Real Estate Lead Qualification AI: Fine-Tune a Scoring Model on Your Conversion History

A Zillow lead that says "just browsing" in the first message has a 3% close rate. A lead that says "we need to be in a new school district before August" has a 40%+ close rate. Human agents know this intuitively after years in the field. A fine-tuned model trained on your actual conversion history can know it in seconds — and route leads accordingly before an agent sees them.

The Lead Qualification Problem

Real estate teams buy leads from Zillow, Realtor.com, Google Ads, and their own website. Typical conversion rates:

Zillow Premier Agent: 2-5% of leads close within 12 months
Realtor.com: 1-3%
IDX website leads: 3-7%

This means 93-99% of the leads agents touch will not close. The question is: which 3-7% are ready to transact soon?

Current approaches:

Manual qualification call: Takes 15-20 minutes per lead. Most agents skip it and respond to all leads equally. Low-intent leads consume as much time as high-intent ones.
Generic AI scoring: Tools like Follow Up Boss AI use generic behavior signals (email opens, site visits). They do not understand what the lead said in their initial inquiry.
Experience-based gut feel: Senior agents have good intuition. Junior agents do not. The difference is years of pattern-matching against conversion history.

A fine-tuned NLP classifier trained on your brokerage's own conversion history encodes that senior agent pattern-matching and applies it instantly.

What the Model Predicts

Input: The lead's initial inquiry message + any available context (source, property searched, price range)

Output:

{
  "intent_score": 0.78,
  "tier": "A",
  "predicted_timeline": "30-90 days",
  "key_signals": ["school district deadline", "financing mentioned", "specific address"],
  "recommended_action": "Priority callback within 1 hour",
  "escalate_to_senior": false
}

Tier definitions:

Tier A (score 0.65+): Strong buying signals, specific timeline, priority routing
Tier B (score 0.35-0.64): Medium-term potential, standard follow-up sequence
Tier C (score <0.35): Low intent, automated nurture only

Building the Training Dataset

Source: Your CRM historical data (past 2-3 years of leads)

Required data per lead:

Initial inquiry message (the raw text the lead sent)
Lead source
Price range / property type they searched
Outcome: Closed (and when), Dead/Inactive, Still Active (exclude these)

Label construction:

Lead closed within 90 days of inquiry → Tier A positive example
Lead closed within 6 months → Tier B positive example
Lead did not close within 12 months → Tier C negative example

Size target: 500-1,500 labeled leads (enough to cover your lead sources and property types)

Important: Your labels need to reflect your team's follow-up behavior too. If your team is bad at following up on Tier B leads, those leads will be undercounted as conversions even though they had potential. Note this and account for it in your evaluation.

JSONL format:

{"messages": [
  {"role": "system", "content": "You are a lead qualification assistant for [Brokerage]. Analyze real estate buyer inquiries and classify intent. Score from 0-1 based on these signals: specific timeline (high weight), financing readiness (high), specific property interest (medium), life event driving search (high), casual browsing language (low). Output JSON."},
  {"role": "user", "content": "Lead inquiry: 'Hi, we are relocating from Chicago for a job in March. We have two kids and need to be in the Jefferson County school district. Pre-approved for $550k. Looking at single family, 3+ beds. Saw your listing on 4th Ave — is it still available?' Source: Zillow. Price: $500-600k"},
  {"role": "assistant", "content": "{\"intent_score\": 0.91, \"tier\": \"A\", \"predicted_timeline\": \"30-60 days\", \"key_signals\": [\"relocation with hard deadline\", \"school district requirement\", \"financing pre-approved\", \"specific property inquiry\"], \"recommended_action\": \"Immediate callback, highest priority\", \"escalate_to_senior\": true}"}
]}

Include a range of inquiry types:

High intent (must move, specific timeline, pre-approved)
Medium intent (interested but early stage, browsing but serious neighborhood)
Low intent (investor window shopping, just curious about prices, no location specificity)

Evaluation

Hold out 15% of your dataset. After training, run the evaluation set and measure:

Primary metric: Recall on Tier A leads (don't miss the hot leads)

Secondary metrics:

Precision on Tier A (how often does a Tier A classification actually close?)
Tier distribution accuracy (does the model assign tiers in proportions similar to your actual conversion rates?)

Aim for 80%+ recall on true Tier A leads. A missed hot lead is costly; a misclassified medium lead (routed to Tier A when it is Tier B) is less harmful.

Integration

Gorgias / Zendesk webhook: When a new lead arrives → webhook fires → send inquiry to model API → score returned → CRM updated with tier + score → routed to appropriate follow-up queue

Follow Up Boss integration: Follow Up Boss supports custom webhooks. Route new leads to a scoring endpoint, return the tier, write it as a custom field using the Follow Up Boss API. Triggers the appropriate automation for each tier.

Slack alert for Tier A: When a Tier A lead is scored, send an immediate Slack notification to the on-duty agent: "🔥 Tier A lead — [lead name] — 'relocating in March, pre-approved $550k' — [CRM link]"

Measuring ROI

Track for 90 days post-deployment:

Time-to-first-contact for Tier A leads (should decrease)
Agent time spent on Tier C leads (should decrease)
Close rate on agent-contacted leads (should increase as low-intent leads filter out)
Overall team close rate as % of total leads received

Ship AI that runs on your users' devices.

Ertas early bird pricing starts at $14.50/mo — locked in for life. Plans for builders and agencies.

View early bird pricing or join the waitlist →