Methodology

Numbers without methodology are marketing. This page documents how we measure response time, conversion lift, and the cohort definitions behind every data point we cite on NZ Leads pages. If you spot a claim on our site without a clear cohort window or a defined metric, treat that as a bug and tell us.

Start your 7-day free trial if you want to see the same dashboards we use to compute these numbers, applied to your own account.

What we measure

Three core metrics drive almost every claim on our marketing pages:

Response time. The interval between an inbound lead webhook arriving at NZ Leads and the first AI-generated reply being sent on the appropriate channel (Thumbtack message, Yelp message, Facebook message, SMS, voice call, etc.).
Lead-to-booking conversion rate. The percentage of inbound leads in a given cohort that result in a calendar event accepted as a job by the contractor within seven days.
Booked-job lift. The change in absolute booked jobs per month between a pre-deploy baseline (typically 14-30 days before activation) and a post-deploy window (typically 14-30 days after activation), holding lead volume roughly constant.

We do not measure or claim "leads generated" — that is the platform's job (Thumbtack, Yelp, etc.), not ours. Our scope starts the moment a lead enters our system.

How we define response time

Response time is the wall-clock interval, measured to the second, between two events:

T0: The inbound lead webhook is received by NZ Leads. For Thumbtack and Yelp this is the official platform webhook timestamp. For Facebook Lead Ads it is the moment the lead form payload is delivered. For Google LSA it is the moment the lead notification reaches our integration.
T1: The first outbound message generated and sent by our AI on the corresponding channel.

Response time = T1 − T0. The metric does not include any latency that occurred before T0 (i.e., time the lead sat inside the originating platform before being delivered). We cannot observe that latency, so we exclude it from our claims rather than estimate around it.

How we define a booked appointment

A booked appointment is a calendar event that meets all four conditions:

Created on the contractor's connected Google Calendar or Microsoft 365 calendar through our integration.
Linked to a customer record sourced from a tracked inbound lead.
Confirmed by the contractor (i.e., not auto-cancelled, not flagged as a misfire).
Scheduled within seven days of the originating lead arrival.

We deliberately exclude "warm replies," "interested but not booked," and "callback requested without confirmed time." Those are softer signals and we choose not to count them as conversions, even though doing so would inflate our headline numbers.

Cohort definitions

Pages on NZ Leads sometimes cite cohorts like "the Thumbtack leads we processed in March 2026." Here is what those phrases mean:

Time window. The calendar month or quarter named in the citation, in US Pacific time.
Channel scope. The named platform — leads from other channels are excluded from a Thumbtack-cohort claim, and vice versa.
Customer scope. All paid customers active on the relevant channel for the full window. Trial accounts that did not convert to paid are excluded. Test accounts and internal traffic are excluded.
Vertical scope. Service-business categories only — HVAC, plumbing, roofing, electrical, landscaping, cleaning, and adjacent trades. We do not include B2B service categories or non-service verticals in service-business claims.

Sample size policy

We do not cite numbers from samples too small to be meaningful. The internal threshold is a minimum of several hundred inbound leads per cited cohort, and a minimum of dozens of contractor accounts per cited cohort. When we make a per-trade claim (e.g., "in our HVAC customer cohort"), the per-trade subset still meets these minimums.

Where a sample is genuinely small — for example, an early-stage feature in pilot — we either decline to cite a number on a marketing page or we explicitly state the sample size next to the figure.

Platform coverage

NZ Leads ingests leads from Thumbtack, Yelp Request a Quote, Facebook Messenger, Facebook Lead Ads, Instagram DMs, Google Local Services Ads, Angi, Nextdoor, Porch, and Houzz. Numbers we publish reflect the platforms where we have native integrations and where data flows through our system end to end. We do not extrapolate to platforms we do not integrate with.

Exclusions

The following data is excluded from every cited cohort:

Test accounts. Accounts created by NZ Leads staff for internal testing.
Internal traffic. Leads from staff or related parties, identified by phone number, email, or IP heuristics.
Cancelled or refunded subscriptions during the cohort window. Their data is excluded from the window even if they were active for part of it.
Channels we do not natively integrate with. If a contractor manually copy-pastes leads from a platform we do not connect to, those leads are not in our cohorts.

Why we publish ranges, not point estimates

Per-business variance is large. Two HVAC contractors at the same Thumbtack lead volume can see meaningfully different conversion lifts from the same auto-responder configuration, depending on service area density, average ticket size, competitive intensity, and seasonality. Publishing a single percentage would imply a precision that does not match the underlying distribution.

Ranges like "roughly 2-3x" or "32-42%" reflect the inter-quartile span across the cohort. Outliers (top and bottom decile) are not used to define the range. If a contractor's results sit outside the published range in either direction, that is normal — the range describes the typical case, not the universe.

Update cadence

Headline numbers across the site are reviewed quarterly. Per-page citations carry the cohort window in the surrounding sentence (e.g., "across the leads we processed in March 2026"), so the data vintage is always visible. When we refresh a number, both the figure and the cohort window in the prose are updated.

If you find a number on our site that is older than the most recent quarter without an explicit cohort window — that is a content bug. Tell us and we will fix it.

Where this methodology is referenced

This page is linked from:

The comparison table footnotes on our Thumbtack auto-responder page.
FAQ answers on the AI receptionist pillar.
Hero stat citations on the lead response automation pillar.

Frequently asked questions

Why do you publish ranges instead of single percentages?

Per-business variance is high. A roofer in Phoenix and an HVAC contractor in Boston do not see the same conversion lift from the same change in response time. Single point estimates would imply a precision we do not have. Ranges (e.g., "32-42% conversion post-deploy") reflect the actual distribution we measure across our customer cohort.

What counts as "response time" in NZ Leads data?

Response time is measured from the moment we receive an inbound lead webhook (from Thumbtack, Yelp, Facebook, Google LSA, Angi, Nextdoor, Instagram, etc.) to the moment our system sends the first AI-generated reply. It does not include the time the lead sat in the originating platform before being delivered to us, which we cannot observe.

How do you define a "booked appointment"?

A booked appointment is a calendar event created on the contractor's connected Google Calendar or Microsoft 365 account, with a confirmed slot and a customer record attached. We do not count "interested replies" or "warm leads" — only calendar events that the contractor has accepted as actual jobs.

How big is the data sample behind the numbers on your pages?

The Thumbtack March 2026 cohort referenced on our pillar pages includes thousands of inbound leads across hundreds of contractors in HVAC, plumbing, roofing, electrical, and adjacent service categories. Smaller cohorts (e.g., per-trade subsets) are noted explicitly when we cite them. We do not publish numbers from samples too small to be meaningful.

How often do you update the data on your pages?

Headline statistics are reviewed quarterly. Per-page citations carry the cohort window in the surrounding sentence (e.g., "March 2026") so readers can see the data vintage. When we refresh a number, the updated cohort window is reflected in the prose.

Want to see the dashboard yourself?

Start your 7-day free trial and run the same response-time and conversion measurements on your own lead flow.