Stage 8: Predictive Churn and Expansion (NRR)

The short answer

Retention work asks: of the revenue we had a year ago, how much do we still have, and how much more did the same customers add? Net revenue retention (NRR) answers it. A GTM engineer owns the data behind that number: usage events, renewal dates, health scores, and a daily job that tells a customer success manager (CSM) which account to call. This stage teaches the math, then builds a model that flags accounts 45 days before they renew.

Why this matters

Winning a customer costs money once. Keeping one pays back every year. The roadmap says: "In the AI era, switching costs are low. Retention and expansion are the only real growth." Test that claim before you repeat it.

• Part true, for cheap AI products. The ChartMogul and Kyle Poyar report The AI churn wave (December 10, 2025) covered 3,500 software companies, about 200 of them AI-native. The authors labeled scraped websites with AI and kept only companies above $250k in annual recurring revenue (ARR, the yearly value of active subscriptions). Median NRR was 82% for B2B SaaS, 49% for B2C SaaS and 48% for AI-native. AI-native products under $50 a month had 23% GRR and 32% NRR. Those above $250 a month had 70% GRR and 85% NRR, which the report calls "essentially the same as B2B SaaS." Caveat: the AI-native price buckets hold about 50 companies each, which the authors call "directional rather than statistically bulletproof."
• Not true as "only". SaaS Capital's 2025 survey of 1,000+ private B2B SaaS companies found median growth of 15% when NRR was below 90%, and 50% when NRR was above 130%. Weak-retention companies still grow by selling to new customers.

These are correlations. SaaS Capital also says GRR shows little link to growth and is a "table stakes" floor near 90%.

Verdict: switching costs are low for thin, cheap, self-serve AI tools and normal for products wired into a workflow. Retention and expansion are a major growth lever, not the only one.

Also skip the folklore "a 5% retention gain lifts profit 25% to 95%." HBR's 2014 article credits Bain's Frederick Reichheld, but I could not verify where the range comes from, so do not quote it.

Core concepts

GRR, NRR and logo churn, with numbers

Take a cohort: every customer you had on one date. Look at them 12 months later. Ignore anyone who joined since.

Four words come up in every formula. Churn is revenue lost because a customer cancelled. Contraction is revenue lost because a customer stayed but pays less, for example fewer seats. Expansion is extra revenue from a customer who stayed. A logo is one customer counted as one name, however much it pays.

• NRR = (start ARR - churn - contraction + expansion) / start ARR. It can exceed 100%.
• GRR = (start ARR - churn - contraction) / start ARR. It cannot exceed 100% because each customer is capped at its starting ARR.
• Logo churn = customers lost / starting customers. Revenue churn = ARR lost to cancellations / starting ARR.

Worked example: 100 customers, $1,000,000 ARR. Twelve small customers cancel and take $90,000. Survivors shrink by $40,000 (contraction) and grow by $160,000 (expansion).

NRR of 103% looks healthy. GRR of 87% says leakage is real and a few expanders cover it. Report both. At 90% NRR a cohort keeps 72.9% of its revenue after 3 years (0.9 cubed).

Definitions differ, so read the footnote. SaaS Capital uses December monthly recurring revenue (MRR) from customers who existed 12 months earlier. Snowflake divides a cohort's product revenue in year two of a two-year window by year one, and counts a customer that stopped as zero. Datadog uses ARR and a weighted average of trailing readings.

The warehouse version, assuming a monthly arr_snapshots(account_id, month, arr) table:

WITH cohort AS (
  SELECT account_id, arr AS arr_start
  FROM arr_snapshots
  WHERE month = DATE '2025-09-01' AND arr > 0
),
later AS (
  SELECT account_id, arr AS arr_end
  FROM arr_snapshots
  WHERE month = DATE '2026-09-01'
)
SELECT
  SUM(COALESCE(l.arr_end, 0)) * 1.0 / SUM(c.arr_start) AS nrr,
  SUM(LEAST(COALESCE(l.arr_end, 0), c.arr_start)) * 1.0 / SUM(c.arr_start) AS grr
FROM cohort c
LEFT JOIN later l USING (account_id);

The left join keeps churned customers at zero, and new customers never enter the cohort. I checked the arithmetic in DuckDB on toy rows.

Benchmarks you can cite

• Snowflake's 10-K (its annual report filed with the US SEC) for the year ended January 31, 2026: NRR 125%, against 126% a year earlier. Usage-based, so existing customers consuming more lifts it.
• Datadog's 10-K for December 31, 2025: trailing 12-month NRR about 120%, against the high 110s a year earlier.
• SaaS Capital 2025, private B2B SaaS: median NRR 101% and median GRR 91%.

Match the benchmark to your price point. SaaS Capital says benchmarking by annual contract value (ACV, the yearly price of one customer's contract) is "the best starting point": median NRR runs 98% to 106% across ACV bands, and median GRR is about 91% below $250k ACV and 95% above. Its by-price charts exclude companies under $1M ARR. Public giants at 120%+ are not peers for $10k contracts. Usage-based NRR also falls when usage falls.

Expansion and contraction signals

A plan quota is the number of API calls a plan includes. A customer at 85% of its quota with rising volume is an expansion candidate. One at 10% is a contraction or churn risk. A feature is one input column the model reads. The feature utilization (calls in 28 days divided by plan quota) feeds both plays: 85,000 calls on a 100,000 quota gives 0.85.

Health scores, version one: weighted rules

Start with rules. They need no training data, are easy to explain, and set the baseline a model must beat. GitLab publishes its method in its customer health scoring handbook: product use 50%, renewal-risk and CSM sentiment 25%, outcomes 10%, customer engagement 10%, support and survey voice 5%. Gainsight's scorecards work the same way: overall score = sum of weight times score, divided by 100.

Worked example with the GitLab weights and invented scores out of 100: product use 40, renewal risk 60, outcomes 70, engagement 50, survey voice 90. Overall: 0.50 x 40 + 0.25 x 60 + 0.10 x 70 + 0.10 x 50 + 0.05 x 90 = 51.5. Product use carries half the weight, so good survey answers cannot lift this account far.

GitLab lists "backtested and validated as effective" as a goal. A weight is a hypothesis until you test it against past renewals.

Health scores, version two: a learned model

With 500+ labeled renewals, let data set the weights. Two models cover most cases.

• Logistic regression gives a probability from a weighted sum of scaled features. Coefficients show direction and size. scikit-learn's docs call it naturally well calibrated, but only unpenalized, and the default adds a penalty, so check the calibration curve. penalty was deprecated in scikit-learn 1.8, so leave it unset and tune C (smaller means stronger regularization).
• Gradient boosting (HistGradientBoostingClassifier) builds many small decision trees and catches interactions, such as "usage down AND no admin login". It has monotonic_cst (binary classification only) if you want to force "more usage never raises risk", and class_weight (added in version 1.2).

Try the simple model first. If boosting wins by little, ship logistic.

Features from a usage time series

Raw daily counts are noisy. Turn them into comparisons.

Compare an account only with itself, over whole weeks so weekends do not look like drops. The +1 terms stop divide-by-zero.

The renewal window, snapshot and label

A churn model needs three dates per account. The renewal date is when the contract ends. The snapshot is 45 days earlier, and only data before it is allowed. The label is what happened at renewal. Example: a contract ends 2026-03-31, so its snapshot is 2026-02-14 and the model sees usage up to 2026-02-13. The label is 1 if the customer did not renew, and 0 if it did.

The renewal date comes from the CRM contract record or from billing. In Stripe, the Subscription object has no top-level current_period_end. It lives on each subscription item (items.data[].current_period_end). The list subscriptions endpoint filters on it and matches the minimum item end date.

To find accounts 45 days out, call GET https://api.stripe.com/v1/subscriptions with status=active, limit=100, and current_period_end[gte] and [lte] set to Unix seconds for that day. Send a restricted secret key from an environment variable as the basic-auth user name. Follow has_more by passing the last id as starting_after.

Monthly plans renew every month, so a 45-day horizon is meaningless for them: use survival analysis (below). Annual and multi-year contracts fit this stage. No study I found proves 45 days is right. Pick it from your contract's notice period and how long a CSM needs for a save play.

Class imbalance, leakage and splits

Imbalance. If 17% of accounts churn, "nobody churns" is 83% accurate and useless. Use ranking metrics. class_weight="balanced" shifts predicted probabilities, so recalibrate if you need real ones.

Leakage means the model sees something it could not know at prediction time. scikit-learn's docs: never call fit on test data, and use a Pipeline so preprocessing sees training rows only. Churn adds traps.

• Features that reach past the snapshot, such as the last 45 days of usage. In my run this lifted boosting AUC from 0.72 to 0.84, a fake gain.
• Fields written because the customer already left, such as "cancellation requested" tickets.
• A random train and test split. Use time.
• The same account in both sets. Split by account.

The split. Pick a cutoff date. Train on renewals before it. Test on renewals at least 45 days after it, because on a test account's snapshot day you only know outcomes of contracts that already ended. scikit-learn's TimeSeriesSplit has a gap parameter, but it counts rows, not days.

Evaluating the model

• ROC AUC (area under the ROC curve): the chance a random churner scores above a random renewer. 0.5 is a coin flip. At 0.75, the churner wins in about 75 of 100 random churner and renewer pairs.
• Precision at k: of the k accounts you flag, how many really churn? Set k to CSM capacity, say 25. Precision is the share of positive predictions that are actually positive, as Google's crash course puts it. If 20 of 25 flags churn, precision at 25 is 0.80.
• PR-AUC (average_precision_score): one number for how well the model keeps precision high as you flag more accounts. It often tells you more than ROC AUC when churners are rare.
• Calibration: when the model says 30%, do about 30% churn? calibration_curve returns the observed fraction first, then the mean prediction. Brier score is the mean squared error of the probabilities.
• Dollars: add up the ARR of flagged accounts. Three $200k accounts outweigh ten $5k ones.

Survival analysis basics

Classification asks "will it churn at this renewal?" Survival analysis asks "how long until it churns?" Its key idea is censoring: a customer still active today has an unknown final lifetime, so do not drop it or treat today as its end. The Kaplan-Meier estimator counts it as at risk up to the date you last saw it. The Cox model adds covariates such as plan or usage. Use them for monthly plans and lifetime value.

from lifelines import KaplanMeierFitter

months = [3, 5, 7, 12, 12, 14, 18, 20, 24, 24, 30, 36]
churned = [1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0]          # 0 = still active (censored)
km = KaplanMeierFitter().fit(months, event_observed=churned)
print(km.median_survival_time_, float(km.predict(12)))  # 24.0 and 0.74 on this toy data

Usage-drop alerts and CS playbooks

A usage alert is a rule, not a model: "the last 7 days average under 50% of the prior 28 days, for an account that averaged over 1,000 calls a day." Keep a volume floor and suppress holidays. Use alerts for speed and the model for ranking. Every score needs an action:

Log every action. If a save call turns a churner into a renewer, your next training set labels it "renewed," and the model learns that falling usage is fine. Store an intervened flag and handle it before retraining.

Key terms

The tools

A platform buys integrations and a CSM workflow, not a better model. Many teams score in the warehouse and push the number into a CRM. Checked October 7, 2026.

Build the project

You will build a churn-risk flagger. It generates fake accounts and daily API usage (no budget needed), builds features as of 45 days before each renewal, compares a rules baseline, logistic regression and gradient boosting on a time split, and then runs a daily job that posts the riskiest accounts to Slack. I ran every script below on October 7, 2026 with Python 3.14, scikit-learn 1.9.1, pandas 2.3.3, numpy 2.5.3 and lifelines 0.30.3. These versions need Python 3.11 or newer.

1. Create a folder and a virtual environment, activate it, and install the libraries with the commands in the block below.
2. Save make_data.py, then run python make_data.py to create 1,500 fake accounts and their daily API calls.
3. Write down your label rule: churned means renewed == 0 at the renewal date, and the snapshot is 45 days earlier.
4. Save features.py and read it. Confirm that build uses only usage dated before the snapshot.
5. Save train.py, then run python train.py to compare the rules baseline, logistic regression and boosting on later renewals.
6. Read the output: AUC, PR-AUC, precision at 25 and the calibration line. Compare each model with the rules baseline.
7. Save flag_at_risk.py, then run python flag_at_risk.py --as-of 2026-08-20 --within 14 to see a flagged list for a past date.
8. Set SLACK_WEBHOOK_URL in your shell and run the same command again to post the top accounts to a channel.
9. Load the toy arr_snapshots table below and run the NRR and GRR query from Core concepts. Customer 3 churned and customer 6 is new, so 6 must not count.
10. Schedule flag_at_risk.py daily with cron or a GitHub Actions schedule, once your usage file refreshes every day.
11. Add an intervened column to the output CSV for the CSM to fill in, so you can handle saved accounts before retraining.

python -m venv .venv
source .venv/bin/activate          # on Windows: .venv\Scripts\activate
pip install scikit-learn pandas numpy requests
pip install lifelines              # only for the survival snippet and the stretch goal

Toy data for step 9. The query should return NRR 108.7% and GRR 92%.

CREATE TABLE arr_snapshots(account_id INT, month DATE, arr DOUBLE);
INSERT INTO arr_snapshots VALUES
  (1, DATE '2025-09-01', 300), (1, DATE '2026-09-01', 450), (2, DATE '2025-09-01', 200),
  (2, DATE '2026-09-01', 180), (3, DATE '2025-09-01', 100), (4, DATE '2025-09-01', 400),
  (4, DATE '2026-09-01', 400), (5, DATE '2025-09-01', 500), (5, DATE '2026-09-01', 600),
  (6, DATE '2026-09-01', 700);

make_data.py

About 20% of accounts churn, and 25% of those give no usage warning, so no model scores perfectly. Some healthy accounts dip briefly, so the model must tell a dip from a decline.

"""Make fake accounts and daily API usage. Writes accounts.csv and usage.csv."""
from datetime import timedelta

import numpy as np
import pandas as pd

AS_OF = pd.Timestamp("2026-10-07")     # "today" in the fake world
N = 1500
rng = np.random.default_rng(7)

PLANS = {"starter": (50_000, 6_000), "team": (500_000, 24_000), "scale": (5_000_000, 90_000)}
names = rng.choice(list(PLANS), size=N, p=[0.55, 0.35, 0.10])
renewal = [AS_OF + timedelta(days=int(d)) for d in rng.integers(-450, 90, size=N)]

accounts, usage = [], []
for i in range(N):
    quota, arr = PLANS[names[i]]
    arr = int(arr * rng.uniform(0.7, 1.4))
    end = renewal[i]
    days = pd.date_range(end - timedelta(days=364), end, freq="D")
    base = quota / 30 * rng.uniform(0.15, 0.9)
    t = np.arange(len(days))
    weekday = np.where(days.dayofweek < 5, 1.0, 0.3)
    level = np.ones(len(days)) * (1 + rng.uniform(-0.1, 0.5) * t / 365)
    churner = rng.random() < 0.20
    quiet = churner and rng.random() < 0.25            # no usage warning
    if churner and not quiet:
        start = len(days) - rng.integers(35, 130)      # decay starts 35-130 days out
        level[start:] *= np.exp(-rng.uniform(0.01, 0.04) * (t[start:] - start))
    if not churner and rng.random() < 0.12:           # healthy account, temporary dip
        s = len(days) - rng.integers(20, 80)
        level[s:s + 21] *= rng.uniform(0.2, 0.5)
    calls = rng.poisson(base * level * weekday * rng.lognormal(0, 0.25, len(days)))
    keep = days <= AS_OF                                # no future usage
    usage.append(pd.DataFrame({"account_id": i, "date": days[keep], "api_calls": calls[keep]}))
    renewed = np.nan if end > AS_OF else float(not churner)     # blank = not renewed yet
    accounts.append(dict(account_id=i, plan=names[i], quota_30d=quota, arr=arr,
                         renewal_date=end.date(), renewed=renewed))

pd.DataFrame(accounts).to_csv("accounts.csv", index=False)
pd.concat(usage).to_csv("usage.csv", index=False)
print(len(accounts), "accounts written")

features.py

"""Turn daily usage into one row of features per account, as of a snapshot date."""
import numpy as np
import pandas as pd

FEATURES = ["ratio_7_28", "ratio_28_prev28", "slope_12w", "drawdown", "active_days_28",
            "days_since_call", "utilization", "log_arr", "plan_rank"]
PLAN_RANK = {"starter": 0, "team": 1, "scale": 2}


def one_account(calls, quota):
    c7, c28, p28 = calls[-7:].sum(), calls[-28:].sum(), calls[-56:-28].sum()
    weekly = calls[-84:].reshape(12, 7).sum(axis=1)
    slope = np.polyfit(np.arange(12), np.log1p(weekly), 1)[0]
    rolling28 = np.convolve(calls[-252:], np.ones(28), mode="valid")
    nonzero = np.flatnonzero(calls)
    return {
        "ratio_7_28": (c7 / 7 + 1) / (c28 / 28 + 1),
        "ratio_28_prev28": (c28 + 1) / (p28 + 1),
        "slope_12w": slope,
        "drawdown": c28 / max(rolling28.max(), 1),
        "active_days_28": int((calls[-28:] > 0).sum()),
        "days_since_call": len(calls) - 1 - nonzero[-1] if len(nonzero) else len(calls),
        "utilization": c28 / quota,
    }


def build(accounts, usage, snapshot):
    """snapshot: Series of dates by account_id. Uses usage dated BEFORE the snapshot."""
    rows = []
    for acc in accounts.itertuples():
        if acc.account_id not in usage:
            continue
        u = usage[acc.account_id]
        last_day = snapshot[acc.account_id] - pd.Timedelta(1, "D")
        days = pd.date_range(u.index.min(), last_day)
        calls = u.reindex(days, fill_value=0).to_numpy(dtype=float)  # a missing day is 0 calls
        if len(calls) < 252:                                       # not enough history, skip
            continue
        row = one_account(calls, acc.quota_30d)
        row.update(account_id=acc.account_id, log_arr=np.log(acc.arr),
                   plan_rank=PLAN_RANK[acc.plan])
        rows.append(row)
    if not rows:
        return pd.DataFrame(columns=FEATURES)
    return pd.DataFrame(rows).set_index("account_id")


def load(path_accounts="accounts.csv", path_usage="usage.csv"):
    accounts = pd.read_csv(path_accounts, parse_dates=["renewal_date"])
    usage = pd.read_csv(path_usage, parse_dates=["date"])
    series = {k: g.set_index("date")["api_calls"] for k, g in usage.groupby("account_id")}
    return accounts, series

train.py

"""Train on past renewals, test on later ones, and print the numbers that matter."""
import joblib
import numpy as np
import pandas as pd
from sklearn.calibration import calibration_curve
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, brier_score_loss, roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from features import FEATURES, build, load

WINDOW = pd.Timedelta(45, "D")
accounts, usage = load()
known = accounts[accounts["renewed"].notna()].set_index("account_id")
snap = known["renewal_date"] - WINDOW                      # we predict 45 days before renewal
X = build(known.reset_index(), usage, snap)
known = known.loc[X.index]
y = 1 - known["renewed"].astype(int)                       # 1 = churned

# Time split with a gap: training labels must be known on the test snapshot dates.
cutoff = known["renewal_date"].quantile(0.7)
train = known["renewal_date"] < cutoff
test = known["renewal_date"] >= cutoff + WINDOW
print(f"train {train.sum()} (churn {y[train].mean():.1%})  test {test.sum()} (churn {y[test].mean():.1%})")


def precision_at_k(y_true, score, k):
    top = np.argsort(-np.asarray(score))[:k]
    return np.asarray(y_true)[top].mean()


def rules_score(f):
    """Baseline: a weighted-rules risk score from 0 to 100."""
    usage_drop = np.clip((1 - f["ratio_28_prev28"]) / 0.5, 0, 1)      # 50% drop = max risk
    trend = np.clip(-f["slope_12w"] / 0.15, 0, 1)                      # falling 15% a week = max
    silent = np.clip(f["days_since_call"] / 14, 0, 1)
    below_peak = np.clip((1 - f["drawdown"]) / 0.7, 0, 1)
    return 100 * (0.35 * usage_drop + 0.25 * trend + 0.20 * below_peak + 0.20 * silent)


models = {
    "logistic": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
    "boosting": HistGradientBoostingClassifier(max_iter=150, learning_rate=0.05,
                                               max_depth=3, random_state=0),
}
scores = {"rules": rules_score(X[test])}
for name, m in models.items():
    m.fit(X.loc[train, FEATURES], y[train])
    scores[name] = m.predict_proba(X.loc[test, FEATURES])[:, 1]

k = 25                                                      # CSM capacity: 25 reviews per cycle
print(f"base churn rate in test: {y[test].mean():.1%}")
for name, s in scores.items():
    print(f"{name:9s} AUC {roc_auc_score(y[test], s):.3f}  PR-AUC {average_precision_score(y[test], s):.3f}"
          f"  precision@{k} {precision_at_k(y[test], s, k):.2f}")

p = scores["boosting"]
frac_pos, mean_pred = calibration_curve(y[test], p, n_bins=5, strategy="quantile")
print("calibration (predicted -> actual):",
      " ".join(f"{a:.2f}->{b:.2f}" for a, b in zip(mean_pred, frac_pos)),
      f" Brier {brier_score_loss(y[test], p):.3f}")

joblib.dump(models["boosting"].fit(X[FEATURES], y), "churn_model.joblib")   # refit on all labels

flag_at_risk.py

The daily job scores accounts whose renewal is 45 days away. --within 14 also catches accounts 32 to 45 days out, for missed runs or a demo. By default the run date is the day after the newest usage row, which is today when your usage file refreshes daily. In step 7 the past-date accounts are in the training data, so their scores look too good. An incoming webhook is a secret URL: you POST JSON with a text field, and Slack shows it as a message and answers HTTP 200 with ok, per the Slack docs. Keep the URL in an environment variable.

"""Daily job: score accounts that renew in about 45 days and flag the riskiest."""
import argparse
import os

import joblib
import pandas as pd
import requests

from features import FEATURES, build, load

ap = argparse.ArgumentParser()
ap.add_argument("--as-of", default=None, help="YYYY-MM-DD, default: the day after the newest usage row")
ap.add_argument("--within", type=int, default=1, help="days of catch-up for missed runs")
ap.add_argument("--top", type=int, default=25, help="how many accounts a CSM can review")
args = ap.parse_args()

accounts, usage = load()
newest = max(s.index.max() for s in usage.values())
today = pd.Timestamp(args.as_of) if args.as_of else newest + pd.Timedelta(1, "D")
accounts["days_left"] = (accounts["renewal_date"] - today).dt.days
due = accounts[(accounts["days_left"] <= 45) & (accounts["days_left"] > 45 - args.within)]
if due.empty:
    raise SystemExit("no accounts in the window")

X = build(due, usage, pd.Series(today, index=due["account_id"]))
if X.empty:
    raise SystemExit("no accounts with enough usage history")
model = joblib.load("churn_model.joblib")
out = due.set_index("account_id").loc[X.index, ["plan", "arr", "renewal_date"]].copy()
out["risk"] = model.predict_proba(X[FEATURES])[:, 1]
out["usage_vs_prev_28d"] = X["ratio_28_prev28"] - 1
out = out.sort_values("risk", ascending=False).head(args.top)
out.to_csv(f"at_risk_{today.date()}.csv")
print(out.head(10).to_string(float_format=lambda v: f"{v:.2f}"))

webhook = os.environ.get("SLACK_WEBHOOK_URL")          # never hard-code this URL
if webhook:
    lines = [f"Account {i} ({r.plan}, ${r.arr:,}) renews {r.renewal_date:%Y-%m-%d}: "
             f"risk {r.risk:.0%}, usage {r.usage_vs_prev_28d:+.0%} vs prior 28 days"
             for i, r in out.head(10).iterrows()]
    resp = requests.post(webhook, json={"text": "Churn risk, 45 days out:\n" + "\n".join(lines)},
                         timeout=10)
    resp.raise_for_status()                              # Slack returns HTTP 200 and "ok"

The no-budget path

Everything above runs on a laptop with fake data and free libraries. For real data, export two CSVs with the same columns: one row per account, and one row per account per day. Skip Slack if you have no workspace. The CSV is the output.

What done looks like

With seed 7, my run printed 879 training and 259 test accounts, a 22.0% churn rate in the test set, and these results:

Precision at 25 of 0.88 means 22 of 25 flags churned, against a 22.0% base rate: about 4 times better than random. Only 57 test accounts churned, so one seed is noisy. Across seven seeds (1 to 7), mean AUC was 0.740 for rules, 0.748 for logistic and 0.738 for boosting: no real difference. Mean precision at 25 was 0.73, 0.76 and 0.86, with boosting ahead in all seven. The learned models earn their keep at the top of the list. Calibration was mixed: the top bucket predicted 0.59 and saw 0.56, but the lowest predicted 0.03 and saw 0.17 (Brier 0.125), so do not read the probabilities literally. Done means: the test set starts 45+ days after the training cutoff, you report precision at k against the rules baseline, and the job has no hard-coded secret.

Stretch goal: fit a Cox model with CoxPHFitter for monthly plans.

Common mistakes

• Random splits. Shuffled rows let the model learn from future renewals. Split by renewal date, with a gap.
• Feature windows past the snapshot. A model fed the final 45 days of usage looks great and is useless 45 days out.
• Reporting NRR alone. Expansion from three accounts can hide falling GRR. Report both, by segment.
• Raw usage alerts. Weekends, holidays and tiny accounts create noise. Compare each account with itself, by whole weeks.
• Ignoring the feedback loop. Saved accounts become "renewed" labels. Log interventions and handle them before retraining.

Check yourself

How to learn this faster

Learn first: GRR and NRR math in SQL, then a weighted-rules score, then logistic regression. Skip neural networks and AutoML until your split is right. Buy a platform only after you can explain your own score.

Drill, the backtest: freeze a date a year ago, build features from earlier data only, predict the next 90 days of renewals, and score against what happened. Repeat with the snapshot 15 days earlier.

• Day 1: Compute GRR, NRR and logo churn on a toy table. Check one result by hand.
• Day 2: Generate the fake data. Plot three accounts: one healthy, one declining, one with a dip.
• Day 3: Train the rules baseline and logistic regression. Report AUC and precision at 25.
• Day 4: Add boosting and the calibration curve. Run a leakage test: move the snapshot to the renewal date and explain the jump.
• Day 5: Run the daily job with a Slack webhook. Write a one-page CSM playbook.

Resources

• SaaS Capital: 2025 B2B SaaS retention benchmarksResearchFree

  Median NRR and GRR by contract size, and growth by NRR band.

• ChartMogul and Kyle Poyar: The AI churn waveResearchFree

  Retention for AI-native products by price band.

• Snowflake Form 10-K for fiscal 2026ResearchFree

  Primary source for Snowflake's 125% NRR and its method.

• Datadog Form 10-K for 2025ResearchFree

  How a public company defines dollar-based net retention.

• GitLab handbook: Customer health scoringDocsFree

  A real, published weighted health score.

• scikit-learn: Common pitfalls and recommended practicesDocsFree

  The official explanation of data leakage and the Pipeline fix.

• scikit-learn: Probability calibrationDocsFree

  Sigmoid versus isotonic calibration, and calibration curves.

• Google Machine Learning Crash Course: Accuracy, precision and recallCourseFree

  Plain definitions of precision, recall and accuracy.

• lifelines: Introduction to survival analysisDocsFree

  Explains censoring with a worked example.

• lifelines: Survival analysis with lifelinesDocsFree

  Covers KaplanMeierFitter and median survival time.

• Fader and Hardie: How to Project Customer RetentionResearchFree

  A classic 2006 paper on modeling subscription retention.

Next module

You now know which accounts will leave and which will grow. Stage 9, Integration-Led Growth, covers the other side of retention: the integrations and partners that make a product hard to remove and open new ways to reach customers.