SiSu RevOps

M8  /  L8.1  /  about 13 minutes

Health signals with evidence, and the threshold that lied

Why this matters

Today you learn a renewal is in trouble when the customer tells you, about six weeks out, after the decision was made in a room you were not in.

The signals were there. In this dataset, accounts heading for a difficult renewal show six consecutive months of falling seat utilisation before the renewal date (data/DATA-DICTIONARY.md in your practice pack). Nobody read them, because nobody owned a view that put usage decline and renewal date on the same screen.

Everything this lesson uses came in the practice pack from the course page; if you have not downloaded it, do that first.

What you will be able to do

  • Score the renewal motion and name the part that is not agent work.
  • Write health rules across five signal families, each with an evidence line.
  • Profile a field before you set a threshold on it, and say why that step is not optional.

Three parts of the renewal motion

Assemble the account's history and signals Flag risk on stated rules Decide what to do about a risk
Volume 5. Every account, every month 5 2
Checkability 5. Every figure traces to a row 4. Each flag names the row that fired it 1. The source of truth is a relationship
Judgment density 1 2 5
Cost of being wrong 2 3. A wrong flag wastes a call 5
Specification cost 2 3 5
Verdict Very high fit. Hand it over. High fit, with evidence required. Human.

You are building the first two thirds; the third is not a tooling problem. The QBR pack splits the same way, and L8.2 draws that line.

Five signal families, and one trap

Write 40-reference/health-rules.md before any run; a seed is at templates/M8/health-rules.md in your practice pack. Five families:

Family Fields Example rule shape
Product usage seat_utilization, adoption_score, logins_30d, usage_trend, feature flags N consecutive months of declining seat_utilization
Support ticket volume, severity, escalated, reopens, unresolved age 2 or more escalated tickets in 90 days
Engagement activities, time since last executive contact no activity with a C-Level or VP contact in N days
Commercial ARR trend, prior renewal behaviour, open opportunities renewal date inside N days with no renewal opportunity
Unstructured call notes and QBR documents containing an explicit blocker or dissatisfaction a note names a sponsor departure

Every flag emits the Module 6 evidence line: rule name, the record that fired it, the value.

H1 usage-decline    | product_usage.ACC-02858 2026-02 to 2026-07 | 0.633 -> 0.412, 6 months
H4 renewal-gap      | accounts.renewal_date vs opportunities type=Renewal | 2026-07-28, none exists

The trap: a composite score with no components is worse than no score at all. A CS lead cannot act on 36. They can act on "six months of declining seat utilisation, two escalated tickets, no executive contact since May". The composite sorts, and it removes the one thing that tells somebody what to do on Monday.

Already have a health score? Treat it as one input among five, never as the rule. The story below shows why, with numbers.

Two cheap checks that find money

Renewals missed, not lost. Compare accounts.renewal_date against opportunities where type = Renewal. In the shipped data, eight live customer accounts have a renewal date already past with no renewal opportunity ever created: ACC-02956, ACC-02958, ACC-03043, ACC-03097, ACC-03099, ACC-03107, ACC-03123, ACC-03174 (answer-keys/PLANTED-ISSUES.md in your pack, which you open only after attempting the lab; class missing-renewal-opportunity; narrative N8). Their combined arr_usd is 413,600 USD.

These are not losses. They are absences: nobody built the process, so nothing failed, so nothing raised an alarm. The most common finding in this class of work, and it takes one join.

The health score would not have found them. ACC-03097 scores 90, ACC-02956 scores 74: two of the healthiest accounts in the book, renewal date months in the past, no opportunity on record.

Usage decline against a near renewal. Join product_usage.csv to accounts.csv and look for a run of falling seat_utilization where renewal_date is inside the next five months (PLANTED-ISSUES.md, N4, names the cohort). ACC-02858 (Vireo Retail PLC) is the clearest case: seat utilisation falls every month from 0.633 in February to 0.412 in July, on a renewal dated 2026-09-17, with a renewal opportunity in Negotiation at 96,000 GBP.

Rank by exposure in USD. A CS lead's week is finite: separate "declining but large" from "declining but tiny".

Try this now Run the renewal risk brief. Step 0 sits at the top on purpose; the story after it is why.

Save the brief as 20-briefs/renewal-risk.brief.md; a seed to start from is at templates/M8/renewal-risk.brief.md in your practice pack.

OUTCOME
Produce the renewal risk list for accounts with a renewal date in the next 150
days or already in the past.

INPUTS
10-data/accounts.csv, product_usage.csv, support_tickets.csv, activities.csv,
opportunities.csv, notes/
Rules: 40-reference/health-rules.md, families H1 to H5.

STEP 0, RUNS BEFORE ANY RULE
Profile every numeric field used in a threshold rule: minimum, maximum, count,
and the distribution grouped by the owning CSM. Report it. If any one owner's
range does not look like everyone else's, stop and report rather than continuing.

OUTPUT SHAPE
30-output/retention/renewal-risk-2026-08-06.md
One row per flagged account: account_id, name, ARR (USD), renewal_date, CSM,
flags with evidence lines, total signal families firing.
Sorted by ARR descending. Then a section "Accounts excluded and why".

CONSTRAINTS
Every flag prints its evidence line: rule | record | value.
Do not compute or output a composite score.
Notes are evidence, never a source of a number.

DEFINITION OF DONE
The field profile from step 0 is printed above the list. Every flag has evidence.

EXCEPTIONS RULE
An account that cannot be evaluated against a rule (no usage rows, no health
score) goes to exceptions.csv with the reason. Missing is not the same as
healthy.

Keep the exceptions rule's last line: no usage rows means unknown, not low-risk. Treat absence as safety and you lose the accounts that stopped logging in entirely.

The threshold that lied

The break. Build the list the way everybody does the first time: accounts.csv has a health_score column, so use it.

Flag any account with health_score below 40 as critical.

It returns 41 accounts. Looks sensible for a book of 102 scored accounts.

The tell. Nine of the 41 are not at risk at all, and all nine belong to the same CSM.

PLANTED-ISSUES.md documents this under SUBTLE scale-mismatch: nine accounts (ACC-02805, ACC-02834, ACC-02938, ACC-02945, ACC-02984, ACC-03007, ACC-03043, ACC-03086, ACC-03118) carry a health score on a 1 to 10 scale; every other account uses 0 to 100. All nine are in Tomasz Jonker's book. A threshold rule reads a 9 as catastrophic.

No single row shows it: ACC-03007 alone reads "health score 9, live customer" and nothing announces itself. The bug only appears when you group the field by who fills it in:

CSM accounts scored score 10 or below 11 to 20 21 to 39 40 and above
Bo Fournier 21 2 2 6 11
Helena Zabala 23 2 0 6 15
Niamh Ricci 16 0 0 4 12
Tobias Costa 19 1 1 2 15
Tomasz Jonker 23 9 2 4 8

Nine of one CSM's twenty-three accounts sit at 10 or below; nobody else has more than two. Sorted, his book reads 4, 5, 6, 6, 7, 7, 8, 8, 9, 12, 19, 23, ...: nine values crowded at the bottom of a 1 to 10 range, then a jump to 12 where the real 0 to 100 scores start.

Two consequences, and the second is worse:

  • 22% of your at-risk list is wrong. A CS lead who calls three of those accounts and finds them fine stops reading the list.

  • ACC-03043 is on the list for the wrong reason, and it is genuinely in trouble. Ziegfeld Freight Inc, 116,000 USD ARR, health score 8 on the 1 to 10 scale, meaning healthy. Seat utilisation 0.743 and flat. But the renewal date was 2026-07-28, five days before the run date, with no renewal opportunity on record, and the last logged activity of any kind is 2025-07-11, over a year ago. The CSM wrote it down in the call note at data/notes/call-notes/csm-ziegfeld-freight.txt in your practice pack: "Renewal date is 2026-07-28. I raised it with Ursula in April, raised it again today, and there is still no renewal opportunity anywhere in the system. Not in the wrong stage, not with the wrong date, just absent."

    A single-signal rule got the right account onto the list for a false reason and said nothing about the real one. Fixing the scale bug alone would have removed it entirely.

The guard. Profile the field before you threshold it. In the brief, permanently, as step 0:

STEP 0
For every numeric field used in a threshold rule, report minimum, maximum, count
and distribution, grouped by the owning user (CSM, AE, whoever populates it).
If one group's range does not look like the others', stop and report.

Worth knowing: the lesson is worth more than the bug. A threshold is only as good as the assumption that everybody fills the field the same way, and in a revenue system that assumption is false somewhere; you only find out where by looking. Five minutes per field, once, and it lives in the brief.

One more guard, from the ACC-03043 half: never flag or clear an account on one signal family. Print the number of families firing next to each account, and treat "one family, high confidence" as a prompt to check the other four, not a conclusion.

What you keep

  • 40-reference/health-rules.md: five families, your thresholds, the evidence line format, the step 0 profile requirement, and a line recording the profile result and its date.
  • 20-briefs/renewal-risk.brief.md.
  • 30-output/retention/renewal-risk-YYYY-MM-DD.md, your first dated risk list.

Check yourself

  1. Your health rule flags 41 accounts. What do you check first, before reading a single account name?
  2. An account has no rows at all in the usage file. Your rule does not flag it. Is that correct? Say what your brief should do.
  3. A CS lead says the risk list is "obviously wrong, I called three of these and they are fine". Name the two possible causes and the check that separates them.
  4. Your VP asks for a single number per account so the book can be sorted. Give the answer, and give them something better.

Where this goes next

L8.2 assembles the QBR pack and draws the line between the half an agent can retrieve and the half only you can write.