You ask for "a data quality report on the CRM export". You get four paragraphs saying the data has some issues. You cannot act on it, compare it to next month, or tell whether it read every row. It is not wrong. It is not a measurement either.
The fix is not a better model or a longer prompt. You never said what a defect is, so the agent picked. This lesson does that job, once, in a file.
What you will be able to do
- Write a defect definition that is falsifiable, and spot one that is not.
- Fix the output shape so this month's audit is comparable to last month's.
- Produce an audit table with counts, percentages and named example IDs, from a brief you can re-run.
A defect definition is a test, not an adjective
Two definitions of the same class:
Stale records. Records that have not been touched in a long time and look abandoned.
Stale opportunity. An opportunity whose
stageis one of the six open stages and whoseclose_dateis earlier than the run date. Counted per opportunity record. Population: all rows inopportunities.csv.
The first cannot be wrong, so it cannot be right either. Two people running it get two answers. The second can be checked by anybody with the file, and argued with. That makes it useful.
A usable definition has four parts:
- The fields it reads, by their real column names.
- The comparison, with any threshold stated as a value you chose, not a vibe.
- The population: "12 records missing an owner" means something different over 528 rows than over 60.
- The unit of count: records, or rows. They differ the moment your file has a duplicated primary key. This dataset does.
Write them in 40-reference/field-policy.md, the policy file you create in this lesson, not in the chat window. Cowork carries no memory between sessions. A definition that lives only in a conversation is gone next month, and the next audit silently uses a different one. Rules in a file in the working folder are the memory.
The output shape is half the brief
Say exactly what the output looks like, or you get a different shape every month and can never draw a trend. The shape for this audit:
| Column | Why it is there |
|---|---|
defect_class |
The name from the field policy. Not free text. |
definition_ref |
The heading in field-policy.md that defines it. Makes the audit auditable. |
count |
Records failing the test. |
pct_of_population |
Count over the stated population, one decimal place. |
example_ids |
Exactly three record IDs, so a reviewer can spot-check in a minute. |
population |
The row count the percentage is over. Printed, not implied. |
Three examples, not all, is deliberate. The audit is a summary a manager reads; the full list belongs in proposed-changes.csv and exceptions.csv, two files your pass creates in 30-output/hygiene/. An audit nobody reads has changed nothing.
Ask for defects by name
Do not ask for "data quality issues". Name the classes. This taxonomy covers most CRM exports, each class definable against a real column:
- missing required fields
- invalid values against a picklist
- stale records by last-modified age
- orphaned records: no owner, or a parent key that does not resolve
- impossible dates: an open record with a close date in the past, a created date after a modified date
- inconsistent formatting: country, industry, currency, owner name
- duplicate candidates (its own lesson, L5.3)
You know which matter in your CRM. That part you cannot delegate, and it takes about twenty minutes. This is why specification cost is a fit-test factor.
See it run
The real brief, run against opportunities.csv in the shipped dataset, in the seven-part shape from Module 4:
OUTCOME
Produce a defect audit of the opportunity export against the definitions in
40-reference/field-policy.md.
INPUTS
10-data/opportunities.csv (528 rows) and 10-data/accounts.csv (308 rows).
accounts.csv is used only to resolve account_id and to read accounts.currency.
Definitions: 40-reference/field-policy.md, sections D1 to D10.
OUTPUT SHAPE
30-output/hygiene/audit-2026-08-06.md
One markdown table, one row per defect class, columns exactly:
defect_class | definition_ref | count | pct_of_population | example_ids | population
Sort descending by count. Percentages to one decimal place.
Below the table, one line: "Distinct opportunity records failing at least one
check: N of 524 (X%)."
CONSTRAINTS
Population for every class is the 524 distinct opportunity_id values unless the
definition says otherwise. Run date is 2026-08-02, which is the dataset's own
"today" (DATA-DICTIONARY.md). Do not clean, modify or rewrite any input file.
Do not propose fixes in this output.
DEFINITION OF DONE
Every class in field-policy.md D1 to D10 appears in the table, including classes
with a count of zero. Every example_ids cell holds exactly three real IDs.
STOPPING CONDITION
Stop after writing the audit file. Do not continue into duplicate detection.
EXCEPTIONS RULE
Any record you cannot classify against a definition goes to
30-output/hygiene/exceptions.csv with columns record_id, reason, definition_ref.
Do not guess.
The result, from the shipped data. Population: the 524 distinct opportunity_id values in a file of 528 rows.
| defect_class | ref | count | pct | example_ids |
|---|---|---|---|---|
| open opportunity, close date in the past | D4 | 14 | 2.7% | OPP-006738, OPP-006894, OPP-006905 |
| owner name variant | D7 | 6 | 1.1% | OPP-006696, OPP-006746, OPP-006795 |
| closed, close date blank | D5 | 5 | 1.0% | OPP-006728, OPP-007099, OPP-007174 |
| closed lost, loss reason blank | D6 | 5 | 1.0% | OPP-006670, OPP-006750, OPP-006884 |
| duplicated primary key | D1 | 4 (on 8 rows) | 0.8% | OPP-006790, OPP-006886, OPP-007099 |
| owner blank | D2 | 4 | 0.8% | OPP-006774, OPP-006841, OPP-007062 |
| currency contradicts account | D8 | 4 | 0.8% | OPP-006697, OPP-006729, OPP-007089 |
| fiscal quarter disagrees with close date | D9 | 4 | 0.8% | OPP-006764, OPP-006773, OPP-006805 |
| amount at or below zero | D10 | 3 | 0.6% | OPP-006774, OPP-006833, OPP-007117 |
| account id does not resolve | D3 | 2 | 0.4% | OPP-006774, OPP-006894 |
Distinct opportunity records failing at least one check: 46 of 524 (8.8%).
Two lines matter most. A careless run leaves them out.
The class counts add to 51, but only 46 records are involved. Four records fail two checks, one fails three: OPP-006774 has a blank owner, an unresolvable account, and a negative amount. Report "51 defective opportunities" and you overstate by 11%. The first checker will find it. Hence the distinct-record line, exactly what the module quiz in the interactive app asks you to diagnose.
The duplicated primary key class counts 4 and touches 8 rows. That distinction is L5.4's failure section, and where your pipeline total goes wrong.
Every ID above is documented in answer-keys/PLANTED-ISSUES.md in your pack, which you open only after attempting the lab.
Try this now: three definitions, one number back
About eight minutes, one real thing end to end. You are the RevOps analyst at Talvera. Sales leadership wants "how much of the open pipeline is not real" by Monday. You have the export and no definitions, because nobody ever wrote any.
Try this now
- Open
opportunities.csvin your pack's data folder and read the columns, or the schema inDATA-DICTIONARY.mdnext to it. The six open stages are Qualification, Discovery, Solution Review, Technical Validation, Proposal, Negotiation.- Create
40-reference/field-policy.mdfromtemplates/M5/field-policy.mdin your practice pack.- Write three definitions, each with all four parts: an open opportunity whose close date has passed; an opportunity with no owner; an opportunity whose currency contradicts its account's billing currency.
- Fill the gaps in the starter brief and run it.
- Compare your counts to the reference below. If one differs, look at the definition first, not the run.
Starter brief (fill the gaps marked <<...>>):
OUTCOME
Count the records failing each of the three defect definitions in
40-reference/field-policy.md.
INPUTS
10-data/opportunities.csv
<<which second file do you need, and what exactly do you need from it?>>
OUTPUT SHAPE
A markdown table printed in the chat, not written to a file. Columns:
defect_class | count | pct_of_population | example_ids (exactly 3)
CONSTRAINTS
Population is <<state the population and say whether it is rows or distinct
opportunity_id values>>.
Today's date for any date comparison is <<which date, and why that one?>>.
Do not modify any input file.
DEFINITION OF DONE
Three rows. Every count reproducible from the file.
EXCEPTIONS RULE
<<what should happen to a record you cannot classify?>>
Stuck? Open the hints in order.
Hint 1 (Concept)
Two of your three definitions can be answered from one file. The third cannot: one class is only visible when two files disagree. Single-row defects are the easy half of data quality. Defects in the relationship between two rows are the half that survives a careless run.
Hint 2 (Move)
Join opportunities.account_id to accounts.account_id and compare opportunities.currency against accounts.currency row by row. Then decide what happens to the two opportunity rows whose account_id resolves to no account: they cannot pass this test and cannot fail it. Your exceptions rule answers that.
Hint 3 (Shape)
A correct answer has three counts in single figures and low double figures, all under 3% of the population, plus one thing not in the table: at least one record routed to exceptions because its account did not resolve. If your exceptions output is empty, re-read your currency definition. An empty exceptions file on messy data is a warning sign, not a clean bill of health.
What you should see:
| defect_class | count | pct_of_population | example_ids |
|---|---|---|---|
| open opportunity, close date in the past | 14 | 2.7% | OPP-006738, OPP-006894, OPP-006905 |
| owner blank | 4 | 0.8% | OPP-006774, OPP-006841, OPP-007062 |
| currency contradicts account | 4 | 0.8% | OPP-006697, OPP-006729, OPP-007089 |
Population: 524 distinct opportunity_id values. Run date 2026-08-02.
Exceptions: OPP-006774 and OPP-006894, reason "account_id does not resolve in accounts.csv, so the currency test cannot be applied".
Your answer can legitimately differ. A population of 528 rows instead of 524 records gives slightly lower percentages: fine if the audit says which you used. Excluding closed opportunities from "owner blank" gives a count below 4: also fine, since a blank owner on a 2023 closed deal is a different problem from one on live pipeline. Not fine: a count you cannot reproduce, or a population you did not state.
If "open, close date in the past" came back far above 14, check whether you included closed stages. If it came back 0, check whether it used your machine's today instead of the dataset's 2026-08-02.
Worth knowing: the three counts took the agent seconds. The twenty minutes of definition writing is the part still worth something next month.
What you keep
40-reference/field-policy.md, your first defect definitions. The artifact of the module. It survives your CRM, your job and this course.20-briefs/crm-hygiene-audit.brief.md, started from the brief above (a seed is intemplates/M5/crm-hygiene-audit.brief.mdof your practice pack).30-output/hygiene/audit-YYYY-MM-DD.md, the first dated audit.
Check yourself
- Which of these is not usable as a defect definition, and rewrite it so it is: (a) accounts with a blank
owner; (b) opportunities whereamountis at or below zero; (c) records that look wrong; (d) contacts whoseemailcontains no@. - Your audit says 51 defective opportunities. A colleague counts distinct IDs in your proposed-changes file and gets 46. Who is wrong, and what does the audit need to add?
- You inherit an audit with counts and percentages but no
populationcolumn and nodefinition_ref. Name the two questions you now cannot answer.
Where this goes next
L5.3 takes on duplicates, the class where an agent will produce confident nonsense, and where the control is a written rule rather than a better model.