Problem and Data Review: Test Your Measure Evidence Before You Hunt for Root Causes

Problem and Data Review is the gate between Measure and Analyse. Its job is narrow and useful. It tests whether the Measure evidence is complete, traceable and strong enough to support root-cause analysis without silently filling gaps.

The word silently matters. Most teams do not fail Analyse because they lack data. They fail because one row of evidence was thin, nobody said so, and the fishbone session treated that row as fact. Problem and Data Review forces each piece of evidence to declare itself: what the problem is, where the number came from, how many observations sit behind it, what the data showed, and how confident you are.

Use it once Measure is finished and before you open any cause analysis tool. Use it again if the scope of the project changes, or if a new data source arrives mid-project. It takes minutes to complete and it is cheap insurance against a root cause built on sand.

Everything in this guide is built on one case. The case is fictional. It is the sample data that ships with the tool, and it describes an accounts payable team whose invoice process is missing several targets at once.

The team has four problem statements on the table. Invoice cycle time exceeds 48 hours against a target of 24. First-pass match rate sits at 65 per cent against a target of 95. Invoice error rate sits at 12 per cent against a target of 2. On-time payment rate sits at 78 per cent against a target of 98.

Four problems, four different data sources, four different sample sizes. That is a normal shape for a Measure phase. It is also exactly the shape that hides weak evidence, because the strong rows make the thin rows look respectable by association.

The input table has five columns per row. You add one row per evidence statement. In the sample case there are four rows.

Row one records the cycle time problem. The source is an ERP export for Q1 2025. The sample size is 1,240 invoices. The key finding is an average cycle time of 46.2 hours and a median of 42 hours. Confidence is recorded at 95 per cent.

Row two covers first-pass match. The source is AP system logs, the sample is 850 records, and the finding is that 35 per cent of invoices require manual rework. Confidence is 95 per cent. Row three covers the error rate. The source is QC audit sheets, the sample is 300 sampled, and the finding is that the top three error types account for 78 per cent of defects. Confidence is recorded at 90 per cent. Row four covers on-time payment. The source is the payment system, the sample is 2,100 payments, and the finding is that late payments correlate with error rate. Confidence is 95 per cent.

Note how specific the source entries are. A source entry such as ERP export Q1 2025 can be found again by somebody else. A vague entry cannot. Traceability is half of what this tool is checking, so write source names that a colleague could follow without asking you.

The result panel summarises the evidence base before it judges it. For this case it reports 4 evidence statements, 4,490 recorded observations, 4 distinct data sources and 4 of 4 complete evidence rows.

Evidence readiness is scored against a simple rule. Each row needs a problem, a source, a sample size, a finding and a confidence statement. The sample case scores 100 per cent, with no obvious evidence gaps in the recorded rows. That is the headline, and it is worth reading carefully. It says the rows you recorded are complete. It does not say the rows you recorded are everything that exists.

Below the score, the tool reprints each statement with its source, sample size, finding and confidence level, so you can read the evidence base as a single block rather than as four separate conversations. A sample coverage by source chart then shows the volume behind each source: 1,240 records from the ERP export, 850 from AP system logs, 300 from QC audit sheets and 2,100 from the payment system.

That coverage chart is where the sample case becomes interesting. The QC audit sheets contribute 300 records against 2,100 from the payment system. Both rows are complete. They are not equally heavy. The tool also offers an AI review that explains the result in plain language.

A 100 per cent readiness score on four complete rows supports a clear decision: you may open root-cause analysis on this evidence base. That is the main call, and the tool exists to let you make it deliberately rather than by habit.

The result also supports sequencing decisions. The error rate row carries the lowest confidence in the set at 90 per cent and the smallest sample at 300. If your first cause analysis leans hard on error types, you now know which plank is thinnest and can decide whether to widen the audit before or after you start.

There is a third decision hiding in the structure. Row four states that late payments correlate with error rate. Row three tells you the top three error types account for 78 per cent of defects. Reviewed together, those two rows suggest where a cause analysis should begin. The tool does not draw that line for you. It simply places the evidence side by side so you can see it.

The most common mistake is treating the readiness score as a verdict on data quality. It is a verdict on record completeness. In this case all four rows are filled in, so the score reads 100 per cent. A row with a weak source and a filled-in confidence field would also score as complete.

The second mistake is padding. Teams sometimes add rows to make the evidence base look larger. The observation count rises, the readiness score stays flat, and the Analyse phase gets slower. Four well-sourced rows beat ten vague ones.

The third mistake is writing conclusions into the key finding column. Average cycle time equals 46.2 hours and median equals 42 hours is a finding. A sentence blaming a team or a system is a conclusion, and conclusions recorded as findings tend to survive the whole project unchallenged.

Problem and Data Review will not tell you whether your data is accurate. It cannot open the ERP export or the QC audit sheets and verify the numbers. If the 46.2 hour average was calculated on the wrong date field, the tool will still mark that row complete.

It will not tell you whether your sample is representative. It reports 1,240 invoices and 300 sampled. It does not know how those were selected, whether the quarter was typical, or whether a seasonal effect sits inside the window. Sample size and sample quality are different things, and the tool only sees the first.

It will not validate your confidence levels. The 95 per cent and 90 per cent figures in the sample case are statements you typed. The tool records them and shows them beside each finding. It does not recalculate them from the underlying data.

It will not find the evidence you never recorded. The sample case reports no obvious evidence gaps in the recorded rows, and that qualifier is deliberate. A missing fifth problem statement, an entire data source nobody pulled, a process step with no measurement at all: none of that appears here. Judging what is absent is still your work.

Finally, it will not do root-cause analysis. It states no causes and ranks no drivers. It tells you whether your evidence is ready for that conversation, and then it hands the conversation back to you and your team.