Sampling Plan: Work Out How Much Data You Actually Need Before You Start Collecting

Most measurement work goes wrong before a single row of data is collected. Someone decides to gather data for a fortnight, or to take whatever the system will export, and then argues about whether the result means anything. Sampling Plan reverses that order. You choose the decision first. Ci Flow then shows only the assumptions and sampling controls needed for that purpose.

That matters because a sample sized to estimate a rate is not the same as a sample sized to compare two groups. The first needs a precision target. The second needs a baseline, a target and a power setting. If you ask for both at once you end up with a form full of fields you do not understand and a number you cannot defend.

Reach for this tool in the Measure phase of RDMAICS, after the problem is defined and before anyone opens a spreadsheet. It fits the moment when you know what question you are answering, you know roughly where the data lives, and you need to tell a collection owner how many records to pull and from where.

Everything that follows uses the built in sample data that ships with the tool. The case is fictional. It is there so you can see the mechanics without having to supply your own numbers first.

The scenario is an invoice error study. A finance and operations team wants to know how often posted customer invoices contain an error. They also want to know whether an improvement moves the rate. Three business units feed the same invoice register: Finance with a population of 2600, Operations with 3900 and Sales with 2000. The combined population size entered in the tool is 8500.

The stated rationale is short and specific. Estimate the invoice error rate with plus or minus three percentage point precision and ensure every business unit and shift is represented. That single sentence drives every setting that follows.

The top half of the tool holds the statistical assumptions. The outcome type is set to percentage or rate, because an error rate is a proportion rather than an average. Confidence level is 95 per cent. Statistical power is 80 per cent. The number of groups is 2, reflecting a before and after comparison rather than a three way split between business units.

The comparison itself is expressed as a baseline rate of 20 per cent and a target rate of 12 per cent. That is the shift the study needs to be able to detect. Expected usable data is set to 90 per cent, which acknowledges that not every record pulled will survive cleaning. Allocation method is proportional.

The lower half holds the governance detail that turns a number into a plan. Sampling frame is the invoice audit population for the four week reporting period. Plan owner is the quality and reporting analysts. Inclusion criteria cover all posted customer invoices during the approved study window. Exclusion criteria remove test, reversed and duplicate transactions. Bias risks are written down rather than assumed away, and the sample data names three: selection bias from daytime only sampling, missing supplier responses, and weekly demand variation.

Below the assumptions sits the stratum table. Each row names a group, its population, an optional manual allocation, a sampling frame, a collector, a collection frequency and free text notes.

In the sample data all three rows draw from the ERP invoice register, and all three are collected daily. What differs is ownership and coverage. Finance is owned by a finance analyst with a note to include all three shifts. Operations sits with the operations quality lead and must cover weekdays and weekends. Sales belongs to the sales reporting analyst and must include direct and channel sales.

Those notes are the part teams skip. They are also the part that prevents a sample from quietly becoming a daytime weekday sample of one channel. The manual allocation column is left empty here, which tells the tool to apply proportional allocation based on the population figures.

The result panel returns four headline numbers and two charts. The first number is 658, the base statistical sample, described as the figure before design and usable data adjustment. That is the raw requirement from the assumptions alone.

The second is 731, the recommended collection target, labelled as 20 per cent to 12 per cent at 80 per cent power. The gap between 658 and 731 is the practical allowance, driven by the expected usable data setting of 90 per cent. The third number is 366, the required per group figure across the 2 comparison groups. The fourth is 731 again, the allocated sample, with a prompt to add groups when representation matters.

Two charts sit underneath. Confidence sensitivity plots the requirement at 90, 95 and 99 per cent on a scale running to 1200, which shows you what tightening confidence would cost in collection effort. Planned sample by group plots Finance, Operations and Sales on a scale running to 340, so you can see the proportional split at a glance. The tool also offers an AI review that explains the result in plain language.

The output is not a statistic to file. It is an instruction set. With 731 as the collection target and a proportional split across three units, each collector knows roughly how many records to pull and from which frame.

It also supports a scoping conversation. The confidence sensitivity chart lets you show a sponsor what moving from 95 to 99 per cent would do to the workload before anyone commits. If 731 records across a four week window is not feasible, you can negotiate the assumptions openly rather than quietly collecting less and hoping.

Finally it supports the handover into Analyse. The inclusion and exclusion criteria, the sampling frame and the named owners travel with the number. When someone later asks why reversed transactions are missing, the answer is already written down.

The most frequent error is treating the base statistical sample as the collection target. In this case that would mean collecting 658 records and ending up short once unusable rows are removed. The recommended figure of 731 exists for that reason.

The second is confusing groups with strata. The number of groups setting is 2 because the study compares a baseline against a target. The three rows in the table are strata used for allocation. They are different things and mixing them produces a per group figure that does not match the design.

The third is leaving the bias section empty. A plan that names selection bias from daytime only sampling has a chance of preventing it. A plan that says nothing simply inherits it. The fourth is overriding proportional allocation without recording why in the notes column.

Sampling Plan sizes a sample. It does not tell you whether your baseline rate is correct. The 20 per cent figure in this case is an input, not a finding. If that estimate is wrong, the resulting target of 731 is wrong in the same direction, and the tool has no way of knowing.

It will not select the records for you. It produces counts and an allocation, not a randomised list. Someone still has to draw the sample from the register in a way that honours the notes about shifts, weekends and channels. The tool cannot check whether that happened.

It will not detect bias that you did not name. The bias field is a prompt for your judgement, not a scan. Missing supplier responses will still distort the result even though the field mentions them. Writing a risk down is not the same as mitigating it.

It will also not validate your inclusion and exclusion criteria, confirm that the sampling frame matches the real population, or tell you whether the improvement from 20 per cent to 12 per cent is worth pursuing. Those are business decisions. The AI review explains the calculation, but it works from the assumptions you supplied. Treat the output as a defensible starting point for a conversation, not as proof that your measurement approach is sound.