What you will be able to do
- Translate a population question into a documented probability sample.
- Calculate precision allowances, design effects and defensible weights.
- Implement verification and hand over data for design-aware analysis.
Before you begin: Basic proportions, multiplication and familiarity with questionnaire data.
Define who the survey is intended to represent
Specify the target population before discussing sample size. All registered enterprises operating in a district on a reference date is different from all enterprises, including informal firms. A business register may omit recent entrants and include firms that have closed. Compare the frame with available listing information and record known coverage gaps. Next, distinguish the selected unit from the respondent: an enterprise may be selected, while an owner or manager supplies information. Establish which respondent is qualified and how substitutions within the enterprise are handled. More interviews cannot repair a frame systematically excluding the population of interest. A coverage note should therefore accompany the sampling plan, explaining exclusions, updates and the boundaries within which the eventual estimates can reasonably be interpreted.
Nearby interviews can contain less independent information
Cluster sampling can reduce travel costs, but enterprises in the same area may share infrastructure, markets and shocks. Their answers therefore contain overlapping information. For a simplified equal-sized cluster design, an approximate design effect is one plus average completed cluster size minus one, multiplied by the intracluster correlation. With twenty completions per cluster and a correlation of zero point zero five, this becomes one point nine five. This outcome-specific approximation does not automatically capture unequal cluster sizes, weighting or stratification. Obtain plausible inputs from comparable data and test sensitivity before finalising the budget. Adding more independent clusters can improve information differently from adding more interviews within existing clusters. Treat field logistics and statistical precision as a joint design problem rather than separate workstreams.
Adjust nonresponse using defensible response groups
A response adjustment tries to account for sampled units whose outcomes are missing. Within a response cell, dividing the base weight by the estimated response rate increases representation for respondents. Its credibility depends on respondents and nonrespondents being comparable for the outcome after conditioning on the adjustment information. Choose cells or a model using relevant variables available for all sampled units, not only respondents. Examine response patterns by size, geography and other pre-existing characteristics, avoiding cells too small for stable estimates. Weighting cannot guarantee removal of bias caused by unmeasured differences. Document callbacks, contact failures, refusals and ineligibility separately because they have different implications. Report the adjustment assumptions and consider sensitivity analyses when plausible nonresponse mechanisms could materially change the substantive conclusion.
Deliver design information with the cleaned data
A spreadsheet is not a complete survey handover. Supply a data dictionary, questionnaire versions, sampling frame notes, selection probabilities, cluster and stratum identifiers, and the final weighting procedure. Include a disposition summary explaining response and eligibility, along with a correction log that preserves original values. Provide analysis code that calculates estimates and their uncertainty using the actual design. For subgroup reporting, use a suitable domain estimation approach rather than casually discarding observations needed for variance estimation. Check weighted totals and identify unstable estimates or sparse domains before publication. Keep identifiers needed for design analysis while removing or protecting direct personal identifiers according to the agreed data arrangements. This package allows a later analyst to reproduce results and judge the limits of what the survey represents.
Apply the method
Work through each task before opening the model answer. Compare your reasoning with the review criteria. Written answers stay on this page and clear when you leave or reload it.
Applied task 1
Show model answer and review criteria
The adjusted completion requirement is ceil(385 × 1.95) = 751. The initial selection requirement is ceil(751/0.80) = 939. Selecting 38 × 25 = 950 yields 760 expected responses, or 20 per cluster, consistent with the design-effect assumption. Achieved response and precision must still be checked.
- Correct rounding and distinction between selected and completed units.
- Allocation is linked to expected completed cluster size.
- States assumptions and avoids guaranteeing achieved precision.
Applied task 2
Show model answer and review criteria
Record both probability stages; base weight is 50 and adjusted weight is 62.5 under defensible response-cell assumptions. Use callbacks and dispositions rather than convenience replacements. Investigate flags with authorised verification, preserve raw data and correction logs, and deliver strata, clusters, weights and design-aware analysis code.
- Correct weights with an explicit response assumption.
- Field procedures preserve selection and verify rather than automatically delete.
- Handover contains design information and reproducible analysis.
Method references
- United Nations Statistics Division — Designing Household Survey Samples: Practical Guidelines
Probability sampling, multistage selection, design effects, weighting and survey documentation.
- J-PAL — Sampling
Population, frame and sampling choices.
- J-PAL — Data quality checks
Operational checks and verification of field data.
- J-PAL — Power calculations
Relationship between sample design, clustering and statistical information; precision examples here are original calculations, not treatment-effect power calculations.