BrandX campaign effectiveness · Step 3 of 6
Research design
From approach to instrument: the design, the sample, and the questionnaire
Introduction to Research Design
Real-world practice. A quasi-experimental post-test-only design is fielded through an online panel: respondents who naturally watch Channels A or B form the exposed stream, while non-viewers become the control.
Portfolio Simulation
This portfolio demonstrates the same research design through a simulated dataset that reproduces the exposed-versus-control assignment using the planned channel-viewing rules. The objective is to mirror the intended commercial workflow while maintaining complete transparency about the simulated implementation.
Survey Methodology
Real-world practice. The questionnaire would be programmed on a panel platform and run until 1,000 qualified Canadian completes are achieved, balanced continuously across the quota grid.
Portfolio Simulation
Rather than recruiting respondents through a commercial online panel, this portfolio uses a simulated 1,000-case dataset that reproduces the planned quota structure, respondent characteristics, routing logic, and exposure assignment. The simulation mirrors how the proposed study would operate under live field conditions while allowing every analytical step to be reproduced transparently. The simulation methodology, source code, and synthetic dataset are documented in Steps 4 and 5, so that every analytical output presented in later steps can be traced back to its source.
Experimental Design
Real-world practice. Mid-survey, exposed respondents watch the live TV spot; control respondents skip directly to brand questions, allowing a clean exposed-vs-control comparison.
Portfolio Simulation
The portfolio simulation applies the same exposure logic that would be used in a commercial campaign evaluation. This allows the subsequent analysis to demonstrate how an exposed-versus-control comparison would be conducted while making clear that no live advertising exposure actually occurred.
Control Group
Respondents who have not viewed Channels A or B during the campaign period
n = 500
Exposed Group
Respondents who have viewed Channels A or B during the campaign period
n = 500
Two principles govern this design. First, "exposed" means opportunity to see (OTS): membership is defined by channel viewership during the campaign window, not verified exposure to the spot, so lift estimates should be read as conservative. Second, impact is derived, never stated: respondents are not asked whether the advertising changed their views. Brand metrics are measured identically in both groups before the video is shown, and the exposed-versus-control difference reveals the campaign's in-market effect. The full reasoning, power calculations, and limitations are below.
Methodological notes in full: measurement logic, statistical power, and limitations
Why derived measurement. Self-reported advertising impact is unreliable: people cannot accurately introspect on what shaped their perceptions, and the question itself leads them. Each brand metric is therefore measured identically in both groups, without reference to the campaign, and conclusions rest on the comparison rather than on respondents' own attributions. Question order protects this logic: all brand health measures precede the video, so group comparisons reflect the in-market effect rather than a forced-exposure effect, while the video enables the creative and message diagnostics that follow it.
Why OTS is conservative. Some respondents classified as exposed will never have encountered the BrandX spot in-market. Unverified exposure dilutes measured differences toward zero, which biases the study against finding effects, not toward them.
Statistical power. With approximately 500 respondents per group, and for metrics near 50% incidence, the minimum detectable difference between groups is roughly 9 percentage points at 80% power; smaller true effects may go undetected, and results are interpreted with this sensitivity floor in mind. For subgroup analyses, where cell sizes are smaller and detectable differences correspondingly larger, results are reported only for cells of n = 50 or more; cells of n = 30 to 49 are flagged as indicative, and smaller cells are suppressed.
Design limitation. Because respondents self-select into viewing Channels A and B, the exposed and control groups may differ in ways beyond campaign exposure. Rim weighting corrects demographic skews but not self-selection. In a live engagement, I would strengthen causal claims with a matched-market design or pre-post brand tracking; for this study, results are interpreted with that caveat in view.
Sampling Strategy
Real-world practice. Stratified quotas are set with the Sampling Plan Builder and enforced in field: gender parity, three grouped age bands, province targets, and minimum completes per viewing tier. The questionnaire captures age in six brackets, which the analysis retains; the grouped bands exist only to keep field quotas manageable.
Portfolio Simulation
The portfolio simulation applies the same quota structure defined in the sampling plan, producing respondent distributions that closely follow the intended commercial design. If this study were conducted using live respondents, any remaining demographic imbalance would be corrected through rim weighting. Because the synthetic dataset is generated directly from the planned quotas, additional weighting is unnecessary.
| Gender Distribution | ||
|---|---|---|
| Gender | Sample Size | Percentage |
| Male | 490 | 49% |
| Female | 490 | 49% |
| Gender diverse / other (accepted to total) | 20 | 2% |
| Age Distribution | ||
| Age Group | Sample Size | Percentage |
| 18-34 | 350 | 35% |
| 35-54 | 340 | 34% |
| 55+ | 310 | 31% |
| Canadian Province Distribution | ||
| Province | Sample Size | Percentage |
| Ontario | 400 | 40% |
| Quebec | 235 | 23.5% |
| British Columbia | 145 | 14.5% |
| Alberta | 95 | 9.5% |
| Other Provinces & Territories | 125 | 12.5% |
Questionnaire Design
The proposed questionnaire is designed to measure brand awareness, perception, and advertising effectiveness across four sections: screening, brand metrics, advertising response, and demographic profiling. Individual items carry section codes (A for screening and demographics, B for brand metrics, and C for advertising response) that support the analysis presented in Step 5.
Questionnaire Architecture and Preview
The proposed questionnaire is intentionally organized into four measurement sections. Each section serves a distinct research objective, and the sequence follows established questionnaire design principles by measuring brand perceptions before advertising diagnostics. This ordering helps minimise response bias while ensuring every research objective introduced in Step 2 is captured systematically.
| Questionnaire section | Purpose | Key measures |
|---|---|---|
| A. Screening and media behaviour | Qualify eligible respondents and classify opportunity-to-see (OTS) exposure while establishing the routing logic used throughout the questionnaire. | Snack purchase, snack consumption, Channel A/B viewing, exposure flag. |
| B. Brand health metrics | Measure baseline brand health before any advertising evaluation, protecting the integrity of the derived brand-lift analysis. | Unaided awareness, aided awareness, impression, familiarity, consideration, recommendation. |
| C. Advertising response | Evaluate advertising recall, creative performance, message communication, and persuasion among respondents classified as exposed. | Ad recall, enjoyment, creative response, persuasion statements, unaided message takeaway, aided message takeaway. |
| D. Demographics and shopper profile | Capture demographic and behavioural characteristics to support segmentation, subgroup analysis, weighting, and interpretation of campaign performance. | Shopper role, purchase frequency, snack spend, employment, education, household profile, income. |
Question order is intentional. Brand-health measures are collected before advertising diagnostics so that campaign evaluation reflects existing perceptions rather than reactions created by immediate ad exposure. This sequencing supports the study's derived measurement approach and helps ensure that observed differences between the exposed and control groups reflect campaign lift rather than survey-induced effects.
The interactive preview below illustrates how respondents would progress through the proposed questionnaire. It demonstrates screening, routing logic, exposure classification, conditional advertising diagnostics, and the overall respondent experience that would be implemented in a commercial online survey.
Note: To review the complete programmed version, including routing logic, embedded variables, and questionnaire annotations, please refer to the accompanying questionnaire documentation.