A data collection instrument is defensible when every item on it can be traced back to a construct in your framework, and examiners check that by asking what a given question was for. Design the instrument from the constructs outwards, never from a list of questions that seem interesting, and put the finished instrument in an appendix so the study could be repeated.
Three templates follow — a questionnaire, an interview topic guide and an observation schedule — each with the structure, the item-writing rules and a skeleton you can fill in, plus the matrix that keeps all three honest. Where a validated scale already exists for your construct, adapting it beats building one, and our comparison of validated instruments for psychology research covers how to choose between the established measures. Research degree regulations and ethics procedures differ by institution; check your own before you build anything.
Step 1: Decide whether to build or to adapt
Build only when you have to. A validated instrument arrives with published reliability and validity evidence, a scoring procedure and a literature you can compare your findings against, and none of that can be manufactured inside a Master’s timetable. The cost is licensing, which some instruments require, and fit, which is often imperfect.
Adapt when an instrument nearly fits: change the wording of items for your setting, declare every change, and note that the published psychometric properties no longer strictly apply to your modified version. Build from scratch only when your construct has no established measure — which is common for organisational and policy constructs and rare for psychological ones — and accept that you will then have to demonstrate the instrument’s adequacy yourself, through piloting and through the reporting of internal consistency where the design allows it.
| Situation | Do this | What to report |
|---|---|---|
| Validated instrument exists and fits | Use it unchanged | Source, version, licensing status, scoring procedure |
| Validated instrument nearly fits | Adapt and declare | Every change made, why, and the caveat on published properties |
| No established measure | Build from the construct definition | How items were derived, pilot findings, any reliability evidence |
| Qualitative study | Build a topic guide from the constructs | The guide itself in an appendix, plus how it changed during fieldwork |
Step 2: Build the instrument matrix first
Before writing a single item, make a table with one row per construct and columns for the research question it serves, the items or prompts that address it, and the analysis that will use them. This is the single most useful half-hour in the whole design, because it exposes two faults immediately: constructs with no items, and items attached to no construct.
The second is the more common. Every instrument accumulates questions that are interesting rather than necessary, and each one costs participant attention and response rate. If an item does not appear in the matrix, it does not go on the instrument. The constructs themselves come from your conceptual framework, and if the matrix reveals constructs you cannot measure, the framework is too large rather than the instrument too small.
Step 3: The questionnaire template
The order below is deliberate. Screening comes early so ineligible respondents do not waste their time or your data; demographics come last because they are the least engaging and the most likely to cause abandonment if they open the instrument.
- Title and purpose — one sentence on what the study is, one on why they were approached.
- Participant information and consent — as approved by your ethics committee, with the consent statement before any question.
- Screening items — the one or two questions that establish eligibility, with a clean exit for those who do not qualify.
- Core sections, one per construct — each with a short instruction line naming the timeframe and the frame of reference.
- Open comment box — optional, one only, placed after the core sections.
- Demographics — only those in your matrix, each with a prefer-not-to-say option.
- Close — thanks, what happens to the data, and who to contact.
The item-writing rules are few and they are the ones pilots reliably catch being broken. One idea per item, so nothing is double-barrelled: “the guidance was clear and easy to apply” is two questions wearing one answer box. No leading wording, which usually means removing an evaluative adjective. Avoid negation, because agreeing with a negative statement is cognitively expensive and produces unreliable data. Match the response scale to the question — frequency scales for behaviour, agreement scales for attitudes, and never a satisfaction scale for something the respondent has no basis to evaluate. Label every scale point rather than only the ends, and keep the number of points consistent across a section.
Step 4: The interview topic guide template
A structured interview uses a schedule, where the wording is fixed and asked in order. A semi-structured interview uses a topic guide, where the domains are fixed and the wording is not. Most postgraduate qualitative work uses the second, and the commonest error is writing a guide that is really a schedule, then reading it aloud.
- Opening — consent confirmed on record, recording started, the purpose restated in one sentence.
- Warm-up — one easy, concrete question about their role or their day. Never start with the construct.
- Domain 1 to n — one domain per construct, each with a single opening question and three or four planned probes.
- Critical incident prompt — “tell me about a specific time when …”, which produces the most usable data in the whole guide.
- Contrast prompt — “has that always been the case?” or “how does that differ from …”, which surfaces conditions and change.
- Closing — what have I not asked about that matters, and the invitation to add anything.
Write the probes out even though you will not read them, because the probe you forget in the moment is the one you needed. Keep the guide to one or two pages: a six-page guide is a schedule, and it will produce thin, compliant answers. Note that the guide will evolve as fieldwork proceeds, which is legitimate and must be reported — say in the methodology chapter which prompts were added after which interview and why. What you then do with the transcripts is a separate craft, covered in our guide to writing up qualitative research when you have forty transcripts and no findings chapter.
Step 5: The observation schedule template
Observation splits into structured, where you record predefined behaviours against a grid, and unstructured, where you write field notes against broad foci. Both need a written instrument; the unstructured one is a set of foci and conventions rather than a grid, but it still goes in the appendix.
A structured schedule needs five things. A behaviour list with each category defined precisely enough that two observers would code the same event identically. A sampling rule — time sampling records what is happening at fixed intervals, event sampling records every occurrence of the defined behaviours, and the choice changes what your counts mean. A recording grid with the setting, date, start and end times and observer at the top. A context field, because a count without context is uninterpretable. And a reflexivity note on your own presence and role, which is data rather than an apology.
For unstructured observation, replace the behaviour list with three or four foci derived from your constructs, and adopt a field-note convention that separates description from interpretation on the page — two columns, or a bracketing convention — so that six months later you can still tell what you saw from what you thought about it.
Step 6: Pilot it, and know what the pilot is for
A pilot is not a small version of the study. It tests the instrument, and it tests four specific things: whether respondents understand the items as you intended, how long completion actually takes, whether the response options cover the real range of answers, and whether the data that comes out can be analysed the way you planned. Run at least three or four people who resemble your sample, and where possible ask them to think aloud while completing it.
Expect to cut. Pilots almost always show that the instrument is too long and that two or three items are ambiguous. Record what you changed and why, because that paragraph belongs in the methodology chapter; a piloted instrument with documented revisions reads as careful design, while an unpiloted one invites the viva question about how you knew the items worked. The full sequence a methods chapter follows, and where the instrument section sits within it, is set out in our guide to writing the methodology chapter.
Step 7: Remember the instrument is part of your ethics approval
Committees approve the instrument, not just the study. The questionnaire, the topic guide or the observation schedule is normally submitted with the application, along with the participant information sheet and consent form, and substantive changes afterwards require an amendment rather than a note in your file. Adding a question about a sensitive topic, changing who you approach, or altering how data is stored are all amendable changes.
Plan the sequence accordingly: constructs, then matrix, then draft instrument, then pilot, then ethics application, then fieldwork. Piloting after approval is possible but means any change the pilot demands has to go back for amendment. Where your study touches health or social care services, a second approval regime applies on top of the university’s, as our guide to ethics approval for sociology and social work research explains.
Step 8: Report it so the study could be repeated
The methodology chapter should say where each instrument came from, how items were derived from constructs, what the pilot changed, how the instrument was administered, and where the full instrument appears in the appendices. Put the complete instrument in an appendix, not a sample of it — an instrument summarised rather than reproduced cannot be evaluated, and reproducibility is one of the things examiners are explicitly looking for.
The instrument also has to agree with everything around it. Who you administered it to is your sampling strategy; what the items measure is your framework; how the resulting data was handled is your analysis section. A mismatch between any two of those is visible to an examiner reading the appendix against the chapter.
What changes at doctoral level
Scale and defence. A doctoral thesis is expected to justify the instrument against alternatives that were considered and rejected, to report reliability evidence where the design permits it, and to address translation and cultural adaptation explicitly where the instrument crosses a language boundary. Multi-phase designs must also explain how the instruments relate: whether the interview guide was derived from the survey findings, and if so, how. Interview-heavy doctoral projects should settle the transcription approach at the same time as the guide, since the two decisions constrain each other — our comparison of transcription tools for research interviews covers the accuracy, ethics and cost trade-offs.
Building the instrument alongside the chapter
The instrument and the chapter that describes it are usually written months apart, which is how instruments end up in appendices that do not match the text describing them. Draft the matrix, the instrument and the methodology section together and keep them in one place. Draft your thesis in Tesify and hold the constructs, the instrument matrix and every chapter in one workspace, so that the appendix a reader turns to is the instrument your chapter actually describes. The design and the items stay yours; there is a free plan, and the useful first step is the matrix — one row per construct, before any item is written.
Frequently asked questions
Should I use a validated instrument or write my own questions?
Use a validated instrument whenever one fits your construct, because it brings published reliability and validity evidence and a literature to compare against. Write your own only where no established measure exists, and then derive every item from your construct definitions and pilot them.
What is the difference between an interview schedule and a topic guide?
A schedule fixes the wording and the order and is used for structured interviews. A topic guide fixes the domains but not the wording, and is used for semi-structured interviews. Most postgraduate qualitative work needs a guide; writing a schedule and calling it a guide produces thin data.
How many questions should a questionnaire have?
As many as your matrix requires and no more. The practical constraint is completion time rather than item count, and a pilot tells you what yours actually is. Every item without a construct behind it costs response rate and buys nothing.
Can I change my instrument after ethics approval?
Substantive changes require an amendment to the approval, not a note in your file. That includes adding questions on sensitive topics, changing who you approach and altering data handling. Pilot before you apply where you can, so the instrument you submit is the one you will use.
Where do demographic questions go?
At the end, after the core sections, because they are the least engaging part of the instrument and opening with them raises abandonment. Ask only those in your matrix, and give every one a prefer-not-to-say option.
What makes an item double-barrelled and why does it matter?
It asks two things at once — “the guidance was clear and easy to apply” — so a single response cannot be interpreted, because you cannot tell which half the respondent was answering. Split it into two items or drop one half.
How do I show an observation schedule is reliable?
Define each behaviour category precisely enough that two observers would code the same event identically, and where the design allows, have a second observer code a subset and report the agreement. Precision of definition does more work than any statistic calculated afterwards.
Does the full instrument have to go in the appendix?
Yes, complete rather than in sample form. A summarised instrument cannot be evaluated and cannot be repeated, and reproducibility is one of the things examiners look for directly. Cross-reference it from the methodology chapter.
What should a pilot actually test?
Four things: whether respondents understand the items as intended, how long completion really takes, whether the response options cover the real range of answers, and whether the resulting data supports the analysis you planned. Record what you changed as a result.