Question Banks: How to Build One So Different Exams Stay Equivalent
Random selection from a bank is only fair when the questions are categorized and leveled by difficulty. Without that, one candidate gets an easy exam and another gets a hard one.
A question bank is a repository of questions from which exams are assembled. Its advantage is reuse and randomization; its risk is inequality between candidates.
The two dimensions every question needs
- Category (topic). Which unit of the syllabus the question belongs to.
- Difficulty level. Easy, medium, hard.
Without these two, random selection means one candidate might get five easy questions from one topic while another gets five hard questions from a different one. That exam tests luck, not knowledge.
A selection rule instead of pure chance
Instead of "10 random questions out of 200," write:
| Category | Easy | Medium | Hard |
|---|---|---|---|
| Core concepts | 2 | 1 | 0 |
| Application | 1 | 3 | 1 |
| Analysis | 0 | 1 | 1 |
Now every candidate sees different questions, but the structure of their exam is identical.
Don't guess the difficulty — measure it
The level the question writer assigns often doesn't match reality. After the first run, look at the percentage of correct answers for each question:
- Above 85% correct: the question is easy (or it contains a clue).
- Below 25% correct: it's either hard or ambiguous — read it again.
- Around 50–70%: the sweet spot for discriminating between candidates.
A question everyone gets right and a question everyone gets wrong both fail to separate candidates, and they are effectively empty slots in the exam.
Bank size
Rule of thumb: the bank should hold at least five times the number of questions on the exam for randomization to be meaningful. For a 20-question exam, 100 questions. Below three times, the overlap between exams is so large that the benefit disappears.
Retire burned questions
A question that has appeared in several live exams has probably leaked. The telltale sign: its percentage of correct answers jumps suddenly without anything changing in the teaching. Set that question aside.