Ten were requested. The model decided otherwise.

We requested ten questions from the generation service. Sometimes we got eight. Sometimes twelve. Sometimes exactly ten, except one of them had a stored correct-answer key that didn't match any of the four options actually listed.

The stored metadata said "10 questions" every time, because that's what we'd requested. What the student actually got varied. The record of what a student took and what they were actually shown could disagree, and nobody would know until someone tried to grade it.

The bug: we stored the request, not the result. Two different pieces of data that happened to usually agree, until they didn't.

Take 1: trust whatever comes back

async function generateQuiz(topicId: string, requestedCount: number) {
  const result = await model.generateQuestions(topicId, requestedCount);

  await db.quizzes.insert({
    topicId,
    questionCount: requestedCount,
    questions: result.questions,
  });
}

questionCount was set from requestedCount, the number we asked for, not result.questions.length, the number we actually got. If the model returned eight questions, the quiz record still claimed ten. Anything downstream that used questionCount to compute a score percentage or check completion was working off a number that had nothing to do with the array sitting next to it.

The malformed-answer case was worse because it wasn't a count mismatch at all. A question with a correct-answer key of 2 when there were only two options, indices 0 and 1, would sit in the array looking completely normal until a student answered it and the grading logic tried to resolve an index that didn't exist.

Take 2: enforce, then recompute

The fix has three parts. Trim excess questions immediately if too many came back. Retry with a corrective prompt if too few came back, and only accept a short quiz on the final retry attempt, not the first. And drop any individual question whose answer key can't be resolved against its own options before anything gets persisted.

function dropMalformedAndRecount(questions: GeneratedQuestion[]) {
  const valid = questions.filter(
    (q) =>
      Number.isInteger(q.correctIndex) &&
      q.correctIndex >= 0 &&
      q.correctIndex < q.options.length,
  );

  return {
    questions: valid,
    questionCount: valid.length,
  };
}

async function generateQuiz(topicId: string, requestedCount: number) {
  let questions = await generateWithRetries(topicId, requestedCount);

  if (questions.length > requestedCount) {
    questions = questions.slice(0, requestedCount);
  }

  const { questions: surviving, questionCount } =
    dropMalformedAndRecount(questions);

  await db.quizzes.insert({
    topicId,
    questionCount,
    questions: surviving,
  });
}

The key change is questionCount comes from valid.length, computed after the malformed questions are already gone. It never reads from requestedCount at the point of persistence. The number we store is a description of what's actually in the array next to it, not a memory of what we originally asked for.

generateWithRetries handles the shortfall case separately: if the model returns fewer than requested, it retries with a prompt that states the exact shortfall ("generate 3 more questions, distinct from the ones already produced") rather than repeating the full original request. Only on the last configured retry does it give up and accept whatever count it has.

What I should have done first

I treated the request and the result as the same fact because in testing they always matched. They matched right up until the day the model returned nine questions instead of ten and nobody had written the code path for that, because nobody had needed it yet.

The general version of the mistake: any time you store a number that describes external output, derive it from the output you actually kept, at the last possible moment, never from the request that produced it. The request is what you wanted. The array is what you got. Only one of those is true after the fact.