Step 2 — Write gold questions

Copy docs/gdf-eval-kit/questions.template.json from your repository checkout and fill in:

{
  "corpus": {
    "gdf_url": "https://guides.co/g/{slug}/gdf",
    "plaintext_url": "https://guides.co/g/{slug}/gdf?format=plaintext",
    "chunks_url": "https://guides.co/g/{slug}/gdf?format=jsonl"
  },
  "questions": [
    {
      "id": "q1",
      "question": "…",
      "gold_answer": "…",
      "expected_item_id": "item-page-1",
      "difficulty": "lookup"
    }
  ]
}

Critical: expected_item_id must match an id in your GDF export (items[].id). After import via API, open GET /g/{slug}/gdf and copy real ids (often item-{number} on live guides).

Write 10-30 questions spanning:

  • lookup - answer on one page
  • procedural - steps spanning one page
  • cross-section - requires choosing among similar pages