Back to Academy
Relevance 7/10Prompting and EvaluationIntermediate5 min read
Context Window Adherence
Context window adherence checks whether responses use available context without ignoring key evidence.
Why it matters for annotators
Context misuse often causes factual and instruction-following errors.
Visual mental model
Provided context -> response alignment check.
Examples (bad vs good)
Scenario: Real annotation scenario involving Context Window Adherence
Bad: Labeling quickly without applying project rubric.
Good: Applying rubric criteria, documenting rationale, and escalating uncertainty.
Common mistakes
- Skipping guideline details for edge cases.
- Applying inconsistent criteria across similar samples.
- Avoiding escalation even when uncertain.
Submission checklist
- Read the latest guideline update before each batch.
- Apply rubric dimensions explicitly in each decision.
- Escalate ambiguous items with concise rationale.