Which of the following best describes inter-rater reliability?

Which of the following best describes inter-rater reliability?

• A. Agreement between different scorers or raters
• B. Consistency of a test across time
• C. Internal consistency of items
• D. Relationship between test content and outcomes

Answer: (A)

Inter-rater reliability is about agreement between different raters or scorers who evaluate the same thing. When multiple observers assess a performance, observation, or credential, high inter-rater reliability means their ratings line up closely, suggesting the scoring is dependable rather than driven by a single person’s bias or interpretation. This is essential for fairness and trust in assessment results because it shows that the measurement reflects the construct being evaluated rather than who did the rating.

This concept is distinct from other reliability concepts and validity. Consistency of a test across time—test-retest reliability—looks at stability of scores on repeated administrations. Internal consistency examines how well the items on a test hang together to measure the same construct. The relationship between test content and outcomes concerns validity, not reliability. In practice, inter-rater reliability can be quantified with statistics like Cohen’s kappa for categorical ratings or intraclass correlation for continuous ratings.