An Emphasis on Statistical Soundness

Reliability and validity are the statistical criteria used to assess whether an assessment is a good measure. The reliability and validity of an employment assessment are critically important.

Birkman understands the impact our behavioral assessments have on people. We take the reliability and validity of our assessments extremely seriously and have done so since the founding of our company over 60 years ago.

April-May 2018 Test-Retest Reliability Analysis

Birkman hired a third-party market research firm to recruit potential respondents with no knowledge of The Birkman Method that could be presumed willing to take the assessment twice in exchange for compensation. The firm was tasked with drawing an audience sample that closely matches the United States workforce. Procedures of the study included:

  • 583 respondents fit the criteria and were sent the Birkman assessment
  • 422 respondents completed the questionnaire once
  • 15 days later, a new link was sent to all initial respondents
  • 414 respondents successfully completed the questionnaire a second time

The range of possible values for the test-retest reliability coefficient is 0 to 1, with neither 0 or 1 to be found occurring within the psychometrics industry. Results between these extremes indicate the following attributes:

  • Less than 0.5: Unacceptable
  • 0.5 to 0.6: Poor
  • 0.6 to 0.7: Questionable
  • 0.7 to 0.8: Acceptable
  • 0.8 to 0.9: Good
  • 0.9 to 1.0: Excellent

Snapshot of Results: 

Across all three areas of Birkman scales,  results were statistically sound, indicating ongoing scientific reliability and validity as shown on the chart below.

Validity

Reliability

An assessment is considered reliable when the results can be repeated. To be reliable, assessments must meet two requirements:

  • Internal consistency
  • Test-retest reliability

To demonstrate empirical evidence for reliability, we examine both the internal consistency and test-retest reliability statistics against the best practices defined by organizational psychology.

 

Validity

An assessment is considered valid when the assessment measures what it is stated to measure. To demonstrate empirical evidence of this for Birkman, we test for several types of validity:

  • Face validity
  • Content validity
  • Construct-convergent validity
  • Construct-divergent validity
  • Criterion-related validity

At Birkman, we are constantly innovating to provide deeper insights into personalities and express the information in an accessible, memorable format. Any reports or scales that we develop are tested rigorously across large sample populations to confirm their validity and reliability.

 

To learn more about the reliability, validity, and other psychometric properties of our behavioral assessments, please contact us.