An Emphasis on Statistical Soundness
Reliability and validity are the statistical criteria used to assess whether an assessment is a good measure. The reliability and validity of an employment assessment are critically important.
Birkman understands the impact our behavioral assessments have on people. We take the reliability and validity of our assessments extremely seriously and have done so since the founding of our company over 60 years ago.
April-May 2018 Test-Retest Reliability Analysis
Birkman hired a third-party market research firm to recruit potential respondents with no knowledge of The Birkman Method that could be presumed willing to take the assessment twice in exchange for compensation. The firm was tasked with drawing an audience sample that closely matches the United States workforce. Procedures of the study included:
- 583 respondents fit the criteria and were sent the Birkman assessment
- 422 respondents completed the questionnaire once
- 15 days later, a new link was sent to all initial respondents
- 414 respondents successfully completed the questionnaire a second time
The range of possible values for the test-retest reliability coefficient is 0 to 1, with neither 0 or 1 to be found occurring within the psychometrics industry. Results between these extremes indicate the following attributes:
- Less than 0.5: Unacceptable
- 0.5 to 0.6: Poor
- 0.6 to 0.7: Questionable
- 0.7 to 0.8: Acceptable
- 0.8 to 0.9: Good
- 0.9 to 1.0: Excellent
Snapshot of Results:
Across all three areas of Birkman scales, results were statistically sound, indicating ongoing scientific reliability and validity as shown on the chart below.
The findings of the study can be found here.
An assessment is considered reliable when the results can be repeated. To be reliable, assessments must meet two requirements:
- Internal consistency
- Test-retest reliability
To demonstrate empirical evidence for reliability, we examine both the internal consistency and test-retest reliability statistics against the best practices defined by organizational psychology.
An assessment is considered valid when the assessment measures what it is stated to measure. To demonstrate empirical evidence of this for Birkman, we test for several types of validity:
- Face validity
- Content validity
- Construct-convergent validity
- Construct-divergent validity
- Criterion-related validity
At Birkman, we are constantly innovating to provide deeper insights into personalities and express the information in an accessible, memorable format. Any reports or scales that we develop are tested rigorously across large sample populations to confirm their validity and reliability.
To learn more about the reliability, validity, and other psychometric properties of our behavioral assessments, please contact us.