How Chally Does Its Validation
The HR Chally Group routinely completes validity studies for specific clients as well as Meta studies across multiple research projects to build validity generalization statistics.
This report summarizes some of the studies that have validated Chally assessment scales as to their accuracy in measuring and predicting sales skills.
Criteria
The criteria used in all studies include one or more of:
- Actual financial sales performance (volume, margin, etc.)
- Intensive interview-gathered ratings by large samples of customers, and
- Supervisor ratings
In
instances where random variables could differentially effect the
criteria, moderator factors are calculated and factored out of the
correlational analyses.
Data Analysis
After the assessments are completed and the criterion collected, the data is input and verified using the Statistical Package for Social Sciences (SPSS) Data Entry Module. The assessments are then scored and tabulated using SPSS. Correlation and Regression solutions are then computed to predict the criterion from the potential predictor scales.
Sampling Variance
Statistical tests indicate that all useful predictions are made at better than 0.05 chance levels. In addition, the results are checked for congruence with similar results found in other studies, and that the data indicate that the predictability achieved is, practically speaking, an important improvement over the usual interview recommendation or related selection procedures.
New data are continuously collected when possible and these scales are updated frequently. The results achieved using the selection scales described in this report are carefully evaluated and improved as new data become available.
Restriction of Range
The range among incumbents usually is smaller than it would be among applicants selected by chance. Due to a variety of factors (e.g., turnover of poor performers, self-selection of individuals with no interest in sales choosing not to apply), it is likely that current employees are more homogeneous than would be a randomly selected group of job applicants. These considerations imply that the correlations typically computed on the basis of the sample data are an underestimate of the accuracy which would be obtained when new applicants are hired.
Technical Results
The results identify scales that predict top performance for the position studied. Included are:
- Descriptive information about each of the validity studies cited for each scale
- The sample size of the study
- The mean or average score achieved by the sample group for the scale
- The standard deviation of the scale
- The correlation or the strength of the scale in predicting top performance
- Significance - the likelihood that the correlation shown occurred only by chance
Definitions of Terms Used Mean: The average score achieved on the scale by sample group. Standard Deviation: Usually examined in conjunction with the mean. The amount of clustering of individual scores around the mean (average). Lower standard deviations indicate in scores across the sample of individual testers, and thus more similarity over the group as a whole. Higher scores indicate greater differences and thus a wider spread between top, average and poor performers. Two digit standard deviations are typically from percentile scores while one digit standard deviations are generally from raw scores. Correlation: The predictive effectiveness of a scale. A higher correlation (closer to 1.00) has stronger predictive power of performance. Conversely, the closer the correlation is to 0.00, the weaker the scale. Significance: The likelihood that the correlation shown occurred only by chance. |