Week 12 Slides

Interview Assignment Feedback

• Display Competence & Conscientiousness– What changes did you decide to keep?– How did you decide to score?– What KSAO gap did the interview fill?– Some of you included assessments that were not in the scenario.

• What was the purpose of the interview? – Fill the KSAO gap or measure all KSAO’s?

• Be careful with the selection plan– Use SME judgments if you can

• Pilot your instructions and forms • You can make your points back by revising and resubmitting

once.

Assigned Readings

• Allen et al (2013) Socialization• Orvis et al (2008) Contract Breach

Section 6: Raw data Interpretation (10 points)

• Create a correlation matrix for our scale, TI, and PW.– Summarize the results presented in the

correlation matrix. Does it make sense theoretically?

• Summarize the sample by analyzing the demographic variables and reporting the sample size.– Generalizable?Explain.

Preliminary Cleaning

• Deleted those that did not finish.• Removed all identifiable data• Converted old SAT scores– There may still be some cases I missed

• Removed the GRE question

What you need to do!

• Determine what data should be included– Incomplete data?– Response sets?– Working and tenure?– Sample size

• Code the Occupational (free-response) data into meaningful categories.

• Dummy code accordingly.

Statistical Procedures• Screen out bad data• Conduct demographic and frequency analyses for the

demographic variables you feel are relevant.• Generate reliability estimates.

– Should any items be removed to improve reliabilities?• If so, are the items bad or do they need to be reverse scored?

• EFA for conscientiousness– Method: Principle Axis Factoring; Rotation: Promax

• Make adjustments to scale based on the reliability, factor analysis, and definition.

• Compute scales for TI, Conscientiousness, and PW• Run correlational analysis of TI, Conscientiousness, and, PW

Turn in to me

• All syntax (organized and labeled)• Part of section 6 & 2– A list of what cases you removed and why

• Also; explain if you dummy coded any variables and then compared screening vs. no screening and decided not to screen.

– Discuss and summarize the data regarding the free-response occupation variable.

– Output: Demographics (in APA table form),All EFA (highlight pattern matrix with your loadings) and correlation (in APA table form).

Full Week 11 Slides

Final Project Information

• Due Thursday November 21st

– Turn in on Dropbox• Around 8-10 Pages APA style. – Also include your output and syntax

• Worry more about showing competence than providing a one-sided convincing argument

• I will help except for the Monday, Tuesday and Wednesday in which it is due.– Bring specific questions.

Section 1: Content Analysis (10 points)

• Define Conscientiousness and content domain. Do you think there is enough evidence of content validity.

• How congruent is the scale with the content domain? Contamination and deficiencies.

• Would you make any changes to the current scale (i.e., directions, response options, or question content)?

Section 2: Factorial Validity (10 points)

• What factor structure do you expect to see with our scale?

• Calculate reliability. High enough? Should items be removed?

• Summarize how many observed factors from the EFA and name the factors. Is the scale homogeneous? How do you know?

• Is there “enough” evidence of factorial validity? What are the implications for the usefulness of the scale?

Section 3: Construct Validity (10 points)

• Come up with an expected pattern of relationships – Convergent & Discrimination (relative)

• Compare expectations to observed relationships– What findings were unexpected?– How does this influence your interpretation of the

evidence of validity?

Section 4: Criterion-Related Validity (10 points)

• -Summarize the results presented in the two regression tables.– R2 , Beta values?

• -Does conscientiousness explain enough unique variance in outcomes when compared to other assessments. Is there enough evidence to convince you of it’s utility?

• -Is multicollinearity an issue? Elaborate.

Section 5: Recommendation (10 points)

• -Make an official recommendation given the current situation– Talk about the different types of evidence and potential tradeoffs/preferences.– Should we add the scale?

• Discuss the selection and baserates for before and after adding the measure. Is there enough of a change to impress you?

• Does the organization currently have an issue with adverse impact?– Does adding our measure help or harm in any significant way?

• Is adding the scale financially and practically feasible? Explain. Are you concerned with faking? If so, what can we do to address the issue?

Section 6: Raw data Interpretation (10 points)

• Create a correlation matrix for our scale, TI, and PW.– Summarize the results presented in the

correlation matrix. Does it make sense theoretically?

• Summarize the sample by analyzing the demographic variables and reporting the sample size.– Generalizable?Explain.

Preliminary Cleaning

• Deleted those that did not finish.• Removed all identifiable data• Converted old SAT scores– There may still be some cases I missed

• Removed the GRE question

What you need to do!

• Determine what data should be included– Incomplete data?– Response sets?– Working and tenure?– Sample size

• Code the Occupational (free-response) data into meaningful categories.

• Dummy code accordingly.

Statistical Procedures• Screen out bad data• Conduct demographic and frequency analyses for the

demographic variables you feel are relevant.• Generate reliability estimates.

– Should any items be removed to improve reliabilities?• If so, are the items bad or do they need to be reverse scored?

• EFA for conscientiousness– Method: Principle Axis Factoring; Rotation: Promax

• Make adjustments to scale based on the reliability, factor analysis, and definition.

• Compute scales for TI, Conscientiousness, and PW• Run correlational analysis of TI, Conscientiousness, and, PW

Turn in to me

• All syntax (organized and labeled)• Part of section 6 & 2– A list of what cases you removed and why

• Also; explain if you dummy coded any variables and then compared screening vs. no screening and decided not to screen.

– Discuss and summarize the data regarding the free-response occupation variable.

– Output: Demographics (in APA table form),All EFA (highlight pattern matrix with your loadings) and correlation (in APA table form).