eeo & testing quarterly review - biddle › articles › bcgnews_2009-01.pdfpreparing for an...

Quarterly ReviewQuarterly ReviewEEO & Testing Published by Biddle Consulting Group, Inc. • HR News & Information • February 2009

Inside...Preparing for Audits ... 1 | Measuring Proficiency in In-House Systems ... 3 | OFCCP Directive on Facility Exemptions ... 4Criterion Validation ... 6 | Snapshot Date for AAPs ... 7 | EEO Insight ... 9 | BCG Training/Event Calendar ... 11

Copyright © 2009 • Biddle Consulting Group, Inc.

Preparing for an On-Site Auditby Marife Ramos

On September 10, 2008, the OFCCP announced its new compliance initiative where every 50th audit will result

problem areas. You will find yourself better prepared for the on-site if you know ahead of time what the OFCCP will find during their review --- Is it adverse impact in hiring? Is it recordkeeping concerns? Is it compensation issues? It is important to know what the OFCCP may find ahead of time with the data you will provide.

4. Know your auditor. Give him/her a call upon receipt of the letter. Introduce yourself. Assure the Compliance Officer (CO) that you intend to extend full cooperation during the audit. More than you think, the CO can actually help manage the expectations of the audit process. It will be a lot easier to ask the contact how to be of help to ensure a smooth review process if you know the CO involved.

5. Meet with the Executives, HR managers, department managers and other decision-makers before the on-site date.

Don’t keep them in the dark. • Clarify what authority the OFCCP has and what • their expectations are.Explain what to expect during an on-site visit • (e.g., possible interview, tour of the facility, etc.)Let them know why you are getting a visit from • a compliance officer.Review the results of the analyses, especially • the potential problem areas.

This will better prepare the company for an OFCCP visit and in turn, could also assist you in getting incumbents prepared for an interview. Contractors can take this opportunity to solicit ideas on how to improve the work environment. Take caution though: Use discretion on the specifics of any potential problem areas you may want to share. If not sure, get the advice of a corporate counsel.

6. Inspect the facility being visited. Ensure that the facility is ADA compliant. • Does the facility have all the required postings?•

in an automatic on-site review for Federal contractors. For most employers, a desk audit in and of itself is enough to cause a panic attack. Add to that the possibility of an on-site audit, and it might send an HR professional over the edge.

Calm down and fear not. For the most part, contractors may actually be able to take control of the review process and create a positive outcome. Being proactive rather than reactive in the AAP development process, understanding the AAP analyses results, and its implementation can help contractors overcome most of the difficult hurdles in an OFCCP compliance review.

Here are some helpful tips that can put companies in a better situation when the dreaded on-site notice is received. Note that the first four (4) items are associated with preparations to be made prior to the OFCCP coming on-site. However, these four items are critical as they could make or break your chances of having a favorable on-site review.

1. Prepare a professional looking and compliant Affirmative Action Plan. Remember the phrase: “You never get a second chance to make a first impression.” The submitted AAP is your first impression. Don’t give the compliance officer a reason to question your ability to comply with the requirements of the regulations by providing a non-compliant and incomplete AAP. A common element of non-compliance is missing elements such as applicant data.

2. Do not miss your submission deadline. Again, this goes hand-in-hand with managing first impressions. Remember, you have only 30 days to respond to the OFCCP. So get to work as soon as you receive the audit letter and if at all possible, submit the AAP on time. It demonstrates your willingness to cooperate during the audit phase and compliance with the requirements. If an extension was requested and is granted by the OFCCP, do not miss the extension deadline.

3. Review and analyze the data before submittal. Don’t get blind-sided by data problems and potential ...see Audit Prep page 2

Quar

terly

Revie

w • 2

• Fe

brua

ry 20

09


Staff ContributionsManaging Editor ........................................................Shelli JohnsonContributors ................................................... Patrick Nooren, Ph.D..............................................................................Jim Higgins, Ed.D................................................................................... Chris Lindholm...................................................................................Michael Callen....................................................................................Marife Ramos..............................................................................Christine AnthonyProduction .......................................................................C. Lori LeeWebsite ................................................................... www.biddle.com

Biddle Consulting Group, Inc. is located at 193 Blue Ravine Road, Suite 270, Folsom, California 95630. For questions or comments about the EEO & Testing Quarterly Review, please e-mail us at [email protected].

BCG Speaking at SHRM 2009 in New Orleans

Patrick Nooren, Ph.D., Executive Vice President of Biddle Consulting Group, Inc. will be conducting a concurrent session titled, “Adverse Impact and the New OFCCP: Analysis Done Right!” on Tuesday, June 30, 2009, as a part of the Society for Human Resource Management’s annual conference being held in New Orleans, Louisiana in 2009.

To see a complete list of BCG presentations and conferences where staff will be present, see page 11 newsletter for the 2009 Training & Events Calendar.

Do a mock tour. Focus on the common areas.• 7. If possible, prepare a room for the auditors and

your representatives to meet and conduct necessary work. A dedicated room (or workstation) will enable the compliance officers and company representatives privacy to conduct their work and interviews. Ensure that the room (or workstation) is equipped with a computer or computer hookup. This type of preparation will be translated as working cooperatively with the CO.

8. Gather materials and documents that could assist you in effectively responding to possible requests and questions. It is far better to be overly prepared than to have to scramble for any requested materials during an on-site audit. A tabbed binder that contains materials that the company is willing to share with the OFCCP would be ideal as a reference tool for both the CO and the company representative. It is also recommend to prepare a separate tabbed binder that contains the same materials submitted to the OFCCP PLUS additional relevant information (e.g., analyses results, cheat sheets, notes, etc.) for the company’s representative use only.

It will be close to impossible to anticipate every single item that the CO might want to see during the on site. Place yourself in the CO’s shoes. Knowing the probable cause of this on site visit, what would be the materials you think the CO would want to see? How can you help in the smooth facilitation of the audit? Will you be able to easily answer questions with regards to the analyses, the data, the company policy, the selection process, etc.?

Following is a list of items that you might consider placing in your OFCCP shared binder:

Prepare a contact list of relevant company • personnelCopies of I-9s• List of employees that the OFCCP will likely •

want to interview Copy of any validation materials or job analysis. • Focus on the practices, procedures or tests identified as having potential problem areas.Copy of the current Federal Contract. • Copies of Purchase Orders (PO’s). Ensure that • PO reference to the EEO Clause and EO 13201 (if union employees are present) is included.Copies of three years of EEO-1 Reports• Copies of three years of VETS-100 Reports• Copy company EEO Policy Statement• Copies of any selection process policies and • guidelines (for example, Promotions Policies/Guidelines, etc.)Copy of the Desk Audit and On-Site letter• Copy of the Record Retention Policy• Copy of current roster• Copy of roster at the time the potential problem • area occurredOutline of Selection Process for at issue jobs• Outline of Application Process and disposition • codesElectronic copy of data and AAP provided to the • OFCCPExecutive summary/Cheat Sheet that includes • data and results of:

Compensation Analysis1. Utilization Analysis2. Adverse Impact Analysis3.

Remember, contractors can receive a notice for an on-site audit for any reason --- meaning, the compliance officer found a potential problem area that he/she may want to investigate further. The least a contractor can do is to be prepared for such an important review.

Audit Prepcontinued from page 1

Quarterly Review • 3 • February 2009


At a large utility company in the western states, supervisors hover over several clusters of low-walled cubicles assembled into a circular shape, each one seating trainees answering live customer calls. The trainees have spent six weeks working in a simulated environment, and are answering live calls for the first time today. They will spend another six weeks being trained and groomed by their superiors until such time as they will either be deemed ready for permanent assignment or unable to make the grade. But how will that decision be made?

Every year, thousands of medium to large-sized organizations spend millions of dollars to implement or upgrade their enterprise software applications. These “mission critical” software programs are the applications entire departments rely upon to perform their tasks daily. In many cases, the job and the software have become so completely intertwined, that one cannot function without the other.

The fact that positions have become more technical is not a surprise, but it is rather ironic that because the software is so sophisticated and customized to the organization, the means by which a trainee is deemed “proficient” and ready to advance to a permanent status has become more time-consuming and subjective than ever before.

There appears to be an inverse relationship between the sophistication of the technology in use and the sophistication of the means by which aptitude to operate that technology is measured. For example, the only way one can learn if a new employee has proficiency in operating critical software is to have a manager stand behind her and directly observe for hours at a time. This method is neither objective nor practical.

While it is peculiar that this emergence of technology has created such reliance upon subjective measures, it is certainly understandable. While off-the-shelf options to measure basic software proficiency (e.g., Word, Excel, Access, PowerPoint, and Outlook) are well within reach, companies cannot afford to produce self-scoring job simulation software designed to measure proficiency on their own proprietary software systems… that is, until now.

Test2Spec is a new service that brings affordable custom testing solutions to organizations faced with this

challenge. Test2Spec is a dynamic, four-step process that involves BCG personnel travelling to the job site to meet with the employer’s staff for a mini-job analysis workshop to determine the critical tasks to be measured. A programmer will “plug in” to the local network to capture all of the software applications screens, menus, dialog boxes, etc., and then begin creating the custom application. The final steps include delivery of the custom application for local validation (we include an automated

validation wizard in the software) and implementation of the test.

While Test2Spec is not cheap (a typical 20 item test with two parallel forms costs about $40,000), the expense is a far cry from what it would cost to create a custom software application ($150K to $250K and up) from the ground-up. The resulting simulation software is amazing because it looks and responds exactly like the “live” software, without exposing any of the functionality or confidential data contained in the “live” application.

Think about it this way: If you wanted to know if someone understood how to use Outlook, you certainly wouldn’t want them to sit down at your computer and start messing around with your confidential emails. You would want to simulate an Outlook environment so that you could measure a person’s skill sets without exposing your critical data or systems.

For the aforementioned utility company, the stakes are even higher than just maintaining confidentiality and protecting data. “The contact center is the gateway for every white-collar position in the company,” the contact center manager told me recently. “[The contact center] is the proving-ground for the rest of the organization. The quality of the people we hire directly affects every other department because this is the pool from which they all hire.”

If your organization operates a mission-critical application and would like to see how the Test2Spec application works, visit the online demo section of our product website at www.test2spec.com.

If you would like to discuss the product, please call Michael Callen at (800) 999-0438 ext. 121 or e-mail him at [email protected].

Employers Have New Options toMeasure Proficiency in In-House Systems

by Michael Callen

Quar

terly

Revie

w • 4

• Fe

brua

ry 20

09


OFCCP Directive on Facility Exemptionsby Chris Lindholm

There are a great many decisions that Federal contractors have to make when they are required to comply with Executive Order 11246. While most contractors can read the regulations and agree that most of the rules for Affirmative Action Planning are black and white, there are always certain areas of the regulations that are considered vague and therefore subject to interpretation. One common example of regulations that have stumped contractors for years is the criteria for a facility exemption using Federal contractor status.

One of the most commonly asked questions that Biddle Consulting Group (BCG) receives is about facility exemptions. The question being “We create an AAP for Company A, but we also own Company B and they have never had an AAP. Do we need to create an AAP for Company B even though they are a separate business?”

Consider the fact that there is an infinite number of company configurations and industries represented in the business world and it becomes virtually impossible to create one global definition of “Who has to create an AAP and who does not”. Typically, contractors assume that if one of the businesses that they own has a Federal contract then that entity needs an AAP while the other businesses that the contractor owns, but does not have their name on the contract, should not be subjected to the Executive Order. Unfortunately, the Department of Labor takes a more detailed approach, and more often than not, the Office of Federal Contract Compliance Programs (OFCCP) will require other businesses under the corporate umbrella to create Affirmative Action Plans.

While a perfect rule simply does not exist, and never will, there are boundaries provided by the OFCCP that contractors can use in an attempt to decide if Company B falls under the umbrella of the Executive Order. Unfortunately, as most experienced contractors know, the directive does not favor the contractor so much as it does the governing body.

For many years, BCG has seen attorneys and consultants provide guidance in the form of questionnaires and papers in an attempt to assist contractors with the boundaries for determining which businesses owned by the contractor need an AAP and which ones may not. While those documents may be helpful, contractors are always required to work within the guidelines provided by the OFCCP no matter how thin their documentation may be when it comes to the detailed questions contractors have about their specific situation.

There is a specific Directive from the OFCCP dated September 13, 2002 that defines the OFCCP guidelines for obtaining a facility exemption from Executive Order 11246 and VEVRAA. The OFCCP directive issued by Charles James, the acting Deputy Assistant Secretary for Federal Contract Compliance, is titled “ADM Notice/Separate Facility Exemptions/Waivers”. The directive can be found on the OFCCP website at:

http://www.dol.gov/esa/ofccp/regs/compliance/directives/dir260.htm.

The fundamentals of the Directive are simple. The OFCCP is basically mandating that if a Federal contractor believes that any part of their organization should be exempt from developing an AAP then they need to submit a request and the authority rests with the OFCCP to make the decision. The directive specifically states, “Historically, OFCCP has narrowly interpreted the exemption and waiver provisions and has granted requests for exemptions only in rare and compelling circumstances. The OFCCP intends to continue this practice when considering requests for separate facility exemptions/waivers under EO 11246 and the affirmative action provisions of VEVRAA.” BCG’s experience has been that the OFCCP has followed this quote for many years and that getting a facility exempted from the AAP requirement is a challenge.

In the past, much of the discussion of facility exemption has been centered around the link to fulfilling the Federal contract. However, there has also been discussion linked to the relationship of the corporate offices to the various businesses that it owns. It has been widely perceived that if the same executive team oversees the business that owns the Federal contract in addition to the business that do not have a Federal contract, then the entire corporation and all of its locations would fall under Executive Order 11246 and VEVRAA. Other common links have been associated to tax ID’s, shared staffing, legal representation, use of company name in branding, ownership of HR processes, and more. While all of these factors may affect the decision that the OFCCP will make, the requirements as they are defined in the OFCCP Directive are listed below:

POLICY: The Deputy Assistant Secretary may grant a separate facility exemption/waiver from the requirements of EO 11246 or the affirmative action provisions of VEVRAA, based on a finding that, 1) the facility is in all respects separate and distinct from activities of the contractor related to the performance of its Government contract; and 2) such an exemption/waiver will not interfere with or impede the effectuation of the Order or the statute.

The Deputy Assistant Secretary’s finding as to whether a. the facility for which an exemption/waiver is sought is separate and distinct from the performance of the contractor’s Government contracts shall be based upon a consideration of the following factors:

Whether any work at the facility directly i. or indirectly supports or contributes to the satisfaction of the work performed on a

...see Facility Exemptions page 5



Government contract; The extent to which the contractor derives ii. benefits from a Government contract, directly or indirectly, at the facility to be exempted; Whether any costs associated with operating the iii. facility are charged to a Government contract; Whether working at the facility for which an iv. exemption/waiver is sought is a prerequisite for advancement in job responsibility or pay at facilities connected to a Government contract; and whether working at facilities connected to a Government contract is a prerequisite for advancement in job responsibility or pay at the facility for which an exemption/waiver is sought; Whether employees who normally work at the v. facility are required to perform work related a Government contract at another facility; Whether the facility regularly or substantially vi. transfers employees to or from facilities at which a Government contract is performed; Such other factors that the Deputy Assistant vii. Secretary deems are necessary or appropriate for considering whether the facility is in all respects separate and distinct from the activities of the contractor related to the performance of a contract. Other factors could include the number of facilities connected to the contractor’s Government contracts and the nature of the contractor’s contractual relationship with the Government. Example: A Government agency seeks to enter into a contract with a company that has several hundred retail stores nationwide. Employees from the company’s corporate office would perform the work on the contract and the total costs of the services would not exceed $65,000. The proposed contract would be in effect for a 6-month period. Under the regulations implementing EO 11246, the company would be required to develop and maintain written affirmative action programs for all of its establishments, including its retail stores. The company has requested an exemption from the requirement to develop and maintain written AAPs for its retail stores. In deciding whether the retail stores are separate and distinct from activities related to the proposed Government contract, the Deputy Assistant Secretary considers the total number of establishments connected with the proposed Government contract and nature of the contractual relationship.

The Deputy Assistant Secretary’s finding as to b. whether granting the request for the separate facility

exemption/waiver will interfere with or impede the effectuation of EO 11246 or the affirmative action provisions of VEVRAA shall be based upon a consideration of the following factors:

Whether the waiver will be used as a subterfuge i. to circumvent the contractor’s obligations under federal, state, or local equal employment opportunity laws; The contractor’s record of compliance with ii. Federal, State or local equal employment opportunity laws; and Such other factors that the Deputy Assistant iii. Secretary deems are necessary or appropriate for; considering whether the granting of the exemption/waiver would interfere with or impede the effectuation of either the Executive Order or the affirmative action provisions of VEVRAA.

Three critical items extracted from the directive, OFCCP has not provided a deadline for a response 1. but they do state “the agency will normally issue a decision within ninety (90) days of the contractor’s submission of the necessary information.”Contractors are expected to certify every three 2. years that the exempted facility has not changed its status. If a facility covered by the exemption gets a Government contract, the exemption for the facility automatically terminates.Contractor will notify OFCCP when a facility covered 3. by an exemption gets a Government contract. The contractor will then have 120 days to develop an affirmative action program.

A few tips when deciding whether to generate AAPs for facilities or businesses that do not appear to be linked to a Federal contract:

Conduct research into the contract status of all 1. businesses owned by the company. It is common for contracts to get misplaced or even forgotten, especially during changes in management. BCG often receive calls from contractors that have received a 30-day audit letter when they didn’t even know they were required to develop an AAP, and it is usually the HR/Personnel function that has to deal with the OFCCP when they come calling.Never assume that since a Federal contractor owns 2. businesses that do not appear to be linked to a Federal contract that they do not have to create an AAP. The OFCCP will not grant lengthy extensions or show a significant amount of sympathy to a facility that is developing a new AAP because the contractor didn’t realize they needed one. In fact, the lack of an existing

Facility Exemptionscontinued from page 4

...see Facility Exemptions page 9

Quar

terly

Revie

w • 6

• Fe

brua

ry 20

09


At its heart, the process of validating pre-employment tests is all about amassing evidence demonstrating that a test score is related to a target job in some meaningful fashion that permits distinguishing between those who are likely to perform well from those who are not. As a result, in the truest sense of the word it is not a particular test that can be said to be “valid” or “not valid” so much as the conclusions draws from the individual test scores. Validity, therefore, refers to the determination—based on the available evidence—as to whether the test scores serve as an effective a proxy that is indicative of an examinee’s likely success on the job.

In other words, determining a test’s validity is a lot like presenting a legal case to a jury. The question you should keep in mind is, “Does the evidence suggest, beyond a reasonable doubt, that a person’s test score is an acceptable estimate of his or her likely future job performance?” Continuing the analogy, it is obvious that some kinds of evidence carry more weight than others. DNA or eye witness evidence, for example, is usually considered strong. Circumstantial evidence is generally considered weak—unless there is so much of it that it tips the balances toward being highly convincing.

There are a number of strategies that have been proposed for demonstrating a test’s validity (many of them newer and more reliant on some of the more advanced statistical or theoretical approaches). To learn about these strategies, the reader is encouraged to review the Society for Industrial and Organizational Psychology’s Principles for the Validation and Use of Personnel Selection Procedures (available from www.siop.org) or the American Psychological Association’s Standards for Educational and Psychological Testing (available from www.apa.org).

Perhaps the most widely recognized guide for the validation and use of pre-employment tests is the Federal Uniform Guidelines on Employee Selection Procedures (1978). While allowing for the development of other approaches to establishing test validity as the state-of-the-art develops, specifically endorses three general test validation strategies. These three approaches are a) content validation, b) criterion-related validation, and c) criterion validation. Because the courts tend to defer to the Guidelines as the standard of practice, most employers utilize one or more of these strategies to help reduce their exposure to legal liability resulting from charges of employment discrimination.

Criterion Validation: An OverviewThis brief article discusses criterion validation; an

approach used to validate approximately 15% of US pre-employment tests (Biddle, 2007). Criterion validation is a strategy that uses statistical techniques to demonstrate that an examinee’s performance on a test (his/her test score) is predictive of their actual job performance as measured by

some criterion (e.g., job performance, number of products assembled, number of calls handled, etc.)

It should be obvious that this is a very different approach from the most common validation strategy—content validation—which relies on the assertion that the pre-employment test is measuring the actual knowledge, skills and abilities required on the job as identified through a job analysis.

Since criterion validation focuses on the relationship between test score and job performance, the key to validation using this approach is the use of a statistical test known as the “correlation coefficient” where each examinee’s test score is correlated with his or her job performance. If the correlation coefficient equals or exceeds r=0.20, it means the test is sufficiently related to job performance to make judgments about a candidates likelihood of job success based on his or her test score (Biddle, 2008). In other words, you can use test score to determine who is appropriate to hire.

Note that a correlation of r=0.20 is the minimum that should be considered acceptable. Ordinarily, you would like to see a correlation that is larger—which indicates the relationship between test score and job performance is stronger. The larger the correlation, the better! To illustrate the importance of the size of the correlation between test score and job performance consider the following example.

A correlation coefficient can be converted to a statistic called the coefficient of determination by squaring it (i.e., multiplying it by itself). The coefficient of determination (also called r-squared) tells you the percent of variation in one score that is related to the variation in the other score. In other words, it can tell you the percent of variation in job performance that tends to be related to—or explained by—the variation in his or her test score.

If there is a correlation of 0.20 between a group of test scores and job performance, it means that about 4% of job performance is related to test performance (0.20 multiplied by 0.20). Stated differently, about 96% of the variation in job performance seems to be related to factors other than the test score! While it could be argued that the test, while helping a little to identify effective employees, it is not measuring a very large amount of what makes a person a good employee. In other words, it is a weak predictor of performance and many might question its use for determining who is hired and who is not.

On the other hand, if the correlation between a pre-employment test and job performance is 0.60, the coefficient of determination becomes 36% (0.60 multiplied by 0.60). In this case the test is explaining 36% of the variation in employee performance. Thus, making hiring decisions based

Criterion Validationby Jim Higgins, Ed.D.

...see Validation page 8



An employee snapshot date represents the annual date identified to represent the contractor’s current workforce to be analyzed as part of their Affirmative Action Plan (AAP). Selecting a snapshot date is a task that all federal contractors must decide on as it defines all of the dates that impact the AAP development process. For existing contractors, the decision may be an easy one; simply use the snapshot that has been established for previous AAPs. However, new contractors should review several points before committing to a particular date. This article will introduce just a few of the key points that should be considered when selecting a snapshot date.

New contractors have 120 days after signing a Federal contract to have an AAP in place. If a contract is signed on January 1, 2009 then the AAP needs to be in place by April 30, 2009. The contractor should pick a snapshot date that is between January 1 and April 30 of that year. Keep in mind, however, that the plan needs to be complete and ready to be implemented by the end of the 120-day period. All contractors should give themselves at least 30 days to complete an AAP and new contractors might require more time if they are creating an AAP for the first time.

Contractors will also want to choose a snapshot date that will be easy to work with from year-to-year. While an AAP that wraps around the calendar year may be the most comfortable timeframe to visualize, December and January are often busy times for organizations that can cause a delay in completing the AAP. While convenience is good if it can be managed, it is important to remember that the regulations do not force any particular date on the contractor. However, the OFCCP has little tolerance for expired AAPs so do not allow for gaps in the dates in which the plan is active. If a contractor does not have the time to devote to creating and implementing an AAP in December or January, then a calendar year plan might not be a fit. Especially when the Office of Federal Contract Compliance Programs (OFCCP) audits are on the rise and companies often have to develop an AAP in a panic because the contractor’s staff could not complete the AAP in a timely manner. Even if contractors outsource their plan, they need to be available to work on the data itself and to work with their consultant on various AAP related decisions. As a general rule, contractors should select a date that will allow for an appropriate amount of time to devote to the project on an annual basis.

Many contractors choose to pull their AAP data at the same time that they pull their EEO-1 Report data in July, August or September. A summer AAP cycle can eliminate the need for two data pulls and can help the organization save time

and money. Also, the OFCCP requests three years of EEO-1 Reports during compliance evaluations. If the same data is used for each reporting requirement, the OFCCP will favor the fact that the two report counts will match. If an AAP is created from different data than the EEO-1 Report, the OFCCP may ask questions if they see significant differences in the number of employees identified in each report.

Another thought to consider when choosing a snapshot date is the desired implementation period that the organization would like to enforce. The OFCCP prefers to see implementation periods that begin on the snapshot date or the day immediately following the snapshot date (December 31 snapshot followed by a January 1 active date for example). The literal translation of this statement is that contractors only have one day to collect data, analyze the data, and start implementing action programs to meet their goals. A 24-hour turnaround on the development of an Affirmative Action Plan, let alone several AAPs, would be a monumental achievement. However, the general expectation of contractors is that they will begin working immediately on their plan as their snapshot date passes. If a contractor receives a scheduling letter from the OFCCP, they would have 30 days to complete the plan but trying to complete a plan in this time period drastically reduces the contractor’s ability to clearly identify problem areas and potentially reduce or eliminate them through legitimate data refinement techniques. Contractors that wish to implement their plans on a calendar year basis would want to have a December 31 snapshot date and a January 1 plan date.

In many instances, contractors choose to give themselves a month or more to properly collect and analyze their data. In these cases, the snapshot date would be one to two months prior to the implementation period. For example, if the desired implementation period is from January 1, 2009 through December 31, 2009 the snapshot date would be November 1, 2008. This type of date structure may come under question during a compliance evaluation. However, arguments can be made that the organization would like a full 12 months to implement their plan. If the snapshot date is the day prior to the implementation start date, then the time used to complete the plan overlaps with the time the contractor has to implement action programs.

The selection of an employee snapshot date can be more complex than it may seem. Snapshot dates need to stay consistent from year to year. When selecting a date, contractors should review the pros and cons of all possible dates before making a final decision.

Tips for Selecting an Appropriate Snapshot Date for AAPsby Christi Anthony

Biddle Consulting Group, Inc.Online: www.biddle.com | Toll-Free: (800) 999-0438

Quar

terly

Revie

w • 8

• Fe

brua

ry 20

09


on scores from the second test will be much more defensible because it is demonstrably more related to—and therefore predictive of—job performance.

How To Conduct A Criterion Validation StudyConducting a criterion validation study appears to be

quite simple. All that is needed is a pre-employment test and a measure of job performance. Step one is to administer the test to a large sample of current or potential employees. Step 2 is to collect performance ratings on the criteria of interest (e.g., job performance, customer service ratings, number of phone calls fielded in a given time). Finally, step 3 requires that the correlation between test score and job performance rating be computed. If the correlation is .20 or greater, the test can be said to be at least marginally valid.

As simple as this sounds, one might ask, “Why doesn’t everyone use this validation approach?” There are several very good reasons. These are briefly presented below:

1. Knowledge of statistics: Because criterion validation is a purely statistical process, it is important that anyone who plans to use this approach become well informed with respect to the issues, problems, requirements and practices of establishing relationships between variables using a correlation coefficient.

2. Sample size requirements. Statistics such as correlation coefficients require minimum sample sizes to be considered both valid and stable (i.e., repeatable). Considering the need to demonstrate that a test is valid across shifts, geographic work locations, gender and ethnic groups, etc., BCG typically recommends a sample size of at least 300.

3. Requirement that scores from both the test and criterion be available. In order to calculate correlation coefficients, two values are required. In the case of criterion validation, each test score must also have an associated job performance/criterion measure. This means that the sample size requirement of 300 listed above requires 300 hires—not just 300 test takers. Therefore, unless an organization is large, the sample size requirement alone tends to result in criterion validation not being an acceptable option for validating pre-employment tests.

4. Criterion measures are frequently of low quality. Criterion validation requires a measure of performance that is to be correlated with test scores. For many jobs, performance ratings are of questionable validity. For example, supervisor ratings tend to be influenced by factors other than job performance. These influences can range from a tendency of some supervisors to rate employees more harshly while others tend to rate them more highly. Occasionally, supervisors unconsciously rate employees who are similar to themselves highly while rating others lower. The best type of performance measure to use is one that is highly objective such as number

of products manufactured, number of customer service calls handled, etc.

5. Restriction of range. It is not uncommon for supervisors to rate the majority of their staff as simply “average” or “meets expectations” with the result being little variability in ratings. This tendency may even be built into the performance appraisal system in that any deviation from “meets expectations” requires documentation which tends to discourage using the entire range of performance scores. In this case, a test which typically has significant amounts of variation (people score from very low to very high) will appear to have little or no validity in terms of predicting performance not because it is invalid but because there is a problem with the way job performance was measured.

6. From a theoretical perspective it is difficult to develop tests. This pitfall of criterion validation warrants an article all on its own. In its simplest form, the issue is as follows. A valid pre-employment test must be reliable, meaning that it provides consistent results. A reliable test tends to be one that is “homogeneous” meaning that it consists of items that tend to be related to each other and therefore measure the same general knowledge domain. However, a test constructed of homogeneous items tends to have a low correlation between its total test score and a job-related criterion because all of the items are measuring a very similar construct. Therefore, a test that has maximal predictive validity must consist of items with low correlations among each other (resulting in a test that is not homogeneous) and yet each item has a high correlation with the criterion. As a result, a very powerful predictive test will look strange because it will consist of items that have little or nothing to do with each other. This will make the test less acceptable to test-takers. Furthermore, there is no theory to drive, in a consistent manner, an employment test containing such items.

When completed correctly, criterion validation can be an effective approach to establishing the validity of attest. It is easy for people (i.e., members of the jury) to understand that the relationship between test score and job performance is such that as test score increases job performance also tends to increase. As a result, hiring those with the highest test score is likely to result in hiring those who will perform the best.

It should also be clear, however, that there are a number of methodological and practical issues that make criterion validation less than ideal for the vast majority of employers. These include the requirements for large sample sizes, a lack of high-quality performance measures, the need to have a strong background in the application of statistics to ensure that all relevant statistical issues are identified and addressed, and the theoretical difficulties in constructing heterogeneous tests that are highly predictive while still appearing to be job related.

Validationcontinued from page 6



The inaugural issue of EEO Insight was released electronically in December 2008. The journal, published by Biddle Consulting Group, Inc., includes articles written by experts throughout the industry to provide a comprehensive view of the industry -- past, present, and future.

The EEOC, OFCCP, and “Systemic Discrimintaion”: •The Rules Have Changed by Mickey Silberman, Esq. and Stephanie Lewis, Esq., Jackson Lewis LLPWhere are the Courts Today? •Proving and Defending A g a i n s t a n “A d v e r s e Impact ” Claim: OFCCP ’s New Approach to Employer Selection Systems by John C. Fox, Esq. and Alexa L. Morgan, Esq., Manatt, Phelps & Phillips, LLPFive Steps to Successful AAP Goal Development by •William H. Truesdell, The Management Advantage, Inc.Diversifying Your Organization: How to Actually •Make it Happen (Part I of II) by Patrick M. Nooren, Ph.D. and Dan A. Biddle, Ph.D., Biddle Consulting Group, Inc.Claims of Employment Test Validity: Who Can You •Trust? by Lisa Harpe, Ph.D., Peopleclick Research Institute (PRI)

EEO Insight is a cutting-edge journal unique to the Equal Employment Opportunity (EEO) and Affirmative Action (AA) fields. The articles you will find throughout EEO Insight will

New EEO Industry Journal

AAP will likely increase suspicion that the contractor has something to hide. As part of the EEO-1 filings, there is a question that 3. specifically asks whether or not the company is a Federal contractor or subcontractor. Be aware that answering this question in the affirmative places the company into the Federal Contractor Selection System (FCSS) to potentially be selected for an audit. Once selected for an audit, OFCCP will require thorough documentation to receive an exemption. Contractors should take care to make sure that this box is correctly checked or unchecked on your EEO-1 filings. This is not to say that the FCSS is the only method that OFCCP uses to determine who is as contractor, it is known that the majority of audits are driven by the use of this system. It is not recommended that contractors leave the checkbox blank in order to avoid being identified as a Federal contractor.

Facility Exemptionscontinued from page 5

provide clear strategies, fresh insights, and the technical information necessary to recruit, hire, select, and retain the most effective workforce while complying with both the letter and spirit of EEO laws and regulations. The content of EEO Insight is narrowly-tailored to matters covered by Title VII of the 1964/1991 Civil Rights Act, EO 11246, and the Code of Federal Regulations[1].

Each quarterly issue contains noteworthy articles written by practicing professionals on current events and trends in EEO, AA, and Testing. EEO Insight is intended for professionals of all levels and experience.

Free electronic (PDF) subscriptions are available by signing up at: www.EEOInsight.com/subscribe.html.

For more information on submitting an article, e-mail the editor at [email protected].

[1] Federal Code of Federal Regulations Pertaining to ESA, Title 41, Chapter 60: 1-4, 20, 30, 40, 50, 250, and 741 (sections pertaining to Affirmative Action regulations, the Uniform Guidelines, Sex Discrimination, Executive Order 11246, OFCCP Recordkeeping, Religion or National Origin, Disabled Veterans and Veterans of the Vietnam Era, and Individuals with Disabilities.

If a contractor is submitting a request for a facility 4. to be exempted, then the contractor should provide all reasonable arguments for the request. While the OFCCP’s list may identify the primary items they will consider for an exemption, it is possible that there may be other legitimate variables that OFCCP would take under advisement when considering an exemption request.

BCG has encountered many different scenarios when it comes to facility exemptions. While it may seem impossible to identify every variable that could be used to determine if a facility is related to a Federal contract, it is possible. As it is with any research project, if the evidence is provided then the expected answer will likely be the result. It may be difficult to convince the OFCCP that a site is not subject to the regulations if they have that address on a list. However, the OFCCP is not unreasonable and if contractors make a legitimate case for a facility exemption then they should get it.

Quar

terly

Revie

w • 1

0 • F

ebru

ary 2

009


In October 2008 Biddle Consulting Group (BCG) attended the National Employment Law Institute’s (NELI) Affirmative Action Briefing conference in San Francisco, CA. These conferences are designed to cover a wide-variety of current EEO/AA topics. Below is a summary of the different topics covered throughout the two-day conference and a brief overview of a few select topics.

Recent Major Developments At OFCCP• Recent ly Publ i shed Federa l Government • DocumentsPrimer On Legal Challenges To Employer Tests• Where Are The Courts Today? Proving And Defending • Against An “Adverse Impact” Claim: OFCCP’s NEW Attack On Employer Selection SystemsProtected Veterans’ Reporting Explained And • UntangledHigh-Level Questions & Answers Re: Audits• Compensation Data Requests•

With a new administration comes a new Assistant Secretary of Labor. This person will be responsible for appointing the new head of the Office of Federal Contracts and Compliance (OFCCP). The OFCCP is the governing agency who enforces Executive Order 11246. Although this change will typically not occur until Spring of 2009, contractors should be prepared for changes that a new administration will bring.

There have been a lot of OFCCP developments within the past few years. A more recent development is the modification of the OFCCP’s compensation trigger analysis. Most contractors are familiar with the trigger analysis known as the “2/30/3.” The OFCCP has recently begun using a modified version; known as the “5/10/3.” This analysis has the same concept as the “2/30/3” but changes the thresholds. For more information please visit our blog spot in February at (http://affirmativeactionnews.blogspot.com/) . BCG recommends conducting this type of analyses on compensation data before submittal to the OFCCP. With the introduction of the new trigger test the industry has seen an increase in the OFCCP’s requests for 12-factor data to analyze

compensation. This proves the OFCCP is trying to focus their efforts on finding systemic disparities within compensation. That is not to say this is their only focus, because they are still focusing on finding selection rate disparities within hires, with a specific focus on low-lying fruit (a.k.a. high volume positions).

There has also been some buzz about a system that will allow the OFCCP to keep an up-to-date account of audits with the capability of connecting the dots amongst regions. For example, this may allow an auditor in California to be aware of another facility’s audit in the Mid-West within the same company. This is something the OFCCP is actively pursuing and has requested the budget to obtain a computer program that will help this accomplish this. The program would help them identify trends amongst the regions for national contractors. One trend the OFCCP will be focusing on is finding a pattern of systemic discrimination within a company across regions.

Another recent OFCCP directive is to ensure contractors are in compliance with their veteran requirements. They recently introduced a new G-5 initiative. While this is a

great initiative, contractors should be aware, the award process is essentially an audit. However, the recipients of the G-5 recognition are exempt for three years from audits for the individual facility or establishment.

The introduction of the Vets-100A report (reporting beginning September 2009) has caused frenzy within the contracting community. The reason it’s so important is because contractors may be required to submit both the Vets-100 and Vets-100A in 2009. See Figure 1 for the important breakdown.

Please note that a renewal or modification to a contract after December 1, 2003 will fall under 60-300 (Vets-100A) reporting requirements.

Check out the NELI website at http://www.neli.org/ for more

information. These conferences are conducted on an annual basis throughout the U.S. and are an excellent resource for anyone involved in Affirmative Action planning.

NELI’s Affirmative Action Briefing Brings New Lightby Danielle Yokoi

FIGURE 1: Vets-100 and Vets-100A

For contracts prior to December 1, 2003

Regulations: 41 CFR § 60-250 »Vietnam Era Veterans Readjustment Assistance Act (VEVRAA)Requires: Vets-100 Report »Due: September 30, 2009 »Contract threshold: $25,000 »Veteran categories: »

Special disabled veteran,•Veteran of the Vietnam era,•Other protected veteran, and•Recently separated veteran•

For contracts on/after December 1, 2003

Regulations: 41 CFR § 60-300 »Jobs for Veterans Act (JVA)Requires: Vets-100A »Due: September 30, 2009 »Contract threshold: $100,000 »Veteran categories: »

Disabled Veteran,•Recently separate veteran,•Armed Forces service medal veteran, and•Other protected veteran•



For more information on any of the events listed, please callBiddle Consulting Group, Inc. toll-free at (800) 999-0438 or e-mail [email protected]

2009 BCG Training & Events CalendarWebinars [Register at www.biddle.com/eeowebinars.stm; cost of attending is free, unless otherwise noted]

: February 11 : Updated Best Practices for OFCCP Audits: June 10 : Understanding Statistics used in EEO: July 13 : Update on Recordkeeping and Definition of Applicant: July 23 : Keeping Up With the Changes in Veterans and Disability Requirements

Seminars [To register: call 800.999.0438 ext. 172 or e-mail [email protected]]

: April 21-22 : Folsom, CATwo (2) Day Seminars :: AAP Methodology & AutoAAP Software TrainingCost: $990

Presentations [BCG will be speaking at the following meetings and conferences]

: February 18 : Sacramento, CA California Employment Development DepartmentSpeaker: Jim Higgins, Ed.D. | Topic: Workforce Analysis Using Excel

: March 12-13 : Sacramento, CA PTC/NC Annual ConferenceSpeaker: Jim Higgins, Ed.D. | Topics: Part 1 - Basic Statistics; Part 2 - Statistics for HOUR Professionals

: April 2 : Lincolnshire, IL American Association for Affirmative Action (AAAA) Annual ConferenceSpeaker: Jim Kuthy, Ph.D. | Topic: Title VII of the Civil Rights Act

: June 30 : New Orleans, LA SHRM Annual ConferenceSpeaker: Patrick Nooren, Ph.D. | Topic: Adverse Impact and the New OFCCP: Analysis Done Right!

Conferences [BCG will have a booth and staff available at the following conferences]

: May 18-20 : New Orleans, LA Association of Legal Administrators (ALA) Annual Conference

: June 8-9 : Fort Worth, TXNational Emergency Numbers Assoc. (NENA) Annual Conference

: June 28-July 1 : New Orleans, LASociety for Human Resource Management (SHRM) Annual Conference

: July 28-31 : Atlanta, GANational Industry Liaison Group (NILG) Annual Conference

: August 16-19 : Las Vegas, NV Association of Public-Safety Communications Officials (APCO) International Conference