east meets west in assessment development: western technical assistance to eastern needs. part ii:...

Millman, J., & Greene, J. (1989). The specification and development of tests of achievement and ability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 335-366). New York: American Council of Education, MacMillan.

Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical ap- proaches for assessing measurement bias. Applied Psychological Measure- ment, l< 297-334.

Mislevy, R. J. (1996). Test theory recon- ceived. Journal of Educational Mea- surement, 33, 379-416.

Nandakumar, R. (1993). Simultaneous DIF amplification and cancellation: Shealy- Stout’s test for DIF. Journal of Educa- tional Measurement, 16, 159-176.

Nichols, P. (1994). A framework of devel- oping cognitively diagnostic assessments. Review of Educational Research, 64,

Nichols, P., & Sugrue, B. (1999). The lack of fidelity between cognitively complex constructs and conventional test development practice. Educational Measure- ment: Issues and Practice, 18, 18-29.

O’Neill, K. A,, & McPeek, W. M. (1993). item and test characteristics that are associated with differential item functioning. In P. W. Holland & H. Wainer (Eds.), Duerential item functioning (pp. 255-276). Hillsdale, NJ: Erlbaum.

Oshima, T. C., Raju, N. S., Flowers, C. P., & Slinde, J. A. (1998). Differential bundle functioning using the DFIT framework:

575-603.

Procedures for identifying possible sources of differential functioning. Ap- plied Measurement in Education, 11,

ROUSSOS, L., & Stout, W. (1996a). A multidimensionality-based DIF analysis paradigm. Applied Psychological Mea- surement, 20, 355-371.

ROUSSOS, L. A,, & Stout, W. F. (1996b). Simulation studies of the effects of small sample size and studied item pa- rameters on SIBTEST and Mantel- Haenszel type I error performance. Journal of Educational Measurement,

ROUSSOS, L. A,, Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with hierarchical cluster analysis to de- tect multidimensionality. Journal of Educational Measurement, 35, 1-30.

Shealy, R., & Stout, W. F. (1993a). A model-based standardization approach that separates true bias/DIF from group differences and detects test bias/DIF as well as item bias/DIF. Psychometrika,

Shealy, R., & Stout, W. F. (1993b). An item response theory model for test bias and differential test functioning. In P. W. Holland & H. Wainer (Eds.), Dzferen- tial item functioning (pp. 197-239). Hillsdale, NJ: Erlbaum.

Silver, E.A., & Kenney, P.A. (1993). An examination of relationships between the 1990 NAEP mathematics items for grade 8 and selected themes from the NCTM standards. Journal .for Research

353-369.

33, 215-230.

58, 159-194.

in Mathematics Education, 24, 159- 166.

Snow, R. E., & Lohman, D. F. (1989). Im- plications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 263-331). New York American Council on Education, Macmillian.

Snow, R. E., & Peterson, P. L. (1985). Cog- nitive analyses of tests: Implications for redesign, In S. E. Embretson (Ed.). Test design: Developments in psychology and psychometrics (pp. 149-166). New York: Academic Press.

Standards for Educational and Psycho- logical Testing. (1999). Washington, DC: American Educational Research Association, American Psychological As- sociation, National Council on Measure- ment in Education.

Sternberg, R. J. (Ed.). (1994). Encyclope- dia of human intelligence. New York: Macmillian.

Sudweeks, R. R., & Tolman, R. R. (1993). Empirical versus subjective procedures for identifying gender differences in science test items. Journal of Research in Science Teaching, 30, 3-19.

Walter, C., 81, Young, B. (1997). Gender bias in Alberta Social Sciences 30 examinations. Canadian Social Studies,

Zieky, M, (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland & H. Wainer (Eds.) Differential item functioning (pp. 337-347). Hillsdale, NJ: Erlbaum.

31, 83-86, 89.

NCME ZntemationaZ News

East Meets West in Assessment Development: Western Technical Assistance to Eastern Needs. Part 11: Project and Team Management in the AERo Project in Romania Steven Bakker, Gerben van Lent, and Annemarie de Knecht-van Eekelen CITO, National Institute for Educational Measurement, Arnhem, The Netherlands

This is Part I1 of a three-part series describing the consultancy and training demands that are encountered in

technical assistance projects in Cen- tral and Eastern Europe. This part highlights project management and cooperation aspects in delivering support to the Romanian Assessment and Examination Service. Part I dealt with strategic issues for a successful deliv- ery of “Western Technical Assistance to Eastern Needs.” Part I11 will address training of Polish subject experts, preparing to take on responsibility for a decentralized and drastically changed final exam, the Matura.

Introduction Assessment and Examinations in Ro- mania (AERo) is one of the largest projects of the Dutch Institute for Ed- ucational Measurement (CITO) in the

Steven A. Bakker is Director Interna- tional Projects, CITO, National Institute for Educational Measurement, P. 0. Box 1034, 6801 MG Arnhem, The Netherlands. His specialization is consultancy, training, and management of large-scale assessment projects.

Gerben van Lent is Senior Consultant, CITO, National Institute for Educational Measurement P.O. Box 1034, 6801 MG Arnhem, The Netherlands. His specialization i s consultancy, training, and management of large-scale assessment projects.

Annemarie de Knecht-van Eekelen is Senior Consultant, CITO, National Insti- tute for Educational Measurement, P.O. Box 1034, 6801 MG Arnhem, The Nether- lands. Her specialkation is consultancy, training, and management of large-scale assessment projects.

36 Educational Measurement: Issues and Practice

Central and Eastern Europe (CEE) region. It offers the opportunity for a more in-depth discussion of the concepts introduced in Part I of this series. Part I focused on key issues in assessment development. It addressed at a strategic level the issue of technical assistance and forms of cooperation. It also described the mutual benefits for the parties involved and the necessary skills and expertise for consultants and trainers. That descrip- tion did not provide sufficient examples of “how it actually works.” In this part, we give a more detailed de- scription of how some of the key issues of policy development, institutional organization, and capacity building are addressed in the course of giving technical assistance to the Ministry of Edu- cation in Romania. The article addresses the implementation of technical assistance and cooperation at an organizational level.

Outline of the Romania Project The AERo project is the result of a contract between the Romanian Min- istry of Education and CITO to provide technical assistance to the newly established National Examinations and Assessment Centre (Serviciul National de Evaluare 5i Examinare [SNEE]). The project comprises training and consultancy to set up the institutional and logistical infrastructure for national assessment and examinations that meet the purposes of certification, selection, and public information on the quality of education.

A main focus of the project is to assist SNEE in introducing a new examination, the Capacitate, in mathematics, mother tongue, and history/ geography at the 14+-year age level. The technical assistance of CITO cow ers all of the important aspects needed for the establishment of a modern sustainable examination system: test construction, examination production, administration and logistics of exams, organization of pretests, and the analysis and reporting of results of the pretest and of the actual exam. The assistance also addresses the modernization of national assessment in Romania and assistance in the field of public relations. The new SNEE had to develop a corporate im- age and professional public relations (PR) in order to satisfy the stakehold- ers-students, parents, teachers, and the Ministry of Education. The techni-

cal assistance team consists of key consultants in the fields of training, administration and logistics, psychometrics, national assessment, PR, and subject consultants.

Management of the Project The project leadership had to deal pro- fessionally with the three key issues in managemenk

0 Strategic level and external fit: Does the project fulfill the expectations of the outside world? These expectations not only concern the project objectives as described in the terms of reference (TOR) of the technical assistance in the contract, but also concern new demands and requests from the Ministry of Education, chang- ing circumstances, sustainability, etc. Organizational level and internal fit: How should a project framework be developed to ensure the desired outcomes? How should project activities be geared to one another? What are shared responsibilities and what are separate responsibilities of CITO and SNEE on an organizational level and on the individual level. Operational level: How to realize the implementation of the project-the distribution of tasks, co- ordination monitoring, internal communication, etc.

Counterpart Approach In this project, CITO and SNEE decided to work as countevarts. For official project documents such as the TOR, the contract specifies obligations and expected outcomes in terms of a client-consultant relationship, and in all official accounting procedures. However, the actual realization of project goals and objectives required, as much as possible, SNEE and CITO forming one team working together ef- fectively and efficiently to:

support and “drive” planned changes in education in Roma- nia; realize the introduction of the new Capacitate exam and the modernization of the Baccalaure- ate exam;

0 ensure the improvement of the technical quality of assessment and examination; facilitate the establishment of the SNEE.

In practice, this means that the relationship between SNEE and CITO is part of the internal fit of the project, instead of CITO and SNEE each per- ceiving the other as part of the project’s external environment. This made it possible to create an environment within the project where “East meets West” on equal footing, each “side” bringing with them their capa- bilities, needs, constraints, attitudes, and expectations. These situational factors can be categorized in two ways: First, as to whether they are intrinsic or extrinsic to the project partners; second, as to whether they have an “Eastern” or “Western” background. In Table I, some of these are listed as examples.

Organization In practice, the team directors, while carrying the overall responsibility, del- egated most of the actual day-to-day running of the project to the team managers. The latter are responsible for planning, communicating between the teams, briefing the directors, reporting, etc. The key consultant for training supervises the subject consultants. The subject consultants form a flexible group, with the SNEE members indicating which subject consultants they would like to assist them during a specific training session. Be- cause until recently the SNEE had only a limited number of staff members, the key consultants often work together in pairs or in groups of four, especially for the more technical aspects of the assistance. On the other hand, seminars combined with work- shops were organized at regular inter- vals in which almost all members of the two teams were involved in training and production activities.

Communication Communication is a key issue. A communication scheme has been developed where the team manager of the SNEE and the team manager of the technical assistance (TA) team are the key persons. The TA manager pre- pared a weekly bulletin for the SNEE manager and this bulletin, together with the response of the SNEE manager, was sent to all the consultants. The two managers also received copies of all direct communications between consultants and their counterparts. Af- ter each intervention, an extensive

Summer 2001 37

Table 1 Situational Factors East West

~

From outside Little change in administration and logistic system Ministry wants reform now; no time for pilot Ministry fixes rules for grading and cut-off point Limited resources for organizational aspects of exams not covered by

Exams are written on blackboard Expectation of level of performance of students is high Deployment of SNEE-staff members’ expertise in other educational fields

Limited number of personnel Limited capability to absorb technical assistance Aspirations to be recognized as a professional institute

the project

From inside

From outside Fixed notion on quality standards Demands from the World Bank Contractual obligations Atlas syndromea

From inside Limited ti me Expected level of cooperation Tendency to take sides

The tendency to feel responsible for everything, “to carry the world on one’s shoulders.”

aide memoir was produced containing an updated list of deliverables. Planning and Adapting Obviously, a project proposal contains a plan of how to deliver the desired results. In the case of Romania, it was clear from the beginning that chang- ing circumstances required a flexible approach. Factors that influenced planning included: delay in the start of the project, reduction of overall project time, SNEE assuming executive responsibility at shorter notice than was presumed earlier. The latter meant that instead of working in a more or less protected environment, where pilot tests are organized first, SNEE im- mediately took the responsibility for a live exam: the new Capacitate exam. The solution was to apply a strategy of training with production. All compo- nents of the project were realized si- multaneously. A timetable for the period July 1998 through June 1999 was developed, integrating the Capaci- tate exams, training, and assistance in the fields of test and item construction, psychometric analysis, and administration and logistics.

In general, linking training to pro- ductive activities has been very successful and the project will continue to apply this strategy. The establishment of a modern sustainable examination system is not only a matter of technical assistance and cooperation. It requires political decisions and con- siderable investments and time. Part of establishing this system is capacity building of SNEE and its externals

(item writers). The output of technical assistance addressing the issue of sustainability, apart from the installed capacity itself, would be a reference guide that includes exam models, pi- loted specimen tes t s , marking schemes, and procedures for administration and analysis. It could serve as a standard for a future system to provide quality control.

The contribution of the project to the educational reform could thus be characterized as delivering products to implement innovation, and install- ing capacity in terms of human resources as well as in terms of the development of professional assessment instruments.

From Product to Policy How can a project like the AERo keep its professional integrity and influence the decision of policy makers? As an example, the case of the difficulty level of the Capacitate exam in Roma- nia is discussed-not to dispute the difficulty level as such, but to draw attention to the processes involved in assisting the decision-making process.

The Capacitate exam comes at the end of compulsory education in Roma- nia. Its main purpose is certification. It consists of three subjects: mother tongue, mathematics, and combined history/geography. The population has a wide ability range. In the future, the Ministry of Education would also like to use the Capacitate exam to select students for the next educational cycle: the Lycee.

When the specifications of the exam were formulated, several procedures were discussed to control the difficulty level:

0 The difficulty levels of items could be controlled by expert judgment, try-out, and pretesting.

0 The difficulty level of the test could be controlled by including, for example, many easy items and leaving out extremely difficult items, and allowing for partial scores.

0 The grade per subject could be influenced by the transformation formula applied to translate scores to grades, and by using differential weighting of easy and more difficult items.

0 The overall result of the exam could be influenced by varying the cut-off score, by allowing for compensation between the subjects, and by a posteriori adjust- ment of the grades based on an analysis of the exam.

The Ministry of Education, however, decided that no compensation would be allowed, the cut-off score was fixed, and scores were to be equivalent to grades. The desired pass rate should be about 80% of the candidates. The first reaction of “West” was “this is impossible:’’ There is no way to organize an exam on a professional basis that can meet these requirements. The reaction of “East” was “In the short term, the statement ‘impossible’ is not relevant: we have been given the task to organize the exam taking into


account the ordinances from the Min- istry.”

A strategy was developed jointly comprising the following steps:

1. Sensitize the test developers to the problem of determining the difficulty level of the test;

2. Collect information about the expected performance level of the population;

3. Develop the final tests as closely as possible to the desired param- eters;

4. Establish SNEE as the local ten- ter of expertise through inten- sive PR activities;

5. Develop a strategic paper leaving the policy makers options for re- viewing their policy, taking into account the two purposes of the Capacitate exam.

The strategy highlights the importance of psychometric capacity for SNEE. To influence educational poli- cies you need professional educational research, professional analysis, and professional reporting techniques. The

strategy also shows the importance of public relations. To secure SNEE’s po- sition as the professional institute for the conducting and administering of quality examinations in Romania and to increase support for the establishment of a modern sustainable examination system, adequate PR is crucial. The project takes care of this need: a PR consultant assists the PR officer of SNEE.

The concept of strategic papers as a tool for supporting educational reform has been used more extensively in the SMART project in Poland (see Part 111). The main difference between the use of policy papers in Poland and in Romania is that in Poland they as- sisted in decision-making before some- thing new was implemented, and in Romania they assist in decision- making to possibly change an already- introduced practice.

Possible Future Developments After the project in Romania finishes, there are various ways the contacts on

an institutional and on a personal level could develop. One is the usual continued exchange of information on the counterpart level. But the CITO- SNEE partnership has moved beyond this by accepting the opportunity of working together in the educational reform project in Moldova, with CITO as main contractor and SNEE delivering the regional expertise required.

Thus, the partnership developed within the AERo project widens. The two partners jointly have to develop a professional working relationship with a third party. They have to present a common purpose and approach, and forms of communication have to be adapted.

If CITO and SNEE successfully fin- ish the AERo project in Romania, and work together successfully in Moldova, the project might serve as an example for a future trend of international cooperation: from delivering TA for establishing a regional center of expertise, to jointly disseminating this expertise in the region.

2001 NCME Election Results

Directors (to serve 3-year terms):

Mark Reckase, Michigan State University Sharif Shakrani, National Assessment Governing Board

Vice President (to assume Presidency in 2002):

Linda Crocker is Professor of Educational Psychology and Associate Dean of Education at the University of Florida. Her research interests focus on validation design, curricular alignment, and performance assessment. Linda was an AERA vice president in 1998-99; an NCME Board member, 1997-2000; and journal editor of Educational Measurement: Issues and Practice, 1995-97. She currently serves on the GRE Board of Directors and technical advisory committees for the National Board for Professional Teaching Standards, the MCAT, and several state assessment programs.

Summer 2001 39

For Committee Use Only: Proposal I.D. Session I.D.

1.

2.

3 .

4.

5.

6.

7.

8.

9.

2002 NCME PAPER PROPOSAL COVER SHEET

(One sheet per proposal) Paper title

Presenting author’s name

Institutional affiliation

Mailing address

(Include ZIP)

Telephone ( ~ ) Fax ( - 1 E-mail

Topic descriptor name and number (one only-see Call for Proposals)

Three key words

Are you an NCME member? Yes If not, name NCME sponsor

Names and institutional affiliations of co-authors

Last First Initial

No

Will you need audiovisual equipment other than an overhead projector, screen, and microphone? Yes - No - If yes, list equipment (be specific). (It i s NCME policy that charges for special equipment be paid by the individual requesting it.)

If you wish this paper grouped in the same session with other papers submitted to NCME, please attach a separate sheet listing titles and authors of the other papers.

I certify that this paper has not been published or previously presented at a scientific meeting. I further certify that I will appear and participate in the session unless prevented by conditions beyond my control.

Signature Date

Enclosure checklist (see instructions)

~ Cover sheet ~ 1,200-1,500-word summary (6 copies, without author identification) ~ Three stamped, self-addressed envelopes ~ 3” x 5” labeled index card (see instructions) ____ Separate listing of authors’ names and addresses (see instructions)

THIS INFORMATION MUST BE RECEIVED BY THE PROGRAM CHAIR BY AUGUST 10, 2001. FIRST CLASS MAIL I S STRONGLY RECOMMENDED (AIR MAIL FROM ABROAD).

(This form may be photocopied)

Send to: Sue Wollrab

The University of Iowa Iowa Testing Programs 334 Lindquist Center Iowa City, IA 52242


For Committee Use Only: Proposal I.D. Session I.D.

1 .

2.

3 .

4.

5.

6.

7 .

8.

2002 NCME COORDINATED SESSION PROPOSAL COVER SHEET

(One sheet per proposal) Session title

Name of organizer

Institutional affiliation

Mailing address

(Include ZIP)

Telephone ( - ) Fax ( ~ ) E-mail

Moderator (if different from organizer)

Name

Type of session: Symposium ~ Debate ~ Demonstration ~ Related Paper ___

Roundtable discussion ~ Other ~ If ”Other,” please specify

Topic descriptor name and number (one only-see Call for Proposals)

Three key words

Requested length of session: 11/2 hours ~ 2 hours ~

Will you need audiovisual equipment other than an overhead projector, screen, and microphone? Yes ~ No - If yes, list equipment (be specific). (It i s NCME policy that charges for special equipment be paid by the individual requesting it.)

Last First Initial

Last First Initial

Are you an NCME member? Yes If not, name NCME sponsor

No ~

I certify that this session has not been published or previously presented at a scientific meeting. I further certify that I will appear and participate in the session unless prevented by conditions beyond my control.

Signature Date

Enclosure checklist (see instructions)

~ Cover sheet ~ 1 SO-word individual summaries (6 copies) ~ 500-word summary of Coordinated Session (6 copies) ___ Three stamped, self-addressed envelopes ~ 3” x 5” labeled index card (see instructions) ~ Separate listing of authors’ names and addresses (see instructions)

THIS INFORMATION MUST BE RECElVED BY THE PROGRAM CHAIR BY AUGUST 10, 2001. FIRST CLASS MAIL IS STRONGLY RECOMMENDED (AIR MAIL FROM ABROAD).

(This form may be photocopied)

Send to: Sue Wollrab

The University of Iowa Iowa Testing Programs 334 Lindquist Center Iowa City, IA 52242

Summer 2001 43

Call for Proposals 2002 Annual Meeting

National Council on Measurement in Education New Orleans, Louisiana April 2-4, 2002

Important Information

Due date, all proposals:

August 10, 2001

Submit training session proposals only to:

James Roberts University of Maryland

Department of Measurement, Statistics & Evaluation

1230 F Benjamin Building College Park, MD 20742

Submit all other proposals to:

Sue Wollrab The University of Iowa Iowa Testing Programs 334 Lindquist Center Iowa City, IA 52242

Proposals must contain all the documents and information specified in this Call for Proposals.

Notification of acceptance/rejection decisions:

November 14, 2001

Date papers due to moderators and discussants:

March 1, 2002

INTRODUCTION

The 2002 NCME Annual Meeting offers a unique form to exchange your ideas, results, and concerns with fellow members of the organization. The submission of a proposal consti- tutes the first step in taking advantage of such an opportunity. However, submitting a proposal involves a commitment to complete the promised work in a timely manner and to attend the Annual Meeting. Please do not submit proposals for papers or other sessions unless you fully expect to attend the Meeting if your proposal is accepted. Should you be un- able to attend the Meeting, please ensure that someone else can assume your role in a session. Furthermore, it is criti- cally important that papers be sent to discussants at least 4 weeks before the Meeting so that they will have time to review your work and prepare appropriate comments.

The Annual Meeting Program will consist primarily of paper presentations and coordinated sessions. This Call for Propos- als provides instructions for submitting proposals for sessions at the 2002 Annual Meeting.

When submitting proposals, please keep the following guidelines in mind:

All presentations must be related to some aspect of measurement in education.

Howard Everson & Audrey Qualls Co-Chairs, Program Committee

Sue Wollrab. Administrative Coordinator

Topics of proposals should not have been published previously or presented previously at another scientific meeting. Proposals describing work in progress will be considered if the activities still to be completed are clearly indicated in the proposal and if the work will be completed prior to the Meeting. The presentations should have clear relevance for practitioners.

The Program Chairs solicit your ideas and proposals for other types of sessions not explicitly listed in the Call. They also welcome suggestions for invited speakers and symposium topics. Moreover, members who are willing to serve as proposal reviewers, session moderators, and/or discussants should contact the Program Chairs by e-mail (HEverson@ collegeboard.org and audrey-quails@ uiowa.edu ). Sugges- tions provided by regular mail should be sent to Sue Wollrab at the address given above. Note, e-mail will NOT be accepted for formal proposals in response to this Call.

PROCEDURES FOR PROPOSAL SUBMISSION All members of NCME are invited to submit paper, coordi-

nated session, and training session proposals. A nonmember may submit a proposal if sponsored by an NCME member.

Participation Rules The following rules have been established to ensure the

widest possible range of participation by NCME members.

A member may not be a presenter in more than one paper session. A member may not be a presenter in more than one coordinated session, but may participate as a discussant in one other coordinated session. A member may chair or organize one event in addition to those listed previously. In all, no person may have an active role in more than four sessions.

Evaluation Criteria Each proposal will be evaluated by at least two persons

according to a blind-review process. The following criteria will guide the evaluations.

The scientific merit of the topic of the proposed session; The extent to which the material is redundant with previous

Whether the topic can be presented in the time allotted; The relevance to practitioners in the field: The clarity and completeness of expression of the proposal.

Topic Descriptors To complete No. 5 on the Cover Sheet for Proposals, use

the following topic descriptors: 1. Policy, legal, and ethical issues; 2. Classroom assessment; 3. Performance or alternative assessment; 4. Large-scale assessment; 5. Licensure and certification testing;

publications and presentations;


6. 7 . 8. 9.

10.

11. 12. 13.

Computer-based testing; Standard setting: Technical and statistical issues in test development; Statistical characteristics of items (including DIF); Statistical measures of test quality, including reliability and validity; Equating and other test score transformations. Test use with specific populations; Other issues (describe).

In addition to the main topic selection, please provide three keywords that relate to the topic of the paper.

Instructions for preparation of proposals for the different types of sessions follow. NCME will provide an overhead projector, screen, and microphone for each session. NCME will arrange for other equipment for a session (e.g., VCR, slide projector, computer, second overhead projector) at the ex- pense of the presenters.

PAPER PROPOSALS General Information

Paper sessions generally include four presentations of 10- 15 minutes each on related topics with one or two discussants. Questions and comments from the audience follow the formal presentations. This selection applies to proposals for individual papers. Related papers that are submitted together should follow the instructions for Coordinated Session Proposals.

Procedures for Submission 1. Complete the Paper Proposals Cover Sheet. A clear photocopy is acceptable.

2. Prepare six copies of a double-spaced summary (1,200 to 1,500 words) of your paper. The heading should consist of the title of your paper only. (Do not include the authors or institutional affiliations on any of these six copies.) In general, the summary should include the following sections: (a) objectives of the inquiry; (b) source(s) of the information presented in the paper (including sample, instruments, etc.); (c) meth- ods andlor techniques; (d) results and/or conclusions; and (e) educational importance of the study.

3. Submit the Paper Proposal Cover Sheet and the six copies of the paper summary with the following:

Three self-addressed, stamped envelopes A 3 x 5 index card, typed as follows: -Primary author’s name (last name first); -Complete mailing address; -Telephone number; -Fax number; -E-mail address; -Title of paper.

0 Separate listing of all authors’ names, affiliations, and mailing addresses. All authors’ names and affiliations should be typed as you would like them to appear in the Program.

COORDINATED SESSION PROPOSALS

General Information Coordinated Sessions are any of a number of different

types of sessions where a proposal for a completely organized session is submitted as a unit, including all presenters, a moderator, and discussants. Coordinated Sessions provide an opportunity to present specific challenges or topics from a variety of perspectives or to present complex topics that require a number of papers to describe the full scope of the issue. Coordinated Sessions are usually 1 l/z hours in length, but special requests can be made for a limited number of 2-hour sessions. Coordinated Sessions may include

(a) symposia, (b) debates, (c) demonstrations, (d) related papers, (e) roundtable discussions, or other innovative formats that provide extensive information about challenging issues. The following are brief descriptions of these session types.

Symposia. A series of papers that present alternative views, solutions, or interpretations for a topic or issue of broad scope and major importance. Symposium proposals should make clear the alternative viewpoints that will be presented in the session.

Debates. A formal debate on a significant issue currently facing the field of evaluation. The debaters must hold oppos- ing viewpoints on the issue. The submitter will serve as moderator for a controlled, but frank, debate between the two debaters and with the audience. The proposal must describe the issue to be debated and the views of the participants.

Demonstrations. A formal classroom-style discussion of a useful measurement concept or tool. Proposals must describe how the presenter will walk attendees through a clear, step-by-step explanation of the concept or tool; how it can be useful; how it compares to other measurement concepts or tools; its strengths and weaknesses; and how it can best be applied. This session differs from a training session in that the demonstration will provide an intellectual awareness and understanding of the concept or tool but not necessarily a hands-on opportunity to actually use the concept or tool.

Roundtable Discussions. A group discussion of an important topic. The moderator will lead a discussion of this topic and the various perspectives offered, facilitate a group effort to reach consensus, identify pros and cons, and draw conclusions. Proposals must include a detailed analysis of the topic.

Related Papers. A session with related papers provides a more extended analysis or discussion of a topic than can be provided in a single paper presentation. Related papers should either build on one another or cover difference aspects of a single topic or issue.

The organizer of a Coordinated Session should select a topic or issue of broad scope and major importance that mer- its an extended presentation. The proposal for this type of session should indicate how the interaction between the presenters will be structured. The differences in the separate presentgtions should be emphasized. Coordinated Sessions are not to be used for collections of papers that happen to be on the same general topic. There must be a common theme and an organizing structure for the session.

The organizer should identify the topic, obtain commitments from a moderator, speakers, and discussants, and plan with them the issues to be explored and the manner for exploring them. The Program Chair may suggest an additional presenter or discussant. An organizer may also act as a participant or moderator.

Organizers of accepted proposals will be expected to no- tify the moderator, speakers, and discussants promptly and should coordinate the presentations well in advance of the Meeting.

Procedures for Submission 1. Complete the Coordinated Session Proposal Cover Sheet. A clear photocopy is acceptable.

2. Prepare six copies of a 500-word summary of the topic and organizational structure of the session. The summary should make clear the distinctions between the positions taken by each presenter. In addition, six copies of a 150-word summary of each participant’s paper should be included.

If an innovative format is proposed that does not lend itself to the standard submission form, six copies of a 1,500-word

Summer 2001 45

east meets west in assessment development: western technical assistance to eastern needs. part ii:...

Documents