findings and recommendations regarding best practices in the evaluation, ranking and assignment of...

Presented 9/10/2013 1

Findings and RecommendationsRegarding Best Practices in the

Evaluation, Ranking and Assignmentof Officials

2Presented 9/10/2013

Background

At the request of several members and of the Executive Committee, a working group was convened to explore Best Practices.

Marc Block was assigned the task of chairing this group.

Interviews were conducted, and correspondence reviewed.


Findings – Evaluation (Purpose)

Development of cadets New cadets should be evaluated not more than twice in

a season. Referees with prior experience (such as USSF) may need

fewer.

Given the short length of the season, more than two assessments would not be of benefit, as an official can only develop so much in two months’ time.



Promotion to Varsity status Not all chapters require cadets to pass a field evaluation

as a requisite for attaining Varsity status. In the RVSOA chapter, once a cadet takes the

classroom training, s/he is given the Varsity exam (in class), and is rewarded with a NJSIAA badge upon successful completion. They then rely on feedback from paired officials and good assignor judgment to time their entry into Varsity level matches.

Other sports, such as girls’ lacrosse, do have a promotion system.

SJSOA’s use of the JV Tournament as a capstone of the Varsity promotion process seems to be a best in class approach.



Promotion to Varsity status Number of assessments required – Cadets wishing to

be promoted to Varsity status should be required to pass two evaluations in the season prior to their promotion. If a cadet should fail two evaluations in the same season, s/he will not be eligible for promotion for the following season.



Promotion to Varsity status Frequency of requests to be promoted – cadets

should be considered for promotion to Varsity status not more than twice in any four year period.

If a cadet fails to succeed in achieving promotion in two consecutive years, then s/he may not apply for promotion again until an additional two years have passed. This is to allow time for the cadet to develop as an official and prevent serial applicants.

Cadets may attempt to achieve promotion in alternating years, if they should desire.



Maintenance – periodic evaluation of an experienced referee can be beneficial. Such a practice will catch bad habits which develop over

time, as well as changes in the Rules not being correctly applied.

Experienced officials should not be assessed more than once in a season (for maintenance purposes), but at least once each 5 years would be appropriate.

For Cause – if the Board receives notice that credibly questions an official’s competence, then that referee should be assessed as soon as possible.

JV Tournament – to the extent possible, all games played in the JV Tournament should be watched by an assessor.


Findings – Evaluation (Style)

Pairing with experienced officials - This appears to be the most common form of evaluation used by all chapters, and other sports. The fatal flaw in the technique however, is that an

official cannot simultaneously critically observe their partner AND perform their own duties well.



“Sideline” assessment by retired officials – SJSOA has made use of this technique for some time. The obvious benefit is that we are not pulling officials away from games in order to observe other officials (thereby depriving the observing official of a game fee as well as potentially creating assigning issues). Two issues have been observed with this style: Knowledge level of the assessors – some assessors scored poorly on

the same recertification exam taken by the current officials and/or had incorrect knowledge of the current Rules.

Familiarity with the current game – Back in the 1990’s, FIFA lowered the retirement age of FIFA referees down to 45. This was NOT done for physical or mental fitness reasons. Rather it was done to ensure that the referees were in touch with the ever-evolving style of play. For similar reasons, the ability of a person who has been long removed from the center, or learned the game decades ago, should not be taken as a given. Some former officials may well put forth the effort to stay current; others may not.



“Sideline” assessment by current officials – this style is utilized very successfully by girls lacrosse. Pros – Current officials have the most recent knowledge

about what needs to be done to manage a game. It encourages current officials to re-invest in the sport and should be considered an ethically-correct thing to do.

Cons – Using current officials to evaluate will reduce the pool for assignment on that day. It will also deprive the assessors of a game fee.



Recommendation – current Varsity officials should be “highly encouraged” to assess at least once per season, if asked. Clearly, not all Varsity officials will be asked to assess. To lessen the impact to the assignor, current officials

should be used to assess on “light” days only. To lessen the financial impact to the current officials

asked to assess, they should be asked to assess an afternoon match on a day when they can be assigned to work an evening match.


Findings – Evaluation (Cost to Officials)

Development of cadets – new cadets should not be charged directly for assessment.

Promotion to Varsity status – candidates for promotion should not be charged directly for their two required assessments. If, however, a candidate should fail one

assessment and wishes to attempt a third assessment (to meet the requirement of two passing scores), then the candidate should be billed the cost of that extra assessment.


Findings – Evaluation (Cost to Officials)

Maintenance – Officials should not be charged directly for a maintenance assessment.

For Cause – Officials should not be charged directly for assessments done for cause. Should that official fail such an assessment, they

should be charged for any further assessment done that season.

JV Tournament – Officials should not be charged directly for evaluations performed at the JV Tournament.


Findings – Evaluation (Payment to Assessors)

In season (Developmental, Promotion, Maintenance, For Cause) – assessors should be paid a fee to be set by the Executive Board. Not only will this compensate our retiree assessors, but will also reduce the impact of a current official taking a game off to watch another.

JV Tournament – Volunteers should be solicited to assess games at the JV Tournament. Being weekend-base, the tourney is not scheduled on

any “heavy” days, so assignment conflicts should be negligible. Additionally, developing the next generation of

Varsity officials should be viewed as professional obligation.

In lieu of a per match payment, a travel stipend could be considered to defray any costs to the current officials volunteering their time.


Findings – Ranking

While good in theory, any formal ranking of officials beyond Varsity and sub-Varsity is of little use, and will only lead to dissention and rivalry within the Chapter. In order to be of any use, the games would need to be ranked as well.


Findings – Ranking

Problems with formal ranking of officials include: Who would do the ranking? We would spend more time resolving appeals than

administering the Chapter. It is already done informally by the assignor – if people

do not like their level of assignment, they can talk to assignor. It is also an annually elected position.

Problems with ranking games include: On what basis would the teams be ranked? (Courier Post

ranking, records, etc.) How often would the ranking be updated? How to factor in rivalries? How to factor in star player injuries? How to factor in “difficult” coaches?


Findings – Assignment

Given the large number of variables in the greater equation, the assignor does a remarkable job. Officials are not treated equally, nor should they be. Not every licensed CPA would make a good CFO, just as not all doctors are equally skilled, even within a specialty.



Things that need to be considered: Skill level of the official – this could be done by formal

ranking, but not recommended as above. However, this is a potential weakness in the assignment process, as most of this knowledge is resident in the assignor himself. It would need to be carefully passed along to future and/or additional assignors.

Difficulty of the match – not only the skill of each team, but also the rivalry between the schools, the meaning of the game (tourney, playoffs, division , intra-district play, etc.), the difficulty of the coaches, the speed of the teams, etc.

Travel requirements of the officials – not only the travel radius set within Arbiter, but also traffic considerations (not a feature within Arbiter!)



Things that need to be considered: Physical readiness of the officials History of officials with the teams Personality conflicts between officials Pairing of senior and junior officials Rescheduling due to weather – reshuffle 10 officials, or

only 2? General care and feeding of the referee pool – gentle

“rewards” when possible for going the extra mile (or 20 miles)


Findings – Assignment (Auto-assign)

While Arbiter does offer an auto-assign feature, it should only be used as a first pass tool. Auto-assign will easily develop the 80% solution and

should continue to be used for just that. The schedule then must continue to be reviewed by the

assignor for tweaks necessitated by the above mentioned considerations.


Findings – Assignment (Accept/Decline)

The Arbiter settings should be changed to allow officials to decline matches without sending an e-mail or placing a phone call to the assignor. This will force each official to actively accept or decline

each match, rather than the current passive acceptance triggered simply by viewing the assignment.

Allowing such action will show respect for the officials, and should in total lighten the work load of the assignor, with there being fewer phone calls and direct e-mails to deal with.


Findings – Responses to unsolicited comments

An official’s ranking should be adjusted based on his/her age. This would be amount to age-based discrimination and

is illegal in our country.

SJSOA should enforce a limit on the number of Varsity matches scheduled on any day regardless of conference requirements. The schools are our customers. No Varsity match has

deliberately been left without full coverage in recent years.



We need to gainfully employ our small pool of assessors or risk losing them due to inactivity and the uncertainty of their next assignment. As already noted above, we should rely more on current

officials to assess and less on former officials. Also, we should not create assessment opportunities simply to guarantee an income stream for a small group of folks.

Very few e-mail reports were received from Varsity officials commenting on the capabilities and limitations of the sub-Varsity officials they worked with. The need for such feedback needs to receive greater

emphasis, as this is a very important source of information.



All new cadets claiming USSF referee experience should provide copies of yearly USSF registration forms. This information is largely kept electronically now, not

by a stack of “pink sheets” as all USSF referees used to collect. When such claimed experience is questioned, it is easily verifiable by other means.

Provide yearly assessor training to increase the assessor pool This could be a good additional use of the JV

tournament!

Schools should have the ability to block officials They already have this capability



Officials blocked by schools should be so informed Agreed, if this is not the current policy, it should be.

Referees should block schools they should not or do not wish to officiate Already done

No sub-Varsity official should receive a Varsity assignment if a varsity referee is available Good in theory, taking into account the many

considerations listed earlier.



Schools should be able to prepare a list of preferred referees for the playoffs, referees should be told when they are placed on such a list, and the assignor should attempt to reward officials who have been so honored by scheduling them to one of that school’s playoff games. Schools already make such lists Informing referees that they were put on a list by a

school will lead to problems. While a compliment to the referees listed, it will also lead towards ill-will against the schools that did not list a certain official.

The assignor already does make use of such lists, but there are only so many playoff matches!



A committee should be used to compile the list of officials sent to NJSIAA and recommended for assignment in the Regional/State finals. Anyone on the committee should not place themselves on the list. This would lead to a committee of officials not “good

enough” to be assigned to a Regional/State final being used to select those who are “good enough”. The greater good is served by having those of higher experience and ability put together the list.

Officials should receive a similar number of games for all schools within their travel limits. Not all referees are equal, nor should their assignments

be equal.



A referee communication system (such as Ref Talk) should be purchased and used to enhance the training and development efforts, as well on “big games” (such as Coaches Cup finals) Agreed. Such a system could be used in pre-season, to

allow a veteran official to communicate with a paired cadet during the game. It could also be used on sub-Varsity matches to allow an assessor to communicate with and mentor the on-field officials.

findings and recommendations regarding best practices in the evaluation, ranking and assignment of...

Documents

varsity promotion process

promotion system

field evaluation

varsity exam

common form of evaluation

experienced officials

varsity level matches

following season