2007 orthern new york judges manual · you were asked to serve on the first tech challenge judging...

Northern New York Robotics Competition at Clarkson University Adapted from FTC and FLL Judge’s Manual 2007 Copyright © 2007 FIRST

2007 NORTHERN NEW YORK

JUDGES’ MANUAL Adapted From FTC and FLL Judges Manuals

For Inspiration and Recognition of Science & Technology

200 Bedford Street, Manchester, NH 03101-1103

(603) 666-3906 • (800) 871-8326 • Fax (603) 666-3907 www.usfirst.org

FIRST Tech Challenge Judge’s Manual 2007 Copyright © 2007 FIRST 2

Table of Contents

1 WELCOME TO THE NORTHERN NEW YORK ROBOTICS COMPETITION! ....................4

2 INTRODUCTION................................................................................................................................5 2.1 WHAT IS FIRST?.............................................................................................................................5 2.2 WHAT IS THE FIRST TECH CHALLENGE?........................................................................................5 2.3 WHAT IS THE FIRST LEGO LEAGUE? ..............................................................................................5 2.4 JUDGES, JUDGE ADVISORS AND JUDGE ASSISTANTS RESPONSIBILITIES..........................................6

2.4.1 Judge Responsibilities............................................................................................................6 2.4.2 Judge Assistant Responsibilities ............................................................................................6

2.5 JUDGE ROLE PHILOSOPHY...............................................................................................................7 2.6 JUDGE PANEL ATTIRE ..............................................................................................................7

3 FTC AWARDS AND JUDGING CRITERIA...................................................................................8 3.1 OVERVIEW ......................................................................................................................................8 3.2 SELECTING RECIPIENTS FOR AWARDS................................................................................8 3.3 ENGINEERING NOTEBOOKS .............................................................................................................9 3.4 FIRST TECH CHALLENGE AWARD DESCRIPTIONS ........................................................................10

3.4.1 FIRST Tech Challenge Think Award ...................................................................................10 3.4.2 FIRST Tech Challenge Innovate Award ..............................................................................10 3.4.3 FIRST Tech Challenge Amaze Award..................................................................................10 3.4.4 FIRST Tech Challenge Motivate Award ..............................................................................11 3.4.5 FIRST Tech Challenge Winning Alliance Award ................................................................11 3.4.6 FIRST Tech Challenge Finalist Alliance Award .................................................................11

3.5 JUDGING PROCESS AND JUDGING SCHEDULE ................................................................................11

4 FTC APPENDIX................................................................................................................................12 4.1 AWARD GUIDELINES.....................................................................................................................12

4.1.1 FIRST Tech Challenge Amaze Award Guidelines ...............................................................12 4.1.2 FIRST Tech Challenge Innovate Award Guidelines............................................................13 4.1.3 FIRST Tech Challenge Think Award Guidelines.................................................................14

4.2 SAMPLE AWARD QUESTIONS AND SUMMARY SHEET.....................................................................15 4.2.1 FIRST Tech Challenge Inspire Award Sample Questions ...................................................15 4.2.2 FIRST Tech Challenge Think Award Sample Questions .....................................................15 4.2.3 FIRST Tech Challenge Judge Summary Sheets ...................................................................16

5 FIRST LEGO AWARD DESCRIPTION ........................................................................................17

5.1 KEEP THE CHILDREN IN MIND ......................................................................................................17 5.2 FLL JUDGING ...............................................................................................................................17 5.3 TEAM PERFORMANCE AWARDS ...................................................................................................19

5.3.1 Power Puzzle Project Award ...............................................................................................19 5.4 TECHNICAL AWARDS....................................................................................................................19

5.4.1 Robot Design Award ............................................................................................................19 5.4.2 Robot Performance Award ..................................................................................................20

5.5 NON-TECHNICAL AWARDS............................................................................................................20 5.5.1 Against All Odds Award.......................................................................................................20 5.5.2 Team Spirit Award ...............................................................................................................20 5.5.3 Judges’ Award .....................................................................................................................20


6 FLL CHALLENGE PROJECT JUDGING MATERIALS...........................................................21 6.1 PROJECT JUDGING MATERIALS .....................................................................................................21 6.2 PROJECT SCORE SHEET .............................................................................................................24 6.3 ROBOT DESIGN JUDGING MATERIALS...........................................................................................25 6.4 ROBOT DESIGN SCORE SHEET ...................................................................................................28 6.5 TEAM SPIRIT JUDGING FORM ........................................................................................................29

7 MISCELLANEOUS FORMS FOR BOTH FTC AND FLL..........................................................31 7.1 TEAM INTRODUCTION PAGE..........................................................................................................31 7.2 TEAM COMMENT FORM................................................................................................................32


1 WELCOME TO THE NORTHERN NEW YORK ROBOTICS COMPETITION!

First of all, on behalf of the organizing committee, thank you very much for volunteering your time to judge the Northern New York Robotics Competition at Clarkson University. We appreciate the time you have given to us. This competition started last spring when we received funding to seed robotics teams in St. Lawrence County. During the summer two camps were run at Clarkson University to teach middle and elementary school teachers how to design and program robots. For many of the teachers this was their first experience with engineering design practices, building and programming. The teachers were given either Lego Mindstorm or VEX kits to take back to their schools to use with the students. The effort continued as the teachers started robotics programs at their home schools. Mentors from Clarkson University helped support the teachers and students throughout the build period. What you will see are the results of the kid’s efforts to solve a challenge problem with self designed and programmed robots. Know that your interaction with the children is critical. This competition is the highlight of the process for the kids, and the awards mean a lot to them. They are all looking to take a trophy home to show off at school. Please keep in mind that most of these kids have never designed, built, or programmed a robot. For many of the kids this is the first time they have been exposed to the design, build, test, and revise pieces of the design process. We hope that this will lead the kids towards future technology related careers. The kids are excited, and we hope you are as well! Thanks Again, The Northern New York Robotics Competition Organizing Committee


2 INTRODUCTION Welcome and thank you for agreeing to serve as a FIRST Tech Challenge competition judge, a Judge Advisor, or a Judge Assistant! You were asked to serve on the FIRST Tech Challenge judging panel because your professional accomplishments make you an ideal role model for the high school students as well as for some of the engineers and other professionals participating in the FIRST Tech Challenge. Thank you for making time in your busy schedule to assist FIRST in reaching its mission.

The purpose of this manual is two-fold: it is intended to provide FIRST Tech Challenge Judge Advisors and Judge Assistants with guidance in planning a competition and also to provide FIRST Tech Challenge judges with information that will help them judge at the competition. Judge Advisors assigned to each competition will use the information in this manual to develop the detail process used during the competition. Judges can use the information in this manual to supplement the information provided by the competition’s Judge Advisor. Most competitions will have a judge orientation meeting or teleconference before the competition to review the specifics of judging for that competition. For any questions or suggestions on how to apply this manual, please contact [email protected].

2.1 WHAT IS FIRST? FIRST (For Inspiration and Recognition of Science and Technology) was founded by inventor Dean Kamen to inspire young people’s interest and participation in science and technology. Based in Manchester, N.H., the 501 (c) 3 not-for-profit organization designs accessible, innovative programs to build self-confidence, knowledge, and life skills while motivating young people to pursue opportunities in science, technology, engineering, and math.

A volunteer-driven organization, FIRST is built on partnerships with individuals as well as businesses, educational institutions, and government. Some of the world’s most respected companies provide funding, mentorship time and talent, and equipment to make FIRST’s mission a reality. FIRST has over 60,000 committed and effective volunteers who are key to introducing 120,000 children to the joy of problem solving through engineering.

FIRST provides four well-known programs, the FIRST Robotics Competition (FRC) for high-school-aged young people, the FIRST Tech Challenge for pre-college students, the FIRST LEGO® League (FLL) for 9 to 14 year-olds (9-16 in some countries), and the Junior FIRST LEGO® League (JFLL) for 6 to 9 year-olds. Also located at FIRST Headquarters is the research and development facility called FIRST Place. FIRST Place is integral to new program development, evaluation, and professional development of FIRST mentors.

2.2 WHAT IS THE FIRST TECH CHALLENGE? In 2005, the RadioShack Corporation partnered with FIRST and Innovation First, Inc. to pilot the FIRST Vex Challenge (FVC) at the FIRST Championships. This 50-team demonstration was a huge success and lead to the 2005 and 2006 pilot seasons. In the 2006 season, the program engaged over 5,500 students and mentors on 550 teams in 25 tournaments around Canada, Mexico, and the United States. In 2007, the program was approved and renamed the FIRST Tech Challenge (FTC).

2.3 WHAT IS THE FIRST LEGO LEAGUE? FLL tournaments provide teams with an opportunity to showcase their achievements on the FLL Challenge. FLL believes that all teams who complete the Challenge are successful, and the awards represent the special achievement of particular teams. What is FIRST LEGO League?


FIRST LEGO League (FLL) introduces 9 to 14 year-olds to the fun and experience of solving real-world problems by applying math, science, and technology. FIRST LEGO� League is an international program for children created in a partnership between FIRST and the LEGO Group in 1998. Each September, FLL announces the annual Challenge to teams, which engages them in authentic scientific research and hands on robotics design using LEGO MINDSTORMSTM technologies and LEGO bricks. After eight intense weeks, the FLL season culminates at high-energy, sports-like tournaments. In 2006, over 80,000 children participated in 40 countries. The FLL Challenge Each year, we provide FLL teams around the world with an annual Challenge that has two parts: a robot game and a project. In the robot game, teams design, build, test, and program autonomous robots that must perform a series of tasks, or missions. In the project, teams conduct research and create a technological or engineering solution to an aspect of the Challenge and present that solution.

2.4 JUDGES, JUDGE ADVISORS AND JUDGE ASSISTANTS RESPONSIBILITIES 2.4.1 Judge Responsibilities

Review team documents • Engineering notebooks

Conduct team interviews • Listen to what teams have done • Identify candidate teams for awards

Observe Competition matches • Demonstrations of gracious professionalism • Robot performance

During deliberations, judges must: • Determine the top ranked teams for each award • Come to consensus on the recipient of each award • Share their notes on winning teams with the Judge Advisor and other

judges for use in the Awards Ceremony script All judges should review the following information prior to the event:

• The FIRST Tech Challenge competition description • Details of the judging process supplied by the Judge Advisor including

the competition schedule • Award descriptions and Award Guidelines

Judges need to participate during the entire competition!

2.4.2 Judge Assistant Responsibilities

Keep judges and teams on schedule

• Act as time keeper for judge sessions as required • Update judges on any schedule changes that occur throughout the day • Provide judges with team Engineering Notebooks • Provide an extra pair of eyes and ears for the judges


2.5 JUDGE ROLE PHILOSOPHY A FIRST Tech Challenge judge takes on four distinct roles during the competition:

° FIRST Ambassador: The judges are ambassadors for the FIRST mission throughout the competition. Their shirts so identify them. This role should be played to the hilt at all times during the event. Giving positive encouragement to the adults and students is very important.

° Role Model: Many of the students have never had the opportunity to interact with a real engineer or scientist, or have not been exposed to individuals who have excelled in other career fields. To be a good role model you should demonstrate a sense of humor, portray your successful career and share real life-experiences. Remember the FIRST mission.

° Detective: It is up to the judges to discover the contestants deserving of the awards. They must find the facts and sort them out in minimum time and amid utmost confusion.

° Reporter: Part of the judge duties includes writing a short script that explains why each team was recognized for their award. This script is read at the award ceremony, and as such, your notes will be useful to construct a succinct and informative description of each team's performance. Take notes! These notes will be useful to construct a succinct and informative description of the selected teams’ performance. The judges must arrive at consensus with a group of strangers, using their best detective work and be perceived as fair and wise. Judges may desire to review the team’s Engineering Notebook for information that can be used in the award scripts.

It is important to note that FIRST judges are not referees of the Competition! Judges do not need to understand all the intricate rules applying to the robots participating in the Competition but may find it fun and useful in evaluating teams for certain awards.

2.6 JUDGE PANEL ATTIRE FIRST judges are sometimes provided with a shirt that you may wear during the competition to identify you in your position. Please keep in mind that you will be doing a lot of walking – so wear comfortable shoes!

Award ceremonies are held on the playing field shortly after the final rounds have been completed. Judges are introduced and are encouraged to hi-five the award recipients as they receive their well-earned trophies and medallions.


3 FTC AWARDS AND JUDGING CRITERIA 3.1 OVERVIEW FIRST Tech Challenge recognizes both on-field performance, and the attributes that help FIRST achieve its mission. FIRST Tech Challenge judged awards are a combination of machine performance, creativity and innovation, and team spirit and performance. These awards are designed to help FIRST achieve its mission of demonstrating that science and technology can be enjoyable and rewarding. Awards represent another positive way for mentors to instill important values like teamwork, creativity, innovation, and the value of the engineering design processes. As you read the criteria for each award please make sure to use the Judges’ Guidelines found in the Appendix. You should also bring these guidelines to the team interviews. The judging process has some level of subjectivity by design but the Judging Guidelines provide judges with a tool to bring some objectively in determining the top contenders for each award. Teams are provided with the guidelines for each award to help them self assess and establish higher goals.

3.2 SELECTING RECIPIENTS FOR AWARDS There will be 16 teams competing at the event. Keep in mind that there is a great deal of work to complete during the day. Teamwork will be a key component of the judging process.

At the competition, the Judge Advisor will divide judges into groups and assign 2 to 3 judges to specific teams with a specific interview time. There should be about 2 judges to every 8 teams present at the competition. Judges then interview and learn about these teams and make an assessment of their competitiveness for each award.

Early on the day of the Competition, judges will be provided with each team’s Engineering Notebook. These notebooks are completed by each team and are a valuable tool for the judges. These notebooks are used for robot design and should show the team’s progress with their robot through the season. Its secondary mission is to describe the team’s background and explain the journey of the team, highlighting their proudest moments and provide judges with insight for use in determining award winners. Take an opportunity to review the notebooks for the teams that you will interview. It might be useful to bring the notebook to the interview. It is suggested that judges hold on to the Engineering Notebooks until after the awards scripts are written. Judges may find them useful for information and anecdotes that can be used in the awards scripts.

Judges will also have an opportunity to see teams in action on the competition floor and see teams busy repairing their robots or reviewing strategy for an upcoming round during visits to the pit area. Judges will be given a pit map that will indicate the location of each team’s workstation in the pit. Judges are free to visit teams in the pit, in the viewing stands or while they are waiting to compete. FIRST teams are easily identifiable since most showcase their team number on their team shirts or uniforms. Please remember to lead by example and wear safety glasses whenever you are in the pit area.

To view a team’s robot in action, there is often a judge seating area adjacent to the playing field. Judges are free to view the play from this location. FIRST asks that the judges be considerate of the audience and not block the view for others.

Judges will need to take notes. When doing so, make sure to capture the team number within the notes. A sample Judge Summary sheet is provided in the Appendix to help judges evaluate each team. This sheet should be used with the Judge Guidelines found in the Appendix. Judge notes will be a key element in helping to determine the winning team for each award and be the foundation for the award script that will be read at the awards ceremony. The integrity of the judging process is contingent upon the judges being able to document and explain why a team earned a coveted award. The teams take the


explanations very seriously, and accordingly, the judges need to be sure and do their homework on each team.

Finally, to protect the integrity of the awards, FIRST requires that judges with any affiliation to a team advise other judges of the affiliation, refrain from commenting upon the team, abstain from voting for the team, and refrain from influencing the judges’ decisions on such team in any manner. Judges will be asked to sign a Conflict of Interest form (see Appendix) prior to the start of the competition disclosing any team affiliations you may have. Please discuss any questions about this form with the competition Judge Advisor.

3.3 ENGINEERING NOTEBOOKS An Engineering Notebook should not be a scrapbook of the team, or full of non-robot information. While some insight as to the team’s thinking and an introduction to team members are both perfectly fine, this should serve first and foremost as a design notebook for the team’s robot and/or their strategic thinking for playing the game. While we encourage the students to be creative, an Engineering Notebook is a collection point for a team’s design ideas. As a judge, please keep this in mind as you search for the best notebook. Some questions to ask yourself as you review these notebooks:

• How do I envision industry professionals using a design notebook? Are these teams doing the same?

• Does it seem as though the team truly followed their progress throughout the season?

• What practical thinking is displayed throughout the notebook?

• Could a team complete a patent application using their drawings, ideas, and designs?

• Did this team ‘dress up’ the notebook and not put enough emphasis on the robot/design?

See some examples of Engineering Notebooks prior to the competition by visiting websites such as http://www.bookfactory.com/special_info/engr_notebook_guidelines.html. They give outlines as to how to write in an Engineering Notebook as well as what a proper notebook should include. Please give this some thought as you are reviewing the notebooks – how much research and time went into this notebook? How does this reflect what a career engineer might produce?

There are a few other things to take into consideration:

• All teams MUST submit a FIRST Tech Challenge Engineering Notebook. Teams may only submit one notebook each. Should a team decide not to submit a notebook, they are choosing to be ineligible for certain awards.

• The FIRST Tech Challenge Engineering Notebook may have been maintained by a single team member or contain contributions from multiple team members. Regardless of the individual contributors, entries need to highlight the thoughts of all team members and mentors of the team.

• Engineering Notebooks should be well organized so judges can determine how the team is put together, with their processes, brainstorms, and strategies well documented. Each journal should have 5-10 journal entries marked that the team feels are good examples of the different types of experiences the team encountered. These entries may include:

o What the team experienced during the engineering design process,

o Drawings and descriptions of the robot during the different stages of design,

o Experience of team work and communication during the building the robot,

o Defining and interesting moments, obstacles and AHAs!,

o Lessons learned from the entire experience


• The FIRST Tech Challenge Engineering Notebooks do not need to be perfect. They should reflect the team’s personality and spirit and reflect an ongoing work in progress. Spelling mistakes, scratches or abandoned pages, or other blemishes should not be held against the team. In contrast, it should mean that the team truly used the notebook for its purpose – last minute designs, a quick thought while designing, or just a brief note to remind themselves of an idea for later on.

• Each team will submit their Engineering Notebook to the Pit Administration or Team Check-In desk when they arrive at the competition site. Detail on the requirements for completing a team’s Engineering Notebook can be found in Chapter 5 – Engineering Notebooks of the FTC Game Manual – http://www.usfirst.org/community/FTC/content.aspx?id=962.

• At the end of the competition, notebooks must be returned to each team. Judges should not take notes directly in notebooks. Post-It notes should be available for use in noting things that may need a second look within a notebook. Judges will be able to hold on to the Engineering Notebooks until all awards scripts have been written. Notebooks will be returned to teams after the awards ceremony.

3.4 FIRST TECH CHALLENGE AWARD DESCRIPTIONS Judged Awards - The judges select teams to win the following awards at each of the Championship Tournaments and at the World Championship Event.

3.4.1 FIRST Tech Challenge Think Award This award is given to the team that the judges feel best reflects the “journey” the team took as they experienced the engineering design process during the build season. The Engineering Notebook is the key reference for judges to help identify the most deserving team. The team’s Engineering Notebook should focus on the design and build stage of the team’s robot. Journal entries of interest to judges for this award will include those describing the steps, brainstorms, designs, re-designs, successes, and “those interesting moments” when things were not going as planned. A team will not be a candidate for this award if the team has not completed the section of the Engineering Notebook describing the team’s experience.

3.4.2 FIRST Tech Challenge Innovate Award The FIRST Tech Challenge Innovate Award celebrates a team that not only thinks outside the box, but also has the ingenuity and inventiveness to make their designs come to life. This award is given to the team that the judges feel has the most innovative and creative robot design solution to the FIRST Tech Challenge to any and/or all specific field elements or components. Elements of this award include elegant design, robustness and “out of the box” thinking related to design. This award may address the design of the whole robot or some sub-assembly component attached to the robot. The creative component needs to work consistently, but a robot does not have to work all of the time during matches to be considered for this award. The team’s Engineering Notebook should be marked with journal entries to show the design of the component(s) and the team’s robot in order to be eligible for this award, and entries should describe succinctly how the team arrived at that solution.

3.4.3 FIRST Tech Challenge Amaze Award During the course of the competition, judges may encounter a team whose unique efforts, performance, or dynamics merit recognition. This judged award is given to the team that defies any award category. This may include:

- A team who has gone above and beyond to assist another team at the competition - A team whose robot performed consistently well, but did not win the Innovate Award - A team whose Engineering Notebook was above and beyond, but did not win the Inspire Award - A team who has spread the word of FIRST throughout their school or community


3.4.4 FIRST Tech Challenge Motivate Award This judged award celebrates the team that exemplifies the essence of the FIRST Tech Challenge competition through team spirit and enthusiasm. This team shows their spirit through costumes and fun outfits, a team cheer or outstanding spirit, and through their collective efforts to make FIRST known throughout their school or community. 3.4.5 FIRST Tech Challenge Winning Alliance Award This award will be given to the winning alliance represented in the final elimination match.

3.4.6 FIRST Tech Challenge Finalist Alliance Award This award will be given to the finalist alliance represented in the final elimination match.

3.5 JUDGING PROCESS AND JUDGING SCHEDULE The entire judging process will take place during the competition. Judges will have an opportunity to review the Engineering Notebooks developed by teams shortly after arriving at the competition. Competitions judges will work in assigned panels of 2-3 judges who will get know up to 8 teams through interviews and observation. Each panel will identify candidates for each of the judged awards using the information they learned about the team. The entire panel of judges will discuss candidates for each award and come to consensus on the team most deserving of each award. Judges will write a short script for each award that will be read when the award is given out.

Notebook Review – Judges will have limited time early in the day to review team notebooks. Judges should make every attempt to review the notebooks for teams they will interview prior to the interview. Judges may also find it helpful to bring notebooks to interview sessions for the teams they will interview. Please treat notebooks with care and do not write anything in the notebooks. Use post-it notes to make any notes. At the end of each competition they will be returned to teams so make sure they are returned to the judging room prior to the end of the day.

Interviews – The panels of judges will interview teams for a predetermined time (recommend 10 minutes) in a private judging area early in the day. These sessions should be closely timed, as slips in schedule will impact the entire competition schedule. The Judge Assistant will keep time during the interview period. Team interviews should conclude when the Judge Assistant notifies that time has expired. Judges should use the Judge Guidelines and Judge Summary Sheets during the session. Engineering Notebooks submitted by the teams being interviewed may also be a useful tool. Teams should bring their robot to the interview session to help explain elements of their design. Judges should look for highlights about the team; its history and make up; what they team achieved during the competition season; and, the experiences that were gained. Student team representatives should be able to answer the questions or elaborate on robot design functions or attributes with minimum direct assistance from the mentor. The team mentors’ contribution during interviews should be kept to a minimum. Additional follow up interviews can be conducted in the pits or at the competition arena as needed to help judges reach consensus on award determination.

Deliberation - Judges deliberation sessions should be conducted with all competition judges present. The Judge Advisor facilitates deliberation sessions with the entire judging panel and brings the judges to consensus over the winner of each award. There will likely be many good candidates for each award at each competition and there is no discrete measurement to determine the award winners. Judges need to keep in mind during deliberation that the goal is to identify a deserving team for each award. It is also strongly advised that a team receive no more than 1 judged award to allow the judges to “spread the wealth”.

Scripts - A script needs to be completed for each award. Templates for each award are included in the Appendix of this guide. Scripts should be limited to 2-3 sentences that clearly describe why the team is being awarded the specific award. Be creative, have fun and make the scripts personal to the individual team.


4 FTC APPENDIX The following appendices are intended to help the judges in the interview process, the awards selection process and the script writing process.

The award guidelines provide a break down of each award and the factors that are judged as part of that award. Each factor has suggestions to help determine if a team is “excellent”, “good”, “fair” or “need improvement”. These determinations are used to rank the teams that each judge interviewed and are not provided to the team.

The sample questions and summary sheet are tools that the judges can use when interviewing the teams. They provide a quick way for judges to come up to speed for the interview process. This also gives all the judges at an event a starting point so that each team can be evaluated on similar questions.

The judges should fill out the award scripts after the awards are determined. The goal should be for the judges to build a level of excitement and to give the award winning teams clues that they are getting the award but it should not announce the winner until the last moment.

Additionally in the appendix are the conflict of interest form and the consent and release form. All of the judges and the judge advisor should fill out these forms.

4.1 AWARD GUIDELINES 4.1.1

FIRST Tech Challenge Amaze Award Guidelines The FIRST Tech Challenge Amaze award recognizes a team that merits recognition but defies the other awards categories. This is based on the judges’ decision and no guidelines are applicable.


4.1.2 FIRST Tech Challenge Innovate Award Guidelines Needs Improvement Fair Good Excellent

Team has difficulty assembling robot & components during interview

Base Structure has some stability

Base and Modules Stable

Robot is modular and parts are used efficiently

Parts are used inefficiently Modules/extremities are difficult to apply

Robot has some difficulty with variations

Base and components very stable

Modules/extremities are difficult to maintain

Robot handles environmental variations very well

Robot unstable during motion robot displays a wide range of capabilities

Stru

ctur

al

Difficulty with environmental variations

Robot has difficulty moving at all Robot motion is appropriate Robot motion is very appropriate

Robot drive train is extremely solid

Robot motion is jerky Robot motion is not repeatable

Robot motion is repeatable

All motions are appropriate for the tasks they do

Robot motion is not precise Robot has minor difficulties with precision

Robot is very controllable Lo

com

otio

n

Robot actions are easily repeated

Robot has one or no manipulators (attachments)

Robot has 1 or more manipulators

Robot manipulators are capable

Robot has 1 or more manipulators

If there is one, it has extreme difficulty completing tasks Manipulators are not precise Some manipulators

are unreliable Manipulators perform tasks extremely well

Manipulator success is not repeatable

Some manipulators are overly complex Manipulators are robust

Manipulators are reliable

Man

ipul

atio

n

Teams can install manipulators with ease

Robot is solid Robot is solid (base and components)

Robot is a complete system-

Robot consists of a basic base design that lacks creativity and/or detailed explanation of how and why team came to that solution

Robot shows signs of team's design ideas

Robot was designed by the team

Robot is consistent with team plan and strategy-

Sensors are not used to make the robot tolerant of variations

Sensors are well-utilized

All components work together-

All components look like they belong together-

Design is unique-

Ove

rall

Des

ign

Design demonstrates creative flair


4.1.3 FIRST Tech Challenge Think Award Guidelines Needs Improvement Fair Good Excellent

Did not follow format and/or guidelines found in the FTC manual

Followed few formatting elements and/or guidelines stated in the FTC Manual

Followed most formatting guidelines stated in the FTC manual

Followed all formatting elements and/or guidelines stated in the FTC manual

Overall organization is poor and impossible to follow the team's journey and robot design ideas

Overall organization of the Engineering Notebook is poor and only some of the team's journey and robot design ideas are noted

Overall organization of the Engineering Notebook is good and the team's journey and robot design ideas are defined

Overall organization of the Engineering Notebook is excellent and the team's journey and robot design ideas are clearly defined and engaging

No pictures/drawings with no detail are found in the Engineering Notebook

Few pictures/drawing are with little to no detail are found in the Engineering Notebook

Some pictures/drawings with details that highlight only parts of the Engineering Notebook

Pictures/drawings with details documenting all stages of robot design

Form

at a

nd O

rgan

izat

ion

Team biographies have little to no information about who they are and why they wanted to participate on the FTC team

Team biographies have little information about who they are and why they wanted to participate on their FTC team

Team biographies have some information about who they are and why they wanted to participate on their FTC Team

Team Biographies have a lot of information about who they are and why they wanted to participate on the FTC team

Team member ideas within the Engineering Notebook is not integrated

Team member ideas within Engineering Notebook is not well integrated

Team member ideas within Engineering Notebook is integrated

Team member ideas within Engineering Notebook is well integrated

Does not demonstrate an understanding of the engineering design process

Engineering Notebook poorly demonstrates an understanding of the engineering design process

Engineering Notebook demonstrates an understanding of the engineering design process

Engineering Notebook demonstrates a clear understanding of the engineering design process and team actively used as a tool throughout the FTC season

Jour

ney

and

Des

ign

Entries do not highlight the team's journey

Few entries highlight the team's journey

Entries in the Engineering Notebook highlight the team's journey

Entries not only highlight the team's journey but offer insight to competition and design strategies

Team Personality is not present Team Personality is not clearly defined

Team Personality is defined

Team Personality is clearly defined

Problems were identified but no steps were taken to identify a solution

A problem was present, but the chosen solution was inadequate to some team members

A problem was identified and there is compromise evident in the solution

A problem was identified and the team worked together to find a solution

Only one member's ideas are evident Decisions made by simple majority without collaborative discussion

Cooperation is dominant theme

Collaboration and co-ownership are dominant themes

Decisions made by one team member and/or mentor Decisions made by most

of the team Decisions made by entire team

Pers

onal

ity

Robot built by only one or two team members Team focuses on

individual tasks

Team members show equality and value for each other roles

Northern New York Robotics Competition at Clarkson University Adapted from FTC and FLL Judge’s Manual 2007 Copyright © 2007 FIRST

4.2 SAMPLE AWARD QUESTIONS AND SUMMARY SHEET o In the section below are some sample questions that can be used when interviewing teams for

the awards. The questions are based off the criteria in the award descriptions.

4.2.1 FIRST Tech Challenge Inspire Award Sample Questions o What roles did each of you play on your team?

o How did you decide to break up the work?

o How did you manage your time?

o Who participated in your Engineering Notebook? Who came up with the designs?

o Can describe what “gracious professionalism” means to your team?

o Have you seen “gracious professionalism” during your build season or at this event? Describe what happened.

o Describe a problem that you faced while building the robot.

o How did you look for solutions?

o How did you solve it?

o Was it a group effort or did one person find the solution on independently?

o Do you look at science, technology, engineering, and math differently after having gone through the FIRST Tech Challenge?

o Are you interested in pursuing careers in science, technology, engineering, or math?

o Has this experience taught you any new skills?

4.2.2 FIRST Tech Challenge Think Award Sample Questions o Tell us about the process you went through to design your robot.

o How did you decide what parts of the challenge to concentrate on?

o Did you sketch out or plan any of your designs or pieces before building them? Are they in your Engineering Notebook?

o How did you decide on the design you have today?


4.2.3 FIRST Tech Challenge Judge Summary Sheets

This sheet is intended to help judges organize information they hear about each team when used in conjunction with the Judge Guidelines. Judges should have one sheet for each team they interview. This is not intended as a discrete scoring sheet.

Team Name: Team Number:

Needs Improvement Fair Good Excellent

Think Award

The Engineering Notebook is well organized, and follows the proper format.

The Engineering Notebook not only described the physical construction of the robot, but also team strategy, objectives, and the reasoning behind their design.

The content reflects the creative design process: identifying the problem, research, brainstorming, choosing the best solution, and developing and prototyping.

Innovate Award

The team has build a robust, well engineered robot that demonstrates the execution of a planned design that shows creativity and "out of the box" thinking.

The robot has a competitive drive system that is tailored to support the strengths of the teams’ strategy towards the game.

The team has designed and built an exceptional manipulator for game objects that performs consistently and effectively.

The team has developed and executed a well-planned strategy for maximizing their ability to play the game that takes scoring systems, alliance interaction, and changes and events during a match into consideration.

Amaze Award – Please list teams you feel are eligible, and examples why.


5 FIRST LEGO AWARD DESCRIPTION 5.1 KEEP THE CHILDREN IN MIND The most important thing for you to know about an FLL tournament is that it is supposed to be FUN. FIRST’s mission is to get children excited about science and technology. If you experience any special challenges during the day, focus on that goal. Everything else will fall into place. Also remember that these are children who worked hard all season to make it to the tournament. Treat their accomplishments and their work with respect, and be sure that other judges do as well. One negative comment from a judge can have a devastating effect on teams. Make it your goal as a judge to ensure that the teams know what they did well, and that they have a positive experience showcasing their achievements. 5.2 FLL JUDGING FLL teams compete in four categories: Robot Performance (score on the playing field), Robot Design, Teamwork and Project Presentation. Judges are usually grouped in teams to judge the major award areas. Larger tournaments generally have several teams of judges for each award. Each pair or group of judges should see between ten and fifteen teams during the course of the day. More than that, and we find that judges start to lose the differential between teams. Within this range, we find that the judging process runs more smoothly, volunteers and teams have an easier day, and the event stays on schedule. Judging sessions are at least ten minutes long, including questions and answers, with several minutes between sessions for judges to evaluate and score teams. Check with your tournament organizer to determine how your event will be scheduled. Award distribution is spread as equitably as possible among the teams, with no team winning more than two awards (Champion’s, Robot Design, Robot Performance, teamwork and Project Presentation). A team can only win a second award if one of the awards they win is for Robot Performance. The Pits The area that teams use as their home during the day is called the pit. Teams are usually given a table for their presentation materials, robot, laptop, and other materials. This is their gathering place for the duration of the tournament, and where they stay between robot competition rounds and judging interviews. This is a great place to meet with teams in an informal way, and see them in a more relaxed environment. It can be difficult to catch teams at their pit tables, as competition schedules are typically very tight, and the teams have little down time between interviews and robot rounds. The Competition Floor Robot rounds take place on the competition floor, on two 4’ x 8’ FLL tables, placed together to form one 8’ x 8’ competition table. Rounds last 2 ½ minutes, and are scored by referees. Teams typically have one or two practice rounds, and three competition rounds to complete during the course of the day. This is a great place to see the teams in action, and to observe how team members and coaches interact when they are focused on a goal. Adult Intervention In FLL, the children are expected to do the work – the programming, the research, and the decisionmaking. Adult coaches and mentors are guides, helping the children find the answers. FLL judges should be wary of teams where adults are overly involved, and ask questions to determine if the children did the work themselves. After questioning the children, if judges believe that adults did the work for the children – or if children tell the judges that their coach or mentor did the work – that team should be marked down to reflect this problem.


But don’t assume that the children couldn’t do a project or certain programming – ask them! Children are usually very honest, and if they can explain why they programmed the robot a certain way, why they chose a certain project topic, or how they arrived at their solution, then the judges will have evidence that the children did the work. Many tournaments limit the number of adults allowed into judging sessions. Your Judge Advisor can tell you what your tournament’s policy is, and what to expect from adults in terms of involvement during the day. During the Interviews Take plenty of notes during judging sessions, and turn in your notes to the Judge Advisor at the end of the day. Please do not take them home, as sometimes, questions about the judging deliberations process come up after the tournament. Be sure to observe teams during setup, breakdown and throughout the day. A team’s actions during unguarded moments can tell you volumes, and give you even more information than the interview can. If score sheets are being used, turn them in regularly. This allows the scorekeeper to enter information on an ongoing basis, and makes the deliberation process easier. Keep an eye on your schedule. Every time your judging team falls behind a minute or two, the schedules for other judging sessions and robot rounds are affected. Just ten minutes can mean that all judging and robot performance rounds are off schedule, and can cause havoc with the schedule for the entire day. What to Expect You will find that some children are talkative, while others are very shy. You may have to ask more questions of a team of introverts to arrive at the same information that a team of extroverts gave you voluntarily. Be prepared to re-word your questions if you find that the children are struggling to understand or answer. Try not to ask questions that allow the teams to answer with a yes or no, and encourage the teams to elaborate on their answers. Keep in mind that all judging should consider the age of the team members. Age-appropriate expectations are critical to success. Be polite and respectful, but do not allow the coach to answer questions for the team. Take note when teams look to their coach for answers, and try to determine if the children know the answer and are just nervous, or if they’re looking to their coach to find out how to answer. The children will be nervous. A tournament is a stressful experience. Asking them questions about their robot or their project can help to put them at ease. Try to ensure that each team leaves your judging room feeling positive about their performance in FLL. Team Dynamics Some teams will have clearly defined roles. Two children may program, two others are the robot drivers, and two others directed the project preparation. This is a completely acceptable team dynamic. You may find that not all of the children can answer all of the questions. All of the children should be able to tell you what their role on the team was, and what they did to contribute. If one or two children don’t answer any questions, target your questions to those children, and find out what they did during the season. Don’t pre-judge team dynamics, or believe that there is only one right answer. Listen to the children carefully, and expect different levels of appropriate coach involvement based upon the age or maturity of the team members.


5.3 TEAM PERFORMANCE AWARDS 5.3.1 Power Puzzle Project Award Part of the annual FLL Challenge requires teams to identify an existing problem in the scientific specialty related to the Challenge, and find a solution to that problem. Then teams must share their findings and their solution with others. At tournaments, these presentations are done for the Project judges. Teams are given at least five minutes for their project presentation – including setup time. This time is followed by a question and answer period. Teams may perform a skit, a PowerPoint presentation, songs, or choose another creative way to share their project solutions. The project presentations are judged on the creativity of the presentation, the innovativeness of the project solution, and the quality of the research. These interviews take place in a separate judging area, and the tournament organizers set guidelines for what audiovisual aids can be used. FLL presents the Project Award to the team whose quality research, innovative solutions, and creative presentation best reflect an in-depth understanding of the various scientific disciplines and issues involved with the Challenge Project. Research Quality The use and understanding of diverse resources to formulate an in-depth and thorough explanation of the team’s point of view and solution to the Challenge Project. Innovative Solution Thought-provoking and innovative resolution, including how and why it was chosen. Creative Presentation An imaginative, creative presentation demonstrating the team’s research and solution. 5.4 TECHNICAL AWARDS 5.4.1 Robot Design Award Robot Design is a judged award based upon the mechanical design and programming of each team’s robot for the robot game portion of each year’s Challenge. Technical judging begins with an interview with each team to discuss the design and programming of their robot. Technical judges should have a background in mechanical design and/or programming. Pairing judges with different technical backgrounds can be an effective way to ensure that judges have the required knowledge to choose the winners. Interviews take place in a separate judging area, which should include an FLL Challenge table with a field setup kit. This 4 x 8 table allows judges to observe robot action, and discuss robot design and programming challenges and choices with the FLL team. Many tournaments require that teams bring a printout of their programming to the technical judging sessions, so that judges can review the team’s programming during this interview. Some tournaments require that teams do a technical presentation, followed by questions and answers; others use a question and answer format for the entire session. Your tournament organizer will tell you what format your technical interviews will use. Judges look for teams whose work stands out for innovation and/or dependability. To assess innovation, the judges watch the robots work and look for things that make them say “Wow!” They interview team members to reveal the less obvious unique and inventive ideas. To assess dependability, the judges interview the teams to learn what solid principles and best practices they used to reduce variability and errors, with preference to robots that best “back it up” throughout the matches. Innovative Robot This award goes to the team best demonstrating its ability to think “out of the box.” Judges consider the


most original robot design approach to solving the Challenge missions. Robot Dependability The best designs make products that are consistent over time and dependable under changing conditions. This award goes to the team whose robot most consistently and dependably works every time. Programming FLL presents this award to the team that understands outstanding programming principles. This team’s robot demonstrated programming mastery. 5.4.2 Robot Performance Award Robot performance is score-based, and is judged by FLL Referees. The Referees score according to the condition of the field at the end of the 2 ½ minute round. Referees use the Challenge Mission, Rules, Field Setup and Question and Answer materials to guide their scoring. This award goes to the team whose robot achieves the best score on the competition field, or in the elimination round, at the tournament. There are several options judges use to determine the winner:

• If no elimination round is held, the team with the single highest score from one of three rounds receives the trophy.

• If elimination rounds are held, the team(s) whose robot achieved the highest score in the elimination round receives the trophy.

• If elimination rounds are held, the highest scoring team (using high score between two teams as the factor for advancement to the next round) receives the award.

If elimination rounds are held, participants are the top teams according to each team’s single highest score after three robot performance rounds. Scores may not be averaged or added together.

5.5 NON-TECHNICAL AWARDS 5.5.1 Against All Odds Award This award goes to the team that improvises and overcomes a difficult situation while still making a respectable showing. We can overcome incredible odds if we never give up, no matter what! 5.5.2 Team Spirit Award Some teams really know how to have fun. This award goes to the team that most enthusiastically demonstrates a commitment to getting others to see how accessible, fun, and rewarding science and technology can be, especially when you are part of a great team. 5.5.3 Judges’ Award During the course of competition the judges may encounter a team whose unique efforts, performance, or dynamics merit recognition. Some teams have a story that sets them apart in a unique way. Sometimes a team is so close to winning an award that the judges choose to give special recognition to the team. This award gives the judges the freedom to recognize the most remarkable teams for which a standard award does not exist. Judges should provide details on why they chose to recognize this team to be shared during the awards ceremony.


6 FLL CHALLENGE PROJECT JUDGING MATERIALS 6.1 PROJECT JUDGING MATERIALS


6.2 PROJECT SCORE SHEET


6.3 ROBOT DESIGN JUDGING MATERIALS


6.4 ROBOT DESIGN SCORE SHEET


6.5 TEAM SPIRIT JUDGING FORM


7 MISCELLANEOUS FORMS FOR BOTH FTC AND FLL 7.1 TEAM INTRODUCTION PAGE


7.2 TEAM COMMENT FORM

2007 orthern new york judges manual · you were asked to serve on the first tech challenge judging...

Documents