Real Time Evaluation: Tips, Tricks, and Tools of the TradeYuqi WangWednesday, November 11, 2015AEA 2015, Chicago, IL @yuqi_2w6

Survey Creating real time surveys

The short method of surveying in real time is to use software with the ability to survey people in real time, and analyze the results instantly. Often times you can embed the software in webinars and PowerPoint presentations. Some commonly used online software are:

o www.GoToWebinar.como www.PollEverywhere.como www.Participoll.com

The long method of surveying in real time is to use hard copies of surveys to collect real time data, and use Excel for data entry and analysis. For a step by step process of how to do this, go to this link: http://bit.ly/1Mhs5Mg.

Tips and resources An Excel resource for doing the long method of surveying in real time is www.excel-easy.com. Attention to detail and organization is important, especially for the long method. Make sure all

your formulas function correctly before data entry day. Make sure the survey you are using has mostly, if not all, closed-ended questions, like a Likert

Scale. Closed-ended questions have a short analysis time, and the analysis will be easy for both Excel and the software to do.

An AEA 2015 session on the topic you can go to is “Getting Real with Real Time Data Collection” Thursday, November 12, 3:00-4:30PM, Roosevelt room, Ignite 31. Danielle Toussaint, Ruthie Chang, and Kristie Glatze of Hatchuel Tabernik and Associates will review the evaluation of a 2014 international tech conference, and show how the use of mixed methods helped to enhance and produce memorable results.

H-formCreating real time H-formsStart with writing the question or topic you want participants to provide their feedback on at the top of a blank piece of paper. Draw a horizontal line in the middle of the paper, and label it with the spectrum of opinions you want participants to provide feedback on (e.g. strongly disagree, disagree, neutral, agree, and strongly disagree). Leave enough space on the left and right side of the paper to write “Why do you disagree?” and “Why do you agree”, respectively.

Once everyone has gotten a chance to provide their feedback, the group can begin to discuss what they shared, and start to reach a group understanding about the question or topic.

1

Real Time Evaluation: Tips, Tricks, and Tools of the TradeYuqi WangWednesday, November 11, 2015AEA 2015, Chicago, IL @yuqi_2w6

Tips and resources Provide participants with dot stickers they can place anywhere on the spectrum of opinions. You

can also provide post-it notes for participants so they can jot down their reasoning behind their opinion.

An AEA 2015 session on the topic you can go to is “Supercharge Your Focus Groups! Using Adult Facilitation Techniques to Capture rich Data and Ensuring Your Participants Have Fun in the Process!” Wednesday, November 11, 6:15-7:00PM, Grand Suite 2A/B. Lynne Franco of EnCompass LLC will introduce a technique and process that involves the use of a large sticky wall and index cards – tools that can draw out perspectives and foster group processing data.

Timeline Creating real time timelines Timelines can be a good for participants to think about what worked and what didn’t work for a particular event so they are able to apply what they learned to future events. But before creating a timeline, it’s important to make sure participants are clear on why they are creating a timeline, and what it is going to be used for.

Once you have a good understanding of the timeline’s purpose and utility, one way to create a timeline is to use blue painters tape, and large butcher-block paper. Use tape to create a single line on the paper – this will represent the past and present sections of the timeline. Use smaller pieces of tape to create a dotted line representing future. Have participants write on post-it notes milestones that have occurred within the span of the timeline, and stick it on to the appropriate sections. You can also have participants choose a few milestones they listed to explore the milestones’ conditions of success and challenges.

Tips and resources You can begin a timeline when a project starts. This is called doing a “pre-mortem”. Harvard

Business Review has a great article on pre-mortems: http://bit.l y/1EMTn7a Angie Ficek wrote a great AEA365 blog on creating timelines with line charts:

http://bit.ly/1KGlUM8 Color-coding past and present events or groups of past and present of events on your timeline

will be helpful for you to organize what happened in the past and present, and will be helpful for you to spot any patterns within your timeline.

An AEA 2015 session on the topic you can go to is “Putting Data in Context: Timelining for Evaluators” Friday, November 13, 7:00-7:45AM, Burnham room. Kat Athanasiades, Smriti Bajracharya, and Katherine Haugh of Innovation Network will demonstrate how to incorporate a timeline into a report, use a timeline to track progress internally, and utilize data visualization principles to create a visual timeline.

2