personalized extensions: democratizing the programming of virtual-physical interactions

Personalized Extensions:

Democratizing the Programming of Virtual-Physical Interactions

MASSACH-USL-1-S INST

by OF TECHNOLOGY

Abdulrahman Y. idlbi

B.S. Computer Engineering, Damascus University (2009) LIBRARIES

Submitted to the Program in Media Arts and Sciences,School of Architecture and Planning,

in partial fulfillment of the requirements for the degree of

Master of Science in Media Arts and Sciences

at the

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

September 2014

Massachusetts Institute of Technology 2014. All rights reserved.

Signature redactedAuthor

Program in Media Arts and SciencesAugust 22, 2014

Signature redactedCertified by

Mitchel ResnickLEGO Papert Professor of Learning Research

Academic Head, Program in Media Arts and SciencesThesis Supervisor

Signature redacted

Accepted byPattie Maes

Alex W. Dreyfoos Professor of Media TechnologyInterim Academic Head, Program in Media Arts and Sciences



byAbdulrahman Y. idlbi

Submitted to the Program in Media Arts and Sciences,School of Architecture and Planning,

on August 22, 2014, in partial fulfillment of therequirements for the degree of

Master of Science in Media Arts and Sciences

Abstract

This thesis describes a new framework that enables a wider range of people to extendand customize programming environments to support more personalized explorations,especially in the physical world.

While the vocabulary of the Scratch programming language (http://scratch.mit.edu) canbe extended to access hardware devices and web services through Scratch Extensions,creating an extension is not trivial and requires a significant level of technical expertise.This occasionally means having less personalized, more abstract language for thoseextensions. I describe and analyze the design of Personalized Extensions, an extensionmechanism that allows the users of Scratch to personalize abstract Scratch extensions,and consequently help other users to have more meaningful learning experiences in theworlds which Scratch extensions allow them to access. I conclude by reflecting on somecase studies where personalized extensions were used, and how they affected thelearning experience of the users.

Thesis Supervisor: Mitchel ResnickTitle: LEGO Papert Professor of Learning Research, Program in Media Arts and Sciences

This thesis is based on research supported by the National Science Foundation under grantnumbers 1002713 and 1027848. Any opinions, findings, and conclusions, or recommendationsexpressed in this document are those of the author and do not necessarily reflect the views ofthe National Science Foundation.

3



byAbdulrahman Y. idlbi

The following people served as readers for this thesis:

Signature redacted

Thesis Reader!/ Eric Klopfer

ProfessorDirector, The Scheller Teacher Education Program

Massachusetts Institute of Technology

Signature redacted

Thesis Reader

Amon MillnerVisiting Assistant Professor of Computing Innovation

Director, EASE LabFranklin W. Olin College of Engineering

5

Acknowledgements

I would like to begin by thanking my advisor, Mitch Resnick. Your ideas have been asource of inspiration since I knew about Scratch and the Computer Clubhouse sevenyears ago. I am grateful for giving me the opportunity to be part of this adventure, andfor supporting me in every possible way while exploring and following my passions.Thank you for your kindness, patience, and all the wonderful work you've been doing toempower youth.

I am also grateful to my thesis readers, Eric Klopfer and Amon Millner. Eric was on asabbatical and Amon was expecting a baby, but they generously provided thoughtfuladvice, reflections, and comments on my work.

The members of the MIT Scratch Team and Lifelong Kindergarten group have been a bigfamily filled with love, compassion, and creative ideas. Each has contributed in their ownawesome way. A special note of thanks goes out to J. Philipp Schmidt, a conversationwith whom inspired the idea behind this work; David Mellis, John Maloney, SayaminduDasgupta, and Shane M. Clements, whose work on Scratch Extensions was essential forthis thesis; Ricarose Roque for valuable advice and lessons on conducting research andrunning workshops (and for being with Sayamindu the kind, caring elder siblings theyhave been); and Amos Blanton and Natalie Rusk for their love and support all the way.Gobos forever v

My childhood and teenage LEGO building experiences, by myself or with my fourbrothers (collaborating peacefully most of the time), had deeply affected my interests inlearning in the physical world and connecting it to the virtual one. BuildingScratch/LEGO WeDo chain reactions was a favorite activity while being at LLK. Thanksto the folks at the LEGO Company for the creative construction opportunities they havebeen providing over generations.

Thanks to SparkFun (especially Nathan Siedle, Founder and CEO) for generouslycontributing most of the electronics that were used in the workshops I conducted as partof this work.

My gorgeous family...

I.-i~I .9 'x,9 .i~ PLL -4i1 -Ir - eA -

.4s1 4iI .Ii . 44, S VS 4Jhi .4

.r.. lIyJI Lic J Ly 4i1 Liyj ai oiI j J49 L L : "ol LaJJ V UI a.

7

Contents

1 INTRODUCTION .......................................................................................................................... 13

1.1 THESIS OVERVIEW ....................................................................................................................... 15

2 OVERVIEW OF VIRTUAL-PHYSICAL CONSTRUCTION KITS............................................................ 17

2.1 SCRATCH 1.X M ODIFICATIONS AND EXTENSIONS............................................................................ 25

2.2 SCRATCH 2.0 EXTENSIONS ............................................................................................................ 27

2.2.1 HTTP Extensions ............................................................................................................... 28

2.2.2 JavaScript Extensions .................................................................................................... 29

2.2.3 Accessing Scratch Extensions ...................................................................................... 32

3 PERSONALIZED EXTENSIONS ...................................................... 35

3.1 PERSONALIZING SCRATCH EXTENSIONS......................................................................................... 38

3.1.1 Procedures to Libraries, Custom Blocks to Personalized Extensions ............................. 41

3.2 HYPOTHETICAL USE CASE.............................................................................................................. 44

4 EXPERIENCING PERSONALIZED EXTENSIONS............................................................................ 51

.4.1 M ETHODOLOGY.......................................................................................................................... 51

4.2 W ORKSHOP FORMAT ................................................................................................................... 53

4 .3 F IN D IN G S...................................................................................................................................5 8

4.3.1 On Connecting the Virtual to the Physical................................................................... 58

4.3.2 On Personalized Extensions ........................................................................................ 58

4.3.3 On Setup and Support M aterials.................................................................................. 60

5 LOOKING BACK, LOOKING AHEAD ........................................................................................... 63

5.1 PERSONALIZING AN EXTENSION, BUILDING A COMMUNITY ............................................................. 63

5.2 LEVELS OF PARTICIPATION ............................................................................................................. 66

5.3 GUIDING THE DESIGN OF PERSONALIZED EXTENSIONS..................................................................... 67

5.4 PERSONALIZED EXTENSIONS AS SCRATCH LIBRARIES........................................................................... 68

9

5.5 EMPOW ERING SCIENCE EXPLORATIONS WITH COMPUTATIONAL TOOLS ............................................. 69

A IMPROVEMENTS TO SCRATCH'S "MAKE A BLOCK" ................................................................... 71

B W ORKSHOP M ATERIALS ............................................................................................................. 75

REFERENCES..................................................................................................................................87

10

In the name of Allah;the Most Gracious; the Most Merciful;

God of Abraham, Moses, Jesus, and Muhammad, peace be upon them all.

11

Introduction

Nowadays, children are increasingly involved in various virtual activities. However,

physical objects still have their own attractiveness. Our physical experiences happen

way before our virtual ones and stick with us. Physical objects surround us everywhere

and are more intuitive to manipulate and play with, and patterns of collaboration

emerge more easily and naturally around them, even when we are talking about young

children. This is why it makes sense not to forget about the interactions with the

physical world as we are creating new learning adventures for the children in the virtual

one.

Michael and Ann Eisenberg wrote about the relationship that should be between the

crafts and computer applications [1], which I think can be generalized to the kind of

relationship that should be between the physical and virtual worlds:

It's a natural desire to employ all one's senses and cognitive powers in the

course of a single project. We do not feel that a love of crafts is

incompatible with technophilia, nor that an enjoyment of computer

applications must detract from time spent in crafting. The world is not, or

13

P

should not be at any rate, a battleground between the real and the virtual.

It is instead a marvelous continuum, a source of wonders that blend and

knead together the natural and artificial, the traditional and novel, the

scientifically objective and the personally expressive, the tangible and the

abstract. We anticipate a future in which ever more astonishing things will

present themselves to our minds, and ever more astonishing ideas to our

hands.

Having construction kits that support creating projects that connect the virtual and

physical worlds allows providing multiple entry points for people with various interests

and backgrounds to start the learning and creation process. Additionally, learning about

certain physical principles in the world can be more effective when developed through

interactions with actual physical items. However, creating such connections and making

them accessible for a wide range of users and creators has not been easy. Looking at the

programming language and environment Scratch for example [2], creating interfaces

between the virtual authoring environment and the physical world (through what is

called Scratch Extensions) requires a relatively high level of technical expertise. This

means only a small set of physical devices can be supported, and that the users of the

environment have little choice in customizing the extensions to meet their creation and

learning needs and styles.

This thesis explores how the barrier of creating Scratch extensions can be lowered to

allow people with diverse backgrounds and interests, including educators, hobbyists

and teenagers, to build their own customized interactions with the natural world with a

shorter learning curve. As extensions become easier to create, people will have the

freedom to connect a wider range of hardware kits, including ones of their own design.

Throughout this thesis I describe and analyze the design of Personalized Extensions, an

extension mechanism that allows the users of Scratch to personalize abstract Scratch

extensions, and consequently help other users to have more meaningful learning

14

experiences in the worlds which Scratch extensions allow them to access. I conclude by

reflecting on some case studies where Personalized Extensions were used, and how they

affected the learning experience of the users.

1.1 Thesis Overview

In Chapter 2, I overview the various construction kits for children that combine both the

virtual and physical worlds. That covers the different ways through which Scratch has

supported hardware connections.

In Chapter 3, I describe the motivation and design of Personalized Extensions, a low-

floor mechanism to offer more meaningful learning experiences in the worlds which

Scratch extensions allow to access.

In Chapter 4, I present case studies that describe the experiences of learners using

Personalized Extensions.

In Chapter 5, I reflect on the design of Personalized Extensions and the experience of

their users, discussing design implications and future directions.

15

Overview of Virtual-Physical

Construction Kits

There has been a long history of creating construction kits that connect the physical and

virtual worlds. When Seymour Papert and his colleagues at MIT developed the Logo

programming language for children in the late 1960's, what we remember now as a

"screen turtle" was actually a "floor turtle" (figure 2-1): a simple mechanical robot

connected to the computer by a long cable. Commands like "forward 50" and "right 90"

were used to control an actual robot instead of graphic images on the computer screen [

3].

In the following decades several kits in various forms were created like LEGO/Logo,

LEGO Mindstorms, and Crickets; which were designed to support computerized

construction in the physical world.

17

Figure 2-1: Logo 'floor turtle"[4]

LEGO/Logo was a computer-based robotics system that supports a variety of design

activities. Children used LEGO bricks (including nontraditional ones like gears, sensors,

and motors) to build machines that could be connected to a computer and programmed

using a modified version of the Logo programming language (figure 2-2).

LEGO/Logo was meant to encourage the children to learn through the design process as

they created a variety of their own machines before programming them. It was "viewed

as a 'multi-media construction kit,' allowing students to build and create in several

different (though interconnected) media." [5]

18

PoCu :Computer and Serial Interface box interpret the input

optaoo .. r

Inpnp[ oup:

KeyboardSensorsMouse

Action by LEGO componentsComputer screen display

F igure 2-2: Principles of LEGO/Logo learning environment [6

A follow-up of the LEGO/Logo project was the Programmable Brick (figure 2-3). It was a

tiny, portable computer embedded inside a LEGO brick, capable of interacting with the

physical world in a variety of ways. The Programmable Brick was designed to support

multiple activities, multiple input/output modalities, multiple (parallel) processes, and

multiple programmable bricks that interact with each other. The design activities and

application for the brick included autonomous creatures, active environments, and

personal science experiments. Similar to LEGO/Logo, a program had to be written on a

computer using a special version of Logo known as Brick Logo. However, the programs

are to be downloaded afterwards via a cable to the Programmable Brick, then the cable

can be disconnected as the programs remain stored on the brick [7]. This project led to

the development of LEGO Mindstorms and the PicoCricket robotics kits.

19

SENSOR INPUTS

OPERATION

ON/OFF + + * * + /ESWITCH

UTR LOW-BATTERYSTARTLEBUTTON .

SCREENSTOP CHOICEBUTTON KNOB

Figure 2-3: The Programmable Brick from MIT [81

LEGO Mindstorms is a programmable robotics construction kit that was released in

1998 to become a commercial success. The kit includes LEGO bricks, gears, motors,

sensors, and an intelligent brick which is based on MIT's Programmable Brick (figure 2-

4); and has been designed to be particularly well-suited for developing mobile robots,

and has been the foundation for many robot competitions around the world; most

notably, the FIRST LEGO League. On the other hand, the PicoCricket kit, which was very

similar to LEGO Mindstorms, was designed for making artistic creations (e.g. musical

sculptures, interactive jewelry, and dancing creatures) with lights, sound, music, and

motion (figure 2-5) [9].1

1 The PicoCricket kit was released in 2006 and has been discontinued since 2012.

20

4.t

(a) (b)

Figure 2-4: (a) A robot made with EV3, the third generation kit in LEGO Mindstorms robotics line (b)An EV3 program

4 /

Lmmot

on .

foreverset 100 - bripowe.

(a) (b)

Figure 2-5: (a) A PicoCricket project (b) A PicoCricket program created with PicoBlock

21

r

After helping develop the robotics kits mentioned above, the Lifelong Kindergarten

group started developing Scratch: a programming language with an online community

for children to create and share their own interactive media such as games, stories, and

animations (figure 2-6). LLK designed Scratch to be more tinkerable, more meaningful,

and more social than past programming languages [10]. Since launching the website in

2007, Scratch has become a vibrant online community with over 6 million projects

shared by thousands of creators, primarily between the ages of 8 and 16 years old.

U---- 't.bJ WH.

..*~n bx:: . t.- 6

*~.wIo.l~r6

iMuWPs

O~0r b.O .q Yb. '

u -ub01 rM.B.~

(b)(a)

Figure 2-6: (a) The Scratch community website (b) The Scratch programming environment

Right from the beginning, Scratch has provided support for the PicoBoard (also known

as Scratch Sensor Board): an electronic board with multiple sensors that allow Scratch

projects to sense and respond to things going on in the real world (figure 2-7). Later,

Scratch also added programming blocks for the LEGO WeDo robotics kit which has a

motor and a couple of sensors. What makes Scratch different from the examples

mentioned above is that Scratch can be used to control objects in both the virtual and

physical worlds and make them interact in various ways.

22

IW" ® a.' as r I -. J

slider

lightF: sensor

4>g) sop

Figure 2-7: The PicoBoard and a Scratch script using the board's sound sensor

A Scratcher can for example create an interactive story or game like the Ferris wheel in

figure 2-8. The Ferris wheel will keep running (both the real one and the one on the

screen) until the LEGO minifigure is brought closer to the WeDo light sensor. Another

example is building a digital-physical chain reaction: Something from the physical world

triggers a sensor, which kicks off a chain reaction within the virtual world of the Scratch

project (figure 2-9). At the end of that virtual chain reaction, a motor is activated, which

continues the chain reaction in the physical world. That triggers the next WeDo / Scratch

project on the next laptop, on and on for as many links as there are in the chain [11].

23

set Whirl effect to sensor value

J % 0

- .} kJ )file dUt Share Hae

Ly~~jpm

:+ n

Figure 2-8: A Scratch/WeDo project by Rita Chen, Inna Koyrakh, and Jinha Lee; created for MAS.714,a course offered at MIT Media Lab in the Fall 2009 term [121

N,

?Ake

.- d 'O

-is Lft- 0. I

e.+: 00-0 00"I

N

1d, ~

Figure 2-9: A virtual/physical link from a Scratch/LEGO WeDo chain reaction

However, to understand the real potential of connecting Scratch to the physical world,

one has to look at the history of Scratch modifications and extensions, many of which

were intended to access hardware devices using Scratch.

24

2.1 Scratch 1.x Modifications and Extensions

The source code of the first generation of Scratch (1.x) has been freely available,2 and

this has allowed advanced Scratch users and enthusiasts to modify and extend the

programming language in different ways [13]. The modifications created of Scratch (also

called mods) would typically introduce new functionality to the language, notably

controlling a hardware device, and in some cases extend the grammar. There was not a

standard process to follow to create a Scratch mod. One could simply download the

source code and modify it as they wish as long as they meet the Scratch trademark

policy, license the code properly, and not upload projects created by the mods to Scratch

website.

The MIT Scratch Team would also create mods to experiment with new ideas without

affecting the normal users. For instance, NetScratch [14] was an early modification that

had new dynamic blocks to receive data from the Internet. In another mod, Scratch was

extended to support image-processing primitives, so that children programming with

Scratch could create their own image filters [15]. Some of these extensions made their

way into the official version of Scratch - one such example being the extensions that

made it possible to use hardware in the LEGO WeDo kit with Scratch [16].

Outside of efforts by the MIT Scratch Team, Chirp was a Scratch mod that focused on

adding new features like exporting and importing scripts as XML and compiling projects

as executable files (to make standalone applications) [17]. Another notable mod, Build

Your Own Blocks (later called Snap!) [18] extended the language by allowing users to

define new blocks (procedures and functions). Additionally, multiple BYOB projects

could interact by sharing variables and broadcasts.

2 The first generation of Scratch was written in Squeak, an open-source implementation of the Smalltalk-80 language.

25

BYOB also extended the grammar of Scratch by adding new computational concepts like

programmatic cloning to let sprites duplicate themselves while the program is running,

and new data structures like lists of lists. The MIT Scratch Team considered adding some

of these features to Scratch but they were always cautious in establishing a balance

between making Scratch more powerful by adding new functionality and still having a

language that is easy to get started with.3 Those are two of the three elements which the

MIT Scratch Team focus on while designing Scratch: having a "low floor" (easy to get

started), a "high ceiling" (opportunities to create increasingly complex projects over

time), and "wide walls" (supporting many different types of projects so people with

many different interests and learning styles can all become engaged) [19]. When

introducing new features to Scratch, the Team tends to prefer maintaining a low floor to

raising the ceiling; however, enabling people to create their Scratch mods allowed them

to break away from those design constraints to try new ideas and concepts on their own.

Mods not only extended Scratch within the realm of virtual world, but also they were

created to connect Scratch to hardware devices, in order to provide a friendlier access to

the physical world, as compared to a traditional hardware programming language like C

or assembly. S4A is a mod that provides new blocks to handle sensors and actuators

connected to an Arduino board. Enchanting is another mod that is used to program

LEGO Mindstorms NXT robots. In contrast to most other mods, Enchanting removed

some of the Scratch blocks, thereby making itself more specialized, and incompatible

with projects created with the official version of Scratch.

Apart from direct source code modification, another method to extend Scratch was

introduced with version 1.3.1 through what was called the remote sensors protocol.

When the remote sensor connection feature is enabled, custom Scratch blocks can be

created with programs that talk to Scratch through a network protocol. Two notable

3 Some of these features (e.g. cloning and procedures) have been added to Scratch 2.0 in a modified form.

26

examples of Scratch extensions made using this protocol are JoyTail, which adds joystick

support to Scratch, and Kinect2Scratch, which allows creating Scratch programs that

communicate with Microsoft Kinect.

(10) ffi

value of sensor AM F90L

Elba:

Blocks to control a hardwaredevice - Enchanting (LEGOMindstorms), S4A (Arduino), andKinect2Scratch (Microsoft Kinect)

temperature in02139

.betkle .edu

W-1 0 -- "_ -h scratch.mit.edu

Blocks to access the web - NetScratch,Snap!, and Insanity.

New control blocks and data New operations - Insanity and Snap!structures - Snap!

Figure 2-10: Examples of blocks from different categories from Scratch mods

It should be noted here that regardless of the method that is used to extend Scratch,

projects created with a mod could not be shared on Scratch online community website.

Links to get new mods can only be shared on the forums or Scratch Wiki, but they are

not endorsed or curated by the MIT Scratch Team.

2.2 Scratch 2.0 Extensions

Noticing the various mods and extensions created by Scratch community, the plans for

the second generation of Scratch included creating an infrastructure to support building

and sharing extensions to the functionality of the programming language.

The design of the extension mechanism went through two main stages:

27

[90ipt vari b4s

^3s

insanity - {

1 foo.txt

2.2.1 HTTP Extensions

The main focus at the beginning was extending Scratch 2.0 to control external devices

(e.g. robotics kits, musical instruments) and to access data from external sensor

hardware (e.g. sensor boards). As Scratch 2.0 runs in the browser,4 it cannot interact

with hardware devices directly due to browser security restrictions. To run an

extension, a separate application, called a helper app, must be installed and run on the

computer. Scratch communicates with the helper app via HTTP requests, and the helper

app talks to the hardware [20].

In addition to the helper app, an extension description file is required to use the

extension. The description file is a text file in JSON formats that includes the extension's

name, the TCP/IP port number used to communicate with the helper app, and a list of

Scratch block specifications (figure 2-11).

Every HTTP extension needs a separate helper app to be developed, installed, and run;

which makes these extensions difficult to create, maintain, and distribute. With the

introduction of the next version of extensions, JavaScript extensions, the online version

of Scratch 2.0 no longer supports HTTP extensions.6

4 An offline editor was introduced later.s See http://www.json.org/6 The Scratch offline editor still supports HTTP extensions.

28

1 {2 "extensionName": "A4S (Arduino For Scratch)",3 "extensionPort": 12345,4 "blockSpecs": [5 1" ", "set pin %n as 9mi.mode", "pinMode", 2, "Digital Input"],6 [" ", "digital write pin %n %m.highLow", "digitalWrite", 13, "high"],7 [" ", "analog write pin %n value %n", "analogWrite", 3, 2551,8 [" ", "servo write pin %n degrees %n", "servoWrite", 5, 1801,9 ["b", "digital read pin 'n", "digitalRead", 2],

10 ["r", "analog read pin %n", "analogRead", 0],11 ],12 "menus": {13 "mode": ["Digital Input", "Digital Output", "Analog Input", "Analog Output((PW )","Servo"],14 "highLow": ["high", "low"],15 },16 }

Figure 2-11: An HTTP extension description file -A4S (https://github.com/damellis/A4S)

A4S (Arduino For Scratch) v s

se i s2 gtldigital write pin 6)high

analog write pin 3.255

servo write pin degrees

Lanalog read pin

Figure 2-12: A4S (Arduino for Scratch) extension blocks (https://github.com/damellis/A4S)

2.2.2 JavaScript Extensions

The HTTP extensions were cumbersome to manage, share, and run as each extension

needed a different helper app that has to be installed. A new method for extending

Scratch was introduced which allows Scratch to be expanded to interact with hardware

devices (e.g. LEGO WeDo or the Kinect), or access online web-services (e.g. the US

National Oceanic and Atmospheric Administration - NOAA weather service). The

extensions themselves are implemented in Javascript and can use a single browser

plugin to access hardware devices [21].

A JavaScript program for an extension has basically the following parts: a cleanup

function when the extension is unloaded; status reporting code to report missing

hardware, plugin or unsupported browser; block and block menu descriptions and

29

corresponding functions for the blocks; and the code to register the extension. Figure 2-

13 shows the simplest code possible for an extension. It defines an extension called "My

first extension" with a single command block "My first block" that does nothing.

1 (function(ext) {2 // Cleanup function when the extension is unloaded3 ext._shutdown = function() {};45 // Status reporting code6 // Use this to report missing hardware, plugin or unsupported browser7 ext._getStatus = function() {8 return {status: 2, msg: 'Ready'};9 };1011 ext.myfirst block = function() {12 I/ Code that gets executed when the block is run131415 // Block and block menu descriptions16 var descriptor = {17 blocks: [18 // Block type, block name, function name19 [' ', 'my first block', 'my_first_block'],20 ]212223 1/ Register the extension24 ScratchExtensions.register('My first extension', descriptor, ext);25 })({});

Figure 2-13: The basic code for a javaScript extension [22]

The JavaScript code for the extension includes an extension descriptor to define the

extension blocks, which is similar in format to the HTTP extension's description file. It

lists the blocks and their type, parameters, menu description, and corresponding

functions (figure 2-14).

30

1 var descriptor = {2 blocks: [3 ['w', 'turn motor on for 96n secs', 'motorOnFor', 1],4 [' ', 'turn motor on', 'allMotorsOn'],5 [' ', 'turn motor off', 'allMotorsOff'],6 1' ', 'set motor power %n', 'startMotorPower', 100],7 [' ', 'set motor direction %m.motorDirection', 'setMotorDirection', 'this way'],8 ['h', 'when distance %m.lessMore %n', 'whenDistance', '<', 20],9 ['h', 'when tilt %m.eNe %n', 'whenTilt', '=', 1],

10 ['r', 'distance', 'getDistance'],11 ['r', 'tilt', 'getTilt']1213 menus: {14 motorDirection: ['this way', 'that way', 'reverse'],15 lessMore: ['<', '>'],16 eNe: ['=','not =']17 },18 url: 'http://info.scratch.mit.edu/WeDo'19 };

Figure 2-14: A JavaScript extension descriptor - LEGO WeDo extension

LEGO WeDov Q

t~nmotor tuon for 0 secs

tr motor doetont hi a

Figure 2-15: LEGO WeDo extension blocks

In both methods, a Scratch 2.0 extension extends Scratch with a collection of command,

reporter, and event blocks that can be used to interact with a particular device or source

of information on the web. When an extension is enabled, its blocks appear in the "More

Blocks" palette.

An extension is limited to adding a set of new blocks to expand the Scratch vocabulary to

read or control objects that reside outside Scratch (i.e. hardware devices or web

31

services), but cannot alter the language grammar nor affect the look or behavior of

sprites or the stage.

2.2.3 Accessing Scratch Extensions

Extensions that are officially curated by the MIT Scratch Team, also called approved

extensions, are accessible through the Extension Library (figure 2-16). Developers

building new extensions can share them by sharing Scratch project files that use these

extensions.

When an extension is added or a project that already uses extensions is loaded, the new

blocks appear in the "More Blocks" palette.

XExtension Library

N1Hardware

PicoBoard LEGO WeDo

OK Cancel

Figure 2-16: Scratch Extension Library, from where approved extensions can be added to a project

32

Finch V

Move Finch left: right

Finch bz a Hz for 00Ms

Finch LED color R0 0 : 0

"' Finch temperature

L~ Finch ri

ft .bstHelle

LFinch rlgt obstacle

""Finch left light" Fbsnch

X acceleration

LEGO WeDov Q C

K iT m. .r .ii~i . . 1 -l~

se moo dieto to thsa

n20

rrr"" Finch Orientation

(a) (b)

Speech To Text vWeather extension v

current temperature in city Boston, MA

current condition in city Boston, MA

(c)

--- 0@

(d)

Figure 2-17: Examples of Scratch 2.0 extensions (a) An HTTP extension for the Finch robot (b)JavaScript extension for the LEGO WeDo kit (c) A JavaScript extension for a weather web service (d) A

speech-to-text JavaScript extension that uses the browser's API

33

0

Finch Y acceleration

Finch Z acceleration

recognize speech and wait

turn motor on

Personalized Extensions

In a conversation with a colleague about the different ways Scratch could support

teachers in classrooms, he mentioned a potential use of Scratch by teachers to create

their own science kits and share them with their students to explore various disciplines.

Commercial science kits are expensive and the MIT Scratch Team had already been

working on developing the mechanism to extend Scratch, mainly to connect it to

hardware devices.

It would be great if teachers could use equipment that was available and relatively cheap

(e.g. Arduino board connected to various sensors) to design and build their own science

kits, then use Scratch to control and access that kit to explore the world around.

The challenge was that Scratch extensions could only be developed by some of the more

advanced Scratch users, as well as developers of hardware kits who want to see their

devices supported in the Scratch language. In many cases that would be just fine. Take

the LEGO WeDo extension for example (figure 3-1) [23]; the parts of the hardware kit

are known: motors, lights, distance sensor, and light sensor. Consequently, the

developers of the LEGO WeDo extension could define blocks that are meaningful to the

35

young users and reflect how the hardware will be used in real life. If a Scratcher wants to

know the distance between the distance sensor and an object, she simply uses the

"distance" reporter block.

LEGO WeDo - Q

when distance <

zwO

(a) (b)

Figure 3-1: (a) The LEGO WeDo robotics kit (b) The LEGO WeDo extension blocks

However, for an abstract hardware device like the Arduino board,7 one cannot predict

how it will be used in the real world. An extension developer for the Arduino can only

define generic blocks that access the basic low-level functionality of the board - mainly

reading or setting the pins on the board (figure 3-2). A Scratcher cannot intuitively use

the blocks to read a distance sensor that is connected to the board. She will have instead

7 Arduino (http://www.arduino.cc/) is an open-source physical computing platform based on a simple

microcontroller board, and a development environment for writing software for the board.

36

set motor power to

turn motor off

to read the value of the pin to which the sensor is connected, using a block that reads

"analog read pin 1".

A4S (Arduino For Scratch) v

set pin 2 Digital nput

_ 13

"' "3255sdagnal write pin haluh

digital read plnG

a n a lo g re ad p A

(a) (b)

Figure 3-2: (a) An Arduino Uno board (b) The A4S (Arduino for Scratch) extension blocks

Figure 3-3 shows how a temperature sensor (TMP36) can be connected to an Arduino

board, and how a script that uses the abstract extension blocks will look like. The

example script sets the color effect for a Scratch sprite to the temperature value

(measured in Celsius). The value read on the analog input (which is a 10-bit analog-to-

digital converter) has to be converted to millivolts, then plugged into the formulas

indicated at the bottom of the figure.

37

-. . - -.O 00 02 00 1

*C t n m -E

OSemp in (ou0nm-5001/0

Temp in *F = [(Temp in *C) x 1.8 ]+ 32

Figure 3-3: Connecting a TMP36 temperature sensor to Arduino, and a script using the A4S extensionto read the temperature

3.1 Personalizing Scratch Extensions

In the previous example, the end users of the hardware kit could have been helped by

providing programming blocks that had been personalized, given the fact that the

configuration of the kit became known. To approach this challenge, I built on a new

feature that was introduced in Scratch 2.0 to allow users to define custom blocks.

Custom blocks are Scratch's version of what is usually referred to as procedures,

functions, or methods in other programming languages [24].

38

Scripts Costumes Sounds

*Motion l EventsLooks * ControlSound ' Sensing

* Pen ' OperatorsData Mor Bck

Make a Block

jump pixels in 0 secs

Add an Extension

Edit Block

0jum h ght pxes ntime sc

v Options

Add number input:

Add string input:

Add boolean input:

Add label text:

!t' Run without screen refresh

OK Cancel

.h5 tisspit 1i~.

text

Figure 3-4: Adding and defining a custom block in Scratch 2.0

Custom blocks are created and found in the "More Blocks" palette, the same place where

Scratch extensions appear when added. A custom block is shown in the palette for each

Define hat block. To create a custom block, the "Make a Block" button in the "More

Blocks" palette is used. A new block window will open, where one can name the custom

block and define its parameters. A Define hat block appears in the scripting area where

the script that defines the new block can be built.

Figure 3-4 shows how a simple "jump" can be defined using standard Scratch blocks.

Similarly, a new block can be defined using blocks from an extension. The definition of a

39

Sho

ii

F

i

II

"blink" block for the Arduino board using the A4S extension 8 looks like what figure 3-5

shows. The new "blink" block will make the LED connected to the digital pin #13 blink

10 times.9

Make a Block

Add an Extension

A4S (Arduino For

'i o 1

Scratch) v

se 2i a igtl nu

digtalwrte inC 25ig

1

diia rie 1

* 1

digital read pin

an.g read pin

Figure 3-5: Defining a custom "blink" block that uses A4S extension blocks

The creator of the project can now use the new block in his project, and even share the

project so other Scratchers can use it as a whole or use the individual scripts (including

new blocks) in it. For the creator of the project, being able to see the definition of the

"blink" block makes sense: she created it and might be iterating on its design. Although

there is no intuitive connection between "blinking" and "digital write pin 13" high and

8 I use the A4S (Arduino for Scratch) extension through this document to demonstrate creatingPersonalized Extensions based on it. A4S is being developed and maintained by David Mellis from theLifelong Kindergarten group, and is publicly available on https://github.com/damellis/A4S9 It should be noted that the Scratch script or program do not get uploaded to the connected hardware, incontrast to what is usually understood when talking about programming an Arduino board. When theprogram is run, Scratch maintains a live connection with the board, and commands are sent to the boardin the real time. This paradigm keeps the programmable objects in Scratch and the outside objects (theconnected hardware in this case) all in synch.

40

1

low, she still knows what that means. That could be also the case with some other users

of the project, but for many others, they would just like to use the new block to make an

LED blink. Their focus is what it does, not how it does it, which is similar to the concept

of using a library in a conventional programming language.

3.1.1 Procedures to Libraries, Custom Blocks to Personalized Extensions

Sharing new custom blocks with the community where their definition is directly visible

to the users has its benefits, especially within the principles that are advocated for by

the MIT Scratch Team regarding openness. 10 However, encapsulating and hiding some

functionality has its own value. In one of my favorite papers on designing construction

kits for kids, Mitchel Resnick and Brian Silverman talk about the importance of choosing

black boxes carefully. In designing Scratch, much thought was put into choosing the basic

building blocks. These choices do not only affect how easy or difficult it is to use the

programming language (or the construction kit in general), but also "what ideas users

can explore with the kit - and what ideas remain hidden from view." [25]

In the case of a hardware platform like the Arduino, working on a low level using

commands that deal with individual pins is good if that is what the user is eager to learn

about (or expected to). 11 However, if the user is more interested in exploring with

10 Once a project is shared on Scratch website, visitors of the website not only can run the project, but also"look inside" it to inspect the programming scripts, objects, images, and sounds; experiment with andmodify them; use these elements into their own projects; or even remix the whole project. This aspect ofScratch design has always triggered discussions among the members of the community about sharing,remixing and giving proper credits. Additionally, the source code of Scratch has been freely available sinceits release.11 Low-level commands can be viewed as abbreviations designed for a chip to understand (e.g. "analogread pin 1"), while higher-level commands are those designed for humans to understand (e.g. "turn LEDon").

41

sensors in the world, higher-level commands that deal with the sensors directly will help

the user in focusing on the topic in hand, encouraging deeper explorations. 12

However, as the developer of the original, abstract extension can only see the plain

hardware board, but not how it will be used, she can only develop a Scratch extension

with abstract, low-level blocks. But a user who has a specific use in mind for that piece of

hardware, can create a modified version of the hardware extension, the same way she

can create a custom block out of Scratch standard blocks, then black-box it to encourage

certain learning experiences when the modified extension, which I will call Personalized

Extension, is shared with others.

[1: blink extension "23 by adlogi (unshared)

I

x: 240 y: 180

Stage SK o Dinoaurl

New backdrop


Motion ' EventsLooks 3 Control

' Sound ' SensingPen 3 OperatorsData

Make a Block

Add an Extension

A4S (Ardulno For Scratch) vi®

s1 3i s D i ia o u

d3i a 2r5e p5 h g

anlgw 0e i au

Backpack

Figure 3-6: Sharing a Scratch 2.0 project with the Scratch online community

12 It is worth mentioning that Arduino has been successful partly due to its encapsulation of variousaspects of electronics and microcontrollers, which made it easier for makers and hobbyists to use [26].

42

y:.54

i

-- I

q= q

When a creator wants to share her project, she currently presses the "share" button

(figure 3-6). Then, other Scratch users can view and open the project, and they will see

exactly the same objects and scripts the project owner can.

If the creator wants instead to share an encapsulation of the behavior, leaving the

definition invisible to create a Personalized Extension, I add the "Export a Personalized

Extension" and "Import a Personalized Extension" to the File menu in Scratch editor

(figure 3-7). When a creator chooses to export a Personalized Extension, all the custom

blocks definitions are saved to an external file, which can be shared later with other

users.

blink exten stumes Sounds p a

w23 by adlogi (ur vents

Control

SensingOperators y

ion

- A4S (Ardulno For Scratch) r

se3 p; s D gia nStage Sprte1 Dinoauri d

I backdrop n

Lanalog red pin vau 25New backd~rop:

4= Q

Backpack

Figure 3-7: A mockup of Scratch 2.0 interface to export and import Personalized Extensions

When another user imports that Personalized Extension, neither the custom blocks

definitions nor the original extension blocks that were used to define the custom blocks

will be visible. The blocks that were defined as custom blocks will now be available as a

new extension (figure 3-8). However, if the Personalized Extension is opened as a

Scratch project (File -+ Upload from your computer) instead of importing it (File -+

43

Import a Personalized Extension), the original extension and block definitions will

appear again, and can be edited to modify or remix the Personalized Extension.

d z~r File I Edit , Tips About J "1,0& adlogi v

r - Untitled-6

4a2 by d"og (unshared)

x; 240 y: -180 ,

i/p

Now backdrop:

a/ra


* Motion 3 EventsLooks ControlSound ' SensingPen ' opertorS

Data

Make a Block

Add an Extension

blink extension v

Backpack

Figure 3-8: A "blink" Personalized Extension after being imported

The current interface for creating custom blocks in Scratch serves as a good basis to

build Personalized Extensions. However, a couple of features should be introduced to

make Personalized Extensions support blocks of different types. Those changes are

listed in Appendix A.

3.2 Hypothetical Use Case

Nada is a high school teacher who wants her students to learn about some

environmental issues. She wants to provide them with some tools to observe and

explore these issues in the real world, experiment with it, not only read about it.

One option is to seek a commercial science kit with various sensors which has software

that does the data acquisition and present the data in a collection of graphs and tables.

Even when a carefully-designed kit is easy to use, several challenges arise. Assuming the

44

=q 0

D

I

financial cost is not an issue, Nada still wants to be able to customize the kit to fit

explorations in different areas in the future, especially to meet unexpected questions

coming from the students themselves.

Additionally, she does not want her students to deal with dry, abstract data as the only

option they have. Using abstract data to create graphs can be useful in many cases to

represent and understand a phenomenon, but this kind representation does not always

fit everyone. For example, when numbers that represent the readings of a carbon

monoxide sensor are shown, students might not be sure what these numbers mean.

They would wonder "Is a reading of 75 a bad thing? What caused this value? How does it

affect the environment around us?" Allowing the students to create more concrete

representations of the readings (e.g. projects with pictures where the readings are

indicated by the color of the sky, the traffic on the highway, and the number of trees) can

make certain ideas about environmental conditions more accessible. Nada wants to

support and validate multiple ways of thinking and knowing about the subject in hand

- both abstract and concrete [27]. She expects that some of her students would have

better learning experiences by creating stories or animations that use what they observe

in the real world. She would even like to have them build live virtual-physical

interactions that combine the intuitiveness and concreteness of the physical world with

the computational power the virtual world offers.

Nada knew Scratch could be used to create personal representations of the data. She had

also some experience using the Arduino platform after building some projects using

online tutorials, and knew she could have some sensors attached to an Arduino board,

then connect the board to Scratch to read data from the real world. She would lose some

accuracy, and would have to build the Arduino kits herself (or with the help of her

students), but once that is done her students can start their own explorations, using a kit

that deals with many of the issues mentioned above.

45

As she wants to explore environmental issues, Nada builds a kit that has a carbon

monoxide sensor, a light sensor, a temperature sensor, and a tri-color LED, all connected

to an Arduino board; then connects it to Scratch by adding the A4S (Arduino for Scratch)

extension available in the Scratch Extension Library (figure 3-9).

XExtension Library

AllHardware

PftoBoard WeDo A45

OK Cancel

Figure 3-9: Adding an extension from the Scratch Extension Library

The A4S extension provides Nada with abstract block to access the board, with

commands like "digital write pin 13" and reporters like "analog read pin" (figure 3-10),

but her board now has very specific features and capabilities. She enhanced it by

connecting several sensors to reflect certain needs and applications, using instructions

from various online tutorials. While she can use the abstract blocks, she wants her

students to use blocks that reflects how the board (or the kit) is used in the real world;

not a bunch of electronic pins anymore, but sensors that read gas, light, and temperature

values.


2digital write pin

high

* m* 3 255analog write pin bvalue

digital read pin

Lanalog read pin

Figure 3-10: A4S (Arduino for Scratch) extension blocks as shown in the "More Blocks" palette

46

She defines the new blocks as she knows what sensors she is using and how they are

connected to the board. She creates new blocks to report the values of the temperature

sensor (temperature), the light sensor (light), and the carbon monoxide sensor (carbon

monoxide); in addition to several blocks that can control a tri-color LED (turn LED on,

turn LED off, and set LED color to) to indicate some status (figure 3-11).

70 My Envlroinmen KitsZ3 by adlogi(ushed)

Sl240 L 5 m 8

et/can

Ne bakrp

Scipts Costumes n.c'

Moon Evnt

* Looks , Cuntun

Socund lUSesincJ Pen 1 O~emt-,r

Doata

Meke a Bc

Add an Extes~o

US (Ardulno for Scratch)' vs

v d (Z

- ----.

Backpack

Figure 3-11: Defining custom blocks using the A4S extension blocks to create a PersonalizedExtension called "My Environment Kit"

Afterwards, Nada exports the new blocks as a Personalized Extension she calls "My

Environment Kit", and shares the resulting file with her students who import it into their

projects to use the new blocks.

47

uJ

son FRO " Edit " -nips About 4 A .mss u; i odlopl

!n

'! >dennt carbon monoxide

report analog read pine n

e r: s

to z.

tt

looking for green

Ks4

r/~ a


*Motion sEvents3Looks Control

* Sound * Sensing3Pen 3Operatorsa a

Make a Block

Add an Extension

My Environment Kit "

Backpack

Figure 3-12: A project using the "My

- - -: -1 3-

Ii

Environment Kit" Personalized Extension

Some students will use the Personalized Extension to create Scratch projects that show

graphs representing collected data, but other will seek different forms of representation,

including pictures and animations (figures 3-12 and 3-13).

I

Figure 3-13: Screenshots from a hypothetical Scratch project, where the creator chooses to representthe readings of a carbon monoxide sensor by the color of the sky, the traffic on the highway, and the

number of trees, indicating changing environmental conditions [28]

Moreover, Nada can now share her creation - the environment kit - with other

interested educators and students. She can do that by sharing the Personalized

Extension and how she designed and built her Arduino-based kit (figure 3-14 and

Appendix B for more examples of cards that can be shared to explain how to use the kit).

48

FRO W Edlt v Tips About j . r w r A

Temperature Sensor

Components

" TMP36 - Temperature Sensor

' The yellow wire an be connected to any othe ANAIOG IN pins, numbered Ao-A&.

Blocks

Tell the board which analog pin thetemperature sensor is connected to.

Get the temperatrein Fahrenheit.

Example

Lr-

F .

3-

1

-

4-

i-F

- SO

4- -..--. ----t---;

- - 71-ft..

-+. --4 ..

1

saratch.mit.edu gthub.com/dameHis/A48 aratch.mit.edu github.com/daneis/A4S

Figure 3-14: A card used to support the learners while using the "My Environment Kit" Personalized

Extension

49

7t.

-IT iT.

rI

Experiencing Personalized

Extensions

In this chapter, I describe a number of case studies of abstract and personalized

extensions being used to connect Scratch to the physical world, specifically, via Arduino

boards. The audiences varied: high school students, college students and educators. I

look at learning outcomes and challenges that arose while using the extensions to

connect Scratch to the physical world.

4.1 Methodology

To understand the efficacy and implications of my design, I conducted several informal

studies with students and educators, and gathered feedback through face-to-face

conversations, and discussions on feedback forms. I worked with educators to get their

feedback and perspective on the potential use of Personalized Extensions in their

classrooms, and with students who had no previous experience with electronics to

observe their first experiences with Arduino while using Personalized Extensions. A

listing of the contexts that I used for my studies follows:

51

Workshop 1

The first study took place at the Media Lab with educators who had attended Scratch

Educator Meetups. 13 It was a three-hour workshop with three participants, all of whom

were familiar with Scratch, but only one had worked with electronics and Arduino.

Workshop 2

The second workshop lasted four hours with a different audience. It was held at the

Flagship Computer Clubhouse at the Boston Museum of Science with seven

participants:14 the Clubhouse coordinator and six high-school students who were

volunteering there. The youth were new to Scratch and Arduino.

Workshop 3

This workshop was also four hours long. It took place at the Media Lab with five college

students. They were new to Scratch, but a couple had some experience of coding and

none of Arduino.

Workshop 4

This one was of shorter duration (90 minutes). It happened during the Scratch@MIT

conference. There were 16 participants, mostly educators who were familiar with

Scratch. Most of them had little or no Arduino experience.

13 See http://scratched.gse.harvard.edu/discussions/scratch-educator-meetups14 The Computer Clubhouse (http://www.computerclubhouse.org/) is a creative and safe out-of-schoollearning environment where young people from underserved communities work with adult mentors toexplore their own ideas, develop new skills, and build confidence in themselves through the use oftechnology.

52

Figure 4-1: Pictures from the Personalized Extensions workshops

4.2 Workshop Format

The topic of all workshops was connecting Scratch to the physical world via Arduino.

A workshop starts with 30- to 45-minute introduction to Scratch if the participants were

new to it. That is followed with a brief introduction to the Arduino board showing how it

could be used with Scratch to read sensors and control LEDs (figure 4-2), how to use a

breadboard, and how to import an extension.

There were two aspects of the Personalized Extension experience I could study: creating

Personalized Extensions, and using them to explore the physical world. Extension

creators had to be familiar with both Scratch and Arduino, so I decided to focus on the

53

larger audience of extension users. Most of the workshops focused on how the

participants' first experience with Arduino would be when a Personalized Extension is

used. There was a case where a participant moved to use an abstract extension after

using the personalized one. In the last workshop, participants started by trying the

abstract extension,15 and then moved on to use the Personalized Extension.

Figure 4-2: A high striker example using Arduino with a Personalized A4S Extension to read a force-sensitive resistor and control LEDs while running an interactive animation on the screen

The abstract extension that was used was A4S (Arduino for Scratch), which is an

experimental HTTP extension that can be used with Scratch offline editor [29]. A4S is

written in Java. The Personalized Extension was based on A4S and had the blocks shown

in the figure 4-3.

1s See the first handout in Appendix B.

54

A4S (Scratch for Arduino) v 0

connect light sensor to pin AO

connect button to pin 7

A4S (Arduino For Scratch) v S2. .se. pED 13 Dt

dgal3 write pin h255

digital read 10 10:10

analog read pin

(a) (b)

Figure 4-3: (a) A4S extension blocks (b) A4S personalized extension blocks

After an introduction to Arduino, the participants were given handouts that explained

how to connect 3-5 sensors and two kinds of LEDs to an Arduino board (see Appendix

B). They were asked to try connecting a sensor then an LED and program them as a start.

Afterwards, they were encouraged to try different sensors or expand what they already

had connected into larger projects using craft materials and LEGO bricks they had

around. For example, one of the participants decided to integrate the tricolor LED and

force sensor he had connected into a robot. To complete his project he wanted to add

another LED, which led him to explore using the abstract extension with the

personalized one (figure 4-4).

55

N

P4,01

;r -MW

A.Figure 4-4: A robot expressing anger when its head is poked. A Scratch program plays a sound while

the eyes (tri-color LEDs) changes color from green to red. The head is hiding a force sensitive resistorunder a LEGO plate. Video on http://youtu.be/FroOEUSKpdU

The handouts were inspired by Scratch Cards, which are used to help beginners in using

Scratch (figure 4-5). The handouts were designed to be simple, created with software

56

available for everyone, so teachers knew they could easily build support materials if

they were to produce their own Personalized Extensions (figure 4-6).16

M4o~Jfl pt1MDIl ~n

______ Clkk tooe the sprite Ikbrry

Choe sprite that has 2Lmoeosum_

You can change its Mutat tytttt.

QictM( ttw

Figure 4-5: A Scratch Card

LED

LED

. 330 7a~a c . n)- n*

wbsd 21-P

Turnan LED an or o1ELEs arenumbe.ed accrding to the DIOFYAL pinthqiae conce toi (a..7, n ea.

Figure 4-6: A handout that was used to support creating with the personalized A4S extension

16 The handouts were produced mainly using Google Docs (https://docs.google.com), and Fritzing(http://fritzing.org) was used to create the breadboard view of the circuits.

57

X20

t

-

ij- +

T.1~

h4T4

4:

t

aithb.ao/deneBa./A48

-Moving .An Mol,

acrateh.mdt.edut

to

4.3 Findings

My focus was observing how Personalized Extensions could lower the barrier for

Scratchers as they make connections between the virtual and physical world.

I developed the following points by observing the participants during the workshop and

by face-to-face conversations. At the end of every workshop, I used a reflection tool

called Green-Yellow-Red, asking the participants about what went well, what was fair or

questionable, and what could be improved; then opened the floor to final remarks.

4.3.1 On Connecting the Virtual to the Physical

Making connections with the physical world provides additional entry points for

learners and supports a multiplicity of styles and experiences. A participant who

was using Scratch for the first time said the blocks (both standard and

personalized) were intuitive to use. However, it was the connection to the

Arduino what made Scratch make sense to him as a college student. He

later added that he liked "seeing [his] program going into something real out in

the world."

4.3.2 On Personalized Extensions

* Many participants expressed that the Personalized Extension made using the

Arduino board easier. One educator who had previously attended Arduino

workshops said she felt for the first time she could replicate what she learned to

share it with her students. A college student said she had felt intimidated to try

[learning] electronics before, but liked her experience as a beginner using the

Personalized Extension.

" The Personalized Extension made the learning and building experience more

personal, as creators could focus on what they can create, rather than how to make

it run in the first place. A college student described her experience saying: "Come

58

out with something, and create it, and make it your own". Another added he felt

he was "only limited by his imagination, not the tools."

* Personalized Extensions can also provide a multiplicity of entry points, whether

depending on interests or levels of expertise. An educator mentioned she had

students with different skills and interests attending her computing class. With

offering abstract and personalized extensions, each of her students could choose

the level of abstraction they are comfortable with, while exploring similar

computational ideas.

* Personalized Extensions allowed for a more participatory, engaging approach to

the workshops. Because abstract behaviors were black-boxed, there was no need

to explain them right from the beginning. The young participants particularly

liked that they did not have to be lectured at the beginning to explain how things

worked. Instead they could start the hands-on activity quickly.

* The abstract and personalized blocks can complement each other, even for the

same learner. In the third workshop, one of the participants wanted to use two

tri-color LEDs in his project. To make a balance between the capabilities of the

Personalized Extension and its simplicity, it provided blocks to control only one

tri-color LED. To control another, the participant had to import the abstract

extension and use its blocks with my help. He was eager to go through the

process because he had then a clear idea to bring into realization. His scripts to

control the LEDs had a combination of blocks coming from the personalized and

abstract extensions, and looked like the script in figure 4-7. His comment on the

experience was that the abstract blocks revealed the actual functionality and

provided higher flexibility in using the board, but it was better to use the

modified extension to get him started.

59

c nnect force sensor to pin Ao

force en

Oeotetat sd robotPl y s udset RGB LED to red

analog write pin value

analog write pin value

analog write pin h value s

set RGB LED to green

analog write pin value b

analog write pin avalue

analog write pin a value

Figure 4-7: A script using blocks from the original and personalized A4S extensions

S One of the main challenges that face the creator of a Personalized Extension is

choosing his kit contents and extension blocks carefully. O ffering few options can

be over-constraining; offering too many can be overwhelming. In my case, the kit

that was used with the Personalized Extension had five sensors (light,

temperature, force, potentiometer, and push button) and two kinds of LEDs.

Some participants said it was a bit overwhelming in the beginning to have so

many sensors available.

4.3.3 On Setup and Support Materials

Providing a good tool is not enough. Creating good support materials and guidelines for

its use is essential to have a good learning experience.

* Many educators expressed that the handouts that were provided served well in

leading the participants as they were building their first circuits. The handouts

60

themselves evolved over time to address issues raised or experienced by the

participants. For example, actual images of the sensors with polarity information

were added because some participants found difficulty in recognizing the

drawings that were on the layouts.

* Most of the trouble the participants faced was due to difficulties in troubleshooting

and debugging the hardware problems. Some participants expressed the need to

have a checklist of techniques to be tried when something goes wrong with the

hardware part of their projects.

* Some high school and college participants asked for more examples and

applications that show various ways of using a specific sensor.

* The participants in the third workshop (college students) had the chance to use

craft materials in their projects. They liked that, and it was reflected in the styles

and diversity of the projects they came up with.17 LEGO bricks and craft materials

were placed on the worktable they were sitting at. I learned my lesson the hard

way in the second workshop (high schoolers) where the participants were

surrounded by all kinds of craft materials on shelves and cabinets around the

room, but not in immediate reach of their hands on the table where they were

working. They ended up using none.

17 Check these videos: http://youtu.be/FroOEUSKpdU, http://youtu.be/KTHd08-sTOk, andhttp://youtu.be/Wa426j8DIv4.

61

Looking Back, Looking Ahead

When I started thinking about the topic of this thesis, there were several themes I was

interested in exploring with regard to connecting Scratch to the physical world: I wanted

to learn how people with different backgrounds could design and create their own

extensions, how they would use these extensions in projects that are personally

meaningful, how the creators and users of the extensions would share their creations

and experiences, and how they could be supported through these experiences of

creating, sharing, and building with these extensions.

Obviously, I could only explore a small set of these aspects through this work. In this

chapter, I reflect on the lessons I learned and the insights I gained as well as future

directions in design and research.

5.1 Personalizing an Extension, Building a Community

In the hypothetical use case in Chapter 3, the high school teacher, Nada, created her own

kit (a variety of sensors connected to an Arduino board), and shared it with her students

to learn about some environmental issues. She turned the generic board (with its

63

abstract extension blocks) into a toolkit that can be used with blocks that give a concrete

sense of they can do ("turn LED on" vs. "digital write pin").

Afterwards, Nada could share her experience and her students' (i.e. the Personalized

Extension, guides to assemble the Arduino kit, projects created by the students, etc.)

with other educators who are interested in this area, and who in turn can use and remix

the Personalized Extension and the materials related to it.

Communities around various aspects of the Personalized Extensions experience are

expected to emerge. The creators of the Personalized Extensions can share their

creations on ScratchEd for example, which is an online community where Scratch

educators share stories, exchange resources, ask questions, find people, and discover

events. 18 ScratchEd can be used to share the Personalized Extensions and the

specifications of the hardware kits associated with them. Extension creators can also use

the platform to share their reflections on design practices and experiences, and get

feedback from other designers or users of the extensions. Other educators or hobbyists

may introduce changes to existing Personalized Extensions to improve the user

experience or add new functionalities (figure 5-1).

The educators using the Personalized Extensions can use the same platform, ScratchEd,

to share their stories and experiences with the extensions. They can also share new

materials, guides, and lesson plans they develop to be used with them. They may also

add their requests asking for new Personalized Extensions (or improvements to existing

ones) that support more learning experiences or explorations in the physical world.

18 ScratchEd (http://scratch-ed.org/)

64

COnneCt fu- sensor to AO

connect button to pin 7

connect button to pin 7 +"

setn pn as on a nu

digital write pin 13 high

dagnal wrie pin vau 25

analog read pin

Figure 5-1: Creators of Personalized Extensions can build on abstract approved extensions or remixexisting Personalized Extensions

Young Scratchers, who might not be familiar with ScratchEd, should be able to share

their projects that use Personalized Extensions on Scratch website (because they are

basically custom Scratch blocks made with approved extensions, which are shareable on

Scratch website). Through the project page (figure 5-2), the creator and viewers of the

project can share their notes, comments and feedback on the projects, including the use

of the Personalized Extensions.

65

Figure 5-2: A Scratch project page

5.2 Levels of Participation

Before introducing the Personalized Extensions, there were only two levels of

participants in the Scratch extensions ecosystem: the few developers who could develop

extensions using JavaScript (to be curated by the MIT Scratch Team in Scratch Extension

Library), and Scratchers who used those extensions.

With the introduction of Personalized Extensions, a new layer of Scratchers is added in

between, which is the Scratchers who can use the standard extensions to create new

personalized extensions and share them with the rest of the community (figure 5-3).

This layer of participation is added by democratizing the process of creating extensions,

allowing everyone to extend the language to explore worlds in their own language to

meet their own needs and interests.

66

wwrrm

A..Ls W, *AS

.a as rr

~i~~w- macuapmmum

.a rwr~a~ vow

Create with Personalized Extensions(or approved Scratch extensions)

Many Scratchers

-Imported throughPersonalized Extensionfiles (or from ScratchExtension Library)

Create PersonalizeExtensions

dI

Some Scratchers

'Created using Scratch (usingstandard blocks or availableextensions)

*Shared through PersonalizedExtension files

Create StandardExtensions Small group of developers

-Created using JavaScript*Shared through Scratch Extension Library-Have to be approved and curated by the MITScratch Team

Figure 5-3: The three levels of participation with Personalized Extensions. Derived from a graphiccreated by Natalie Rusk of the MIT Scratch Team to represent three levels of participation with

extensions in the current version of Scratch (extension developers, extension testers, and extensionusers)

5.3 Guiding the Design of Personalized Extensions

The MIT Scratch Team puts lots of thought and effort into the design of the Scratch

programming environment and blocks to achieve several overarching goals that include:

helping children become more fluent and expressive with new technologies; helping

them explore important concepts through their expressive activities; and, most broadly,

helping them become better learners [30].

The MIT Scratch Team has the power to make design decisions that support these goals,

usually stressing the importance of having a "low floor" (easy to get started), a "high

ceiling" (opportunities to create increasingly complex projects over time), and "wide

walls" (supporting many different types of projects so people with many different

interests and learning styles can all become engaged) [31]. However, some of this power

is delegated to Scratchers as they get enabled to design and create Personalized

67

(

I

Extensions, and share them with the Scratch community; and with great power, comes

great responsibility.

As Scratchers create Personalized Extensions, they have to make decisions about the

functionality they want to reveal or hide, which affects what ideas can be explored with

the extension (and the associated kit). These design decisions affect how simple or

overwhelming the extension is to use, an aspect that usually contests with the

capabilities offered by the extension. The question of how to support the creators in

making good design decisions remains an open area for exploration.

5.4 Personalized Extensions as Scratch Libraries

While the Personalized Extensions were inspired by the need to make personalized

explorations of the physical world more accessible, they do not have to be used solely

for that. They do not even have to be modifications of approved extensions.

A Personalized Extensions can generally be seen as an encapsulation of complex

behaviors defined in Scratch. A Scratcher might build a gaming Personalized Extension,

which has for example blocks that implement trajectories among other things (figure 5-

4). Other Scratchers would see this extension as a gaming library and import it to use its

blocks in building their games without worrying about the mathematics behind them.

There would be others who would like to modify the implementation of those functions

to meet different needs, and they would be able to do that by opening the Personalized

Extension as a Scratch project instead of importing it.

68

Figure 5-4: The definition of "launch with velocity" custom block, which can be used to create aPersonalized Extension (Based on code from "Gift Toss" project by Paddle2See -

http://scratch.mit.edu/projects/2238694/)

5.5 Empowering Science Explorations with Computational Tools

After introducing Personal Extensions, a rich area to study is how teachers and students

would use this tool to support their science explorations. Realizing that they have a

computational tool to access the physical world, teachers are expected to start creating

their own science kits that can be connected to Scratch and meet their needs in the

classroom.

With Personalized Extensions, the students would be able to focus on using the science

kit in hand, which would be developed by their teacher, to explore various areas of

interests without being overwhelmed by learning how to use the kits beforehand.

It will also be interesting to see how the students will make use of having Scratch

connected to their physical-world explorations. Scratch was designed to be more

meaningful and more social than other programming environments. More meaningful

means allowing people to create projects that are personally meaningful to them

through supporting many different types of projects (stories, games, animations,

simulations) and making it easy for people to personalize their projects by importing

69

photos and music clips, recording voices, and creating graphics. More social means

building a community around Scratch where people can support, collaborate, and

critique one another and build on one another's work [32]. These two aspects of Scratch

combined with Personalized Extensions and the right hardware tools would allow for

multiple pathways and representations (which correspond to different styles, interests,

and backgrounds) of exploring the world around 19 and sharing these explorations with

diverse audiences around the globe.

19 An example of personally-meaningful projects to explore the physical world is provided by the end ofChapter 3.

70

Improvements to Scratch's"Make a Block"

The current interface for creating custom blocks in Scratch serves as a good basis to

build Personalized Extensions. However, a couple of features should be introduced to

make Personalized Extensions support blocks of different types. The first of them is

supporting the creation of reporters: blocks that return values (figures A-1 and A-2).

Scratch currently supports only the creation of command blocks.

71

touching ho wor ld

I've reached the ceiling! 2

Figure A-1: Examples of Scratch reporter blocks and how they can be used

Figure A-2: A mock-up of the definition of a reporter block

Another feature is defining pull-down menus to be used as block parameters (figure A-3

and A-4). Currently, custom blocks can have numbers or strings as arguments.

Restricting the value of an argument to the elements of a menu can simplify using the

new block and eliminate the need to deal with unexpected values.

72

I

New Block

New Block

s Options

Add number input:

Add string input:

Add boolean input:

Add label text:

1 Run without screen refresh

text

OK Cancel

(a)

. Options

Add number input:

Add string input:

Add boolean input:

Add label text:

Add menu input:

W Reporter blockI Run without screen refresh

OK Cancel

(b)

Figure A-3: (a) The current window for creating a new custom block (b) A mock-up of the window forcreating new custom blocks that support reporter blocks and using pull-down menus as parameters

New Block

0

v Options

Add n :

Add string input:

Add boolean input:

Add label text: text

Add menu input:

* Reporter blockO Run without screen refresh

OK Cancel

Figure A-4: A mock-up of defining a pull-down menu for a new custom block

73

text

analog write pin valueanalog write pin Wvalue

Cooanalog write pin value

(a) (b)

Figure A-5: (a) A mock-up a script defining a new block using a pull-down menu as a parameter (b)the resulting new custom block

II

74

analog write pin m value IManalog write pin W value 40

red '-.

Workshop Materials

I list here the tools and materials that were used in the workshops described in Chapter

4.

The participants used the Scratch 2.0 offline editor 20 to work with abstract and

personalized Scratch extensions for Arduino boards. The abstract extension was A4S

(Arduino for Scratch), a preliminary HTTP extension for talking to Arduino boards

running Firmata that is being developed by David Mellis from the Lifelong Kindergarten

group. 21 The personalized extension was a modification of the A4S code (figure B-1). 22

The workshop participants had kits containing the following parts:

" Arduino Uno board

" Breadboard

- Jumper wires

20 See http://scratch.mit.edu/scratch2download/21 See https://github.com/damellis/A4S22 See http://bit.ly/A4Scratch

75

Resistor kit

Assortment of LEDs

Common cathode RGB LEDs

Mini photocell

Momentary Push Button Switch - 12mm Square

Trimpot 10K with Knob

TMP36 - Temperature Sensor

Force Sensitive Resistor

A4S (Scratch for Arduino) v

conctlgh eno t inA

cm

button pressed

turn LED 2 on

turn LED 2 off

turn on RGB LED

turn off RGB LED


set 2 Digital Input

digital write pin hI h

anlo 3rt 255DaueD

-0e m 1

(a) (b)

Figure B-1: (a) The original A4S extension blocks (b) The personalized A4S extension blocks

76

U'

digital read pie

analog read pi

The participants were provided with handouts (figures B-2 to B-9) that showed how to

connect different components to the Arduino board and the associated Scratch blocks.

The handouts were designed to be simple, created with software available for everyone,

so teachers knew they could easily build support materials if they were to produce their

own Personalized Extensions. The handouts were produced mainly using Google Docs

(https://docs.google.com), and Fritzing (http://fritzing.org) was used to draw the

breadboard view of the circuits.

77

t

ITi

.1-------------------------------

----

--it1 ii -

717

~1i

.- -

-.

--_-

-.

_

-._

.

-j

'. .

.l...

...

M

H4

0

1

-+

Q

II

.u

.

Figu

re B-2: A h

an

dou

t for connectin

g a tem

perature sen

sor usin

g the a

bstra

ct A4S exten

sion

78

Ib

t

0

if iii

1

JV.i

'I0

t..0

t

T

9a ,.F

igu

re B-3: A

han

dou

t for con

nectin

g a tem

pera

ture sen

sor u

sing a

perso

na

lized A

4S ex

tensio

n

79

4'!

0j" {J I

ja"

7

""

0 0 0 Oka

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

........

....

...............

44

-1-

o ..

_-I A

.._.....

IW

N

e F

a.,

fE-4.

S

FigureB-4: handot

forco

nnecin

g a

po

tento

mete u

singa p

eroaie

4

xeso

80

88U

iE

f

--~~

--

I

YTh:1II p

c'

s

lb.

Fig

ure B

-5: A ha

nd

ou

t for co

nnectin

g a

ph

oto

cell usin

g a perso

nalized

A4S extensio

n

V1

81

PT

IU,~

IAr

,00t

IH

y,

rN

III lI

7!CV

-

1~ K

sire

I05

O

COC

"

_1

v

d

a

00

-.w

3 .

Figu

re B-6: A h

an

dou

t for con

nectin

g a force sensitive resistor u

sing a person

alized A4S exten

sion

82

8

8

0

0I

1

i

iIFF

4~-_.

Q

1i

I

a

j

.I

I

I.

IlF

igu

re B-7: A

han

dou

t for co

nn

ecting a

pu

sh b

utto

n u

sing a

perso

na

lized A

4S ex

tensio

n

83

08

rI~L

I

Ia!'

"

............

t V i

~ }t~l:}Ej

--

.-

--

I

-1-

---

t7---I ~U

('A

An

illI -*aU

m

d0*10b

Figu

re B-8: A h

an

dou

t for conn

ecting an

LED u

sing a person

alized A4S exten

sion

84

1~~-

-H -

~

0ir

42"

E1L7*I----..--'--.--.

I t L iJ

02

.711% -~ ~K~<

) jill

1111 -.-..---

-- -

--4--

1~1

_

--V

.

wzzt

-~

_ 1 111121.1 zztzx

~

I ~

-

_ _Lit

III _

_ --

~-r-

*1

V..A

1......... ....

"...

iii

ii I Ike

vau

0a0.rI

U

I0I

Figu

re B-9: A h

andou

t for con

nectin

g a tri-color LED u

sing a person

alized A4S exten

sion

85

aI0M

a.

1R

1

VM

References

[1] Eisenberg, M. and Eisenberg, A. (2000). The Developing Scientist as Craftsperson.

In N. Roberts, W. Feurzeig, and B. Hunter, eds. Computer Modeling and Simulation

in Pre-College Science Education, NY: Springer-Verlag.

[2] Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan,

K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch:

Programming for All. Communications of the ACM, 52:60-67.

[3] Resnick, M., and Ocko, S. (1991). LEGO/Logo: Learning Through and About

Design. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing.

[4] Papert, S. (1980). Mindstorms: Children, Computers, and Powerful Ideas. Basic

Books, Inc..

[5] Resnick, M., and Ocko, S. (1991). LEGO/Logo: Learning Through and About

Design. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing.

[6] Lego Dacta. (1993) Control Lab Setup Guide and Introductory Explorations. Enfield,

CT: Lego Systems, Inc.

[7] Resnick, M., Martin, F., Sargent, R., and Silverman, B. (1996). Programmable

Bricks: Toys to Think With. IBM Systems Journal, vol. 35, no. 3-4, pp. 443-452.

[8] Resnick, M., Martin, F., Sargent, R., and Silverman, B. (1996). Programmable

Bricks: Toys to Think With. IBM Systems Journal, vol. 35, no. 3-4, pp. 443-452.

[9] Rusk, N., Resnick, M., Berg, R., & Pezalla-Granlund, M. (2008). New Pathways into

Robotics: Strategies for Broadening Participation. Journal of Science Education

and Technology, vol. 17, no. 1, pp. 59-69.

[10] Resnick, M., Maloney, J., Monroy-Hernindez, A., Rusk, N., Eastmond, E., Brennan,



87

T

[11] Blanton, A. (2012, February 21). Physical-Digital Chain Reaction using WeDo

Robotics with Scratch. Posted to:

http://scratched.gse.harvard.edu/resources/physical-digital-chain-reaction-

using-wedo-robotics-scratch

[12] Brennan, K. (2009, October 3). We Do WeDo. Posted to:

http://scratched.gse.harvard.edu/stories/we-do-wedo

[13] Scratch 1.4 Source Code. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki:

http://wiki.scratch.mit.edu/wiki/Scratch_1.4_Source_Code

[14] Stern, T. I. (2007). NetScratch: a Networked Programming Environment for

Children. MIT Masters Thesis

[15] Eastmond, E. (2006). New Tools to Enable Children to Manipulate Images through

Computer Programming. MIT Masters Thesis.

[16] LEGO WeDo Construction Set. (n.d.). Retrieved August 26, 2014 from the Scratch

Wiki: http://wiki.scratch.mit.edu/wiki/LEGO_WeDo_Construction_Set

[17] Jens. (2008, February 27). Chirp: A Community Edition Based On Scratch.

Message archived at:

http://scratchforums.blob8108.net/forums/viewtopic.php?id=3779

[18] Harvey, B. and M6nig, J. (2010). Bringing "No Ceiling" to Scratch: Can One

Language Serve Kids and Computer Scientists. Proceedings of Constructionism

2010. (2010).




[20] Maloney, J. (2013, September). Creating Scratch 2.0 Extensions [PDF document].

Retrieved from the Scratch Wiki:

http://wiki.scratch.mit.edu/w/images/ExtensionsDoc.HTTP-9-11.pdf

[21] Scratch Extensions. (2014). Retrieved August 26, 2014 from LLK GitHub

repository: http://lk.github.io/scratch-extension-docs/

88

[22] Scratch Extensions. (2014). Retrieved August 26, 2014 from LLK GitHub

repository: http://lk.github.io/scratch-extension-docs/

[23] LEGO WeDo Construction Set. (n.d.). Retrieved August 26, 2014 from the Scratch

Wiki: http://wiki.scratch.mit.edu/wiki/LEGO_WeDoConstructionSet

[24] Custom Blocks. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki:

http://wiki.scratch.mit.edu/wiki/CustomBlocks

[25] Resnick, M., and Silverman, B. (2005). Some Reflections on Designing

Construction Kits for Kids. Proceedings of Interaction Design and Children

Conference, Boulder, CO.

[26] Torrone, P. (2011, February 11). Why the Arduino Won and Why It's Here to Stay.

Retrieved from Make Magazine blog: http://makezine.com/2011/02/10/why-

the-arduino-won-and-why-its-here-to-stay/

[27] Turkle, S., & Papert, S. (1991). Epistemological Pluralism and the Revaluation of

the Concrete. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing.

[28] Resnick, M., and Rusk, N. (2009). Sensing 2.0: Integrating Sensor Data with

Interactive Media to Engage a Broader Audience in STEM Learning. Unpublished

project proposal.

[29] Mellis, D. A. (2014). A4S (Arduino For Scratch) [code repository]. Available from

https://github.com/damellis/A4S







[32] Resnick, M., Maloney, J., Monroy-Hernindez, A., Rusk, N., Eastmond, E., Brennan,



89

personalized extensions: democratizing the programming of virtual-physical interactions

Documents