Multimodal literacy learning with

interactive technologies

Sue NicholsCentre for Research in

Education

Multimodality

The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning-making resources eg print language, sound & visual images.

Some resources used by text producers:

Multimodality

The term ‘multimodal’ is used to refer to a text constructed from different kinds of meaning-making resources eg print language, sound & visual images.

Some resources used by text producers:

Graphic symbolsSpoken language

Sound effectsMoving images

Hyper-linkingConcrete materials

Written language

Sally (aged 4) uses visual resources

CONTEXT: Sally had noticed that a researcher visiting her kindergarten was writing in a notebook. She asked to draw something in the notebook. A yoghurt container full of coloured feathers was on the table. Sally announced she would draw that and asked if the researcher would draw the container first. Then the following conversation happened …

Sally uses spoken language and social resources

Sally: Sally has green feathers and red feathers and purple feathers and yellow feathers and purple flowers. You can write it in your story now.

Sue: OK.

Sally: I’m going to draw a feather. I’ll do the purple one. You do the next one.

Sue: OK. I’ll do the green one.

Sally: You do it like that. These are the fluffy bits. You have to go down to here. You have to be careful with textas. You’re doing it nicely, aren’t you?

Sue: How’s that?

Sally: Good. I’ll do the next one.

Sally employs written language resources via an expert helper.

Here, the adult helper is the resource, or indeed the technology, which is utilised by Sally to produce the written part of her text.

Multimodality is not new

• However, digital technologies offer a different set of resources for producing and consuming multimodal texts.

• They can make it easier to work across modes and to include modes that have become neglected.

Digital word processing has been changing the nature and pace of children’s writing development

One of the children had actually sent e-mail messages in September with spaces between words, but did not begin to consistently add spaces in her handwriting for 18 months.

One child typed his name, “Jeffery”, highlighted it, selected the WingDing font type, and gleefully exclaimed, “I made my name in code.” The typed name “Jeffery” had become jefferry

Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours. IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta

Digital technologies can create different opportunities for children’s communication.

• New audiences• New forms of written communication • New kinds of collaboration• Students becoming ‘prosumers’

Collaborative online writing

In the United States, Common Core State Standards (CCSS) requires teachers to “engage students in collaboration and online publication of writing”.

Teachers Mary Blow and Ronica Lawrence, working in schools over 100k apart, decided to partner their classes in an online science research project.

http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writing-and-online-publishing accessed 22/8/12

Online collaboration using Knowledge Forum

Many children were motivated to make entries themselves using whatever strategies they had available, including use of invented spelling. Pelletier et al 2012

Children could see the photographs and the ideas posted by their peers. By clicking on a peer’s entry, text scaffolds in the database would appear that gave children choices; for example: “my theory is…” or “I want to know.”

Integrating reading & writing with digital devices

don’t worry Humphrey i have a terrible life with my sister

i wonder what garths house looked like

what does that line mene

that is werd

Sourced from Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The Reading Teacher 64(1) pp 15 – 22.

Amy ( aged 7) & Winnie (aged 8) made notes on their Kindles while reading Friendship According to Humphrey .

Amy made 43 notes and Winnie made 33.

www.curiouspuppy.com

Literacy Apps – Blending old and new

First Published 1979

This app is a version of a class children’s book. It is not unusual to see older children’s literature appearing in app form, as it is cheaper for producers than employing authors to create original content. Given what Susan Hill’s research is revealing about the popularity of mobile phones with very young children, this offers a way to engage them with quality children’s literature at little cost to parents.

• What has changed?• How many of these functions could be carried out without

digital technology? • There are significant continuities in so-called ‘New Literacy’

texts and practices.

Personal Examples

• E-book making in libraries

• Photo studio in classrooms

In the following slides I share some examples of multimodal texts produced by children in two projects I have been involved in.

E-book Making in Libraries• Sessions for children and carers

were run in school holiday programs at two metropolitan libraries.

• Separate sessions were run for younger (aged 3 – 6) and older (aged 7 – 10) children.

• Sessions were 90 minutes long and utilised the iPad app ‘My Story’.

• Sessions began with a ‘hands off’ pre-production phase.

PRE-PRODUCTION:1. Icon recognition (It is vital for

users to understand how to access functions like ‘undo.’)

2. Explanation of multiple modes & non-linear production (Users can commence creating their story with drawing, photography, writing or oral narration.)

3. Narrative planning ( A ‘hands-off’ time for children and their carers to decide on characters and story line. Deciding on a title is a good focusing move.)

PRODUCING THE E-BOOK

RESOURCES:• iPads• paper & textas • puppets & figurines• large pieces of

coloured cardboard as backdrops

• cubes for setting up scenes

PROCESS:• Most children began with the visual

element. Children chose to either draw or to stage and photograph scenes using props. Some combined both.

• The language part of the narrative was either dictated to an adult to type or was recorded as an oral narration by the child. Some children chose to type their own text.

• Narratives that had been well planned could be produced more efficiently since children generally moved quickly to the complication part of the story.

NARRATED BY CHILD: Once there was a princess

NARRATED BY CHILD: Once the princess met a dragon.

NARRATED BY CHILD: The dragon ate the princess.

NARRATED BY CHILD: He didn’t like her and he spat her out.

Created on an iPad by ‘Jack’, a Junior Primary child, using photography, drawing, handwritten text & text typed by an adult

but no oral narration.

Once there was a fire breathing dragon

One day the dragon found that he couldn’t breath fire any more.

So the dragon went to see the giraffe.

The giraffe waved his magic wand.

Suddenly the dragon could breathe fire again. The end.

Classroom Photo Studio: Year 1 Children making design choices

• For this project, children worked with digital cameras and PowerPoint to produce representations of significant personal objects.

• A photo booth was set up in the classroom with a digital camera, tripod, table and display board. Cardboard provided the backdrop.

PROCESS:• Children posed and photographed

their objects.

• The photographs were saved to the class computer. Children then took turns to dictate written text in one or more of three formats:– Speech bubble– Thought bubble– Caption

• All the slides were combined in a class presentation that was shown during the school’s Art Night.

Eagles

Aaaah!!

Eagles are scary.

DESIGN DECISIONS:

What object to show;

How to pose it;

Whether to add text;

What text to add;

Speech or thought bubble or caption;

Colour for text and fill.

My car knows everything about the world. My car has been all over the world

Wheeeeee!!!

Images of his toy shark taken by a 5 year old boy

Adult: Is this how you want it to look?

Child: Shakes head

Adult: How do you want it to look?

Child: Fierce.

Image selected by child.

Propping shark on plastic cup to show mouth.

Questions about any technology

• Is it flexible?• Can it be used in combination with existing

tools?• Does it foster the integration of skills?• Does it facilitate both scaffolded and

independent use by children?• Can it be used for text production as well as for

the consumption of texts produced by others? • Does it support different ways of learning and of

displaying learning?

REFERENCES• Larson, L. (2010) Digital Readers: The next chapter in e-book reading and response. The

Reading Teacher 64(1) pp 15 – 22.• Pelletier, J., Reeve, R., & Halewood, C. (2006). Young children's knowledge building and

literacy development through Knowledge Forum. Early Education and Development, 17(3), 323 - 346.

• Yost, N. (2003) Computers, Kids, and Crayons: A comparative study of emergent writing behaviours. IFIP Working Group 3.5 Conference: Young Children and Learning Technologies, held at UWS Parramatta

• http://www.scholastic.com/teachers/top-teaching/2012/01/collaborative-writing-and-online-publishing accessed 22/8/12

TO REFERENCE THIS PRESENTATION:

Nichols, S. (2013) Multimodal literacy learning with interactive technologies. Presentation for the Centre for Research in Education, University of South Australia, Adelaide, July 3rd 2013.