The effectiveness of multimedia lexical

knowledge representation as a learning tool

Takeshi SATO (Tokyo Univ. of Agri. & Tech., Japan)

Tyler BURDEN (Meisei Univ., Japan)

Presentation for AsiaLex 2017 on 11th of June @ Guangdong University of Foreign Studies

Overview

1. Introduction

2. Research Questions

3. Research Procedures

4. Findings & Discussion

5. Conclusion

Is an electronic dictionary merely a reference tool?

Genius English-Japanese Dictionary (3rd version)

Advantage of computational knowledge representation

•Multimodality: the combination of text, images and sound• Longer knowledge retention (Lindstromberg & Boers 2008, Sato, Lai & Burden 2014, Yoshii & Flaiz, 2002 etc.)

•Better text comprehension (Sato, Matsunuma, & Suzuki 2013)

Advantage of online dictionaries

•On-screen presentation can make linguistic features of a lexical item salient. from a reference tool to a learning tool (Pachler, 2001)

Which lexical items to focus on?

Which linguistic features to make salient?

Which lexical items to focus on?

English Phrasal Verbs

Phrasal verbs (PVs)

• Multiword unit consisting of a verb and a preposition (adverb)

• Both words are polysemic.

• “[O]ne of the most challenging features of the English language” (Garnier & Schmitt 2016, p.30)

• Cannot acquire PVs by memorizing as an idiom (Lindstromberg, 2001b)

• Not only language teachers but also cognitive linguists are interested in PVs (ex. Dirven, 2001; Rice 2003; Rudzka-Ostyn, 2003)

Complicated semantic network

Take Over

(Norbig & Lakoff, 1987) (Tyler & Evans, 2003)

“Go off”

1. explode

2. start making a noise

3. electricity, etc.: not work

4. leave

5. happen

6. become angry

7. food/drink: not fresh

8. become worse

9. stop liking someone/something

10. begin sleeping

(from Macmillan Dictionary)

•Professor Buden went off to get a drink.

(concrete sense)

•Professor Burden went off Mr. Sato after an argument.

(figurative sense)

Drawbacks of a dictionary (Lindstromberg, 2001a)

1. jumbled or fragmented positioning of information

2. lack of attention to metaphor

3. lack of information about paradigmatic semantic contrasts

4. lack of pictorial illustration

5. flawed sense information

6. difficult language

7. omitted usages

Which linguistic features to make salient?

Schema & Prototype

Schema-extension model (Langacker, 1998, 2000, Tayler, 2002)

C. Schema

B. ExtensionA. Prototype

Semantic extension cannot be done without schematization

abstract commonality between A and B

schematization

Process to understand L2 PVs with schema

C. Schema

The fire alarm went off this morningThe meeting went off well.

Prof. Burden went off Mr. Sato after an argument .

Prof. Burden always goes off at a tangent

Prof. Burden went off to get a drink.

Prof. Burden went off with all the money.

Relates the senses with each other

Metaphorical comptence is crucialfor L2 learners (Boers, 2000, Littlemore, 2001)

Examples of schema

Dewell (1994)

Suggestions to improve the drawbacks (Lindstromberg, 2001)

• pictorial representation of meaning

• explicit division between literal and metaphorical uses

• clear division between a preposition and other related prepositions

• stop treating uses in order of frequency

• standardization of the organization of entries according to semantic views such as prominent highlighting of succinct rubrics.

Suggestions to make a dictionary a learning tool

•Representing schemata of the target words

•Showing a prototypical sense first

•Dividing literal and metaphorical senses

•Dividing clearly a preposition/verb and other related prepositions/verbs

2. Research Questions

1. Do schematic visual glosses enhance L2 text comprehension with L2 PVs compared with verbal glosses?

2. Which senses of L2 PVs do the schematic visual glosses enhance, concrete senses or figurative senses?

3. Research Procedures

1. 121 students randomly divided into two groups (handout/application)

2. Pre PV test (24 questions within 10 minutes)

3. Self-study with the treatments (5 minutes)

4. Post PV test (Same as the pretest)

5. Analysis (t-test)

Participants

• 121 Freshmen from two Japanese universities • Technology• Commerce

• Randomly divided into two groups

1. Handout (PVs, core sense, index of the meanings, example and its translation)

2. Application with schematic images of the core sense

Cited 8th of June fromhttps://goo.gl/1T4pJp

Target PVs

give off

give in

give out

give over

go in

go off

go out

go over

keep in

keep off

keep on

keep out

Application

Handout語句 基本イメージ 意味 例文 和訳

1 give offgive:与える＋off:～から

離れて

【他】(1)＜熱・におい・光・煙

＞を発する (2)＜木などが＞

（小枝などを）出す

I switched off the heater 20

minutes ago, but it is still

giving off heat.

20分前にヒーターを止

めたのだが、まだ熱が

出ている

2 give in give:与える＋in:～の中に

【自】＜…に＞屈服する・降参

する 【他】(1)＜書類・答案な

どを＞（…に）提出する・手渡

す[to] (2)（候補者などとして）

［…に］＜名前を＞届ける[to]

He gave in without a fight. 彼は戦う前に降参した.

3 give outgive:与える＋out:～の外

に

【他】(1)（…に）＜商品・用

紙・ビラなどを＞配る・配布す

る [to] (2) ＜…を＞発表する

【自】(1)＜供給物・力などが＞

尽きる・なくなる

There are people at Shinjuku

station who give out tissues.

新宿駅でティッシュを

配っている人がいる

4give

over

give:与える＋over:～を

超えて

【他】(1) ＜物・人など＞を

（…に）渡す・預ける・譲る、

（犯人などを）［警察などに］

引き渡す [to] 【自】（命令法

で）＜英口語＞やめる

She gave her all property

over to her daughter.

自分の全財産を娘に

譲った.

Self-study

After the instructor’s explanation of the core sense of each word, the participants were asked to learn the PVs by connecting the core senses with each sense

24 Questions (12 concrete & 12 figurative senses)

Analyses

•T-test between the two groups

•T-test between the pre and post tests in each group•Questions about concrete/figurative senses

Pre & post tests (overall)

6.928.92

7.56

9.33

0

2

4

6

8

10

12

14

16

18

20

pretest posttest

software

handout

T-test for the pretest

6.928.92

7.56

9.33

0

2

4

6

8

10

12

14

16

18

20

pretest posttest

software

handout

p = 0.21

T-test for the posttest

6.928.92

7.56

9.33

0

2

4

6

8

10

12

14

16

18

20

pretest posttest

application

handout

p = 0.21

p = 0.46

T-test results for the handout group

4.09

2.83

5.13

3.34

0

1

2

3

4

5

6

handout-concrete handout-figurative

pretest

posttest

p = 0.10

p = 0.00

T-test results for the app group

4.52

3.04

5.59

3.91

0

1

2

3

4

5

6

application-concrete application-figurative

pretest

posttest

p = 0.01

p = 0.00

Answers of our RQs

• RQ1: Do schematic visual glosses enhance L2 text comprehension with L2 PVs compared with verbal glosses?No significant difference between the

treatments• RQ2: Which senses of L2 PVs do the schematic

visual glosses enhance, concrete senses or figurative senses?Both aids enhanced the concrete senses

of the PVsSchematic aids enhanced their figurative

senses

Conclusion and limitations

•Traditional and multimodal PV representation facilitated its comprehension. •Multimodal PV representation facilitated improved comprehension of the figurative senses of the PVs compared to the traditional one. •Large-scaled research required.

References

• Boers, F., & Lindstromberg, S. (2008). How cognitive linguistics can foster effective vocabulary teaching. In F. Boers, & S. Lindstromberg (Eds.), Cognitive linguistic approaches to teaching vocabulary and phraseology (pp.1-64). Berlin: Mouton de Gruyter.

• Dewell, R. B. (1994). Over again: Image-schema transformations in semantic analysis. Cognitive Linguistics, 5(4), 351-380.

• Garnier, M. & Schmitt, N. (2016). Picking up polysemous phrasal verbs: How many do learners know and what facilitates this knowledge?, System, 59, 29-44.

• Lakoff, G.(1987) Woman, fire and dangerous thing. Chicago: Chicago University Press.

• Langacker, R, W.(1987) Foundations of Cognitive Grammar, Volume I, Theoretical Prerequisites. Stanford, California: Stanford University Press.

• Lindstormberg, S. (2001a). Preposition entries in UK monolingual learner’s dictionaries: Problems and possible solutions. Applied Linguistics 22(1), 79-103.

• Lindstromberg, S. (2001b). (Sometimes) Against the grain. Humanising Language Teaching Magazine, 3(3). Retrieved 12th of November, 2016 from http://www.hltmag.co.uk/may01/lind.htm

• NORVIG P., & LAKOFF G. (1987). Taking: A study in lexical network theory, A. Jon, B. Natasha, M. Laura A. & F. Hana (Eds), Proceedings of the Thirteenth Annual Meeting of the Berkeley Linguistics Society (pp. 195-206).

• Rudzka-Ostyn, B. (2003). Word power phrasal verbs and compounds: A cognitive approach. Berlin, Germany: Walter de Gruyter.

• Sato, T., Lai, Y., & Burden, T. (2014). Examining the Impact of Individual Differences of Information Processing Styles in Technology-Enhanced Second Vocabulary Learning. Proceedings of CLaSIC 2014. p. 432-440.

• Yoshii,M., & Fraitz, J.(2002). Second Language Incidental Vocabulary Retention: The Effect of Text and Picture Annotation Types. CALICO Journal, 20 (1), 33-58.

• Yeh, Y., & Wang, C. (2003). Effects of Multimedia Vocabulary Annotations and learning styles on vocabulary learning. CALICO Journal, 21(1). 131-144.

This work is supported by JSPS KAKENHI Grant Number 26370658

ご清聴ありがとうございましたThank you for listening

Takeshi SATO ( tsato@cc.tuat.ac.jp )

Tyler BURDEN ( tylerbuden1@gmail.com )