Pictures Meet Words: Learners of English Describing Motion Situations ^†

Abstract

This paper analyses the choices of English learners describing the opening scene of Mayer’s (1969) Frog, where are you? which depicts a frog escaping from a jar. A number of results were later tested using drawings that portray a person climbing. Speaking multiple languages might allow adjustment to ways of thinking, when formulating utterances and pointing to different details when describing pictures (Thinking for Speaking Hypothesis, Slobin 1996). The present paper contends that fewer contexts evoke mental images of climbing for speakers of Italian and Ladin who are learning English than for German-speaking English-learners, due to different cross-linguistic influences.

1. Introduction and Theoretical Framework

This paper presents an analysis of how multilingual English-learners who speak different first languages (L1) interpret and describe the image of the frog’s escape from the jar, in Frog, where are you? [1]. Subsequently, the choices made by learners, in describing a series of climbing pictures, are examined and interpreted. It is argued that learning different languages implies that ways of thinking are adjusted when formulating utterances and referring to different details to describe pictures.

The analysis is carried out with reference to the Thinking for Speaking Hypothesis [2,3,4], according to which, events depicted in picture books are understood differently by speakers of various languages—in the process of constructing a verbalised story out of them [2].

Descriptions and narrative tasks based on pictures are typical forms of linguistic data-elicitation in applied linguistics and language-acquisition research. As emphasised by [5] (p. 171), “picture stories are effective for assessing various linguistic features, including lexical variety, syntactic complexity, expression of space and motion or participants’ intentional states”. Initially inspired by [6,7], many researchers have employed the wordless picture book Frog, where are you? [1] to elicit narratives from participants of various ages learning different first (L1) and further languages (Ln). This picture story has proved useful in analysing the acquisition and use of motion verbs, expressing movement from one place to another, across typologically similar or dissimilar languages [4,8,9].

The so-called Frog Story [1] is about a journey and a search. The pictures represent a number of dynamic interactions between a boy and various animals in different physical settings. The child and his dog awake in the morning to find the pet frog has escaped from the glass jar in which it was kept. The boy and his friend, the dog, undertake a quest that takes them into the forest, where they make a number of intimidating encounters, before finding the lost frog in sweet company (or another one to take its place) [6].

The descriptions of various Frog Story images have been analysed by researchers who have sometimes concentrated on the specific events depicted in them [10,11]. The escape of the frog from the glass jar, at the onset of the plot, is a narratively significant event. The creature’s exit from the glass container can be interpreted and described in various ways, which include jumping, crawling, and climbing out. In the description of a motion scene, narrators can describe both the manner of movement and the route followed by the subject. As illustrated by [12], two path segments can be identified as the frog leaves the jar: one that can be labelled frog-leaves-jar (e.g., climbs out of the jar) and one that can be labelled frog-departs scene (e.g., leaps through the window).

Verbs of motion have been classified as manner verbs and as path verbs. The stem of manner verbs includes information about the way in which the figure physically moves (e.g., run, jump, climb). Path verbs (e.g., enter, get, come, go) and their adnominal and adverbial encodings (e.g., into, out of, away) describe the trajectory the figure follows. (A number of verbs (e.g. escape) have been argued to be hybrid or mixed, since they appear to denote both path (e.g. away from a ground) and manner (e.g. motion that is quick and stealthy) [9]) It has been confirmed by some researchers that manner of motion is mentioned more frequently in some languages than in others [13]. While English and German are considered high-manner-salient languages, Italian and Ladin are normally viewed as low-manner-salient languages, since manner is characteristically expressed outside of the main verbs and is often omitted, e.g., English He ran into the house vs. Italian Entrò (di corsa) ‘He entered by running’ [11,13].

In interpreting and describing a figure’s movement, learners can adopt a deictic perspective. The moving entity can be viewed as approaching or receding from the fons et origo or deictic centre of orientation, which can be the story’s protagonist or narrator. Learners who choose the verb come to lexicalise a frog’s exit from a jar, in a story, probably see the animal as somehow moving towards themselves or the story’s protagonist. (While some studies separate the high-frequency verb types come and go from the path verbs [8,14,15], others include them [9,16], like the present study.)

Narrators need to interpret lines and curves as objects (e.g., a jar), animate beings (e.g., a frog) and relations (e.g., frog in jar). Objects and animate beings are subsequently interpreted as involved in situations (e.g., moving somewhere). After several perceptions and interpretations, narrators finally need to find adequate linguistic forms to communicate the complex ideas in the portrayed events [7]. Learners of further languages have to choose among expressions and structures they have available at their linguistic developmental stage and their choices will often be variously influenced by their previously learnt languages. A growing number of studies suggest that different language speakers make a variety of choices when interpreting and verbalising depicted events [17].

An image featuring a person clinging onto a rocky mountain with a helmet and a safety harness fastened around his waist will probably elicit the English verb climb in most learners. On the other hand, images that clearly depict an individual climbing into bed, out of a cot, through a window, or down a tree, will probably prompt the use of the verb climb in fewer learners. The verb climb is officially learnt by children at Movers level (beginner level A1 on the Common European Framework), but learners normally acquire the unmarked (standard) meaning first, whereas other form-meaning connections are established later. As explained by [18] (p. 5), a form-meaning connection “is initially made when a learner somehow cognitively registers a form, a meaning, and the fact that the form encodes that meaning in some way”. Ln learners frequently map Ln words to L1 meaning or lexical semantics in another acquired language, and it is difficult for new meanings to get into Ln lexical entries [19]. If an L1 form encodes fewer meanings than the equivalent Ln form, it might be hard for the Ln learner to recognise and acquire the multiple meanings in the new language.

The Ladin se arampiché and the Italian arrampicarsi, ‘climb’ are used in fewer contexts than the English climb: they normally denote upward movement and are usually connected with mountains, hills, trees, and walls. Ladin and Italian English-learners will, therefore, hardly view a frog struggling to get out of a jar as ‘climbing’. Various forms of klettern ‘climb’ in German indicate upward movement (e.g., hochklettern, heraufklettern, hinaufklettern), downward movement (e.g., herabklettern, herunterklettern, hinunterklettern), movement over something (e.g., über den Zaun klettern ‘to climb over the fence’), and effortful movement into and out of something (e.g., hineinklettern, hinausklettern). Klettern is therefore also used to describe boundary-crossing events. Moreover, the German steigen is frequently used as an equivalent of the English climb to indicate movement in various directions in various contexts. German speakers might therefore associate a frog’s movement out of a jar more readily with climbing than Ladin and Italian speakers.

The study participants are presented in Section 2. The analysis procedure and the research questions are specified in Section 3, while the main results are presented and discussed in Section 4. The paper concludes with Section 5, which underlines the main points that have been raised.

2. Participants

The data was collected in schools and at a university faculty in the northern Italian province of Bozen-Bolzano, which recognises three official languages: German (69.41%), Italian (26.06%), and Ladin (4.53%) [20]. As shown below, in

Table 1. Participants who provided the frog-story narratives.

Group	Participants	L1	English	Education
1	73	Ladin	L4	LL 1
2	43	Ladin	L4	LU 2
3	34	German	L3	GU 3/University
4	21	Italian	L2/L3	University

¹ Ladin lower secondary school; ² Ladin upper secondary school; ³ German upper secondary school.

, the first set of data was obtained from the following four groups of English-learners:

• Pupils in their 8th school year who speak Ladin, Italian, and German. They were learning English (L4) for the 5th year.
• 13th graders who speak Ladin, Italian, and German. They were learning English (L4) for the 8th year.
• 13th graders and university undergraduates who speak German (dialect and standard) and learnt Italian (L2) and English (L3).
• University undergraduates who speak Italian and learnt German and English (L2/L3).

It was estimated that the level of English of the youngest group of learners (group 1) was A2 on the Common European Framework of Reference for Languages, whereas it was estimated to range between B1 and B2 in the other groups (2, 3, and 4). (At the Ladin lower (L_L) and upper (L_U) secondary schools, 59% and 44% of the participants were male, whereas almost all the German and Italian participants were female. No significant gender differences could be detected in the results obtained from the Ladin speakers. Due to the vast majority of female participants in the other groups, the gender issue was not pursued further in the analysis reported here.)

Table 2. Participants who described the climb-drawings.

Group	Participants	L1	English	Education
1	13	Ladin	L4	University
2	40	German	L3	University
3	40	Italian	L2/L3	University

below illustrates that the second set of data was provided by three groups of undergraduates at university, who spoke different languages: (i) Ladin, Italian, German, and English; (ii) German (dialect and standard), Italian, and English; (iii) Italian, German, and English. The participants’ level of English proficiency was estimated to be around B2.

3. Analysis Procedure and Research Questions

The first data collection comprised the subjects’ task-based written stories, prompted by a shortened version of Frog, where are you? [1]. The learners were provided with the beginning of the story, and were asked to complete it with the aid of a sequence of photocopied black-and-white pictures. The analysis carried out for the present paper concentrated on the second picture of the book, which is provided below (Figure 1). While the first book picture had shown a bedroom in which a boy and a dog were observing a frog in a jar, the picture analysed in the present paper (Figure 1) depicts the boy and the dog asleep and the frog in the process of escaping its imprisonment (with one leg out of the jar).

At first, the choices made by the two groups of Ladin participants at both school levels (lower and upper secondary school), and with different proficiency levels, were compared to assess the degree of similarity. Subsequently, the Ladin, German, and Italian groups at a similar proficiency level of English were observed to identify similarities and discrepancies ascribable to their other languages.

In the light of results and impressions obtained from this first investigation, a second data collection was organised, using 12 drawings depicting an individual climbing in different environments (e.g., mountains, trees, walls, stairs, and ladders) and in different directions (e.g., upwards and downwards). See drawings 1–12 below.

More specifically, the following questions were addressed in the analysis:

• How do the learner groups interpret and describe the frog’s ‘journey from the jar’ in the opening scene of the Frog Story [1]?
• Is a deictic perspective detectable in the description of the opening frog-story scene?
• Does it follow, on the basis of the present analysis, that different L1s in a multilingual context might arrive at diverse picture interpretations, along with the Thinking for Speaking Hypothesis [2]?

4. Results and Discussion

4.1. The Frog’s Journey from the Jar

Most of the younger Ladin participants, 81%, and 95% of the older Ladin participants explicitly mentioned the frog and the jar explicitly in their descriptions of the amphibian’s exit from the glass container.

The lower percentage among the younger learners was partly due to non-target like constructions with verb-less prepositional phrases that focused on the locations of the boy and the dog (i.e., in bed, in their room). It seems that several of younger participants had difficulties switching their attention from the boy and the dog to the frog within the same sentence.

The verb-phrases reported in

Table 3. Verb-phrases in the frog-leaves-jar scene at the Ladin lower (L_L) and upper (L_U) secondary schools.

Frog + Verb-Phrase + Jar-Noun-Phrase	LL School (59 Learners)	LU School (41 Learners)
come out of/from	34%	10%
jump out of	32%	24%
go out of/into	22%	7%
leave	5%	5%
climb out of	2%	13%
get out of	2%	-
fall out	2%	-
run out at	2%	-
escape from/out of	-	34%
sneak out of	-	2%
slip out of	-	2%
drop out from	-	2%
TOTAL	100% 1	100%

¹ All percentage figures are rounded to whole numbers.

were used by the Ladin participants at the two school levels, when the frog and the jar were mentioned explicitly. It can be noticed that the favourite verb-phrases among the younger subjects were come out of/come out from, with 34% of the participants opting for these expressions. This means that 34% of the pupils at lower secondary school concentrated on the frog’s route from inside to outside of the jar, and disregarded the manner of its movement. A similarly high number of pupils in this group, namely 32%, opted for the motion verb of manner jump, viewing the frog not so much with one leg in the jar anymore, but already as moving quickly from the glass by pushing itself with its legs and feet.

The verb-phrase jump out of was also a favourite at the upper secondary school, where 24% of the participants chose it. However, the verb-phrases escape from/escape out of accounted for the highest percentage in this group, namely 34%, whereas this verb was not used by the younger participants, despite the formal similarity with Ladin sciampè and Italian scappare, from Vulgar Latin *excappare, literally ‘get out of one’s cape’.

A similar developmental trend was noticed by [6], who observed that English adults often used the Latinate form escape, while children did not. However, [12] reports that most adults (33%) used escape to describe the second path segment (the frog-departs scene, e.g., escaped through the window), while fewer adults (25%) employed it to describe the first path segment (the frog-leaves-jar scene, e.g., escaped from the jar). By contrast, in our data, escape was used by fifteen Ladin learners at upper secondary school, fourteen of whom used it in the frog-leaves-jar scene.

Examples 1 and 2 illustrate two typical descriptions of Ladin learners at lower secondary school, while example 3 shows how a Ladin learner at upper secondary school described the scene.

• One night, while he and his dog were sleeping, the frog comes out of the jar. (L_L).
• One night, while he and his dog were sleeping, the frog jumped out of the jar. (L_L).
• One night, while he and his dog were sleeping, the frog escaped from the jar. (L_U).

Table 4. Verb-phrases in the frog-leaves-jar scene, in the Ladin upper secondary school (L_U), German, and Italian groups.

Frog + Verb-Phrase + Jar-Noun-Phrase	Ladin LU (41 Learners)	German (33 Learners)	Italian (21 Learners)
escape from/out of	34%	21%	-
jump out of/out/from/outside	24%	36%	33%
climb out of/from	13%	18%	-
come out of/from	10%	3%	5%
go out of/from	7%	9%	24%
leave	5%	6%	-
sneak out of	2%	-	-
slip out of	2%	-	-
drop out from	2%	-	-
run outside	-	-	5%
walk out of	-	3%	-
fly by the open top of	-	3%	-
get out of/from	-	-	29%
open	-	-	5%
TOTAL	100%	100%	100%

, is a comparison of the Ladin, German, and Italian groups at approximately the same proficiency level revealing that the Ladin and German groups preferred three motion verbs to describe the manner of movement, namely escape (secretly), jump (quickly), and climb (with effort). In contrast, the Italians preferred one manner verb, jump, and two path verbs that encoded nothing about manner, get and go. This might be an influence from Italian L1, which mentions manner less frequently than German and English [13].

In

Table 5. Group comparison of segment-coding for the frog’s ‘journey from the jar’.

Path Segment	Ladin LL 1 (73)	Ladin LU 2 (43)	German (34)	Italian (21)
frog leaves jar	64%	74%	65%	67%
frog leaves jar + departs	18%	21%	35%	33%
frog departs	11%	5%	/	/
frog enters jar	1%	/	/	/
other	5%	/	/	/
TOTAL	100%	100%	100%	100%

¹ Ladin lower secondary school; ² Ladin upper secondary school.

, a comparison of segment-coding for the frog’s ‘journey from the jar’ revealed no striking differences between the four multilingual learner groups. By contrast, [12] found a clear difference between speakers of typologically distant languages, some of whom concentrated on the frog-leaves jar scene, while others focused on the frog-departs scene. In our data, most Ladin speakers and all German and Italian speakers focused on the frog’s exit from the jar. The group that elaborated the most on the frog’s journey, by mentioning both its exit from the container and its departure, was the German group, when 35% of its speakers encoded both path segments. The percentage difference between the German and Italian speakers is too small (35% vs. 33%) to allow generalisations, but it was expected that the German speakers would add more path elements than the Italian speakers. [22] argued that the number of path elements per trajectory is longer in some languages (English and German) than in others (Italian). Unlike the German and Italian speakers, a small number of Ladin speakers concentrated on the frog-departs scene. This was the case especially, among the younger Ladin speakers (see Table 5).

Example 4 shows how a Ladin upper secondary school speaker focused on the frog-leaves-jar scene, while example 5 illustrates how a German speaker attended to both path segments; the frog’s exit and its departure. The clause in example 6 shows that the Ladin learner at the lower secondary school did not pay explicit attention to the source (the jar) but was already projected towards a place some distance away.

4.

One night, while he and his dog were sleeping, his pet frog jumped out of his jar. The next morning, when the sun rose, the boy couldn’t find the frog.

5.

One night, while he and his dog were sleeping, the frog jumped out of the large jar and escaped through the window.

6.

One night, while he and his dog were sleeping, the frog ran away.

4.2. Deictic Perspective

The frog was seen by several narrators to be approaching them or the protagonist, or to be distancing itself.

Table 6. Total deictic perspectives (come vs. go) in the frog-leaves-jar scene.

Types	Total of 153 Participants
come out of/from	17%
go out of/from	15%

illustrates, the verb-phrases come out of/come out from, and go out of/go out from were chosen by almost the same number of learners, which shows how subjective the interpretation of directionality can be, in a depicted scene.

Examples 7 and 8 show how two learners interpreted the frog-leaves-jar scene from two different deictic perspectives.

7.

One night, while he and his dog were sleeping, the frog comes out of the glass. (Ladin, L_L).

8.

One night, while he and his dog were sleeping, the frog went out from the glass. (Italian).

As illustrated in

Table 7. Deictic perspective (come vs. go) in the frog-leaves-jar scene.

Frog + Come/Go + Jar-Noun-Phrase	LL 1 (58) 2	LU 3 (41)	German (33)	Italian (21)
come out of/from	34%	10%	3%	5%
go out of/from	21%	7%	9%	24%

¹ Ladin lower secondary school; ² One Ladin L_L learner viewed the frog as entering the jar, but the phrase go into has not been counted here; ³ Ladin upper secondary school.

, a group comparison of deictic perspectives revealed that come and go were used especially by the younger participants, who probably used these common verbs in part to fill their lexical gaps. In both Ladin groups, come was employed by more narrators than go, unlike in the German and Italian groups, where the reverse is true. The Ladin language frequently describes an exit from something, with the verb-phrase gnì fora ‘come out’, which has no synthetic alternative like the Italian uscire ‘exit’. Ladin learners of English therefore seem to be prompted by their L1 to use the form come out of to describe an exit. In contrast, a higher number of Italian speakers interpreted uscire as ‘going out’ rather than as ‘coming out’. The lower percentages in the German group might display stronger focus on manner than simply on direction.

4.3. Climb-Scenes

When analysing the descriptions of the frog’s exit from the jar, it was noticed that the highly-specific verb climb was seldom used, contrary to expectations. The verb climb was expected at least in the older learner groups, as it seems to be a basic verb often used by adults in descriptions of the frog-leaves-jar scene [23]. In Section 1 it is suggested that the German speakers would associate the frog’s exit from the jar more readily with the act of climbing than the Ladin and Italian speakers, due to a different cross-linguistic influence. As

Table 8. Group comparison of the use of climb out of and climb out from in the frog-leaves-jar scene.

	LL 1 (59)	LU 2 (41)	German (33)	Italian (21)
climb out of and climb out from	2%	13%	18%	-

¹ Ladin lower secondary school; ² Ladin upper secondary school.

shows, this hypothesis is supported, since the analysis revealed that climb was used most often by the German group, whereas no Italian learner chose it.

To further corroborate the hypothesis that Italian speakers associate fewer situations with the act of climbing, thirteen Ladin, forty German, and forty Italian speakers were asked to describe twelve drawings that showed a person ‘climbing’ in different directions and contexts (see Section 2 and Section 3).

The German group used the verb climb more often than the Ladin and Italian groups, as shown in

Table 9. The most frequent verbs in the description of the twelve drawings in the learner groups.

Group	Total of Drawings Described	Climb	Go	Get	Jump	-Pass
Ladin	156 1	43%	34%	2%	4%	1%
German	480	58%	27%	2%	2%	2%
Italian	480	31%	41%	7%	4%	2%

¹ 12 drawings × 13 participants = 156 (100%); 12 drawings × 40 participants = 480 (100%).

below. Climb represented 58% and 43% of the total number of verb tokens in the German and Ladin groups, where it was the preferred option, whereas the Italian group favoured the verb go. Table 9 shows that the verb get represented 7% of the total number of verb tokens in the Italian group, but that no other verb reached 5% in the three groups. (Verbs that occurred less than 2% of the time have been omitted.) The high percentage of climb in the German group confirmed again that German speakers tend to focus more on the manner of movement than Ladin and Italian speakers.

Table 10. Percentage of learners using climb in each drawing.

Drawings	Ladin (13)	German (40)	Italian (40)
1 (mountain, up)	100%	100%	97%
2 (tree, up)	85%	90%	85%
3 (wall, up)	92%	95%	70%
4 (roof, up)	77%	82%	45%
5 (wall, over)	31%	57%	10%
6 (window)	23%	55%	5%
7 (ladder, up)	31%	70%	40%
8 (stairs, up)	8%	10%	2%
9 (bed, into)	8%	5%	2%
10 (cot, out)	23%	67%	5%
11 (tree, down)	23%	50%	12%
12 (water, into)	15%	17%	-

shows that all the groups saw an act of climbing, especially in the drawings that depicted an individual ascending a mountain (drawing 1), a tree (drawing 2), a wall (drawing 3), and a roof (drawing 4). On the other hand, the percentage of participants using climb was lowest in the descriptions of the drawings that depicted a person moving up the stairs (drawing 8), into bed (drawing 9), and down into the water (drawing 12). As expected, no Italian speakers saw an act of climbing in drawing 12.

It can be noticed in

Table 11. Group comparison of the climb percentage in drawings 1, 2, 3, and 4.

	Ladin Group (67) 1	German Group (280)	Italian Group (150)
Drawings 1–4	68%	53%	80%

¹ Total number of climb-instances in the description of the twelve drawings.

that the concentration of the climb-instances in the descriptions of the first 4 drawings was particularly high in the Italian group, where 80% of the total number of climb-tokens occurred in these initial 4 scenes. Table 10 shows that the percentage of the Italian speakers using climb already drops below 50% in the description of drawing 4 (up the roof). In the Ladin group, the percentage falls considerably in drawing 5 (over the wall). In contrast, the percentage of German speakers using climb is above 50% in the descriptions of 9 drawings out of 12. This result therefore seems to confirm the hypothesis that more contexts evoke a mental image of climbing in German than in Italian and in Ladin learners of English.

5. Conclusions

As emphasised by [6] (p. 613), “the picture-story method makes it possible to compare the ways in which the identical picture is described by speakers of different languages”. The present paper analyses learners’ narrations of the frog’s ‘journey from the jar’ in the picture book Frog, where are you? [1] and it subsequently examines descriptions of twelve drawings that portray a person climbing in different environments [21]. In line with the Thinking for Speaking Hypothesis [2], it is assumed that speakers of different languages pay different kinds of attention to depicted events when writing about them and, consequently, make different lexical choices, influenced by their previously learnt languages. English was learnt by the research participants mainly as an L3 and L4, so that their lexical choices in English might be influenced by more than one previously learnt language. However, differences in word selections were detected among the groups of multilingual learners with different L1s: Ladin, German, and Italian:

• In the narration of the frog’s ‘journey from the jar’, the older Ladin and German speakers seemed to attend to the frog’s manner of movement more than the younger Ladin participants and the Italian speakers, who focused more on the trajectory. The older Ladin and German speakers preferred three manner verbs (escape, jump, and climb), whereas the younger Ladin participants favoured two path verbs (come and go) and one manner verb (jump), and the Italian participants one manner verb (jump) and two path verbs (get and go).
• In general, the deictic verbs come and go were chosen by approximately equal numbers of participants. A group comparison revealed that come out of and come out from were preferred by the Ladin speakers, while German and Italian speakers favoured go out of and go out from. Hence, the Ladin speakers appeared to view the frog exiting from the jar as approaching the deictic centre, whereas the German and Italian participants saw it mainly to be distancing itself.
• In the description of the twelve climb-drawings, climb was the preferred option of the German and Ladin groups, whereas the Italian group favoured the verb go.

The frequent use of go and get by the Italian speakers in both tasks could be due to an influence from their L1. Drawings intended to depict movement from one place to another might evoke a mental image of path rather than of manner in Italians, whose L1 tends to express manner less frequently than German and English [13]. The fact that the Italian arrampicarsi fulfils fewer functions than the English climb, might have influenced the Italian group of participants in their interpretation and description of the twelve drawings. As was anticipated, some learners did not use the verb climb in their descriptions of certain drawings. This might be because characteristic features of the drawings did not suggest an image of climbing, which normally entails some physical exertion and perhaps evokes a grimace (see drawings in Section 3). However, it might also be the case that the learners had not yet understood the multiple meanings of climb in the English target language and, therefore, did not associate certain actions, such as ascending stairs or getting into bed, with climbing. In line with the Thinking for Speaking Hypothesis [2], learning and using further languages might involve adjusting ways of thinking when formulating utterances, and attending to different details when describing pictures, such as the path of in contrast to the manner of the depicted movement.