A "Scanpath" down Memory (for pronunciation) Lane: The Eyes have it!

"Eye catching" new study on the function of eye scanning movements in memory creation and access, Eye-movement replay supports episodic remembering, by Johansson, Nystrom, Dewhurst and Johansson in the recent Proceedings of the Royal Society of Biological Sciences. (Also summarized informally in Neurosciencenews.com). The general concept is that eye movement patterns ("scanpaths") that accompany creation of a memory are virtually the same as those used in recalling features of the memory later. 

From the abstract:

Our findings provide direct evidence that such scanpaths are replayed to assemble and reconstruct spatio-temporal relations as we remember and further suggest that distinct scanpath properties differentially contribute depending on the nature of the goal-relevant memory.

In other words, at some level the eyes, at least in episodic memory creation, as part of the overall visuo-spatial processing system of the brain, move in minute scan patterns of millisecond duration, which are not only accessible in recall of that event/episode, but are integral to it. How that relates to internal "events" inside the brain is not entirely clear, of course, but the research again confirms the correlation between path and event-in-time. As the research literature review in the research report relates, those eye movements and related phenomena have been the object of research and various therapeutic applications for over 70 years. 

And how does this relate to remembering or learning pronunciation? The key idea is what constitutes an "episode" in this context " . . . that the sequential replay of eye movements serves to facilitate pure episodic reconstruction in the absence of visual input." (from the article) According to that model, when a gesture is used, for example, associated with a new sound or perhaps an intonation contour, the two are "stored" together in a sense. Either may, in principle, then prompt recall of the other. If the pair are practiced together, the components of the two, the sound, itself, the kinesthetic "track" of the gesture, the "trail" in the visual field can be further joined and strengthened. 

That is, in effect, the basis of the KINETIK Method, as well having gotten its early inspiration from what is known as Observed Experiential Integration Therapy.  

From about 30 years of experience in working with gesture and sound change, the connection between what we call the "movement, tone and touch technique," the MT3, the complex of sound and gesture (and touch), when mapped onto a sound or word or phrase or clause or passage, is incredibly powerful. At times the eyes are actually tracking a gesture across the visual field; at others, the MT3 is out of direct line of sight, but, as the research reveals the "episode" is still embodied, in part, by one or more eye movements that are associated with it. How to exploit those complexes effectively is the question, of course. To learn more about how that works, go to www.actonhaptic.com/kinetik. 

Source: 
Eye-movement replay supports episodic remembering
Roger Johansson, Marcus Nyström,Richard Dewhurst and
Mikael Johansson
Published:15 June 2022 https://doi.org/10.1098/rspb.2022.0964

Proceedings of the Royal Society of Biological Sciences

Why haptic pronunciation teaching should be seen as more "memorable!"

(Eye) moving study by Johansson, Nystrom, Dewhurst and Johansson of Lund University, entitled, 
"Eye-movement replay supports episodic remembering." The idea that our eyes may move in the same or similar pattern or path in recalling an event that they experienced earlier when the even occurred has been recognized for several decades, used in various therapeutic applications and has been reported in several earlier blog posts over the last decade. 

This research study provides new, strong empirical confirmation of that underlying process. Subjects' eye tracking was recorded as they were shown various configurations of objects and asked to search out specific features. When later asked to recall details from that earlier phase, the eyes in effect closely "retraced" the paths used initially.  The implications are striking, even for haptic pronunciation work (KINETIK), which, itself, was inspired by the work of Bradshaw and colleagues in developing Observed Experiential Observation (OEI) therapy.

In the KINETIK English Fluency and Pronunciation System, sounds and words or phrases to be remembered are introduced and later recalled, associated with dynamic gestural patterns, accompanied by touch across that are performed in various locations in the visual field of the learner--either by the instructor, the learner or by both, simultaneously. (See examples of what are termed: movement, tone and touch techniques, from an earlier, version 4.5, of haptic pronunciation teaching.) After about 15 years of work, developing and teaching with the KINETIK Method, clearly the "eyes have it!" There are many well-established techniques and methods in various fields that involve synchronization of movement and speech and prompted recall, such as the Total Physical Response Method of language teaching. This research provides further justification for such embodied approaches. 

Speaking of, KINETIK v6.0 will be available for classroom use beginning this fall. More details will be available shortly, but if you would like to check out the general format of the courses, go here!

And, of course, Keep in Touch!

Published:15 June 2022https://doi.org/10.1098/rspb.2022.0964

Improved pronunciation "in the blink of any eye!"

How important is general/not directly task-based body movement, especially the lack of it, to learning pronunciation, creativity or just learning? In haptic pronunciation teaching learners are encouraged or required to move almost constantly, primarily through speech-synchronized gesture, but also through "Mindfulness-like" practices that monitor the state of the muscles and posture of the body, along with breathing patterns. 

But what about the impact on learning when students' bodies are held more in check, with restricted motor engagement? A new study by Murali and Händel of Julius-Maximilians-Universität Würzburg, Motor restrictions impair divergent thinking during walking and during sitting, summarized by ScienceDaily.com, not only affirms our intuitions about the central role of embodiment in thought and learning, but suggests something more: even while seated, a little movement appears to go a long way in maintaining creativity and attention. (What a shocker, eh? Hope you were sitting down when you read that!)

Ciker.com

The actual protocols of the research, which involved measurement of eye "blinking" responses as indices of degree of engagement, are not described in the summary, but the title of the original piece is interesting. To quote from the summary of the study: "Our research shows that it is not movement per se that helps us to think more flexibly," says neuroscientist Dr Barbara Händel from Julius-Maximilians-Universität Würzburg (JMU) in Bavaria, Germany. Instead, the freedom to make self-determined movements (emphasis, mine there) is responsible for it." 

In other words, messing with the body's incredible range of what appear to be random movements, apparently unassociated with the task being consciously in focus, may have dramatic consequences. An extreme analogy might even be talking with friends who are somewhere on the autism or ADHD spectrums. Their body and eye movements seem to suggest that they are not pay sufficient attention when in fact that is not the case at all. 

Now I am not saying that "thinking more flexibly" at any moment in instructional time is necessarily a good thing, of course, but the principle of allowing the body to also think and create on its own on an ongoing basis, in some sense "non or extra-verbally," if  you will, certainly is. On behalf of all elementary school boys on the planet who have had to sit in/through years of class to learn with girls when we should, instead, have been outside learning with our hands and whole bodies, I can only say, AMEN! 

Think about it. While you were reading this blogpost, what "else" was your body doing? If you can't remember . . . Q.E.D (quod erat demonstrandum)

Keep in touch!

Bill

Original source: 

Supriya Murali, Barbara Händel. Motor restrictions impair divergent thinking during walking and during sitting. Psychological Research, 2022; DOI: 10.1007/s00426-021-01636-w

Pronunciation in the eyes of the beholder: What you see is what you get!

This post deserves a "close" read. Although it applies new research to exploring basics of haptic pronunciation teaching specifically, the complex functioning of the visual field, itself, and eye movement in teaching and learning, in general, is not well understood or appreciated.

For over a decade we have "known" that there appears to be an optimal position in the visual field in
front of the learner for the "vowel clock" or compass in basic introduction in haptic pronunciation teaching to the (English) vowel system. Assuming:

• The compass/clock below is on the equivalent of an 8.5 x 11 inch piece of paper
•  About .5 meters straight ahead of your 
• With the center at eye level--or equivalent relative size on the board or wall or projector, 
• Such that if the head does not move, 
• The eyes will be forced at times to move close to the edges of the visual field 
• To lock on or anchor the position of each vowel (some vowels could, of course be positioned in the center of the visual field, such as schwa or some diphthongs.) 
• Add to that simultaneous gestural patterns concluding in touch at each of those points in the visual field (www.actonhaptic.com/videos) 

Something like this:

11.  [uw] “moo” 10.  [ʊ] “cook” (Northwest)	(North)	1.  [iy] “me” 2.  [I] “chicken” (Northeast)
9.  [ow] “mow” 8.  [Ɔ] “salt”  (West)	(eye level)	3.  [ey] “may” 4.  [ɛ] “best” (East)
7.    [ʌ] “love” (Southwest)		5. [ae] “fat”  (Southeast)
	6. [a]        “hot/water” (South)

Likewise, we were well aware of previous research by Bradshaw, et al. (2016), for example, on the function of eye movement and position in the visual field related to memory formation and recall. A new study Eye movements support behavioral pattern completion” by Wynn, Ryan, and Buchsbaum of Baycrest’s Rotman Research Institute, summarized by Neurosciencenews.com, seems (at least to me) to unpack more of the mechanism underlying that highly "proxemic" feature.

Subjects were introduced to a set of pictures of objects positioned uniquely on a video screen. In phase two, they were presented with sets of objects containing both the original and new objects, in various conditions, and tasked with indicating whether they had seen each object before. What they discovered was that in trying to decide whether the image was new or not, subjects' eye patterning tended to reflect the original position in the visual field where it was introduced. In other words, the memory was accessed through the eye movement pattern, not "simply" the explicit features of the objects, themselves. (It is a bit more complicated than that, but I think that is close enough . . . )

The study is not claiming that the eyes are "simply" using some pattern reflecting an initial encounter with an image, but that the overt actions of the eyes in recall is based on some type of storage or processing patterning. The same would apply to any input, even a sound heard or sensation with the eyes closed, etc. Where the eyes "land" could reflect any number of internal processing phenomena, but the point is that a specific memory entails a processing "trail" evident in or reflected by observable eye movements--at least some of the time!

To use the haptic system as an example, . . . in gesturing through the matrix above, not only is there a unique gestural pattern for each vowel--if the visual display is positioned "close enough" so that the eyes must also move in distinctive patterns across the visual field--you also have a potentially powerful process or heuristic for encoding and recalling sound/visual/kinesthetic/tactile complexes.

So . . . how do your students "see" the features of L2 pronunciation? Looking at a little chart on their smartphone or on a handout or at an LCD screen across the room will still entail eye movement, but of what and to what effect? What environmental "stimulants" are the sounds and images being encoded with and how will they be accessed later? (See previous blogpost on "Good looking" pronunciation teaching.)

There has to be a way, using my earlier training in hypnosis, for example, to get at learner eye movement patterning as they attempt to pronounce a problematic sound. Would love to compare "haptic" and "non-haptic-trained" learners. Again, our informal observation with vowels, for instance, has been that students may use either or both the gestural or eye patterning of the compass in accessing sounds they "experienced" there.  Let me see if I can get that study past the human subjects review committee . . .

Keep in touch! v5.0 will be on screen soon!

Source: Neurosciencenews.com (April 4, 2020) Our eye movements help us retrieve memories,

Ferreting out good pronunciation: 25% in the eye of the hearer!

Clker.com

Something of an "eye opening" study, Integration of Visual Information in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding, by Town, Wood, Jones, Maddox, Lee, and Bizley of University College London, published on Neuron. One of the implications of the study:

"Looking at someone when they're speaking doesn't just help us hear because of our ability to recognise lip movements – we've shown it's beneficial at a lower level than that, as the timing of the movements aligned with the timing of the sounds tells our auditory neurons which sounds to represent more strongly. If you're trying to pick someone's voice out of background noise, that could be really helpful," They go on to suggest that someone with hearing difficulties have their eyes tested as well.

I say "implications" because the research was actually carried out on ferrets, examining how sound and light combinations were processed by their auditory neurons in their auditory cortices. (We'll take their word that the ferret's wiring and ours are sufficiently alike there. . . )

The implications for language and pronunciation teaching are interesting, namely: strategic visual attention to the source of speech models and participants in conversation may make a significant impact on comprehension and learning how to articulate select sounds. In general, materials designers get it when it comes to creating vivid, even moving models. What is missing, however, is consistent, systematic, intentional manipulation of eye movement and fixation in the process. (There have been methods that dabbled in attempts at such explicit control, e.g., "Suggestopedia"?)

In haptic pronunciation teaching we generally control visual attention with gesture-synchronized speech which highlights stressed elements in speech, and something analogous with individual vowels and consonants. How much are your students really paying attention, visually? How much of your listening comprehension instruction is audio only, as opposed to video sourced? See what I mean?

Look. You can do better pronunciation work.


Citation: (Open access)

DOI: https://doi.org/10.1016/j.neuron.2017.12.034

Back to the future of pronunciation teaching (and the "Goldfish" standard for attention management)

You apparently have a bit more than 8 seconds to read this post. So you may want to just scroll down to the conclusion and start there . . .

Clip art: 

Capturing and holding attention, if only for a few seconds, is the key to effective change in pronunciation work, especially for "mechanical" adjustments--and most other things in life. In earlier blog posts, the "gold standard" or is sine qua non of haptic pronunciation work has been seen to be about 3 seconds. In other words, for a learner to adequately experience the totality of a new sound or word, physically, auditorily, visually and conceptually--connecting things together, before moving on to practice or at least noticing or any chance at "uptake"-- takes complete, undivided attention for at least that long or longer.

Even that is often an unrealistic requirement with all the other potential distractions in the classroom or visual field. Research on the effectiveness of recasting learner utterances by instructors, for example, (Loewen and Philip, 2006) suggests that most of the time that strategy is relatively ineffective. One critical variable is always the quality or intentionality of learner attention, both in term of what the function the instructor is attempting to carry out and general learner receptivity.

Clker.com

Recall that Microsoft claims that our collective attention span, in part due to the impact of technology, has now dropped to about 8 seconds, just below that of the goldfish. (The UK Telegraph report is much more entertaining than that from the techies.

A new study by Moher, Anderson and Song of Brown University, summarized by Science Daily.com, adds a fascinating piece to the puzzle and may suggest how to begin to maintain attention better in class. What they discovered in an experimental study was that their subjects were, in effect, better able to "block" obvious distractions than they were more subtle ones. Backgrounded images in the visual field had more effect on subsequent action than did foregrounded, more striking elements which appeared to be easier for the brain to manage or ignore. They seem to have "discovered" one possible path into the mind by subliminal stimuli, evading first line conceptual or perceptual defences.

What is the obvious "subtle, unobtrusive, yet potent" application to pronunciation teaching? If you don't have "full body, mind and visual field" attention, there is no telling what is interfering with anchoring of sound change in the brain and subsequent total or partial recall.

Early on in EHIEP (Essential Haptic-integrated English Pronunciation) work I experimented extensively with controlling eye movement, in part to maintain concentration and attention, based primarily on the research underlying the therapeutic model of "Observed experiential integration" (See citation below) developed by  Bradshaw and Cook (2011). The effect was dramatic in working with individuals but applying those techniques to the classroom proved at least impractical. In part because the haptic pedagogical system was just developing, I backed off from eye patterning techniques in pronunciation work in 2009.

Based on Moher et al's research, however, it is perhaps time to again give directed eye movement management a "second look" in our work, going back to what I believe is the (haptic) future of pronunciation instruction, especially in virtual, computer-mediated applications.

Will report back on an in progress exploratory study with one learner using some eye movement management later this summer. Not surprisingly I am already "seeing" some promising results, attending to features of the teaching session that I would normally not have noticed!

Full citations:

Brown University. "Surprise: Subtle distractors may divert action more than overt ones." ScienceDaily. ScienceDaily, 16 July 2015, www.sciencedaily.com/releases/2015/07/150716123831.htm. (Jeff Moher, Brian A. Anderson , Joo-Hyun Song. Dissociable Effects of Salience on Attention and Goal-Directed Action. Current Biology, 2015 DOI: 10.1016/j.cub.2015.06.029)

Bradshaw, R. A., Cook, A., McDonald, M. J. (2011). Observed experiential integration (OEI): Discovery and development of a new set of trauma therapy techniques. Journal of Psychotherapy Integration, 21(2), 104-171.

Loewen, S., and Philip, J. (2006). Recasts in the adult English L2 classroom: Characteristics, explicitness, and effectiveness. The Modern Language Journal, 90, 536-556.