Zoom(h)aptic: Haptic Pronunciation Teaching online

Keeping in touch doing pronunciation online with your students a problem? We've been working with what we call "haptic videos" for over a decade. Basically by that we mean using video models that learners move along with and in the process use gestures that are mediated and regulated by touch. (The touch usually occurs in the path of a speech-synchronized gesture where the stressed syllable in the word or phrase is articulated.) 

Just read a fun piece by Powers and Parisi on Techcrunch.com (hat tip to haptician Skye Playsted) The hype, haplessness and hope of haptics in the COVID-19 era, I'll focus a bit on the latter! What they get to is a number of haptic technologies, some of which at least promise to help us touch during COVID so we don't pass on something, such as virtual bank "touch" screens that feel to your fingers like you are actually touching the buttons when, in fact, you aren't. They also mention the sort of thing we have been following here for years such as haptic prosthetics, full-body suits and vests and gaming consoles. 

What we have discovered in doing haptic pronunciation teaching online for the last few years is that having learners "dance" along with us haptically, with extensive use of gesture and touch as they repeat or speaking spontaneously from various perspectives, is that the work really does "connect" us. Because the gesture complexes (pedagogical movement patterns - PMPs) are very easy to teach and conduct on Zoom, for instance, everybody (or every body) should get the sense of greater participation and what we term "haptic presence." 

Years of research on mirror neurons has demonstrated that if you are paying careful attention to the motions of another your brain is experiencing much of what is happening as if you, yourself, were the source of the action. What that means is that after students have been introduced to the gestural patterns--by doing them along with select phrases, when they see them again, it should (and generally does from our experience) resonate with them. In informal experiments where we ask students NOT to move along with us, they report that their bodies generally cannot help but move along to some degree. (That is a doctoral degree research project for any haptician who is interested!!!)

So . . . pack up your mirror neurons and go over the www.actonhaptic.com and look at the demonstration videos. And, while you are at it, check our our latest webinar with IATEFL on HaPT! After you do, come back and we'll sign you up for some HaPT training. Right now we are still in v4.5 but v5.0 "ActonHaptic Pronunciation Complement" will be rolling out later this fall!

Why using gesture in (pronunciation) teaching (sometimes) doesn't work well--or at all

Reviewed a manuscript recently that reported on a study exploring the use of gesture in pronunciation teaching, based on a method that seemed to require learners to repeatedly mirror

Clip art:
Ckker

instructor movements. The case made was not convincing, for at least a couple of reasons.

Research on mirror neuron function (even in monkeys, according the Association for Psychological Science - see full citation below!) has important implications for use of gesture in teaching, especially pronunciation. Normally, our mirror neurons mimic observed movement, giving us something of the sensation that we are actually doing what we are seeing, or perhaps moving along in synchrony with a person in our visual field. (Watch the audience at a dance recital, most discretely "moving along with" the dancers.) That should, in principle, make using gesture a potentially powerful vehicle for instruction. For most it probably is; for some, it isn't.

There are any number of reasons why gesture may not be that effective or why some learners and instructors simply do not feel comfortable with much "co-gesticulation." After decades of wondering exactly why gestural techniques were not more generally adopted (and adapted) in pronunciation instruction--when it was so natural and easy for me, personally--I got an answer from a student: the REVEG and ADAEBIP effects.

Rui was what I would term extremely "visually eidetic," meaning that she had a near photographic memory, such that if she learned a gesture in one position in the visual field and an instructor used that motion even very slightly off the original pattern, she could not process it or at least became very frustrated. Likewise, even looking in a mirror at herself performing gestures was maddening, since she, too, could not consistently move her hands in precisely the same track.

That encounter was a game changer. Within 6 months the EHIEP methodology had been changed substantially.

Since then we have encountered any number of learners who appear to have had varying degrees of "REVEG" (Rui's extreme visual-eidetic "gift") or "ADAEBIP" (aversion to doing anything potentially embarrassing with your body in public!) What that means is that for them, whatever the underlying cause, being required to mimic with any degree of accuracy someone's gesture can be maddening, near-traumatic or impossible.

The solution, at least in part, has been "haptic"-- to use touch to anchor the patterns to the relatively same locations in the visual field--along with anchoring the stressed syllable of a targeted word or phrase at the same time with touch. In addition, instead of the sometimes "wild and crazy" or "over the top," spontaneous gesturing used by some instructors, the idea is now to use highly controlled, systematic, "tasteful" and regular movements for pointing out, noticing and anchoring, and homework.

In fact, one of the advantages of using video models in EHIEP (as in AHEPS, v3.0) is that at least the patterns students are trained on are consistent. In that way, when a pattern (what we call a "pedagogical movement pattern) is used later in working on "targets of opportunity," such as modelling and correction, learners tend to be more accepting of the instructor's slight deviations from the "standard" locations.

In general, haptic anchoring of patterns (PMPs) tends to keep positions of prescribed gestures within range even for the more REVEG among us. Extreme accuracy in actually producing the PMPs in practice and anchoring is really not that critical for the individual learner.

So, if gesture work is still not in within your perceptual or comfort zone, we may have a (haptic) work-around for you. Keep in touch.

Citation:
Association for Psychological Science. (2011, August 2). Monkey see, monkey do? The role of mirror neurons in human behavior. ScienceDaily. Retrieved December 10, 2014 from www.sciencedaily.com/releases/2011/08/110801120355.htm

Guidelines for using (haptic) gesture in pronunciation teaching

Literally for decades I was working under the assumption that gesture and general body movement work, in principle, was a good way to loosen learners up and get them engaged, let alone teach aspects of pronunciation. For some, it is, but the impact on others, especially those from less "gesticular" cultures, can be unproductive, at best. I have evolved, somewhat at least, from cheerleader to coach/consultant in that regard. A few general principles:

• If you do make extensive use of gesture with adolescents and adults, you must be able to explain why FIRST, at least initially persuade them that it is research and success-based. 
• The directed movement must be highly controlled, both in terms of range of motion and emotional loading, and very easy to teach and to follow. 
• The gesture work is most effective when done "in chorus," as a class, with learners visually attending to and following the instructor, not being able to see what each other is doing. 
•  It must not be forced. If a learner choses not to participate, or do so only minimally, that is fine. (Research on mirror neurons and years of experience with this kind of teaching confirms the power of engaged observation.)
• The gesture must be consistently coupled with strong vocal resonance to make sure that it is well anchored. That is based, in part, on the work of Lessac in voice training.
• Learners must experience early success. Using baton-like gesture on stressed syllables to enhance memory for vocabulary is often a good start. 
• The gestural patterns, what we ("haptic" pronunciation teachers) term "pedagogical movement patterns" need to be used consistently in integrated classroom instruction, presentation and correction. 

If you'd like to see an example of what the haptic patterns look like, here is a demonstration video clip that shows the set that can be used for presenting (not training in or practicing) the lax vowels of general American English. https://vimeo.com/60977241 Especially the set of low and mid-back vowels in that demonstration would have to be adjusted accordingly were you working with specific regional dialects of English or "World Englishes." 

Seeing (haptic-integrated pronunciation) is (mirror neuron) believing!

This article from CriticalDance.com makes an important observation. The response of the mirror neurons of dancers is much stronger when they are watching familiar dance elements of new dances that they are familiar with, as opposed to seeing new elements in new dances. They are, in effect, able to learn the complete routines "simply" by watching, without physically being on the dance floor because their brains are mirroring and then committing to kinaesthetic memory a new arrangement of familiar elements.

That explains, in part, why learners are often able to quickly "uptake" haptic feedback or correction by instructors. Learners both see the pedagogical movement patterns and (usually) hear the "correct" form or pronunciation performed by the instructor--which they have been introduced to earlier in the course. Their mirror neurons should lock on the PMPs, which are anchored to the felt sense of the sounds.  It is a case where we learn best what we know already, what we have been touched by or touched. See what I mean?

Mirror Neurons, Dance Movement Therapy and Haptic Mirroring

Clip art:
Clker

This 2006 article by Berol looks at the underlying neurophysological basis of mirroring in dance-movement therapy. Of particular interest to our work is both the full-body engagement with the model and the strong emotional, empathetic grounding involved. As noted in earlier posts, carefully managed mirroring offers an extraordinary level of attention to aspects of communication generally thought to be outside of the domain of the classroom, to be learned inductively. It can be, however, very difficult to incorporate into instruction consistently and in a way that is psychologically "safe."

Dance therapy protocols do an extraordinary job of setting the stage for engaging the client/learner in the "moving" experience. The language used and therapeutic process of DMT offer valuable insights into effective task staging of the pedagogical process for pronunciation instruction. HIPoeces employs haptic mirroring (mirrored movement with a range of touch intensities and configurations), using a number of DMT strategies,including teaching from a mirror-image perspective at times. A good first step is to set your mirror neurons down in front of a video of yourself teaching intonation for a couple of hours . . .

Collaborative haptic-integrated instruction

As explored in earlier posts, for any number of reasons, HICP work has been restricted to engaging but culturally "tasteful" touching of one's own hands, deltoids (or clavicle) and quadriceps. The "collaborative music controller" developed at Stanford, would, in principle, function like the "haptic mirror neurons" in the brain, guiding and synchronizing the hand of the other.

Clip art: Clker

Imagine the possibilities: being able to quickly train learners in the correct pedagogical movement patterns (virtually) without touching them. As it is now, if a learner is having difficulty picking up a pattern, given the right setting and relationship with that learner, I might occasionally physically guide a hand or arm momentarily--but do not recommend that as a regular classroom practice. If necessary, brandishing a pointless, "guiding" pencil will usually be sufficient.

Were EHIEP  to be imported into a virtual reality system, much of the basic training could probably be done using similar haptic-mirroring technology. By that point it would also be far easier to persuade the "haptically challenged" to "mirror-ly" get with it as well . . .

Kinesthetic empathy and haptic listening

Here is the first of two very cool videos from a neuroscience/dance project and conference: "From Mirror neurons to Kinesthetic Empathy." (The sound quality is problematic in places.) Dance-related research in kinesthetic empathy explores, in part, how the observer of dance "moves along" with the dancer--and how that experience can be utilized and enhanced.

Credit: www.watchingdance.ning.com

One frequent observation by EHIEP learners is that near the end of the program their listening skills have improved in a somewhat unexpected manner. Specifically, they have become better at remembering what is said, how it is said and able to repeat what they have heard (often using EHIEP pedagogical movement patterns). The "felt sense" of that experience seems to be very much body-based, non-cognitive, as if the whole body is recording the conversation. Although we have for sometime been terming that "kinesthetic listening," we have not yet developed the advanced listening comprehension protocol systematically. We should soon. Hapticempathy?