Haptic cinema and EHIEP-tic pronunciation training

My discovery of "haptic cinema" and that approach to experiential entertainment and teaching about 6 years ago was a game changer. The integration of the senses, especially the place of perceived texture in that media became the phenomenological model for "haptic-integrated clinical pronunciation," and still is. Here is a great example, "Haptic cinema: a sensory interface to the city."  It is about 11 minutes long. Put on some earphones, sit someplace where you'll have no visual distractions and experience it. 

Clip art: Clker

That is what it should feel like, the felt sense of haptic anchoring in EHIEP instruction, when the learner articulates a sound or word with rich vocal resonance as hands move across the visual field (with some degree of eye tracking) and the hands touch on the stressed vowel--possibly followed by a short continued movement completing an intonation "denouement." 

To prepare for watching it, you might go outside and hug a tree first . . . 

Do-it-Yourself! haptic-integrated pronunciation teaching

Clip art: Clker

Clip art: Clker

Haptic work is, by definition . . . touching! As explored in several previous posts, there are a wide range of conditions under which haptic anchoring of movement, visual images and sound may or may not be effective in instruction. (According to new research, by Patterson and colleagues at the University of Liecester, summarized by Science Daily, there may even be a bias in favor of those of us over the age of 65 in responding to the typical "fuzziness" of haptic cinema!)

One of the most striking discoveries in our work has been the realization that some of the EHIEP pedagogical movement patterns can be taught well face-to-face but others may be better introduced by a video model, especially vowels, vowel "compaction" and intonation. That video model can be the instructor, him or herself, or someone else--such as in the EHIEP system of videos and student workbooks that I am developing, of course! Why that should be is complex but understood (See this blogpost by Grant on http://filmanalytical.blogspot.ca/)

In essence, it is emotionally and interpersonally very powerful. In some contexts, either because of the personality of the instructor or the class, video is a better option for perhaps half of the PMPs. One reason for that is the impact of eye contact on mirroring in a classroom setting. In essence, vivid "moving" visual feedback from students, whether negative  or positive can dramatically undermine an instructor's ability to teach PMPs. Once they are introduced, however, classroom use of a PMP to anchor vowels, stress, rhythm, intonation or pitch/volume/pace seems to be less susceptible to disruption.

Bottom line: It takes training to do pronunciation work of any kind effectively or efficiently. Either you get trained or have somebody else do it for you, either in your program or through technology. Haptic video and its post-production technology is very promising. I am tempted to use a term like "CAPT Video," Computer-Assisted-Pronunciation-Teaching with Video, were there not already a near-relevant song by that name .  .  .  

Pronunciation & body & media fit

Clip art: Clker

If you have been reading the blog occasionally, you are aware of the basis of the EHIEP model: (a) initial pronunciation teaching and (b) practice outsourced to video with subsequent (c) integrated use in the classroom, (d) strong haptic engagement (movement and touch) and (e) somatic or body awareness and training. For the latter piece, body monitoring, maybe what we need is something like the "BodyMedia FIT" system. I love the company's come on line: "Your body talks. We listen." Wish I had the spare change to buy one of those arm bands, just for fun. The research on effectiveness of the technology, using web-based systems,  is interesting. "Body training," in general, is biofeedback of one kind or another. This type of technology could easily be adapted to provide constant feedback on the quality of movement, relaxation, energy expenditure and body resonance. For much less money and hassle--with a modicum of self-discipline and persistence, learners can experience the same kind of integrated experience of speaking and pronunciation change with us. The future, however, is with technology such as this linked to CAPT (see previous post.) and haptic cinema. But if you have difficulty consistently managing your "current classroom body image" and its caloric correlates, consider "arming yourself" with such a band. 

Disembodied pronunciation: computer generated, animated images of learners' inappropriate articulation

Clip art: Clker

Clip art: Clker

May start a new series of blogposts focusing on amazing-looking pronunciation techniques that, from a HICPR perspective, are so thoroughly disembodied or "dys-haptic" (generally depending heavily on only visual modalities, lacking a somatic, physical basis) such that chances of them working are probably not all that good, at best, such as this one:
"Improvement of animated articulatory gesture extracted from speech for pronunciation training," by Manosavan, Katsurada, Hayashi, Zhu, Nitta of Toyohashi University, a paper from the 2012 IEEE Convention--available for 31 bucks to nonmembers. (Have not read the full paper, just the abstract. My general policy is to pay for no research papers that cost more than 6 Starbucks Vente Carmel Frappuccinoes.) Computer-assisted Pronunciation Training (CAPT) is probably the future of the field, but a system that creates a moving cartoon-like representation of what a learner is doing wrong and then juxtaposes that with an animated image of how to do it right cannot possibly work effectively or efficiently-expect perhaps for those who are CAPT designers and gamers. (What do they need appropriate pronunciation for anyway?) 

However, if that video image were to be merged with "haptic cinema" technique and technology, (linked is a very "a-peeling" example, in fact!) they may still be on to something. 

The (haptic) handwriting on the (virtual) wall for pronunciation instruction

"MENE, MENE, TEKEL, PARSIN!" (Possible pronunciation: Many, many tech'cle person!) 

At the TESOL convention in March, I'll be giving a talk in a Symposium on integration of pronunciation teaching. The title will be something like "Post-pronunciation, pronunciation instruction." Will argue three points: (a) The movement toward integration of pronunciation teaching into all skill areas signals the end of what we do as we do it. (b) Those can operate comfortably in virtual technology are going to take over and , (c) but the emergence in the last decade of haptics technology, haptic engagement in pedagogy, and haptic video and cinema, among others . . . offers exciting possibilities! (Rough translation and extrapolation of the Babylonian above: It's about over, gang. The field has had near enough of our disembodied,  insiders' club attitude. Our best tricks are about to be passed out to tech- and haptic-savy Newbees.) Could be worse . . . we could be in a lion's den . . .  or Philadelphia . . . 

"Off the wall" example of the felt sense of haptic anchoring

If you haven't seen Mission Impossible: Ghost Protocol, yet, it is worth the ticket just to see Tom Cruise go up the side of Burj Khalifa, the tallest building in the world . . . from about the 120 floor to the 130th, wearing "haptic gloves" that stick to glass . . . at least the left one! In IMAX the special effects are fantastic; you feel like you are free climbing with him. One of the best "haptic videos" I have seen . . .or felt! With one glove malfunctioning he, of course, accomplishes the impossible. Every time the "magnetic" left hand clamps onto the glass (with terrific sound effects) you can feel it in your left had as well. (If you are not a great fan of extreme heights, your palms will be activated already, of course!)

So if you would like to experience a couple of minutes of extreme haptic anchoring, not entirely unlike your average EHIEP protocol lesson . . . your mission, should you accept it . . . is just to accept it!

Engaging haptic hormone?

The hormone oxytocin has been associated with a range of social functions such as trust, facial recognition, massage technique and lactating. In this 2008 study by Gordon,  Zagoory-Sharon, Leckman, and Feldman, summarized by Science Daily, it was also found to be associated with parenting styles that involved richer communication and engaging touch with infants in both fathers and mothers.

Clip art: Clker

The concept of "haptic visuality" was proposed by media/art critic Marks, that the eyes, in some multi-literacy, multimedia environments, such as haptic cinema, become very much "tactile-like," interpreting experience more as through the skin, potentially bypassing higher cognitive critical functioning and filters. What that means for development of haptic interfaces is that in a vivid multisensory "nexus" or event, the brain becomes much more holistic in interpreting what is in front of it, in effect interpreting the experience as a "felt" whole, not allowing critical analysis or deconstructing to enter into the process much. This is, of course, also the "Holy Grail" of advertising, movie makers and marketing.

A shot of oxytocin before your next haptic pronunciation lesson? Touching . . .