How YOU elocute is how I elocute: Collaborative haptic motor skill (and pronunciation) learning

For a glimpse into the future of instruction, have a look at Chellali, Dumas and Milleville-Pennel (2010) "A Haptic Communication Paradigm For Collaborative Motor Skills Learning." Their WYFIWIF (What you feel is what I feel) model illustrates nicely just what haptic technology is, in essence using a computer-mediated interface to guide movement, using basically pressure translated through some kind of device such as a glove. In the study, subjects were guided to better performance on a focused manual task, moving a needle, by a haptic-assisted instructor. Not surprisingly, the control group, the visual or verbally-guided only group, did not perform as well. 

Another example of haptic communication, as defined in WYFIWIF, might be an instructor first leading a learner through a gesture pattern with haptic technology and then continuing to provide haptic guidance as the learner attempts to practice and master the pattern. The researchers note that in a virtual environment, as in haptics-assisted surgery or training, " . . . haptic communication is combined (more and more with complementary) visual and verbal communication in order to help an expert to transfer his knowledge to a novice operator."

Although the haptic application to our pronunciation work does not involve haptics technology, but rather hands touching on target or stressed sounds--following the visual and spoken guidance of an instructor or peer--the parallel is striking. It is the collaborative haptic-embodied task (instructor and learner engaged in a tightly linked, synchronous, communicative, embodied "dance") that greatly enables and facilitates learning. 

In the conclusion of the study, there is a truly striking recommendation for further research: the impact on haptic communication of the "verbal communications between the instructor and the leaner." We have  over a decade of experience--and a few dozen blogposts--with that! Now "needle-less" to say,  if we can just get our hands in some of those gloves . . .

Full citation:
Amine Chellali, C ́edric Dumas, Isabelle Milleville-Pennel. WYFIWIF: A Haptic Communication Paradigm For Collaborative Motor Skills Learning. IADIS. Web Virtual Reality and Three-Dimensional Worlds 2010, Jul 2010, Freiburg, Germany. IADIS, pp.301-308, 2010.

Research supporting haptic pronunciation teaching

Acton, W. (in press). OEI and accent reduction, in Bradshaw, R. (Ed.) Toward integration: Clinical applications, Vancouver: Sync publications.

Burri, M., Baker, A., & Acton, W. (2019). Proposing a haptic approach to facilitating L2 learners' pragmatic competence. Humanising Language Teaching, 3. Available at http://hltmag.ng3.devwebsite.co.uk/june19/proposing-a-haptic-approach

Burri, M., Acton, W., & Baker, A. (2019). Moving to L2 fluency: The tai ball chi technique. Speak Out! Journal of the IATEFL Pronunciation Special Interest Group, 60, 43-51.

Kielstra, N. and Acton, W. (2018). A haptic pronunciation course for Freshman ESL college students!, in Murphy, J. (Ed.) Teaching the Pronunciation of English: Focus on whole courses, Ann Arbor: University of Michigan Press.

Acton, W. , Burri, M. and Baker. (2016). Anchoring Academic Vocabulary with a "Hard Hitting" Pronunciation Teaching Technique, in Jones, T. (Ed.) Pronunciation in the classroom: the overlooked essential. New York: TESOL.

Acton, W. , Burri, M. and Baker. (2016). Anchoring Academic Vocabulary with a "Hard Hitting" Pronunciation Teaching Technique, in Jones, T. (Ed.) Pronunciation in the classroom: the overlooked essential. New York: TESOL. 

For earlier general references go to this page:

General theory on embodiment

• Embodiment is critical to 2LA (Holme, 2012)
• The body as an instrument for change of voice and persona (Lessac, 1984 and 1997)
• Haptic cinema (Marks, 2012)

Gesture

• Gesture and language are tightly interrelated (Kendon, 2004)
• Mirror neurons and learning movement (Simpson, 2008)
• Gesture observed enhances learning (Wagner et al., 2013)
• Gesture use enhances speaking  (Beliah, 2013)

Gesture in second language learning

• Gesture supports second language learning in several ways (McCafferty, 2006)
• Gesture enhances second language learning (Macedonia et al, 2012)
• Gesture  is closely related to prosodics (rhythm, stress and intonation) in L1 and L2 acquisition (McCafferty, 2004)
• Gesture and grammar (Churchill et al., 2013)

Touch

• Touch serves to bond the senses together (Fredembach, et al, 2009).
• Touch influences/manages memory for events (Propper, et al., 2013)
• Touch is remembered through movement, sound and visual images (Charite, 2011)
• Intensity of touch determined by intentions/set up (Gray, 2013)
• Tactile Metaphors (Lacy et al., 2012)
• Binding of movement, sound and touch (Legarde, J. and Kelso, J., 2006)

Haptics

• The nature of haptics (Harris, 2013)
• Haptic technology (Kuchenbecker, 2012)
• Sense of touch technology (Umeå universitet, 2012)

Gesture and haptics in teaching

• Haptics in education (Minogue et al., 2006)
• Gesture widely used in phonetics and L2 pronunciation teaching (Wrembel, et al., 2011)
• Gesture use in second language teaching (Hudson, 2011)

Introduction to some haptic gadgets - II

Kudos to CNN Tech Trends for this nice 14-slide piece by Arion McNicoll on haptics and new haptic gadgets. If you are just getting "in touch" with haptics, you'll like this. See especially slide #8 on Tesla Touch. I have done some research on that technology recently, an approach that may have promise for our AH-EPS haptic pronunciation. (See also the recent blogpost linking the TED talk on haptics, too.)

Credit: CNN.com

Introduction to haptics and some possible applications

If you are new to the idea of haptics and "haptic," here is a neat 6 minute TEDYouth 2012 talk by Kuchenbecker of University of Pennsylvania (Hat tip to Karen Rauser.) Our work in haptic-integrated pronunciation teaching is something of the flip side of this. Whereas Kuchenbecker's work digitizes touch and movement to accompany video, we create the haptic felt sense of sound (through awareness of vocal resonance, upper movement and touch) to accompany the positioning of the hands and arms in the visual field. Have been working on the outlines of a TED talk proposal myself for next year. Keep in touch!

The future of pronunciation teaching: It's simple, Haptics!

Clip art: Clker

Well . . . actually what Moussette proposed in a 2012 doctoral thesis at Umeå University, Sweden, reported by Science Daily, is the term "Simple Haptics," which " . . . advances that haptic attributes (how things feel through touch) are increasingly part of the qualities that make up the interactions and the experiences we have with objects and the interfaces that surround us, and that these considerations can and ought to be knowingly and explicitly designed by designers." This is from the abstract: "The main knowledge contribution relates to the massification of haptics, i.e. the intentional realization and appropriation of haptics—with its dimensions and qualities—as a non-visual interaction design material." Now I'm not exactly sure what "massification" means in that context but I like it, especially as it is used in this example by Wordnik online dictionary--which couldn't locate a definition apparently--quoting ANC Daily News Briefing: “This process, known as massification, would have to be accompanied by changes in curriculums and qualifications and the introduction of multiple entry and exit points.” (Italics, mine.)

That is us; that is Haptic-integrated Clinical Pronunciation! Along with Moussette, now we "simply" need to get the word out to the "masses" of language teachers and curriculum designers who have yet to be "touched" by haptics in pronunciation teaching!

Why "haptic-integration" should enhance pronunciation work

Junko Kimura/Getty Images

Linked is a good, readable nontechnical summary of what computer haptics (and haptic) is about. (This one nicely complements the 2007 AERA review of research to that date by Minogue and Jones, discussed in an earlier post that reported on the application of haptics in education as well.) This blog post is titled why haptic-integration SHOULD work in our work, in part, because the hard, research-based evidence is, generally  speaking, only indirect, coming from five areas:

(a) Basic research on the neurophysiology of movement and touch, including its close relationship to visual and auditory modalities,
(b) Developments/recent successes in computer haptics-applications such as virtual reality training, gaming and prosthetics,
(c) Practice in several related fields such as sports, dance, rehabilitation and the arts
(d) A few relevant empirical studies examining the effectiveness of haptic enhancement in approaches to helping children learn to connect up orthography and sound, and
(e) About five years of explicit application of haptic pedagogical movement patterns in the EHIEP system and its early predecessors.

I have designed two or three, small scale empirical Stick!-based studies of haptic-integration techniques that I hope to carry out in the next few months. It is a problem inherent to most experiential educational methods. In the meantime, we'll continue to focus primarily on the how . . . and Wow!

Pronunciation futures: the iPad (haptic) HD

Photo courtesy of Slashgear

If you have to explain what "haptics" is to a colleague or student, from now on just let them touch your new iPad HD. Delighted to see that the case is now closed on how effective haptic interfaces are in engaging the user/learner. Thanks to Apple, the research on haptics and widespread application to touch technology will quickly become available. I'm told by friends who work with iPhone programming that our work is a natural for that interface--especially now that the door has been opened in Cuperinto. Now, rather than buying a couple dozen research articles online, I'm going to get an iPad HD. Welcome to the future of pronunciation teaching. Keep in touch. 

Haptic sine machina (HICP)

Clip art: Clker

How about a 2005 conference announcement with a succinct definition of "haptic video" that includes the concept of machine controllers. As was pointed out to me recently, that is the "industry standard," that haptic, as currently used in most fields assumes proactive transmission of movement and force through a mechanical interface of some kind.

HICP/EHIEP, of course, does not yet involve such a "machine." The training program firmly establishes the felt sense or feel of the pedagogical movements; the regular warm up reestablishes that awareness before each session. That felt sense includes (as noted in earlier posts) four types of resonance, pressure (between hands), four distinct types of skin contact (depending on what part of the left hand impacts where on the right hand or upper body), relative speed, precise 3-dimensional positioning in the visual field. Given that degree of "haptic" engagement, when a sound pattern is later signalled by the instructor the effect of the visual model should be nearly as "controlling" for physically realized as being connected to a mechanical arm or controller.

But perhaps we do need a more precise term here, at least for the name of the blog. How about "Haptic-integrated Clinical Pronunciation (Haptic sine machina)". In the classical sense, of course, the field of Haptics is the true "deus ex machina"in Greek tragedy where a mechanical crane was used to lower actors on to the stage to effect a solution "from the gods," something far outside the normal (literally, god from the machine). HICP, on the other hand, still relies just on the well-conditioned human body to effect its "miraculous" endings. 

Haptic Video and a haptics video

The terms "haptic" and "haptics" are used inconsistently today in various fields but the distinction is important for HICP work. The general definition of "haptic" is " . . .relating to the sense of touch." HICP methodology falls within that definition. Technically, we should refer to our work as something like "kinaesthetic-haptic," but we have been using "haptic" as a shorthand of sorts to include both movement and touch. "Haptics," on the other hand, is the "science of applying touch to human-computer interaction." (See this somewhat glitzy video of some haptics devices.)

As noted many times in blogposts, however, the ultimate GOAL of HICP development is to create a system that is readily integrated with "Wii-like" virtual reality technology. In a very real sense, it is approaching that. Here is an important caveat, however. As it is now the "felt sense" of saying a word, focusing on the vowel resonance, while performing a pedagogical movement pattern, culminating usually in both hands touching on the stressed syllable in most cases, is a very powerful,  proven holistic anchor. It is not entirely obvious that the virtual reality embodiment of the system will be as effective (or how?), although it would contribute a range of possibilities of voice recognition options such as modelling and offering various types of corrective feedback.

The two current versions, one done face-to-face by a "live" instructor and EHIEP haptic video system (EHVS), both have the same haptic requirements on the part of the learner of following either the instructor in the classroom or the instructor on the video. The haptic video system is a substantial step toward the virtual reality embodiment design. In other words,  HICP work is currently "haptic" in application but "haptics" in outlook and destination. It is just a matter of keeping in touch . . . and time. 

Introducing Haptic-integrated pronunciation work

Clip art:
Clker

Clip art:
Clker

Here's an interesting note on a website that develops haptic technology:

 "As with any UI element, haptics must be designed thoughtfully in order to achieve the desired experience. Users expect the sight, sound and feel of their experience to be consistent, rational and integrated. The combination of haptics with audio and visual UI can be breathtakingly effective if all these components work together, but can be confusing if poorly designed. To help guide the developer through the process, we’ve created a series of design recommendations for the most commonly implemented gestures."(Italics, mine.)

Those four terms in italics, "consistent, rational, integrated--and breathtakingly effective" provide a good set of criteria for assessing haptic pronunciation work--especially the latter! The second, rational, is worth more discussion. Providing a simple, direct, verbal explanation for learners as to why they should be excited about EHIEP is a challenge. Leading off with a warm up and demonstration where students join in, on the other hand, almost invariably convinces them that it is "designed thoughtfully" and worth engaging with. To quote my favorite athletic shoe logo: Just do it!

Affective haptics

Clip art: Clker

In the earlier post on the upcoming 2012 Haptics Symposium, I mentioned the Workshop on Affective Haptics. Here is the note from the summary of the workshop:

"Affective Haptics is the emerging area of research which focuses on the design of devices and systems that can elicit, enhance, or influence the emotional state of a human by means of sense of touch. Human emotions can be easily evoked by different cues, and the sense of touch is one of the most emotionally charged channels. Affective Haptics is a wide interdisciplinary area, strongly related to such fields as multi‐modal interfaces, affective computing, neuroscience, psychology, mediated communications, telepresence, robotics, etc."

HICP work draws from most of those areas, even the last. Note the EHIEP logo, which draws something of its character from the robot-like control of the upper body required for mosts protocols. Keep in touch. 

Haptic testimonials from the stars - II

Clip art: Clker
/Library of Congress

Talk about being out (side) of the (Cornell university research) box:

"I loved the haptics. They were much better than Cats." -Matthew Broderick, actor

"A haptic box? What's the deal with a haptic box? Someone tell me, because I'd like to know." -Jerry Seinfeld, actor

"I felt like I was living inside the Cornell box." -Shaquille O'Neal, professional basketball player

"These haptics made me a better-looking man. After I experienced the Cornell box, I had a date every night for a year!" -Steven Adler, the original drummer for Guns and Roses

Talk about endorsements for all things haptic! Rather than unpack what they think that are talking about specifically, I'll just leave it at that . . .