Accurate mirroring in (haptic-integrated) pronunciation teaching

Clip art: Clker

Clip art: Clker

Mirroring, having learners move along with a model, is a common technique in pronunciation teaching, especially at more advanced levels such as this by Goodwin at UCLA. There are any number of applications of the concept, for various purposes. In EHIEP work, mirroring figures in prominently from the beginning. As noted in previous posts, some highly visual learners find imprecise modelling by the speaker being mirrored to be very disconcerting. For example, one pedagogical movement pattern (PMP) involves moving the left hand across the visual field in an ascending motion as a rising intonation contour is spoken. For sometime we have been looking at the possibility of using avatars that would perform perfectly precise PMPs in new versions of the haptic videos to compensate for the fact that a human model (namely me on the current videos) cannot possibly be consistent enough to satisfy the few most radically visual learners. Research by Thomaz at Georgia State University seems to suggest that the only way to do that with robotic models--would be to build in "human-like" variability of motion into the repetitions of PMPs in training. In other words, the slight differences in the track of the gestural patterns is essential to creating a sufficiently engaging model to effectively keep subjects' attention. Rats. Better go back to figuring out both how to be more "humanly" precise in modelling PMPs and developing techniques that will assist the "visually-challenged" in loosening up a bit. Figuring out exactly what acceptable deviations from ideal PMPs are is, in principle, doable, of course. Just a matter of studio time and field testing. Keep in touch. 

Intonation deafness? The fix may be "psycho-or-somatic!"

Clip art: Clker

Clip art: Clker

Have students who have great difficulty imitating intonation contours? Could be tone deafness or "amusia," in varying degrees. (If you have a question, there is a self-administered test off the first link that may be of some help.) Both studies, summarized by Science Daily, relate to singing, not speech intonation directly. The "phenotypes" identified, however, are of interest to us: (a) those who have difficulty perceiving pitch and pitch change and (b) those who sing away, regardless. I have worked with several over the years of the latter type. (In diagnostic work I often will have learners sing along with me on some simple songs to check their singing pitch control.) The relationship between intonation problems and inability to sing is complex, of course. Occasionally, serious hearing or speech production disorders were involved; other times, there appeared to be interpersonal, psychological issues of identity and self-confidence. I have been unable to find credible, published research on effective "treatment" for tone deafness. Intonation "deafness" may be something of another matter, caused by a wider range of factors. Sometimes haptic anchoring of pitch (by using PMPs set high or low in the visual field as a phrase or word is articulated) can be moderately effective, or at least serve to enhance slightly the lack range or production of pitch change, such as "rise-fall" contours. That type of work, based loosely on Lessac and Observed Experiential Integration Therapy, has to be done one-on-one at this point in time and it can be time consuming, but the potential for "embodying" that concept in a computer-mediated system is intriguing. The approach, regardless, must be very much "whole-person," mind and body, "psycho(and)somatic!" 

Visual intonation in pronunciation instruction

Clipart: Clker

Clipart: Clker

One of the early inspirations for the EHIEP system, as reported earlier, was my work with deaf students at the university who communicated with American Sign Language (ASL). Sign gestures, along with upper body and facial expressiveness have been researched extensively; I have referred to those studies several times in trying to better understand how ASL behaviors can be effectively exploited in our pronunciation work. I found this 2009 study on "visual intonation"in Israeli Sign Language recently which demonstrates vividly the role of the systematic movement of facial muscles (e.g. raised eyebrows, squints, knitting brows) in conveying prosody, especially as it accompanies pitch change. The analogous pedagogical movement patterns of EHIEP are designed primarily to help the learner get the felt sense of intonation, where learners' hands and arms move around the visual field to anchor pitch (H, M and L) and pitch change (fall-rise, rise-fall, etc.) However, once a student is able to "repeat" a PMP along with the instructor in articulating a target word or phrase, the "visual intonation" portrayal also serves to communicate to the instructor and student something of accuracy of the pitch or pitch change. In other words, an instructor also uses the visual PMPs in both assessing and providing feedback in class on intonation. Just as in the case of the ISL study, spatial-gesture serves more than just as an analogy for pitch and pitch change--it is potentially a very powerful physical anchor. See what I mean?