Professor
Department of Psychology
Department
of Otolaryngology
Queen's University
telephone: (613) 533-6012
fax: (613) 533-2499
e-mail: munhallk@post.queensu.ca
Research Interests
My general research interest is the understanding of the processes of speech production and speech perception. My speech production research focuses on the control of vocal tract and facial motion: what are the units of coordination; how are sequences of actions implemented; what form do the central commands take in fluent speech. I use a number of techniques to gather physiological data, including electromyography, kinematic analysis of OPTOTRAK data, and image processing of X-ray images from the X-ray Film Database.
My speech perception research focuses on audiovisual integration during face-to-face communication: what are the visual cues for individual speech sounds; what are the necessary and sufficient conditions for audiovisual fusion; what patterns of eye movements are observed during face-to-face communication. In this area, much of our work has focused on the McGurk effect.
Recently, I have begun a project on facial and vocal tract modeling. The purpose of the work is to produce realistic facial animations that can be used in controlled audiovisual experiments and to generate models which summarize our knowledge of the speech production process.
MacDonald, E.N., Purcell, D.W., and Munhall, K.G. (2011). Probing the independence of formant control using altered auditory feedback. Journal of the Acoustical Society of America, 129(2): 955-966. PDF 989 KB
Zheng, Z., Munhall, K.G. & Johnsrude, I.S. (2010). Functional Overlap between Regions
Involved in Speech Perception and Monitoring One’s Own Voice During Speech Production.
Journal of Cognitive Neuroscience 22 (8), 1770-1781. PDF 375 KB Mansour, J. K., Lindsay, R. C. L, Brewer, N., & Munhall, K. G. (2009). Characterizing
visual behaviour on a lineup task. Applied Cognitive Psychology. 23(7), 1012-
1026. PDF 151 KB Tahmasebi, A., Abolmaesumi, P., Zheng, Z.Z., Munhall, K.G. Johnsrude, I.S. (2009).
Reducing Inter-subject Anatomical Variation: Effect of Normalization Method
on Sensitivity of Functional Magnetic Resonance Imaging Data Analysis in
Auditory Cortex and the Superior Temporal Region. Neuroimage 47, 1522-
1533. PDF 1917 KB Munhall, K.G., ten Hove, M., Brammer, M., Paré, M. (2009). Audiovisual integration of
speech in a bistable illusion. Current Biology, 19(9), 1-5. PDF 232 KB Munhall, K.G., MacDonald, E., Byrne, S. & Johnsrude, I. (2009). Talkers alter vowel
production in response to real-time formant perturbation even when
instructed not to compensate. Journal of the Acoustical Society of America. 125,
384-390.
PDF 153 KB Lucero, J.C. & K.G. Munhall (2008). Analysis of facial motion patterns during speech
using a matrix factorization algorithm. Journal of the Acoustical Society of
America 124, 2283-2290.
PDF 744 KB Buchan, J.N., Paré, M., & Munhall, K.G. (2008). The effect of varying talker identity and
listening conditions on gaze behavior during audiovisual speech perception.
Brain Research, 1242, 162-171. PDF 763 KB Wilson, A., Wilson, A., ten Hove, M., Paré, M. & Munhall, K.G. (2008). Loss of central
vision and audiovisual speech perception. Visual Impairment Research, 10,
23-34.
PDF 1955 KB Tajima, K., Kato, H., Rothwell, A., Akahane-Yamada, R. & Munhall, K.G. (2008).
Training English listeners to perceive phonemic length contrast in Japanese.
Journal of the Acoustical Society of America. 123, 398-413.
PDF 589KB Everdell,
I.T., Marsh, H., Yurick, M.D., Munhall, K.G. & Paré, M.
(2007). Gaze behaviour in audiovisual speech perception: Asymmetrical
distribution of face-directed fixations. Perception, 36, 1535-1545.
PDF
311 KB Buchan,
J.N., Paré, M. & Munhall, K.G. (2007). Spatial statistics
of gaze Fixations during dynamic face processing. Social Neuroscience,
2(1), 1-13. PDF
382KB
Purcell,
D.W. & Munhall, K.G. (2006). Adaptive control of vowel formant
frequency: Evidence from real-time formant manipulation. Journal of
the Acoustical Society of America, 120, 966-977. PDF
287 KB Purcell,
D.W. & Munhall, K.G. (2006). Compensating following real-time
manipulation of formants in isolated vowels. Journal of the Acoustical
Society of America, 119, 2288–2297. PDF
192
KB Jones,
J.A. & Munhall, K.G. (2005). Remapping Auditory Motor Representations
in Voice Production. Current Biology, 15, 1768–1772. PDF
153 KB Callan,
D., Jones, J.A., Munhall, K.G., Kroos, C., Callan, A. & Vatikiotis-Bateson,
E. (2004). Multisensory-integration sites identified by perception
of spatial wavelet filtered visual speech gesture information. Journal
of Cognitive Neuroscience,16, 805-816. PDF
2.74 MB Munhall,
K.G. & Buchan, J. (2004). Something in the way she moves.
Trends in Cognitive Sciences, 8, 51-53. Munhall,
K.G., Kroos, C., Jozan, G. & Vatikiotis-Bateson, E., (2004).
Spatial frequency requirements for audiovisual speech perception. Perception
and Psychophysics, 66, 574-583. PDF
466 KB Santi,
A., Servos, P., Vatikiotis-Bateson, E., Kuratate, T. & Munhall, K.
(2003). Perceiving biological motion: Dissociating talking from walking.
Journal of Cognitive Neuroscience. 15, 800-809. [PDF
(486 KB)] Callan,
D., Jones, J.A., Munhall, K.G., Kroos, C., Callan, A. & Vatikiotis-Bateson,
E. (2003). Neural processes underlying perceptual enhancement
by visual speech gestures. Neuroreport, 14 2213-2218. Nicholson,
K.G., Baum, S., Kilgour, A., Koh, C.K., Munhall, K.G., & Cuddy, L.L.
(2003). Impaired processing of prosodic and musical patterns after right
hemisphere damage. Brain and Cognition, 52, 382-389. [PDF
(234 KB)] Paré,
M., Richler, R., ten Hove, M., & Munhall, K.G. (2003). Gaze
Behavior in Audiovisual Speech Perception: The Influence of Ocular Fixations
on the McGurk Effect. Perception and Psychophysics, 65, 553-567. Munhall,
K.G., Jones, J.A., Callan, D. Kuratate, T., & Vatikiotis-Bateson,
E. (2003). Visual prosody and speech intelligibility: Head
movement improves auditory speech perception.
Psychological Science, 15(2), 133-137. [PDF
143 KB) Jones,
J.A. & Munhall, K.G. (2003). Learning to produce speech
with an altered vocal tract: the role of auditory feedback. Journal
of the Acoustical Society of America, 113(1), 532-543. [PDF
409 KB] Jones,
J. A. & Munhall, K. G. (2002). The role of auditory feedback
during phonation: Studies of Mandarin tone production. Journal of
Phonetics, 30, 303-320. [PDF (233
KB)] Munhall,
K.G., Servos, P., Santi, A. & Goodale, M. (2002). Dynamic
visual speech perception in a patient with visual form agnosia. Neuroreport,
13(14), 1793-1796. [PDF (116
KB)] Nicholson,
K.G., Baum, S., Cuddy, L. & Munhall, K.G. (2002). A Case
Of Impaired Auditory and Visual Speech Prosody Perception After Right
Hemisphere Damage. Neurocase, 8, 314-322.
Pitermann, M. & Munhall, K.G. (2001) An inverse dynamics
approach to face animation. Journal of the Acoustical Society of America,
110, 1570-1580. [PDF (492 KB)] Munhall,
K.G. (2001). Functional imaging during speech production. Acta
Psychologica, 107, 95-117. [PDF (356
KB)] Munhall,
K.G., Kross, C. & Vatikiotis-Bateson, E. (2001). Audiovisual
Perception of band-pass filtered faces. Journal of the Acoustical
Society of Japan, 21, 519-520. Pitermann,
M. & Munhall, K.G. (2001). A face-to-muscle inversion of
a biomechanical face model for audiovisual and motor control research.
Proceedings of ISCA. Nicholson,
K., Baum, S., Cuddy, L. & Munhall, K. G. (2001). A case of
multimodal aprosodia: impaired auditory and visual speech prosody perception
in a patient with right hemisphere damage. Proceedings of AVSP01.
Jones,
J.A. & Munhall, K.G. (2001). Cross-Linguistic Studies of
Fundamental Frequency Control. Proceedings of the Nijmegen speech
motor conference. Munhall,
K.G. (2001). Review of "Coarticulation: Theory, Data, and
Techniques". Journal of the Acoustical Society of America,
109, 19. Jones,
J.A. & Munhall, K.G. (2000). Perceptual Calibration of F0
Production: Evidence from Feedback Perturbation. Journal of the Acoustical
Society of America, 108, 1246-1251. [PDF
(75 KB)] Vatikiotis-Bateson,
E.,Kuratate, T., Kroos, C. Munhall, K.G. & Pitermann, M.
(2000). Task constraints on robot realism: The case of talking heads.
Proceedings of IEEE International Workshop on Robot and Human Interaction
(RO-MAN 2000). Osaka, Japan. IEEE. Vatikiotis-Bateson,
E., Kroos, C., Kuratate, T., Munhall, K.G., Rubin, P., & Yehia, H.
(2000). Building talking heads: Production based synthesis of audiovisual
speech. Proceedings of the First IEEE-RAS International Conference
on Humanoid Robots. MIT, Cambridge, MA. Jones,
J.A. & Munhall, K.G. (2000) Perceptual Contributions to Fundamental
Frequency Production. Proceedings of the Fifth Seminar on Speech Production:
Models and Data, Munich, Germany. Munhall,
K.G., Kawato, M., & Vatikiotis-Bateson, E. (2000) Coarticulation
and physical models of speech production. In M. Broe & J. Pierrehumbert
(Eds.) Papers in Laboratory Phonology V. Acqusition and the Lexicon.
Cambridge University Press. Vatikiotis-Bateson,
E., Kuratate, T., Munhall, K. G., & Yehia, H. C. (2000).
The production and perception of a realistic talking face. In O. Fujimura,
B. D. Joseph, & B. Palek (Eds.), Proceedings of LP'98, Item order
in language and speech 2 (pp. 439-460). Prague: Charles University
(Karolinum Press). Lucero,
J.C. & Munhall, K.G. (1999) A model of facial biomechanics
for speech production. Journal of the Acoustical Society of America,
106, 2834-2842. [PDF (255 KB)] Flanagan
JR, Jakobson LS, Munhall KG. (1999) Anticipatory grip adjustments
are observed in both goal-directed movements and movement tics in an individual
with Tourette’s syndrome. Experimental Brain Research,
128, 69-75. [PDF (157 KB)] Gracco,
V. & Munhall, K.G. (1999) The neurophysiology of speech.
In F. Fabbro (Ed.) Concise Encyclopedia of Language Pathology.
New York: Elsevier Science Publishers. Kuratate,
T., Munhall, K.G., Rubin, P.E., Vatikiotis-Bateson, E., & Yehia, H.
(1999). Audio-visual synthesis of talking faces from speech production
correlates. Proceedings of EuroSpeech’99, ESCA. [PDF
(375 KB)] Hoole,
P., Munhall, K.G., Mooshammer, C. (1998) Do air-stream mechanisms
influence tongue movement paths? Phonetica, 55, 131-146. Munhall,
K.G. & Tohkura, Y. (1998) Audiovisual gating and the time
course of speech perception. Journal of the Acoustical Society of
America, 104, 530-539. [PDF
(196 KB)] Vatikiotis-Bateson,
E., Eigsti, I.-M., Yano, S., & Munhall, K. G. (1998). Eye
movement of perceivers during audiovisual speech perception. Perception
& Psychophysics, 60(6), 926-940. [PDF
(245 KB)] Leeper,
H.A., Tizzington, L., & Munhall, K.G. (1998). Temporal characteristics
of velopharyngeal function in children. The Cleft Palate Journal:
An International Journal of Craniofacial Anomalies, 35, 215-231.
Munhall,
K.G., & Vatikiotis-Bateson, E. (1998). The moving face during
speech communication. In R. Campbell, B. Dodd & D. Burnham (Eds.),
Hearing by Eye, Part 2: The psychology of speechreading and audiovisual
speech. London: Taylor & Francis, Psychology Press. Jones,
J. A. and Munhall, K. G. (1997). The effects of separating auditory
and visual sources on audiovisual integration of speech. Canadian
Acoustics, 25, 13-19. Campbell,
R., Zihl, J., Massaro, D., Munhall, K., & Cohen, M. (1997).
Speechreading in the akinetopsic patient, LM. Brain, 120, 1793-1803.
[PDF (300 KB)] Lucero,
J.C., Munhall, K.G., Gracco, V.L., and Ramsay, J.O. (1997). On
the registration of time and the patterning of speech movements. Journal
of Speech and Hearing Research, 40, 1111-1117. Ramsay,
J., Munhall, K.G., Gracco, V. & Ostry, D.J. (1996). Functional
data analysis of lip motion. Journal of the Acoustical Society of
America, 99, 3718-3727. Munhall,
K.G., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal
constraints on the McGurk Effect. Perception and Psychophysics,
58, 351-362. [PDF
(211 KB)] Vatikiotis-Bateson,
E., Munhall, K.G., Hirayama, M., Lee, Y.V., & Terzopolous, D.
(1996). Dynamics of facial motion in speech: Kinematic and electromyographic
studies of orofacial structures. In D.G. Stork & M. Hennecke (Eds.),
Speechreading by Humans and Machines: Models, Systems & Applications.
Kluwer Academic Publishers. Vatikiotis-Bateson,
E., Munhall, K.G., Kasahara, Y., Garcia, F., and Yehia, H. (1996).
Characterizing audiovisual information during speech. In Proceedings
of International Conference on Spoken language Processing (ICSLP-96).
Philadelphia PA. Gracco,
V., Löfqvist, A., Ramsay, J., Munhall, K., & Ostry, D.
(1996). Characteristics of Speech Movements. In Proceedings of 4th
Speech Production Seminar: Models and Data. Autrans, France. Vatikiotis-Bateson,
E., Munhall, K.G., Hirayama, M., Kasahara, Y., & Yehia, H.
(1996). Physiology-based synthesis of audiovisual speech. In Proceedings
of 4th Speech Production Seminar: Models and Data. Autrans, France.
Lucero,
J.C., Maciel, S.T.R., Johns, D.A. & Munhall, K.G. (2005).
Empirical Modeling of human face kinematics during speech using motion
clustering. Journal of the Acoustical Society of America, 118,
405-409. PDF
109 KB