|
ERC Proof of Concept Grant, MAP: "Music-assisted
programmes: Developing communication in autism spectrum disorder
through music making." 2019-2021, European Research Council, €150,000.
Role: PI.
My ERC Proof of Concept project aims to develop a set of
multifaceted, individualized, easily implemented, music-assisted
intervention programmes (MAP) to increase spoken language ability in
2-5-year-old, nonverbal or minimally verbal children with ASD.
ERC Starting Grant, CAASD: "Cracking the pitch code in
music and language: Insights from congenital amusia and autism spectrum
disorders." 2016-2022, European Research Council, €1,488,814. Role: PI.
My ERC project aims to investigate the underlying
mechanisms of pitch processing in language and music through
comparative studies of two neurodevelopmental disorders: CA –
congenital amusia, and ASD – autism spectrum disorders. The two
disorders demonstrate intriguing differences in music, language,
emotion, pitch, memory, and cognitive processing that may provide us
with a unique opportunity for examining the underlying mechanisms of
normal music and language functioning. My central hypothesis is that
normal linguistic, musical, and emotional functioning requires a
delicate balance in the encoding and decoding of form and function in
each domain, with musical communication centred on form and linguistic
and emotional communication focused on function. With three Ph.D.
students, three post-docs, and my collaborator in China (Cunmei Jiang, Shanghai Normal University),
we will conduct a series of behavioural and EEG experiments to study
the cognitive and neural bases of music/language deficits in CA and
ASD, and how language background affects those characteristics.
Pitch processing in typical and atypical populationsSince
2008, I have been investigating pitch processing in speech and music in
individuals with congenital amusia, a neurodevelopmental disorder of
musical processing, in collaboration with Cunmei Jiang (Shanghai Normal University, China), Aniruddh D. Patel (Tufts University, USA), Peter Pfordresher (University at Buffalo, USA), Lauren Stewart (Goldsmiths, University of London, UK), Bill Thompson (Macquarie University, Australia), and Yufang Yang
(Chinese Academy of Sciences, China). Given that individuals with
congenital amusia can neither sing in tune nor detect an out-of-tune
note in a melody, but rarely report language problems, it has been
hypothesized that musical pitch processing involves different
mechanisms than linguistic pitch processing in congenital amusia.
However, our recent studies indicate that individuals with congenital
amusia are also impaired in speech intonation processing (Liu et al., 2010), lexical tone perception (Liu et al., 2012a), speech comprehension in both quiet and noise (Liu et al., 2015a), and speech and song imitation abilities relative to normal controls in laboratory conditions (Liu et al., 2013a).
In addition, pitch-processing deficits in congenital amusia appear to
be associated with insensitivity to the direction of pitch movement in
language and music (Liu et al., 2012b), with pitch thresholds of individuals with congenital amusia significantly dependent on task demands (Williamson et al., 2012).
Together, our findings suggest that congenital amusia is a
domain-general pitch-processing deficit, and that pitch processing in
language and music involves shared mechanisms.
In collaboration with Cunmei Jiang,
I have also started examining pitch processing in speech and music in
individuals with Autism Spectrum Disorders (ASD). We recently conducted
the first study to investigate processing of speech intonation and
melodic contour in Mandarin-speaking individuals with high-functioning
autism (HFA). Results indicate that Mandarin-speaking individuals with
ASD demonstrated inferior prosodic processing but superior musical
processing compared to matched neurotypical controls (Jiang et al., 2015),
which is in direct contrast to the findings of linguistic (relatively
intact) and musical (severely impaired) processing in congenital amusia.
Cognitive and neural bases of pitch processing
At the Chinese University of Hong Kong, I worked with Patrick Wong
on the cognitive and neural bases of pitch processing in congenital
amusia using auditory training, EEG, and fMRI experiments. In one
study, we investigated whether musical processing deficits in
congenital amusia can be treated through auditory training in pitch
direction identification via a scaffolding, incremental learning
approach (Liu et al., 2017). We designed and implemented an
adaptive-tracking training paradigm to help amusics consciously label
the direction of fine-grained pitch movement in both speech syllables
and piano tones. Compared with those untrained, trained amusics showed
not only improved pitch direction identification thresholds for speech
and music, but also increased melodic contour processing abilities. In
another study, we investigated brainstem representation and behavioral
identification of lexical tones in Cantonese-speaking individuals with
congenital amusia (Liu et al., 2015b).
Measurements of the FFR (frequency-following response) waveforms
revealed no evidence of abnormal brainstem representation of lexical
tones for amusics relative to controls, in terms of timing, frequency,
and amplitude. However, amusics performed significantly worse than
controls on identification of these tones. These findings suggest that
amusics’ subcortical neural responses simply represent acoustic/sensory
properties of the tone stimuli, rather than reflecting their
higher-level pitch-processing deficits.
Pitch production and perception in music and languageMy postdoctoral work supported by the Economic and Social Research Council
in the UK examined the mechanisms of pitch production and
perception in music and language. In our study on intonation processing
in congenital amusia (Liu et al., 2010),
both the amusic and control groups performed better on imitation than
identification of statements and questions in English, suggesting
unconscious pitch processing during imitation. In our speech and song
imitation study (Liu et al., 2013a),
both the amusic and control groups imitate pitches more accurately in
singing than in speaking, suggesting shared pitch-production mechanisms
but distinct requirements for pitch-matching accuracy in language and
music processing. This may be due to the different nature of speech and
music: speech is function-driven while music is form-driven. In
particular, pitch patterns in speech are used for representing
functional contrasts (e.g., lexical tone/stress, focus, sentence
modality, etc.), and as such their execution only needs to satisfy
contrastive adequacy. For music, musical understanding or communication
relies on pitch accuracy and aesthetics, which are obvious aspects to
be perfected in performances. In other words the “form” taken by pitch
patterns acts as a means for communication in speech, but is the
intended end product for music.
Speech prosody in tone versus non-tonal languages
In collaboration with Yi Xu, I have been investigating
the phonetics
and phonology of speech prosody in tone (e.g., Mandarin) versus
non-tonal languages (e.g., English) using speech production and
perception experiments and computational modeling techniques. Our main
findings include:
1. Linguistic functions such as lexical tone (in
Mandarin), word stress (in English), focus (sentence emphasis), and
sentence type (statement vs. question) are encoded in parallel via
language-specific schemes by specifying the pitch target, pitch range,
articulatory strength, and duration of each syllable in spoken
utterances (Liu, 2009; Liu and Xu, 2005; Liu et al., 2013b).
2. For
both tone and non-tonal languages, underlying pitch targets (e.g.,
high, mid, low, rising, falling) are prosodic primitives in speech,
realized through a process of syllable-synchronised target
approximation (Xu et al., 2015).
3. The neutral tone in Mandarin is
not targetless. Rather, it is likely to have a [mid] pitch target plus
a weak articulatory strength. The same is true of unstressed syllables
in English. Such a weak strength can be simulated for both languages
through computational modeling (Liu et al., 2013b; Prom-on et al.,
2012).
4. Focus in Mandarin consists of both on-focus pitch range
expansion and post-focus pitch range compression, which is the same as
in English. While post-focus pitch range is lowered in both statements
and yes-no questions in Mandarin, it is lowered in statements but
raised in yes-no questions in English (Liu and Xu, 2005; Liu et al., 2013b).
5. The syllable is the basic temporal organization structure
that assigns time intervals to both segmental and laryngeal units (Xu
and Liu, 2006, 2007). This explains the nature of coarticulation, time
intervals of segments, and temporal alignments of segmental and tonal
events. It also unites segmental and suprasegmental aspects of speech,
treating them as following the same basic articulatory dynamics (Xu and
Liu, 2012).
ESRC
Postdoctoral Fellowship, "How do musically tone-deaf
individuals produce and perceive pitch targets in speech?", 2009-2010,
Economic and Social Research Council, UK, �92,772. Role: PI.
Between December 2009 and November 2010, I was awarded a
one-year Postdoctoral Fellowship by the Economic and Social Research Council
(ESRC)
in the UK. For my Fellowship project, I conducted a set of experiments
to investigate whether, and to what extent, congenital amusia affects
Mandarin-speaking amusics' tone and intonation processing, speech
versus song imitation, and song imitation with lyrics versus 'la'.
My research outputs can be seen from the following links: http://gtr.rcuk.ac.uk/projects?ref=PTA-026-27-2480-A
http://gtr.rcuk.ac.uk/projects?ref=ES/H023895/1
My Ph.D. dissertation examined how communicative functions, such
as lexical
tone (in Chinese), word stress (in English), focus (sentence emphasis),
and sentence type (statement vs. question), interactively affect
surface pitch contours of spoken utterances in English and Chinese.
Using a comparative approach, I found that in both languages surface
prosodic forms result from articulatory implementation of underlying
pitch targets (ideal production goals) that are associated with
syllables and exhibit different trajectories depending on the
communicative functions conveyed. Specifically, while tonal targets in
Chinese remain unchanged regardless of focus or sentence type, pitch
targets of English stressed syllables vary with the stress pattern of
the word, focus and sentence type conditions of the utterances. For
example, in English statements, 'blackboard' has a [high] pitch target
on the stressed syllable followed by a [low] target on the unstressed
syllable, and 'black board' (a board that is black) has a [low] pitch
target on the unstressed syllable and a [fall] target on the stressed
syllable. In questions, pitch targets of English stressed syllables
change (from [high] or [fall] in statements) to [rise]. In both
languages pitch range of the focused word is expanded in statements as
well as in questions. In Chinese, post-focus pitch range is compressed
and lowered in both statements and questions, although the latter is
smaller in magnitude. In English, post-focus pitch range is compressed
and lowered in statements but compressed and raised in questions.
Overall, my PhD research demonstrates that pitch modulations in Chinese
and English are complicated yet regulated so that the function-carrying
pitch targets are maintained and modified in time, place, and occasion.
|
