Training Programs for Improving Speech Perception in Noise: A Review
Article information
Abstract
Understanding speech in the presence of noise is difficult and challenging, even for people with normal hearing. Accurate pitch perception, coding and decoding of temporal and intensity cues, and cognitive factors are involved in speech perception in noise (SPIN); disruption in any of these can be a barrier to SPIN. Because the physiological representations of sounds can be corrected by exercises, training methods for any impairment can be used to improve speech perception. This study describes the various types of bottom-up training methods: pitch training based on fundamental frequency (F0) and harmonics; spatial, temporal, and phoneme training; and top-down training methods, such as cognitive training of functional memory. This study also discusses music training that affects both bottom-up and top-down components and speech training in noise. Given the effectiveness of all these training methods, we recommend identifying the defects underlying SPIN disorders and selecting the best training approach.
Introduction
Accurate speech perception in everyday life is dependent on the hearing system’s ability to process complex sounds in the presence of background noise. Speech perception in noise (SPIN) is also difficult for children and even some young adults with normal hearing and cognitive abilities. To understand speech in noisy situation, children with normal hearing require a higher signal-to-noise ratio (SNR) than adults. Because the most childhood learning takes place in noisy environments, the emergence of any speech perception disorder in children can result in learning, academic, and communication problems [1]. Adults with difficulty speech understanding in noise, on the other hand, complain of being tired of listening to or hearing something without understanding the meaning, discomfort from background noise, and misunderstanding of conversations in the presence of competing sounds [2-4]. Speech impairment in noisy environments is one of the most important challenges for children with central auditory processing disorder (CAPD) [5], learning disabilities (LD) [6] attention deficit hyperactivity disorder (ADHD) [7], and hearing loss [8] and likewise for the elderly (over 65 years) [9-12]. The neurological and processing mechanisms associated with SPIN include pitch perception, neural coding and decoding of temporal and intensity cues, and cognitive skills. Each of these mechanisms is important for SPIN [13].
Pitch perception is an important indicator in the processing of complex stimuli and SPIN. Speech perception is related to the speaker’s fundamental frequency (F0). The frequency of speech components is used to identify the speaker [14]. Studies have shown that listeners recognize F0 in the presence of ambient background noise and use other cues such as harmonics and formants [15]. Oxenham [16] showed that the ability to receive F0 and pitch is involved in concurrently grouping and segregating speech sounds in normal hearing and people who have hearing loss or those using cochlear implants.
Temporal and intensity cues are related to interaural temporal difference (ITD) and interaural level difference (ILD), respectively. This is the primary condition for distinguishing target auditory data from non-target auditory data and also for normal auditory function [17,18]. Studies have shown that correct localization can help people with normal hearing thresholds to understand conversations with a lower SNR, and also demonstrated that auditory localization improves the SNR by 2 to 3 dB when the nature of the noise and signal are different, but increases the spatial advantage by 10 dB when the noise and signal are homogeneous [19,20]. When the noise and the signal have different frequency textures, the noise causes energetic masking. By contrast, when the noise and the signal of the same type coexist, e.g., target speech in the presence of distorted speech signals, the distracted noise causes both energetic and informational masking. Target signal processing in the presence of masking noises, particularly informational masking, necessitates the proper operation of cognitive mechanisms [21].
The main cognitive functions are attention, short-term memory, and working memory [3]. In cases such as hearing loss and the lack of temporal spectral encoding or sensory input, cognitive functions serve as compensatory mechanisms for the auditory system [22]. Speech perception exhibits a relationship with attention and auditory memory in the presence of noise [23,24]. Studies on patients with auditory attention and memory deficits showed that they had difficulty understanding speech in noise [25,26]. It can be hence concluded that higher levels of cognitive performance through top-down pathways can enhance bottom-up pathways and increase signal quality [3,27]. In addition, some exercises can influence both top-down and bottom-up activities [8].
The modification of the hearing system using educational tasks to strengthen speech comprehension is one of the most important and common areas of research today. Ample evidence suggests that hearing training exercises can help normal hearing and peoples with hearing loss, to improve spectral and temporal properties perception. Auditory training tasks are divided into three main categories: bottom-up, top-down, and mixed exercises (a combination of bottom-up and top-down methods). They aim to ameliorate auditory events comprehension via repetitive listening tasks [28]. Bottom-up tasks focus on the acoustic cues of the signals (i.e., spectral, temporal, and intensity characteristics), and require acoustic identification and differentiation. On the other side, top-down tasks improves signal perception by increasing attention to stimuli and encouraging the use of background information [28].
Various exercises and methods (bottom-up and top-down) have been designed to improve SPIN in different clinical populations, including patients with auditory processing disorder (APD) [29], the elderly [30-32], hearing-impaired people [33], and autism patients [34]. This review study aims to search and analyze all the studies that investigated bottom-up and top-down hearing training programs for improving SPIN in different populations.
Training Programs Based on Cues and Factors Involved in Bottom-up Processes for Improving SPIN
Pitch training
Equivalent to the human sensory perception of sound frequency, the pitch is one of the psychological components of sound, along with loudness and timber, that aid in understanding music, speech perception, and sound separation in the presence of competing sound sources. Understanding the pitch of compound sounds depends on the discovery of the F0 and its harmonics and the periodicity of sounds [35]. Harmonics, along with other sound properties (spectral properties, fundamental frequency, synchronicity, or asynchronous at stimulus onset and offset), contribute to the formation of a hearing object by grouping/separating consecutive and concurrent sounds [36]. Because pitch is an important component of SPIN, various studies have investigated the training programs for improving pitch perception, including fundamental frequency [37,38] and harmonics trainings [8].
Fundamental frequency (F0) training
Changing the F0 is one of the methods examined in various studies on speech materials to improve pitch differentiation. Vowel differentiation has been used in F0 training studies. Vowels are components of verbal signals that contain a lot of information about F0, low-frequency components (first formant), and second formant. The acoustic information obtained from these formant features carries phonological information and, along with consonants, is considered a basis for speech comprehension [39]. The effect of F0 differences on the separation of concurrent sound sources is studied using concurrent vowel pairs [37,40]. These results show that increasing F0 differentiation between vowel and target speech from competitive speech improves speech perception [38].
Vowel interventions aim to improve the ability to distinguish between vowels. Meaningless syllables are frequently used in this training method to reduce the effects of top-down mechanisms (based on the schema and prior knowledge) and enhance vowel comprehension, as the basis of speech transmission and perception, by nourishing bottom-up pathways more accurately. Vowels in various formats are presented during vowel training [41]. A review of the literature revealed that two studies have specifically investigated the vowel training interventions on hearing-impaired children [34] and the elderly with normal hearing [42]. A training pattern proposed by Talebi, et al. [33] involved teaching five vowel sounds (/æ/, /e/, /ɒː/, /i:/, and /u:/) in a meaningless pattern of monosyllabic vowels including unvoiced syllables as /pæ/, /shæ/, /sæ/, /hæ/, and /kæ/. This training method is repeated for other sounds. The syllables were spoken behind the hearing impaired children, and they were asked to identify and produce syllables verbally. At the end of the training, the effect of this type of training on double-vowel separation was evaluated both behaviorally and electrophysiologically, and the results showed the effects of vowel training on concurrent sound separation [32,33]. There was a difference between pairs of vowels in their fundamental frequency [43]. Heidari, et al. [42] conducted a vowel training program based on hearing exercises for the elderly. The training program involved teaching six vowels (/æ/, /e/, /a/, /i/, /o/ and /u/) as meaningless monosyllabic vowels for a period of 5 weeks. For example, the monosyllabic vowels /pæ/, /ʃæ/, /sæ/, /hæ/, and /kæ/ were spoken at a distance of one meter behind the person at a hearing comfortable level. This process was repeated for the vowels /e/, /a/, /i/, /o/ and /u/. This training program also improved SPIN among the elderly.
Harmonic training
Mistuning harmonics is a behavioral [8] and electrophysiological [44] method of evaluating concurrent sound separation. It is assumed that when one of the harmonic components is mistuned by 3% of its harmonic value, it is heard as a separate tone from the complex tone. Mistuned harmonics produce and perceive beats as a result of amplitude modulation in the temporal envelope of the stimulus waveform. Increasing the mistuning rate from 3% to 16% and the duration to more than 50 milliseconds results in a decrease in the detection threshold of beats [36]. Moossavi, et al. [8] tested the ability of several hearing-impaired children to perceive and distinguish harmonics and, for the first time, designed a new method of harmonic differentiation training. According to the prevalence of the fundamental frequency of human speech sounds in men (100-146 Hz) and women (188-221 Hz) [45], they employed a complex tone of 100, 200, and 300 Hz with their first 10 harmonics and mistuned one of the first harmonics each time by 2%, 4%, 8%, and 16%. Then they asked the children to compare the complex tone with the tuned harmonic and the mistuned harmonic. The training was performed at frequencies of 100, 200, and 300 Hz in the first, third, fifth, seventh, and ninth harmonics after obtaining the mistuning differentiation threshold in each harmonic of the complex tone. The training began with the lowest level of detection in each harmonic and increased by 70% of the difficulty level if differentiated. The study findings revealed that this training improved the performance of hearing-impaired children in the frequency pattern sequence test and consonant-vowel perception in noise [8].
Localization training
For sound localization, the human auditory system employs two delicate cues. Horizontal level localization is based on ITD for sounds below 1,500 Hz, and ILD and spectral cues for sounds above 2,500 Hz. These spatial and spectral cues may be important for spatial stream segregation, which aids in speech perception [46]. ITD is more important to sound localization [47]. The ITD in unmodulated signals is only processed up to 1,500 Hz and is referred to as the ITD fine structure (ITD FS) [48,49]. A slow carrier (low frequency) modulates the ITD information at higher frequencies, and this is known as ITD envelope (ITD ENV) [50]. Speech, as a modulated signal, contains two types of ITD: ITD FS and ITD ENV [51].
Wright and Fitzgerald [52] evaluated the effectiveness of training based on the simulation of ITD and ILD cues under headphones and reported the improved ability of participants to diagnose ITD and ILD after training. Furthermore, a study by Kuk, et al. [53] demonstrated the reliability of the localization training program in people with hearing loss. Sound Storm (previously known as the Listen & Learn Auditory Training Software) is a training program designed and evaluated by Cameron, et al. [54] to improve the binaural processing skills of children with spatial processing disorder (SPD). SPD is defined as the inability of individuals to use binaural cues to selectively pay attention to sounds coming from one direction while suppressing sounds coming from other directions. As a result, children with SPD struggle to understand speech in noisy environments, such as classrooms. The Sound Storm software creates a three-dimensional hearing environment through headphones. The child is required to repeat a word, including the target, which appears in the background noise. A comparative method according to the patient’s response determines the level of sentence intensity. In their study, they used the Sound Storm auditory training software to train 9 suspected SPD children who performed abnormally in the speech-innoise assessment (LiSN-S PGA) for 15 minutes per day for a period of 12 weeks. They examined the effect of the intervention after three months and found that the speech perception threshold improved by 10 dB in the Sound Storm auditory training software [54].
Lotfi, et al. [29] studied the effectiveness of an auditory localization rehabilitation program based on ITD cues on spatialized speech in noise and monaural low redundancy tests in a group of children with suspected APD. After 12 training sessions, the mean speech perception score was significantly better in the experimental group in comparison to the control group. Lotfi, et al. [30] also investigated the effect of a spatial processing training program called the Persian spatialized speech in noise test on speech perception skills in the elderly with normal hearing who complaint of difficulty with SPIN. Their findings revealed that the SNR significantly reduced to 50% in speech perception after the training, whereas auditory spatial rehabilitation could assist the older adults in benefiting from the spatial difference of sound sources and noise for better speech perception.
Recent investigations have dealt with the role of ITD ENV in spatial hearing and SPIN [55,56]. Majdek, et al. [51] confirmed the role of ITD ENV in speech localization and understanding in noisy environments. In a study by Delphi, et al. [31] stimulus ENV in various ITDs were presented to the elderly with normal hearing. The ENV was designed in 10-millisecond steps from 10 to 100 milliseconds and 50-millisecond steps from 100 to 350 milliseconds. ITD ENV was initially implemented according to the results of previous assessments and then it was gradually reduced. Each ITD ENV stimulus was presented again and again until the participant correctly identified it. SPIN was re-evaluated at the end of the training to measure the effectiveness of the training. The results revealed that the ITD ENV training not only improved the localization abilities of participants but also increased their mean score of SPIN.
Temporal training
The temporal processing is a broad category of auditory processing abilities that includes temporal sequencing, temporal integration, temporal resolution, and temporal masking [32]. According to perceptual and neurophysiological studies, temporal processing skills are more affected by age than other central auditory processing skills. There are numerous biological reasons for the auditory system’s poor functioning in the elderly, including decreased myelin integrity, nerve longterm recovery, decreased brain connections, and decreased neural synchronicity [57]. Inability to process time can be a major cause of language disorders. For example, autism has been linked to temporal processing disorders and their relationship with SPIN [34]. Because of the importance of time processing, it is necessary to develop training approaches based on this skill. Temporal training programs in the form of formal and organized homework, and even computer games, have been used in studies with various goals [58]. One of the most important goals of temporal processing training is to improve time and language processing skills, as well as speech perception in noisy environments.
Ramezani, et al. [34] assessed the effectiveness of temporal processing based rehabilitation on sequencing skills and temporal resolution using temporal pattern detection and noise detection exercises on the speech perception in noise ability and on the speech ABR components in 28 autistic adolescents. Their findings showed a significant improvement in SPIN and the efficiency of time processing at the auditory brainstem level (reduction of wave latency) in response to speech signals. Sattari, et al. [32] also designed the rehabilitation tasks related to auditory temporal processing in 5 domains, including detection of the number of stimuli, pitch detection, detection of duration patterns, detection of the number of meaningless speech stimuli in noise, and gap detection in noise, to improve SPIN in the elderly aged 75-60 years who were using hearing aids. In another study, Rasouli Fard, et al. [59] separated the fine temporal structure and the stimuli envelope of vowel-consonantvowel (VCV) and then designed a training method based on the fine temporal structure and differentiation of 16 consonants. Their results showed an improvement in SPIN among the elderly with mild to moderate hearing loss.
Phonemic training
Bottom-up processes (e.g., as sensory speech processing) and top-down processes (e.g., selective attention, short- and long-term memory, and the use of lexical and textual concepts) are always involved in the process of speech perception. Both types of processes are constantly interacting with one another and are inseparable. By reducing semantic information (content and context), individuals rely on phonetic features (such as formant frequencies and voice onset time) for speech perception [60]. However, it has been reported that bottom-up training protocols improve everyday listening ability less than topdown training [61]. Schumann, et al. [60] reported that a computer-based listening training program, which included phonological differentiation using VCV and consonant-vowelconsonant (CVC), was effective in improving SPIN in individuals using cochlear implants. As the most common disorder in auditory processing, phonological differentiation training is one of the components of defect decoding training in the Buffalo model of therapy for APDs [62]. This training program, which is known as phonemic training program, aims to improve speech comprehension, reading ability, auditory spelling improvement, and speech clarity. According to the results of phonemic error analysis, this training program begins with easier tasks, such as providing phonemes that are easier to perceive for participants, and gradually increases in difficulty as auditory processing goes on. Practices in this training program are based on the repetition and differentiation of learned phonemes [39].
Training Methods for Improving SPIN Based on Factors Involved in Top-Down Processes
Memory-based training
Many studies have extensively dealt with cognitive training in recent years and demonstrated its remarkable effects on improving academic performance [63], and language learning [64], and lowering the risk of dementia [65]. Cognitive training includes attention, functional memory, and short-term memory reinforcement [3]. However, more emphasis is usually placed on improving functional memory [63,66]. As part of the cognitive process, memory related to the reception, process and store of verbal stimuli, and eventually recall what have been heard. Auditory memory serves as the foundation for the development of language skills (including the ability to learn and memorize words as well as the ability to understand and apply grammar, spoken language, and written language) and the learning process. As a result, language could not be imagine without memory [67]. More extensive functional memory capacities are associated with improved academic performance and language learning as well as a lower risk of pathological aging [63,66]. After functional memory training, the elderly with unknown cognitive impairment showed an increase in functional memory capacity [68,69]. Functional memory has also been shown to play a role in listeners’ ability to understand speech in noisy environments. Among people with normal hearing and those with hearing impairments, listeners with greater functional memory capacity appear to be more successful in understanding speech in noise. Greater functional memory capacity predicts greater success in speech recognition in noise in older people who use hearing aids [70,71]. Because functional memory capacity predicts the success of SPIN, it can be suggested that cognitive training and increasing listeners’ functional memory capacity can be a basis for developing an effective intervention. Many functional auditory memory training tasks rely on straightforward test items including numbers, letters, or monosyllabic words. In addition, since these assignments do not have a high semantic load, they are relatively strong interventions for improving hearing loss [72]. Ingvalson, et al. [73] reported that 10 days of reverse digit training significantly improved reading comprehension and SPIN among native Mandarin Chinese speakers and native Chinese English speakers. Therefore, this type of training can be used to improve SPIN.
Music training
Music is a source of pleasure, learning, and well-being. Prolonged exposure to music creates plasticity from the cochlea to the auditory cortex [74]. Several studies have shown that exposure to music can lead to the transfer of specific listening proficiencies to non-musical domains [74,75]. The theories presented by Patel shows that music education increases adaptive flexibility in speech processing neural networks [76]. It also reinforces numerous auditory processing components involved SPIN among musicians, including syllable differentiation [77,78], speech temporal signals processing [78], prosody [79], pitch and harmonics [80], melodic contour [81], and auditory cognition such as working memory [82,83], attention [84,85], and neural representation of speech [86,87].
Zendel and Alain [88] employed ERP responses to investigate the separation of concurrent sounds in musicians with a mean age of 28 years. N1 and P2 waves were used as evidence to study the music training effectiveness on the separation process. The findings demonstrated the musicians’ exceptional ability to separate concurrent sounds. The researchers in this study hypothesized that training methods, such as music therapy, could have a significant effect on the separation of sounds presented concurrently, ultimately improving sound perception (especially speech). In another study conducted by Swaminathan, et al. [89], the spatial release of noise was higher among musicians than non-musicians. Parbery-Clark, et al. [90] demonstrated improvement in speech perception in both multi-talker and continuous pseudo-speech noise in musicians.
Considering the benefits of music in improving the performance of bottom-up and top-down auditory processing as well as the better performance of musicians in SPIN, Slater, et al. [91] used a long-term music training (2 years) to improve children’s ability to understand speech in noisy environments. Jain, et al. [92] performed an 8-day music training for adults aged 18-25 years and observed an improvement in SPIN. It has also been shown that short-term music training in the elderly improves neural speech coding and facilitates SPIN [93]. Jayakody [94] examined the effects of a computer-based music perception training program and concluded that this program improved music comprehension and speech perception in people using cochlear implants or hearing aids, more significantly in those who were using hearing aids.
Speech-in-noise training
Speech-in-noise training (SINT), which focuses on improving listening skills in noisy environments, is an important component of an auditory training program for the hearing impaired and a wide range of individuals with hearing impairment, learning disabilities, and LD [95]. Various speech materials, including syllables, words (monosyllabic and bisyllabic), sentences, phrases, and texts, at specific SNRs can be used in SINT. Such methods, which were first described by Katz and Burge [96] employed noise desensitization techniques to improve SPIN. This method increases the level of noise tolerance and memory and, as a result, improves SPIN. This technique helps participants desensitize to noise and develop strategies to obtain more information about auditory signals in difficult-to-hear situations [56]. This training program involves listening to speech stimuli at different levels of noise, because exposure to a gradually increasing level of noise reduces the effect of noise. In this program, the words are first presented in silence and then in noise, and the SNR is gradually increased from easy (e.g., +12 or +15 dB) to difficult (0 dB or negative SNRs) conditions [97]. Masters propose that a treatment hierarchy begins with white noise and ends up with the most problematic type of noise for the client. It is also suggested that such interventions take into account the type of noises that one is more exposed to in the surrounding environment [98]. Several studies have reported the effectiveness of noise speech training, particularly the noise desensitization method. According to Jutras, et al. [99], children with APD exhibited a significant increase in noise tolerance after receiving noise-specific speech training. They investigated how SINT affected speech perception test scores, electrophysiological criteria, and listening and living habits in the child’s real-life conditions and environment. Speech perception test scores and electrophysiological components improved due to this rehabilitation. Furthermore, improvement was not limited to noisy environments but it happened in other challenging situations of everyday life [99]. Kumar, et al. [95] designed a computerized tasks for speech in noise training and evaluated its effectiveness using auditory processing behavioral tests and auditory electrophysiological responses. This computer-based program (module) included teaching words in noise, in which single and three-syllable words were presented as target words in the presence of spoken noise and multi-talker babble at various SNRs ranging from +20 to -4 dB. The performance of experimental children in speech noise and auditory processing was better than the control group. In addition, after training, the range of evoked responses with high latency in silence and noise decreased significantly. Their study found that a targeted program of SINT improved auditory behavioral skills and electrophysiological responses. Another study in 2021 investigated the effects of a noise word training program using monosyllabic words in the presence of spoken and multi-speaker noise in the SNR range of +20 to 4 dB on processing and cognitive skills (working memory) of 20 children with APD (10 children in the experimental group and 10 children in the control group). The results of this study showed that SINT significantly improved the mean SPIN score, temporal processing (gap in noise test and duration pattern), and the backward digit’s span score [100].
To design speech tasks in noise, it is recommended that noise be used in conjunction with the speech signal in a systematic manner, beginning with energetic noise and progressing to informational noise. Energetic masking is associated with noise that has a high-frequency compression and no gap between the energy spectrums produced, such as white noise. In this case, it seems that the neural networks in charge of signal and noise processing are sensitive to the all energy level of the signal and noise, and thus the signal with a slightly higher absolute energy level than the noise is easily detected. Speech comprehension in a noisy environment will be possible with lower SNRs and the use of pitch and spatial cues. The difficulty and the challenging level of the auditory environment are controlled and adjusted in this type of test based on SNR. Providing speech in the presence of informational masking, on the other hand, targets a different level of ability of the neural networks responsible for signal processing in noise. Distracting noises of the signal type are referred to as informational masking. When speech is in the speech noise, a person’s ability to understand usually suffers. It is preferable to begin with multiple speech streams and end up with two speech streams (from easy to hard). In the presence of informational noise, the auditory system typically uses signal monitoring in the ears at any frequency and at any time, according to theories of speech signal detection. In other words, it is assumed that our auditory system searches for target signal among the noise gaps or tries to collect the target signal from the better ear with higher SNR, an approach which is known as cross-ear dip-listening [101-104].
Conclusion
Difficulty with SPIN is a common condition affecting all children and adults with central hearing impairment, children with LD, ADHD, autism, and hearing loss, and the elderly. Each person with this problem suffers from a defect in one or more of the underlying neural mechanisms of SPIN, which can be improved through appropriate training programs. Pitch training, spatial training, temporal training, phoneme training, functional memory training, musical training, and SINT are all used to improve SPIN. Pitch training is based on fundamental frequency and harmonics. Changing the fundamental frequency is one of the methods to improve pitch differentiation based on vowel differentiation at various fundamental frequencies. Mistuning complex tone harmonics and distinguishing tuning stimuli from mistuning from the base of harmonic training. Localization training involves simulating the space environment with headphones and investigating localization disorders in various ITDs. The detection of the number of stimuli, pitch stimuli, duration pattern, number of meaningless speech stimuli in noise, and the gap in noise is the central part of temporal training, whereas phoneme training focuses on meaningless syllables of VCV and CVC. Memory training is performed for patients with poor memory using simple stimuli, such as numbers and letters. Music training has been shown to have a significant effect on SPIN by improving the reception of acoustic stimuli and involving top-down processes of attention and memory. Finally, SINT using stimuli, such as syllables, words, and sentences, in energetic and informational background noise has been shown to improve SPIN.
Notes
Conflicts of Interest
The authors have no financial conflicts of interest.
Author Contributions
Conceptualization: Nasrin Gohari, Zahra Hosseini Dastgerdi. Data curation: Nasrin Gohari, Zahra Hosseini Dastgerdi. Formal analysis: Nasrin Gohari, Zahra Hosseini Dastgerdi. Investigation: all authors. Methodology: Nasrin Gohari, Zahra Hosseini Dastgerdi, Nematollah Rouhbakhsh. Project administration: Nasrin Gohari, Zahra Hosseini Dastgerdi. Supervision: Nasrin Gohari, Zahra Hosseini Dastgerdi. Validation:Nasrin Gohari, Nematollah Rouhbakhsh, Sara Afshar, Razieh Mobini. Visualization: Nasrin Gohari, Nematollah Rouhbakhsh, Sara Afshar, Razieh Mobini. Writing—original draft: all authors. Writing— review & editing: Nasrin Gohari, Zahra Hosseini Dastgerdi. Approval of final manuscript: all authors.