Search Results (10)

To address the challenges of slow synthesis speed, unstable quality, limited emotional expressiveness, and the lack of controllable speaking rate in Mongolian emotional speech synthesis, this paper proposes a speech Rate-Controllable Mongolian emotional speech synthesis model based on improved Tacotron2 (SRC-IT2). First, an end-to-end Mongolian speech synthesis module is constructed based on an improved Tacotron2 framework, incorporating the unique linguistic characteristics of the Mongolian script. The front-end processing is optimized accordingly, and a G2P-Seq2Seq model is employed to achieve accurate grapheme-to-phoneme conversion for Mongolian characters. Next, on top of the end-to-end synthesis framework, a joint text-audio emotion analysis module is integrated to effectively learn and represent emotional style features specific to Mongolian speech. Finally, a style encoder and speaking rate control variable are embedded into the acoustic modeling process, further enhancing Tacotron2’s ability to dynamically adjust the speaking rate during emotional speech generation. Experimental results demonstrate that the proposed model produces more natural-sounding speech with improved emotional expressiveness and enables effective real-time control over speaking rate in Mongolian emotional speech synthesis. Full article

The assimilation–accommodation theory provides a crucial theoretical framework for understanding the neural mechanisms of second language (L2) processing. Chinese characters, as logographic scripts, contain diverse strokes and components with high visual complexity, and their grapheme–phoneme conversion differs fundamentally from alphabetic writing systems. Existing studies have identified unique neural patterns in Chinese language processing, yet a systematic synthesis of L2 Chinese processing remains limited. This review focuses on the brain mechanisms underlying Chinese language processing among L2 learners with diverse native language backgrounds. On the one hand, Chinese language processing relies on neural networks of the native language (assimilation); on the other hand, it recruits additional right-hemisphere regions to adapt to Chinese characters’ visuospatial complexity and grapheme–phoneme conversion strategies (accommodation). Accordingly, this review first synthesizes current brain imaging studies on L2 Chinese processing within this theoretical framework, noting that prevailing paradigms—limited to lexical and sentence-level processing—fail to capture the complexity, hierarchy, and dynamics of natural language. Next, this review examines the application and implications of naturalistic stimuli paradigms in neuroimaging research of L2 Chinese processing. Finally, future directions for this field are proposed. Collectively, these findings reveal neuroplasticity in processing complex ideographic scripts. Full article

Background/Objectives: Dyslexia, a learning disability affecting reading, has been extensively studied using eye movements. This study aimed to examine in the same design the effects of different psycholinguistic variables, i.e., grammatical category, lexical frequency, word length and orthographic consistency on eye movement patterns during reading in adults. Methods: We compared the eye movements of forty university students, twenty with and twenty without dyslexia while they read aloud a meaningful and a meaningless text in order to examine whether semantic context could enhance their reading strategy. Results: Dyslexic participants made more reading errors and had longer reading time particularly with the meaningless text, suggesting an increased reliance on the semantic context to enhance their reading strategy. They also made more progressive and regressive fixations while reading the two texts. Similar results were found when examining grammatical categories. These findings suggest a reduced visuo-attentional span and reliance on a serial decoding approach during reading, likely based on grapheme-to-phoneme conversion. Furthermore, in the whole text analysis, there was no difference in fixation duration between the groups. However, when examining word length, only the control group exhibited a distinction between longer and shorter words. No significant group differences emerged for word frequency. Importantly, multiple regression analyses revealed that orthographic consistency predicted fixation durations only in the control group, suggesting that dyslexic readers were less sensitive to phonological regularities—possibly due to underlying phonological deficits. Conclusions: These findings suggest the involvement of both phonological and visuo-attentional deficits in dyslexia. Combined remediation strategies may enhance dyslexic individuals’ performance in phonological and visuo-attentional tasks. Full article

Grapheme-to-phoneme conversion (G2P) is the task of converting letters (grapheme sequences) into their pronunciations (phoneme sequences). It plays a crucial role in natural language processing, text-to-speech synthesis, and automatic speech recognition systems. This paper provides a systematical overview of the G2P conversion from different perspectives. The conversion methods are first presented in the paper; detailed discussions are conducted on methods based on deep learning technology. For each method, the key ideas, advantages, disadvantages, and representative models are summarized. This paper then mentioned the learning strategies and multilingual G2P conversions. Finally, this paper summarized the commonly used monolingual and multilingual datasets, including Mandarin, Japanese, Arabic, etc. Two tables illustrated the performance of various methods with relative datasets. After making a general overall of G2P conversion, this paper concluded with the current issues and the future directions of deep learning-based G2P conversion. Full article

The construction of pronunciation dictionaries relies on high-quality and extensive training data in data-driven way. However, the manual annotation of corpus for this purpose is both costly and time consuming, especially for low-resource languages that lack sufficient data and resources. A multilingual pronunciation dictionary includes some common phonemes or phonetic units, which means that these phonemes or units have similarities in the pronunciation of different languages and can be used in the construction process of pronunciation dictionaries for low-resource languages. By using a multilingual pronunciation dictionary, knowledge can be shared among different languages, thus improving the quality and accuracy of pronunciation dictionaries for low-resource languages. In this paper, we propose using shared articulatory features among multiple languages to construct a universal phoneme set, which is then used to label words for multiple languages. To achieve this, we first developed a grapheme−phoneme (G2P) model based on an encoder−decoder deep neural network. We then adopted a near-optimal active learning method in the process of building the pronunciation dictionary to select informative samples from a large, unlabeled corpus and had them labeled by experts. Our experiments demonstrate that this method selected about 1/5 of the unlabeled data and achieved an even higher conversion accuracy than the results of the large data training method. By selectively labeling samples with a high uncertainty in the model, while avoiding labeling samples that were accurately predicted by the current model, our method greatly enhances the efficiency of pronunciation dictionary construction. Full article

This article presents a rule-based grapheme-to-phoneme conversion method and algorithm for Polish. It should be noted that the fundamental grapheme-to-phoneme conversion rules have been developed by Maria Steffen-Batóg and presented in her set of monographs dedicated to the automatic grapheme-to-phoneme conversion of texts in Polish. The author used previously developed rules and independently developed the grapheme-to-phoneme conversion algorithm.The algorithm has been implemented as a software application called TransFon, which allows the user to convert any text in Polish orthography to corresponding strings of phonemes, in phonemic transcription. Using TransFon, a phonemic Polish language corpus was created out of an orthographic corpus. The phonemic language corpusallows statistical analysis of the Polish language, as well as the development of phoneme- and word-based language models for automatic speech recognition using statistical methods. The developed phonemic language corpus opens up further opportunities for research to improve automatic speech recognition in Polish. The development of statistical methods for speech recognition and language modelling requires access to large language corpora, including phonemic corpora. The method presented here enables the creation of such corpora. Full article

An encoder–decoder with attention has become a popular method to achieve sequence-to-sequence (Seq2Seq) acoustic modeling for speech synthesis. To improve the robustness of the attention mechanism, methods utilizing the monotonic alignment between phone sequences and acoustic feature sequences have been proposed, such as stepwise monotonic attention (SMA). However, the phone sequences derived by grapheme-to-phoneme (G2P) conversion may not contain the pauses at the phrase boundaries in utterances, which challenges the assumption of strictly stepwise alignment in SMA. Therefore, this paper proposes to insert hidden states into phone sequences to deal with the situation that pauses are not provided explicitly, and designs a semi-stepwise monotonic attention (SSMA) to model these inserted hidden states. In this method, hidden states are introduced that absorb the pause segments in utterances in an unsupervised way. Thus, the attention at each decoding frame has three options, moving forward to the next phone, staying at the same phone, or jumping to a hidden state. Experimental results show that SSMA can achieve better naturalness of synthetic speech than SMA when phrase boundaries are not available. Moreover, the pause positions derived from the alignment paths of SSMA matched the manually labeled phrase boundaries quite well. Full article

The learning of writing skills involves the re-engagement of previously established independent procedures. Indeed, the writing deficit an adult may acquire after left hemispheric brain injury is caused by either an impairment to the lexical route, which processes words as a whole, to the sublexical procedure based on phoneme-to-grapheme conversion rules, or to both procedures. To date, several approaches have been proposed for writing disorders, among which, interventions aimed at restoring the sub-lexical procedure were successful in cases of severe agraphia. In a randomized double-blind crossover design, fourteen chronic Italian post-stroke aphasics underwent dual transcranial direct current stimulation (tDCS) (20 min, 2 mA) with anodal and cathodal current simultaneously placed over the left and right temporo-parietal cortex, respectively. Two different conditions were considered: (1) real, and (2) sham, while performing a writing task. Each experimental condition was performed for ten workdays over two weeks. After real stimulation, a greater amelioration in writing with respect to the sham was found. Relevantly, these effects generalized to different language tasks not directly treated. This evidence suggests, for the first time, that dual tDCS associated with training is efficacious for severe agraphia. Our results confirm the critical role of the temporo-parietal cortex in writing skills. Full article

Out-of-vocabulary (OOV) words are the most challenging problem in automatic speech recognition (ASR), especially for morphologically rich languages. Most end-to-end speech recognition systems are performed at word and character levels of a language. Amharic is a poorly resourced but morphologically rich language. This paper proposes hybrid connectionist temporal classification with attention end-to-end architecture and a syllabification algorithm for Amharic automatic speech recognition system (AASR) using its phoneme-based subword units. This algorithm helps to insert the epithetic vowel እ[ɨ], which is not included in our Grapheme-to-Phoneme (G2P) conversion algorithm developed using consonant–vowel (CV) representations of Amharic graphemes. The proposed end-to-end model was trained in various Amharic subwords, namely characters, phonemes, character-based subwords, and phoneme-based subwords generated by the byte-pair-encoding (BPE) segmentation algorithm. Experimental results showed that context-dependent phoneme-based subwords tend to result in more accurate speech recognition systems than the character-based, phoneme-based, and character-based subword counterparts. Further improvement was also obtained in proposed phoneme-based subwords with the syllabification algorithm and SpecAugment data augmentation technique. The word error rate (WER) reduction was 18.38% compared to character-based acoustic modeling with the word-based recurrent neural network language modeling (RNNLM) baseline. These phoneme-based subword models are also useful to improve machine and speech translation tasks. Full article

Grapheme-to-phoneme (G2P) conversion is the process of generating pronunciation for words based on their written form. It has a highly essential role for natural language processing, text-to-speech synthesis and automatic speech recognition systems. In this paper, we investigate convolutional neural networks (CNN) for G2P conversion. We propose a novel CNN-based sequence-to-sequence (seq2seq) architecture for G2P conversion. Our approach includes an end-to-end CNN G2P conversion with residual connections and, furthermore, a model that utilizes a convolutional neural network (with and without residual connections) as encoder and Bi-LSTM as a decoder. We compare our approach with state-of-the-art methods, including Encoder-Decoder LSTM and Encoder-Decoder Bi-LSTM. Training and inference times, phoneme and word error rates were evaluated on the public CMUDict dataset for US English, and the best performing convolutional neural network-based architecture was also evaluated on the NetTalk dataset. Our method approaches the accuracy of previous state-of-the-art results in terms of phoneme error rate. Full article