Neural Coincidence Detection Strategies during Perception of Multi-Pitch Musical Tones
Abstract
:1. Lead Paragraph
2. Introduction
3. Method
3.1. Listening Test
3.2. Cochlea Model
3.3. Post-Processing of Cochlea Model Output
3.4. Correlation Between Perception and Calculation
4. Results
4.1. ISI Histogram
4.2. Perception of Separateness and Roughness
4.3. Correlation Between Perception and Simulation
Sound | Interval | Roughness Mean | Roughness Std | Separateness Mean | Separateness Std |
---|---|---|---|---|---|
Mean | Std | Mean | Std | ||
Saung Gauk | Major seventh | 3.25 | 1.80 | 4.86 | 2.53 |
Saung Gauk | Major sixth | 3.39 | 1.87 | 4.66 | 2.27 |
Saung Gauk | Major second | 4.00 | 2.64 | 3.72 | 2.31 |
Dutar | Major seventh | 4.93 | 2.23 | 3.79 | 2.16 |
Hulusi | Major third | 6.07 | 2.85 | 3.52 | 2.53 |
Hulusi | Fifth | 5.29 | 3.00 | 2.345 | 1.82 |
Hulusi | Minor third | 5.71 | 2.417 | 3.31 | 2.19 |
Hulusi | Major seventh | 6.07 | 2.90 | 3.17 | 2.14 |
Mbira | Minor third | 2.60 | 1.97 | 3.66 | 2.57 |
Bama | Fundamental | 3.43 | 2.43 | 2.55 | 1.84 |
Big Gong | ∼b2 | ||||
Bama | Fundamental | 2.82 | 1.90 | 2.38 | 1.72 |
Big Gong | ∼g#2 | ||||
Roneat Deik | Major second | 3.71 | 2.27 | 3.28 | 2.31 |
Roneat Deik | Two octaves | 2.61 | 2.04 | 2.17 | 2.16 |
+ major 2rd | |||||
String Pad | Minor second | 4.71 | 2.48 | 3.24 | 2.23 |
String Pad | Four octaves | 3.07 | 1.92 | 7.86 | 2.08 |
+ Major third | |||||
Piano | Fifth | 2.46 | 1.69 | 4.72 | 2.23 |
Strings | Minor seventh | 5.75 | 3.04 | 2.55 | 2.06 |
5. Conclusions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Schneider, A. Pitch and Pitch Perception. In Springer Handbook of Systematic Musicology; Springer: Berlin/Heidelberg, Germany, 2018; pp. 605–685. [Google Scholar]
- von Helmholtz, H. Die Lehre von den Tonempfindungen als Physiologische Grundlage für die Theorie der Musik [On the Sensations of Tone As a Physiological Basis for the Theory of Music]; Vieweg, Braunschweig: Decatur, IL, USA, 1863. [Google Scholar]
- Schneider, A. Perception of Timbre and Sound Color. In Springer Handbook of Systematic Musicology; Springer: Berlin/Heidelberg, Germany, 2018; pp. 687–725. [Google Scholar]
- Bregman, A.S. Auditory Scene Analysis: The Perceptual Organization of Sound; MIT Press: Cambridge, MA, USA, 1994. [Google Scholar]
- Benetos, E.; Dixon, S. Multiple-instrument polyphonic music transcription using a temporally constrained shift-invariant model. J. Acoust. Soc. Am. 2013, 133, 1727–1741. [Google Scholar] [CrossRef] [PubMed]
- Jansson, A.; Bittner, R.M.; Ewert, S.; Weyde, T. Joint singing voice separation and f0 estimation with deep u-net architectures. In Proceedings of the 2019 27th European Signal Processing Conference (EUSIPCO), Coruña, Spain, 2–6 September 2019; pp. 1–5. [Google Scholar]
- McLeod, A.; Schramm, R.; Steedman, M.; Benetos, E. Automatic transcription of polyphonic vocal music. Appl. Sci. 2017, 7, 1285. [Google Scholar] [CrossRef]
- Schramm, R. Automatic Transcription of Polyphonic Vocal Music. In Handbook of Artificial Intelligence for Music. Foundations, Advanced Approaches and Developments for Creativity; Springer: Berlin/Heidelberg, Germany, 2021; pp. 715–736. [Google Scholar]
- Lin, K.W.E.; Balamurali, B.T.; Koh, E.; Lui, S.; Herremans, D. Singing voice separation using a deep convolutional neural network trained by ideal binary mask and cross entropy. Neural Comput. Appl. 2018, 32, 1037–1050. [Google Scholar] [CrossRef]
- Dessein, A.; Cont, A.; Lemaitre, G. Real-time polyphonic music transcription with non-negative matrix factorization and beta-divergence. In Proceedings of the ISMIR—11th International Society for Music Information Retrieval Conference, Utrecht, The Netherlands, 9–13 August 2010; pp. 489–494. [Google Scholar]
- Matsunaga, T.; Saito, H. Multi-Layer Combined Frequency and Periodicity Representations for Multi-Pitch Estimation of Multi-Instrument Music. IEEE/ACM Trans. Audio Sep. 2024, 99, 3171–3184. [Google Scholar] [CrossRef]
- Vincent, E.; Bertin, N.; Badeau, R. Adaptive harmonic spectral decomposition for multiple pitch estimation. IEEE/ACM Trans. Audio, Speech, Lang. Process. 2010, 18, 528–537. [Google Scholar] [CrossRef]
- Grey, J.M.; Moorer, J.A. Perceptual evaluations of synthesized musical instrument tones. J. Acoust. Soc. Am. 1977, 62, 454–462. [Google Scholar] [CrossRef]
- Iverson, P.; Krumhansl, C.L. Isolating the dynamic attributes of musical timbre. J. Acoustic. Soc. Am. 1993, 94, 2595–2603. [Google Scholar] [CrossRef] [PubMed]
- McAdams, S.; Winsberg, S.; Donnadieu, S.; De Soete, G.; Krimphoff, J. Perceptual scaling of synthesized musical timbres: Common dimensions, specifities, and latent subject classes. Psychol. Rev. 1995, 58, 177–192. [Google Scholar]
- Bader, R. Nonlinearities and Synchronization in Musical Acoustics and Music Psychology; Current Research in Systematic Musicology; Springer: Berlin/Heidelberg, Germany, 2013; Volume 2. [Google Scholar]
- Ziemer, T.; Schultheis, H. PAMPAS: A Psycho-Acoustical Method for the Perceptual Analysis of multidimensional Sonification. Front. Neurosci. 2022, 16, 930944. [Google Scholar] [CrossRef]
- Bader, R. Pitch and timbre discrimination at wave-to-spike transition in the cochlea. arXiv 2017, arXiv:1711.05596. [Google Scholar] [CrossRef]
- Haken, H. Brain Dynamics, Springer Series in Synergetics, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
- Baars, B.J.; Franklin, S.; Ramsoy, T.Z. Global workspace dynamics: Cortical binding and propagation enables conscious contents. Front Psychol. 2013, 4, 200. [Google Scholar] [CrossRef] [PubMed]
- Fries, P.; Neuenschwander, S.; Engel, A.K.; Goebel, R.; Singer, W. Rapid feature selective neuronal synchronization through correlated latency shifting. Nat. Neurosci. 2001, 4, 194–200. [Google Scholar] [CrossRef] [PubMed]
- Buhusi, C.V.; Meck, W.H. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 2005, 6, 755–765. [Google Scholar] [CrossRef] [PubMed]
- Hartmann, L. Neuronal synchronization of musical large-scale form: An EEG-study. Proc. Mtgs. Acoust. 2014, 22, 035001. [Google Scholar] [CrossRef]
- Barrie, J.M.; Freeman, W.J.; Lenhart, M.D. Spatiotemporal analysis of prepyriform, visual, auditory, and somesthetic surface EEGs in trained rabbits. J. Neurophysiol. 1996, 76, 520–539. [Google Scholar] [CrossRef] [PubMed]
- Kozma, R.; Freeman, W.J. (Eds.) Cognitive Phase Transitions in the Cerebral Cortex—Enhancing the Neuron Doctrine by Modeling Neural Fields; Springer Series Studies, System, Decision, and Control; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
- Ohl, F.W. On the Creation of Meaning in the Brain—Cortical Neurodynamics During Category Learning. In Cognitive Phase Transitions in the Cerebral Cortex—Enhancing the Neuron Doctrine by Modeling Neural Fields; Springer Series Studies, System, Decision, and Control; Kozma, R., Freeman, W.J., Eds.; Springer: Berlin/Heidelberg, Germany, 2016; pp. 147–159. [Google Scholar]
- Bader, R. Cochlea spike synchronization and coincidence detection model. Chaos 2018, 023105, 1–10. [Google Scholar]
- Joris, X.; Lu, H.-W.; Franken, T.; Smith, H. What, if anything, is coincidence detection? J. Acoust. Soc. Am. 2023, 154, A242. [Google Scholar] [CrossRef]
- Joris, P.X.; Carney, L.H.; Smith, P.H.; Yin, T.C.T. Enhancement of neural synchronization in the anteroventral cochlea nucleus. I. Responses to tones at the characteristic frequency. J. Neurophysiol. 1994, 71, 1022–1036. [Google Scholar] [CrossRef] [PubMed]
- Joris, P.X.; Carney, L.H.; Smith, P.H.; Yin, T.C.T. Enhancement of neural synchronization in the anteroventral cochlea nucleus. II. Responses in the tuning curve tail. J. Neurophysiol. 1994, 71, 1037–1051. [Google Scholar] [CrossRef]
- Kreeger, L.J.; Honnuraiah, S.; Maeker, S.; Goodrich, L.V. An anatomical and physiological basis for coincidence detection across time scales in the auditory system. bioRxiv 2024. [Google Scholar] [CrossRef]
- Stoll, A.; Maier, A.; Krauss, P.; Schilling, A. Coincidence detection and integration behavior in spiking neural networks. Cogn. Neurodyn. 2023, 1–13. [Google Scholar] [CrossRef]
- Schonfield, B.R. Central Decending Auditory Pathways. In Auditory and Vestibular Efferents; Ryugo, D.K., Fay, R.R., Popper, A.N., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 261–290. [Google Scholar]
- Buzsáki, G. Rhythms of the Brain; Oxford University Press: Oxford, UK, 2006. [Google Scholar]
- Ferrante, M.; Ciferri, M.; Toschi, N. R&B—Rhythm and Brain: Cross-subject Decoding of Music from Human Brain Activity. arXiv 2024, arXiv:2406.15537v1. [Google Scholar]
- Thaut, M. Rhythm, Music, and the Brain: Scientific Foundations and Clinical Applications; Routledge: London, UK, 2007. [Google Scholar]
- Dau, T.; Püschel, D.; Kohlrausch, A. A quantitative model of the “effective” signal processing in the auditory system. I. Model structure. J. Acoust. Soc. Am. 1996, 99, 3615–3622. [Google Scholar] [CrossRef] [PubMed]
- Dau, T.; Püschel, D.; Kohlrausch, A. A quantitative model of the “effective” signal processing in the auditory system. II. Simulations and measurements. J. Acoust. Soc. Am. 1996, 99, 3623–3631. [Google Scholar] [CrossRef] [PubMed]
- Lewicki, M.S. Efficient coding of natural sounds. Nat. Neurosci. 2002, 5, 356–363. [Google Scholar] [CrossRef] [PubMed]
- Cariani, P. Temporal Codes, Timing Nets, and Music Perception. J. New Music. Res. 2001, 30, 107–135. [Google Scholar] [CrossRef]
- Cariani, P. Temporal Coding of Periodicity Pitch in the Auditory System: An Overview. Neural Plast. 1999, 6, 147–172. [Google Scholar] [CrossRef] [PubMed]
- Goldstein, J.L.; Gersen, A.; Srulovicz, P.; Furst, M. Verification of the Optimal Probabilistic Basis of Aural Processing of Pitch of Complex Tones. J. Acoust. Soc. Am. 1978, 63, 486–497. [Google Scholar] [CrossRef] [PubMed]
- Bader, R. How Music Works—A Physical Culture Theory; Springer: Berlin/Heidelberg, Germany, 2021. [Google Scholar]
- Sawicki, J.; Hartmann, L.; Bader, R.; Schöll, E. Modelling the perception of music in brain network dynamics. Front. Netw. Physiol. 2020, 2, 910920. [Google Scholar] [CrossRef]
- Bader, R. Modeling Temporal Lobe Epilepsy during Music Large-Scale Form Perception Using the Impulse Pattern Formulation (IPF) Brain Model. Electronics 2024, 13, 362. [Google Scholar] [CrossRef]
- Linke, S.; Bader, R.; Mores, R. Modeling synchronization in human musical rhythms using Impulse Pattern Formulation (IPF). arXiv 2021, arXiv:2112.03218. [Google Scholar]
- Rossing, T. Science of Percussion Instruments; World Scientific: Singapore, 2001. [Google Scholar]
No. | Instrument | Pitches |
---|---|---|
1 | Western guitar | Unisono, g3-g3 |
2 | Western guitar | Minor second, g3-g#3 |
3 | Western guitar | Major second, g3-a3 |
4 | Western guitar | Minor third, g3-b flat3 |
5 | Western guitar | Major third, g3-b3 |
6 | Western guitar | Fourth, g3-c4 |
7 | Western guitar | Tritone, g3-c#4 |
8 | Western guitar | Fifth, g3-d4 |
9 | Western guitar | Minor sixth, g3-e flat4 |
10 | Western guitar | Major sixth, g3-e4 |
11 | Western guitar | Minor seventh, g3-f4 |
12 | Western guitar | Major seventh, g3-f#4 |
13 | Western guitar | Octave, g3-g4 |
14 | Saung Gauk | Minor seventh, f#3-e4 |
15 | Saung Gauk | Minor sixth, f#3-d4 |
16 | Saung Gauk | Major second, f#3-g#3 |
17 | Dutar | Major seventh, f#2-e3 |
18 | Hulusi | Major third, c4-e4 |
19 | Hulusi | Fifth, a3-e4 |
20 | Hulusi | Minor third, e4-g4 |
21 | Hulusi | Major seventh, f#3-e4 |
22 | Mbira | Minor third, c4-e4 |
23 | Bama Big Gong | Fundamental ∼ b2 |
24 | Bama Small Gong | Fundamental ∼ g#2 |
25 | Roneat Deik | Major second, f#4-g4 |
26 | Roneat Deik | Two octaves plus |
Major second, b3-c6 | ||
27 | String Pad | Minor second, d#4-e4 |
28 | String Pad | Four octaves plus |
Major Third, f#2-a#6 | ||
29 | Piano | Fifth, c4-g4 |
30 | String Pad | Minor seventh, c2-b flat2 |
Sound | Roughness Mean | Roughness Std | Separateness Mean | Separateness Std |
---|---|---|---|---|
Unisono | 3.36 | 2.16 | 3.97 | 3.17 |
Minor second | 6.21 | 2.36 | 5.17 | 2.45 |
Major second | 4.79 | 1.87 | 5.66 | 2.45 |
Minor third | 3.89 | 2.06 | 5.79 | 2.57 |
Major third | 4.18 | 1.84 | 6.24 | 2.10 |
Fourth | 2.93 | 1.96 | 5.45 | 2.63 |
Tritone | 4.93 | 2.37 | 5.34 | 2.41 |
Fifth | 3.89 | 2.06 | 5.62 | 2.41 |
Minor sixth | 4.25 | 2.17 | 6.10 | 2.16 |
Major sixth | 4.71 | 1.90 | 5.83 | 2.32 |
Minor seventh | 4.74 | 1.98 | 5.97 | 2.49 |
Major seventh | 5.64 | 2.02 | 7.00 | 2.10 |
Octave | 3.93 | 2.18 | 5.03 | 2.91 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bader, R. Neural Coincidence Detection Strategies during Perception of Multi-Pitch Musical Tones. Appl. Sci. 2024, 14, 7446. https://doi.org/10.3390/app14177446
Bader R. Neural Coincidence Detection Strategies during Perception of Multi-Pitch Musical Tones. Applied Sciences. 2024; 14(17):7446. https://doi.org/10.3390/app14177446
Chicago/Turabian StyleBader, Rolf. 2024. "Neural Coincidence Detection Strategies during Perception of Multi-Pitch Musical Tones" Applied Sciences 14, no. 17: 7446. https://doi.org/10.3390/app14177446