AI based algorithms for the detection of (ir)regularity in musical structure

Mihelač, Lorena ; Povh, Janez

International Journal of Applied Mathematics and Computer Science, Tome 30 (2020) no. 4, pp. 761-772.

Résumé

Regularity in musical structure is experienced as a strongly structured texture with repeated and periodic patterns, with the musical ideas presented in an appreciable shape to the human mind. We recently showed that manipulation of musical content (i.e., deviation of musical structure) affects the perception of music. These deviations were detected by musical experts, and the musical pieces containing them were labelled as irregular. In this study, we replace the human expert involved in detection of (ir)regularity with artificial intelligence algorithms. We evaluated eight variables measuring entropy and information content, which can be analysed for each musical piece using the computational model called Information Dynamics of Music and different viewpoints. The algorithm was tested using 160 musical excerpts. A preliminary statistical analysis indicated that three of the eight variables were significant predictors of regularity (E_cpitch, IC_cpintfref, and E_cpintfref). Additionally, we observed linear separation between regular and irregular excerpts; therefore, we employed support vector machine and artificial neural network (ANN) algorithms with a linear kernel and a linear activation function, respectively, to predict regularity. The final algorithms were capable of predicting regularity with an accuracy ranging from 89

Détail

Mots-clés : musical structure regularity, perception, AI algorithms

Bibliographie

[1] [1] Agres, K.R., Abdallah, S. and Pearce, M.T. (2018). Information-theoretic properties of auditory sequences dynamically influence expectation and memory, Cognitive Science 42(1): 43–76.

[2] [2] Arthur, C. (2018). A perceptual study of scale-degree qualia in context, Music Perception 35(3): 295–314.

[3] [3] Bader, M., Schröger, E. and Grimm, S. (2017). How regularity representations of short sound patterns that are based on relative or absolute pitch information establish over time: An EEG study, PlosONE 12(5): e0176981.

[4] [4] Bendixen, A., Schröger, E. and Winkler, I. (2009). I heard that coming: Event-related potential evidence for stimulus-driven prediction in the auditory system, The Journal of Neuroscience 29(26): 8447–8451.

[5] [5] Benward, B. and Saker, M. (2008). Music in Theory and Practice, William Glass, New York, NY.

[6] [6] Bharucha, J.J. (1987). Music cognition and perceptual facilitation: A connectionist frame-work, Music Perception 5(1): 1–30.

[7] [7] Boltz, M.G. (1999). The processing of melodic and temporal information: independent or unified dimensions?, Journal of New Music Research 28: 67–79.

[8] [8] Bouwer, F. and Honing, H. (2012). Rhythmic regularity revisited: Is beat induction indeed pre-attentive?, Proceedings of the 12th International Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece, pp. 122–127.

[9] [9] Burns, E.M. (1999). Intervals, scales, and tuning, in D. Deutsch (Ed.), The Psychology of Music, Academic Press, New York, NY, pp. 215–264.

[10] [10] Busch, R. (1985). On the horizontal and vertical presentation of musical ideas and on musical space (I), Tempo (154): 2–10.

[11] [11] Butler, D. and Brown, H. (1994). Describing the mental representation of tonality in music, in R. Aiello and J.A. Sloboda (Eds), Musical Perceptions, Oxford University Press, New York, NY, pp. 191–212.

[12] [12] Cleary, J.G., Teahan, W.J. and Witten, I.H. (1995). Unbounded length contexts for PPM, Proceedings of the Data Compression Conference, DCC’95, Snowbird, UT, USA, pp. 52–61.

[13] [13] Cleary, J.G. and Witten, I. (1984). Data compression using adaptive coding and partial string matching, IEEE Transactions on Communications 32(4): 396–402.

[14] [14] Dahlhaus, C. (2014). Studies on the Origin of Harmonic Tonality, Princeton University Press, Princeton, NJ.

[15] [15] Edmonds, B. (1995). What is complexity? The philosophy of complexity per se with application to some examples in evolution, The Evolution of Complexity, Brussels, Belgium.

[16] [16] Feldman, J. (1997). Regularity-based perceptual grouping, Computational Intelligence 13(4): 582–623.

[17] [17] Finnas, L. (1989). How can musical preferences be modified? A research review, Bulletin of the Council for Research in Music Education 102: 1–58.

[18] [18] Fritsch, S., Guenther, F. and Guenther, M.F. (2019). Package ‘neuralnet’, https://cran.r-project.org/web/packages/neuralnet/neuralnet.pdf.

[19] [19] Gold, B.P., Pearce, M.T., Mas-Herrero, E., Dagher, A. and Zatorre, R.J. (2019). Predictability and uncertainty in the pleasure of music: A reward for learning?, Journal of Neuroscience 39(47): 9397–9409.

[20] [20] Grassberger, P. (2004). Problems in quantifying self-organized complexity, Helvetica Physica Acta 62(5): 498–511.

[21] [21] Griffiths, T.D., Johnsrude, I., Dean, J.L. and Green, G.G. (1999). A common neural substrate for the analysis of pitch and duration pattern in segmented sound?, NeuroReport 10(18): 3825–3830.

[22] [22] Herbert, R. (2012). Young people’s use and subjective experience of music outside school, Proceedings of the 12th International Conference on Music Perception and Cognition and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece, pp. 423–430.

[23] [23] Holleran, S., Jones, M.R. and Butler, D. (1995). Perceiving implied harmony: The influence of melodic and harmonic context, Journal of Experimental Psychology: LMC 21(3): 737–753.

[24] [24] Huron, D. (2001). Tone and voice: A derivation of the rules of voice-leading from perceptual principles, Music Perception 11(1): 1–64.

[25] [25] Jackendoff, R. (2009). Parallels and nonparallels between language and music, Music Perception 26(3): 195–204.

[26] [26] Jones, M.R. and Boltz, M. (1989). Dynamic attending and responses to time, Psychology Revue 96(3): 459–491.

[27] [27] Justus, T. and Bharucha, J. (2003). Music perception and cognition, in A. Yantis and H. Pasler (Eds), Stevens Handbook of Experimental Psychology, Volume I: Sensation and Perception, Wiley, New York, NY, pp. 453–492.

[28] [28] Kotsiantis, S., Kanellopoulos, D. and Pintelas, P.E. (2006). Handling imbalanced datasets: A review, GESTS International Transactions on Computer Science and Engineering 30(1): 25–36.

[29] [29] Kramer, J.D. (1988). The Time of Music: New Meanings, New Temporalities, New Listening Strategies, Schirmer Books, New York, NY.

[30] [30] Krawczyk, B. (2016). Learning from imbalanced data: Open challenges and future directions, Progress in Artificial Intelligence 5(4): 221–232.

[31] [31] Krumhansl, C.L. (2000). Rhythm and pitch in music cognition, Psychological Bulletin 126(1): 159–179.

[32] [32] Krumhansl, C.L. (2004). The cognition of tonality—As we know it today, Journal of New Music Research 33(3): 253–268.

[33] [33] Lerdahl, F. and Jackendoff, R. (1983). An overview of hierarchical structure in music, Music Perception 1(2): 229–252.

[34] [34] Loui, P. (2012). Learning and liking of melody and harmony: Further studies in artificial grammar learning, Topics in Cognitive Science 4(4): 554–567.

[35] [35] Manjunath, B.S., Wu, P., Newsam, S. and Shin, H.D. (2000). A texture descriptor for browsing and similarity retrieval, Signal Processing: Image Communication 16(1–2): 33–43.

[36] [36] Melara, R.D. and Algom, D. (2003). Driven by information: A tectonic theory of stroop effects, Psychological Review 110(3): 422–471.

[37] [37] Meyer, L.B. (1957). Meaning in music and information theory, Journal of Aesthetics and Art Criticism 15(4): 412–424.

[38] [38] Mihelač, L. (2017). Napovedovanje slušne sprejemljivosti na osnovi entropije harmonije, Master’s thesis, School Center, Novo Mesto.

[39] [39] Mihelač, L. and Povh, J. (2017). Predicting the acceptability of music with entropy of harmony, 14th International Symposium on Operations Research in Slovenia, SOR’17, Bled, Slovenia, pp. 371–375.

[40] [40] Mihelač, L. and Povh, J. (2020). The impact of the complexity of harmony on the acceptability of music, ACM Transactions on Applied Perception 17(1): 1–27.

[41] [41] Mihelač, L. and Povh, J. (2019). The impact of harmony on the perception of music, 15th International Symposium on Operations Research in Slovenia SOR’19, Bled, Slovenia, pp. 360–365.

[42] [42] Mihelač, L., Wiggins, A.G., Lavrač, N. and Povh, J. (2018). Entropy and acceptability: Information dynamics and music acceptance, Proceedings of ICMPC15/ESCOM10, Graz, Austria, pp. 313–317.

[43] [43] Näätänen, R., Paavilainen, P., Rinne, T. and Alho, K. (2007). The mismatch negativity (MMN) in basic research of central auditory processing: A review, Clinical Neurophysiology 118(12): 2544–2590.

[44] [44] Narmour, E. (1990). The Analysis and Cognition of Basic Melodic Structures: The Implication–Realisation Model, University of Chicago Press, Chicago, IL.

[45] [45] Parncutt, R. (1989). Harmony: A Psychoacoustical Approach, Springer, Berlin.

[46] [46] Pauly, M., Mitra, N.J., Wallner, J., Pottmann, H. and Guibas, L.J. (2008). Discovering structural regularity in 3D geometry, ACM Transactions on Graphics 27(3), Article no. 43.

[47] [47] Pearce, M.T. (2005). The Construction and Evaluation of Statistical Models of Melodic Structure in Music Perception and Composition, PhD thesis, City University, London.

[48] [48] Pearce, M.T. (2018). Statistical learning and probabilistic prediction in music cognition: Mechanisms of stylistic enculturation, Annals of the New York Academy of Sciences 1423(1): 378–395.

[49] [49] Pearce, M.T., Müllensiefen, D. and Wiggins, G.A. (2010a). Melodic grouping in music information retrieval: New methods and applications, in Z.W. Raś and A.A. Wieczorkowska (Eds), Advances in Music Information Retrieval, Springer, Berlin, pp. 364–388.

[50] [50] Pearce, M.T., Müllensiefen, D. and Wiggins, G.A. (2010b). The role of expectation and probabilistic learning in auditory boundary perception: A model comparison, Perception 39(10): 1365–1389.

[51] [51] Pearce, M.T., Ruiz, M.H., Kapasi, S., Wiggins, G.A. and Bhattacharya, J. (2010). Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation, NeuroImage 50(1): 302–313.

[52] [52] Pearce, M.T. and Wiggins, G.A. (2012). Auditory expectation: The information dynamics of music perception and cognition, Topics in Cognitive Science 4(4): 625–652.

[53] [53] Pearce, M. and Wiggins, G. (2006). The information dynamics of melodic boundary detection, Proceedings of the 9th International Conference on Music Perception and Cognition, Bologna, Italy, pp. 860–865.

[54] [54] Peretz, I. and Zatorre, R.J. (2005). Brain organization for music processing, Annual Review of Psychology 56: 89–114.

[55] [55] Piotrowska, M., Korvel, G., Kostek, B., Ciszewski, T. and Czyżewski, A. (2019). Machine learning-based analysis of English lateral allophones, International Journal of Applied Mathematics and Computer Science 29(2): 393–405, DOI: 10.2478/amcs-2019-0029.

[56] [56] Platt, J.R. and Racine, R.J. (1994). Detection of implied harmony changes in triadic melodies, Music Perception 11(3): 243–264.

[57] [57] Plaut, D.C. (2000). Methodologies for the computer modeling of human cognitive processes, in F. Boller et al. (Eds), Handbook of Neuropsychology, Elsevier, Amsterdam, pp. 259–267.

[58] [58] Pole, W. (2014). The Philosophy of Music, Routledge, Taylor Francis, Milton Park, Abingdon.

[59] [59] Povel, D.-J. and Jansen, E. (2002). Harmonic factors in the perception of tonal melodies, Music Perception 20(1): 51–85.

[60] [60] Prince, J.B. (2011). The integration of stimulus dimensions in the perception of music, Quarterly Journal of Experimental Psychology 64(11): 2125–2152.

[61] [61] Prince, J.B., Schmuckler, M.A. and Thompson, W.F. (2009a). The effect of task and pitch structure on pitch-time interactions in music, Memory and Cognition 37(3): 368–381.

[62] [62] Prince, J.B., Thompson, W.F. and Schmuckler, M.A. (2009b). Pitch and time, tonality and meter: How do musical dimensions combine?, Journal of Experimental Psychology Human Perception Performance 35(5): 1598–1617.

[63] [63] Rohrmeier, M. (2011). Towards a generative syntax of tonal harmony, Journal of Mathematics and Music 5(1): 35–53.

[64] [64] Rohrmeier, M. and Pearce, M.T. (2018). Musical syntax I: Theoretical perspectives, in R. Bader (Ed.), Springer Handbook of Systematic Musicology, Springer, Berlin/Heidelberg, pp. 473–486.

[65] [65] Schröger, E. and Winkler, I. (1995). Presentation rate and magnitude of stimulus deviance effects on human pre-attentive change detection. viewpoint systems for music prediction, Neuroscience Letters 193(3): 185–188.

[66] [66] Shannon, C.E. (1948). A mathematical theory of communication, Bell System Technical Journal 27(3): 379–423.

[67] [67] Sloboda, J.A. and Parker, D.H.H. (1985). Immediate recall of melodies, in P. Howell et al. (Eds), Musical Structure and Cognition, Academic Press, London, pp. 143–167.

[68] [68] Solomon, J.W. (2019). Music Theory Essentials—A Streamlined Approach to Fundamentals, Tonal Harmony, and Post-Tonal Materials, Taylor Francis, Milton Park, Abingdon.

[69] [69] Steinbeis, N., Koelsch, S. and Sloboda, J.A. (2006). The role of harmonic expectancy violations in musical emotions: Evidence from subjective, physiological, and neural responses, Journal of Cognitive Neuroscience 18(8): 1380–1393.

[70] [70] Steinruecken, C., Ghahramani, Z. and MacKay, D. (2015). Improving ppm with dynamic parameter updates, Data Compression Conference 2015, Snowbrid, UT, USA, pp. 193–202.

[71] [71] Thompson, W.F. and Cuddy, L.L. (1989). Sensitivity to key change in chorale sequences: A comparison of single voices and four-voice harmony, Music Perception 7(2): 151–168.

[72] [72] Tillmann, B., Bharucha, J.J. and Bigand, E. (2000). Implicit learning of tonality: A self-organizing approach, Psychological Review 107(4): 885–913.

[73] [73] Volk, A. (2016). Computational music structure analysis: A computational enterprise into time in music, in M. Müller et al. (Eds), Computational Music Structure Analysis, Vol. 6, Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Leibniz, p. 159.

[74] [74] Wiggins, G.A., Pearce, M.T. and Müllensiefen, D. (2009). Computational modeling of music cognition and musical creativity, in R.T. Dean (Ed.), Oxford Handbook of Computer Music and Digital Sound Culture, Oxford University Press, Oxford.

[75] [75] Williams, L.R. (2005). Effect of music training and musical complexity on focus of attention to melody or harmony, Journal of Research in Music Education 53(3): 210–221.