I Choi
Interactive sonification exploring emergent behavior applying models for biological information and listening
Choi, I
Authors
Abstract
Sonification is an open-ended design task to construct sound informing a listener of data. Understanding application context is critical for shaping design requirements for data translation into sound. Sonification requires methodology to maintain reproducibility when data sources exhibit non-linear properties of self-organization and emergent behavior. This research formalizes interactive sonification in an extensible model to support reproducibility when data exhibits emergent behavior. In the absence of sonification theory, extensibility demonstrates relevant methods across case studies. The interactive sonification framework foregrounds three factors: reproducible system implementation for generating sonification; interactive mechanisms enhancing a listener's multisensory observations; and reproducible data from models that characterize emergent behavior. Supramodal attention research suggests interactive exploration with auditory feedback can generate context for recognizing irregular patterns and transient dynamics. The sonification framework provides circular causality as a signal pathway for modeling a listener interacting with emergent behavior. The extensible sonification model adopts a data acquisition pathway to formalize functional symmetry across three subsystems: Experimental Data Source, Sound Generation, and Guided Exploration. To differentiate time criticality and dimensionality of emerging dynamics, are applied between subsystems to maintain scale and symmetry of concurrent processes and temporal dynamics. Tuning functions accommodate sonification design strategies that yield order parameter values to render emerging patterns discoverable as well as , to reproduce desired instances for clinical listeners. Case studies are implemented with two computational models, Chua's circuit and Swarm Chemistry social agent simulation, generating data in real-time that exhibits emergent behavior. is introduced as an informal model of a listener's clinical attention to data sonification through multisensory interaction in a context of structured inquiry. Three methods are introduced to assess the proposed sonification framework: Listening Scenario classification, data flow Attunement, and Sonification Design Patterns to classify sound control. Case study implementations are assessed against these methods comparing levels of abstraction between experimental data and sound generation. Outcomes demonstrate the framework performance as a reference model for representing experimental implementations, also for identifying common sonification structures having different experimental implementations, identifying common functions implemented in different subsystems, and comparing impact of affordances across multiple implementations of listening scenarios.
Citation
Choi, I. (2018). Interactive sonification exploring emergent behavior applying models for biological information and listening. Frontiers in Neuroscience, 12, 197. https://doi.org/10.3389/fnins.2018.00197
Journal Article Type | Article |
---|---|
Acceptance Date | Mar 12, 2018 |
Online Publication Date | Apr 27, 2018 |
Publication Date | Apr 27, 2018 |
Deposit Date | May 25, 2018 |
Publicly Available Date | May 25, 2018 |
Journal | Frontiers in Neuroscience |
Print ISSN | 1662-4548 |
Publisher | Frontiers Media |
Volume | 12 |
Pages | 197 |
DOI | https://doi.org/10.3389/fnins.2018.00197 |
Keywords | biological information, cognitive cycle, emergent behavior, interaction design, listening, media psychology, sonification, supramodal attention |
Publisher URL | https://doi.org/10.3389/fnins.2018.00197 |
Related Public URLs | https://www.frontiersin.org/journals/neuroscience |
Files
Choi Frontiers Sonification Final Accepted with Figures.pdf
(8.4 Mb)
PDF
Licence
http://creativecommons.org/licenses/by/4.0/
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
Version
Final Accepted Version including Figures, Tables and Appendices
fnins-12-00197.pdf
(6.1 Mb)
PDF
Licence
http://creativecommons.org/licenses/by/4.0/
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
Downloadable Citations
About USIR
Administrator e-mail: library-research@salford.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search