Phase 1 of this project emerged out of a European crucible collaboration with Dr Olga Barrera, a reader in mechanical engineering at Oxford Brookes, Dr Jun Pang, an assistant professor in Computer science at the University of Luxembourg and Dr William Peveler, a lecturer in the school of chemistry, also based at the university of Glasgow. This pilot project was relatively small scale, looking at the possibilities of taking an audiovisual approach to representing data associated with the knee meniscus.
Although this is still very early-stage work, these results suggest that an audiovisual approach has the potential to be extended to larger datasets to explore how similarities and differences can be audibly recognized across multiple samples. The study was recently published in Frontiers in Materials, here, and we hope to apply for another grant to extend this work further.
Phase 2 of the project used longitudinal climate change data in 6 categories:
- Global fossil fuel emissions by type
- Global CO2 emission trends overall
- Reduction in glacier mass
- Global temperature anomalies
- Temperature anomalies by hemisphere and geographical are, and
- Sea level rise.
These datasets were chosen as, although broadly they will have a relatively linear progression, each one presented a slightly different challenge in terms of clarity and legibility.
The final work is intended as a 6-screen expanded audiovisual format installation, but though the visual components are now in place the audio is still being finalised. Through these pieces I have begun to develop a compositional methodology for the sonic elements I describe as composed sonification, in which data is used not only to drive the development of pitched or rhythmic material over time, but also to intervene with that material. This is a process I developed further in the third part of this project.
Phase 3 involved utilising data from the Growing Up in Scotland project. GUS is a longitudinal study, following up 5,217 children born in the 2004/5. The study is led by NatCen Social Research and funded by the Scottish Government. Every 1-2 years children and their families are asked a range of questions including on health, education and relationships.
Mental health data from GUS for the mother and child, respectively, were used to create both the sonic and visual components of the work. Mental health data for the mother was self-reported and comprises two different scales: the Depression, Anxiety and Stress scale (when child is aged 2 and 4 years), and the SF12 mental health component (when child is aged 10 months, 3, 5, and 10 years). Mental health data for the child uses Goodman’s Strengths and Difficulties Questionnaire (SDQ) Total Difficulties score, completed by the parent at ages 4, 5, 6, 8, 10, 12, and 14 years.
In the audiovisualisations, each reported score has a particular pitch and rhythmic pattern associated with it and the mother and child scores, in pairs, are clustered together in groups by colour, with maternal mental health represented by circular shapes and child mental health represented by conical shapes. Increaasing amounts of distortion, dissonance or interference in the audio component, alongside increasing size and motion in the visuals, indicates higher scores on the relevant utilised scale. The most extended version of the work, which I’ll show a little bit of now, involves a 6 screen, 6.1 speaker audiovisualisation intended for expanded audiovisual format installation, in which each screen is representative of a particular urban or rural population area in Scotland, from inaccessible rural to accessible developed urban.
Working through these three very different and distinct datasets over the course of the past year has been complex and difficult at times, but also very illuminating. It has refined my skills in working with data to make it clear and legible in the outcome but also create audiovisual works that are engaging and challenging for the viewer. With these three small examples it’s only really been possible to scratch the surface of what we can do in exploring data audiovisually, but I believe there is significant potential to develop these ideas further and begin to establish some of these methodologies as common practice, both within fields that utilise data and within audiovisual composition.