In June 2021, LxDII moved into a new phase of research with its partner: the Centre for Analysis of Social Exclusion (CASE), London School of Economics. Dr Tania Burchardt and Dr Bert Provan shared their research on the different dimensions of social disadvantage, particularly from longitudinal and neighbourhood perspectives, as well as impacts on public policy.
Our partner Dr Sandra Pauletto, an expert in sonification and sound design, is integral to the work we are doing with CASE. In this blog post, Sandra provides a detailed explanation of the sonification that she has produced from their social exclusion data.
Dr. Sandra Pauletto
Sonification is a design and research area concerned with displaying data (which can come from a variety of domains) with sound so that experts and/or non-experts can perceive and engage through listening with the properties of the data and their meaning.
For example, one of the most well-known sonification example is the Geiger counter where the detection of ionizing radiation is transformed, via an electronic instrument, into a sound signal.
As an inherently time-based medium, sound is particularly well suited to representing time-based data (for example, earthquake data which evolve in time):
as well as data gathered in real-time (for example, data from a muscle performing an action can be sonified as an auditory feedback):
However, other kind of data, for example spatial data, can be rendered into sound too. As people are particularly good at perceiving changes in sound, sonification can alert listeners to anomalies and new events especially when vision is occupied in other tasks. Finally, sonification is a fundamental tool for making data and information accessible to visually impaired users. To give an example, visually impaired astrophysicists use sonification to access and work with data from telescopes:
Within Listening Across Disciplines II, we wanted to explore how a specific data set from the social science domain might be sonified, and whether interesting insights could be derived from this process. The data set Social policies and distributional outcomes in a changing Britain (SPDO) propose a number of indicators (related to poverty, life expectancy, health, education and so on) as a way to measure how inequalities might have changed over time in the UK.
In a number of meetings with our collaborators from social science, we shared and discussed a variety of sonification examples as well as the SPDO data set, its development and current use.
From these discussions a number of realisations came to the surface. We did not anticipate, for instance, the role that the heterogeneous nature of the data set (it includes many different variables that use different measurement scales) would play in the design process, but ultimately it led us to create a number of distinct sonification examples, rather than a coherent single example. Additionally, despite numerous meetings and discussions, we realized that we were not immune from having to deal with the tension – so often found in sonification projects – between attempting to create a sound display that reflects the properties of the data, and our personal expectations for a specific sonic result.
In the end, we produced a series of sonification examples that show how by using different sound characteristics – e.g. duration, rhythmical regularity vs. randomness, pitch, timbre – we might be able to listen to the data from three different perspectives: (1) as a whole, (2) looking overall changes for each main indicator (in this case poverty, or compulsory education, etc.), and finally (3) looking at how each main indicator changes in time depending on characteristics such as gender, ethnicity, etc.
The data we used can be found here.
Summary of the data
As part of the Social policies and distributional outcomes in a changing Britain (SPDO) programme, an indicator set was created to measure inequalities across different outcomes. The aim was to systematically evaluate the progress that has been made in addressing inequalities in relation to key social policy areas, such as health and education, for different groups.
We sonified data from the two spreadsheets provided. The first shows differences in outcomes by ethnicity, age, disability, sex, geographic area and socioeconomic deprivation. The second focuses on change over time – whether outcomes have got better or worse for different groups – using a traffic light colour scheme, for example with red signalling a worsening of outcomes.
Sonification approach
Using different sound to data sonification mappings, we aimed to provide examples that range from the overview to the detail.
We also wanted to create sonification examples that exploit three fundamental sound parameters: rhythm (regularity versus randomness), timbre, pitch.
Three examples of sonification.
- an overview of all the changes in the data between 2010-2015 and 2015-2019
- an overview of the changes for specific social categories (poverty, compulsory education, etc.) between 2010-2015 and 2015-2019
- a representation to how the data changed over time between 2010 and 2019 for each indicator and by factor (e.g. gender, ethnicity, etc.)
These sonifications should be considered initial sketches and ideas, usable to foster discussions about sonification in the context of social data, rather than fully finished displays of data.
An overview of all the changes in the data between 2010-2015 and 2015-2019
The data provided by our collaborators have been gathered using a variety of methods and refer to a number of different indicators (for example, some “Life expectancy” is measured in years, while “Compulsory Education” is measure in percentage of students achieving a specific grade). These numbers are therefore not comparable across the indicators. To overcome this, our collaborators have colour coded each data cell to indicate whether indicators have worsened or not. This additional information allowed us to create a sonic overview of spreadsheet two (which focuses on changes overtime).
We have first visualized the count of worsening, improving, etc. cells in each period and then sonified this graph.
This sonification uses duration and rhythm regularity and density as main parameters for representing number of cells of a particular colour and worsening of the change (colour changing from green to red) respectively. The number of cells without data is sonified with synthesized speech. This sonification has been created using a free real-time synthesizer in Pure Data.
An overview of the changes for specific social categories between 2010-2015 and 2015-2019
In this sonification, we aim to sonify changes between 2010-2015 and 2015-2019 for a single social category (e.g. poverty).
As poverty is describe by three indicators (Relative poverty AHC, Anchored poverty AHC, and Spotlight indicator:
Relative child poverty AHC), we have first sequenced the data for these indicators and then sonified the column for 2010-2015 and then the column for 2015-2019.
In this sonification, white noise is passed through a band of filters.
Bands corresponding to the worsening cells are turned up in gain a lot, bands corresponding to improving cells are turned up not so much in gain.
The more noise we hear, the more worsening or no change cells in the data.
Example for Poverty 2010-2015
Band filter:
Example for Poverty 2015-2019
Example for Compulsory Education 2010-2015
Example for Compulsory Education 2015-2019
A representation to how the data changed over time between 2010 and 2019 for each indicator and by factor (e.g. gender, ethnicity, etc.)
Here we are sonifying the first spreadsheet which shows differences in outcomes by ethnicity, age, disability, sex, geographic area and socioeconomic deprivation.
For this, we simply modulate the pitch of a sinewave following the numerical data.
Overall Poverty
This corresponds to the first row of data for the three indicators of poverty.
Poverty and gender
You can listen to the separate genders first, and then both genders together
Poverty and Ethnicity
You can listen to the separate ethnicities first, and then all of them together.