Artificial intelligence has become a byword for industrial disruption – new research shows that even the creative industries are not immune to its machinations. What does this automated incursion mean for the ‘black arts’?
I t’s 2040, and yes, the Academy Awards are still grinding on. When the Best Original Score award winner is announced a buffed humanoid fitted with legs adapted to tentatively negotiate the stage steps appears from the crowd to claim the prize.
This would have been a fanciful thought maybe 10 or 20 years ago but not now that advanced artificial intelligence (AI) has done what many had thought impossible by boring into the once impenetrable world of creatives.
This community includes artists, musicians, writers, and film-makers, whose work involves high levels of intuition, affective knowledge and gut instinct once considered beyond the capabilities of machines.
You found ways of breaking the rules and that’s why they called it the black art because people would say, how do you get that level on that record with that length? And I’m going, you know, I’m not telling.Mastering Engineer, 2017
“Much has been written about AI and the disruption it causes for blue-collar and white-collar workers, but relatively little research has been done on creatives, sometimes referred to as ‘no collar’ workers,” says Dr Thomas Birtchnell, Senior Lecturer at the University of Wollongong’s School of Geography and Sustainable Communities in the Faculty of Social Sciences.
In a paper published recently in the journal Geoforum, Dr Birtchnell explores how one section of the music production industry, mastering engineering, is grappling with a fledgling AI incursion.
Dr Birtchnell defines modern AI as comprising three key areas: machine learning, algorithms (set of rules to be followed in calculation) and big data analytics drawn from online data repositories.
His case study focused on a low-cost automated system for audio mastering catering for musicians, sound producers and film score composers developed by the Montreal-based company LANDR in 2014. It uses advanced AI called ‘deep learning’. The AI adapts to large and complex data – it learns how to learn.
“Its big data analytics are drawn from music files and it has the ability to sort through and run queries on these data bases, and do that in an automated fashion,” Dr Birtchnell says.
“While the company is not suggesting their artificial intelligence is able to think about and have an experience of music and sound in the same way a human does, the provocation upsets long-standing ideals about the role of human expertise.”
The Black Art
Audio mastering, the last component of music post-production, is sometimes referred to as a black art which defies simplistic definition. It involves a chain of skills, technologies and intuitive talent to improve the aural quality of a piece of music without compromising the original content.
Much of the work involves volume control and decisions on how ‘loud’ and accurate a song sounds against play-back technologies (your car stereo versus iPhone earbuds or a club’s PA). The aim is to ensure a cohesive experience for the listener but also one that evokes an emotional response regardless of the listener’s environment or space.
One analogy compares it with a picture frame, which enhances a painting or photograph without compromising the original artistry. Another compares it with the spray-paint job on a new car.
There is no doubt this kind of technology is going to replace a huge chunk of what I do.Rick O'Neill, Turtlerock Studios
Veteran mastering engineer Rick O’Neill, from Turtlerock Studios in Sydney’s inner west, who has worked with artists such as Bob Dylan, Madonna, Kylie Minogue and Bryan Adams pulls no punches on LANDR’s impact.
“There is no doubt this kind of technology is going to replace a huge chunk of what I do,’’ he says. However, while acknowledging this software provides an acceptable base-line level of quality for certain mastering tasks, he argues it has not evolved to equal humans in quality control.
“But I totally believe somebody in the future could develop software to figure that sort of stuff out.”
These comments are consistent with the qualitative research gathered from some of the mastering engineers interviewed by Dr Birtchnell.
“I expected them to say this is completely rubbish, and some did, but some were more agnostic about it,” he says. “They were saying the product that this AI is producing is not bad, it’s not good but they could certainly see it had a market, and that really surprised me.”
The study details some of the suggestions by freelancers and pioneering companies where LANDR could be used to build alternative business models, rather than undercut their human counterparts who cannot financially compete with the cost of cheap mastering software.
“First is the affordance for artists to ‘test’ their mixes before sending them to human audio mastering engineers,” the study reads.
Musicians who now more heavily rely on the ‘gig economy’ could cheaply use AI to master their own demo tapes to attract work. The relatively untapped market for commercial music for short films, elevators, shopping malls, and advertisements was also a potential AI niche.
Changing work spaces
AI, in various forms, has been nibbling away on the edges of creative spheres for some time, Dr Birtchnell says. “We saw it, for example, in 1997 when reigning world chess grand master Garry Kasparov was defeated by IMB’s ‘Deep Blue’ computer,” he says.
In response to the defeat, Kasparov has become an advocate of human creativity as he highlights in the title of his 2017 book Deep Thinking: Where Artificial Intelligence Ends and Human Creativity Begins.
As a human geographer, spatial considerations are critical in Dr Birtchnell’s research.
“It’s often the spatial aspects that are the most important when thinking about how jobs can be replaced or made obsolete by robots or AI because jobs are a lot about space – the spaces we work and play in, or listen to music in the case of audio mastering engineers, and the spaces we travel to,” he says.
“The type of AI used for audio mastering can access the internet rather than a studio to engage with their clients, which means part of the appeal is that the work can be done anywhere.”
Rick O’Neil, the engineer who started in the industry at 15, explains how this spatial shift has impacted on his career.
“Digital technology began disrupting the industry a decade ago and AI has continued the run. For my first 20 years the musician turned up at the mastering process to finish his record. He sat in the room to make sure it was fine, and signed off on it.
“In the past 10 years with the rise of the internet, there is now about 10,000 mastering engineers worldwide where there used to by a few hundred, and most of us are doing it remotely.
What I miss about working with a musician in a studio is that point when you listen back and the hairs on your arms stick up and it feels incredible.Rick O'Neill, Turtlerock Studios
“This means about 80 per cent of my week, nobody sits with me anymore. What I miss about working with a musician in a studio is that point when you listen back and the hairs on your arms stick up and it feels incredible. When there’s nobody else in the room that doesn’t happen to me.”
The removal of the musician from the process has had a profound effect on Mr O’Neil.
“For me that means I no longer know who I’m working with. I look at my email and stick the name of the sender on the job. That night my partner will ask me what I’ve done today and I’ll answer ‘some guy’s record’.”
He says this change has a flow-on effect for the industry. “If nobody sits with me anymore, then I may as well be at home, and if I’m at home I can charge less. If I can charge less then suddenly the whole system changes,” he says.
“LANDR is $7 a song and I’m $200 a song so there is a point where that trajectory can’t be maintained. And that’s going to put me out of business because the client doesn’t give a shit if the mastering is done by a human or a robot.”
However, Dr Birtchnell contends this creation of remote work spaces, where humans are using forms of AI technology like LANDR, spells major changes now and in the future for skills, employment and work.
Instead of falling into a hackneyed appraisal of the consequences of AI on jobs— either doomsday prophecies about the destruction of industries or lauding its economic efficiencies – Dr Birtchnell’s work takes a more nuanced view of audio mastering engineers who both collaborate and compete with AI-driven technological innovation.
The AI in question depends upon vast databases of human-mastered music to build its algorithms and replicate human decision-making. Audio mastering engineers still fulfil a crucial role.Dr Thomas Birtchnell
“The AI in question depends upon vast databases of human-mastered music to build its algorithms and replicate human decision-making in pre-mastered music,” Dr Birtchnell says. “Audio mastering engineers still fulfil a crucial role in understanding the variability of atmospheres of sound to the human ear.
“In the pursuit of the most affective ‘master’ version, human skills aren’t reduced to mechanical processes nor are they and their machine assistants reconstructed as the ideal ‘cyborg’.”
Like most experienced mastering engineers, Mr O’Neil’s repertoire of post-production music skills beyond just mastering will allow him to see out his career, but in a much less engaged fashion than before digital disruption.
“I guess the point I’m trying to make is that the component of mastering that LANDR does, could absolutely do away with the younger versions of me without a doubt,” he says.
“But it can’t do the entire process, at least not yet.”