AI-based mastering, human audio mastering and the future
By Rob Stewart - JustMastering.com - May 29, 2016
Last updated July 22, 2017
- LANDR and other AI-based platforms offer a compelling service at a compelling price point
- How you define audio mastering will help you decide if automated audio mastering it is right for you
This is not a review of LANDR, or other AI-based mastering services. I will discuss how LANDR and other automated audio mastering services have and will continue to shape the Audio Mastering industry.
What is LANDR?
LANDR is an online automated audio mastering service that uses artificial intelligence (AI) to analyze your mixes and apply different mastering processes to it based upon what the AI engine determines your mix needs.
Seriously? Automated audio mastering?
Yes! Well, it depends upon how you define audio mastering but absolutely, LANDR offers a fully automated service that uses learning AI. Automation is synonymous with progress because it means saving time, which ultimately means saving money. Automation works in virtually any industry from automotive to financial services to even healthcare. Driverless cars are coming, too. The music industry is no different.
What is Audio Mastering?
Audio Mastering has become an umbrella term for a whole range of services but at its core, mastering is the process of preparing music for distribution and broadcast. There are other tasks associated with mastering but this is the heart of what mastering is all about. I have written a few articles that expand upon this in different ways:
- Why master your music?
- Song Mastering: The mastering process explained
- Audio Mastering Process - Step By Step
Because the audio mastering process is creative, scientific and highly specialized, it is often shrouded in mystery. Some refer to it as a "dark art", and mastering engineers are often asked to "work their magic" before sitting down to work on a mix. Mastering is not magic, though. While mix engineers focus on crafting compelling mixes, mastering engineers focus on making sure that what you hear at the mix position is heard out in the real world. To do this, they listen on a system that allows them to hear the mix objectively - revealing issues that are not always obvious in the mix environment. Then, they either make or recommend adjustments that will help it sound more musical, more listenable or playable.
Is audio mastering really an art and a science?
Yes. Science first. To be effective, the mastering engineer needs to understand and consider psycho-acoustics (how our brains perceive and process sound), along with many other technical aspects of sound (e.g. the physics of sound) and audio (e.g. electronics, signal flow, etc.). It is helpful for the engineer to understand recording technology, microphone placement, various mixing techniques and countless other aspects of recording engineering because without that knowledge, it is much more challenging to recognize and diagnose issues.
Audio Mastering is also an artistic process because when the situation calls for it, the engineer uses different tools - often creatively - to help align with the producer's vision. The mastering engineer will sometimes bring their own aesthetic to a project which has been created over time, with experience, just like any evolving artist.
Different mastering engineers will have different approaches. You can read more about my approach and philosophy here:
Can this mix of creativity and technical expertise really be automated?
I believe parts of the process can be automated. Think of how audio processing has evolved over the years. We started with very specific tools such as compressors and equalizers. While they haven't gone away, many new and popular mix processors are combining these tools together into a simple, easy-to-use interface. For example, you can buy plugins that - with just one knob - can make something sound louder or wider, whereas in the past you might have needed two or more tools, and two or more adjustments to accomplish the same thing. That's a very basic example of automation. While the trade-off is less control, the benefit is in its ease of use, saving the mix engineer time.
Anything that is purely process-driven can be automated. That is why robots can paint cars. Processes like monitoring and reporting on gain and adding metadata can easily be made automatic. Creativity is not as easy to automate, but that is changing. I have no knowledge of how LANDR's current AI works, but, I do know that as of the time of this writing, AI in general has become quite "human like" on many levels. Creativity is largely about making experience-based choices to leverage the capabilities of the available tools to accomplish a goal. There's certainly more to it than that, but it's absolutely conceivable to me that AI could be fed or develop a set of experiences that it could use to determine what tools it needs to apply to a given situation. So, science first: observe, measure, define. Then art and creativity: adjust, enhance. Then science again: verify intended result.
How does LANDR sound?
I have not used LANDR enough to form an opinion. For example, I don't know if it has a sound character of its own or if its character depends upon the material you submit. I also haven't completed any technical analysis of the results. I have used LANDR to test drive it, but have not used it enough to know how well it performs under different types of circumstances or if it's better suited to one genre or another, for example. As with any mixing or mastering service, let your ears guide you.
Any tips for assessing the sound quality?
Absolutely. Check out my article "How to judge a finished master" which covers some basic guidelines. One key concept that I mention in the article is that louder almost always sounds better to our ears, so to fairly judge the quality of any master (human or AI-made), listen to it at the same loudness level as whatever you are comparing it to. This will minimize the chance of you being swayed by gain alone. One easy way to do this is if you have Apple's iTunes installed on your computer, load your master into iTunes, and turn on Sound Check in the playback settings. Sound Check will adjust the gain on each song in your library so that everything aligns with a target loudness level (no dynamics changes are made, it's purely a non-destructive gain adjustment). With Sound Check on, you can compare your master to your original mix while keeping them both on a level playing field.
Is LANDR really audio mastering?
It depends upon how you view audio mastering. Whether you are a mastering engineer, or if you are an artist or label considering hiring a mastering engineer, here are some questions to consider.
Do you view audio mastering as:
- separate from the rest of the production process, or as a collaborative effort, where the mastering engineer is part of the team?
- an audio processing service, or as an audio and mix consulting service?
- an audio adjustment process, or as a quality monitoring and assurance process?
Do you pay a mastering engineer to:
- "finish the job" by working with whatever they are given, even if it could be improved before mastering?
- help you achieve the best possible result by offering suggestions to improve your mix before proceeding with mastering?
There is no right or wrong answer to these questions. The answers depend upon who you are, your budget and your needs. That is why I believe that AI-based mastering and traditional audio mastering both have a strong future.
But what about the gear?
Mastering equipment is an important tool of the trade. If it weren't for calibrated monitoring systems and rooms, mastering engineers would struggle to hear mixes objectively. On one hand, that makes the gear a key component, but on the other, most mastering engineers could still work in a "compromised" environment if they were forced to. It would just take them longer to work. In other words, the gear is an enabling tool, but it is not a measure of a mastering engineer's skills or value. As a mastering engineer, if you are trying to compete with AI-based mastering services based on gear alone, you'll be fighting a losing battle. This is no different than trying to compete with other mastering engineers based purely upon gear.
What about years of experience?
Absolutely. The more experience you have, the more competent you are (usually!). Keep in mind that a learning AI system that can evolve based upon many thousands of experiences is impossible to compete with on the basis of experience alone. AI can recall exact details about every experience it has, forever. Sadly, it doesn't work that way for us humans. Our memory is quite plastic in nature. While we remember many aspects of our experiences, our memory of them changes over time, and details change. That doesn't automatically mean that AI-based mastering is better or worse than human-based mastering, it just means we store and leverage our experience differently.
What about having "golden ears"?
While I love the name, I think the term "golden ears" has contributed to Audio Mastering's reputation as a dark art surrounded by folklore. What exactly are golden ears? It is all about experience and focus. A mastering engineer has the experience to know when something could be improved (and how - e.g. "turn down the cymbals"), and the focus to hear it among everything else that is happening in a mix. I expand upon this here:
Can AI be made to have the equivalent of golden ears? To a degree, I think it can, and may already have. AI can measure dynamics, gain, frequency balance, timing and more. It's not too far a stretch for AI to then use its experience-based logic to make decisions based upon what it has measured. Humans still have the edge at the moment because I believe only a human mastering engineer might say something like "the low C note on the bass guitar track is too loud relative to the rest of the bass track", whereas an AI-based system - at best - might be able to determine that there's an issue in that range of frequencies but it likely cannot differentiate a bass guitar from a bass note on a piano (yet!).
Will Mastering Engineers become extinct?
We have witnessed the fall of the traditional recording industry over the last 20+ years. Many great recording studios closed, and many recording engineers were left without work. At the time of this writing, the most expensive package that LANDR offers would put many mastering houses out of business, which is scary on the surface. What is a mastering engineer to do?
Step 1 is for mastering engineers to accept AI as a valid competitor. Denial will accomplish nothing. LANDR and other AI-based mastering services are not going anywhere because they offer a compelling service, at a compelling price. Inevitably, more AI-based services will come. Welcome to disruptive technology :)! Our industry has seen other disruptive technologies before. Remember Napster?
Step 2 is to respond to the disruption and evolve. Mastering engineers will need to adapt and stay ahead of AI by redefining the value that they offer to the world. Renovate, Innovate or Evaporate :)! While traditional audio mastering was aligned with the "manufacturing" side of music (e.g. final prep, adding metadata etc.), the role has and continues to evolve to meet the changing needs of recording artists.
How will mastering engineers survive the new reality of AI-based audio mastering?
It depends! Mastering engineers need to continually ask themselves the following questions:
- Using your gear, what additional value do you offer your clients over AI-based mastering services?
- Using your experience, what additional value do you offer your clients over AI-based mastering services?
- For the price you are asking, what additional value do you offer your clients over AI-based mastering services?
- Is your value in the transaction, or in the service you offer?
I believe that there will always be a place for human audio mastering engineers. There is and will always be a place for automated AI-based mastering services, too. It is up to the human mastering engineers to help clients understand the differences so that they can make the choice that best meets their needs. If we fail to do that, we certainly will face extinction.
I see many mastering engineers already doing this. I don't think AI-based mastering necessarily means changing what mastering engineers do. While it will for some, in many cases it just means changing how we approach our work and communication.
P.S. I hope you'll check back with me in 5 or 10 years to find me still in business, happily co-existing with LANDR!