After Apple discontinued its Music Memos app favored by musicians for developing song ideas, a new startup called Tape It stepped in to fill the void with an app that leveraged AI to automatically detect the instrument and annotate the recording. Now that startup is taking the next step in its journey to improve the audio-recording process with the introduction of an automatic, studio quality noise reduction algorithm, also powered by AI, that works on any audio — not just speech.
The AI denoiser shipped this week as a free web app, with a plan to license the technology to vendors in the future. It will also later be integrated into the company’s flagship Tape It app, the company says.
Founded in 2020 by musicians and friends Thomas Walther and Jan Nash, Tape It’s initial focus was on an iOS recording app for musicians. Before creating Tape It, Walther had spent three and a half years at Spotify after it acquired his audio detection startup Sonalytic. Nash, meanwhile, is a classically trained opera singer, who’s also a bassist and engineer. The duo were originally inspired to build Tape It because it was something they wanted for themselves as fellow bandmates that would be as simple to use as Apple’s Music Memos, but made more powerful through the use of AI.
The original version of the app was able to automatically detect the instrument, and then annotate the recording with a visual indication to make those recordings easier to find by looking for the colorful icon. Musicians could also add their own markers to the files, as well as notes, and photos to review later on.
The app has since gained traction with around 10,000 monthly active users, the company says.
But as Walther told TechCrunch at the time of Tape It’s 2021 debut, the team aimed to broaden their use of AI over time.
Image Credits: Tape It
That led to the startup’s latest development — an AI-powered denoiser they’ve been building over the past two years. The challenge with recordings, the company explains, is background noise. In order to reduce environmental noise and electrical interference, musicians record in studios and leverage complex software. Tape It wants to provide a more affordable alternative using AI. Their software automatically removes noise like hums and hisses, not only spoken word, with the goal of producing studio-quality results on songs, single-instrument tracks and field recordings.
“What we developed is we created an automatic version of the denoising software that you have been finding in professional recording studios in the last 15 years,” explains Walther.
To verify its results, Tape It is releasing an academic study with a scientific listening test that shows the software’s quality in competing with industry-leading denoisers.
In a video, the company explains that while speech enhancement systems have advanced significantly, they generally only work for speech and distort or corrupt music signals. Meanwhile, professional denoising systems require manual control of complex software by professional users. Tape It’s technology involves connecting a neural network controller to a signal processing-based noise reduction algorithm. This allowed for automatic denoising of general audio signals, including music. The company plans to present its work at the AES conference next week.
“The reason people haven’t automated those [professional systems] is because you can’t traditionally put them into a neural network…you can’t train such a system,” notes Walther. “We are actually the first ones to train such a system and that’s why we’re quite excited about this larger area.”
He adds that the academic community will most likely be less interested in the denoising product itself but more in how they managed to get it to work this way because of the implications it has for other applications of automating studio software.
Still, the denoising software already has some interested potential customers, including a large studio software vendor and a large hardware manufacturer. In those cases, enterprise pricing will be made available but for smaller startups, less expensive plans will be offered.
“Everyone is excited about AI being creative,” said Walther, when announcing the news. “We are excited about AI solving boring problems. We take care of background noise, so you can entirely focus on the creative parts and write more songs,” he said.
AI technologies aren’t only being used to reduce background noise for musicians, of course, other companies are also turning to AI to create near-studio quality sound for podcasters too. For example, Podcastle just this month launched its Magic Dust AI, a generative AI tool that eliminates background noise and enhances its dynamic range.
Tape It’s five-person team is based in Berlin, London, Los Angeles and Stockholm, and includes designer and musician Christian Crusius, previously of the design consultancy Fjord, which was acquired by Accenture. The bulk of the work on the denoising software was done by Christian Steinmetz, a PhD researcher in AI and audio.
The company is continuing to bootstrap, having previously turned down offers of funding.
“This is fundamental research and we just didn’t know how long it would take,” Walther explained as to why they went this route. “We thought it was a bit risky if you get an investor who isn’t that patient — [they’d push you to ] just take an open source model and move on. But we wanted to have a larger technological advantage,” he said.
The company is now considering raising funds and is having those discussions, given the pace of the AI market, but hasn’t made any formal decision as of yet.
