Magenta Studio

Demos Blog RealTime Studio DDSP-VST Research Magenta Studio (Ableton Live Plugin) (v2.0) Download Ableton Live Plugin View on GitHub Magenta Studio is an Ableton Live plugin built on Magenta’s open source tools and models. They use cutting-edge machine learning techniques for music generation. Magenta Studio was formerly available as a collection of standalone applications. They are not actively maintained but may still work on your operating system. For more information, please see Magenta Studio v1.0. Table of Contents Overview Installation Usage Continue Generate Interpolate Groove Drumify Overview Magenta Studio is a MIDI plugin for Ableton Live. It contains 5 tools: Continue, Groove, Generate, Drumify, and Interpolate, which let you apply Magenta models on your MIDI clips from the Session View. Installation Requirements Ableton Live 10.1 Suite* or greater *Previous versions of Ableton Live require using a non-bundled version of Max 8. Installation Drag the downloaded Magenta Studio amxd file into any available MIDI track within Live. Usage Launching the plugin Each of the tools can be launched by clicking on its name within the plugin. Clip selection All of the plugins work by choosing one or more clips from Ableton's Session View. You must choose a track before selecting your clip. Only MIDI tracks will show up as options. Once all of your selections are made, the Generate button will become enabled. Temperature All of the plugins have a temperature slider. Temperature is a parameter used for sampling in the last layer of the neural network. You can think of it as controlling randomness: higher values produce more variation and sometimes even chaos, while lower values are more conservative in their predictions. Limitations Melody input is limited to monophonic melodies (one note at a time), and drums input uses this MIDI mapping. Notes outside this range will be mapped to these 9 instruments: Instrument Pitch Bass drum/Snare drum 36/38 Closed/Open hi-hat 42/46 Low/Mid/High tom 45/48/50 Crash/Ride cymbal 49/51 † Continue Continue uses the predictive power of recurrent neural networks (RNN) to generate notes that are likely to follow your drum beat or melody. Give it an input clip and it can extend it by up to 32 measures. This can be helpful for adding variation to a drum beat or creating new material for a melodic track. It typically picks up on things like durations, key signatures and timing. It can be used to produce more random outputs by increasing the temperature. How to use Select a clip which you would like to extend, then click Generate. The output clips will be added to the clip slots after your selected clip. Note: This video uses the v1 version of Magenta Studio. The interface now launches within the Ableton Live window, but the functionality is the same. Generate Generate is similar to Continue, but it generates a 4 bar phrase with no input necessary. Choose where you would like the output to go, the number of variations, temperature, and click Generate. This can be helpful for breaking a creative block or as a source of inspiration for an original sample. Under the hood, Generate uses a Variational Autoencoder (VAE) that has been trained on millions of melodies and rhythms to learn a summarized representation of musical qualities. Generate chooses a random combination of these summarized qualities and decodes it back to MIDI to produce a new musical clip. How to use Generate does not require any input files, so the clip selection determines where you'd like the output clips to go. Note: This video uses the v1 version of Magenta Studio. The interface now launches within the Ableton Live window, but the functionality is the same. Interpolate Unlike the other plugins, Interpolate takes two drum beats or two melodies as inputs. It then generates up to 16 clips which combine the qualities of these two clips. It's useful for merging musical ideas, or creating a smooth morphing between them. Interpolate also uses a Variational Autoencoder (VAE) similar to Generate. One way to think of the VAE is as a mapping from MIDI to a compressed space in which similar musical patterns are clustered together. Each of your input patterns is represented by a position on this map. Interpolate draws a line between these positions and returns clips along this line. The number of returned clips is set by the "steps" slider. How to use Interpolate requires two inputs, and these inputs must be on the same track. The outputs are inserted after the second clip. The clips should be the same length and less than 4 measures. Note: This video uses the v1 version of Magenta Studio. The interface now launches within the Ableton Live window, but the functionality is the same. Groove Groove adjusts the timing and velocity of an input drum clip to produce the "feel" of a drummer's performance. This is similar to what a “humanize" plugin does, but achieved in a totally different way. We recorded 15 hours of real drummers performing on MIDI drum kits. These recordings were quantized, removing all velocity and microtiming and were used to train a neural network to predict the unquantized beats as the output. How to use Groove takes one clip as an input and places the output clip one slot below the input. Note: This video uses the v1 version of Magenta Studio. The interface now launches within the Ableton Live window, but the functionality is the same. Drumify Drumify creates grooves based on the rhythm of any input. It can be used to generate a drum accompaniment to a bassline or melody, or to create a drum track from a tapped rhythm. It works best with performed inputs, but it can also handle quantized clips. We used the same dataset of drum performances as Groove to train Drumify. However, instead of learning a translation from quantized drum patterns to performances, here we map from rhythms to performances. We extract a rhythm from each performance by removing the pitches and velocities, while keeping the precise timing details. When you provide an input sequence -- be it a melody, bassline, chord progression, or drum pattern -- we extract a rhythm in the same way and have the model turn it into a groove. How to use Drumify takes one clip as an input and places the output clip one slot below the input. Note: This video uses the v1 version of Magenta Studio. The interface now launches within the Ableton Live window, but the functionality is the same. --- Demos Blog RealTime Studio DDSP-VST Research Demos A primary goal of the Magenta project is to demonstrate that machine learning can be used to enable and enhance the creative potential of all people. The demos and apps listed on this page illustrate the work of many people--both inside and outside of Google--to build fun toys, creative applications, research notebooks, and professional-grade tools that will benefit a wide range of users. Collections Plugins + Apps Plugins + Apps Web demos built with Magenta.js Magenta.js Community contributions using Magenta models Community contributions Colab notebooks to interact with Magenta Python Colab Notebooks Applications using specialized hardware Hardware Featured play read blog The Infinite Crate Ashu Desai desaiashu The Infinite Crate is a DAW plugin prototype that integrates the Lyria RealTime API directly into your favorite music software. play read blog DDSP-VST Jesse Engel jesseengel jesseengel DDSP-VST is a DAW plugin to morph audio from one instrument to another in realtime and is even playable with MIDI. play read blog see code Listen to Transformer Monica Dinculescu notwaldorf notwaldorf An app to make it easier to explore and curate samples from a piano transformer. play read blog see code DrumBot Monica Dinculescu notwaldorf notwaldorf Play real-time music with a machine learning drummer that drums based on your melody. read blog see code Fruit Genie Mike Derrick mikederrick Nico Paris nico-paris Matthew Pegula pegula pegula A real-time intelligent musical instrument which combines Magenta’s Piano Genie model with a physical interface consisting of fruit (or whatever else you can dream up)! Developed in partnership with The Flaming Lips for their performance at Google I/O 2019. play read blog see code MidiMe Monica Dinculescu notwaldorf notwaldorf MidiMe is a machine learning experiment to train a small model to sound like you. All the training happens directly in the browser using TensorFlow.js – no servers or backends here! play read blog see code Magenta Studio Yotam Mann tambien yotammann Cassie Tarakajian catarak Magenta Studio is a collection of music plugins for Ableton Live built on Magenta’s open source tools and models. It can also be downloaded as standalone, native apps with no additional dependencies. play read blog see code Runn Vibert Thio vibertthio vibertthio RUNN = 🏃Run + 🤖RNN. A side-scrolling game where the player has to finish the level to listen to the full song. Each level is generated realtime with a MusicRNN model. play GANHarp Tero Parviainen teropa teropa Samuel Diggins samueldiggins GANHarp is an experimental musical instrument based on AI-generated sounds. It uses Magenta.js and the GANSynth model to generate continuously morphing waveform interpolations, and lets you play them with your computer or MIDI keyboard. play see code Magic Sketchpad Monica Dinculescu notwaldorf notwaldorf Every time you start drawing a doodle, Sketch RNN tries to finish it and match the category you’ve selected. play read blog see code Piano Genie Monica Dinculescu notwaldorf notwaldorf Chris Donahue chrisdonahue chrisdonahuey Have some fun pretending you’re a piano virtuoso using machine learning. play read blog see code Piano Scribe Converts raw audio to MIDI using Onsets and Frames, a neural network trained for polyphonic piano transcription. read blog see code NSynth Super Google Creative Lab NSynth Super is an experimental physical interface for the NSynth model, which uses deep neural networks to generate sounds at the level of individual samples. play read blog see code The Incredible Musical Spinners From Latent Space Tero Parviainen teropa teropa An “AI-powered interactive music experience” that presents a playable 7x7 grid of musical measures. Powered by the chord-conditioned Multitrack MusicVAE and Magenta.js. play see code Tenori-off Monica Dinculescu notwaldorf notwaldorf A creative take on a rare electronic sequencer. Uses the Magenta.js to generate drum patterns when you hit the “Improvise” button. play read blog see code Neural Drum Machine Tero Parviainen teropa teropa An experimental drum machine powered by Magenta.js using the DrumsRNN and MusicVAE models. play read blog see code Beat Blender Torin Blankensmith torinmb tBlankensmith Kyle Phillips hapticdata hapticdata Generate two dimensional palettes of drum beats and draw paths through the latent space to create evolving beats. Built by Google Creative Lab using MusicVAE. play read blog Latent Loops Catherine McCurry currycurry axlsetback Zach Schwartz zischwartz Harold Cooper hrldcpr Sketch melodies on a matrix tuned to different scales, explore a palette of generated melodic loops, and sequence longer compositions using them. Built by Google’s Pie Shop using MusicVAE. --- Demos Blog RealTime Studio DDSP-VST Research Blog posts Open-sourcing The Infinite Crate DAW plugin Today we're fully open sourcing The Infinite Crate, a DAW plugin built on the Lyria RealTime API, for developers to fork, modify, and make their own under the Apache 2.0 license. March 9, 2026 Lyria Camera: Soundtrack your life Lyria Camera is an app that uses Lyria RealTime to make music with your camera. By combining Gemini’s image understanding and the Lyria RealTime API, Lyria Camera generates a musical score that adapts to your environment. December 3, 2025 Space DJ: Navigating a Musical Universe Space DJ is a web application that turns music exploration into an interactive journey through a constellation of sounds. Pilot a spaceship through a galaxy where each star represents a musical genre, and Space DJ will use the Lyria RealTime API to generate a continuous stream of music in real-time. November 3, 2025 Lyria RealTime VST: The Infinite Crate The Infinite Crate is a DAW plugin prototype built on the Lyria RealTime live music model in the Gemini API. The plugin allows users to mix together text prompts to steer a live music model in real-time, feeding audio directly into your DAW for sampling, live performance, or a backing track to jam with. July 9, 2025 Magenta RealTime: An Open-Weights Live Music Model We’re happy to share a research preview of Magenta RealTime (Magenta RT), an open-weights live music model that allows you to interactively create, control and perform music in the moment. June 20, 2025 Introducing Lyria RealTime API Introducing the Lyria RealTime API, a new experimental tool offering real-time control over a generative AI music model, enabling users to explore and create dynamic music experiences through text prompts and various musical parameters. June 12, 2025 Magenta Studio 2.0 Magenta Studio has been upgraded to more seamlessly integrate with Ableton Live. It is a collection of music creativity tools built on Magenta’s open source models, using cutting-edge machine learning techniques for music generation. August 24, 2023 The 2023 I/O Preshow – Composed by Dan Deacon (with some help from MusicLM) A look into Dan Deacon's creative process for the 2023 Google I/O preshow. June 21, 2023 The Wordcraft Writers Workshop: Creative Co-Writing with AI We invited 13 professional writers to explore the limits of co-writing with LaMDA and foster an honest and earnest conversation about the rapidly changing relationship between technology and creativity. December 1, 2022 The Chamber Ensemble Generator and CocoChorales Dataset We combine Coconet and MIDI-DDSP into a system called the Chamber Ensemble Generator, which we use to make a giant dataset of four-part Bach chorales called CocoChorales. September 30, 2022 Autoregressive long-context music generation with Perceiver AR We generate music with clear long-term coherence and structure in both symbolic and audio domains, by attending to inputs spanning up to several minutes. June 16, 2022 DDSP-VST: Neural Audio Synthesis for All DDSP-VST is a neural audio synthesizer for your digital audio workstation, powered by DDSP. June 8, 2022 MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling We show a hierarchical extension of DDSP to use note and expressive performance conditioning. January 20, 2022 Paint With Music Turn your paint brush into musical instruments and compose on sensorial canvases! January 6, 2022 HCI and ML: Putting People First We highlight work done in collaboration with Human Computer Interaction (HCI) colleagues that optimize models for human values. December 15, 2021 Modern Evolution Strategies for Creativity Modern evolution to artistically fit concreate images and abstract concepts. November 18, 2021 Music Transcription with Transformers We discuss Magenta's research into using off-the-shelf Transformer models for music transcription. November 9, 2021 Hip-Hop EP co-produced using Machine Learning In this guest post, MJ Jacob discusses how he used Magenta models in the process of creating his new EP. May 26, 2021 Natya*ML Through Natya*ML, I sought to showcase the beauty of Bharatanatyam in a way that’s easy and understandable. January 28, 2021 Maestro: An AI-guided vocal coach We leveraged the Magenta.js library to create Maestro, an application that combines music theory with practical lessons for music enthusiasts. January 26, 2021 DearDiary.ai Type some words—thoughts, feelings, poems, goals, stories, a to-do list—and you’ve created an original song. January 22, 2021 BitRate: Recap Last year, Magenta and Gray Area partnered to host BitRate, a month-long series focused on experimenting with the possibilities of Music and Machine Learning. January 20, 2021 Stepping Towards Transcultural Machine Learning in Music How can we move towards building creative technology for all? January 12, 2021 AI Song Contest: Human-AI Co-Creation in Songwriting What are the challenges in using AI as a tool in songwriting? What are the design implications? October 13, 2020 Tone Transfer Transform everyday sounds into musical instruments. Record and hear yourself as flutes, saxophones and more! October 1, 2020 Lo-Fi Player A magical room where you can interact with music and have fun. Explore the possibilities by tinkering with the objects in the room. September 1, 2020 The Musician in the Machine Guest blogger Dan Jeffries discusses how he and his team dug deep to find out if neural nets can compose Ambient music with the great masters of the art. August 7, 2020 BitRate: Machine Learning & Music Series This August, Magenta and the Bay Area non-profit Gray Area present BitRate, a month-long series focused on experimenting with the possibilities of Music and Machine Learning. July 31, 2020 Make a song together Work together with friends to create your very own piece of music! April 30, 2020 Improving Perceptual Quality of Drum Transcription with the Expanded Groove MIDI Dataset OaF Drums enables high precision automatic drum transcription with included velocity prediction. April 2, 2020 Making an Album with Music Transformer Nobody's songs is an album composed with the help of Magenta’s Music Transformer. February 18, 2020 Listen to Transformer An app to make it easier to explore and curate samples from a piano transformer. February 13, 2020 Encoding Musical Style with Transformer Autoencoders We introduce the Transformer autoencoder, allowing control over both the global and local structure of a generated music sample. January 22, 2020 DDSP: Differentiable Digital Signal Processing Fusing interpretable digital signal processing with end-to-end learning. January 15, 2020 DrumBot: your real-time ML drummer Play real-time music with a Machine Learning drummer that drums based on your melody. December 2, 2019 SVG VAE: Generating Scalable Vector Graphics Typography Code, Colab notebook and data open-sourced for ML-assisted SVG generation of fonts. October 15, 2019 Generating Piano Music with Transformer Interactive Colab notebook for generating piano performances. September 16, 2019 YACHT's new album is powered by ML + Artists The LA-based dance-pop trio YACHT just released their new album, a collaboration with Magenta and other ML researchers and artists. September 13, 2019 MidiMe: Personalizing MusicVAE A machine learning experiment to train a small model to sound like you. July 23, 2019 Visualizing the Bach Doodle Dataset Interactive visualizations of the Bach Doodle compositions. July 16, 2019 ML-Jam: Performing Structured Improvisations with Pre-trained Models Interactive jamming with machine learning models. June 19, 2019 Magenta + Deeplocal + The Flaming Lips = Fruit Genie Creating an AI-assisted performance as part of the headline concert at I/O 2019. May 13, 2019 GrooVAE: Generating and Controlling Expressive Drum Performances GrooVAE models expressive drumming. May 2, 2019 WiMIR Workshop 2018 Building Bridges at WiMIR 2018. April 22, 2019 Coconet: the ML model behind today’s Bach Doodle We present Coconet, the ML model behind today's Bach Doodle. It is a versatile model of counterpoint that can infill arbitrary missing parts by rewriting the musical score multiple times to improve its internal consistency. March 20, 2019 GANSynth: Making music with GANs GANSynth enables high-fidelity audio synthesis with GANs. February 25, 2019 Magenta Studio Magenta Studio is a collection of music creativity tools built on Magenta’s open source models, available both as standalone applications and as plugins for Ableton Live. They use cutting-edge machine learning techniques for music generation. February 12, 2019 Porting Arbitrary Style Transfer to the Browser Reiichiro Nakano describes how he contributed arbitrary image style transfer to Magenta.js using model distillation to improve performance in the browser. December 20, 2018 Music Transformer: Generating Music with Long-Term Structure We present Music Transformer, a self-attention-based neural network that can generate music with long-term coherence. December 13, 2018 ML as Collaborator: Composing Melodic Palettes with Latent Loops Catherine McCurry, a musician and a creative technologist with Google’s Pie Shop, writes about designing tools that help musicians make use of Magenta’s musical models. November 6, 2018 The MAESTRO Dataset and Wave2Midi2Wave MAESTRO (MIDI and Audio Edited for Synchronous TRacks and Organization) is a new dataset composed of over 172 hours of virtuosic piano performances captured with fine alignment (~3 ms) between note labels and audio waveforms. October 30, 2018 Piano Genie: An Intelligent Musical Interface We introduce Piano Genie, an intelligent controller that maps 8-button input to a full 88-key piano in real time. October 15, 2018 A train window Inspired by Steve Reich’s Music for 18 musicians, Damien Henry uses machine learning to create a visual to go along with it. October 3, 2018 Piano Transcription in the Browser with Onsets and Frames Many of the generative models in Magenta.js require music to be input as a symbolic representation like MIDI; but what if you only have audio? September 20, 2018 Multitrack MusicVAE: Interactively Exploring Musical Styles June 5, 2018 Connecting with Music Through Magenta.js May 3, 2018 Magenta.js May 2, 2018 MusicVAE: Creating a palette for musical scores with machine learning. March 15, 2018 Hands on, with NSynth Super March 13, 2018 Onsets and Frames: Dual-Objective Piano Transcription February 12, 2018 Real-time Performance RNN in the Browser October 5, 2017 Using NSynth to win the Outside Hacks Music Hackathon 2017 September 12, 2017 Performance RNN: Generating Music with Expressive Timing and Dynamics June 29, 2017 Draw Together with a Neural Network June 26, 2017 Generate your own sounds with NSynth June 19, 2017 Waybackprop June 1, 2017 SketchRNN model released in Magenta May 18, 2017 Making a Neural Synthesizer Instrument May 18, 2017 NSynth: Neural Audio Synthesis April 6, 2017 Magenta returns to Moogfest March 16, 2017 Learning from A.I. Duet February 16, 2017 Magenta wins "Best Demo" at NIPS 2016! December 16, 2016 Tuning Recurrent Neural Networks with Reinforcement Learning November 9, 2016 Multistyle Pastiche Generator November 1, 2016 Human Learning What WaveNet Learned from Humans September 23, 2016 Magenta MIDI Interface August 2, 2016 Generating Long-Term Structure in Songs and Stories July 15, 2016 Music, Art and Machine Intelligence (MAMI) Conference July 11, 2016 Reading List June 29, 2016 A Recurrent Neural Network Music Generation Tutorial June 10, 2016 Welcome to Magenta! June 1, 2016 --- Demos Blog RealTime Studio DDSP-VST Research The Infinite Crate (Lyria RealTime VST) A VST plugin that integrates Lyria RealTime directly into your DAW, reducing barriers to interactively create, control, and perform music in the moment. Download for macOS Download for Windows Download macOS Standalone View on GitHub volume_off Table of Contents Overview Usage Controls Installation Troubleshooting Overview Welcome to the Infinite Crate! An imaginary crate of vinyl you can dig though to create a continually evolving stream of music filled with latent interpolations and wild new sounds for you to find. The Infinite Crate is an experimental DAW plugin built on the new Lyria RealTime live music model in the Gemini API. The plugin allows users to mix together text prompts to steer a live music model in real-time, feeding audio directly into your DAW for sampling, live performance, or just giving you a practice partner to jam with. Usage Dig in the Crate Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block. Jam Along Jam with Lyria RealTime by using it to create dynamic backing tracks that are different every time, varying and evolving as you play on top. Be a Prompt DJ Play Lyria RealTime like a live instrument. Use prompts and advanced controls to vary the instrumentation, texture, and genre. Incorporate with a DJ set to apply effects and mix with other loops. Controls BPM: Influences the model to generate audio at the specified BPM; it is imprecise. Set this to [SYNC] to instruct the model to generate audio at the specified BPM. Set this to [DAW] to use the BPM setting from your DAW. Key: Instructs the model to generate audio in the specified key. It's imprecise and does not differentiate between major and relative minor. TopK: Top-k sampling is a technique used to control the diversity of the generated audio. Lower values make the model more deterministic. Temp: Influences the randomness of the model. Lower values make the model more deterministic. Density: Controls the density of the audio. Higher values generate busier sounds. Turn this off to let the model decide. Bright: Controls the brightness of the audio. Higher values generate more high frequency content. Turn this off to let the model decide. Mute Bass / Drums / Other: Attempts to mute bass, drums, and other sounds from the plugins. Guidance: Influences how much the model will try to match the text prompts and parameters (prompts) vs the most recently generated audio (context). Installation Download the plugin Download for macOS Download for Windows Get a Gemini API key To use the plugin, you'll need to create a Gemini API key. Install the plugin Apple Silicon / M-series processors: Unzip "The-Infinite-Crate-v0.1.0.zip" Manually install the plugin VST3: Drag the .vst3 file into /Library/Audio/Plug-Ins/VST3/ AU: Drag the .component file into /Library/Audio/Plug-Ins/Components/ Standalone: Drag the .app file into /Applications/ Restart your DAW / rescan plugins Plugin appears under Magenta->The Infinite Crate Supported Platforms & Plugin Formats Mac (Universal) - VST3 (Ableton), AU (Logic, Ableton) Windows (x86_64) - VST3 (Ableton) May work on additional DAWs supporting VST3 on Mac/Windows. Linux and AAX (Pro Tools) are not supported. Troubleshooting Known Limitations After 10 minutes of continuous music generation, you'll need to restart your session by clicking the "Reset" button. This is because the experimental Lyria RealTime API currently has a 10 minute limit. It will take at least 1 second for the audio to respond to your actions in the plug-in. Be patient, but also try turning up the sliders if you're not getting what you want. Some audio dropouts may occur (eg. during buffering or resetting). Some server dropouts may occur (eg. when using multiple plugin instances, or hitting the 10 minute limit). Windows Ableton DPI fix To improve sharpness on high DPI monitors using Windows/Ableton: Click the "..." button in the top right of the plugin Turn off "Auto Scale Plug-In Window" Updating API Key On Mac, open the terminal and execute command: rm ~/Library/Application\ Support/Magenta/The\ Infinite\ Crate/The\ Infinite\ Crate.settings Learn More Join The Discussion Leave feedback, ask questions, and share what you've created. How does Lyria RealTime work? Learn more about the Lyria RealTime model. Explore AI and Music Learn more about how Magenta is exploring the future of Machine Learning and Audio.