#web audio api

After giving up hope of ever finding my first coding project ever (which I recently found!), I decided about a year ago to build a new chordAnalyzer - this time with a keyboard as the input.

Like my college freshman code, this one identifies notes, rearranges them into thirds, and identifies the root and inversion - as well as any extra "non-chord tones."

Also, it looks like a piano. That's thanks to some clever CSS that makes the keys different shapes and sizes and colors and positions based on some dynamic building instructions:

Above, the Javascript that constructs the HTML. Below, the CSS that styles it:

Finally, this one also makes sound, meaning you can use it as a regular piano even without the chord analysis component. When I first built it, the only timbre was actual piano - individual mp3 files for the 25 notes depicted, which play through a simple <audio> player:

But now that toneMaker (from this old post) is up and running, I figured I'd refactor this to leverage those sounds as well. So in addition to a whole bunch of audio files...

...there's the same audio.js "library" I built for the make-your-own-synth app, which honestly needed just a few extra lines to fully integrate.

If you've used the tool to build your own tones before, you might also notice they're magically available in the drop-down menu here; that's because localStorage is shared across the whole domain - so audio.js can find your instruments no matter which app you're in.

This was so easy that I went back to pitchPlayer (from this old post) and added the instruments there too. So now all three of my music web apps are generating audio using the same code, making your custom-made sounds available everywhere.

tl;dr: visual inputs are better than typing numbers in a command line; you can build an interface in Javascript instead of manually in HTML; applications that connect are stronger together

So far, my inputs, across all my applications, have been limited to mouse, keyboard, and touch screen. Not anymore.

I honestly thought this would be the most challenging part of the toneMaker project, but connecting to a MIDI instrument was actually really simple (thanks to this tutorial and this documentation).

The .requestMIDIAccess() method gives you access to all the input devices in an annoying Iterator, but you can loop through to add some good old event listeners.

Every time you press or lift a key, you get an array of three numbers: [action, pitch, velocity]. There are other actions (mod wheel, sustain pedal, instrument switch, etc.), but all I really need are 144 for key press and 128 for key lift.

Then I just send the pitch into the getFrequency() function, like I was already doing for the on-screen <button>s. From there, I decided to add a new velocity gain node ahead of the envelope to make sounds quieter or louder (up to twice as loud!) depending on how hard the key is hit.

And... that's it. Now you can use a MIDI keyboard to play a custom Javascript synthesizer in your browser. tl;dr: MIDI input is surprisingly easy to decipher (compared to, say, raw websocket); velocity can be controlled with a new gain node; if I wanted in-between pitches, the generalized MIDI-to-frequency formula is 440 * Math.pow(2, (x - 69) / 12)

Chrome 29 Beta Brings WebRTC and Web Audio API To Android, Omnibox Improvements, VP9 Support To Desktop

Google today released the first beta model of Chrome 29 for desktop and robot. Most of the major changes in this update are happening on the robot platform, which now supports the Web Audio API for processing and synthesizing audio and WebRTC, the up-and-coming real-time communication API.

On the desktop (Windows, Mac and Linux), today’s update introducesa tweak to Chrome’s omnibox suggestions,…

I'd like to think I'm great at taking feedback. But sometimes I'm not. Sometimes it just takes a while.

So, this is toneMaker (again). A common tool in any self-respecting synthesizer is an echo effect, depicted here with two adjustable poles. For the first, dragging the height changes the volume of a gain node that siphons off some sound on its way to the speakers. For the second, moving it left and right changes the delay - the pause between pulses.

Note that the gain node is connected to the delay node... but the delay node is connected back to the gain. This creates a feedback loop, which will circle around until the volume eventually reaches zero. (I found this online tutorial really helpful for understanding how it happens!)

Originally, the whole tool was this abstract, label-less art piece. It made perfect sense to me, having built it, but it was impossible to just figure out. I didn't want to admit this, but my friend was right. So now, icons and text labels.

A tougher pill to swallow was that the filter tool was in need of a complete overhaul. My friend showed me how this is handled in other audio tools, emphasizing that confusing users is the most likely outcome of unexpected design.

As it turns out, there was also a problem with the audio setup of these biquadFilterNodes too; sometimes, the sound would just cut out if you played the same note too frequently (no pun intended). I honestly don't know why that happened, but in fixing the filters front-end, I simplified the "back-end", (I mean, technically, this is all client-side,) and the problem went away.

Basically, filters boost or attenuate selected frequencies. Now you can drag up or down from the central line to add them, or drag back to the middle to delete them. It was good feedback, and it made the project both more usable and more functional.

tl;dr: solicit feedback - if you're afraid of it, that means you could really use it; use .createDelay() and .createBiquadFilter() for some more audio effects; label stuff and make it easy for users

As a rule, if I’m not writing about coding, I’m probably coding. And, boy, have I been coding. But before we get to the big project(s) of the last few weeks, let’s dive into the world of Javascript audio.

Every Wednesday at my job, there’s a “tech talk” - kind of like the lunch & learns, but only for engineers. Well, they made an exception, and I got to present on the web audio API, made accessible by Javascript (and hopefully by my tech talk). This one’s gonna be a sing-along, so get out your code editor!

All right, ignoring the code above, note the complex web of nodes along the right. Like a pedal board, the audio API lets you build a chain of nodes - oscillators or buffers, amplifiers, distortion and effects, and an output - that sends an electric signal from one end to the other. Let’s look at the simplest chain: an oscillator and a destination.

So first, we build our environment using new window.AudioContext(). (If you’re in an older browser, you might need to use a vendor-prefixed version, like webkitAudioContext() or mozAudioContext().) Below that, we build a standing wave - a pure tone like you’d hear on a hearing test - and connect it to the audio.destination, ie, your computer speakers.

Most nodes have adjustable parameters. In this case, we can set the type of wave (sine, square, triangle, or sawtooth), and the frequency (440 Hz is the standard tuning A for an orchestra). Note that we use the .setValueAtTime() function to set this frequency at the current time - represented by audio.currentTime. And then... .start() it up!

Oh, yeah, I should have warned you - there’s no way to .stop()! Build out your HTML page like this so we can control when and how loud the sound plays. Note that I gave the <button> a value - this will let us play different pitches.

Okay, so now, we .createOscillator() inside an .onmousedown function, which means the sound starts when you press the <button>. And when you lift (.onmouseup), we .stop()! Simple enough.

For volume, use .createGain() to make a gain node - you can think of this as a guitar amplifier. Using the .onchange event, we can set the volume - as long as you connect your oscillator to this new node instead of straight to the speakers. Note that this volume is between 0 and 1 - if you go too high, you’ll just destroy your speakers!

So far, we’ve been setting the volume immediately - but what if you could schedule that change for the future? And what if you could do it gradually? Well, friends, then you could make an envelope - the attack/decay/sustain/release framework that transforms a never-ending tone into a pseudo-instrument.

These parameters are all adjustable - attack, decay, and release all represent the time (in seconds) that it takes to transition from one volume to the next. In other words, when you first hit a note, the volume is 0, but it ramps up - using .exponentialRampToValueAtTime() or .linearRampToValueAtTime() - in the attack phase, then back to some medium level in the decay phase.

An instrument you pluck or strike - like guitar and piano - will have a shorter attack and decay than an instrument you bow - like a violin.

At this point, while you’re holding the <button>, you’re in the sustain phase. Some instruments are unsustainable (like a drum) while others, hypothetically, can be sustained forever (like a woodwind or brass instrument). Here, the sustain parameter represents a volume.

But when you’re done sustaining, the release phase is the fade-out - again, a time value. Note that for math-y reasons, you can’t .exponentialRampToValueAtTime() to 0, though you can get pretty close (say, 0.0001).

Anyway, play around with it and see what sounds you can make! And try adding more keys for more frequencies!

tl;dr: web audio API is native to your browser; use Javascript to create, adjust, and connect nodes; use an oscillator to make waves and some gain to control volume and envelope; you should really read this one, honestly