Sound Design Resources

Streaming enables a number of important facilities:

Listeners can listen to complex sonic events, such as orchestral music, and understand them in useful ways. By tracking streams and making comparisons with other streams comparisons can be made. This enables the listener to find relationships between streams of sound.

Listeners can predict what will happen to a stream in the future. If a stream is behaving as predicted then more attention can be paid to other streams until the stream in question changes requiring a re-evaluation.

Streaming can be done by applying extra-sonic qualities to sounds. Words are actually a set of sonic symbols that carry meaning for a listener. Therefore the ability to stream a conversation at a party, in part, involves the ability to make sense of what is being said. Words that don’t fit into the general sense of a conversation, probably come from another speaker. The different qualities of voice, such as tone and accent also assist in streaming.  Other extra-sonic qualities, such as associations can be important in listening. For example the sound of a siren or car tyres screeching, will demand attention due to the potential for danger associated with these sounds.

In music, however the main qualities that assist in streaming are:

Register/pitch

Dynamics/size

Timbre

Gesture

Time/rhythm

Space

Musical Illusions

Examples from: Pierce, J. R. The Science of Musical Sound.  Freeman, New York 1983 . pp. 208-13

Example 1.

The ability to stream based on the parameters above has been tested in auditory experiments and a number of interesting musical illusions have been discovered. These illusions involve sending two scrambled streams to the left and right ears. It has been shown that listeners unscramble the streams based on the best fit according to parameters such as register and timbre. In Example 1. above, stimuli presented to the left and right ears through headphones are shown in part A. What the subject heard is shown in part B. Although both high tones and low tones were presented to each ear, the right ear heard only the high tones, the left ear only the low tones (some people do not experience this illusion).

Example 2.

In Example 2, tones presented to the left and right ears, in part A, make up an ascending scale and a descending scale. What was heard is shown in part B. The right ear heard the higher tones as running partway down the scale and then up again. The left ear heard the lower tones as running partway up the scale and then down again..

Simple Orchestration Example

Example 3.

Using principles derived from streaming it is possible to create rich textures that retain the original line but add depth by including streamed layers as a sustaining texture.

Example 4.

In example 4, the notes shown by the cross and the triangle may be played with the same, or different timbres. When played with the same timbre they are heard as a sequence of 3 ascending notes. When the timbres differ enough, two voices are heard, as shown in Part B. Each consists of of a sequence of descending notes.

Orchestrational Functions

Orchestration involves combining instrumental sounds into a texture. The way that this is done depends on the objectives of the orchestrator. Understanding streaming is useful in forming objectives and putting them into action.

There are only two basic functions that can be carried out in orchestration:

Separation – the creation of streams or layers in a musical texture that can be heard distinct from one another.

Blending – combining sounds together so that they are heard as a single stream.

Figure 1 illustrates the relationship between musical parameters and separation and blending.

Figure 1.

There are a range of objectives possible in orchestrating a musical texture however the most common strategies involve the creation of layers that suggest listening focus. Listening focus relates to a kind of hierarchy of attention in that the most important or interesting musical lines are indicated or made clear by the structure of the texture. The most basic, and traditional approach is to separate streams into established or designate musical roles:

Melody (most important stream),

Harmony (supporting stream) and

Rhythm (supporting stream).

Many musical textures however do not fit into these roles (such as a fugue for example) and therefore a more global description is required. It is helpful to analyse musical textures into:

Foreground,

Middleground and

Background.

Figure 2 illustrates the relationships between; separation and blending, musical parameters and common musical streams. Note also that the level of predictability of a musical line will also help determine whether it has a background vs. foreground role. For example, melodic lines are typically more novel than bass lines and as such, demand more attention of the listener. In fact if the bass line was to become too demanding, the musical texture may be difficult to listen to as there will be a competition for attention between the melodic line and the bass.

Figure 2.

Listening Focus

In terms of listening focus, there are two aspects to the structure of musical texture. Foreground and background relationships can be:

Stable, or,

Mobile.

Typically, developed musical structures shift between stable organization and mobile organizations.

The most common parameters used to differentiate foreground are:

Dynamics – if a stream is louder it is, by implication, closer and therefore in the foreground.

Space (and virtual space in recorded or electronic music) – a stream can actually be closer

Note that the other musical parameters a likely to used in conjunction with dynamics and space.

Listening Examples

John Adams, Violin Concerto – as is common in the concerto form, the relationship between the solo instrument and the orchestra remain, for the most part, fixed or stable.

J. S. Bach, 1st Bandenburg Concerto, 2nd Movement – this work is a concerto grosso, a form in which a small group of soloists (the concertino) are presented as a foreground stream while the orchestra (the ripieno) provides a supporting role. This can mean that at some times there is a solo or a duet , trio or quartet that take the foreground in the texture. In this example the oboe starts in a stable relationship then the violin takes over. Then these to instruments weave around each other shifting in a very fluid manner between foreground and middleground.

Igor Stravinsky, The Rite of Spring, Introduction – this section start with the solo bassoon which is presented in a stable relationship. The texture however becomes increasingly mobile as more and more instruments are given momentary focus. The mobility of the texture is made increasingly more complex to the point where the foreground/background distinction becomes ambiguous.

György Ligeti, Lux Aeterna – here the choir sings long sustained notes that form aggregated pitch clusters. As such, there are little in the way of markers that determine foreground/background relationships. Instead we have very momentary foreground markers that happen as each choir member enunciates the first syllable of the word being sung.

Peter Mcilwain, MinimalistTexture (mp3 file available from the muso website) – here the repeated material of this texture ebbs and flows as the polyrhythmic nature at the parts cycle in and out of phase with each other. This creates a complex cannon in which the foreground/background relationships are in constant flux. In fact it is possible to listen in a number of different ways to this texture each time picking out different relationships between the parts. Therefore the listener makes up there own points of listening focus.

Sound localisation is part of a discipline called Auditory Scene Analysis (which studies the way we make mental picture of the world through sound linked to other senses).

Inter-aural Intensity Difference (IID) - headshadow effect

Inter-aural Timing Difference (ITD) - pathways from a sound source to each ear can be different in length therefore sound waves arrive at different times.

Head Related Transfer Function (HRTF) - reflections off the body differ depending on the location of the sound (forward facing ears help resolve front/back confusion in IIDs and ITDs).

Contextualisation - micro movements of the head comparing changes in IID, ITD & HRTF, echo and reverberation patterns, correlated visual cues, other information (memory of previous similar events).

Perception Lecture Notes: Auditory Pathways and Sound Localization by Professor David Heeger

Good introduction (if the neuroscience stuff is confusing skip to the general discussion of Auditory Scene Analysis):

Making Utterance

Stages in sound making

There are various ways that theorists have described the phases or stages that sound goes through. The stages described below are my own descriptions that I have found useful in approaching sound design. In general an ecological perspective on perception understands perception from the basis of affordance, that is perceptual attributes have evolved that afford survival advantage because they inform the listener about critical events in the environment (such as the the magnitude, kind and manner or energy expenditure within meaningful proximity to the listener).

1) Accumulation: the building of potential energy prior to a sonic event occurring. Examples include the intake of breath prior to singing. 

2) Action: the release of energy via a gesture of some kind prior to impact.

3) Transference (or Impact): the imparting of kinetic energy to an object. When a string is displaced by an exciter the process can by understood as noise being introduced into a system in equilibrium.

4) Response: the response of that object to the introduction of energy. This is the process by which kinetic energy is channeled into steady oscillation by virtue of the physical properties of the excited object.

5) Interaction: the modification of the vibration where for example a performer may be in contact with the excited object which may cause the sound production process to change in some way. Vocalisation is the most significant example of interaction.

6) Resonance: the response of the environment to sound bring transmitted, the most obvious example being reverberation.

The way the elements connect together are as follows:

Notes: 

Envelope generators (EG) create a series of ramps that can be used to control the level of the sound (by controlling the amplifier level).  

In Schindler's List the sound design is often focused on detail. We are drawn into the actuality of the scenes being depicted. The film portrays in great detail, the experience of the Jews during the Holocaust. In the early sequences we are being taught to notice the details, the details matter.

For more on using the ES1 sythn see: