Level 3: My Non-Linear Life

Learning Outcomes

Linear vs non-linear—the nuts and bolts of why games are different from film/TV
Understand how interactivity affects audio design
Some of the major technical developments in game audio history

Challenges of Interactive Media

Non-linear Nature of Game Audio
Unpredictability and Indeterminacy
Action-based Sound Triggering
Integration with Game Design and Programming

Early Console and Arcade Game Development Challenges

Review

In the 1970s and 1980s, creating game audio required direct programming of chips, with no advanced tools available for non-programmers.
Sound design was predominantly done by programmers, often with little knowledge of music or sound, due to the lack of specialized tools.
Early consoles like the NES could only produce a few notes simultaneously, limiting the complexity of music and sound effects.
Despite basic technology, creative designers managed to produce significant audio achievements, contributing to the evolution of game sound design.

Summary of YouTube Transcript

Part 1: Beeper Speaker

Early Home Computers: Basic “beeper speaker” for sound, found in IBM PC and Apple II.
Functionality: Controlled by the CPU, producing tones through timed clicking noises.
Limitation: Required significant CPU resources, hindering other functions.

Part 2: FM Synthesizer

1980s Evolution: Dedicated sound chips in computers and consoles.
Personalities of Systems: Unique sounds and styles for each system.
Voices/Channels: Importance of the number of voices (or channels) for sound complexity.
Voice Flexibility: Ability to create different waveforms.
Examples:
- Nintendo Entertainment System: Five voices but limited to specific sound types.
- Commodore 64: Three voices with four waveform types (square, triangle, sawtooth, noise).
Programming Techniques: Advanced methods allowed for dynamic reassignment of voices.

Sound Chips in Computers

IBM PC Sound Upgrade: Introduction of AdLib card with Yamaha YM 3812 sound chip, followed by the SoundBlaster card.
Features: YM 3812 chip had nine voices and was used in Yamaha keyboards.
Example: Music from the game Ultima 6, and its recreation on a Yamaha keyboard.

Sampling

Sampling Keyboard (1985): Four voices, showcasing the sampling capability.
Commodore Amiga: First affordable home computer with a four voice stereo sampling system.
Modtracker Music: Music files containing samples and music information, originally designed for the Amiga sound chip.
Modern Music Composition: Shift from Mod Tracker format to large sample files like mp3s due to increased storage and memory.

Classic video game sound explained

Nobuo Uematsu

How PS1 Audio Mechanics Work

Summary

Summary of the YouTube Transcript

Introduction to Media Consumption and Sound in Media

The video begins with a discussion about how media consumption is an integral part of our lives, and how sound has played a crucial role in enhancing the immersion and enjoyment of media.

Evolution of Sound Storage: From Phonographs to Compact Discs

The history of sound storage is explored, highlighting the transition from early phonograph-like devices to the development of the compact disc (CD) by Philips and Sony in 1982. This innovation marked a significant change in media storage, rendering many older mediums obsolete.

The Compact Disc: A Game Changer in Media Storage

The compact disc revolutionized media storage with its release in Japan, offering around 10 megabytes of storage in the 1980s, which was impressive for the time. The CD was initially used by music labels to distribute albums.

Advancements and Adaptations in CD Technology

Over time, CDs evolved to store videos, became rewritable, and even saw size reductions. Video game companies began using CDs, with the TurboGrafx-16 CD-ROM add-on being an early example. By the mid-1990s and 2000s, CDs became a dominant medium for video games.

From CDs to DVDs and Beyond

The evolution continued with DVDs, Blu-ray discs, and 4K Blu-ray discs, with storage capacities reaching up to 100 gigabytes on triple-layer discs.

Focus on PlayStation and CD-ROMs

The video then shifts focus to the PlayStation, which used CD-ROMs, a format similar to CDs but with pre-written, read-only data. The PlayStation’s audio capabilities and how it processed sound for games are explored in depth.

Understanding the PlayStation’s Sound Processing

The PlayStation used two chips for sound: the Sound Processing Unit (SPU) and its DRAM. These chips processed sound and stored audio data, respectively. The video examines how PlayStation games used tracks for soundtracks, similar to traditional audio CDs.

Technicalities of PlayStation Audio

The intricacies of PlayStation audio are discussed, including the use of ADPCM (Adaptive Differential Pulse Code Modulation) for audio file compression. The PlayStation could understand two subformats of ADPCM: SPU-ADPCM and XA-ADPCM, each with its own characteristics.

The Role of VAG and VAB Formats

PlayStation games often used VAG files for individual sounds and VAB files for sets of sounds. The video describes how these formats are used in conjunction with the PlayStation’s hardware to produce game audio.

Advanced Sound Processing Capabilities of the PlayStation

The PlayStation’s SPU had advanced capabilities like producing 24 voices, stereo sound, and various sound effects. The video goes into detail about these features, including looping, pitch modulation, and digital reverb.

Conclusion: Exploring PlayStation Sound in Action

The video concludes with a practical exploration of the PlayStation’s sound capabilities, using the console’s BIOS as an example. The BIOS sequence demonstrates the use of SPU voices, reverb, looping, and other audio features, providing a comprehensive understanding of PlayStation sound processing.

The Emergence of Middleware in Game Audio

Introduction of Middleware in 2000s
Middleware as a Solution for Audio Designers
Popular Middleware Applications

Internet and Flash in Game Audio

Impact of Internet on Game Audio
Flash’s Role in Early Internet Games
Evolution of Audio Quality with Internet Speed

Modern Challenges and Innovations in Game Audio

Advancements in Mobile and Tablet Gaming
New Platforms and Audio Challenges
Evolution of Handheld Game Audio

Level 3: My Non-Linear Life

Learning Outcomes

Challenges of Interactive Media

Early Console and Arcade Game Development Challenges

Classic video game sound explained

Nobuo Uematsu

How PS1 Audio Mechanics Work

The Emergence of Middleware in Game Audio

Internet and Flash in Game Audio

Modern Challenges and Innovations in Game Audio

Current Audio Middleware