During the Fall of 2020, I had the honor of teaching a new course at the University of California in Los Angeles (UCLA) Department of Design Media Arts (DMA) entitled “Cultural Appropriation with Machine Learning”. This provocatively titled course came together after wrestling with many questions I had that year in the wake of the pandemic, black civil rights movements, and a crushed economy.

Rather than teach a course that focuses purely on the “how” of machine learning, like Creative Applications of Deep Learning does, I wanted to also include a critical component to guide students through the questions they should be asking as they learn and employ these tools. I also wanted students to understand how these tools and algorithms came to be in today’s society, so that they knew better what questions to ask when they were using them. It became clear early on that cultural appropriation was a central theme across most generative arts practices. I say this because machine learning requires large amounts of data which tend to come from existing corpora of creative content, such as flickr archives, or instagram collections. What does it mean when an algorithm owned by Google or Microsoft is capable … Continue reading...

The following article is written for the LUCID Studio for Speculative Art based in India.

Introduction

My work in audiovisual resynthesis aims to create models of how humans represent and attend to audiovisual scenes. Using pattern recognition of both audio and visual material, these models use large corpora of learned audiovisual material which can be matched to ongoing streams of incoming audio or visual material. The way audio and visual material is stored and segmented within the model is based heavily on neurobiology and behavioral evidence (the details are saved for another post). I have called the underlying model Audiovisual Content-based Information Description/Distortion (or ACID for short).

As an example, a live stream of audio may be matched to a database of learned sounds from recordings of nature, creating a re-synthesis of the audio environment at present using only pre-recorded material from nature itself. These learned sounds may be fragments of a bird chirping, or the sound of footsteps. Incoming sounds of someone talking may then be synthesized using the closest sounding material to that person talking, perhaps a bird chirp or a footstep. Instead of a live stream, one can also re-synthesize a pre-recorded stream. Consider using a database … Continue reading...