john oswald - http://archive.pkmital.com https://archive.pkmital.com computational audiovisual augmented reality research Sun, 23 Aug 2015 06:57:28 +0000 en-US hourly 1 https://wordpress.org/?v=6.9.4 Memory Mosaic iOS – Generative Audio Mashup App https://archive.pkmital.com/2015/08/23/memory-mosaic-ios-generative-audio-mashup-app/ https://archive.pkmital.com/2015/08/23/memory-mosaic-ios-generative-audio-mashup-app/#respond Sun, 23 Aug 2015 06:55:54 +0000 http://pkmital.com/home/?p=1977 I had a chance to incorporate some udpates into Memory Mosaic, an iOS app I started developing during my PhD in Audiovisual Scene Synthesis. The app organizes sound in real-time and clusters them based on similarity. Using the microphone on the device, or an iTunes song, any onset triggers a new audio segment to be created and stored in a database. The example video below shows how this works for Do Make Say Think’s song, Minim and The Landlord is Dead:

Memory Mosaic iOS from Parag K Mital on Vimeo.

Here’s an example of using it with live instruments

Memory Mosaic – Technical Demo (iOS App) from Parag K Mital on Vimeo.

The app also works with AudioBus, meaning you can use it with other apps too, adding effects chains, or sampling from another app’s output.  Available on the iOS App Store: https://itunes.apple.com/us/app/memory-mosaic/id475759669?mt=8… Continue reading...

The post Memory Mosaic iOS – Generative Audio Mashup App first appeared on http://archive.pkmital.com.

]]>
https://archive.pkmital.com/2015/08/23/memory-mosaic-ios-generative-audio-mashup-app/feed/ 0
Intention in Copyright https://archive.pkmital.com/2012/06/29/intention-in-copyright/ https://archive.pkmital.com/2012/06/29/intention-in-copyright/#respond Fri, 29 Jun 2012 14:49:27 +0000 http://pkmital.com/home/?p=1106 The following article is written for the LUCID Studio for Speculative Art based in India.

Introduction

My work in audiovisual resynthesis aims to create models of how humans represent and attend to audiovisual scenes. Using pattern recognition of both audio and visual material, these models use large corpora of learned audiovisual material which can be matched to ongoing streams of incoming audio or visual material. The way audio and visual material is stored and segmented within the model is based heavily on neurobiology and behavioral evidence (the details are saved for another post). I have called the underlying model Audiovisual Content-based Information Description/Distortion (or ACID for short).

As an example, a live stream of audio may be matched to a database of learned sounds from recordings of nature, creating a re-synthesis of the audio environment at present using only pre-recorded material from nature itself. These learned sounds may be fragments of a bird chirping, or the sound of footsteps. Incoming sounds of someone talking may then be synthesized using the closest sounding material to that person talking, perhaps a bird chirp or a footstep. Instead of a live stream, one can also re-synthesize a pre-recorded stream. Consider using a database … Continue reading...

The post Intention in Copyright first appeared on http://archive.pkmital.com.

]]>
https://archive.pkmital.com/2012/06/29/intention-in-copyright/feed/ 0