• cuda, gpu, i hate matlab, matlab, optical flow, source code

In 2007, a very nice implementation of a variational implementation of optical flow was described in: A Duality Based Approach for Realtime TV-L1 Optical Flow by C. Zach et. al. I won’t get into the details too much, but the formulation is described by this equation:

E = int_?{?|I0(x) ? I1(x + u(x))| + |?u| dx}

If you are familiar with the seminal work of Horn and Schunck, you will notice it is fairly similar to their variational formulation:

min_u{ int_?{ (|?u1|^2 + |?u2|^2) d? } + ? int_?{ ((I1(x + u(x)) ? I0(x))^2) d? }

And although it looks incredibly simple now, it is in fact fairly difficult computationally since now both terms are not continuously differentiable. To overcome this difficulty, they follow the work of Rudin-Osher-Fatemi energy for total variation image denoising.

Another big contribution comes in their implementation on the GPU. By linearization of the generally non-convex energy functional shown above, the problem is reduced to a pixel-wise convex energy minimization problem. Additionally, by employing coarse-to-fine image pyramids, they are able to account for both small and large movements. Luckily, graphics cards are great at doing both of these sorts of computations very quickly. You … Continue reading...

• 4 comments
• Permanent Link

• cuda, gpu, i hate matlab, matlab, sift, source code

Changchang Wu has a beautiful implementation of David Lowe’s scale invariant feature transform (SIFT) inspired by Andrea Vedaldi’s sift++ and Sudipta N Sinha et al’s GPU-SIFT. Adam Chapman has also made a MATLAB mex version which will allow you to pass in the filename of an image and retrieve the SIFT descriptors and keys as well as perform the matching. If that sounds like a lot of people have implemented this algorithm, then check this out.

I had tried using Adam Chapman’s version though, unfortunately, I already had my images loaded into the MATLAB workspace after performing some manipulations and didn’t want to keep writing/reading from disk, thinking that it would be a waste of computation time. I was also processing a lot of images in turn and was running into a lot of crashes, perhaps from continually loading and unloading the library? I haven’t seen anyone complain about this version on the mathworks site, so maybe it is just me.

In finding a way to avoid writing and reading to disk, I did not foresee a problem in the way MATLAB and OpenGL handle their image data. After a brief exchange with Changchang Wu, he led me on … Continue reading...

• 15 comments
• Permanent Link

• close up, conference, diem, edinburgh, eye movements

An event organized by the new Center for Film, Performance, and Media Arts (CFPMA) was held today in Edinburgh University discussing recent topics in… film, performance, and media arts.

It was an interesting group of people that strangely somehow all had much in common. I had the fortune of presenting my research as it relates to DIEM in the place of Tim J. Smith.

The schedule:

Close-Up 2: Schedule for Wednesday 17^th June 2009

10am coffee and tech checks (G.11, William Robertson Building)

10.30 Welcome, Annette Davison (Music, ACE and Director, Cfpma), Martine Beugnet (LLC, Convener of Film Studies)

Who is who, where is what? People and resources for which the Cfpma will provide a point of convergence.

11am Individual Presentations (MAX 10 mins each):

Andrew Lawrence (African Studies, SSPS) — Difficult satire under austerity: the films of Sissako and Amoussou

Martine Beugnet (Film, LLC) — “The Wounded Screen”

Richard Williams (Architecture, ACE) — “The Modernist City on Film”

Stephen Cairns (Architecture, ACE) — “Cultures of Legibility: Emergent Urban Landscapes in Southeast Asia”

Simon Frith and Annette Davison (Music, ACE) — “The Role of Cinemas in the History of Live Music”

Mary Fogarty (Music, ACE) — “The … Continue reading...

• No comments yet
• Permanent Link

• face tracking, glass, installation, opencv, openframeworks, pictures, source code

I’ve recently finished up a project in collaboration with a Glass Artist, Agelos Papadakis. We built a structure of 25 glass neurons the size of a face and chainded them together in a 3x3x5 meter sculpture. We had 2 cameras hidden in the piece tracking peoples faces and a projector then creating visualizations of the recorded faces resembling something like a cloud of neurons firing in different patterns. We presented it first in Edinburgh at Lauriston Castle’s Glasshouse, and then at the Passing Through exhibit in the James Taylor Gallery in Hackney: http://jamestaylorgallery.co.uk/exhibitions/2009/03/passing-through.html

It’s a bit tricky trying to film the piece since it uses projection onto glass. Sadly I’m left with only a few images that try to portray what went on.

Here’s the code, http://ccrma.stanford.edu/~pkmital/share/Memory.zip It makes use of the openframeworks library so you will need to be familiar with how to setup an XCode project with the openframeworks library if you plan on using it.

The original idea was to use glass balls so that’s why all the code says glassBalls instead of say glassNeurons. If you manage to get it running, press ‘d’ to see the live video input. As it collects faces, it fills up … Continue reading...

• No comments yet
• Permanent Link

• jared ali, mr. magoo's poop experiment, sacrifice, shiza

I felt like making a song today:
2009 – Mr. Magoo’s Poop Experiment – Sacrifice.mp3

And I also found these old ones I hadn’t uploaded:
2008 – Mr. Magoo’s Poop Experiment & Jared Ali – fuckeditup.mp3
2008 – Mr. Magoo’s Poop Experiment – Requiem for Shiza.mp3… Continue reading...

• No comments yet
• Permanent Link

• exhibit, installation, london

… Continue reading...

• No comments yet
• Permanent Link

• glass, installation, pictures

Some close ups of the testing:

Artists interpretation by Varun Cursetji:

… Continue reading...

• No comments yet
• Permanent Link

• eye movements, obama

Part of my research entails investigating ways of visualizing eye-movement data during dynamic images.

Some vids:

And one of President Obama’s inaugural speech:

from VisCogEdinburgh… Continue reading...

• No comments yet
• Permanent Link

• face tracking, glass, installation, real-time

Working with Agelos Papadakis, this installation will investigate interactivity, agency, and memory within an installation space. Can an art piece evolve and contain the memory of its participants? Is reaction simply a side-effect of art or *the* effect?

More to come… Exhibit in late April.

((MEMORY)) installation on Vimeo.… Continue reading...

• No comments yet
• Permanent Link

• background subtraction, blob detection, blob tracking, dmsp, eclipse, edinburgh, face tracking, motion analysis, opencv, processing, processing.org, university

My students for the Digital Media Studio Project here at the University of Edinburgh have asked me to present a small workshop on using some aspects of the Processing.org environment. I’ve worked up something and thought I could share it online as well. I’ve setup a google code repository with the necessary files. The code simply highlights what you could find throughout the Processing.org discourse and the OpenCV example files though is more thoroughly commented and organized.

A few notes, I really dislike the Processing IDE. Maybe it’s just because I’ve used IDE’s like VS, Netbeans, Eclipse, XCode etc… and I haven’t really played with Processing enough to have a well founded basis in the functions available. I believe going through a few extra steps to setup an IDE like Eclipse makes the task of programming much easier though at the cost of a bulky editor that may not be so easy to setup at first…

Eclipse is an (Integrated Development Environment) IDE for many coding languages, one of which is Java. Some advantages:

• code completion – automatically see possible choices for all members belonging to a class definition, such as functions and their arguments.

• javadocs – javadoc is

• 2 comments
• Permanent Link