Using the libraries pkmFFT and pkm::Mat, you can very easily perform a highly optimized short time fourier transform (STFT) with direct access to a floating-point based object.
Get the code on my github:
http://github.com/pkmital/pkmFFT
Depends also on: http://github.com/pkmital/pkmMatrix… Continue reading...
Apple’s Accelerate Framework can really speed up your code without thinking too much. And it will also run on an iPhone. Even still, I did bang my head a few times trying to get a straightforward Real FFT and IFFT working, even after consulting the Accelerate documentation (reference and source code), stackoverflow (here and here), and an existing implementation (thanks to Chris Kiefer and Mick Grierson). Still, the previously mentioned examples weren’t very clear as they did not handle the case of overlapping FFTs which I was doing in the case of a STFT or they did not recover the power spectrum, or they just didn’t work for me (lots of blaring noise).
Get the code on my github:
http://github.com/pkmital/pkmFFT… Continue reading...
As part of a system of numerous dynamic connections and networks, we are reactive and deterministic to a complex system of cause and effect. The consequence of our actions upon our selves, the society we live in and the broader natural world is conditioned by how we perceive our involvement. The awareness of how we have impacted on a situation is often realised and processed subconsciously, the extent and scope of these actions can be far beyond our knowledge, our consideration, and importantly beyond our sensory reception. With this in mind, how can we associate our actions, many of which may be overlooked as customary, with for instance, the honey bee depopulation syndrome or the declining numbers of Siberian Tigers.
Responsive Ecologies is part of an ongoing collaboration with ZSL London Zoo and Musion Academy. Collectively we have been exploring innovative means of public engagement, to generate an awareness and understanding of nature and the effects of climate change. All of the contained footage has come from filming sessions within the Zoological Society; this coincidentally has raised some interesting questions on the spectacle of captivity, a issue which we have tried to reflect upon in the construction and presentation of … Continue reading...
Over the past 2 years, I have been working under the direction of Prof John M Henderson together with Dr Tim J Smith and Dr Robin Hill on the DIEM project (Dynamic Images and Eye-Movements). Our project has focused on investigating active visual cognition by eye-tracking numerous participants watching a wide-variety of short videos.
We are in the process of making all of our data freely available for research use. As well, we have also worked on tools for analyzing eye-movements during such dynamic scenes.
CARPE, or more bombastically known as Computational Algorithmic Representation and Processing of Eye-movements, allows one to begin visualizing eye-movement data together with the video data it was tracked with in a number of ways. It currently supports low-level feature visualizations, clustering of eye-movements, model selection, heat-map visualizations, blending, contour visualizations, peek-through visualizations, movie output, binocular data input, and more. The videos shown above on our Vimeo page were all created using this tool. Head over to Google code to check out the source code or download the binary. We are still in the process of stream-lining this process by creating manuals for new users and uploading more of the eye-tracking and video data so … Continue reading...
Taken from the changelog:
… Continue reading...New functionality, features: <<< – General:
* The brand-new C++ interface for most of OpenCV functionality
(cxcore, cv, highgui) has been introduced.
Generally it means that you will need to do less coding to achieve the same results;
it brings automatic memory management and many other advantages.
See the C++ Reference section in opencv/doc/opencv.pdf and opencv/include/opencv/*.hpp.
The previous interface is retained and still supported.* The source directory structure has been reogranized; now all the external headers are placed
in the single directory on all platforms.* The primary build system is CMake, http://www.cmake.org (2.6.x is the preferable version).
+ In Windows package the project files for Visual Studio, makefiles for MSVC,
Borland C++ or MinGW are note supplied anymore; please, generate them using CMake.+ In MacOSX the users can generate project files for Xcode.
+ In Linux and any other platform the users can generate project files for
cross-platform IDEs, such as Eclipse or Code Blocks,
or makefiles for building OpenCV from a command line.* OpenCV repository has been converted to Subversion, hosted at SourceForge:
http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary
where the very latest snapshot is at
http://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk,
and the more or less stable version can be found
In 2007, a very nice implementation of a variational implementation of optical flow was described in: A Duality Based Approach for Realtime TV-L1 Optical Flow by C. Zach et. al. I won’t get into the details too much, but the formulation is described by this equation:
E = int_?{?|I0(x) ? I1(x + u(x))| + |?u| dx}
If you are familiar with the seminal work of Horn and Schunck, you will notice it is fairly similar to their variational formulation:
min_u{ int_?{ (|?u1|^2 + |?u2|^2) d? } + ? int_?{ ((I1(x + u(x)) ? I0(x))^2) d? }
And although it looks incredibly simple now, it is in fact fairly difficult computationally since now both terms are not continuously differentiable. To overcome this difficulty, they follow the work of Rudin-Osher-Fatemi energy for total variation image denoising.
Another big contribution comes in their implementation on the GPU. By linearization of the generally non-convex energy functional shown above, the problem is reduced to a pixel-wise convex energy minimization problem. Additionally, by employing coarse-to-fine image pyramids, they are able to account for both small and large movements. Luckily, graphics cards are great at doing both of these sorts of computations very quickly. You … Continue reading...
Changchang Wu has a beautiful implementation of David Lowe’s scale invariant feature transform (SIFT) inspired by Andrea Vedaldi’s sift++ and Sudipta N Sinha et al’s GPU-SIFT. Adam Chapman has also made a MATLAB mex version which will allow you to pass in the filename of an image and retrieve the SIFT descriptors and keys as well as perform the matching. If that sounds like a lot of people have implemented this algorithm, then check this out.
I had tried using Adam Chapman’s version though, unfortunately, I already had my images loaded into the MATLAB workspace after performing some manipulations and didn’t want to keep writing/reading from disk, thinking that it would be a waste of computation time. I was also processing a lot of images in turn and was running into a lot of crashes, perhaps from continually loading and unloading the library? I haven’t seen anyone complain about this version on the mathworks site, so maybe it is just me.
In finding a way to avoid writing and reading to disk, I did not foresee a problem in the way MATLAB and OpenGL handle their image data. After a brief exchange with Changchang Wu, he led me on … Continue reading...
I’ve recently finished up a project in collaboration with a Glass Artist, Agelos Papadakis. We built a structure of 25 glass neurons the size of a face and chainded them together in a 3x3x5 meter sculpture. We had 2 cameras hidden in the piece tracking peoples faces and a projector then creating visualizations of the recorded faces resembling something like a cloud of neurons firing in different patterns. We presented it first in Edinburgh at Lauriston Castle’s Glasshouse, and then at the Passing Through exhibit in the James Taylor Gallery in Hackney: http://jamestaylorgallery.co.uk/exhibitions/2009/03/passing-through.html
It’s a bit tricky trying to film the piece since it uses projection onto glass. Sadly I’m left with only a few images that try to portray what went on.
Here’s the code, http://ccrma.stanford.edu/~pkmital/share/Memory.zip It makes use of the openframeworks library so you will need to be familiar with how to setup an XCode project with the openframeworks library if you plan on using it.
The original idea was to use glass balls so that’s why all the code says glassBalls instead of say glassNeurons. If you manage to get it running, press ‘d’ to see the live video input. As it collects faces, it fills up … Continue reading...