Loading...
WiiJ Video VJ Software
Live Visuals with the Nintendo Wii Remote
WiiJ Video is customizable VJ software for mixing video and effects in a live performance environment. It uses the Nintendo Wii remote to manipulate live 2D and 3D video projections in real time. You can control movie clips, OpenGL graphics and effects using gestures and motion tracking, and also record and loop your gestures for a completely improvised performance.
WiiJ Video Demonstration
A demonstration of the WiiJ Video interface.
Using the Wii Remote (a.k.a. Wiimote)
WiiJ Video uses the Wiimote's pointing ability and accelerometers to control movies and effects with simple hand gestures. The two-handed nature when using the nunchuck allows for a powerful control style: selecting effects with one hand and manipulating them with the other.
There are four layers with four effects each. You can control the composite blending of the layers with the D-pad and switch movie clips and effect presets with the buttons. You can also record and loop automations for each effect to layer many effects over each other.
Judgement Day - Finis Omne
This is a music video of a live video performance for the band Judgement Day. The video is entirely improvised using custom-shot clips.
How it Works
Effects are divided into manipulators and generators. Generators can be movies, camera input, cellular automata, spiderwebs and other textures. Manipulators transform the image from the generator into somthing else, like color bias, hue rotation, convolution/blur, kalidescope/mirror transformations, and more advanced effects.
Most of the effects are implemented as shaders in the GPU, so the system is fast enough for real-time performance. WiiJ Video is written in Javascript using the Max/MSP/Jitter platform.
The Wiimote connects via Bluetooth thanks to the aka.wiimote interface by Masayuki Akamatsu. I built a simple USB-powered "sensor bar" using two infared LEDs and attached it to my MacBook Pro for the Wiimote's IR pointing functions.
Gallery
These are some stills from a Hey Three Eyes/French Miami concert at Bottom of the Hill, San Francisco, California.
All photos by Drew Beck
Performance Highlights
I have used WiiJ Video for improvisational cinematography in many different shows for many different types of music. If you would like me to do visuals for your band in the San Francisco Bay Area, please contact me.