Hi Everyone
I’ve been working on a very interesting update that I wanted to share. It involves the company MBD and their work in the arts and projects related to disability and outreach.
Project Brief
We were asked if we could help create a video cave-like system for a woman/child whose disabilities left her able to only move her eyes. Currently, she and her carers wish to remain anonymous, but once we have something that works for them, and with their permission, I’ll be posting an update to include them directly.
The brief is to create a playback system with multiple projectors for a full 180-degree environment in a conservatory that can play back 360-degree content and can be operated by the client.
The software should work with all video and image formats and be easy for a carer to use.
Approach
The job did not require us to make any new software, so we looked at a variety of existing options. These are the two we’ve worked with before.
Madmapper
https://madmapper.com/
This was the first option we looked at as we’ve used Madmapper previously for map buildings. The software allows for multiple projectors to work together to create a single image, and it has a lot of amazing features. An issue I found is how to make this work as a media playback system and how to feed the different 360 formats and have them map correctly.
Resolume
https://www.resolume.com/
Resolume is also a video playback system that can utilize multiple projectors. Again, this is a great piece of software and one that we’ve used previously for exhibitions. It works for this type of project, and can run multiple video tracks, multiple projectors, setup loops, random plays etc. The issue I had with this was similar to that of Madmapper – how to make it work with all the different video formats and how to map that.
Other options considered
Blendy Dome VJ
Scalable Display Technologies
TouchDesigner
Watchout by Dataton
Vioso
These solutions are mostly intended for exhibitions and large projection spaces and are very in-depth and complex. The projection mapping I want for this project is aimed at a smaller space and with a simpler playback system.
My Conclusion
After looking at the alternatives, I wondered if I could simply add projection mapping into Whirligig. (I’m aware that the word “simply” is doing a lot of lifting here!)
Many may see this as an impossible task, but I like a challenge, so here we are.
My solution
Whirligig works differently to most video players as it has to create a 3D world that the user can be in and place the videos around them and take into account what types of videos they are, 360, flat screen, fuldome etc.
For example, when it’s an equirectangular video, you are surrounded by a video sphere, and when it’s a flatscreen video, it’s mapped onto a plane in front of you.
My solution, then, is if you can create a series of cameras in the environment pointing in different directions, they could represent the projectors in real life.
I’ve mocked up some images to demonstrate.
This is a 3D rendering of what you could be doing. The projectors are set up to demonstrate overlap etc. In this case, your computer has a monitor for controlling the setup and three projectors to play back the content. The monitor has all the controls on it which you don’t want on playback.
So imagine that you had these three projectors in your 3D space and around you you have the 360 degree projection mapped onto a sphere.
These three cameras can then take what they see and send them to projectors, which produces something like this:
While the projection doesn’t match perfectly, the alignment makes sense. The real 3D space isn’t a sphere so the projection will need adjusting. Here, I started adding tools to alter the image to make it fit the space. This is the setup that I’ve built into Whirligig:
I started with the setup of each camera and then kept adding stuff. For example, the keystone helps you get closer to the space looking right, the corner pinning helps shift points to correct for the room shape, and the masking allows you to blend between the projects reducing overlap.
I’ve also added options for overlays etc to be able to line things up better.
Here is Paul (MBD boss man) setting up the projectors for the installation:
During the installation, the warping I had implemented wasn’t lining up in some areas. However, we managed to get it working and, as the projection space acts like the VR viewer, you can throw everything you want at it in terms of projection types; fulldome, equirectangular, flat screen etc. Anything that Whirligig can play, will now work in this space.
Final Thoughts
Like so many of my projects, the implementation was much harder than originally hoped. I will keep adding features to help with the projection warping, but right now, it’s about installing the system and seeing what needs changing and how it can be improved. We may find, after all the work we’ve done here, that a different piece of software would work better, but for the moment, I’m hoping that the approach we’ve taken will be useful example to others attempting the same thing.
This setup is now part of Whirligig and can be used by anyone. It’s currently in the latest beta on Steam, so please feel free to update and play around with it.
If you have any suggestions at all, I’d love to hear from you. Feedback on these kinds of projects is truly invaluable!
Happy New Year, and thanks for reading.
Philippa