Kinect has taken the gaming world by storm by enabling motion-controlled gaming on the Xbox 360 without the need for a controller since your body is the controller. But Hyderabad-based Shantanu Goel has managed to get Kinect working on the PS3 and we talk to him in great detail about this unique project. What made you attempt to make Kinect work on the PS3? Well, I didn't really have a goal in mind when I started. I just like dabbling in new things and learning about different technologies. I've been interested in image manipulation for some time now, but I don't have a CS background, so learning all the algorithms to do things from scratch was way too much for my time-bound condition. Kinect seemed like a nice platform for the same, with all the open source libraries around it. I saw that everyone was doing hacks with it on the PC already, so I thought, ‘why not try something that no one has thought about?’ Judging by some comments on my Youtube video, it looks like people still don’t believe that it can be done. I had come across a nice project called Sixaxis Emulator or DIYPS3Controller some time back, which can be used to emulate a regular PS3 Dualshock 3 (DS3) controller on a PC. So I put two and two together and created this mash-up.  The future? What procedure did you use to get Kinect working on the PS3 and what were the components involved in the process? The program makes use of several other programs in order to provide this functionality. To compile/use it, you also need the following: 1. OpenNI Libraries – To get data from Kinect 2. NITE Libraries – Builds on top of OpenNI to provide gesture detection, etc 3. PrimeSense Libraries for kinect – Kinect drivers to get raw data 4. DIYPS3Controller – The main component for PS3 connection. Also known as Sixaxis Emulator. This emulates a DS3 on the PC and sends the inputs to PS3 through Bluetooth. 5. A Bluetooth dongle capable of custom BT MAC address. Most dongles from CSR with Bluetooth 2.1 + EDR marking should work. How much technical expertise and monetary expense would it take to get Kinect working on PS3? If you are looking to do it my way (a PC is used as man-in-the-middle), monetary expense is limited to a cheap Bluetooth dongle worth a couple of hundred Rupees (assuming you already have a PC, Kinect and PS3). Technically, my code is pretty much pick-up-and-go for basic usage, but you need to be familiar with basic compilation stuff. If you want to extend it, then you need to read and understand the (pretty sparse) OpenNI and NITE documentation. You should be pretty conversant with writing C code. How does the system work? What is the extent of Kinect functionality on the PS3? My code glues all the components listed above together to get gesture data from Kinect, convert it into DS3 format expected by PS3 on my laptop and then send it to PS3 over Bluetooth through the laptop/Bluetooth dongle. My proof-of-concept code allows you to browse the PS3 menus fluidly, but game control mapping isn't robust yet and a lot is pending there. Are you able to play PS3 games using Kinect? How did you map motion controls to games that use the traditional controller? Which PS3 games were you able to get working with Kinect? I would say that I am able to make a few movements in PS3 games with Kinect right now, but it still needs a lot of work before I can say it is good enough for playing. Six-axis Emulator can translate mouse and keyboard movements to DS3 data, so for my initial approach, I create a fake mouse and keyboard and convert Kinect input data to this fake mouse and keyboard, and provide it to the Sixaxis Emulator. This is not optimal though. For my next approach, I'm modifying the Sixaxis Emulator code and trying to directly translate Kinect data to DS3, eliminating few redundant steps in between to increase performance and accuracy. So far, I have tried Flower and Killzone 3, with Flower giving more suitable results as it is already geared towards motion controls and doesn't require as much accuracy and reflex action as Killzone 3 does.  Will this be possible on the PS3? Do you plan to keep working on this project? What do you hope to achieve via Kinect functionality on PS3? I do plan to keep working on it and I hope to make it a fully usable alternative for at least Playstation Move-enabled games on PS3. Another wish of mine is to eliminate the man-in-the-middle approach (PC usage) and do it natively on the PS3. With custom firmware it might be possible using Linux on PS3, but right now, I don't want to chart that territory because of this being a legally grey area. Has the recently released Kinect SDK for Windows helped in any way? Do you plan to work with Kinect further outside its uses on the PS3? No. I do all my work on Linux, so the Kinect for Windows SDK hasn't had any impact on it. As of now, I don't have a plan to work with Kinect outside PS3 since a lot of other folks are already doing that work much better than me.  What's next? How about Playstation Move on the Xbox 360? Do you think that would be possible? Of course. With my PC man-in-the-middle approach, it should be possible. We also have open source drivers for the Xbox 360 controller available, so getting that format, creating an Xbox 360 controller emulator and then using the same process with the Xbox 360 should be possible. About the innovator: Shantanu Goel is a B.E in Electronics and Electrical Communication from Punjab Engineering College, Chandigarh. He began his corporate career with Aricent (Hughes Software Systems) working on GSM technology and drivers, and platform development for TI DSPs. He is currently working at Qualcomm, where he develops device drivers for a variety of mobile processors across Android, Windows Phone 7 and various other OSs. |
