Project ideas

Plug and Play Kinect Hacking

• Open-source operating system for ARM boards that makes the Kinect plug-and-play
• This OS has libfreenect pre-installed and includes common scripts (generate maps of rooms, track skeletons, etc), so it's really quick to get started
• In progress: [1]

Workout RPG gamification

• Adapt any existing RPG game where one can gather experience points
• In order to gather them or for fights different workout exercises like push-ups have to be done
• Thus the player gathers in workouts as (s)he gathers in the game

Video Conferencing on Steroids

Gproud 14:25, 11 February 2011 (CET)

• Universities use a powerful, large format videoconferencing system called Access Grid (see [2]). It is almost 3D because they use multiple cameras and multiple large displays (typically 3 projectors).
• With OpenKinect, it could actually become real 3d - the immersive conferenceing experience!

Actor-Triggered Stage Lighting/Sound Board Control

Leal Vona @ 04:21, 4 January 2011 (CET)

• Attach a Kinect to an existing lighting rig.
  • 12-watt power draw is negligible against existing dimmer packs
  • USB length limits allow 5-hubs with 5m of cable allowing approx 25m (~82 ft) distance from control host to stage
• Wire Kinect to control host via USB.
• Write middleware to allow for OSC or MIDI code conversion of Kinect positional data
• Use OSC / MIDI data to control lighting/sound board through board's existing input mechanisms

A simple example: This could allow actors on stage to cue lighting changes and sound effects without the use of a dedicated board op, and with more precision.

Sequenced 3D Environment Mapper

Leal Vona

• 3-4 Kinects in triangular/circular array
• Taking pictures (depth camera on/off) in sequence at rate < 30 FPS
  • Kinect 1 - Frames 1,4,7,10, etc.
  • Kinect 2 - Frames 2,5,8,11, etc.
  • Kinect 3 - Frames 3,6,9,12, etc.
• Depth camera IR arrays theoretically would not interfere with each other.
  • Requires ability to completely shut off IR beams in rapid succession. I don't know if this has been realized in the API yet.
    • see Oliver Kreys work with multiple kinects reconstructing one model scene - Sean Kean

Live Dance perspective changer

Sean Kean

• I am in talks with a performance venue here in new york city which has acrobats and other dancers as well as a gallery space. I'd like to set up an installation which displays both live and recorded "volumetric video" and allows viewers to experience the performances from traditionally impossible perspectives.

Japanese Gameshow Human Tetris game

• Game idea seams like a reasonably simple project.
  • http://www.youtube.com/watch?v=U7NKhQDBW6I

Floating Keyboard

• Create a simple gesture for bringing up a "floating" typing keyboard
• View would zoom in to user's face and upper torso
• User would see the keyboard with opacity to see his hands and face "behind" the keyboard
• Each "finger" press would really be a hand motion, a few inches movement rather than a couple centimeters.

Socratesone 20:47, 21 December 2010 (CET)

3D Drum Kit

Ben X Tan

• Using MIDI so you can alter the sound to what ever you want
• Placements of drums will be configurable too
• Use physical objects as the drum components (Mark Esthers idea as discussed this morning with me :))
  • This way you can actually hit physical objects
  • You tell the software whether the object is a Tom, snare, kick, hat, etc. and then you can move it around in 3d space.
  • For example you set up a room with card board boxes and trash cans with AR markers on top of each object and you tap on them (next to and not on top of the markers) Or you might not even need markers...just use object detection by shape and size.

I think the latency (which is around 150ms from what I've seen) would make this unusable. 18:58, 5 January 2011 (CET)

Gesture Based Key Mapping

Brian Schiesl

• Using a full body skeleton to map a gesture and bind it to a single key press or a string of key presses.
• Along with a key press it would also need to have a programmable gesture (toggle on/off) that would have the mouse track a hand.
• Record and Bind Gestures
• gesture brings up start/stop/save interface
• This would allow gamers to feel more like the actual character in a game as well as allow non gamers the ability to control their computer with their own set of gestures.
• View visible representation of gesture, modify properties
• Name gesture
• Associate gesture with an application or trigger an event

Standardized Gesture Key Mapping

• Sign language base of some gestures?
• Painting interface:
  • shake up spray can gesture to bring up painting
  • left hand spins object 3-axis to be painted, right hand controls spray volume
  • shifting weight to one leg to scroll through items, pages, photos (further you lean, faster items are paged through)
  • Moving hand to chin to stroke beard(think) brings up options.

Learn Karate

• If you try to make this game, please try not to hurt yourself or others
• Player would try to learn a "form", and he would be scored for his performance
• A mini-game in which the player repeats fundamental movements, perhaps hitting objects thrown at him?
• You know those movies where one ninja beats up 20 other ninjas? That's a game.
• Possible to register movements? :
  • Forward/back/side
  • Punch/kick/knee/elbow
  • Left/right
  • Hook/jab/backhand/open
  • That move where you hit them from behind with the back of your fist

Gesture Semantics

• identify multiple gestures in sequence ("combos")

Mouse Driver

Farid Wardan (http://far.id.au/)

• Recreation of the native Kinect menu interface on xbox
• Map hand movement to the mouse cursor
• Gestures for clicking and right-clicking

Joystick Emulation

• Standardized set of gestures for joystick and buttons
  • Leaning/punching, open and close fists, twisting motions
  • Define a discrete number of major areas on the screen accessible with a hand (ei, a 3 x 3 grid).
  • Need modes to handle different joystick implementations (such as an XBox Controller)?

Video conferencing

Chris Duncanson-Hales

• utilize camera's voice recognition and tracking capability for video conferencing with multiple participants
• I am organizing an international conference with no travel budget. Having Kinects at each table group would increase the number of participants and would save literally 1000's of dollars (probably significantly more). The potential for international conference is staggering. At the last American Academy of Religion, the consultation I am a part of used Skype for a paper presentation. The presenter could not afford a cross continent air fare. This was the first use of this type of technology for the AAR. I'd like to push the envelope further with Kinects.

3D Scanner

nikione

• Easiest/simplest way to use Kinect to scan an object in 3D
• Software could be coupled for interfacing with one of the many RapProt or 3D Printers around (many opensource) effectively creating a "3D copier"

See 3D BodyScanner demonstration at http://www.youtube.com/watch?v=9MwS_nk9n2A

Sign Language Interpreter

Per Arneng

• Sign Language to speech
• Sign Language to text
• An chat application so that sign language speakers can chat with non sign language speakers
• Sign language-based programming language - Socratesone 22:20, 21 December 2010 (CET)
• Sign language-based operating system interface

Bodytracking for Weightlifting

• Software analyzes angle of extremities to each other, records for later analysis
• Overlay of actual movement on optimal bio-mechanical movement
• Possible correction cues via speech output (Example: "Go further down", "Straighten your back", etc.)
• Workout stats: Average declining speed, average lifting speed, uncontrolled movements (e.g. shaking knees)

Virtual Dance Pad for Rhythm games like DDR/StepMania

Waltbosz

• Users would dance on a virtual dance pad being tracked by Kinect
• Eliminate the need for cumbersome plastic dance pads that slip on floor and stick to feet.
• Inspired by the Kinect piano by Pete & Matt

Home Monitoring System

• real time video streaming of kinect located inside the home for the purposes of monitoring while I am away.
• Could also be used to monitor older children who get home before their parents get home from work.

Portable 3d Survey System

• use the kinect to "scan" an environment and create a complete 360 point cloud.
• could be handheld or mounted on a tripod.
• would be able to output to .fbx or another format that can be read by the majority of 3d applications.

Vehicle Blind Spot Camera

• detect when vehicles are in the blind-spot regions of a semi truck or car.

Presentation program using augmented reality

• can allow one to manipulate a virtual object using his hands, without sign or card, to show virtual object.
• a simple feature would be to change page on a presentation document with a natural move.
• interact with the audience with gestural language and enhance the interactivity during a lecture.

View-angle-based 3D Model DIsplay

• display a 3D model as if the user is looking at the monitor as if it is a window. Perspective should change as the users face moves closer. Moving in any direction should move the camera to that direction.
• this would require calibration depending on the screen and user initial distance.
• could include gesture control to rotate the entire scene around a pivot point (center of the model in focus should be default). a straight raised arm would control direction ans speed.
• objects could be moved using input devices simultaneous to the user. Eventually this method should be used in an actual game to help you look around/aim slightly. Due to constraints of the camera's FOV and monitor distance head tracking can Only be used to slightly modify the users view. Using this method also allows for the scene to appear 3D inside of your TV to the viewer without the need for glasses.

Enhanced Bio-metric Identification

• use Three-dimensional Identity recognition can significant improve the accuracy of face recognition and body shape recognition.
• use Mic-phone array to make sure the voice source is on the face.
• use Voice recogition to double check person's identity.