Init Analysis

Concept

This page should be superceded by the info in the camera section of Protocol Documentation. Not that anything here is wrong, just that that page is substantially more informative.

We'd like to know exactly what the Kinect init sequence is doing, since right now we're just replaying USB control commands blindly. Here's where we'll document what happens when we try to init the Kinect in different ways.

Minimal init sequence

 # Depth camera
 03,126f,12000300,0000
 03,1278,06000200,0000
 # RGB camera
 03,127f,0c000000,0000
 03,1281,0d000100,0000
 03,1282,0e001e00,0000
 03,1283,05000100,0000

The inits for the two cameras can be interleaved, but the inits for each camera need to follow the given order.

Actually, that's not strictly true. For the depth camera, you can't reorder the inits, but the RGB camera will work if you reorder inits...kinda. If the above init ordering is [1,2,3,4], then the following things happen:

• Sending [2, 1, 3, 4] will horizontally flip the RGB camera image. This will persist (including across device reboots) until init 28 is sent.
• TODO: test more permutations

Partial init behavior

I modified marcan's code to send only the first N inits in the send_init() function. I recompiled the source for each test, and unplugged and replugged the Kinect between each test. Results, in "sent N of 28 inits" order:

1. Nothing visibly interesting
2. Nothing visibly interesting
3. Nothing visibly interesting
4. Nothing visibly interesting
5. Nothing visibly interesting
6. Nothing visibly interesting
7. Nothing visibly interesting
8. Nothing visibly interesting
9. Nothing visibly interesting
10. Receive a single full-size depth frame, then nothing
11. Receive nothing.
12. Receive nothing.
13. Receive a single full-size depth frame, nothing else
14. IR projector blinks on for ~1/4th second, then turns back off
15. IR projector blinks on for ~1/4th second, get one correct-size depth frame, projector turns off
16. IR projector turns on, stays on. Depth cam streams correct data. Nothing from the RGB cam yet.
17. Same visible effects as previous
18. Same as previous
19. Same as previous
20. Projector on, depth camera screwy, sends increasingly large buffers, no RGB.
21. Projector on, depth camera works, rgb streams fuzz, mostly large frames of random data. Depth camera works consistently hereafter.
22. RGB frames stream, but look like random data. Frames vary in size: I saw sizes from ~321092 to ~329472 bytes.
23. RGB frames stream, but look like random data. Much larger frame-size variation than previous.
24. Receive 2 RGB frames, RGB frame transfers stop.
25. Receive 3 RGB frames of different sizes, then RGB streaming stops. Timing note: RGB frames start arriving after init 21.
26. Receive 3 RGB frames of different sizes, then RGB streaming stops.
27. Get some RGB updates, but most are very small, likely invalid (~3kB)
28. All systems go.

The RGB camera will stream at 30FPS if you omit all but four of the inits. It seems the minimum commandset to get RGB streaming at 640x480 is:

• First xfer
• init 23 - if skipped, you get a whole lot of "Invalid magic" messages
• init 25 - if skipped, you get a whole lot of "Invalid magic" messages
• init 26 - if skipped, you get image data, but it looks strange and appears to either starve USB bandwidth or CPU when you move objects, since we stop receiving depth frames.
• init 27 - if skipped, no frames try to stream at all. This suggests that this is the "start streaming RGB frames" command.

The depth camera only requires inits 7 and 16. Skip either one, and the depth camera doesn't stream any data at all. Switch their order, and you get a whole bunch of "Invalid magic" packets.

TODO

• Restart the init sequence after the 1st init, 2nd, etc.
• Try switching the order in which inits are sent.
• Send random data for the 17th init, and see what command turns off the projector. (Actually, we got a USB dump from rrivera, so this shouldn't be needed)
• See if the datasheets from Hardware_info#datasheets prove relevant