Picking apart the Joy-Con: Pro Controllers, Extended HID and uinput Drivers
With my hidtest utilities out in the open, getting results from other’s Pro Controllers took very little time. Initially, Pro Controllers worked out of the box except they disconnected after a short bit. Additionally, sending commands to only one Joy-Con would cause the other to eventually disconnect sometimes. The solution to this ended up being an extra 80 04 HID command being required in order to keep the controller from switching over to the Bluetooth default later on. While I haven’t checked exactly what this is doing, I’d hazard to guess that one of the Joy-Con pins ends up getting pushed high in order to keep UART communication. 80 05 undoes the effects of 80 04.
As an interesting note, this much progress actually enabled Switch Pro Controller support to be added to the Wii U through Maschell’s HID to VPAD, definitely worth checking out here.
With Pro Controller support out of the way and a small test kit for Joy-Con, I went into dekuNukem’s collection of UART logs in Nintendo_Switch_Reverse_Engineering to find some more interesting commands to investigate. Specifically, I went to check out the vibration commands since Nintendo’s linear resonant actuator “HD Rumble” feature is certainly not something all that common with game controllers yet (notable controllers which do contain them include Valve’s Steam Controller, the HTC Vive wands and Oculus Touch). Examining his Breath of the Wild vibration packet sample, the vibration commands were fairly obvious:
19 01 03 11 00 92 00 0a 00 00 aa e0 10 0a c2 12 4b 40 c2 12 4b 40
In this packet, the extended command header and checksum is contained in the first 12 bytes, with the extended command in this case being command 0x10. For reference, the extended command for fetching input packets was command 0x1f. To test and experiment with different packets I had implemented a replay feature into my test program and the results when given vibration packets were… disappointing. No vibration at all, the input packets returned as usual but not much else actually happened. Between having my Joy-Con connected to my Switch over Bluetooth and also connected to my computer via USB HID, I managed to find an interesting quirk with the 80 04 command mentioned earlier: By having the controller connected to a console and then quickly rushing through init (skipping higher baudrates and 80 04), it was actually possible to get the controllers to preserve an internal ‘vibration enabled’ state, provided I talked to the controller enough that it wouldn’t disconnect from me. Using this, I was able to replay existing vibration packets and also craft my own, resulting in some very strange vibrations:
After a bit of experimentation, I eventually found that vibration packets were formatted as follows for USB HID:
80 92 00 0a 00 00 eb 01 10 TT XX YY YY YY XX ZZ ZZ ZZ
TT is a timekeeping byte, it increments from 0 to 15 forever. XX seems to control the frequency of vibration for the left and right vibration respectively. YY YY YY and ZZ ZZ ZZ seem to be three signed bytes denoting intensity in certain sides of the Joy-Con, however I’m not 100% certain. By leaving YY YY YY and ZZ ZZ ZZ set to all 01 and only increasing the frequency, the following result is gained and, for now, still resides as a simple vibration test in hidtest:
With vibration working somewhat (except for actually enabling it on our own), most of the functionality present in a left Joy-Con is workable, except for maybe setting the player number on the side. The real goal is still reversing the actual Bluetooth protocol, and with that still comes the requirement of getting some sort of real firmware to reverse. With more people being able to dump their SPI flashes without hardware modification, I did hope that there might be a few more eyes on the firmware to try and figure things out a bit. While the full code is still a mystery, there is still some interesting things to note as far as the SPI flash goes:
Depending on the age of the Joy-Con (and possibly other factors?), some Joy-Con have an older firmware at 0x10000 and others have an older one at 0x28000. Generally, both slots are filled with the exception of some Pro Controllers which only have the 0x10000 firmware slot filled. The active firmware is determined by a pointer at 0x1FFC in the SPI flash, which will point to 0x28000 if that firmware should be booted, otherwise it will be unwritten. 0x0 appears to be some sort of initial loader, containing twice a u64 magic D297E5680FF055AA to be checked at 0x1FF4 if 0x1FFC is filled with a pointer to 0x28000. The block at 0x2000 also includes the MAC of the last connected console and joystick calibration data. The block at 0x6000 contains the Joy-Con serial, type, color, and diffing between a right, left and pro controller dump, what looks to be factory joystick calibration data or something similar. The Pro Controller, interestingly, does not have a serial number and only has a body color with its buttons left unwritten. This data is also writable, more on that later.
For Joy-Con and Pro Controllers out of the box, a single byte at 0x5000 is set to 0x01 and the 0x2000 block is entirely written to 0xFF.
As nice as examining the SPI flash was, it was somewhat suggested to me by shuffle2 and others that the firmware on SPI might just be a patch over the existing firmware data, so reversing the full firmware would require the ROM of what was being patched on the Broadcom chip. Examining some of the oddities I had encountered, I’m honestly inclined to agree, if it’s not a patch it’s definitely something strange and having the ROM would be nice. Given some existing Pro Controller board shots previously, I noticed an abundance of test pads and ended up buying a Pro Controller to try maybe getting JTAG on it. Specifically, an array of pads next to the Broadcom chip looked pretty candid:
After fiddling with different JTAG possibilities and examining any outputs with my logic analyzer, it appeared to me that the JTAG functionality was either disabled or those pins were not JTAG at all. Also exposed on the board, however, was a 4-pin SWD header for the STM32 and what looked like another SWD header for some other chip, however I had no luck finding anything on the other end of that as well. The line of 7 throughholes, however, is also accessible from the back and contains the UART Rx, Tx and two flow control lines on the top four pins, followed by VCC, GND and a reset pin.
A bit disappointed, I decided to try getting a wired USB HID driver going for my Pro Controller. For the most part, it was fairly straightforward; uinput was able to provide a fairly straightforward interface for exposing my own custom unified controller, and by tweaking some of my existing HID test code it was possible to make a responsive userland input driver for both Pro Controllers and for two Joy-Con in a charging grip with added hotplugging support. It also passed the Super Meat Boy test with full analog controls in the charging grip and 16ms latency, or 8ms with the Pro Controller:
After several weeks of not that much progress, I realized that I really wanted to get a bit more information on potential commands so that I could maybe try getting code execution on the Broadcom chip some other way. While I do have my doubts, I also wanted some specific commands for enabling vibration, IMU data, and for writing to SPI. As such, with some help from hedgeberg I got a small UART sniffer going using a miniSpartan6+:
Since the sniffer was able to parse the UART in realtime, I could create detailed captures of all data sent during any event. Using this, I was able to go into System Settings, calibrate my joystick, and then watch a packet get generated which wrote some data to SPI flash. The packet ended up being the command next to the SPI read command, extended command 0x01, double extended command 0x11:
19 01 03 38 00 92 00 31 00 00 1a ad 01 04 00 00 00 00 00 00 00 00 11 10 80 00 00 0b b2 a1 0a c5 40 97 a7 8d a4 f4 48 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
This packet does a 0x0B-large write to 0x00008010 of data b2 a1 0a c5 40 97 a7 8d a4 f4 48.
To test this, I did some haphazardous writes into the 0x6000 block to change my Joy-Con color, and through this I was able to create the first pair of spice orange Joy-Con and also have some fun messing with the serial text:
Interestingly, because the console has a colored effect on the side of the screen when Joy-Con are clicked in, most of the metadata in the 0x6000 block is actually cached on the console itself. In order to clear this, Settings > Controllers > Disconnect Controllers has to be used to unpair and uncache all controllers to then show the actual updated color/serial data, which are presumably cached according to the controller MAC.
Additionally, I found out that the commands to enable vibration and IMU are called relatively soon after the metadata SPI reads called to fetch the serial, color and other settings.
19 01 03 38 00 92 00 31 00 00 ae fb 01 03 00 00 00 00 00 00 00 00 48 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
19 01 03 38 00 92 00 31 00 00 cb 8f 01 04 00 01 40 40 00 01 40 40 40 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
For these packets, extended command 0x01, double extended command 0x48 with an argument 01 will enable vibration, or with its argument set to 00, disable vibration. Double extended command 0x40 with argument 01 enables IMU data to be sent along with input packets, occupying the large amount of what is normally zeroed data there.
Even with those bits figured out, my big UART mysteries now mostly lie in the right Joy-Con, specifically with Amiibo/NFC, the infrared camera on the bottom of the controller, home button lights and also the player LED lights present on both the left and right Joy-Con in addition to the Pro Controller. However, the home button lights and player LED lights will probably have to wait a while since those are never called over UART (though the home button lights will likely recieve an update later on which could be sniffed out). Additionally, the infrared camera’s only game is 1-2 Switch which I do not own, and on top of that it requires that the controller be wireless from the Switch, which effectively leaves NFC as the only thing I can really snoop around.
After wiring up my right Joy-Con to be investigated, I ended up with a rather interesting sequence of commands. First, extended command 0x01 double extended command 0x3 is called with argument 0x31, as opposed to argument 0x30 sent on init. Additionally, extended command 0x01 double extended command 0x22 is called with argument 0x01, while it is only called on init on the right Joy-Con with argument 0x00, so I’m guessing this just enables NFC. After NFC is enabled, input packets stop getting sent, and with those commands called, extended command 0x11 double extended command 0x01 is called ~13 times with an extended command 0x01 double extended command 0x21 inbetween. The console then sends extended command 0x11 double extended command 0x02 until an Amiibo is found. To these commands, the Joy-Con responds with an input packet with an additional 0x138 bytes of NFC data attached to it (with the last byte being a sort of checksum byte it seems). This data seems to contain a status byte, a nearby NFC tag ID, and once in contact, one of two alternated portions of the full NFC tag data. After the Amiibo is scanned, NFC is disabled and the smaller input packet requests are resumed. A full console->Joy-Con packet dump can be found here.
Naturally with some packets to replay, I attempted to make this work over USB HID. After all, a USB NFC reader which doubled as a controller would be a great feature for 3DS and Wii U emulators, and it may even work with non-Amiibo NFC tags. However, there’s a fairly critical issue: The STM32 firmware was never tested or meant to work with NFC packets at all. While most other packets stay within the set 0x40 byte limitation, these NFC packets extend much larger. Because most of the UART input buffers end up being 0x40 bytes on the stack, all NFC packets are truncated to just input packets (with just a fragment of the NFC extension). It might be possible to modify the STM32 firmware to allow NFC to work, however in the long run it would probably be easier to just look further into Bluetooth where it is guaranteed to work.
And with that, some things going forward:
- There are definitely some limitations to snooping UART. Specifically, some things aren’t even transmitted over UART at all due to not being implemented (Home Button LED) or due to game limitations (IR camera).
- The STM32 also has limitations to what it can transmit and recieve, especially with NFC and probably with the IR camera as well
- It may be possible to fuzz around for commands like the player LEDs or Home Button LED, but ultimately it would be easier to identify commands with an actual firmware to reverse engineer. As such, that and Bluetooth in general will be the focus with this going forward, probably.
