Hello and welcome back to my GPRO project as I like to call it. We are now entering the third semester already during which the prototype setups developed during the previous semester will be completed to final products. Instead of overthinking my next steps and planning too much ahead, I decided to jump right back in and continue where I stood at the end of the second semester.
As noted in the documentation and presentation of the second phase, both setups, in particular the right hand setup would benefit from being wireless, with no cables obstructing the guitar player’s movements. Hence, this was the first issue tackled. Although there exist several possibilities to send data wirelessly, it was quickly decided to try out OSC (open sound control) which works over WIFI.
Open Sound Control (OSC) is an open, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices. OSC messages are transported across the internet and within local subnets using UDP/IP and Ethernet. Additionally, OSC is sometimes used as an alternative to the 1983 MIDI standard.
This decision was taken among other reasons because the author was provided with an ESP32 microcontroller at the end of the second semester by his supervisor which already has WIFI capabilities on board. As the name already suggests, the ESP32 is not an Arduino product, however, it still can be programmed using the Arduino IDE after conducting a manual implementation process. With only a few difficulties, said process was completed and the ESP32 ready to be programmed.
The first step was to write an Arduino sketch which connects the ESP32 to the local WIFI network of the author in order to be able to send OSC messages. Using the OSC library for Arduino, such a code was achieved in a rather short time. Furthermore, the OSC library also contained a very useful example sketch to send OSC messages which served as a great learning tool to understand how OSC messaging works. In order to test the OSC messaging function, a Pure Data patch was made which uses the “netreceive” and “oscparse” objects among others to receive OSC messages coming in from a predefined port. The “netreceive” objects allows Pure Data to listen to a specific port, in this case 8000. In the Arduino sketch the port 8000 was defined as the outport from which OSC messages are sent. As a result, OSC messages could now be sent from the ESP32 via WIFI to Pure Data.
After achieving basic OSC messaging capabilities, the subsequent step was to merge the original sketch for the right hand setup, containing the code to access the IMU data readings, with the new OSC messaging method. On the other hand, the Pure Data patch from the second semester which is set up to receive the y-value of the orientation data from the IMU sensor to control a Wah-Wah effect was updated was well. Instead of receiving the orientation data as MIDI input, it now receives the orientation data via OSC, using the objects mentioned above.
After establishing the pin layout of the ESP32, the IMU sensor (still attached to the wrist watch device) could be connected to the microcontroller and data transmission from the ESP32 to Pure Data was successful. Latency seemed to be comparable to that of MIDI. With basic OSC transmission working, it is time to test the new setup it in conjuncture with real guitar playing.
Source:
