With my left hand setup kind of working I decided to start with my right hand setup which, unfortunately, I have totally neglected so far. Short recap: the right hand setup is planned to consist of an IMU sensor that picks up the natural strumming patterns of the right hand and uses the movement parameters to modulate the guitar sound. First of all, what is an IMU sensor? According to Wikipedia, an inertial measurement unit (IMU) is an electronic device that measures and reports a body’s specific force, angular rate, and sometimes the orientation of the body, using a combination of accelerometers, gyroscopes, and sometimes magnetometers. As the definition suggests, it is quite a complex device and is really on another level coding-wise than the left hand setup featuring the time-of-flight sensor.
At the very beginning of the semester, my supervisor gave me one of his IMU sensors, namely a MPU-92/65. However, as I approached him last week concerning the IMU sensor business for my right hand setup, he recommended me using another kind of IMU sensor, the BNO055 from Bosch. Apparently, there are better/easier-to-use Arduino libraries for the BNO055, and it is capable of sensor fusion – something I will get into below. Luckily, he also had one of those and gave it to me for experimenting.
Additionally, my supervisor told me the basics of IMU sensors which I will relay to you now:
As already mentioned in the definition, an IMU sensor basically combines an accelerometer, a gyroscope and a magnetometer and they can be used alone or in combination to obtain information about the position and/or movement of an object. When used in conjuncture (=sensor fusion), one can determine the pitch, roll and yaw movements of said object which is what I think I need. Since I have to actually wear the sensor on my wrist while playing the guitar, I cannot yet say, what kind of information I need from the IMU sensor. Of course, the pitch, roll and yaw movements make sense, but I could also try acceleration values for example. My goal for now is to get sensors readings in general and in the next step, I will try to figure out what kind of readings work best for my cause.
I found an Arduino library that lets me calculate the orientation of the sensor giving me readings of the x, y and z axes. My supervisor also highlighted the need to calibrate the sensor each time otherwise the readings are inaccurate. Luckily, the library also has a function that reads me the calibration status of each of the sensors in the IMU (accelerometer, gyroscope, magnetometer) – 0 means not calibrated at all; 3 means the sensor is fully calibrated. I watched a YouTube video that explains how to calibrate each of the three sensors: to calibrate the gyro, the sensor just needs to sit still for like 1-2 seconds (easy!). To calibrate the magnetometer, one needs to tilt and move the sensor for a bit into all directions which also works quite well. Calibrating the accelerometer is the most complex of all three approaches. One must tilt the sensor in different angles and hold each position for about five seconds. It takes a little bit of time and experimenting, but it works.
With the calibration and the orientation readings going, I decided to test it by putting the sensor on my wrist – easier said than done! After some tinkering I came up with the very rough solution of using my (seldomly worn) wristwatch and sticking the IMU sensor onto it using double-sided tape. Now I should be able to strap the watch over my hand and start playing.
