In this chapter, the project’s progress, made during the second semester, will be evaluated and successful as well as failed outcomes will be discussed. The evaluation will be based on the goals that were set out to be fulfilled during the second phase of the project.
| Goal Number | Description |
| Goal 1 | Determine and acquire necessary equipment |
| Goal 2 | Determine ideal placement of sensors and microcontrollers on guitar neck and pick/right hand and install them accordingly |
| Goal 3 | Program microcontrollers to pick up the movements of the fretting and picking hands using the Integrated Development Environment (IDE) by Arduino |
| Goal 4 | Program a Pure Data patch that handles the incoming data and transforms it to useable parameters to trigger effects |
| Goal 5 | Either program custom effects in Pure Data or integrate Pure Data in a DAW to trigger commercial effect plug-ins |
| Goal 6 | Determine suitable effects and parameters to be triggered by the movements of the fretting and picking hands |
1.1. Goal 1 – necessary equipment
As far as the first goal is concerned, it can be stated that this task was accomplished. As outlined above, the necessary and suitable equipment for both the left and right hand setups was determined. This was achieved by thoroughly researching potentially suitable components and submitting them to tests in order to identify the best solution possible.
With regards to the choice of a microcontroller, the Arduino Leonardo was chosen over the Arduino UNO due to its built-in USB communication and USB MIDI device capabilities. Albeit the Leonardo being rather big and unwieldy, it was decided to stick with it during the experimental phase on simplicity grounds since it could be borrowed from the FH JOANNEUM. However, it is likely that a smaller microcontroller with similar performance such as an Arduino Micro will be used for the final setups in project phase three.
Regarding the sensor for the left hand setup, diligent research and a direct performance comparison lead to the decision to use a time-of-flight sensor to pinpoint the position of the guitarist’s hand along the neck instead of the initially planned ultrasonic sensor. After evaluating the specifications of several ToF sensors, the sensor of the type VL53L1X was finally chosen.
For the right hand setup on the other hand, a suitable IMU sensor was found rather quickly. Although an MPU9250 was briefly considered and used during an initial Arduino library test, it was then dropped for a BNO055 sensor, following the recommendation of the author’s supervisor.
Of course, next to these main components, other equipment including cables, breadboard, electronic components, etc. were acquired.
1.2. Goal 2 – sensor position and placement
All in all, this goal has been largely fulfilled with some improvements possible in the third phase of the project.
As outlined above, the usual positions of the fretting hand have been determined and subsequently analyzed. This analysis led to the conclusion that, unfortunately, the posture and the exposed reflection area of the fretting hand vary a lot depending on what is played, with major differences between playing barre chords and single notes for instance. These inconsistencies in hand posture were and still are a major constraint to the left hand setup and its flawless implementation into the natural playing style of a guitarist.
As far as the ideal place and installation of the sensors is concerned, a lot of progress was made with regards to the left hand setup. Based on the afore-mentioned analysis of the hand posture, several attachment devices were made for the ultrasonic as well as the ToF sensors and, subsequently, compared. One position in particular (IMAGE) proved to be better than the others, albeit being not perfect. The position chosen works best for barre chords as well as the Solo Mode application.
Regarding the attachment device of the right hand setup, there is definite room for improvement. The wristwatch solution was sufficient for the experimental phase and proved that placing the sensor on top of the back of the hand serves to get useful sensor data. However, it is unsuitable for the final product. While a wireless solution is optional for the left hand setup, the right hand setup would certainly benefit from the lack of cables. It would enable an even more natural playing of the guitar.
1.3. Goal 3 – programming microcontrollers
Here, definite progress was made, especially with the left hand setup. With no prior experience in programming the learning curve was quite steep, and a lot of time had to be dedicated just to learn basic coding techniques.
Regarding the left hand setup, Arduino sketches were made firstly for the ultrasonic sensor and, subsequently, for the time-of-flight sensor when the former proved to be unsuitable. In addition to code needed to access the basic sensor data, the mathematical relationship of the guitar fret spacings was established, fret ranges were determined and finally implemented in the code. Next to absolute distance measurements, detecting the fret numbers is possible up to the ninth fret which already enables applications such as the Solo Mode.
The code for the right hand setup on the other hand is not yet as advanced mainly due to time constraints. Using a library, orientation, acceleration, and calibration data could be obtained from the IMU sensor and transmitted to Pure Data via MIDI. The y value from the orientation data is the only data so far that is suitable for further use to control effect parameters of a Wah Wah effect. Here, more variety in data usage would be desirable.
Lastly, different data transmission techniques were tested and evaluated. A lot of time was spent with first MIDI USB libraries and then MIDI only libraries with transmission suffering from a lot if lag initially. Serial port communication proved to be the first viable solution, fast enough to control effect parameters. Finally, the latency problems of MIDI could be eliminated. The current setups work via MIDI communication using a MIDI cable. For the third phase, a wireless means of data transmission would be desirable – especially for the right hand setup.
1.4. Goal 4 – programming of Pure Data patches
This goal was achieved to a large extent. Albeit initially working only with great latency issues, making a patch that receives MIDI data and is able to use it for further processing was achieved rather easily. The interim solution, serial port communication, took some more research but once the basic method was discovered, its application was straightforward. For the left hand setup, the incoming fret numbers can be either used directly to control effect parameters or, using the “moses” object for instance, a fret threshold can be set to make an ON/OFF switch. The patch for the right hand setup is very similar to that of its counterpart and effect parameters can be controlled.
1.5. Goal 5 – effects programming
As far as goal 5 is concerned, mixed results were achieved. Working with the digital audio workstation (DAW) Steinberg Cubase Pro 11 proved to be more difficult than previously anticipated and, consequently, it was decided to work with Pure Data only during the experimental phase. Nevertheless, in order to ensure a seamless integration into the guitarist’s natural workflow, a DAW integration of the final product is desirable. Thus, it will be tried to accomplish this in the third phase of the project.
Regarding the decision between using self-made effects or third-party plugins, both approaches were tested. The first patch contained a self-made delay and overdrive/distortion effect which proved to be useful for the first test but had definite shortcomings tonal quality-wise. Thus, the patches from then on used the object “vstplugin~” to implement third-party plugins in the Pure Data environment. The exception is the Wah-Wah effect for the right hand setup which is self-made and works well.
1.6. Goal 6 – determine suitable effects
With a stable data transmission between Arduino and Pure Data achieved at a relatively late stage of the semester, this goal could not be fully achieved. Tone experiments involving several effects and their parameters were conducted. For the Solo Mode application of the left hand setup, an amplifier, reverb and delay were tested. Less tonal experimenting was done with the right hand setup: three effects tremolo, phaser and Wah Wah were tested with only the latter representing a reasonable effect to be controlled by the right hand setup.
It is evident, that only the surface has been scratched so far and much more in-depth research and experimenting in both setups will be needed to really provide practical applications for extending the range of possible guitar sounds.
1.7. Conclusion
In conclusion, it can be stated that all tasks set to be done during the second phase of the project have been approached and tackled with the majority of goals at least partially achieved. Additionally, setup compatibility with working hypotheses 1 and 2 was consistently ensured, with all effects working so far not invading the usual way people play guitar. As stated in the Exposé and in chapter 4 of this documentation, the overall aim of the second phase was to develop working setups that are sufficiently reliable and allow for further practical research regarding suitable effects, playability and performability. While the left and right hand setups are far from being a final product or ready to be tested by other guitarists, the second, experimental phase yielded a lot of progress. Overall, it can be affirmed that the left and right hand setups, albeit having shortcomings in some areas, are advanced enough to serve as a base for further practical research in the third semester.
