Introduction

We were given the task of creating a musical instrument with some sort of haptic feedback for the user. We first did individual design proposals and were organized into groups based on our ideas in those proposals. Based on our group's proposals, and one member's familiarity with acoustic guitars, we decided that our electronic instrument would be heavily inspired by a six-string acoustic guitar. In our group we found strengths especially in Arduino programming and MIDI/pd communication, and decided to use these as hardware and synthesizer.


The main components that we wanted to create was strumming of the "strings", creating different notes based on the string's position in the frets, and add light and vibrations to correspond with the note being played.

The members of our group are:

Kevin Melotti

Sean Murphy

Lisa Perez

Body Structure, Fret Sensing, and PD code

1) Body Structure

We first considered a minimalist shape, maintaining only the necessary contact points of an acoustic guitar. See an example of this in the Yamaha SVC-210 electric cello -->



However, after further consideration, much of an acoustic guitar is a contact point, and for simpler construction and electronic packaging we went with a traditional shape.

We wanted to keep the body fabrication simple. The guitar body is lasercut from 4mm wood, to create front and back sides. The two sides are joined with standoffs, and the electronics are attached by velcro to reduce necessary hardware. The only complication was the maximum size of the blanks which fit in the lasercutter, which forced us to cut the guitar from four parts instead of three.
We are pleased with the final shape and durability. It's blue paint, block M and recognizeable shape drew people to it at the expo, and it's structure was robust enough to survive with no damage.

2) Fret Sensing Concept

One of the key features of our instrument is its expressive range. We desired to keep all the note range and capability of an acoustic guitar as a baseline functionality. In order to sense each possible fret position of each six strings, we needed to be able to sense 6x24 positions independantly of each other. However, in a real guitar, only 6 positions are needed at a time, which is each string at the lowest fret fingered. Instead of attempting to assemble 144 sensors, we instead built the 24 frets to each carry a unique voltage, and then use the six metal strings themselves to close the circuit to the arduino analog in ports. The guitar neck is a giant voltage devider, and the strings pick off the voltage when applied to a fret. Otherwise the strings are pulled down to ground. As the analog in ports draw little current, the voltage devider sees very little change in Vout when the string 'load' is applied, and all strings can be fingered with reliable results.
In practice, it is difficult to establish a good connection between the strings and the frets, and while possible, each string must be carefully fingered to acheive the desired chord.

3) Pd code
We chose to produce MIDI output from the guitar, and use Pd as a synthesizer. We decided to do this because we were confident we could produce a satisfying output in this manner. MIDI was only used as a communication standard, as we were creating the Pd patch from scratch, we did not use standard MIDI commands.
Basic MIDI includes note-on and note-off commands, and the simplest synthesizer will produce a pure tone of the commanded note at the commanded volume until the note-off command. A guitar is much different, with each string acting largely independantly of each other (ignoring the excitation one string may cause on another) and each sounding at the same time as the others and with their own overtones. Additionally, ignoring fret-work (hammer on/off, bending) the strings work on their own after initial impulse excitation, dynamic volume is not necessary. The structure of the Pd patch reflects the impulse input control, and independance of each string.
The Arduino sends the note and volume when the string is triggered. Pd receives the command and acts on the trigger, sending the command to the appropriate 'string' subsystem, which sounds the note and calculated overtones with the triggered ASDR envelope ("Attach, Sustain, Decay, Release"), tuned according to our observed acoustic guitar behavior. Each of the six strings has its corrosponding subsystem, which follow their own envelopes. This allows chords when multiple subsystems are sounding at the same time, triggered quickly in succession in a natural strumming motion.
There is also capability to pitch bend the final total output of the patch. If there is a continuous sending of MIDI information above the string information range this will work on the pitch shift.
The sound generation is simple: a phasor is constructed at the main frequency and at smaller magnitudes at the first two overtones. The ASDR envelope has an aggressive attack to emulate the aggressive attach in a string pluck. While effective in creating a pleasant and musical tone, there is further room to acheive a more accurate acoustic guitar sound, if desired.

Pd guitar front-end. Note-in sends information to correct 'string' or pitch bending


Example 'string'. Each string has unique asdr envelope


Borrowed ASDR envelope patch

Main Body's Arduino

Strumming

While we were brainstorming we thought of ideas to create a feeling of a pick being forced over multiple strings. To accomplish this task we used a linear actuator with an attached potentiometer to create the haptic environment. Specifically we used the motorized fader provided by class. We had the idea of creating a virtual spring that would give way once the "string" was passed. The force curve would look similar to a ramp function. To create this kind of feel we needed to know the position and direction of the velocity from the potentiometer. We used an op amp circuit in order to find the velocity direction. The circuit diagram below shows how we found the position and velocity from the potentiometer on the fader motor.

Once we had the position and velocity for the Arduino we used an H-Bridge to control the motor of the fader. We compared the current position to the position of the virtual spring and as the current position came closer to the virtual spring the force would increase. Once the position past the string the force would let up. We used the velocity to determine which direction the fader is moving so that we can get the string feeling regardless of which direction the user was strumming. The code for this can be seen in the Appendix Post under older posts.

Detecting and Sending a Note


To determine when a note was being played we monitored the position of the motorized fader. When it crossed a particular position (designated as a "string")we proceeded to send a note and send a signal to the other Arduino. We used a MIDI to communicate with the computer using the connection diagram found in this tutorial. With that we sent a signal to the computer. The signal consisted of three bytes of data (channel, note, volume). We used channel 1 for all communications. The note was determined by the particular string being strummed and which fret that string was pressed into. So for string 1 we sent a number between 0 and 15 (we had 15 frets and open), for string 2 we sent a number between 16 and 30, and so on up to 6 strings. For the volume we sent a byte between 70 and 110. The particular value was determined by an accelerometer located in the guitar's body. We used an axis aligned with the neck of the guitar and used gravity to determine the tilt of the guitar. We then sent a larger value for the volume when the neck was tilted up and a smaller value when the neck was tilted down. The code for this can be seen below.

Communication to the Other Arduino


Another task that the main body's Arduino had was to send a signal to the other Arduino signaling that a string had been strummed. This is required to allow the other Arduino control the lights and pager motors at the head of the guitar. To do this we were going to send the signal that corresponds to the volume of the guitar when a string is strummed. We went about this by using an analog write to a pin that the other Arduino would read. But since an analog write on the Arduino board produces a PWM and reads a specific value we were only getting 0 or 5 volts reading. To overcome this we put the signal through a low pass filter to smoother the signal out.

To create a low pass filter we had a signal go through a 5.6 kOhm resistor then at this node we had a 1 microFarad capacitor connected to ground and a wire going to the other Arduino. The code that we used to communicate can be seen in the Appendix Post in older posts

Head of the Guitar

The head of the guitar is where the vibration of the body and the visual feedback are controlled. This part of the design consists of an Arduino Duemilanove controller, 2 pager motors (vibration), and 6 LEDs (visual feedback). The vibration and visual feedback are discussed separately below.

Vibration: To give the feeling that a real acoustic guitar gives when its strings are plucked, we decided to include 2 pager motors that spun at a certain speed at the neck of the guitar as soon as a one of the virtual strings was plucked. The certain speed was determined by what the volume was – the higher the volume, the higher the speed. To get the volume data, we ran a line from an Arduino Mega Analog Out pin to the Duemilanove’s Analog In pin. We mapped that data to a full PWM range, which created 2 signals that controlled both motors.

As long as the virtual strings were not being plucked, the speed of the motors would attenuate over a given amount of time to simulate the dying down of the vibrations. We based this given amount of time (about 15 seconds for loud plucks) on tests that we did with a real acoustic guitar.

Visual Feedback: We decided to add an element of “Star Power” in our design by creating a light-up block M that had 3 levels of lighting that would react to the volume in a manner similar to the pager motors. To do this, we mapped the same volume data to PWM ranges, which created 3 signals that controlled 3 sets of 2 LEDs in parallel.

As long as the virtual strings were not being plucked, the lighting would attenuate over a given amount of time. The top tier of LEDs was mapped to the smallest PWM range, so it started to dim first. The second tier was mapped to the middle range, so they started to dim about a second after the top tier. The third tier was mapped to the highest range, so it dimmed last.

The code for this can be found in the Appendix section under older posts.

Layout:

Assembly and Tuning

Upon assembly we found some areas did not go together as smoothly as planned. The biggest problem was the strings and accelerometer input into the lower arduino; the values read did not make sense, and did not match voltmeter readings made on the inputs directly. This was finally resolved when we found the Arduino v.17 program to have a bug in the computer software side. When this was resolved, the readings cleared up. Other more minor issues in were:

a) The body structure was not sufficiently reinforced at the body-neck connection, and the neck had become quite heavy with the upper arduino and voltage-devider fret board. This was fixed with a neck-body reinforcement made of balsa, wood glue and through-bolts.

b) The strings were not sufficiently 'pulled-down' and an open fingering did not give consistant results. Resolved with different pulled-down resistor values

c) The accelerometer was not functioning properly. This was cleared up when the datasheet was examined and the part was found to function on 3V instead of the 5V we were running it at.

A video of our final product can be seen below:

Product at MechaMusicTronics Event

We took our design to MechaMusicTronics in the Duderstadt Gallery. We were slightly worried about how it would go over with the crowd, especially since it wasn't tuned perfectly. However, guitar players and non-guitar players alike definitely engaged with the design, from the fader motor's ability to simulate plucking a string, to the voltage divider frets, to the vibrations of the guitar, to the light-up M.

Here are some pictures from Monday:

Future Functionality and Design Revisions

Now that the baseline guitar is constructed, future work can add novel functionality and expand the expressive range of the instrument. First, pitch bending may be implemented - the accelerometer hardware and pd code is in place, just the Arduino code needs to be adjusted, and then overall pitch could be altered by tilting the guitar.


Another idea we had was a built-in sequencer. The arduino memory could be used to store a section of playing, perhaps commanded by another external button, and then that could be played on command, or looped under continued playing. Different expansions on this same idea: the Arduino could loop just the fingering or just the picking, allowing the player to concentrate on the other to produce more complex music more easily.


To expand the 'acoustic guitar' functionality, the arduino could carry multiple tuning maps onboard, to be selected by the player to automatically assign the fretboard different tuning, or a virtual capo.

Appendix

Appendix A:

Code for Main Body's Arduino

const int string[] = {5,4,3,2,1,10};//10,1,2,3,4,5};
int stringval[6];
int pos[6];
const int position = 6;
int dir = 12;
int pwm = 13;
int direct = 7;
int i = 0, q = 0;
int j = 0;
float k = 1;
int dir1 = 0;
int vol = 0, xval=500, var=0;
const int commout = 3, xaxis=8, yaxis=9;
int pitch = (255-146)/2;

void setup() {

Serial.begin(31250);
pinMode(direct,INPUT);
pinMode(pwm,OUTPUT);
pinMode(dir,OUTPUT);
}

void comm(int x, int a, int y, int z, int b){
if (x <> 50)
{
//Sends a MIDI signal and the other Arduino a signal
if ((x-a+3 > z-a-3)) {
noteOn(0x90,16*(z/(1023/7)-1)+stringval[z/(1023/7)-1],b);
analogWrite(commout,map(vol,70,110,0,150));
delay(15);
}
else if (a-x+3 > a-(z-1023/7)-3) {
noteOn(0x90,16*(z/(1023/7)-2)+stringval[z/(1023/7)-2],b);
analogWrite(commout,map(vol,70,110,0,150));
delay(15);
}
//Creates a feedback for the user (virtual spring)
if (y <495){>525)
{
if (x>z-1023/14+15) {
analogWrite(pwm,0);
} else {
analogWrite(pwm,abs(z-x)*k);
analogWrite(dir, 255);
}
}
}
else {
analogWrite(pwm,1);
}
}




void noteOn(byte cmd, byte data1, byte data2) {
Serial.print(cmd, BYTE);
Serial.print(data1, BYTE);
Serial.print(data2, BYTE);
// Serial.print("cmd ");
// Serial.print(cmd, HEX);
// Serial.print(" data1 ");
// Serial.print(data1, HEX);
// Serial.print(" vol ");
// Serial.println(data2, HEX);
}




void loop() {
//reads the value of each string to determine fret position
i=i+1;
q=q+1;
if (i==6) {
i=0;
}
stringval[i]=map(analogRead(string[i]),-14,970,0,15);///(1023/15);


//reads the position of the fader and direction it's traveling
pos[i] = analogRead(position);
dir1 = analogRead(7);
vol = map(analogRead(xaxis),295,400,110,70);
if (i==0) { j = 5; }
else { j = i-1; }

//Sends the position, past position, direction, string it's closet to, and the volumn to the

//commuication function
if (pos[i] <= 1023/7) { comm(pos[i],pos[j],dir1,1023/7,vol); }else if (pos[i] <= 2*1023/7) { comm(pos[i],pos[j],dir1,2*1023/7,vol); } else if (pos[i] <= 3*1023/7) { comm(pos[i],pos[j],dir1,3*1023/7,vol); } else if (pos[i] <= 4*1023/7) { comm(pos[i],pos[j],dir1,4*1023/7,vol); } else if (pos[i] <= 5*1023/7) { comm(pos[i],pos[j],dir1,5*1023/7,vol); } else if (pos[i] <= 6*1023/7) { comm(pos[i],pos[j],dir1,6*1023/7,vol); } else { comm(pos[i],pos[j],dir1,7*1023/7,vol); } //sends the other Arduino a constant flat line analogWrite(commout,1); }





Code for Arduino at Head of Guitar:
int i = 0; int var = 0; int varLed1 = 0; int varLed2 = 0; int varLed3 = 0; int myData = 0;

int ledPin1 = 3; int ledPin2 = 5; int ledPin3 = 6; int motorPin1 = 11; int motorPin2 = 10; int volInput = 1;

void setup(){
Serial.begin(38400);
pinMode(motorPin1, OUTPUT);
pinMode(motorPin2, OUTPUT);
}

void loop() {
myData = analogRead(volInput);
Serial.println(myData);
if(myData > 50) { i = 0;}
var = map(myData, 0, 900, 0, 255);
varLed1 = map(myData, 0, 900, 0, 255);
varLed2 = map(myData, 0, 900, 0, 175);
varLed3 = map(myData, 0, 900, 0, 125);
if(i <255){
analogWrite(motorPin1, var-i);
analogWrite(motorPin2, var-i);
analogWrite(ledPin1, varLed1-i);
if(i <125){
analogWrite(ledPin2, varLed2-i);
analogWrite(ledPin3, varLed3-i);
}
else if(i <175){
analogWrite(ledPin2, varLed2-i);
analogWrite(ledPin3, 1);
}
else if(i <255){
analogWrite(ledPin2, 1);
analogWrite(ledPin3, 1);
}
delay(5);
i = i + 1;
}
else{
analogWrite(motorPin1, 1);
analogWrite(motorPin2, 1);
analogWrite(ledPin1, 1);
analogWrite(ledPin2, 1);
analogWrite(ledPin3, 1);
}
}