I’ve been thinking about how to build a good robot guitar off and on for the past 15 years. My interest in musical robots started when I joined the RobOrchestra project at the Carnegie Mellon Robotics Club in college. After graduation I focused my career on automated industrial equipment but was always on the lookout for cool musical robots popping up on the internet. Now, with more experience, spare time, tools, and funding it is finally time to revisit some of the elusive robotics projects that have been keeping me up at night for over a decade! In this post I’ll be reviewing my favorite guitar playing robots from the internet in order to help me set goals for my own upcoming guitar playing robot design.
Musical Robot Background
When I joined the RobOrchestra group they had a working xylophone, snare drum, and a recorder. During my four years we added a bass drum, hi-hat, whamola, vibraphone, and harmonica (with varying degrees of success). Every project was done on a shoestring $500-$1000 budget! I always thought that these kinds of projects are perfect for a university student: you will never get an opportunity to do something similar in industry because of the complete lack of commercial viability, and the tiny budget forces you to get creative and try interesting solutions to problems.
The vibraphone project (2010-2011) was probably my favorite. Inspired by the viral Animusic videos online we arranged the bars of a vibraphone in a circle and struck them with ball bearings dropped from a chute! It was a fun engineering project because it involved a lot of cost optimization, prototyping, and a variety of manufacturing techniques. I was also pushed outside of my purely utilitarian comfort zone to design something not just to “work”, but to draw people in and captivate them. I was proud of the result, despite its imperfections, and it ended up being a great project to talk about in job interviews.
The only stringed instrument we made during my time at Carnegie Mellon was a whamola. It is played percussively (with a drumstick), and the pitch is changed by altering the string tension (instead of fretting). This made it much easier to build than a guitar robot because it bypasses all of the hard parts of playing a guitar!
I’ve always tried to stay up-to-date on the latest musical robots since leaving college. It helps that my friends are always sending me links to the cool videos they come across (when Wintergatan released the marble machine video absolutely everyone I knew from college had to send me a link because if its similarity to my Vibratron!). Because of this, it was fairly easy to put together an assessment of all of the existing robot guitars out there (that I know of) to try and learn more about what works and what does not work.
Discrete Picking Robots
After watching dozens of guitar robot videos, I tend to categorize them by two criteria: How they pick, and how they fret. Most designs use what I call ‘Discrete Picking’, where each string is assigned a single, dedicated actuator. Rarely I find a design that uses ‘Variable Picking’ where a single actuator is responsible for picking multiple strings.
The most common way to solve the fretting problem is what I call ‘discrete fretting’, where each combination of fret bar and string is assigned a single, dedicated actuator. There are a variety of alternate designs that give each string an actuator (that slides along the fret board), or even hybrid approaches that slide a limited array of discrete fretting actuators along the fret board.
Discrete Fretting
With this style of robot guitar there is usually one actuator per string and one actuator per fret zone+string combination. The obvious advantage is that you can play notes very rapidly (no need to share actuators between notes) and the software is more straightforward (you don’t need to plan any moves). The big disadvantage is that there is only one way to play each note (no ability to bend/slide notes or pluck with a different strength). A secondary disadvantage is that the robots can quickly get absolutely buried in a mess of cables and actuators!
The most representative guitar robot of this style is probably Demin Vladamir’s robot guitar project. He has been working to perfect this design for the past ten years and has gotten it to a very reliable and consistent state. It uses one pick per string (servo controlled) and what appears to be a vertical motion on the pick mechanism for damping. Each potential fret point has a solenoid mounted directly above.
Earlier prototypes were mounted on T-Slot framing, which seems like a great way to iterate a design. Many discrete-picked designs suffer from the problem of sounding too much like a harpsichord because of their inability to vary how strongly a note is plucked, the inability to alter the pitch through bending or other means, as well as the fact that they use a pick instead of a finger. By choosing ‘traditional’ songs and melodies for his robot to play, the creator was able to mask the harsh sounds of his early prototype really well.
While I’m not entirely certain, I think this design ended up on another guitar at an exhibition in 2019, which solidifies its position as the top tier design in its class.
While there are plenty of other examples of similar designs out there, they all share the same basic characteristics. This particular design is the highest performant and most reliable of the discrete-pick and discrete-fret type that I have found so far. It is probably also worth highlighting that the accordion robot from this same creator is absolutely fantastic, especially with the cool human-assisted feature!
Linear Slide Single-Fretting
The most popular non-discrete fret style out there is a simple linear actuator that follows the axis of a string to position a single fretting actuator. The obvious benefits of this style are that you can slide between notes while they are being played (sweet on an electric guitar!) and avoid needing to organize several dozen actuators to ensure a wide range of potential notes. The obvious downside is that it can be very difficult to play two notes on the same string in quick succession if they require fretting at a vastly different location (human players tend to ‘mask’ a note fretted higher on the neck with one finger while they play a note lower on the neck, but on the same string, with a different finger).
The best design I have found in this category is MechBass by James McVay. He uses T-Slot extrusion as a linear guide and controls the position of the fretting mechanism with a stepper-driven timing belt (like a 3D printer). The plucking is done with guitar picks mounted to a rotary device driven by stepper motors, and the actual fretting is done with a guillotine style device (looks to be solenoid driven) mounted to the end of the fretting timing belt.
What really sets this design apart from other linear slide designs is the use of a damping actuator on each string. Without this actuator notes end up blurring into each other, which is usually not what the song requires.
Linear Slide Multi-Fretting
The only two guitar robots I know of that are (were?) part of actual performing bands both use this style of fretting. They have a complicated ‘hand’ (neck-mounted carriage) with multiple different actuators that can be driven up and down the main axis of the guitar into different positions (or to make sweet effects!). Like all of the other examples so far, both of these designs still use discrete picking to actuate each note.
Compressorhead is a German metal band that includes a bassist (‘Bones’) with a linear slide fret design. From what I can tell, each finger (of the left hand) is capable of engaging any of the strings within the fret it is aligned to, which gives the robot a lot of range without having to drive a high performance from the linear slide.
The lead guitarist of Compressorhead (‘Fingers’) has a similar design. The left hand is a carriage that can slide up and down the neck of the guitar with a 5×6 (5 frets on 6 strings) array of actuators for engaging the strings (although I have never seen the carriage slide during a song?). Fingers also has a secondary fixed grid of actuators mounted at the base of the neck, presumably for fast-paced face-melting solos.
Of course, you can’t mention Compressorhead without also bringing up Z Machines! Z Machines is a metal band from Japan that also built a robot guitarist. While they don’t have quite the same grungy charm of Compressorhead they definitely boast some seriously high performance from their guitar: over 1,000 bpm from some tests!
Like Compressorhead this robot also uses alternating (instead of rotary) pick actuators (one per string), but their arrangement of the picks at a steep angle sets them apart from most other discrete pick mechanisms.
They have at least 7 rows (one per fret) of 6 fret actuators (one per string) on their carriage. While the whole carriage of fret actuators does move, I’ve never seen it used for repositioning the actuators (only for creating a slide effect).
While I really appreciate the shear speed of the Z Machines guitar robot, their songs tend to sound like a track out of a Super Nintendo game. Compressorhead on the other hand has a way more human feel to their performances (not just because of all of the extra animatronics) but because of their song selection, amplification, and actuators.
Manual Fretting
The easiest option is obviously to actuate the frets yourself! This is of course how Martin played the strings in his marble machine. Another take on setting the frets by hand is to either tune/tension them ahead of time or pre-fret them (maybe with a capo?) for each arrangement. These sorts of manual fretting options are how many other robotic guitar projects have gotten started, in combination with discrete picking, because it is extremely simple.
Variable Picking
While there is certainly more than one way to actuate a guitar pick dedicated to a specific string (sliding, twisting, spinning, or even a cam system) the design space really opens up when a single pick is shared between multiple strings.
Strumming Cam
One of the only examples I know of with an actual strumming stroke is the lego guitar robot by TECHNICally Possible. His behind the scenes video explains all of the cool cam mechanisms involved in both the strumming and the fretting.
The strumming is interesting because while the normal stroke from the linkage moves the pick back and forth across all six strings, there are multiple other cams and actuators that can alter that behavior. There is a linkage that lifts the pick high enough to avoid touching the strings. This linkage is connected to the output of a cam system with three possible states: do not lift (for playing up and down stroke) lift on the upstroke (for playing only the downstroke) or lift for all strokes (to not play at all). Meanwhile, a separate mechanism rotates the entire assembly so that the pick can ‘fall’ in between the first and second strings for some chords (to avoid playing the first string all together).
The fret actuators are also interesting, although I would still categorize them as ‘discrete’. There are only four specific chords that can be set on the frets, and in one specific order (the robot can basically only play one song). A series of cams sets each fret point as pressed or not pressed based on the position of a single common fretting actuator that has four potential positions (one for each chord). Very cool! I think something like this could be useful in Martin’s marble machine to help hin avoid having to manually fret his strings.
While this approach is very novel, it is extremely limiting. However there is still a lot to learn from this design and it is a really impressive result (even if it wasn’t made out of legos!)
Simple Pick Arm
My favorite example of a multi-axis mechanism for picking a guitar string is from Clayton Darwin and his mandolin project. This two-axis, direct-stepper-drive mechanism can reach out and pluck any of the eight strings! The kinematics remind me of a cross between an industrial 6 axis robot arm and an industrial ‘delta’ style robot arm, but the wooden components don’t give the final product an industrial feel at all, which is awesome.
I’m not sure what the limiting factor for speed is with this setup. With powerful enough motors could you eventually match the speed of Z machines? Delta robots can go really fast with very high accelerations, so I wouldn’t be surprised if the performance ceiling on a design like this was extremely high.
The StrumBot from Richard Vindriis is another great example of single pick meant to interact with multiple strings. In his paper, Richard shows how the position of the pick from the bridge can impact the timbre of the note, and the angle of the pick can impact the final volume of the note.
This robot is also noteworthy (especially when compared to its cousin MechBass) because the sliding frets have a built-in pitch-bender actuator (achieved by driving the damping actuator in reverse) that can execute tremolo-speed effects!
Industrial Pick Arm
But why build a simple, elegant arm to pick the strings when you can just use the $50,000 arm in your university lab? That’s what Georgia Tech did with this project that required manual fretting.
When the robot is strumming it definitely feel like a waste of 5 degrees of freedom (especially because they angle the pick with a custom end effector, not the wrist). However, when a lot of picking is required, the robot is able to easily travel a path between all of the strings to pluck one at a time rather quickly.
Industrial Pick and Fretting Arms
Of course, the next logical step is to add a second industrial robot arm to handle the fretting. That’s exactly what Nigel Stanford did in this partnership with Kuka. Despite the video being an 11 on the awesome scale, this specific robot guitar was fairly underwhelming (only played one string, no damping).
You can see from the behind the scenes video that it was just some manually set points for the guitar portion of the video, but it’s easy to imagine the potential for more automated pathing after lots of additional software work. The obvious downside here is the enormous cost for the robots and the extra infrastructure (power, foundation) that you need to support them.
Goals for this Project
After surveying the progress in robot guitars over the last fifteen years I am convinced that there is still a lot for someone to contribute to the space. The biggest general problem with most of the designs is that the music just doesn’t sound as good as when a human plays. My hypothesis is that there are a few ‘musical degrees of freedom’ missing in these existing robots that prevent them from properly playing the songs they are given. These ‘musical degrees of freedom’ are things like note bending and varying the strength of a note.
There is also a lot of good progress that has been made that I need to not forget to include in my own project. Not only are there ‘musical degrees of freedom’ that have been included with success (like note slides) but also there are plenty of design choices that have succeeded in drawing audiences in. Things like keeping all of the wiring and mechanisms neat and orderly, not obstructing the instrument itself, ‘imitating’ human interactions, and building the mechanisms into the theme and design of the instrument have helped make the final product a very cohesive design.
My overall goals for the project are to:
- Maintain the look of the instrument being played (not hiding it behind a mess of wires or mechanisms)
- Keep the mechanisms quiet enough to be played on an acoustic instrument
- Imitate the human form where possible (but not just making an animatronic)
- Be able to play a large variety of music from standard formats
My goals for the picking aspect are:
- Allow for a large range in note strength (loudness)
- Allow for an optional non-pick string interface that more closely simulates a human finger
My goals for the fretting aspect are:
- Allow for note bends (1/2 step), usually done by moving the pinched string laterally on the fretboard
- Allow for note slides along the neck of the instrument
- Allow for hammer on/offs
Packing all of this into one design will be difficult but I think it can be done. There will definitely be a lot of subscale testing along the way to help provide slow and steady progress towards a solution. I hope to make the solution generally adaptable to stringed instruments of different sizes/counts/arrangements even if the specific builds differ slightly.
The biggest challenge will likely be my poor understanding of music theory. Many of the more creative designs from my survey used their understanding of chord progressions and scales to make clever design reductions. Despite my parent’s best efforts to get me to pick up an instrument as a kid (many disastrous years of piano lessons) and four years in a robotic instrument group at college I really don’t know enough about music to make those sorts of design decisions. This project will hopefully be an opportunity for me to learn more about music theory through personal research and working with more educated people from my community.
The next step will be to turn this set of goals into a series of technical requirements, and from there I can begin to brainstorm solutions and make prototypes to test them!