The Surrogate Hands

Future of Labor, productivity and agency: How will robotic technologies extend the understanding and value of work beyond substituting human or animal labor?

At the age of pandemic and social distancing, learning craft and collaboration between craftspeople is a mirage from the outmost reaches of time. Surrogate hands is an effort to bridge the distance between craftspeople and the physical world of craftsmanship.

The proposed framework is beyond a robotic installation or interactive performance. It aims at exploring the new ways of interfacing distanced tools and companions.

The surrogate hands inquires about applying machine learning and robotics in two directions; first, training robotic tools to serve as a surrogate for the craftspeople who cannot physically attend their workshops. Second, it is an effort to explore machine learning and robotics' affordances to teach craftsmanship to apprentices.  This proposal only focuses on the first goal and leaves the second approach for another opportunity.


In its traditional conceptualization, workshop is a sociophysical platform where apprentices reside and learn from masters and each other. In the age of pandemic, the social aspects of workshop are surviving through digital vehicles, and the collective process of continual learning thrives remotely. However, the physicality of workshop is eroded by social distancing practices and is now abstracted to a virtual experience.

The surrogate hands is an inquiry on the intersection of machine learning, robotics, and craftsmanship to investigate the possibility of collective physical toolmaking through remote means. In this process, craftspeople collectively and interactively train a machine learning model to control a robotic arm to perform a designated task.

Participants collectively contribute to the development of a shared database that encapsulates samples of actions. The robotic arm also constantly actuate samples, generates new ones with slight variations to the original samples to enrich the library.

The ever-growing library of actions will let each participant curate its customized dataset by observing and integrating other participants' samples as well as the synthesized ones. Throughout this process, they will interactively train a machine learning model to control the robot and actuate their bespoken samples. This model will serve as a virtual surrogate of the participant, and the robotic arm plays the hand.

The final demonstration will be a collaborative act where models trained by each participant will be used to execute a designated task. A "master" will curate the demonstration, and an algorithm decides which participant's model should be utilized at each step of the process.