Yushan Chen, Co-founder & COO of Artly AI, shares how the upgraded robotic hand powered by vision-based AI is bringing human-like precision to coffee-making—and beyond.
In a conversation with Robotics Business NEWS, Yushan Chen, Co-founder and Chief Operating Officer of Artly AI, dives into the latest upgrade to their Barista Bot—a leap from an industrial gripper to a bionic hand powered by an advanced AI platform. This transformation enables the robot to handle complex, delicate tasks with a level of dexterity that mirrors human skill, signaling broader potential for robotics in hospitality, healthcare, and service industries.
What are the key advancements in the new natural robotic hand upgrade for the Barista Bot?
The first upgrade is the hardware. We have replaced our custom industrial gripper on our current Barista Bot with a bionic hand. This enhancement allows our robot to manipulate everyday objects, such as fresh fruits and kitchen tools.
The software has also been improved. We upgraded our imitation learning platform to capture human hand and finger movements with millimeter accuracy. By combining this with motion capture data of human actions, our vision-based action AI model can achieve high-precision movements that are ready for commercial applications.
This upgrade is a significant step toward our vision of a humanoid capable of manipulating tools and objects skillfully, or “artly.”
How does AI enhance the dexterity and precision of the robotic hand to replicate human-like coffee-making skills?
AI action policy models are required for a bionic hand to complete daily tasks in the service industry. Unlike simpler industrial grippers, which are purposefully designed and programmed to follow predefined rules, bionic hands have a greater degree of freedom. Consequently, controlling them to manipulate objects and tools is significantly more complex.
It’s challenging to create customized control policies for bionic hands to operate tools and objects of different shapes like a human would. This is where AI comes in. With it, we can use motion capture data to control the bionic hand and replicate human motions.
Compared to the widely available language data used to train large language models like ChatGPT, data for robots with industrial grippers operating in the service industry is extremely scarce. In contrast, images and videos of human hands using daily tools and objects are abundant; this data can be used to pre-train an AI model for a bionic hand.
What challenges did Artly AI face in developing a more natural robotic hand, and how were they overcome?
The application of bionic hands in the service industry is still a nascent use case. As an enabling platform, Artly is supporting various robotic hands, arms, and their combinations. Their form factors, functionalities, and interfaces continue to evolve rapidly. At Artly, we are constantly exploring ways to improve our AI model so that it can adapt to different hands and arms without sacrificing performance.
How does the upgraded Barista Bot improve customer experience and efficiency in coffee preparation?
With a bionic hand, our robot barista can make the coffee experience even more authentic. For example, our robots can now handle a wider range of everyday items, such as glass and ceramic cups of different shapes and sizes. Moreover, our robot can prepare even fancier drinks—like melting real milk chocolate for a mocha or adding garnishes to the finished cup—just like a real barista would.
What role did NVIDIA’s technology play in advancing the AI capabilities of this robotic system?
NVIDIA provides crucial components for our robotic system. Artly’s Barista Bot relies on NVIDIA’s AI accelerators to execute mission-critical, computer vision-based action models that control its movements. NVIDIA’s cloud-based GPUs enable us to train these models, while NVIDIA’s Isaac Sim enhances our training data, allowing us to develop more reliable action models.
Beyond coffee-making, what potential applications does this natural robotic hand technology have in other industries?
The potential applications of the bionic hand extend far beyond making coffee. Its dexterity and precision, combined with the ability to replicate human touch, are perfect for a wide range of tasks in the service sector. From preparing and serving food and drinks in restaurants and cafés to providing personal care in healthcare settings, the bionic hand can assist both the staff and patrons by offering human-like interactions.
What are Artly AI’s future plans for expanding AI-powered robotics in the food service industry?
Artly is working to create a platform that faithfully replicates human craftsmanship through robotics and AI. On the hardware side, our robot will more closely resemble human actions. On the software side, we will enhance our VLA AI models’ efficiency to improve its performance with less data. Our ultimate goal is to enable the robot to perform simple tasks without demonstrations, allowing its operators more time to focus on their businesses.