#sim2real

Abstract—Recent advancements in Artificial Intelligence (AI) have largely been propelled by scaling. In Robotics, scaling is hindered by the lack of access to massive robot datasets. We advocate using realistic physical simulation as a means to scale environments, tasks, and datasets for robot learning methods.

We present RoboCasa, a large-scale simulation framework for training generalist robots in everyday environments.

RoboCasa features realistic and diverse scenes focusing on kitchen environments. We provide thousands of 3D assets across over 150 object categories and dozens of interactable furniture and appliances. We enrich the realism and diversity of our simulation with generative AI tools, such as object assets from text-to-3D models and environment textures from text-to-image models.

We design a set of 100 tasks for systematic evaluation, including composite tasks generated by the guidance of large language models. To facilitate learning, we provide high-quality human demonstrations and integrate automated trajectory generation methods to substantially enlarge our datasets with minimal human burden.

China’s humanoid robot turns into Kung-fu master after dazzling dance debut

Robots train in a virtual setting, learn human actions, then transfer skills to real robots using Sim2Real technology.

Unitree’s latest video shows its compact humanoid robot mastering kung-fu moves, including punches and roundhouse kicks, while maintaining balance. The Chinese company is in the process of constantly upgrading G1’s algorithm, enabling it to learn and perform virtually any movement. The most recent update seems to have expanded the humanoid’s balance capabilities and repertoire of movements.

In a video released last week, G1 showcased agility and balance, performing smooth dances, precise footwork, while being interrupted.

In January, Unitree showcased its G1 robot’s smooth walking and running system, demonstrating agility, stability, and precise control on inclines and uneven terrain.

Humanoid robot’s agile moves

In its most recent video, Unitree’s G1 humanoid performs amazing kung-fu movements with amazing balance, albeit it almost trips once. The new algorithm upgrade that powers these nimble movements improves the robot’s capacity to learn and carry out intricate tasks. The G1 exhibits remarkable coordination and flexibility with 23 degrees of freedom (DoF) and enhanced stability. Notwithstanding this demonstration of martial arts expertise, the film concludes with a warning that users should not alter the robot in any way that could endanger others or teach it actual combat techniques. Unitree claims that the G1 is made to handle difficult, filthy, and repetitive jobs in a variety of settings, including at homes, factories, and hospitals. It embodies Unitree’s concept of humanoid robots serving as useful work and life companions.

Training begins in a virtual setting using Nvidia’s Isaac Simulator, allowing the robots to learn complex behaviors before they physically exist. This method entails building a digital twin of the humanoid robot that uses motion capture and video data to observe human actions. These behaviors are exercised in the virtual environment through reinforcement learning. Through a procedure known as Sim2Real, which smoothly converts simulated actions into real-world applications, the acquired abilities are subsequently transmitted to the physical robot.

Open-source motion

To improve the natural movement of its humanoid robots, such as the H1, H1-2, and G1 models, Unitree Robotics has released an open-source full-body dataset. This information enhances the robots’ flexibility and coordination, allowing them to mimic human-like movements – like dancing. Captured using LAFAN1 motion capture technology, the dataset is fully compatible with Unitree’s main robot models and showcases lifelike movements in a newly released demonstration video. The dataset features a redirection algorithm that combines interactive mesh and inverse kinematics technology. By taking into account end posture restrictions, joint positions, and velocity limitations, this novel method maximizes the robots’ movements.

RECOMMENDED ARTICLES

Unitree hopes to enable researchers and enthusiasts to investigate novel uses for humanoid robots in diverse real-world contexts by making this dataset publicly accessible.

High-performance H1 and H1-2 models from Unitree are built for advanced capabilities; as of March 2024, the H1 held the Guinness World Record for sprinting at 7.38 mph (3.3 m/s). The G1 model, on the other hand, provides a more portable and reasonably priced choice for research and development.

According to the firm, it aims to promote exploration and creativity through this open-source project, expanding the possibilities for more dynamic, human-like interactions and pushing the limits of humanoid robotics.