#imitation learning

NVIDIA has clearly stated its ambitions in the field of robotics, making imitation learning its spearhead for 2025. By capturing and transforming human gestures into actionable data in real time, the Californian firm is paving the way for a new generation of robots, more adaptable and versatile.

A turning point for robotics

This revolutionary approach offers several advantages:

- Democratization of robotics: By reducing the need for complex programming, learning by imitation makes robotics accessible to a greater number of companies.

- Faster innovation: Robots can now acquire new skills much more quickly, opening up infinite possibilities in terms of applications.

- Improved human-robot collaboration: Imitation learning facilitates the integration of robots into existing work environments, fostering more natural and effective collaboration.

Implications for our everyday lives

Progress in the field of robotics, thanks in particular to NVIDIA's initiatives, will profoundly change our daily lives:

- Automation of repetitive and hazardous tasks: Robots will be able to take on tedious or risky tasks, freeing up humans for higher value-added activities.

- Personalization of services: Robots will be able to adapt to individual needs, offering more personalized and efficient services.

- Transformation of industries: Robotics will profoundly transform many sectors, from industrial production to logistics and healthcare.

The future of robotics: a human-machine symbiosis

With the advent of learning by imitation, the future of robotics looks particularly promising. Robots will become increasingly autonomous and collaborative, capable of continually learning from their environment and from their interactions with humans.

The collaboration between NVIDIA and major robotics players, as demonstrated by the launch of GR00T and Blueprint, is accelerating this momentum. The integration of technologies such as Apple Vision Pro, which creates digital twins of human actions, also opens up exciting new prospects.

In conclusion

Imitation learning is a real revolution for robotics. By combining NVIDIA's data processing capabilities with advances in augmented reality, we are witnessing the emergence of a new generation of robots, more intelligent, more adaptable and closer to humans. The future is full of promise, and it's exciting to imagine all the possible applications for this technology.

Enabling humanoid robot movement with imitation learning and mimicking of animal behaviors

Rish Joshi Contributor Rish is an entrepreneur and investor. Previously, he was a VC at Gradient Ventures (Google’s AI fund), co-founded a fintech startup building an analytics platform for SEC filings and worked on deep-learning research as a graduate student in computer science at MIT. More posts by this contributor

Assessing the potential for a gig economy in…

朝、少しポスドク氏と研究テーマについて会話。

「まだ一年目だしコースワークで忙しいよねアメリカの博士課程はそういうものさ」という雰囲気をボスからもかけられていて、それはまぁ助かるっちゃ助かるのだけど、自分としては一年目でもコースワークの合間を縫ってガンガン研究を進めていきたいので、今期は午前中は全て研究関連の時間に充てるという自分ルールを課して進めている。そして、九月頃までに一本論文を書く、それが今年の大きな目標。そのために以下のタイムラインを考えている。

Winter：ひたすらサーベイと勉強、そしてちょくちょくポスドク氏やボスとディスカッション。

Spring：興味があり解きたいオープンクエスチョンを明確にし、さらにサーベイを進め、方法論を検討する。

Summer：授業が無くなるので、全ての時間を前期に定めた方向に注ぎ論文を書く。

One-shot や Imitation learning に興味があり、かつ一般的な機械学習の文脈ではなくて少しニューロサイエンスよりの方向からも問題に取り組んでみたい私の話を踏まえ、ポスドク氏からは Mirror Neuron と Mental simulation という二つの立ち位置を教えてもらった。

文献や読み物もいくつか。

The Myth of Mirror Neurons: The Real Neuroscience of Communication and Cognition：ミラーニューロンに関する盛り上がりに待ったを唱えている本。うちの学校の先生が書いているので、興味があれば直接話が聞ける（笑）

Mental Simulation of Action in the Service of Action Perception まずは一般的な機械学習の文脈で One-shot や Imitation learning のサーベイをする一方で、上記のようなニューロサイエンスにおける二つの立ち位置を学んで行くという感じで進めよう。

One way to teach robots is for them to imitate humans or other robots, and having them extrapolate from there. Berkeley Artificial Intelligence Research are working on a way for this to work better.

PAPER:

I recently came across this interesting paper by NVIDIA autonomous driving team:

End to End Learning for Self-Driving Cars (NVIDIA’s team)

SUMMARY

They take a supervised learning approach to learn a mapping from the image input to the steering command. It is essentially a modern (mid 2010s) version of ALVINN from late 1980s. The function approximator is a convolutional neural network (a normalization + 5 convolutional + 3 fully connected). They use a lot of collected data based on actual driver’s behaviour to train their network (about 70 hours, corresponding to about 2.5M data samples — not explicitly mentioned) and some data augmentation.

COMMENTS:

It is exciting to see an end-to-end neural network learned how to perform relatively well. But there are potential problems: One challenging problem with such a classical supervised learning approach is due to the distribution mismatch caused by the dynamical nature of the agent-environment interaction: When an agent makes a mistake at each time step, the distribution of the future states slightly changes compared to the distribution of the optimal agent (or the driver, for this case). This has a compounding effect and the difference in distributions can potentially grow as the agent makes more interactions with the environment. As a result, as time passes, the agent is more likely to be in regions of the state space from which it doesn’t have much training data. So the agent starts behaving in ways that are not predictable even though it might perform well on the training distribution (this is the distribution mismatch problem in machine learning/statistics).

A solution to this problem is to use DAGGER-like algorithms:

Stéphane Ross, Geoffrey Gordon, and J. Andrew Bagnell, “A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning,” AISTATS, 2011.

Aside the aforementioned work, which analyzes the phenomenon in the imitation learning context, the analysis of how the distribution of the agent’s changes, in the context of reinforcement learning, has been done by several researchers, including myself. I only refer to two papers. See their references for further information.

Remi Munos, Performance bounds in Lp norm for approximate value iteration, 2007.