OpenAI’s ChatGPT has already proved its mettle by generating essays that look like they were written by humans, and basic programming. But Microsoft has figured out that the AI chatbot can also play a role in human-to-robot interactions. Most of the code written to control robots today is hand-written and Microsoft wants to change that. The company is experimenting with using ChatGPT as an intermediary between humans and assistive robots. In a blog post, Microsoft notes the difficulties associated with programming robots the traditional way. Current robotics pipelines begin with an engineer translating a task’s requirements into code for the system. The engineer writes code and then re-writes it to correct a bot’s behaviour until it’s perfected. The process is tedious, expensive, and inefficient. ChatGPT can be used to simplify this process significantly, allowing even non-technical users to program robots. High-level feedback (commands in plain English) can be provided to ChatGPT, which will then generate code, mostly in Python, for controlling the robot accordingly. “Without any fine-tuning we leverage the LLM’s knowledge to control different robots form factors for a variety of tasks,” adds the blog post. As a demo for this technique, Microsoft has shared a video where ChatGPT wrote code to control a real drone from simple prompts inputted by the user. While the objects that the robot interacted with were pre-defined to ChatGPT along with their positional data, the results were remarkable. For example, the prompt “I am still thirsty. Can you find something with sugar and a red logo?” got ChatGPT to understand that the user is referring to the can of coke in the room. The AI chatbot then programmed the bot to navigate the room toward the can. The researchers also noted in their blog post that "ChatGPT asked clarification questions when the user's instructions were ambiguous, and wrote complex code structures for the drone such as a zig-zag pattern to visually inspect shelves." ChatGPT can write code because it was trained on vast amounts of data, which also included code. The chatbot has already proven its ability to solve coding problems while also giving out detailed explanations for them - and also seeking clarifications wherever necessary. Aside from the drone experiment, Microsoft researchers also tested ChatGPT to use a robotic arm to move around blocks to create the Microsoft logo. They also tested if ChatGPT is able to decide where the robot should go based on sensor feedback in real-time.
