A robotic guide dog system developed by researchers at Binghamton University can hold two-way conversations with users, offering real-time navigation assistance to visually impaired people through large language models.
The system uses GPT-4 with voice commands to guide users to their destinations. It describes surroundings, identifies obstacles, and outlines route options before a journey begins. This builds on earlier research in which robotic guide dogs responded only to physical cues such as leash tugs.
From Physical Cues to Conversational Guidance
The latest version adds two core features: plan verbalization and scene verbalization. Before a trip, the robot presents possible routes and estimated travel times. During navigation, it describes the environment continuously, giving users a clearer picture of their surroundings.
“This is very important for visually impaired or blind people, because situational and scene awareness is relatively limited without vision,” said Shiqi Zhang, the lead researcher on the project.
How the Robotic Guide Dog Was Tested
Researchers recruited seven legally blind participants to navigate a large indoor office environment. The robot first asked each user where they wanted to go, then presented route options. Once a route was selected, it guided users step by step, providing spoken feedback including corridor descriptions and obstacle alerts.
After completing the navigation tasks, participants rated the system on usability and effectiveness. Results showed a clear preference for the combined approach, which included both pre-journey explanations and real-time narration during movement.
Language Capabilities Beyond Biological Dogs
Zhang said the conversational ability of the robotic system surpasses what a trained animal can offer. “Real dogs can understand around 20 commands at best. But for robotic guide dogs, you can just put GPT-4 with voice commands. Then it has very strong language capabilities,” he said.
A simulated study supported the user trial findings, suggesting that conversational guidance improves navigation confidence and gives users a greater sense of control. The ability to communicate with the robot also appeared to build trust over time.
Future Plans and Outdoor Expansion
The research team plans to increase the system’s autonomy and extend its use beyond indoor spaces. Future testing will target more complex environments and longer travel distances, with the goal of making the robotic guide dog a practical daily assistive tool.
Zhang noted that blind participants responded positively to the technology during testing. “They were super excited about the technology, about the robots. They asked many questions. They really see the potential for the technology and hope to see this working,” he said.
The study was presented at the AAAI Conference on Artificial Intelligence, where it drew attention for combining mobility support with intelligent voice communication in a single assistive device.
