PHYSICAL AI · HUMAN-ROBOT INTERACTION
Public-private collaboration · NASERTIC · Humanoid robotics
Equip a Unitree G1 humanoid robot to guide visitors at Navarra’s Polo IRIS: moving through the space and holding natural conversations about the centre and its activities.
In a public-private collaboration with NASERTIC, we develop TXISTU’s intelligence layer: autonomous navigation, fluent conversation and environmental perception. We integrate these capabilities on Unitree’s robotic platform to connect information about Polo IRIS with the visitor experience.
Python · ROS 2 / Nav2 · Unitree SDK2 / DDS · Livox MID-360 LiDAR · RGB-D camera · Computer vision · STT / TTS · LLMs · RAG · SQLite FTS5 · MuJoCo / Isaac Sim
TXISTU combines autonomous navigation and fluent conversation in its role as a guide at Navarra’s Polo IRIS, in collaboration with NASERTIC.
We separate conversation, perception and motion into modules connected through common interfaces. This lets us test each capability, replace its models and connect the same software to the simulator or the G1.
Speech passes through echo cancellation, noise suppression, voice activity detection and transcription. The session retains recent context and lets visitors interrupt a response to ask a new question. Spanish and Basque profiles use interchangeable speech recognition and synthesis components.
Before answering, we retrieve information from a Polo IRIS corpus using RAG with SQLite FTS5. Answers include sources and retrieval dates. A router combines cloud models with local alternatives and an extractive fallback when a provider is unavailable.
We integrate the LiDAR point cloud and camera imagery to obtain spatial information. Vision tests use a COCO-trained SSDlite detector to detect people. Depth data estimates their distance and provides structured observations to the rest of the system.
ROS 2 and Nav2 connect localisation, maps, route planning and obstacles. Before sending velocity commands to the robot, a supervision layer checks data freshness, localisation, distances and motion limits. Invalid conditions trigger a stop.
LiDAR, camera and environmental observations.
Speech, context and knowledge of Polo IRIS.
Routes, obstacles and motion supervision.
Conversation and movement as a guide.
We use Isaac Sim and a MuJoCo G1 environment to develop and test integrations. In GISAKI-IRIS, the simulator reproduces the D435 camera, MID-360 LiDAR, odometry and motion services through DDS and Unitree SDK2. This allows data exchange to be tested before checks on the physical robot.
Actual captures from project testing: object detection and an environment representation in the perception interface.
TXISTU is developed in collaboration with NASERTIC on a Unitree G1 humanoid. Unitree manufactures the robotic platform. The work described here concerns the integration of intelligence, perception and interaction capabilities.