Actuate 26: Foxglove Launches Agentic Data Platform for Physical AI, Collaborates with NVIDIA on Semantic Search

Foxglove launches Agent, Semantic Search powered by NVIDIA Cosmos, Comparison Mode, and Remote Access, enabling robotics teams to accelerate the path from data collection to improved models and more reliable fleets.

SAN FRANCISCO--(BUSINESS WIRE)--Foxglove, the agentic data platform for Physical AI, today announced a new intelligent AI layer that enables robotics developers to search, visualize, compare, debug, and curate multimodal datasets by simply describing what they need in natural language. Foxglove’s new agentic capabilities are available through the Agent Sidebar, a built-in agent inside the Foxglove app, and the Foxglove MCP server, which lets external AI agents interact with Foxglove. These new AI-powered capabilities automate the work required to turn raw multimodal data into high-value datasets, evaluations, and insights that improve models and performance.

“Robots are increasingly powered by AI, but the workflows used to develop and improve them remain largely manual,” said Adrian Macneil, co-founder and CEO of Foxglove. “Foxglove brings intelligence into the development loop, allowing engineers to describe the behavior they need to investigate or the dataset they want to create, and let agents do the work.”

Foxglove also introduced Semantic Search powered by the open NVIDIA Cosmos world models, Comparison Mode, and Remote Access, new capabilities that make it easier for developers to find critical events across large robotics datasets, compare system behavior across runs, and debug robots in the field without needing to send engineers onsite.

Robotics programs advance at the rate teams can close the loop between a failure in the field and a validated fix. Today that loop consumes much of an autonomy team’s engineering capacity, while most of the data a fleet collects is never reviewed. Foxglove raises the yield on data teams have already paid to collect and store, converting it into more model improvements per quarter. It frees engineers for model development and lets teams diagnose field incidents the same day, without scheduling an onsite visit.

Foxglove and NVIDIA collaborate to bring semantic search to Physical AI

Foxglove Semantic Search enables users to describe a behavior or scenario in natural language, and Foxglove retrieves matching multimodal segments from across large volumes of unlabeled data. Each result stays connected to the synchronized sensor data, telemetry, logs, and system signals recorded around that moment, so developers move directly from finding an event on camera to understanding what the rest of the robot was doing when it happened.

Foxglove Semantic Search is powered by NVIDIA Cosmos, a video-text embedding model purpose-built for Physical AI. General-purpose image-text models are trained on internet data and lose accuracy on complex manipulation tasks, driving scenes, and temporal action data. NVIDIA Cosmos is a frontier foundation model trained on robotics, driving, and ego-centric human action data, which delivers state-of-the-art retrieval accuracy on physical AI benchmarks. Foxglove and NVIDIA collaborated to bring Semantic Search powered by Cosmos to physical AI teams, with fully managed indexing and inference infrastructure requiring zero operational overhead and maintenance.

When combined with Foxglove's Bring-Your-Own-Storage (BYOS) architecture, Semantic Search is able to index and search over raw logs stored directly in customer cloud storage, without any unnecessary duplication of data.

“Physical AI is moving beyond recognizing objects in a frame to understanding actions as they unfold in real time,” said Amit Goel, head of robotics partner ecosystem at NVIDIA. “With Semantic Search powered by NVIDIA Cosmos world models, Foxglove is helping robotics developers search video and multimodal robot data in natural language and turn real-world context into faster debugging, evaluation and model improvement.”

Turning production data into a learning loop

As robotics fleets scale, teams generate more data across more robots, environments, and model versions. The events that drive improvement are rare, and their value depends on how quickly developers can find them, understand what happened, and turn them into fixes, evaluations, or training data.

Foxglove’s new capabilities connect and accelerate each step:

• Agent Sidebar coordinates work across the platform, allowing developers to find data, create visualizations, investigate failures, compare runs, and curate results into datasets.
• Semantic Search powered by NVIDIA Cosmos finds relevant behaviors and events across large volumes of unlabeled robot data.
• Comparison Mode synchronizes two or more runs so teams can understand where behavior diverged across models, software versions, robots, or scenarios. Comparisons are a critical tool for understanding drift between model versions and validating each release.
• Remote Access streams every topic from a remote robot directly to the browser, letting engineers observe and debug a deployed system in real time from anywhere. Camera, lidar, telemetry, and logs arrive at low latency in the same application teams use to visualize recorded data, so field issues can be diagnosed without sending anyone onsite.

Every capability announced today rests on the same foundation: time-synchronized data from cameras, lidar, radar, transforms, telemetry, and logs, connected as a single record of what a robot actually did. That grounding is what makes agentic workflows dependable in Physical AI, where every answer has to hold up against the physical world. As companies scale from a handful of robots to fleets in production, Foxglove is where the data they collect becomes the engine behind every improvement they ship.

For more details on everything announced at Actuate 26, read the announcement blog.

About Foxglove

Foxglove is the agentic data platform for Physical AI. Built for robotics teams developing real-world systems, Foxglove provides a purpose-built, modular platform to collect, organize, and learn from vast quantities of multimodal data, creating the data flywheel to rapidly scale from development to distributed fleets. Founded in 2021, Foxglove supports hundreds of customers across automotive, aerospace, defense, logistics, agriculture, construction, and consumer robotics to deploy the next generation of intelligent machines. Learn more at foxglove.dev.