Author: admin

Readily-available physical AI, with robotics assisting users in manufacturing, home, and warehouses scenarios, holds immense potential to improve safety, productivity, and assistance across a wide range of tasks. In many ways, AI for the physical world represents a major frontier for AI . Physical AI must operate in open, unpredictable environments, interact with both other robots and people, and work with a diversity of embodiments. Realizing this vision requires advances along three dimensions: robot hardware, embodied AI models, and systems infrastructure for training and inference. While robot hardware and the AI models have advanced rapidly in recent years, we turn our focus on a relatively under-addressed aspect: inference infrastructure of physical AI. Enabling robots to effectively and safely operate in the physical world will require sophisticated systems to handle large volumes of distributed inference compute.

Today, the prevailing approach to physical AI is to provision a GPU onboard the robot, e.g., by wiring a GPU to the robot. In this model, the robot’s inference will be confined to the onboard GPU, and provide the robot with the necessary chunks and sequence of actions for the execution of its tasks. While higher-level planning may be performed in the cloud, task execution typically remains tied to the robot itself. We challenge this assumption.  As physical AI models grow in size and sophistication, the constraints of onboard compute become increasingly apparent. GPUs consume significant power, reduce battery life, add cost and weight, and can limit the ability to run the latest generation of AI models. 

To better understand the systems implications of physical AI, we conducted the first systematic study of robotics workloads. We focused on mobile robotic manipulation, with the canonical task such as “check for rubbish in the kitchen and put it in the trash.” Such a task involves planning the path to the kitchen, perceiving the environment to find rubbish, navigating to the rubbish, picking up the rubbish, and navigating back to the trash can for disposal. We evaluated representative models across three core capabilities: semantic mapping and planning, navigation, and manipulation, as summarized in Figure 2. 

Offloading physical AI inference out of the robot improved its response time and accuracy, along with battery lifetime and cost. We evaluated the inference models across a range of onboard, edge, and cloud compute configurations. Details of the specific test hardware are available in our technical report. 

Benefits in task performance: Our evaluation shows offloading inference can significantly improve robot performance across mapping, planning, navigation, and manipulation workloads. Some smaller GPUs could not accommodate the mobile manipulation stack. On GPUs with sufficient memory, mapping and planning slowed by up to 383% compared to an A100, thus limiting the robot’s abilities in dynamic spaces. Navigation showed a 30% drop in its timely detection of obstacles with lighter GPUs. While the VLA models did not dramatically slow down with smaller GPUs, the slowdown was still sufficient to drop their accuracies by 50%. In other words, onboard GPUs limited the performance of the robots while offloading their inference to an on-premise or cloud GPU boosts their operations, as shown in the videos below and quantified in the graphs. As physical AI models continue to grow in size and complexity, the benefits of offloading are likely to become even more pronounced.

Benefits in battery lifetime: Beyond performance, onboard GPUs also significantly drained the battery life of the robot. We compared the increase in battery lifetime by replacing an onboard GPU with a Raspberry Pi-5 board and shipping all the data to the offloaded GPU. The larger onboard GPUs, such as Jetson Thor, drained robot batteries by up to 160% (or a few hours) for even the larger robots.

The above results show that offloading GPU inference out of the robot is critical for functioning in the open world with large models and long battery lifetimes. Nonetheless, offloading inference out of the robot involves a complex tradeoff involving performance, network latency and bandwidth, and available GPU resources. We believe that our measurement study will inform the design of physical AI inference systems.

We have built a toolset for easy inference offloading out of the robot and distributing inference between the edge GPU and cloud. Kubernetes is a natural platform to provide a uniform abstraction to distribute robotic AI between the robot’s compute, edge GPU, and overflowing to the cloud. The toolset allows automatic containerization and offloading of robotics workloads using declarative specifications, distributes physical AI containers with smart policies using Kubernetes, and integrates with robotic simulators, LeRobot, and ROS2 for easy development. The sequence of steps below shows how the toolset can be prompted with what to offload, and how it creates a separate container for GPU inference and offloads the same.

Microsoft has recently released the Physical AI Toolchain (opens in new tab) for operationalizing physical intelligence at scale. Physical AI Toolchain is an open-source, production-ready framework that integrates Microsoft Azure (opens in new tab) cloud services with NVIDIA’s (opens in new tab) physical AI stack, accelerating robotics and physical AI developers to automate and scale data curation, augmentation, and evaluation across perception, mobility, imitation learning, and reinforcement learning pipelines. We are announcing the addition of an industry-first capability for offloaded physical AI inference for robots as part of the Physical AI Toolchain. This release includes example projects for offloading inference of a SO-101 and a UR10e. The videos below show the offloading of the inference of Microsoft’s Rho model, targeted at dual-arm robots, to a Jetson Thor GPU, which controls the actions of the Mobile Aloha robot (opens in new tab).  

Check out the inference offload feature, look into the source code, and let us know your feedback. We have already tested it with many real-world use cases, and look forward to hearing about your deployment experiences.

Developing new medicines and materials requires making new molecules but planning how to make them is still largely manual, time-consuming, and costly. RetroChimera automatically proposes high-quality synthesis routes. The model combines two strong models with complementary strengths, learning how to rank their proposals to produce better predictions than either alone. In blind tests, PhD-level chemists prefer RetroChimera’s individual reaction predictions over preceding models and recorded literature reactions. 

Custom-made molecules are unlocking advances in modern medicine, smart materials, and sustainable agriculture. Yet, progress is slowed by chemical synthesis—the time-consuming process of making new molecules from simpler building blocks in the lab. In addition, synthesis is a significant driver of drug development costs. So even as computational methods make it possible to explore large numbers of novel molecules, finding practical ways to synthesize them remains a critical challenge. 

Retrosynthesis approaches this problem by working backwards from a target molecule, breaking it down step by step into simpler precursors (Figure 1). This process is comparable to playing strategic board games like chess and Go. It involves contemplating a wide range of possible immediate moves, or individual disconnections, while also requiring high-level strategic thinking to reach the end-to-end synthesis plan. However, the number of possible moves in retrosynthesis is much larger than in board games, and it is not obvious which moves would be available for a given molecule. Existing systems face major challenges, including recalling rare but strategically important reactions, robustness beyond the training distribution, and aligning with chemists’ expectations. As a result, retrosynthesis often requires highly specialized expertise, which hinders scaling and automation of scientific discovery.

In a paper recently published in the journal Nature (opens in new tab), we present RetroChimera (opens in new tab), a new framework for retrosynthesis prediction. It is built around two models (Figure 2). R-SMILES 2, a Transformer-based de-novo model, predicts precursor molecules directly from the input molecule. This gives it the flexibility to learn reaction patterns directly from data. However, its unconstrained generation can also make it prone to hallucination.

NeuralLoc, in contrast, is a graph neural network- (GNN) based model that encodes both the target molecule and reaction templates as graphs. It selects reaction templates and predicts where they should be applied to the target molecule. Its predictions are grounded in reaction patterns extracted from the training data, so it tends to produce more accurate and reliable outputs. But it’s more constrained when encountering reactions not covered by the template library. 

These differences actually turn out to be a strength. Rather than making the same kinds of predictions, the two models capture complementary patterns in chemistry and specialize in different reaction types. R-SMILES 2 performs particularly well on reactions that involve large changes over the course of the reaction, while NeuralLoc excels in reactions of low precedence and those involving more localized changes.

RetroChimera combines the ranked predictions of both sub-models using a learned ensembling strategy. Each model assigns a learned, rank-dependent vote to each predicted reactant set, and votes are added when both models propose the same reaction. By learning how much to trust each model at different ranks, RetroChimera can leverage their complementary strengths, approximately matching the better-performing sub-model across reaction classes.

As a result, RetroChimera performs strongly across both common and rare reaction classes and produces retrosynthesis predictions that better align with chemists’ judgment (Figure 3). In blind tests, expert chemists preferred disconnections of complex molecules suggested by RetroChimera over those obtained from its constituent sub-models, as well as those from more established approaches, and even from the test set itself.

We believe RetroChimera could help researchers identify promising synthesis route more efficiently, supporting faster design-make-test cycle across molecular science applications, including drug discovery and design of smart materials. RetroChimera could enable chemists to assess more—and more complex—candidate molecules at large scale. Paired with increasing levels of laboratory automation, we expect further acceleration toward closed-loop, self-improving systems for synthesis planning and execution.  

RetroChimera is available on GitHub (opens in new tab) (MIT license) and accessible via Microsoft Foundry (opens in new tab). For instructions on how to access the checkpoint, we refer to the GitHub repository.

We invite the broader chemistry community to experiment with RetroChimera, helping us identify its strengths and shortcomings so we can enhance it in the future. We are looking forward to hearing how it performs on various targets you care about!

For a deeper look at the findings, including evaluation experiments with our external research collaborators, see the full Nature publication (opens in new tab) and the accompanying Microsoft Source article (opens in new tab).