AI coding agents are old news. Now the same agents are training factory robots, generating autonomous vehicle scenarios, and catching defects on semiconductor lines.
NVIDIA open-sourced its Physical AI agent skill collection, and Foxconn, Pegatron, and Delta Electronics have already deployed them with real results.
What is Physical AI?
Until now, AI agents mostly lived in text, code, and images — ChatGPT writing emails, Claude Code writing software. Physical AI is the next frontier: AI that moves beyond the digital screen to understand and control the physical world — robots, factories, autonomous vehicles.
The challenge is that training physical AI is brutally hard. Teaching a robot a specific motion requires hundreds of thousands of simulation examples. But collecting that data from real environments is slow, expensive, and sometimes impossible. Getting thousands of images of a rare semiconductor defect from a real production line? Practically unfeasible.
NVIDIA's approach is to solve this bottleneck with AI itself: when real data is scarce, AI generates synthetic versions. The model trains in simulation, then deploys to the real world. And the entire pipeline — data generation, simulation, training, deployment — is now automated by agents.
Factories Were Already Using It
What makes this announcement stand out is that it comes with real numbers, not just a roadmap. Taiwanese electronics manufacturers — including Foxconn and Pegatron — have already deployed these tools and produced results.
The key skill behind these results is Defect Image Generation. When real defect images are scarce, AI generates synthetic ones to supplement training data. In Pegatron's case, this single skill cut AI model training and deployment time by 67%.
In autonomous vehicles, the scale is even larger. Li Auto, Afari, and DeepRoute.ai are running 1,000+ scene reconstructions and 300,000+ renderings and simulations per day. You can't film 300,000 driving scenarios on real roads in a day — but with agent skills, you can generate them.
What Skills Have Been Released?
The skills released through GitHub and skills.sh cover four domains. They work with any coding agent.
| Domain | Key Skills & Tools | Key Use Cases |
|---|---|---|
| Manufacturing & Vision AI | Defect Image Generation, Video Augmentation, Metropolis VSS Blueprint | Pegatron 67% time cut, Delta 17% detection improvement |
| Robotics | Isaac Sim 6.0, Isaac Lab, COMPASS Navigation, Cosmos-H-Surgical-Simulator | RL automation, surgical robot training data generation |
| Autonomous Vehicles | Neural Reconstruction, OmniDreams, AlpaGym, Alpamayo 2 Super (32B VLA) | Li Auto 300K daily simulations |
| Industrial & Healthcare | Omniverse Digital Twin, Healthcare Simulator, MEGA Blueprint | Siemens·Cadence digital twins, hospital environment simulation |
The underlying model is Cosmos 3 — a full omnimodel that handles visual reasoning, world prediction, and action generation in one architecture. It has been downloaded over 2 million times on Hugging Face, and the Physical AI Dataset has hit 15 million downloads.
What about security?
The NemoClaw blueprint and OpenShell runtime provide policy-based governance — you can control exactly what tools an agent is allowed to run, whether in local or cloud environments.
How to Get Started
NVIDIA Brev offers free H100 GPU credits for Physical AI Launchables, so you can start without any infrastructure. Here's a 5-step path to running your first Physical AI agent skill.
- Browse the skills on GitHub
Visit github.com/NVIDIA/skills and pick the skill for your domain. Manufacturing and vision AI → vision-ai folder; robotics → robotics folder. - Launch a free environment on NVIDIA Brev
Select a Physical AI Launchable for your domain. A Jupyter environment with H100 GPU access will be ready in minutes — no setup required. - Connect to your coding agent
Skills packaged in skills.sh format plug directly into Claude Code, Cursor, or any other coding agent via MCP. - Generate synthetic data → Train → Validate
Use the Defect Image Generation skill to create training data, Isaac Sim 6.0 for simulation, and Isaac Lab to automate training — all orchestrated by your agent. - Deploy to Jetson edge devices
Finish with Jetson-optimized deployment skills that push the trained model to edge devices. Cloud training → on-site inference: the full cycle in one agent pipeline.
