A systems-oriented map of robotics: hardware, classical foundations, simulation, autonomy stacks, robot learning, embodied foundation models, datasets, and interview preparation.
Robotics is not one stack. A useful robot joins mechanics, actuation, sensing, estimation, planning, control, real-time software, data, learned policies, and safety into one closed loop. This repository is organized around those interfaces rather than around hype cycles.
Last reviewed: 2026-08-03 · Catalog version: 0.1.0
Contents
- Start here
- The robotics stack
- Hardware landscape
- Core software stack
- Robot learning and foundation models
- Essential papers
- Datasets and benchmarks
- Foundations and courses
- Interview preparation
- How to evaluate claims
- Repository map
- Contributing
Start here
| Goal | Recommended path |
|---|---|
| Build solid fundamentals | Math primer → Modern Robotics → Underactuated Robotics |
| Learn a practical robot stack | ROS 2 → tf2/URDF → ros2_control → MoveIt 2 or Nav2 → Gazebo |
| Learn simulation and control | MuJoCo or Drake first; Isaac Lab when GPU-parallel robot learning matters |
| Study robot learning | behavior cloning → ACT/Diffusion Policy → Octo/OpenVLA → openpi/SmolVLA |
| Understand humanoids | Hardware landscape → whole-body control → loco-manipulation → GR00T/Gemini Robotics/Helix |
| Prepare for interviews | 12-week roadmap and interview guide |
| Select data or a benchmark | Dataset and benchmark guide |
| Track the frontier | Foundation-model guide and field signals |
The robotics stack
flowchart TB
GOAL[Task / language goal] --> TASK[Task and behavior planning]
TASK --> MOTION[Motion planning / learned policy]
MOTION --> CONTROL[Whole-body and joint control]
CONTROL --> ACT[Actuators and mechanism]
ACT --> WORLD[Physical world and contacts]
WORLD --> SENSOR[Sensors]
SENSOR --> EST[Calibration, perception, state estimation]
EST --> TASK
EST --> MOTION
EST --> CONTROL
SAFETY[Independent safety system] -. constrains .-> TASK
SAFETY -. constrains .-> CONTROL
SIM[Simulation + digital twins] -. test/train .-> EST
SIM -. test/train .-> MOTION
DATA[Demonstrations + robot data] -. train .-> MOTION
The central engineering fact is the feedback loop. An agentic planner may choose what to do, but a controller must still produce stable commands at the robot's required rate, and an independent safety path must be able to stop it.
Hardware landscape
The table is deliberately compact. Availability means something different for a research kit, an enterprise deployment, and a prototype announcement; verify the linked primary source before making a purchasing or investment decision.
Humanoids and general-purpose mobile manipulators
| Company | Important platform | Focus | Public posture |
|---|---|---|---|
| Boston Dynamics | Atlas | Electric humanoid; industrial whole-body manipulation | Enterprise product/deployment path |
| Agility Robotics | Digit | Logistics and material movement | Commercial deployments |
| Apptronik | Apollo 2 | General-purpose industrial humanoid and data-collection platform | Partner/deployment program |
| Figure | Figure 03 | Humanoid hardware paired with Helix 02 | Company-operated deployments |
| Tesla | Optimus | General-purpose humanoid | Internal development/prototype |
| Unitree | G1 and H1 | Research-accessible humanoids and locomotion | Commercial hardware |
| Fourier Intelligence | GR-2 | Humanoid research and embodied AI | Commercial/research platform |
| 1X | NEO | Home-oriented humanoid | Early access/development |
| Sanctuary AI | Phoenix Generation 8 | Dexterous work and teleoperation/data flywheel | Partner deployments |
| PAL Robotics | TALOS, TIAGo | Torque-controlled humanoid and mobile-manipulation research | Commercial research platforms |
| UBTECH | Walker S2 | Industrial humanoid deployments | Commercial/partner program |
| AgiBot | G-series and AgiBot World | General robots, data, simulation, and models | Commercial + open research artifacts |
Manipulators, hands, mobile bases, and legged robots
| Category | Representative products | Why they matter |
|---|---|---|
| Collaborative arms | Universal Robots UR, ABB GoFa, FANUC CRX, KUKA iiwa | Industrial integration, safety-rated modes, repeatability, ecosystem |
| Research arms | Franka Research 3, Kinova Gen3, UFACTORY xArm | Torque/force control, accessible APIs, manipulation research |
| Low-cost learning | ALOHA, SO-ARM100/SO-101 | Teleoperation, imitation learning, bimanual data collection |
| Dexterous end effectors | Shadow Dexterous Hand, Robotiq grippers | Contact, tactile sensing, grasp planning, underactuation |
| Quadrupeds | Boston Dynamics Spot, ANYbotics ANYmal, Unitree Go2/B2 | Mobility, inspection, mapping, loco-manipulation |
| Mobile research bases | Clearpath Husky, TurtleBot 4, Hello Robot Stretch 4 | Navigation, SLAM, mobile manipulation, education |
| Robot compute | NVIDIA Jetson Thor/Orin | Edge inference, sensor pipelines, hardware acceleration |
| Perception | RealSense depth cameras, Ouster lidar | Calibration, synchronization, depth, point-cloud pipelines |
See the detailed hardware landscape, including selection criteria, actuators, sensors, compute, and what vendor demos do not establish.
Core software stack
| Layer | Important projects | Interview-level understanding |
|---|---|---|
| Middleware | ROS 2 | nodes, topics, services, actions, QoS, executors, DDS, lifecycle, time |
| Robot description | URDF/Xacro, SDFormat, MJCF, USD | links/joints, frames, inertias, collision vs visual geometry |
| Transforms | tf2, Sophus |
frame semantics, timestamped transforms, SO(3) and SE(3) |
| Hardware/control | ros2_control |
hardware interfaces, controllers, update rates, command/state separation |
| Manipulation | MoveIt 2 | IK, planning scene, collision checking, OMPL, time parameterization |
| Navigation | Nav2 | localization, costmaps, planners, controllers, behavior trees, recovery |
| Simulation | Gazebo, MuJoCo, Drake, Isaac Sim/Lab | fidelity, contacts, determinism, throughput, sensor rendering, sim-to-real |
| Estimation | GTSAM, Ceres, SLAM Toolbox, ORB-SLAM3 | factor graphs, nonlinear least squares, observability, loop closure |
| Planning/optimization | OMPL, CasADi, acados, Crocoddyl | sampling vs optimization, constraints, MPC, trajectory optimization |
| Perception | OpenCV, PCL, Open3D | calibration, geometry, point clouds, registration, detection/segmentation |
| Flight | PX4, ArduPilot | estimators, flight-control loops, mission planning, safety boundaries |
Use software-stack.md for simulator selection and layer-by-layer study. For a stable learning environment, prefer a supported ROS 2 LTS distribution over Rolling unless you specifically need unreleased features.
Robot learning and foundation models
A useful lineage
Behavior cloning / offline RL
├─ ACT and Diffusion Policy: action chunks and expressive continuous policies
├─ RT-1 / RT-2 / RT-X: transformer policies, web knowledge, cross-embodiment data
├─ Octo / OpenVLA / SmolVLA / openpi: reproducible open model stacks
└─ GR00T / Gemini Robotics / Helix / π-series: larger-scale generalist and whole-body systems
| Model/system | Organization | Key idea | Openness caveat |
|---|---|---|---|
| RT-1 | Google Robotics | Tokenized multi-task real-robot policy | Paper/artifacts; not a fully open training stack |
| RT-2 | Google DeepMind | Transfer vision-language knowledge into robot actions | Closed model |
| Open X-Embodiment / RT-X | 21-institution collaboration | Cross-embodiment dataset and policies | Dataset licenses vary |
| Diffusion Policy | Columbia / TRI / MIT | Receding-horizon diffusion over action sequences | Open code |
| ACT / Mobile ALOHA | Stanford | Action chunking and low-cost bimanual/mobile data | Open hardware/code/data |
| Octo | UC Berkeley and collaborators | Open generalist transformer policy | Open weights/code |
| OpenVLA | Stanford / Berkeley / TRI | 7B open VLA with efficient fine-tuning work | Open weights/code |
| openpi | Physical Intelligence | π-family code and selected checkpoints | Newest frontier model may not be open |
| SmolVLA | Hugging Face | Compact VLA and asynchronous inference | Open weights/code/data recipes |
| GR00T N1.6 | NVIDIA | Humanoid foundation model with explicit fast/slow structure | Check release-specific weights/license |
| Gemini Robotics | Google DeepMind | Embodied reasoning plus VLA control across embodiments | Early access/closed |
| Helix 02 | Figure | Hierarchical whole-body visuomotor control | Company report/demos |
| π0.7 | Physical Intelligence | Steerable generalist manipulation and compositional transfer | Company paper/demos |
Read foundation-models.md before comparing headline success rates. Embodiment, control frequency, action representation, camera placement, evaluation resets, and human intervention policy can dominate a model-name comparison.
Essential papers
The curated paper reading guide covers operational-space and impedance control, RRT/PRM, GraphSLAM/ORB-SLAM, deep RL and sim-to-real, modern manipulation policies, and the VLA lineage from RT-1 through open models and 2026 frontier systems. It also provides a paper-reading rubric and reproduction ladder.
Datasets and benchmarks
| Resource | Modality / setting | Best used for |
|---|---|---|
| Open X-Embodiment | Multi-institution, cross-robot trajectories | Generalist pretraining and embodiment transfer |
| DROID | In-the-wild Franka manipulation | Scene/task diversity and reproducible hardware |
| BridgeData V2 | Tabletop/mobile manipulation | Offline imitation and representation learning |
| AgiBot World | Large-scale humanoid/general-robot data | Long-horizon and broad-scene pretraining |
| CALVIN | Language-conditioned simulated manipulation | Long-horizon instruction chains |
| LIBERO | Lifelong robot learning suites | Transfer and catastrophic forgetting |
| BEHAVIOR-1K | Household mobile manipulation | Long-horizon, human-centered embodied tasks |
| RoboCasa | Kitchen manipulation simulation | Scalable tasks, demonstrations, scene diversity |
| ManiSkill | GPU-parallel manipulation | RL/IL algorithm development |
| SimplerEnv | VLA evaluation in simulation | Policy ranking and sim-real correlation studies |
| RoboTwin | Digital-twin bimanual manipulation | Data generation and generalization evaluation |
| Habitat | Embodied navigation and rearrangement | Navigation, interaction, and household agents |
See datasets-benchmarks.md for data leakage, reset policy, confidence intervals, intervention accounting, and cross-embodiment normalization.
Foundations and courses
Mathematics, kinematics, dynamics, and control
- Modern Robotics — screws,
SE(3), kinematics, Jacobians, dynamics, planning, and control with code. - Underactuated Robotics — nonlinear dynamics, optimization, planning, control, contact, and learning.
- MIT Robotic Manipulation — perception, planning, contact, grasping, and modern manipulation systems.
- Planning Algorithms — freely available reference for geometric, sampling-based, feedback, and decision-theoretic planning.
- ETH Robot Dynamics — rigid-body dynamics, legged systems, and estimation/control practice.
- Berkeley CS 285 — deep reinforcement learning.
Essential concepts
- coordinate frames, rotations, quaternions, Lie groups
SO(3)andSE(3); - forward/inverse kinematics, Jacobians, singularities, manipulability;
- rigid-body dynamics, contacts, friction, inverse dynamics;
- PID, impedance/admittance control, LQR, MPC, whole-body control;
- Bayes filters, EKF/UKF, factor graphs, SLAM, calibration and observability;
- graph search, sampling-based motion planning, trajectory optimization;
- behavior cloning, offline/online RL, imitation learning, diffusion/flow policies;
- real-time scheduling, concurrency, latency budgets, safety and verification.
The math primer collects the core equations and common mistakes.
Interview preparation
A robotics interview usually samples several layers rather than one:
- Math/algorithms: transforms, Jacobians, least squares, probability, graph search.
- Controls/estimation: stability, feedback, filters, calibration, sensor timing.
- Software: C++/Python, ROS 2, concurrency, memory, networking, debugging.
- Domain depth: manipulation, navigation, perception, locomotion, or learning.
- Systems design: rates, interfaces, failure handling, logging, deployment, safety.
- Experimentation: baselines, ablations, confidence intervals, sim-to-real validation.
The interview guide provides question prompts, expected reasoning, coding exercises, system-design cases, and portfolio projects. The learning roadmap turns the repository into a 12-week curriculum.
How to evaluate claims
Do not equate a polished video with general autonomy. Ask:
- Was the run autonomous, teleoperated, scripted, cherry-picked, or reset between clips?
- How many trials, environments, objects, operators, and hardware units were tested?
- Were failures, safety stops, interventions, and recovery time counted?
- Is success binary, partial-credit, time-bounded, or judged by the authors?
- What runs on-board, what runs in the cloud, and what is the end-to-end latency distribution?
- Does the policy survive camera shifts, payload changes, worn grippers, latency, and disturbances?
- Are train/test tasks, objects, scenes, and embodiments genuinely disjoint?
- Can results be reproduced from released code, weights, data, robot description, and calibration?
The repository records access and maturity separately: open code is not proof of capability, and a commercial deployment is not proof of generality.
Repository map
| Path | Purpose |
|---|---|
catalog/resources.yaml |
Canonical machine-readable resource index |
docs/hardware-landscape.md |
Companies, products, mechanisms, sensing, compute |
docs/software-stack.md |
Middleware, planning, control, simulation, perception |
docs/math-primer.md |
Equations and classical robotics background |
docs/foundation-models.md |
VLA/robot-foundation-model architecture and comparison |
docs/papers.md |
Essential classical, learning, and embodied-model papers |
docs/datasets-benchmarks.md |
Dataset and evaluation methodology |
docs/learning-roadmap.md |
Twelve-week systematic curriculum |
docs/interview-guide.md |
Interview questions, exercises, and project ideas |
docs/safety.md |
Physical safety and agentic autonomy boundaries |
docs/field-signals.md |
Time-sensitive trends and interpretation |
docs/catalog-report.md |
Generated coverage report; do not edit manually |
Contributing
Contributions are welcome. Read CONTRIBUTING.md and update the canonical YAML first. Every entry needs a primary source, company/steward, access level, maturity, review date, and a concrete explanation of why it belongs.
make setup
make report
make validateInclusion is not endorsement. Specifications, availability, licenses, and model access change; verify official sources before purchasing hardware, deploying software, or operating a robot.