u/Ok-Video-2620

From Fusion 360 to IsaacLab: training a custom robot with reinforcement learning
▲ 9 r/IsaacSim+1 crossposts

From Fusion 360 to IsaacLab: training a custom robot with reinforcement learning

Hi everyone,

I recently worked on a small project where I designed a custom robot in Fusion 360 and trained it in IsaacLab using reinforcement learning.

USDZ File

CAD File

The robot is a wheeled biped-style platform. After creating the CAD model, I converted it into a simulation-ready asset, set up the joints, and used it for stabilization and jump-recovery tasks in IsaacLab.

What I found most interesting was how much the physical design affects the learning process. Things like joint placement, link length, wheel contact, collision shapes, inertia, and actuator settings all had a noticeable impact on whether the robot could learn stable behavior.

The first task was basic stabilization, where the robot learns to maintain its posture. I also tested a jump-and-stabilize task, where the robot needs to recover after a more dynamic motion.

This made me realize that building a robot for RL is not just about making a nice-looking CAD model. The morphology, physics properties, and simulation setup are all part of the learning problem.

The workflow was roughly:

Fusion 360 → asset preparation → joint setup → IsaacLab training → policy evaluation

I’m planning to extend this robot to more tasks, including wheeled balance control, push recovery, locomotion, turning, navigation, and object interaction.

I wrote a longer post with more details about the design process and what I learned from training it in IsaacLab.

Stabilize Task

Jump & Stabilize Task

reddit.com
u/Ok-Video-2620 — 7 days ago
▲ 8 r/IsaacSim+1 crossposts

[Project Demo 2] Training a Humanoid agent to shoot a ball in IsaacLab! ⚽🤖

Hey everyone! I wanted to share the second demo from my IsaacLab project series.

For Project Demo 2, I trained a humanoid agent to approach a ball and shoot it toward a goal. Compared to my previous Ackermann-steering soccer agent, this task focuses more on full-body coordination, balance, and contact-rich control.

The main challenge is that the agent has to approach the ball, maintain balance, time the kick properly, and generate contact force in the goal direction without falling too early.

### What’s next?

I’m planning to extend this task to more complex soccer-style behaviors, such as:

  1. dribbling before shooting,
  2. shooting from different angles,
  3. balance recovery after kicking,
  4. staged skills like approach → stabilize → kick.

I’d love to hear any feedback or suggestions, especially on reward shaping for humanoid sports tasks.

Thanks for reading!

https://reddit.com/link/1tc4t6f/video/cadj16njlx0h1/player

reddit.com
u/Ok-Video-2620 — 7 days ago

[Project Demo] Training an Ackermann-steering robot to play soccer in IsaacLab! ⚽🏎️

Hey everyone! Just wanted to share a demo of my latest agent. I've been training an Ackermann robot in IsaacLab to track a ball and kick it into a goal.
Ackermann kinematics make this much trickier than using a simple differential drive, as the turning radius limits how the robot can approach the ball for a shot.
The Setup:
I’m using a 17-dimensional observation space to give the agent a clear "spatial awareness." Here is the breakdown of what the robot sees:

  • Positional Data: rel_robot_ball and rel_ball_goal (all in robot body frame).
  • Heading: I used sin/cos of the heading errors to the ball and the goal to ensure the gradients stay smooth.
  • Velocities: Robot linear/angular velocities + the ball's velocity in the body frame.
  • History: Including the _prev_action to encourage smoother control and prevent jitter.

What’s next for Real-world Deployment?
The goal is to get this running on a 1/10th scale RC car. To get there, I’m working on:

  1. System ID: Better matching the steering lag of real servos.
  2. Noise Injection: Adding sensor noise to the observations during training.
  3. Visual RL: Moving away from ground truth data to using a camera-based input (likely via a Kalman Filter or a small CNN).

Would love to hear your thoughts on Reward Shaping for sports-based tasks or any tips for Sim-to-Real with Ackermann steering!

https://reddit.com/link/1t99lf0/video/sf3bww4ftb0h1/player

reddit.com
u/Ok-Video-2620 — 10 days ago