Jiayi Zhu

Jiayi Zhu

EECS at UC Berkeley

Curious about robotics, perception,
and how things work.

Humanoid table tennis

Teaching a humanoid to return a table-tennis ball takes more than a fast swing. HOPE connects motion-capture perception, physics-based racket planning, and learned whole-body control so the Agibot A3 can coordinate a strike, maintain balance, and recover for the next ball.

ROS 2Isaac Lab / PPOMuJoCoA closer lookShow less

A complete perception-to-action loop

  1. 01ObserveTrack the ball
  2. 02PlanChoose the strike
  3. 03ActMove the whole body
  4. 04RecoverPrepare for the next ball

The planner decides where and when the racket should meet the ball. A learned controller turns that compact goal into coordinated joint motion. This division connects explicit ball physics with a policy that handles the robot’s body.

The challenge: timing, contact, and balance

A useful return needs the right contact position, racket velocity, and strike time. The robot must also stay upright and finish in a state from which it can strike again. HOPE brings these pieces together in an open-source stack for the Agibot A3, with a unified forehand/backhand policy and a shared path from training to deployment.

Tracking the ball and planning a return

OptiTrack/NatNet or Chingmu/VRPN supplies motion-capture measurements through ROS 2. A C++ planner estimates the incoming flight and predicts gravity, aerodynamic drag, and table bounces. The no-spin physics model uses parameters fitted from real venue measurements.

At a virtual hitting plane, the planner chooses forehand or backhand, solves for a return toward the opponent’s half, and produces racket position, velocity, face normal, and timing. One frozen flight packet is used for each solve; the runtime holds the accepted command once the swing engages, keeping the target stable through the strike.

Planner architecture

One policy for both sides

A single feed-forward policy runs at 50 Hz for both forehand and backhand. It observes joint state, body motion and orientation, the offset to its recovery station, and the racket’s target position, velocity, and time-to-strike. Ball trajectories and stroke selection are handled outside the actor.

Observation
110 dimensions
Action
31 joint residuals
Control
50 Hz

The outputs become joint-position targets under the A3’s joint limits; the runtime keeps the two head joints at their nominal pose. The policy’s station feedback supports in-place recovery between strokes.

Learning a continuous rally

Training uses PPO in Isaac Lab with a forehand/backhand reference-motion pair. A privileged critic can use reference motion and true simulated racket state during training, while the deployed actor uses only its own observation contract.

Robot state carries across successive swings instead of resetting after every shot. Training therefore includes forehand-to-backhand transitions, balance, and recovery as part of a continuous sequence. Racket goals include samples fitted to the real venue; mass variation and external pushes add physical variation. Control runs at 50 Hz over 200 Hz simulated physics.

Training design

Keeping simulation and deployment aligned

The trained actor is exported to ONNX with metadata for observation layout, joint order, default pose, action scaling, and joint limits. These shared contracts let the runtime detect incompatible exports rather than silently interpret an action incorrectly.

Evaluation progresses from Isaac Lab to MuJoCo, then to a closed loop with the actual C++ planner and the message format used by the A3 runtime. Physical racket, table, and net contacts are evaluated in MuJoCo before hardware use. The deployment stack includes a native C++ controller and ROS 2 integration.

A complementary model-based serve

The repository also includes a separate serving workflow. It searches for an outgoing ball trajectory, derives the racket’s impact velocity and orientation, then uses damped-least-squares inverse kinematics to convert a Cartesian racket path into arm motion.

The serving video in the gallery shows this MuJoCo application. Its high-level A3 runtime commands the arms while the vendor controller retains the waist, neck, and legs. It is distinct from the learned whole-body return policy.

Serving workflow & demo

Current evidence and open questions

The published October 13, 2026, 30-ball MuJoCo run of model_14600 recorded 30 racket contacts and 30 measured legal landings, with no falls during the continuous motion interval. It included 10 forehand and 20 backhand shots.

The full validation gate did not pass. Station and support-foot behavior, recovery, final heading, event identity and timing, and forehand coverage still failed checks. These are simulator results for a specific run, not a hardware success rate or a completed qualification.

The current system is station-anchored, with in-place recovery. Free locomotion, spin-aware shot planning, opponent adaptation, and shot strategy remain outside the current scope.

Read the experiment report
Explore the repository

HOPE, developed by Hitch Interactive with UC Berkeley ROAR. Cover illustration based on the repository’s A3 model.

Learn & evolve

An AI-guided mobile robot that finds scattered cubes and places them in a basket. Camera observations and execution feedback help its agent choose the next action, adjust failed grasps, and verify each placement.

AI agentsVisual feedbackTask recoveryProject details

AI that closes the loop

Built on the Innate ROS 2 stack, a language-model agent turns a cleanup request into a sequence of pickup and basket-drop skills. After an attempt, execution feedback informs its next skill call and parameters. The task continues through observation, action, verification, and adjustment.

  1. 01ObserveLocate the cube and inspect the scene
  2. 02ActCall a pickup or placement skill
  3. 03CheckRead visual and execution feedback
  4. 04AdjustChange the next attempt and check again

The agent manages task decisions and recovery. Onboard vision, inverse kinematics, and motion routines handle object detection, reachable poses, and physical execution. This gives the AI concrete skills to call and observable results to work with.

Feedback the agent can act on

Each skill reports what happened: no cube found, an unreachable grasp, an empty gripper, or an uncertain placement. Progress feedback can also include annotated camera images. The robot client sends these messages and failure reasons back to the agent, connecting the physical outcome to the next decision.

Main-camera observations guide search and base alignment; wrist-camera observations and gripper state help assess the grasp. A new pickup attempt starts by moving the arm clear of the camera and looking again, so its target comes from a fresh detection.

How a failed attempt changes the next one

The agent is given an explicit recovery policy that maps different failures to different adjustments:

  • SearchIf no cube is found, increase the search budget to 18 scan steps, then 24. The wider search helps distinguish a missed object from a cleared work area.
  • ReachIf inverse kinematics rejects a grasp, retry with a small position offset: first 3 cm backward, then 3 cm to either side. Each new attempt observes the target again and checks reachability.
  • GraspIf the gripper is clearly empty, retry with a grasp target 5 mm lower and an adjusted closure setting. Further retries use small lateral offsets to change the approach.
  • PlaceIf the cube remains in the gripper, retry the basket drop. If the gripper is empty or the cube was lost, return to pickup. If the basket cannot be found, expand the visual search and alignment attempt.

Two levels of correction

Corrections happen both inside a skill and between skill calls. During a pickup, wrist-camera feedback refines alignment before the gripper closes. When the wrist view is insufficient, the skill can return to the main camera and wheels to recover a usable approach. A clearly empty grasp also triggers one local reclose attempt.

If the skill still fails, its result returns to the AI agent. The agent’s recovery instructions then guide a revised call, such as a broader search or a changed grasp offset. Local visual correction and task-level replanning work together across the cleanup loop.

Verify before moving on

An uncertain grasp can proceed to the basket when there is no clear evidence of an empty gripper. Placement uses stricter verification: the robot finds the basket again, aligns, releases, and checks the result. The task policy counts a cube only after a verified successful drop; an uncertain outcome triggers another check or recovery attempt.

Saved camera images, target poses, and arm-state records make each attempt inspectable. They help trace whether a failure arose during detection, alignment, reachability, grasping, or release, supporting further development of the skills and recovery rules.

What improves between attempts

The current implementation adapts the next action within the task: what to look for, which skill to call, and which parameters to change. Its recovery rules are supplied in the agent prompt and skill code. The model’s weights remain fixed during this loop; the project explores how AI-directed retries and physical feedback can support more reliable manipulation.

Watch on YouTube

Drone research

Tethered quadrotor during a visual tracking flight test

Drone Tracking

Visual target tracking on a real UAV, connecting camera perception to onboard control.

ROS 2 / ArUcoPD controlJetson / BetaflightProject details

Camera to controller

  1. LocateROS 2 ArUco detection measures image-space target error.
  2. RespondCascaded PD control on Jetson Xavier sends Betaflight RC commands.
  3. TuneReal flight tests refine the combined tracking and control response.

Signal path

Camera→Target offset→PD control→Flight command

/arducam/image_raw → /marker_pixel → image error → /desired_pose

Onboard integration & calibration

Jetson Xavier hosts ROS 2; Betaflight receives RC output. Calibration includes Kalibr camera calibration and IMU noise characterization with Allan variance.

View project reference
Racing policy flying a gate course with a yellow trajectory trail

Autonomous Drone Racing

Learning to race through gates, with controller evaluation, policy training, and flight dynamics in simulation.

PD / MPCIsaac Lab / PPOLearned dynamicsProject details

Building the simulation workflow

  1. ControlIsaac Sim supports PD and MPC experiments.
  2. LearnIsaac Lab and skrl train PPO racing policies.
  3. ModelA neural model fitted to A2RL flight data replaces simulation dynamics, preparing for later policy fine-tuning.
Training & external control

Isaac Lab defines observations, actions, rewards, and termination. skrl collects rollouts and updates PPO. The ROS bridge exchanges external commands and simulated sensor/state data; PPO steps the environment directly.

Isaac Lab workflow
Task key→Gym + Hydra→EnvCfg + scene↓Policy action→Motor dynamics→Isaac Sim↓Observation · Reward · Done→Rollout + PPO update↻
Task registration

The task key resolves through Gym registration to the environment, EnvCfg, and agent configuration. Hydra loads both configurations.

Environment & assets

EnvCfg combines scene, observations, actions, commands, events, rewards, and termination. Track YAML places gate assets; the scene adds the seven-inch drone, ground, lighting, camera, IMU, and collision sensors.

Actions & physics

ControlAction interprets policy actions. Motor dynamics and control allocation produce thrust and torque, applied to the drone body as Isaac Sim advances physics.

Feedback & PPO

Drone state, target gate, previous action, and sensors form observations. Progress, gate passage, collisions, and stability affect reward. Collision, missed gates, distance limits, or timeout can end an episode. The skrl wrapper passes feedback into rollout collection and PPO updates.

ROS bridge

External motor or RC commands enter the bridge, which steps simulation and publishes pose, velocity, camera, target-gate, and telemetry data. Direct motor commands take priority over RC overrides, followed by hover or zero-action fallback.

Run, replay & inspect

The main entry points are model/skrl/train.py and model/skrl/play.py. ROS launch and telemetry tools support external control. RSL-RL and DreamerV3 entry points are also documented; the source notes uncertain RSL-RL registration.

Configuration & checks

Task, MDP, dynamics, asset, track, and agent files define the experiment. Unit tests, a headless Isaac smoke test, and a Makefile smoke test are separate validation paths.

Original architecture canvas ↗

Training records

Training replayTraining replay from the reference project
Reward behaviorReward behavior from the reference project
Training curvesTraining curves from the reference project
View project reference

Huffman coding

A command-line text compressor built around a Huffman tree. It stores the character frequencies with a packed bitstream, so the decoder can rebuild the tree and restore the original text.

Moby-Dick · repository sample Original text1.25 MB Huffman file719.8 kB 42.6% smallerRestored byte for byte
C++14Huffman treeBinary I/OProject details

From text to bits, and back

  1. 01CountCharacter frequencies
  2. 02BuildPrefix-code tree
  3. 03PackCodes into bytes
  4. 04RestoreRebuild and decode

The command-line program offers compression and decompression. Each operation takes an input path and an output path, writes the result, and exits. The implementation connects data structures to file storage: ordered maps count characters, a priority queue builds the tree, and a separate storage class handles bit packing.

Command-line entry point

A tree shaped by the input

Character counts are stored in std::map. A min-priority queue repeatedly merges the two lightest nodes into a parent until one tree remains. A recursive walk assigns 0 to each left edge and 1 to each right edge, giving every character a prefix code.

The code lengths depend on the input distribution. Frequent characters can use short paths, reducing the number of bits needed for the message. Keeping symbols in an ordered map gives encoding and decoding the same insertion order when rebuilding the tree in this implementation.

Frequency table, tree, and dictionary

The compressed file carries its own dictionary data

Header length→Frequency records→Packed codes + EOF

The header records each symbol as an eight-digit binary string with its frequency. The payload packs variable-length codes into bytes. A pseudo-EOF code marks the end of the message, so the decoder can distinguish real data from padding in the final byte.

Header and bit storage

Decoding across byte boundaries

The decoder reads the stored frequencies and rebuilds the tree. It follows each bit to a leaf, writes the corresponding character, and returns to the root. If a code spans two bytes, the remaining bits carry into the next chunk. Decoding stops when the pseudo-EOF leaf is reached.

Measured on the repository’s samples

All five bundled text files were compressed and restored using the unmodified C++ source. Every restored file matched its input byte for byte.

SampleOriginalCompressedChange
100west20.4 kB11.7 kB42.5% smaller
Sherlock Holmes6.34 MB3.61 MB43.2% smaller
Moby-Dick1.25 MB719.8 kB42.6% smaller
go go gophers17 B121 B104 B larger
2 MB text sample2.17 MB1.15 MB47.1% smaller

Checked October 16, 2026 · Sizes include the header · Decimal kB / MB

The 17-byte sample grows to 121 bytes because the decoding header outweighs the payload savings. Larger samples here shrink by roughly 43–47%; results depend on the text and its character distribution.

Exact sizes & verification record

C++ implementation notes

The Huffman class implements a copy constructor, assignment operator, and destructor, with recursive deep-copy and tree cleanup. Encoding and storage are separated into their own classes.

File format details

The current format targets text input, reserves byte 0x7F for the end marker, and stores a native four-byte header length. The measurements above cover the repository’s five bundled text files on one local build.

Class interface
Explore the repository