NAO Robot Autonomous Ball Search and Kick System

Embodied AI system for autonomous perception, decision-making, and control on NAO humanoid robots

Overview

This project implements a complete embodied AI system on a NAO humanoid robot, enabling autonomous ball search, approach, and kicking.

The system is designed as a closed-loop pipeline:

Perception → Decision-Making → Action

It integrates real-time computer vision, behavior planning, and robot motion control under hardware constraints.

Key capabilities include:

  • real-time ball detection and localization
  • autonomous search and navigation
  • behavior control via finite-state machine (FSM)
  • stable humanoid locomotion and kicking

System Architecture

The system operates as a continuous perception–action loop:

  1. Capture image from onboard camera
  2. Detect and localize the ball
  3. Estimate relative position to the robot
  4. Determine behavior state
  5. Execute motion (walk / align / kick)

This enables the robot to autonomously interact with its environment.

Perception Module

Ball Detection (Lightweight Vision Pipeline)

  • HSV color space transformation for robustness to lighting
  • color thresholding to segment ball region
  • morphological filtering to remove noise
  • contour detection and filtering (area, circularity)

The system extracts the ball center position in image coordinates.

Distance Estimation

  • Based on pinhole camera geometry
  • Uses observed ball size to estimate distance

This provides sufficient accuracy for navigation and kicking.

Behavior Planning (Core Contribution)

A finite-state machine (FSM) was designed to control robot behavior.

Main States

  • Search: scan environment when ball is not visible
  • Approach: move toward detected ball while tracking
  • Alignment: adjust position before kicking
  • Kick: execute kicking motion

State Transitions

Transitions are triggered by:

  • ball visibility
  • estimated distance
  • relative position

This design ensures:

  • stable behavior transitions
  • interpretable decision logic
  • robustness in dynamic environments

Motion Control

Locomotion Control

  • Controlled using NAOqi API (setWalkTargetVelocity)
  • Forward motion and rotation adjusted based on ball position
  • Proportional control used for heading correction

Head Control

  • Active visual scanning using head yaw control
  • Enables recovery when the ball is lost

Kicking Control

  • Implemented using predefined joint trajectories
  • Coordinated hip, knee, and ankle movements
  • Tuned for balance and stability

Technical Stack

Hardware

  • NAO humanoid robot
  • onboard RGB camera
  • IMU and joint encoders

Software

  • NAOqi framework
  • Python / C++ APIs

Computer Vision

  • HSV segmentation
  • contour detection
  • geometric estimation

Algorithms

  • finite-state machine (FSM)
  • proportional control
  • humanoid motion control

Outcome

The system enables the robot to:

  • detect and track the ball in real time
  • actively search for the ball when lost
  • navigate toward the ball
  • perform stable kicking actions

Key Takeaways

  • Built a complete embodied AI system (perception + decision + control)
  • Designed and implemented FSM-based behavior planning
  • Gained hands-on experience with real-world robotic systems

This project provided early exposure to closed-loop intelligent systems, which later influenced my research in autonomous driving and multi-agent perception systems.