ManipulaPy Documentation

A modern, GPU-accelerated Python toolbox for robot kinematics, dynamics, trajectory planning, perception and control.

🤖 Modern Robotics Made Simple

ManipulaPy brings cutting-edge robotics algorithms to your fingertips with GPU acceleration, computer vision integration, and a clean Python API.

⚡ CUDA Accelerated
GPU-powered trajectory planning and dynamics

👁️ Computer Vision
YOLO detection and stereo perception

🎮 Real-time Control
PyBullet simulation and advanced controllers

Getting Started 

If you’re in a hurry, install the package into a fresh virtual-env and try the examples below:

python -m pip install manipulapy[cuda]  # or just manipulapy
python -c "import ManipulaPy; print('🎉 Installation successful!')"

Note

The docs you’re reading are generated from the source in docs/; feel free to improve them and send a pull request 🚀

🚀 Quick Start Examples

1. Your First Robot Analysis

import numpy as np
from ManipulaPy.urdf_processor import URDFToSerialManipulator
from ManipulaPy.ManipulaPy_data.xarm import urdf_file as xarm_urdf_file

# Load the built-in xArm robot model
urdf_processor = URDFToSerialManipulator(xarm_urdf_file)
robot = urdf_processor.serial_manipulator
dynamics = urdf_processor.dynamics

# Compute forward kinematics at home position
joint_angles = np.zeros(6)  # 6-DOF robot at home
end_effector_pose = robot.forward_kinematics(joint_angles, frame="space")

print("🏠 Home position:", end_effector_pose[:3, 3])
print("📐 Home orientation:\n", end_effector_pose[:3, :3])

2. Inverse Kinematics in Action

# Define a target pose for the end-effector
target_angles = np.array([0.5, -0.3, 0.8, 0.0, 0.5, 0.0])
T_target = robot.forward_kinematics(target_angles)

# Solve inverse kinematics
initial_guess = np.zeros(6)
solution, success, iterations = robot.iterative_inverse_kinematics(
    T_desired=T_target,
    thetalist0=initial_guess,
    max_iterations=1000
)

if success:
    print(f"✅ IK solved in {iterations} iterations!")
    print(f"🎯 Solution: {np.degrees(solution):.2f} degrees")
else:
    print("❌ IK solution not found")

3. CUDA-Accelerated Trajectory Planning

from ManipulaPy.path_planning import TrajectoryPlanning

# Setup trajectory planner with GPU acceleration
joint_limits = np.array([[-np.pi, np.pi]] * 6)
planner = TrajectoryPlanning(robot, xarm_urdf_file, dynamics, joint_limits)

# Plan smooth trajectory from start to end
start_angles = np.zeros(6)
end_angles = np.array([0.5, -0.3, 0.8, 0.0, 0.5, 0.0])

trajectory = planner.joint_trajectory(
    thetastart=start_angles,
    thetaend=end_angles,
    Tf=5.0,          # 5 second duration
    N=100,           # 100 waypoints
    method=5         # Quintic time scaling for smoothness
)

print(f"📈 Generated trajectory with {trajectory['positions'].shape[0]} points")
print(f"🎯 Start velocity: {trajectory['velocities'][0]}")
print(f"🏁 End velocity: {trajectory['velocities'][-1]}")

# Visualize the trajectory
planner.plot_trajectory(trajectory, 5.0, title="Smooth Joint Trajectory")

4. Advanced Control with Dynamics

from ManipulaPy.control import ManipulatorController

# Create intelligent controller
controller = ManipulatorController(dynamics)

# Current robot state
current_pos = np.zeros(6)
current_vel = np.zeros(6)

# Desired target state
desired_pos = np.array([0.2, -0.1, 0.3, 0.0, 0.2, 0.0])
desired_vel = np.zeros(6)

# Auto-tune PID gains using Ziegler-Nichols
ultimate_gain = 50.0  # Found experimentally
ultimate_period = 0.5
Kp, Ki, Kd = controller.tune_controller(ultimate_gain, ultimate_period, kind="PID")

print(f"🎛️  Auto-tuned gains - Kp: {Kp[0]:.2f}, Ki: {Ki[0]:.2f}, Kd: {Kd[0]:.2f}")

# Compute optimal control torques
control_torques = controller.computed_torque_control(
    thetalistd=desired_pos,
    dthetalistd=desired_vel,
    ddthetalistd=np.zeros(6),
    thetalist=current_pos,
    dthetalist=current_vel,
    g=np.array([0, 0, -9.81]),
    dt=0.01,
    Kp=Kp, Ki=Ki, Kd=Kd
)

print(f"⚡ Control torques: {control_torques}")

5. PyBullet Simulation

from ManipulaPy.sim import Simulation

# Create realistic physics simulation
sim = Simulation(
    urdf_file_path=xarm_urdf_file,
    joint_limits=joint_limits,
    torque_limits=np.array([[-50, 50]] * 6),
    time_step=0.01,
    real_time_factor=1.0
)

# Initialize robot and planning systems
sim.initialize_robot()
sim.initialize_planner_and_controller()

# Execute the planned trajectory in simulation
waypoints = trajectory["positions"][::10]  # Subsample for demonstration

print("🎬 Running simulation...")
final_position = sim.run_trajectory(waypoints)
print(f"🏁 Final end-effector position: {final_position}")

6. Computer Vision & Perception

from ManipulaPy.vision import Vision
from ManipulaPy.perception import Perception

# Setup camera system
camera_config = {
    "name": "workspace_camera",
    "translation": [0.0, 0.0, 1.0],  # 1m above workspace
    "rotation": [0, 45, 0],           # Look down at 45°
    "fov": 60,
    "intrinsic_matrix": np.array([
        [500, 0, 320],
        [0, 500, 240],
        [0, 0, 1]
    ], dtype=np.float32),
    "distortion_coeffs": np.zeros(5, dtype=np.float32)
}

# Create integrated vision system
vision = Vision(camera_configs=[camera_config])
perception = Perception(vision_instance=vision)

# Detect and analyze obstacles
obstacle_points, cluster_labels = perception.detect_and_cluster_obstacles(
    camera_index=0,
    depth_threshold=3.0,  # Objects within 3m
    eps=0.1,              # DBSCAN clustering parameter
    min_samples=3         # Minimum points per cluster
)

num_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
print(f"👁️  Detected {len(obstacle_points)} obstacle points")
print(f"🔍 Found {num_clusters} distinct object clusters")

💡 Pro Tips for Success

🎯 Start Simple
Begin with forward kinematics before tackling complex control

🔧 Use Built-ins
The xArm URDF model is perfect for learning and testing

📊 Visualize Everything
Use plotting functions to understand robot behavior

⚡ GPU Acceleration
Install CUDA for 7x faster trajectory computations

Key Features at a Glance

🔧 Core Robotics

Kinematics: Forward/inverse with neural network acceleration
Dynamics: Mass matrices, inverse/forward dynamics with caching
Control: PID, computed torque, adaptive controllers
Planning: CUDA-accelerated trajectory generation

👁️ Perception & Vision

Vision: Monocular/stereo camera support
Detection: YOLO-based object detection
3D Processing: Point cloud clustering
Integration: PyBullet camera debugging

⚡ High Performance

CUDA Kernels: GPU-accelerated computations
Parallel Processing: Multi-robot trajectory planning
Optimized Algorithms: Cached mass matrices
Real-time: Sub-millisecond kinematics

Documentation map 

🚀 Get Started

Getting Started with ManipulaPy

🛠️ API Reference

🔧 API Reference Guide

📚 User Guides

Popular Learning Paths 

🆕 Complete Beginner

🤖 Robotics Engineer

💻 Performance Engineer

👁️ Vision Engineer

What’s New 

🎉 Latest in v1.1.0

New: CUDA-accelerated trajectory planning
New: Computer vision and perception modules
New: Neural network inverse kinematics
Improved: 3x faster dynamics computation with caching
Added: PyBullet simulation integration
Enhanced: Documentation with interactive examples

Installation Options

# Basic installation
pip install manipulapy

# With CUDA acceleration (recommended)
pip install manipulapy[cuda]

# With computer vision support
pip install manipulapy[vision]

# Full installation (all features)
pip install manipulapy[all]

# Development installation
git clone https://github.com/boelnasr/ManipulaPy.git
cd ManipulaPy
pip install -e .[dev]

Performance Showcase

⚡ CUDA Acceleration

7x faster trajectory planning
GPU vs CPU for 1000-point trajectories

🧠 Neural Network IK

10x faster convergence
Hybrid approach vs traditional methods

💾 Smart Caching

3x faster dynamics
Cached mass matrices for repeated calls

👁️ Real-time Vision

30 FPS object detection
YOLO integration with 3D localization

Citing ManipulaPy

If you use ManipulaPy in your research, please cite:

@software{manipulapy2024,
  title={ManipulaPy: A Modern Python Library for Robot Manipulation},
  author={Mohamed Aboelnar},
  year={2024},
  url={https://github.com/boelnasr/ManipulaPy},
  version={1.1.0}
}

License 

ManipulaPy is released under the AGPL-3.0 License:

✅ Open Source
Source code freely available

🔄 Copyleft
Derivative works must be open source

🌐 Network Use
Modified network services must provide source

💼 Commercial Use
Permitted with AGPL compliance

For commercial licensing options or AGPL compliance questions, please contact the maintainers.

Indices and tables 

Index - Complete index of all functions, classes, and methods
Module Index - Module index for quick navigation
Search Page - Search the documentation