Usage Guide

This comprehensive guide will help you get started with pSim, from basic usage to advanced features. pSim provides multiple environment types for different use cases: traditional control algorithms, reinforcement learning with Gymnasium, and multi-agent learning with PettingZoo.

Quick Start

First, make sure you have installed pSim with the appropriate dependencies.

Scenario Configuration

pSim scenario configurations are defined natively in Python. You can choose a built-in scenario (like "formation" or "full_random") or pass a custom configuration dictionary directly to the environment:

from pSim import SimpleVSSSEnv

# Use a built-in scenario
env = SimpleVSSSEnv(scenario="formation")

# Or pass a custom scenario dictionary
custom_scenario = {
    "ball_velocity": [0.0, 0.0],
    "ball_position_type": "fixed",
    "ball_position": [0.0, 0.0],
    "agent_robots": {
        "position_type": "fixed",
        "positions": [[-0.1, 0.0, 0.0]],
        "movement_types": ["action"]
    },
    "adversary_robots": {
        "position_type": "fixed",
        "positions": [[0.25, 0.0, 3.14159]],
        "movement_types": ["action"]
    }
}
env_custom = SimpleVSSSEnv(scenario=custom_scenario)

Environment Types

pSim offers three main environment types:

1. SimpleEnv - Traditional Control

Best for traditional control algorithms and manual testing.

from pSim import SimpleVSSSEnv
import numpy as np

# Create environment
env = SimpleVSSSEnv(
    render_mode="human",  # "human" for GUI, None for headless
    scenario="formation",
    num_agent_robots=3,
    num_adversary_robots=3,
    color_team="blue"
)

# Reset and run
obs, info = env.reset()
print(f"Observation keys: {list(obs.keys())}")

# Control loop
for step in range(1000):
    # Random actions for demonstration [v, w] for each robot
    action = np.random.uniform(-1, 1, (env.num_agent_robots, 2))
    obs, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        obs, info = env.reset()

env.close()

2. VSSSGymEnv - Gymnasium Integration

For reinforcement learning with Gymnasium.

from pSim import VSSSGymEnv
from gymnasium.wrappers import FlattenObservation
import numpy as np

# Create environment with flattened observations
env = FlattenObservation(VSSSGymEnv(
    render_mode="human",
    scenario="formation",
    num_agent_robots=3,
    num_adversary_robots=3,
    color_team="blue"
))

print(f"Action space: {env.action_space}")
print(f"Observation space: {env.observation_space}")

# Reset and run
obs, info = env.reset()
print(f"Observation shape: {obs.shape}")

for step in range(1000):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        obs, info = env.reset()

env.close()

3. VSSSPettingZooEnv - Multi-Agent Learning

For multi-agent reinforcement learning with PettingZoo.

from pSim import VSSSPettingZooEnv
import numpy as np

# Create multi-agent environment
env = VSSSPettingZooEnv(
    render_mode="human",
    scenario="formation",
    num_agent_robots=3,
    num_adversary_robots=3,
    color_team="blue"
)

print(f"Agents: {env.possible_agents}")
print(f"Action space: {env.action_space('agent_0')}")

# Reset and run
obs, info = env.reset()
print(f"Active agents: {env.agents}")

for step in range(1000):
    # Random actions for all agents
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    obs, rewards, terminations, truncations, infos = env.step(actions)

    if any(terminations.values()) or any(truncations.values()):
        obs, info = env.reset()

env.close()

Core Concepts

Rendering Modes

pSim supports different rendering modes for various use cases:

• "human": Displays a graphical window for interactive visualization
• None: Headless mode for training and automated testing
• "rgb_array": Returns RGB arrays for custom rendering or recording

# Interactive mode - shows GUI window
env = SimpleVSSSEnv(render_mode="human")

# Headless mode - no GUI, faster for training
env = SimpleVSSSEnv(render_mode=None)

# RGB array mode - for custom processing
env = SimpleVSSSEnv(render_mode="rgb_array")
obs, info = env.reset()
rgb_frame = env.render()  # Returns numpy array with RGB image

Actions and Observations

Action Format

Actions control robot movement as [linear_velocity, angular_velocity] pairs:

# For a single robot
action = [0.5, 0.2]  # Move forward at 0.5 speed, turn right at 0.2 speed

# For multiple robots (SimpleEnv)
actions = [
    [0.5, 0.2],  # Robot 0: forward + right turn
    [-0.3, 0.0], # Robot 1: backward + straight
    [0.0, -0.1]  # Robot 2: stop + left turn
]

Velocity ranges: -1.0 to 1.0 (normalized) - Linear velocity: Forward/backward movement - Angular velocity: Rotation (positive = right turn, negative = left turn)

Observation Format

Observations contain the complete environment state:

obs = {
    'agent_robots': [
        [x, y, theta, vx, vy, omega],  # Robot 0: position, orientation, velocities
        [x, y, theta, vx, vy, omega],  # Robot 1: ...
    ],
    'adversary_robots': [...],  # Same format for opponent robots
    'ball': [x, y, vx, vy],     # Ball position and velocity
    'field': [...],             # Field boundaries and features
    'game_state': {             # Game status information
        'score': {'blue': 0, 'yellow': 0},
        'time': 0.0,
        'episode_length': 1000
    }
}

Core Methods

reset()

Initializes or reinitializes the environment:

obs, info = env.reset()
# obs: Initial observation dictionary
# info: Additional information (usually empty dict)

When to call: At the start of each episode or when you want to restart the simulation.

step(action)

Advances the simulation by one time step:

obs, reward, terminated, truncated, info = env.step(action)

Parameters: - action: Control inputs for robots

Returns: - obs: New observation after the action - reward: Scalar reward value - terminated: True if episode ended naturally (goal scored, etc.) - truncated: True if episode was cut short (time limit, etc.) - info: Additional diagnostic information

Termination Conditions

• terminated = True: Episode ended due to game logic (goal scored, out of bounds, etc.)
• truncated = True: Episode ended due to external constraints (time limit, manual stop, etc.)

# Always check both conditions
if terminated or truncated:
    obs, info = env.reset()  # Start new episode

Reward System Customization

pSim allows you to customize the reward system by subclassing the RewardSystem class and injecting it into your environment. This provides full control over the reward calculation logic.

Creating a Custom Reward System

To create a custom reward system, inherit from RewardSystem and override the calculate_reward method.

from pSim.modules.env_description import RewardSystem
import numpy as np

class CustomRewardSystem(RewardSystem):
    """Custom reward system example."""

    def calculate_reward(self) -> tuple[float, bool, bool]:
        """
        Calculate custom reward.
        Returns: (reward, terminated, truncated)
        """
        # Access simulator state via self.simulator
        ball = self.simulator.ball_body
        agent = self.simulator.robots_agent[0]

        # Example: Simple distance-based reward
        distance = np.linalg.norm(agent.position - ball.position)
        reward = -distance

        # Check termination (e.g., goal scored)
        # You can use self.simulator.contact_listener or check positions
        terminated = False
        if distance < 0.1:
            reward += 10
            terminated = True

        # Check truncation (time limit)
        truncated = self.time_step >= self.truncated_time

        return reward, terminated, truncated

Using Custom Rewards

To use your custom reward system, you need to create a custom environment class that initializes it.

from pSim import VSSSGymEnv
from my_custom_rewards import CustomRewardSystem

class CustomVSSSGymEnv(VSSSGymEnv):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        # Override the default reward system
        self.reward_system = CustomRewardSystem(
            self.simulator, 
            truncated_time=kwargs.get('truncated_time', 600)
        )

# Usage
env = CustomVSSSGymEnv(render_mode="human", scenario="formation")

For a complete working example, see examples/vsss_gym_custom_obs_reward.py.

Configuration System

pSim configurations define scenario parameters, initial positions, and robot behaviors.

Configuration Dictionary Structure

A scenario configuration dictionary contains the following keys:

• ball_position_type: "fixed" or "range".
• ball_position: [x, y] coordinate pair (used if type is "fixed").
• ball_position_range: {"x": [min, max], "y": [min, max]} (used if type is "range").
• ball_velocity: [vx, vy] velocity bounds.
• agent_robots / adversary_robots: Dictionaries specifying:
• position_type: "fixed" or "range".
• positions: List of robot poses [[x, y, theta], ...] (used if type is "fixed").
• position_range: {"x": [min, max], "y": [min, max], "angle": [min, max]} (used if type is "range").
• movement_types: List of robot movement types (e.g. ["action", "ou", "no_move"]).

Movement Types

• "action": Robot controlled directly by control actions.
• "ou": Robot moves according to an Ornstein-Uhlenbeck process.
• "no_move": Robot remains stationary.

Default Scenarios

Default scenarios are defined in game_setup.py under the DEFAULT_SCENARIOS dictionary. - "formation": A standard 3v3 formation with fixed initial positions. - "full_random": Poses and ball coordinates randomized within range bounds.

Human-Machine Interface (HMI)

pSim includes a sophisticated HMI system for manual control with keyboard and joystick support.

Keyboard Controls

Key	Action
W/S	Forward/Backward movement
A/D	Turn left/right
E/Q	Next/Previous robot
X/Y	Switch to Blue/Yellow team
B	Toggle ball control mode
R	Reset environment
ESC	Exit

Joystick Controls (Universal Mapping)

Control	Action
Left Stick Y	Forward/Backward
Left Stick X	Turn left/right (robot) or strafe (ball)
Right Stick X	Turn left/right (robot only)
RB/LB	Next/Previous robot
X/Y	Switch to Blue/Yellow team
B	Toggle ball control mode
BACK	Reset environment
START	Exit

HMI Example

from pSim import SimpleVSSSEnv, HMI

env = SimpleVSSSEnv(render_mode="human", num_agent_robots=3)
hmi = HMI()

obs, info = env.reset()

while hmi.active:
    # HMI returns a dictionary with control state
    control_state = hmi()

    if not control_state['active']:
        break

    if control_state['reset_commanded']:
        obs, info = env.reset()
        continue

    # Get actions from HMI state
    actions = control_state['actions']

    # Apply actions to controllable robots
    # For SimpleEnv with multiple robots, you might need to route actions
    # This is handled internally in main.py, but here is a simplified view:
    env_actions = np.zeros((env.num_agent_robots, 2))
    current_robot = control_state['current_robot_id']
    if current_robot < env.num_agent_robots:
        env_actions[current_robot] = actions

    obs, reward, terminated, truncated, info = env.step(env_actions)

    if terminated or truncated:
        obs, info = env.reset()

hmi.quit()
env.close()

Advanced Multi-Robot Control

The HMI automatically handles robot and team switching. You can access the current selection from the returned state:

control_state = hmi()
current_team = control_state['current_team']
current_robot_id = control_state['current_robot_id']
ball_mode = control_state['ball_control_mode']

Troubleshooting

Common Issues

Wrong robot count - Check that the movement_types lists match the num_agent_robots and num_adversary_robots configured in your scenario.

Environment won't start - Check that all dependencies are installed - Verify your Python version (3.12+ recommended) - Try running with render_mode=None first

Controller not detected - Try unplugging and replugging your controller - Check that pygame is properly installed - Some controllers may need additional drivers

Getting Help

• Check the API Reference for detailed documentation
• Review the examples in the examples/ directory