Skip to content

How TongSIM Works

TongSIM Lite follows a client–server design:

  • Server (Unreal Engine) runs the simulation and exposes APIs via gRPC
  • Client (Python SDK) connects to the server and sends RPC calls (spawn/move/query/voxel, etc.)
Unreal Engine (TongSIM Lite)                 Python (tongsim)
┌──────────────────────────┐                ┌──────────────────────┐
│ TongSimCore + TongSimGrpc│  gRPC/protobuf  │ TongSim / UnaryAPI    │
│ World & physics          │ <-------------> │ examples/*.py         │
└──────────────────────────┘                └──────────────────────┘

Endpoints

  • Client default: 127.0.0.1:5726
  • Server bind: 0.0.0.0:5726 (listens on all interfaces)

Unreal side (server)

What the server does:

  • Owns the world state (levels, actors, physics, navigation, perception)
  • Executes actions (move, navigate, pick up, traces, voxel queries)
  • Streams results back to the client through gRPC responses

How to run it:

  • Open unreal/TongSim_Lite.uproject
  • Start a Play session (PIE) so the gRPC server is available

Python side (client)

What the client does:

  • Connects to the UE server endpoint
  • Calls high-level RPC helpers (for example UnaryAPI.reset_level, UnaryAPI.spawn_actor, UnaryAPI.query_voxel)
  • Runs experiments/training loops (see examples/)

Console commands

The SDK exposes UnaryAPI.exec_console_command(...), which can run UE console commands (for example stat fps, open <level>). The quickstart script examples/quickstart_demo.py uses this to switch to the demo level automatically.

  1. Start Play in Unreal Editor
  2. Run a Python script from examples/
  3. Iterate on:
    • maps/blueprints on the UE side
    • policies/logic/data collection on the Python side

Tutorials

Once you can run examples/quickstart_demo.py, move on to Tutorials:

  • Single-agent RL Navigation — exploration + navigation in multi-room indoor scenes, with Gymnasium-style wrappers and Stable-Baselines3 baselines. Start here: Task Overview
  • Multi-agent RL Navigation — cooperative multi-agent task (MACS) built on a lightweight adaptation of XuanCe, focusing on coordination under partial observability. Start here: Task Overview

Next: Single-agent RL Navigation · Multi-agent RL Navigation