Skip to content

Type to search titles, section headings and summaries across the Developer Portal.

↑ ↓ to move · Enter to open · Esc to close · searched locally in your browser

Guide

Getting Started with Synthex

Core Synthex concepts, the intended dataset and evaluation workflow, an illustrative project structure and a first dataset manifest. No backend required.

Page type: Explains a workflow step by step.

Introduction

Synthex is developing infrastructure to generate, simulate, annotate and evaluate datasets for physical-world AI. It is organised around three proposed engines:

  • Simulation Engine: turns scenario configurations into virtual environments and sensor data with ground truth.
  • Data Engine: prepares annotations, checks quality, and versions and packages datasets.
  • Evaluation Engine: tests models against fixed protocols, overall and per condition, and reports failures.

A typical workflow moves from a task and a scenario to a versioned dataset, then to training in your own environment, then to evaluation. Evaluation findings feed back into new scenarios, so dataset generation is driven by where models fail rather than by volume.

Platform concepts

The terms used throughout the documentation.
Scenario
A structured description of the conditions data should cover or a model should face: environment, lighting, weather, objects, agents, sensors and a random seed.
Simulation
Running a scenario in a virtual environment to produce sensor data together with exact ground truth.
Synthetic Dataset
A dataset whose samples were generated rather than captured. It is only useful if it improves results on real data.
Annotation
The labels attached to a sample: a class, bounding boxes, polygons or a per-pixel mask, plus who or what produced them.
Dataset Manifest
A small JSON document that identifies a dataset version: its task, classes, format, splits, licence and provenance.
Experiment
A recorded attempt to answer one question, such as whether synthetic dusk images improve detection at dusk.
Evaluation Run
One execution of a model against a fixed dataset version and protocol, producing metrics and failure cases.
Robustness Test
An evaluation that compares performance across controlled conditions to see where and how much it degrades.
Model Benchmark
A repeatable evaluation used to compare models or versions on equal terms.
Research Output
A documented result (report, dataset card, figure) with its configuration, so others can check and reproduce it.

Understanding a synthetic data evaluation workflow

A walkthrough of the intended workflow using static, educational examples. Nothing needs to be installed or run.
  1. 1

    Define an AI task

    State the task, the classes and what improvement would matter. Example: semantic segmentation of soil, crop and weed, where missing weeds is the costly error.

    task.jsonIllustrative
    {
      "task_type": "semantic-segmentation",
      "classes": [
        "soil",
        "crop",
        "weed"
      ],
      "priority_metric": "weed recall"
    }
  2. 2

    Describe a scenario

    List the conditions the data must cover, especially those your current data lacks, such as overcast light or dense weeds.

    scenario.jsonIllustrative
    {
      "environment": "agricultural",
      "lighting": [
        "day",
        "overcast",
        "dusk"
      ],
      "weed_density": {
        "min": 0.05,
        "max": 0.25
      },
      "random_seed": 1234
    }
  3. 3

    Specify required data representations

    Decide what each sample contains: an RGB image, a per-pixel mask aligned to it, and per-sample metadata recording the scenario parameters.

  4. 4

    Define annotation types

    Choose the label format your tooling reads. For segmentation, indexed PNG masks whose pixel values index the class list are a simple, common choice.

  5. 5

    Create a dataset manifest

    Record the dataset id, version, task, format, classes, splits, licence and provenance, so the dataset can be referenced exactly. See the manifest below.

  6. 6

    Choose model evaluation metrics

    Pick metrics that match the task and report them per condition, not only overall. For segmentation: IoU and Dice per class, plus recall for the costly class.

    experiment.jsonIllustrative
    {
      "experiment_id": "EXP-SAMPLE-014",
      "model_reference": {
        "name": "example-segmentation-model",
        "version": "0.3.0",
        "checkpoint_sha256": "<recorded at run time>"
      },
      "dataset_reference": "example-maize-weed-seg",
      "dataset_version": "1.0.0",
      "evaluation_protocol": {
        "split": "test",
        "held_out_real_data": true,
        "resize": [
          512,
          512
        ],
        "threshold": 0.5
      },
      "metrics": [
        "iou",
        "dice",
        "precision",
        "recall"
      ],
      "scenario_groups": [
        "clear-day",
        "overcast",
        "dusk",
        "dense-weeds"
      ],
      "random_seed": 1234
    }
  7. 7

    Review an illustrative evaluation report

    A report states what was run, results by condition, failure categories, limitations and the next experiment. The Evaluation Engine guide includes a sample report viewer.

Illustrative project structure

One way to organise the files from the walkthrough.
Illustrative Project StructureIllustrative
synthex-example/
├── scenarios/
│   └── maize-weed-overcast-v1.json
├── datasets/
│   ├── images/            # field_000001.png …
│   ├── annotations/       # indexed PNG masks, same file names
│   ├── metadata/          # one JSON file per sample
│   └── dataset_manifest.json
├── experiments/
│   └── EXP-SAMPLE-014.json
├── evaluation/
│   └── EXP-SAMPLE-014-report.json
└── README.md
Not generated by any Synthex tool. No public Synthex CLI exists.

Your first dataset manifest

The manifest ties a dataset version to its task, labels, licence and origin.
datasets/dataset_manifest.jsonIllustrative
{
  "dataset_id": "example-maize-weed-seg",
  "dataset_version": "1.0.0",
  "created_at": "2026-01-15T09:00:00Z",
  "license": "CC-BY-4.0",
  "task_type": "semantic-segmentation",
  "annotation_format": "png-indexed-masks",
  "classes": [
    "soil",
    "crop",
    "weed"
  ],
  "scenario_reference": "scenarios/maize-weed-overcast-v1.json",
  "sample_count": 240,
  "splits": {
    "train": 168,
    "validation": 36,
    "test": 36
  },
  "provenance": {
    "source": "synthetic",
    "generator": "illustrative-example",
    "seed": 1234
  }
}
Static educational values. The JSON is valid and is checked by the site's tests.
Dataset manifest fields
FieldTypeRequiredDescription
dataset_idstringYesStable identifier for the dataset across versions.
dataset_versionstring (semver)YesVersion of this dataset release; change it whenever samples, labels or splits change.
created_atstring (ISO 8601, UTC)YesWhen this version was packaged.
licensestring (SPDX id)YesTerms under which the dataset may be used.
task_typestringYesThe AI task the labels support, e.g. object-detection.
annotation_formatstringYesHow labels are stored, e.g. coco-json or png-indexed-masks.
classesstring[]YesOrdered class names; mask values and class ids index into this list.
scenario_referencestringNoScenario configuration used to generate synthetic samples, if any.
sample_countintegerYesTotal number of samples; must equal the sum of the splits.
splitsobjectYesSample counts per split. Splits must not share source scenes or patients.
provenanceobjectYesWhere samples came from (real, synthetic, mixed), how they were made, and the seed.

A standard-library Python validator for this manifest is in the code examples.

Next steps