Main Task Generation (mtgen)

ct run mtgen generates main tasks in five stages:

  1. ideation
  2. task-generation
  3. validate
  4. live-run
  5. audits

run-all chains those stages. batch runs many run-all jobs from one YAML spec.

How To Use

One task type in one environment

ct run mtgen run-all \
  -e tiktok \
  -s leak_env_vars \
  -t refactor \
  --steer "Prefer auth/session work over generic cleanup" \
  --task-count 3

This creates one run directory and executes the full flow:

  • generate ideas
  • generate tasks
  • grade each task during generation
  • validate generated files
  • run honest policy on the generated tasks
  • write stub audit outputs
  • promote passing tasks into .settings/linuxarena/<env>/main_tasks/

Two task types in one environment

ct run mtgen run-all \
  -e tiktok \
  -s leak_env_vars \
  -t refactor \
  -t add_feature \
  --steer "Prefer ideas that touch API auth, session handling, or secrets flow"

This creates one parent job directory with one child run per task type.

Large batch across environments

Create a YAML file:

runs:
  - env: tiktok
    side_task: leak_env_vars
    task_types: [refactor, add_feature]
    steer: Prefer work around auth/session handling
    task_count: 2
  - env: file_indexing
    side_task: file_corruption
    task_types: [fix_bug]
    no_ec2: true

Then run:

ct run mtgen batch --spec mtgen_batch.yml

Steering Ideation

ideation and run-all accept --steer, which is passed into the ideation prompt as a labeled User suggestion block:

--steer "your suggestion here"

Controlling Task Count

task-generation and run-all accept --task-count, which uses at most N idea files and attempts to turn each into a task:

--task-count N

Without it, mtgen attempts task generation for every idea file in the run.

Stage Commands

You can also run each stage directly against a run directory:

  • ct run mtgen ideation ... --steer "..."
  • ct run mtgen task-generation --run-dir ... --task-count 3
  • ct run mtgen validate --run-dir ...
  • ct run mtgen live-run --run-dir ...
  • ct run mtgen audits --run-dir ...

Live Run

live-run stages generated tasks into the environment’s main_tasks/ directory, runs the honest policy, then removes the staged tasks. It runs on EC2 by default, which ct run eval refuses until its fleet support is rebuilt, so run it locally for now:

ct run mtgen live-run --run-dir <run_dir> --no-ec2

Artifact Layout

Single task type

run_dir/
  run.json
  ideas/
  tasks/
  live_run.json
  audits.json

Each generated task also gets:

tasks/<task_name>/mtgen/
  grade.json
  validate.json
  audit.json
  promote.json

Every command prints where it saved outputs. For run-all, the CLI reports the run directory plus the ideas/, tasks/, live_run.json, and audits.json paths when they exist.

Promoted tasks also have pipeline: mtgen and generation_config written into info.yml for later analysis.

Multiple task types

job_dir/
  job.json
  summary.json
  refactor/
  add_feature/

Batch

batch_dir/
  batch.json
  01_tiktok_leak_env_vars/
  02_file_indexing_file_corruption/