Action recognition from video sequences. Supports RGB, optical flow, and joint (multi-stream) input types for classifying temporal actions in video clips.

Set model.pretrained_model_path for pretrained backbone weights.

Generated TAO Core schemas are packaged in , with listing available actions. Each generated schema also emits from the schema top-level field. AutoML enablement is declared at the model layer in via . Runnable AutoML still requires and to exist and parse. Use the packaged train schema for , , defaults, min/max bounds, enums, option weights, math conditions, dependencies, and popular parameters. Do not expect at runtime; maintainers regenerate schemas/templates before packaging the skill bank.

This model is AutoML-enabled at the model layer. Before handling any train-stage request, read and resolve the run override from either an explicit value or the user's workflow request. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as for this run only; otherwise default to . When , , and both and are packaged, route the train action through by default with this model's . Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and . Use direct model training only when or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.

Non-train actions such as , , , and deploy flows stay in this model skill. The per-run override does not change model metadata.

Data source overrides are mandatory for every action — the agent MUST construct data source paths from the Per-Action Dataset Requirements table above and include them in .

S3_TRAIN = "s3://bucket/data/train"

{
    "train.num_epochs": 30,
    "train.checkpoint_interval": 10,
    "train.validation_interval": 10,
    "train.num_gpus": 1,
    "dataset.label_map": {
        "catch": 0,
        "smile": 1
    },
    "dataset.batch_size": 2,
    "dataset.train_dataset_dir": f"{S3_TRAIN}/train.tar.gz",
    "dataset.val_dataset_dir": f"{S3_TRAIN}/test.tar.gz",
}

{
    "evaluate.test_dataset_dir": f"{S3_TRAIN}/test.tar.gz",
}

{
    "inference.inference_dataset_dir": f"{S3_TRAIN}/test/smile.tar.gz",
}

Optional. Test dataset is provided as test.tar.gz separate from training.

Launch method: Lightning-managed (single process, Lightning spawns workers).

train.num_gpus

train.gpu_ids

Minimum 1 GPU(s), recommended 2 GPU(s). 16GB+ VRAM per GPU. Memory depends on sequence length and input resolution. batch_size=2 is conservative for video data.

Sequence length mismatch: Ensure video clips have enough frames for the configured rgb_seq_length or of_seq_length.

Model-specific inference mappings belong in this MD file, not in . Generated runners should read this section and apply the mappings with SDK helpers before . This mirrors the old microservices flow.

Inference mappings from TAO Core :

encryption_key

key

evaluate.checkpoint

parent_model

results_dir

output_dir

encryption_key

key

export.checkpoint

parent_model

export.onnx_file

create_onnx_file

results_dir

output_dir

encryption_key

key

inference.checkpoint

parent_model

results_dir

output_dir

encryption_key

key

model.of_pretrained_model_path

ptm_if_no_resume_model

model.rgb_pretrained_model_path

ptm_if_no_resume_model

results_dir

output_dir

train.resume_training_checkpoint_path

resume_model

For or , pass the upstream train/export/AutoML child job id as . The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to and do not patch generated runner scripts to guess checkpoint paths.