Arize Dataset Skill

Concepts

• Dataset = a versioned collection of examples used for evaluation and experimentation
• Dataset Version = a snapshot of a dataset at a point in time; updates can be in-place or create a new version
• Example = a single record in a dataset with arbitrary user-defined fields (e.g.,

  question

  ,

  answer

  ,

  context

  )
• Space = an organizational container; datasets belong to a space

id

created_at

updated_at

Prerequisites

ax

Install ax

ax

1. Check if

   ax

   is on PATH:

   command -v ax

   (Unix) or

   where ax

   (Windows)
2. If not found, check common install locations:
   • macOS/Linux:

     test -x ~/.local/bin/ax && export PATH="$HOME/.local/bin:$PATH"

   • Windows: check

     %APPDATA%\Python\Scripts\ax.exe

     or

     %LOCALAPPDATA%\Programs\Python\Scripts\ax.exe

3. If still not found, install it (requires shell access to install packages):
   • Preferred:

     uv tool install arize-ax-cli

   • Alternative:

     pipx install arize-ax-cli

   • Fallback:

     pip install arize-ax-cli

4. After install, if

   ax

   is not on PATH:
   • macOS/Linux:

     export PATH="$HOME/.local/bin:$PATH"

   • Windows (PowerShell):

     $env:PATH = "$env:APPDATA\Python\Scripts;$env:PATH"

5. If

   ax --version

   fails with an SSL/certificate error:
   • macOS:

     export SSL_CERT_FILE=/etc/ssl/cert.pem

   • Linux:

     export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

   • Windows (PowerShell):

     $env:SSL_CERT_FILE = "C:\Program Files\Common Files\SSL\cert.pem"

     (or use

     python -c "import certifi; print(certifi.where())"

     to find the cert bundle)

6. ax --version

   must succeed before proceeding. If it doesn't, stop and ask the user for help.

Verify environment

ax --version && echo "--- env ---" && echo "ARIZE_API_KEY: ${ARIZE_API_KEY:-(not set)}" && echo "ARIZE_SPACE_ID: ${ARIZE_SPACE_ID:-(not set)}" && echo "--- profiles ---" && ax profiles show 2>&1

ax --version; Write-Host "--- env ---"; Write-Host "ARIZE_API_KEY: $env:ARIZE_API_KEY"; Write-Host "ARIZE_SPACE_ID: $env:ARIZE_SPACE_ID"; Write-Host "--- profiles ---"; ax profiles show 2>&1

• No API key in env and no profile → AskQuestion: "Arize API key (https://app.arize.com/admin > API Keys)"
• Space ID unknown → AskQuestion, or run

  ax projects list -o json --limit 100

  and search for a match
• Project unclear → ask, or run

  ax projects list -o json --limit 100

  and present as selectable options

Space ID and Project

1. User provides it in the conversation -- use directly via

   --space-id

   /

   --project

   flags.
2. Env var is set (

   ARIZE_SPACE_ID

   ,

   ARIZE_DEFAULT_PROJECT

   ) -- use silently.
3. If missing, AskQuestion once. Tell the user:
   • Space ID is in the Arize URL:

     /spaces/{SPACE_ID}/...

   • Project is the project name as shown in the Arize UI.
   • For convenience, recommend setting env vars so they don't get asked again:

     export ARIZE_SPACE_ID="U3BhY2U6..."

     and

     export ARIZE_DEFAULT_PROJECT="my-project"

401 Unauthorized

List Datasets:

ax datasets list

ax datasets list
ax datasets list --space-id SPACE_ID --limit 20
ax datasets list --cursor CURSOR_TOKEN
ax datasets list -o json

Flags

--space-id

--limit, -l

--cursor

-o, --output

-p, --profile

Get Dataset:

ax datasets get

ax datasets get DATASET_ID
ax datasets get DATASET_ID -o json

Flags

DATASET_ID

-o, --output

-p, --profile

Response fields

id

name

space_id

created_at

updated_at

versions

Export Dataset:

ax datasets export

--all

ax datasets export DATASET_ID
# -> dataset_abc123_20260305_141500/examples.json

ax datasets export DATASET_ID --all
ax datasets export DATASET_ID --version-id VERSION_ID
ax datasets export DATASET_ID --output-dir ./data
ax datasets export DATASET_ID --stdout
ax datasets export DATASET_ID --stdout | jq '.[0]'

Flags

DATASET_ID

--version-id

--all

--output-dir

.

--stdout

-p, --profile

REST vs Flight (

--all

)

• REST (default): Lower friction -- no Arrow/Flight dependency, standard HTTPS ports, works through any corporate proxy or firewall. Limited to 500 examples per page.
• Flight (

  --all

  ): Required for datasets with more than 500 examples. Uses gRPC+TLS on a separate host/port (

  flight.arize.com:443

  ) which some corporate networks may block.

--all

id

created_at

updated_at

[
  {
    "id": "ex_001",
    "created_at": "2026-01-15T10:00:00Z",
    "updated_at": "2026-01-15T10:00:00Z",
    "question": "What is 2+2?",
    "answer": "4",
    "topic": "math"
  }
]

Create Dataset:

ax datasets create

ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.csv
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.json
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.jsonl
ax datasets create --name "My Dataset" --space-id SPACE_ID --file data.parquet

Flags

--name, -n

--space-id

--file, -f

-o, --output

-p, --profile

Supported file formats

.csv

.json

.jsonl

.parquet

Append Examples:

ax datasets append

Inline JSON (agent-friendly)

ax datasets append DATASET_ID --json '[{"question": "What is 2+2?", "answer": "4"}]'

ax datasets append DATASET_ID --json '[
  {"question": "What is gravity?", "answer": "A fundamental force..."},
  {"question": "What is light?", "answer": "Electromagnetic radiation..."}
]'

From a file

ax datasets append DATASET_ID --file new_examples.csv
ax datasets append DATASET_ID --file additions.json

To a specific version

ax datasets append DATASET_ID --json '[{"q": "..."}]' --version-id VERSION_ID

Flags

DATASET_ID

--json

--file, -f

--version-id

-o, --output

-p, --profile

--json

--file

Validation

• Each example must be a JSON object with at least one user-defined field
• Fields

  id

  ,

  created_at

  ,

  updated_at

  are auto-generated -- do not include them
• Maximum 100,000 examples per request

Delete Dataset:

ax datasets delete

ax datasets delete DATASET_ID
ax datasets delete DATASET_ID --force   # skip confirmation prompt

Flags

DATASET_ID

--force, -f

-p, --profile

Workflows

Create a dataset from file for evaluation

1. Prepare a CSV/JSON/Parquet file with your evaluation columns (e.g.,

   input

   ,

   expected_output

   )

2. ax datasets create --name "eval-set-v1" --space-id SPACE_ID --file eval_data.csv

3. Verify:

   ax datasets get DATASET_ID

4. Use the dataset ID to run experiments

Add examples to an existing dataset

# Find the dataset
ax datasets list

# Append inline (e.g., from an LLM-generated payload)
ax datasets append DATASET_ID --json '[
  {"question": "What is gravity?", "answer": "A fundamental force..."},
  {"question": "What is light?", "answer": "Electromagnetic radiation..."}
]'

# Or append from a file
ax datasets append DATASET_ID --file additional_examples.csv

Download dataset for offline analysis

1. ax datasets list

   -- find the dataset

2. ax datasets export DATASET_ID

   -- download to file
3. Parse the JSON:

   jq '.[] | .question' dataset_*/examples.json

Export a specific version

# List versions
ax datasets get DATASET_ID -o json | jq '.versions'

# Export that version
ax datasets export DATASET_ID --version-id VERSION_ID

Iterate on a dataset

1. Export current version:

   ax datasets export DATASET_ID

2. Modify the examples locally
3. Append new rows:

   ax datasets append DATASET_ID --file new_rows.csv

4. Or create a fresh version:

   ax datasets create --name "eval-set-v2" --space-id SPACE_ID --file updated_data.json

Pipe export to other tools

# Count examples
ax datasets export DATASET_ID --stdout | jq 'length'

# Extract a single field
ax datasets export DATASET_ID --stdout | jq '.[].question'

# Convert to CSV with jq
ax datasets export DATASET_ID --stdout | jq -r '.[] | [.question, .answer] | @csv'

Dataset Example Schema

id

created_at

updated_at

Troubleshooting

ax: command not found

~/.local/bin/ax

uv tool install arize-ax-cli

401 Unauthorized

No profile found

ax profiles show --expand

ARIZE_API_KEY

~/.arize/config.toml

Dataset not found

ax datasets list

File format error

platform-managed column

id

created_at

updated_at

reserved column

time

count

source_record_*

Provide either --json or --file

Examples array is empty

not a JSON object

--json

{...}

Save Credentials for Future Use

ax

~/.arize/config.toml

~/.zshrc

~/.bashrc

export ARIZE_SPACE_ID="..."

[System.Environment]::SetEnvironmentVariable('ARIZE_SPACE_ID', '...', 'User')

• The API key was already loaded from an existing profile or

  ARIZE_API_KEY

  env var
• The space ID was already set via

  ARIZE_SPACE_ID

  env var
• The user only used base64 project IDs (no space ID was needed)

"Yes, save them"

"No thanks"

1. API key — Check if

   ~/.arize/config.toml

   exists. If it does, read it and update the

   [auth]

   section. If not, create it with this minimal content:
   toml

   [profile]
   name = "default"
   
   [auth]
   api_key = "THE_API_KEY"
   
   [output]
   format = "table"

   Verify with:

   ax profiles show

2. Space ID — Persist the space ID as an environment variable:
   macOS/Linux — Detect the user's shell config file (

   ~/.zshrc

   for zsh,

   ~/.bashrc

   for bash). Append:
   bash

   export ARIZE_SPACE_ID="THE_SPACE_ID"

   Tell the user to run

   source ~/.zshrc

   (or restart their terminal) for it to take effect.
   Windows (PowerShell) — Set a persistent user environment variable:
   powershell

   [System.Environment]::SetEnvironmentVariable('ARIZE_SPACE_ID', 'THE_SPACE_ID', 'User')

   Tell the user to restart their terminal for it to take effect.