Parameter Optimization

Goal

Provide a workflow to design experiments, rank parameter influence, and select optimization strategies for materials simulation calibration.

Requirements

Python 3.10+
No external dependencies (uses Python standard library only)

Inputs to Gather

Before running any scripts, collect from the user:

Input	Description	Example
Parameter bounds	Min/max for each parameter with units	`kappa: [0.1, 10.0] W/mK`
Evaluation budget	Max number of simulations allowed	`50 runs`
Noise level	Stochasticity of simulation outputs	`low` , `medium` , `high`
Constraints	Feasibility rules or forbidden regions	`kappa + mobility < 5`

Decision Guidance

Choosing a DOE Method

Is dimension <= 3 AND full coverage needed?
├── YES → Use factorial
└── NO → Is sensitivity analysis the goal?
    ├── YES → Use quasi-random (preferred; "sobol" is accepted but deprecated)
    └── NO → Use lhs (Latin Hypercube)

Method	Best For	Avoid When
`lhs`	General exploration, moderate dimensions (3-20)	Need exact grid coverage
`quasi-random`	Sensitivity analysis, uniform coverage (preferred)	Very high dimensions (>20)
`sobol`	Deprecated alias of `quasi-random` (emits a warning)	New code (use `quasi-random` )
`factorial`	Low dimension (<4), need all corners	High dimension (exponential growth)

Factorial sizing: the factorial grid is
levels
evenly spaced values per parameter, producing exactly
levels ** params
samples. Set the resolution explicitly with
--levels
(e.g.
--params 2 --levels 4
-> 16 samples). If you use
--budget
instead, the script back-computes
levels = round(budget ** (1/params))
and warns whenever the realized sample count differs from the requested budget (e.g.
--budget 20 --params 2
realizes 16 samples). For an exact design, pass a perfect power (
--budget 16
) or, preferably,
--levels
.

Choosing an Optimizer

Is dimension <= 10 AND budget <= 100?
├── YES → Bayesian Optimization
└── NO → Is dimension <= 20?
    ├── YES → CMA-ES
    └── NO → Random Search with screening

Noise Level	Recommendation
Low	Gradient-based if derivatives available, else Bayesian Optimization
Medium	Bayesian Optimization with noise model
High	Evolutionary algorithms or robust Bayesian Optimization

Script Outputs (JSON Fields)

Script	Output Fields
`scripts/doe_generator.py`	`samples` , `method` , `coverage` ( `count` , `dimension` ; plus `levels` and a top-level `requested_budget` / `note` for factorial)
`scripts/optimizer_selector.py`	`recommended` , `expected_evals` , `notes`
`scripts/sensitivity_summary.py`	`ranking` , `notes`
`scripts/surrogate_builder.py`	`model_type` , `metrics` ( `mse` , `cv_error` , `output_variance` ), `notes`

Workflow

Generate DOE with
```
scripts/doe_generator.py
```
Run simulations at DOE sample points (user's responsibility)
Summarize sensitivity with
```
scripts/sensitivity_summary.py
```
Choose optimizer using
```
scripts/optimizer_selector.py
```
(Optional) Fit surrogate with
```
scripts/surrogate_builder.py
```

CLI Examples

bash

# Generate 20 LHS samples for 3 parameters
python3 scripts/doe_generator.py --params 3 --budget 20 --method lhs --json

# Full factorial with 4 levels per parameter (2 params -> 16 samples)
python3 scripts/doe_generator.py --params 2 --levels 4 --method factorial --json

# Rank parameters by sensitivity scores
python3 scripts/sensitivity_summary.py --scores 0.2,0.5,0.3 --names kappa,mobility,W --json

# Get optimizer recommendation for 3D problem with 50 eval budget
python3 scripts/optimizer_selector.py --dim 3 --budget 50 --noise low --json

# Build surrogate model from simulation data
python3 scripts/surrogate_builder.py --x 0,1,2 --y 10,12,15 --model rbf --json

Conversational Workflow Example

User: I need to calibrate thermal conductivity and diffusivity for my FEM simulation. I can run about 30 simulations.

Agent workflow:

Identify 2 parameters →
```
--params 2
```
Budget is 30 →
```
--budget 30
```

Use LHS for general exploration:

bash

python3 scripts/doe_generator.py --params 2 --budget 30 --method lhs --json

After user runs simulations and provides outputs, summarize sensitivity:

bash

python3 scripts/sensitivity_summary.py --scores 0.7,0.3 --names conductivity,diffusivity --json

Recommend optimizer:

bash

python3 scripts/optimizer_selector.py --dim 2 --budget 30 --noise low --json

Error Handling

Error	Cause	Resolution
`params must be positive`	Zero or negative dimension	Ask user for valid parameter count
`budget must be positive`	Zero or negative budget	Ask user for realistic simulation budget
`argument --method: invalid choice: <value> (choose from lhs, sobol, quasi-random, factorial)`	Invalid method (argparse)	Use decision guidance to pick a valid method
`could not convert string to float: <token>`	Non-numeric value in `--scores` / `--x` / `--y`	Reformat as `0.1,0.2,0.3`
`scores must be a comma-separated list`	Empty `--scores` input	Provide at least one numeric score

Verification checklist

Recorded the exact
```
doe_generator.py
```
```
coverage.count
```
and confirmed it matches the intended design — for
```
factorial
```
, verified
```
count == levels ** params
```
and that no
```
note
```
/
```
requested_budget
```
mismatch warning was emitted (or that the realized count is acceptable).
Confirmed the chosen
```
--method
```
matches the Decision Guidance for the actual dimension/goal, and that
```
quasi-random
```
was used instead of the deprecated
```
sobol
```
alias (no
```
DeprecationWarning
```
in output).
Recorded the
```
optimizer_selector.py
```
```
recommended
```
strategy and
```
expected_evals
```
, and verified
```
expected_evals <= budget
```
so the plan is feasible within the stated evaluation budget.
Logged the
```
sensitivity_summary.py
```
```
ranking
```
and checked whether the top sensitivity is
```
< 0.1
```
(the "All sensitivities are low" note); if so, did not over-interpret the ranking and revisited the output metric.
For surrogate fits, judged quality with
```
metrics.cv_error
```
(leave-one-out), NOT in-sample
```
mse
```
— especially for
```
rbf
```
, where
```
mse
```
is near zero by construction — and compared
```
cv_error
```
against
```
metrics.output_variance
```
to confirm the surrogate beats the constant-mean baseline.
Confirmed any reported
```
cv_error
```
is a finite number (not
```
NaN
```
), i.e. there were enough samples for leave-one-out (
```
poly
```
:
```
n > degree+1
```
;
```
rbf
```
:
```
n >= 3
```
).

Common pitfalls & rationalizations

Tempting shortcut	Why it's wrong / what to do
"RBF surrogate `mse` is ~0, so the model is excellent."	RBF is an exact interpolant — in-sample `mse` is near zero by construction and says nothing about generalization. Judge fit with `metrics.cv_error` and compare it to `output_variance` .
"I asked for `--budget 20` factorial, so I got 20 samples."	Factorial honors `levels ** params` , not the budget; `--budget 20 --params 2` realizes 16 samples and emits a `note` /warning. Use `--levels` for an exact, intended design.
" `sobol` gives me a true Sobol low-discrepancy sequence."	`sobol` is a deprecated alias that emits a `DeprecationWarning` and uses a simplified golden-ratio additive recurrence, not a true Sobol sequence. Use `quasi-random` ; for production Sobol use `scipy.stats.qmc` .
"The optimizer recommendation is just advice — budget doesn't matter."	The recommendation is gated on dimension AND budget (BO only for `dim<=10 AND budget<=100` ), and `expected_evals` is capped at the budget. Record both and confirm the plan fits the real budget.
"One sensitivity score is highest, so that parameter dominates."	The script only sorts the scores you pass in; it computes no sensitivity itself. If the top score is `< 0.1` it flags that all sensitivities are low — get the scores from a real screening/Sobol analysis before trusting the ranking.
"It printed JSON without erroring, so the result is valid."	Exit success only means inputs parsed. Verify the design size, `expected_evals <= budget` , a finite `cv_error` , and that the surrogate beats `output_variance` before trusting any output.

Security

Input Validation

```
sensitivity_summary.py
```
validates
```
--names
```
against
```
[a-zA-Z_][a-zA-Z0-9_ .-]*
```
with a 200-char limit, preventing shell metacharacter injection via crafted parameter names
All numeric list inputs are validated as finite numbers (
```
NaN
```
/
```
Inf
```
rejected)
Comma-separated value lists are capped (10,000 for scores, 100,000 for surrogate data) to prevent resource exhaustion
```
doe_generator.py
```
caps dimension at 1,000 and budget at 1,000,000;
```
optimizer_selector.py
```
caps dimension at 100,000 and budget at 10,000,000
```
--method
```
is validated against a fixed allowlist (
```
lhs
```
,
```
quasi-random
```
/
```
sobol
```
,
```
factorial
```
);
```
sobol
```
is an accepted but deprecated alias of
```
quasi-random
```
```
--noise
```
is validated against a fixed allowlist (
```
low
```
,
```
medium
```
,
```
high
```
)
```
--model
```
(surrogate type) is validated against a fixed allowlist (
```
rbf
```
,
```
poly
```
)
```
--levels
```
(factorial grid resolution) is validated as an integer in
```
[2, 1000]
```

File Access

Scripts read no external files; all inputs are provided via CLI arguments
Scripts write only to stdout (JSON output); no files are created unless the agent explicitly uses the Write tool

Tool Restrictions

Read: Used to inspect script source, references, and user data files
Write: Used to save DOE sample plans, sensitivity rankings, or optimizer recommendations; writes are scoped to the user's working directory
Grep/Glob: Used to locate relevant files and search references
The skill's
```
allowed-tools
```
excludes
```
Bash
```
to prevent the agent from executing arbitrary commands when processing user-provided parameter names and constraints

Safety Measures

No
```
eval()
```
,
```
exec()
```
, or dynamic code generation
All subprocess calls use explicit argument lists (no
```
shell=True
```
)
Reduced tool surface (no Bash) limits the agent to read/write operations only
Parameter names are sanitized before use, preventing injection via crafted identifiers

Limitations

Not for real-time optimization: Scripts provide recommendations, not live optimization loops
Surrogate is lightweight:
```
surrogate_builder.py
```
fits a real 1-D least-squares polynomial (
```
poly
```
) or Gaussian RBF interpolant (
```
rbf
```
) using only the standard library and reports honest residual
```
mse
```
, leave-one-out
```
cv_error
```
, and the data
```
output_variance
```
; for production use scipy/scikit-learn/GPyTorch. For
```
rbf
```
, in-sample
```
mse
```
is near zero by construction (exact interpolation) — judge fit quality with
```
cv_error
```
No automatic simulation execution: User must run simulations externally and provide results

References

```
references/doe_methods.md
```
- Detailed DOE method comparison
```
references/optimizer_selection.md
```
- Optimizer algorithm details
```
references/sensitivity_guidelines.md
```
- Sensitivity analysis interpretation
```
references/surrogate_guidelines.md
```
- Surrogate model selection

Version History

v1.2.2 (2026-06-24): Added Verification checklist and Common pitfalls & rationalizations sections to drive evidence-based use of the DOE, optimizer, sensitivity, and surrogate scripts
v1.2.0 (2026-06-23): Real surrogate fits (
```
poly
```
least-squares,
```
rbf
```
interpolation) with honest
```
mse
```
/
```
cv_error
```
/
```
output_variance
```
; explicit factorial
```
--levels
```
with budget-mismatch warnings; BO dimension cutoff harmonized to dim<=10; corrected Security/Error-Handling/output-field docs to match script behavior
v1.1.0 (2024-12-24): Enhanced documentation, decision guidance, conversational examples
v1.0.0: Initial release with core scripts

parameter-optimization

NPX Install

Tags

SKILL.md Content

Parameter Optimization

Goal

Requirements

Inputs to Gather

Decision Guidance

Choosing a DOE Method

Choosing an Optimizer

Script Outputs (JSON Fields)

Workflow

CLI Examples

Conversational Workflow Example

Error Handling

Verification checklist

Common pitfalls & rationalizations

Security

Input Validation

File Access

Tool Restrictions

Safety Measures

Limitations

References

Version History