devtu-fix-tool

Original：🇺🇸 English

Translated

Fix failing ToolUniverse tools by diagnosing test failures, identifying root causes, implementing fixes, and validating solutions. Use when ToolUniverse tools fail tests, return errors, have schema validation issues, or when asked to debug or fix tools in the ToolUniverse framework.

3installs

Sourcemims-harvard/tooluniverse

Added on2026-02-16

NPX Install

npx skill4agent add mims-harvard/tooluniverse devtu-fix-tool

SKILL.md Content

View Translation Comparison →

Fix ToolUniverse Tools

Diagnose and fix failing ToolUniverse tools through systematic error identification, targeted fixes, and validation.

Instructions

When fixing a failing tool:

Run targeted test to identify error:

bash

python scripts/test_new_tools.py <tool-pattern> -v

Verify API is correct - search online for official API documentation to confirm endpoints, parameters, and patterns are correct
Identify error type (see Error Types section)
Apply appropriate fix based on error pattern
Regenerate tools if you modified JSON configs or tool classes:

bash

python -m tooluniverse.generate_tools

Check and update unit tests if they exist in
```
tests/unit/
```
:

bash

ls tests/unit/test_<tool-name>_tool.py

Verify fix by re-running both integration and unit tests
Provide fix summary with problem, root cause, solution, and test results

Where to Fix

Issue Type	File to Modify
Binary response	`src/tooluniverse/_tool.py` + `src/tooluniverse/data/_tools.json`
Schema mismatch	`src/tooluniverse/data/*_tools.json` (return_schema)
Missing data wrapper	`src/tooluniverse/*_tool.py` (operation methods)
Endpoint URL	`src/tooluniverse/data/*_tools.json` (endpoint field)
Invalid test example	`src/tooluniverse/data/*_tools.json` (test_examples)
Unit test updates	`tests/unit/test_*_tool.py` (if exists)
API key as parameter	`src/tooluniverse/data/_tools.json` (remove param) + `_tool.py` (use env var)
Tool not loading (optional key)	`src/tooluniverse/data/*_tools.json` (use `optional_api_keys` not `required_api_keys` )

Error Types

1. JSON Parsing Errors

Symptom:

Expecting value: line 1 column 1 (char 0)

Cause: Tool expects JSON but receives binary data (images, PDFs, files)

Fix: Check Content-Type header. For binary responses, return a description string instead of parsing JSON. Update

return_schema

{"type": "string"}

2. Schema Validation Errors

Symptom:

Schema Mismatch: At root: ... is not of type 'object'

Data: None

Cause: Missing

data

field wrapper OR wrong schema type

Fix depends on the error:

If
```
Data: None
```
→ Add
```
data
```
wrapper to ALL operation methods (see Multi-Operation Pattern below)

If type mismatch → Update

return_schema

in JSON config:

Data is string:
```
{"type": "string"}
```
Data is array:
```
{"type": "array", "items": {...}}
```
Data is object:
```
{"type": "object", "properties": {...}}
```

Key concept: Schema validates the

data

field content, NOT the full response.

3. Nullable Field Errors

Symptom:

Schema Mismatch: At N->fieldName: None is not of type 'integer'

Cause: API returns

None

null

for optional fields

Fix: Allow nullable types in JSON config using

{"type": ["<base_type>", "null"]}

. Use for optional fields, not required identifiers.

4. Mutually Exclusive Parameter Errors

Symptom:

Parameter validation failed for 'param_name': None is not of type 'integer'

when passing a different parameter

Cause: Tool accepts EITHER paramA OR paramB (mutually exclusive), but both are defined with fixed types. When only one is provided, validation fails because the other is

None

Example:

json

{
  "neuron_id": {"type": "integer"},      // ❌ Fails when neuron_name is used
  "neuron_name": {"type": "string"}      // ❌ Fails when neuron_id is used
}

Fix: Make mutually exclusive parameters nullable:

json

{
  "neuron_id": {"type": ["integer", "null"]},      // ✅ Allows None
  "neuron_name": {"type": ["string", "null"]}      // ✅ Allows None
}

Common patterns:

```
id
```
OR
```
name
```
parameters (get by ID or by name)
```
acronym
```
OR
```
name
```
parameters (search by symbol or full name)
Optional filter parameters that may not be provided

Important: Also make truly optional parameters (like

filter_field

filter_value

) nullable even if not mutually exclusive.

5. Mixed Type Field Errors

Symptom:

Schema Mismatch: At N->field: {object} is not of type 'string', 'null'

Cause: Field returns different structures depending on context

Fix: Use

oneOf

in JSON config for fields with multiple distinct schemas. Different from nullable (

{"type": ["string", "null"]}

) which is same base type + null.

6. Invalid Test Examples

Symptom:

404 ERROR - Not found

400 Bad Request

Cause: Test example uses invalid/outdated IDs

Fix: Discover valid examples using the List → Get or Search → Details patterns below.

7. API Parameter Errors

Symptom:

400 Bad Request

or parameter validation errors

Fix: Update parameter schema in JSON config with correct types, required fields, and enums.

8. API Key Configuration Errors

Symptom: Tool not loading when API key is optional, or

api_key

parameter causing confusion

Cause: Using

required_api_keys

for keys that should be optional, or exposing API key as tool parameter

Key differences:

```
required_api_keys
```
: Tool is skipped if keys are missing
```
optional_api_keys
```
: Tool loads and works without keys (with reduced performance)

Fix: Use

optional_api_keys

in JSON config for APIs that work anonymously but have better rate limits with keys. Read API key from environment only (

os.environ.get()

), never as a tool parameter.

9. API Endpoint Pattern Errors

Symptom:

for valid resources, or unexpected results

Fix: Verify official API docs - check if values belong in URL path vs query parameters.

10. Transient API Failures

Symptom: Tests fail intermittently with timeout/connection/5xx errors

Fix: Use

pytest.skip()

for transient errors in unit tests - don't fail on external API outages.

Common Fix Patterns

Schema Validation Pattern

Schema validates the

data

field content, not the full response. Match

return_schema

type to what's inside

data

(array, object, or string).

Multi-Operation Tool Pattern

Every internal method must return

{"status": "...", "data": {...}}

. Don't use alternative field names at top level.

Finding Valid Test Examples

When test examples fail with 400/404, discover valid IDs by:

List → Get: Call a list endpoint first, extract ID from results
Search → Details: Search for a known entity, use returned ID
Iterate Versions: Try different dataset versions if supported

Unit Test Management

Check for Unit Tests

After fixing a tool, check if unit tests exist:

bash

ls tests/unit/test_<tool-name>_tool.py

When to Update Unit Tests

Update unit tests when you:

Change return structure: Update assertions checking
```
result["data"]
```
structure
Add/modify operations: Add test cases for new operations
Change error handling: Update error assertions
Modify required parameters: Update parameter validation tests
Fix schema issues: Ensure tests validate correct data structure
Add binary handling: Add tests for binary responses

Running Unit Tests

bash

# Run specific tool tests
pytest tests/unit/test_<tool-name>_tool.py -v

# Run all unit tests
pytest tests/unit/ -v

Unit Test Checklist

Check if
```
tests/unit/test_<tool-name>_tool.py
```
exists
Run unit tests before and after fix
Update assertions if data structure changed
Ensure both direct and interface tests pass

For detailed unit test patterns and examples, see unit-tests-reference.md.

Verification

Run Integration Tests

bash

python scripts/test_new_tools.py <pattern> -v

Run Unit Tests (if exist)

bash

pytest tests/unit/test_<tool-name>_tool.py -v

Regenerate Tools

After modifying JSON configs or tool classes:

bash

python -m tooluniverse.generate_tools

Regenerate after:

Changing
```
src/tooluniverse/data/*_tools.json
```
files
Modifying tool class implementations

Not needed for test script changes.

Output Format

After fixing, provide this summary:

Problem: [Brief description]

Root Cause: [Why it failed]

Solution: [What was changed]

Changes Made:

File 1: [Description]
File 2: [Description]
File 3 (if applicable): [Unit test updates]

Integration Test Results:

Before: X tests, Y passed (Z%), N failed, M schema invalid
After: X tests, X passed (100.0%), 0 failed, 0 schema invalid

Unit Test Results (if applicable):

Before: X tests, Y passed, Z failed
After: X tests, X passed, 0 failed

Testing Best Practices

Verify Parameter Names Before Testing

CRITICAL: Always read the tool's JSON config or generated wrapper to get the correct parameter names. Don't assume parameter names.

Example of incorrect testing:

python

# ❌ WRONG - assumed parameter name
AllenBrain_search_genes(query='Gad1')  # Fails: unexpected keyword 'query'

Correct approach:

python

# ✅ RIGHT - checked config first
# Config shows parameters: gene_acronym, gene_name
AllenBrain_search_genes(gene_acronym='Gad1')  # Works!

How to find correct parameter names:

Read the JSON config:
```
src/tooluniverse/data/*_tools.json
```
Check the generated wrapper:
```
src/tooluniverse/tools/<ToolName>.py
```
Look at test_examples in the JSON config

Systematic Testing Approach

When testing multiple tools:

Sample first: Test 1-2 tools per API to identify patterns
Categorize errors: Group by error type (param validation, API errors, data structure)
Fix systematically: Fix all tools with same issue type together
Regenerate once: Run
```
python -m tooluniverse.generate_tools
```
after all JSON changes
Verify all: Test all fixed tools comprehensively

Understanding Data Structure

Tools can return different data structures:

Object:
```
{"data": {"id": 1, "name": "..."}}
```
- single result
Array:
```
{"data": [{"id": 1}, {"id": 2}]}
```
- multiple results
String:
```
{"data": "description text"}
```
- text response

Test accordingly:

python

# For object data
result = tool()
data = result.get('data', {})
value = data.get('field_name')  # ✅

# For array data
result = tool()
items = result.get('data', [])
count = len(items)  # ✅
first = items[0] if items else {}  # ✅

Common Pitfalls

Schema validates
data
field, not full response
All methods need
{"status": "...", "data": {...}}
wrapper
JSON config changes require regeneration
Use
optional_api_keys
for APIs that work without keys
Check official API docs for correct endpoint patterns
Unit tests should skip on transient API failures, not fail
Mutually exclusive parameters MUST be nullable - most common new tool issue
Verify parameter names from configs - don't assume or guess
Test with correct data structure expectations - list vs dict vs string

Debugging

Inspect API response: Check status code, Content-Type header, and body preview
Check tool config: Load ToolUniverse and inspect the tool's configuration
Add debug prints: Log URL, params, status, and Content-Type in the run method

Quick Reference

Task	Command
Run integration tests	`python scripts/test_new_tools.py <pattern> -v`
Run unit tests	`pytest tests/unit/test_<tool-name>_tool.py -v`
Check if unit tests exist	`ls tests/unit/test_<tool-name>_tool.py`
Regenerate tools	`python -m tooluniverse.generate_tools`
Check status	`git status --short \| grep -E "(data\|tools\|.*_tool.py\|tests/unit)"`

Error Type	Fix Location
JSON parse error	`src/tooluniverse/*_tool.py` run() method
Schema mismatch	`src/tooluniverse/data/*_tools.json` return_schema
404 errors	`src/tooluniverse/data/*_tools.json` test_examples or endpoint
Parameter errors	`src/tooluniverse/data/*_tools.json` parameter schema
Unit test failures	`tests/unit/test_*_tool.py` assertions
Tool skipped (optional key)	`src/tooluniverse/data/*_tools.json` use `optional_api_keys`
API key as parameter	Remove from JSON params, use `os.environ.get()` in Python

devtu-fix-tool

NPX Install

Tags

SKILL.md Content

Fix ToolUniverse Tools

Instructions

Where to Fix

Error Types

1. JSON Parsing Errors

2. Schema Validation Errors

3. Nullable Field Errors

4. Mutually Exclusive Parameter Errors

5. Mixed Type Field Errors

6. Invalid Test Examples

7. API Parameter Errors

8. API Key Configuration Errors

9. API Endpoint Pattern Errors

10. Transient API Failures

Common Fix Patterns

Schema Validation Pattern

Multi-Operation Tool Pattern

Finding Valid Test Examples

Unit Test Management

Check for Unit Tests

When to Update Unit Tests

Running Unit Tests

Unit Test Checklist

Verification

Run Integration Tests

Run Unit Tests (if exist)

Regenerate Tools

Output Format

Testing Best Practices

Verify Parameter Names Before Testing

Systematic Testing Approach

Understanding Data Structure

Common Pitfalls

Debugging

Quick Reference