• PILOT_MAX_HOURS = 2 — Skip any pilot estimated to take > 2 hours per GPU. Flag as "needs manual pilot".
• PILOT_TIMEOUT_HOURS = 3 — Hard timeout: kill pilots exceeding 3 hours. Collect partial results if available.
• MAX_PILOT_IDEAS = 3 — Pilot at most 3 ideas in parallel. Additional ideas are validated on paper only.
• MAX_TOTAL_GPU_HOURS = 8 — Total GPU budget for all pilots combined.
• REVIEWER_MODEL =

  gpt-5.4

  — Model used via Codex MCP for brainstorming and review. Must be an OpenAI model (e.g.,

  gpt-5.4

  ,

  o3

  ,

  gpt-4o

  ).

💡 Override via argument, e.g.,

/idea-creator "topic" — pilot budget: 4h per idea, 20h total

.

1. Scan local paper library first: Check

   papers/

   and

   literature/

   in the project directory for existing PDFs. Read first 3 pages of relevant papers to build a baseline understanding before searching online. This avoids re-discovering what the user already knows.
2. Search recent literature using WebSearch:
   • Top venues in the last 2 years (NeurIPS, ICML, ICLR, ACL, EMNLP, etc.)
   • Recent arXiv preprints (last 6 months)
   • Use 5+ different query formulations
   • Read abstracts and introductions of the top 10-15 papers
3. Build a landscape map:
   • Group papers by sub-direction / approach
   • Identify what has been tried and what hasn't
   • Note recurring limitations mentioned in "Future Work" sections
   • Flag any open problems explicitly stated by multiple papers
4. Identify structural gaps:
   • Methods that work in domain A but haven't been tried in domain B
   • Contradictory findings between papers (opportunity for resolution)
   • Assumptions that everyone makes but nobody has tested
   • Scaling regimes that haven't been explored
   • Diagnostic questions that nobody has asked

1. Feasibility check: Can we actually run this experiment with available resources?
   • Compute requirements (estimate GPU-hours)
   • Data availability
   • Implementation complexity
   • Skip ideas requiring > 1 week of GPU time or unavailable datasets
2. Novelty quick-check: For each idea, do 2-3 targeted searches to see if it's already been done. Full

   /novelty-check

   comes later for survivors.
3. Impact estimation: Would a reviewer care about the result?
   • "So what?" test: if the experiment succeeds, does it change how people think?
   • Is the finding actionable or just interesting?

1. Novelty check: Use the

   /novelty-check

   workflow (multi-source search + GPT-5.4 cross-verification) for each idea
2. Critical review: Use GPT-5.4 via

   mcp__codex__codex-reply

   (same thread):

   Here are our top ideas after filtering:
   [paste surviving ideas with novelty check results]
   
   For each, play devil's advocate:
   - What's the strongest objection a reviewer would raise?
   - What's the most likely failure mode?
   - How would you rank these for a top venue submission?
   - Which 2-3 would you actually work on?

3. Combine rankings: Merge your assessment with GPT-5.4's ranking. Select top 2-3 ideas for pilot experiments.

1. Design pilots: For each top idea, define the minimal experiment that would give a positive or negative signal:
   • Single seed, small scale (e.g., small dataset subset, fewer epochs)
   • Target: 30 min - PILOT_MAX_HOURS per pilot on 1 GPU
   • Estimate GPU-hours BEFORE launching. If estimated time > PILOT_MAX_HOURS, reduce scale (fewer epochs, smaller subset) or flag as "needs manual pilot"
   • Clear success metric defined upfront (e.g., "if metric improves by > 1%, signal is positive")
2. Deploy in parallel: Use

   /run-experiment

   to launch pilots on different GPUs simultaneously:

   GPU 0: Pilot for Idea 1
   GPU 1: Pilot for Idea 2
   GPU 2: Pilot for Idea 3

   Use

   run_in_background: true

   to launch all at once.
3. Collect results: Use

   /monitor-experiment

   to check progress. If any pilot exceeds PILOT_TIMEOUT_HOURS, kill it and collect partial results. Once all pilots complete (or timeout), compare:
   • Which ideas showed positive signal?
   • Which showed null/negative results? (eliminate or deprioritize)
   • Any surprising findings that suggest a pivot?
   • Total GPU-hours consumed (track against MAX_TOTAL_GPU_HOURS budget)
4. Re-rank based on empirical evidence: Update the idea ranking using pilot results. An idea with strong pilot signal jumps ahead of a theoretically appealing but untested idea.

• Hypothesis: [one sentence]
• Minimum experiment: [concrete description]
• Expected outcome: [what success/failure looks like]
• Novelty: X/10 — closest work: [paper]
• Feasibility: [compute, data, implementation estimates]
• Risk: LOW/MEDIUM/HIGH
• Contribution type: empirical / method / theory / diagnostic
• Pilot result: [POSITIVE: metric +X% / NEGATIVE: no signal / SKIPPED: needs GPU]
• Reviewer's likely objection: [strongest counterargument]
• Why we should do this: [1-2 sentences]

1. Start with Idea 1 (positive pilot signal, lowest risk)
2. Idea 3 as backup (weak signal, may need larger scale to confirm)
3. Idea 2 eliminated by pilot — negative result documented

• Scale up Idea 1 to full experiment (multi-seed, full dataset)
• If confirmed, invoke /auto-review-loop for full iteration

undefined

• The user provides a DIRECTION, not an idea. Your job is to generate the ideas.
• Quantity first, quality second: brainstorm broadly, then filter ruthlessly.
• A good negative result is just as publishable as a positive one. Prioritize ideas where the answer matters regardless of direction.
• Don't fall in love with any idea before validating it. Be willing to kill ideas.
• Always estimate compute cost. An idea that needs 1000 GPU-hours is not actionable for most researchers.
• "Apply X to Y" is the lowest form of research idea. Push for deeper questions.
• Include eliminated ideas in the report — they save future time by documenting dead ends.
• If the user's direction is too broad, ask them to narrow it before proceeding.