Loading...
Loading...
Found 10 Skills
Serves AI models on AMD Instinct GPU hardware using vLLM. Use this skill whenever the user wants to run, serve, deploy, start, host, or launch a language model on an AMD GPU, AMD Instinct, MI300X, MI325X, MI350X, or MI355X. Also use when the user mentions vLLM on ROCm, vLLM on AMD, serving on HBM, or asks how to get a model running on AMD data center hardware. Use when the user asks "run Qwen3", "serve DeepSeek", "start a vLLM endpoint", "get a model running on my AMD machine", or any similar phrasing. Handles the full flow: GPU detection, environment validation, vLLM configuration, launch, and health verification. Do not use for NVIDIA GPUs, consumer AMD GPUs (RX series, Radeon), Ryzen AI, NPU, MI250X, or MI100.
Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.
Mistral AI efficient open models. Use for efficient AI.
Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.
Meta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails.
Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.
Pick Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes; maps Orin SM 8.7 vs Thor SM 11.0 and JetPack-specific package choices.
Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.
Deploy any HuggingFace model to RunPod as a vLLM server, bridge it to Claude Code (which speaks Anthropic's Messages API, not OpenAI's), smoke-test that tool-calling actually works end to end, and tear everything down with zero trace (pod, network volumes, local processes, scratch files). Use this whenever you want to run a model on RunPod, try a model with Claude Code, test whether a HF/GGUF/abliterated checkpoint supports tool use, benchmark a model for coding-agent use, or spin up/kill a disposable GPU inference box. Also trigger it for anything about hooking Claude Code up to a non-Anthropic backend (vLLM, Ollama, LM Studio) via ANTHROPIC_BASE_URL, or questions like 'what GPU do I need to run X' and 'how much will it cost to run this model'.
Serve a Hugging Face model with vLLM on Runpod (Hub worker-vllm queue endpoint, load-balancing endpoint on a vllm/vllm-openai image, or a Pod), sized, CUDA-pinned and verified with a real OpenAI client. Use for any "deploy/serve/test this LLM with vLLM" request or a vLLM deployment that will not boot; defer to runpod-serverless-deploy for non-vLLM engines and runpod-endpoint-ops for an endpoint that already works.