lightning-deployments
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseLightning AI Deployments
Lightning AI 部署
A Deployment runs a container (or an auto-built vLLM server for a HuggingFace model) behind an HTTPS endpoint with autoscaling. Sources are mutually exclusive: (Docker), (HuggingFace id, GPU required), or (deploy a Studio's environment).
--image--model--studio部署(Deployment)会在支持自动扩缩容的 HTTPS 端点后运行一个容器(或为 HuggingFace 模型自动构建的 vLLM 服务器)。部署源是互斥的:(Docker 镜像)、(HuggingFace 模型 ID,需 GPU)或 (部署 Studio 环境)。
--image--model--studioSetup & auth
安装与认证
bash
undefinedbash
undefinedUse the Lightning AI CLI from the current env; install or upgrade it there if it's missing or older than 2026.9.18
Use the Lightning AI CLI from the current env; install or upgrade it there if it's missing or older than 2026.9.18
v=$(lightning --version 2>/dev/null | sed -n 's/^Lightning CLI version //p')
[ -n "$v" ] && [ "$(printf '%s\n' 2026.9.18 "$v" | sort -V | head -1)" = 2026.9.18 ]
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token
This uses, and if needed installs into, the environment the agent runs in (the user's project
venv, a conda env, …). Then call plain `lightning …` everywhere. If setup doesn't go cleanly:
| Symptom | Fix |
|---|---|
| Both installs fail: no active env, or pip refuses with `externally-managed-environment` | Install it on its own instead: `uv tool install lightning-sdk` (or `pipx install lightning-sdk`), then call `"$(uv tool dir --bin)/lightning"` if that dir isn't on `PATH` |
| `lightning --version` still prints no `Lightning CLI version` line after installing | `command -v lightning` shows which one runs. Another tool owns the name (PyTorch Lightning also installs a `lightning` command), or the env you installed into isn't on `PATH`: activate it (`source .venv/bin/activate`) or call its `bin/lightning` by full path |
| `No such command`, `No such option` or `unexpected extra argument` | The CLI is older than this skill expects: `uv pip install -U lightning-sdk` (or `pip install -U lightning-sdk`) in the env `command -v lightning` points into; `uv tool upgrade lightning-sdk` for a uv tool install |
| The upgrade fails on a version conflict with the project's own pins | Don't fight the pins: install it outside the project with `uv tool install lightning-sdk` and call `"$(uv tool dir --bin)/lightning"` |
| Installing is blocked (read-only env or home, agent sandbox) | Skip it and run each command as `UV_CACHE_DIR="${TMPDIR:-/tmp}/uv" uvx lightning-sdk …` |
| Every call fails on SSL/certificates, even `lightning --version` (`Could not find a suitable TLS CA certificate bundle`), only inside an agent sandbox | A `**/*.pem` read-deny rule is hiding certifi's public CA bundle (`site-packages/certifi/cacert.pem`). Allow that one path; don't disable the sandbox |
The install above also makes `lightning_sdk` importable in that env, so Python snippets run with
its `python`. If the CLI came from a uv tool or `uvx` fallback instead, run one-off scripts with
`uv run --with lightning-sdk python script.py`. For code that stays in the user's project, ask,
then declare `lightning-sdk` as a dependency with the project's own tool (`uv add`, `poetry add`).v=$(lightning --version 2>/dev/null | sed -n 's/^Lightning CLI version //p')
[ -n "$v" ] && [ "$(printf '%s\n' 2026.9.18 "$v" | sort -V | head -1)" = 2026.9.18 ]
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token
此操作会使用 Agent 运行所在的环境(用户的项目 venv、conda 环境等),如有需要也会安装到该环境中。之后所有地方都直接调用 `lightning …` 即可。如果安装过程不顺利,请参考下表:
| 问题现象 | 解决方法 |
|---|---|
| 两种安装方式都失败:没有激活的环境,或 pip 报错 `externally-managed-environment` | 改为独立安装:`uv tool install lightning-sdk`(或 `pipx install lightning-sdk`),如果该目录不在 `PATH` 中,则调用 `"$(uv tool dir --bin)/lightning"` |
| 安装后 `lightning --version` 仍然没有输出 `Lightning CLI version` 行 | 用 `command -v lightning` 查看当前运行的是哪个程序。可能是其他工具占用了这个名称(PyTorch Lightning 也会安装 `lightning` 命令),或者你安装的环境不在 `PATH` 中:激活该环境(`source .venv/bin/activate`)或通过完整路径调用其 `bin/lightning` |
| `No such command`、`No such option` 或 `unexpected extra argument` | CLI 版本低于本技能要求:在 `command -v lightning` 指向的环境中执行 `uv pip install -U lightning-sdk`(或 `pip install -U lightning-sdk`);如果是 uv tool 安装的,则执行 `uv tool upgrade lightning-sdk` |
| 升级因与项目自身的版本锁定冲突而失败 | 不要强行修改版本锁定:用 `uv tool install lightning-sdk` 在项目外安装,并调用 `"$(uv tool dir --bin)/lightning"` |
| 安装被阻止(只读环境或家目录、Agent 沙箱) | 跳过安装,将每条命令改为 `UV_CACHE_DIR="${TMPDIR:-/tmp}/uv" uvx lightning-sdk …` 运行 |
| 所有调用都因 SSL/证书问题失败,即使是 `lightning --version`(报错 `Could not find a suitable TLS CA certificate bundle`),且仅在 Agent 沙箱中出现 | 可能是 `**/*.pem` 读取拒绝规则隐藏了 certifi 的公共 CA 包(`site-packages/certifi/cacert.pem`)。请允许该路径的访问权限,不要禁用沙箱 |
上述安装还会让 `lightning_sdk` 在该环境中可导入,因此 Python 代码片段可以用该环境的 `python` 运行。如果 CLI 来自 uv tool 或 `uvx` 备用方案,则使用 `uv run --with lightning-sdk python script.py` 运行一次性脚本。对于要保留在用户项目中的代码,请先询问用户,然后用项目自身的工具(`uv add`、`poetry add`)将 `lightning-sdk` 声明为依赖项。Resolving org and teamspace (do this first)
确定组织与工作区(请先执行此步骤)
Deployments live in a teamspace owned by an organization or a user. Never guess. Use explicit (Python: ; the separate / arguments are deprecated), env vars / , or the config default (). If none is set, list the options and ask the user which org/teamspace to use:
--teamspace owner/teamspaceteamspace="owner/teamspace"org=user=LIGHTNING_ORGLIGHTNING_TEAMSPACElightning config get teamspacebash
lightning api /v1/memberships | jq -r '.memberships[] | [.ownerType, .name, .projectId] | @tsv'Persist the choice: .
lightning config set teamspace <owner>/<teamspace>From a scoped API key (an agent, no user to ask): gives the teamspace name and the owner id, but needs the owner slug — resolve it via (needs ). Select the entry rather than : the same teamspace is commonly listed twice, once with and once with (identical and ), so index 0 is a coin flip and the row's will not resolve against .
/v1/memberships--teamspace/v1/orgsjqorganization.memberships[0]ownerType: organizationownerType: userprojectIdnameuserownerId/v1/orgsbash
M=$(lightning api /v1/memberships)
ROW=$(echo "$M" | jq -c '[.memberships[] | select(.ownerType=="organization")][0] // .memberships[0]')
TS=$(echo "$ROW" | jq -r .name) # teamspace
OWNER=$(lightning api "/v1/orgs/$(echo "$ROW" | jq -r .ownerId)" | jq -r .name) # owner (org) slug
lightning config set teamspace "$OWNER/$TS" # every command now defaults here; or pass --teamspace "$OWNER/$TS"部署存放在由组织或用户拥有的工作区(teamspace)中。绝对不要猜测。 请使用显式的 (Python 中为 ;单独的 / 参数已弃用)、环境变量 / ,或配置默认值()。如果都没有设置,请列出可选选项并询问用户要使用哪个组织/工作区:
--teamspace owner/teamspaceteamspace="owner/teamspace"org=user=LIGHTNING_ORGLIGHTNING_TEAMSPACElightning config get teamspacebash
lightning api /v1/memberships | jq -r '.memberships[] | [.ownerType, .name, .projectId] | @tsv'保存选择:。
lightning config set teamspace <owner>/<teamspace>使用受限 API 密钥时(Agent 场景,无法询问用户): 会返回工作区名称和所有者 ID,但 需要所有者 slug——需通过 解析(需要 )。请选择 条目而不是 :同一个工作区通常会出现两次,一次是 ,一次是 ( 和 相同),因此直接取第0个是随机的,且 行的 无法通过 解析。
/v1/memberships--teamspace/v1/orgsjqorganization.memberships[0]ownerType: organizationownerType: userprojectIdnameuserownerId/v1/orgsbash
M=$(lightning api /v1/memberships)
ROW=$(echo "$M" | jq -c '[.memberships[] | select(.ownerType=="organization")][0] // .memberships[0]')
TS=$(echo "$ROW" | jq -r .name) # teamspace
OWNER=$(lightning api "/v1/orgs/$(echo "$ROW" | jq -r .ownerId)" | jq -r .name) # owner (org) slug
lightning config set teamspace "$OWNER/$TS" # every command now defaults here; or pass --teamspace "$OWNER/$TS"CLI reference
CLI 参考
Subcommands: , , , , , (+ ), , and / (// on a running deployment). There is no — stopping = scaling to zero via .
createlistinspectupdatedeletelogslogs downloadreload-weightsenvsecretlistsetdeletestopupdate --min-replicas 0 --max-replicas 0bash
undefined子命令包括:、、、、、(含 )、,以及 / (对运行中的部署执行 // 操作)。没有 命令——停止部署等于通过 将副本数缩容到0。
createlistinspectupdatedeletelogslogs downloadreload-weightsenvsecretlistsetdeletestopupdate --min-replicas 0 --max-replicas 0bash
undefinedcontainer deployment (a --port is required for non-model deployments)
container deployment (a --port is required for non-model deployments)
lightning deployment create my-api --teamspace owner/teamspace
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth
lightning deployment create my-api --teamspace owner/teamspace
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth
HuggingFace model via vLLM (GPU machine required; port defaults to 8000)
HuggingFace model via vLLM (GPU machine required; port defaults to 8000)
lightning deployment create llama --teamspace owner/teamspace
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]
lightning deployment create llama --teamspace owner/teamspace
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]
cloud account — if you pass none, the SDK picks one (config cloud-account
, then
cloud-accountcloud account — if you pass none, the SDK picks one (config cloud-account
, then
cloud-accountLIGHTNING_CLUSTER_ID, then the teamspace's default binding) and prints which on stderr.
LIGHTNING_CLUSTER_ID, then the teamspace's default binding) and prints which on stderr.
Other deployments in the same teamspace may sit on a different account, which
Other deployments in the same teamspace may sit on a different account, which
deployment list
shows in its last column; match it with --cloud when relevant.
deployment listdeployment list
shows in its last column; match it with --cloud when relevant.
deployment listlightning deployment create ... --cloud gcp-lightning-public-prod
lightning deployment create ... --cloud gcp-lightning-public-prod
autoscaling / env / secrets
autoscaling / env / secrets
lightning deployment create ...
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas
lightning deployment create ...
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas
endpoint auth — mutually exclusive; OMITTING ALL THREE MAKES THE ENDPOINT PUBLIC
endpoint auth — mutually exclusive; OMITTING ALL THREE MAKES THE ENDPOINT PUBLIC
Run lightning auth whoami
first: --api-key-auth only accepts user keys.
lightning auth whoamiRun lightning auth whoami
first: --api-key-auth only accepts user keys.
lightning auth whoami--api-key-auth # require a Lightning USER API key (Bearer) — see Gotchas
--basic-auth USER:PASS
--token-auth TOKEN # any bearer string you choose; use this from a scoped key
--api-key-auth # require a Lightning USER API key (Bearer) — see Gotchas
--basic-auth USER:PASS
--token-auth TOKEN # any bearer string you choose; use this from a scoped key
operate
operate
lightning deployment list --teamspace owner/teamspace [--all] [--sort-by state]
lightning deployment inspect my-api --teamspace owner/teamspace # full JSON incl. endpoint URLs
lightning deployment logs my-api --teamspace owner/teamspace [-f] [--tail 100] [--timestamps]
lightning deployment logs my-api --teamspace owner/teamspace --query timeout --severity error # filter server-side
lightning deployment logs my-api --teamspace owner/teamspace --since 30m --until 5m # window (replicas merged + labelled)
lightning deployment update my-api --teamspace owner/teamspace --max-replicas 8
lightning deployment update my-api --teamspace owner/teamspace --min-replicas 0 --max-replicas 0 # "stop"
lightning deployment update my-api --teamspace owner/teamspace --image myimg:v2 [--max-surge 1] [--max-unavailable 0] # new release, rolling update
lightning deployment reload-weights llama --teamspace owner/teamspace # hot-reload BYOM weights
lightning deployment delete my-api --teamspace owner/teamspace --yes
undefinedlightning deployment list --teamspace owner/teamspace [--all] [--sort-by state]
lightning deployment inspect my-api --teamspace owner/teamspace # full JSON incl. endpoint URLs
lightning deployment logs my-api --teamspace owner/teamspace [-f] [--tail 100] [--timestamps]
lightning deployment logs my-api --teamspace owner/teamspace --query timeout --severity error # filter server-side
lightning deployment logs my-api --teamspace owner/teamspace --since 30m --until 5m # window (replicas merged + labelled)
lightning deployment update my-api --teamspace owner/teamspace --max-replicas 8
lightning deployment update my-api --teamspace owner/teamspace --min-replicas 0 --max-replicas 0 # "stop"
lightning deployment update my-api --teamspace owner/teamspace --image myimg:v2 [--max-surge 1] [--max-unavailable 0] # new release, rolling update
lightning deployment reload-weights llama --teamspace owner/teamspace # hot-reload BYOM weights
lightning deployment delete my-api --teamspace owner/teamspace --yes
undefinedPython SDK
Python SDK
python
from lightning_sdk import Deployment, Machine
from lightning_sdk.api.deployment_api import (
ApiKeyAuth, BasicAuth, TokenAuth, AutoScaleConfig, Env, Secret,
RollingUpdateReleaseStrategy, HttpHealthCheck,
)
dep = Deployment("my-api", teamspace="my-org/my-teamspace") # owner/teamspace, same as the CLI; org=/user= are deprecated
dep.start(
image="nginx:latest",
machine=Machine.CPU,
ports=[80], # REQUIRED for non-model deployments; a LIST, not an int
autoscale=AutoScaleConfig(min_replicas=0, max_replicas=4, metric="CPU", threshold=90),
env=[Env("MODE", "prod"), Secret("HF_TOKEN")], # or a plain dict for env vars
auth=ApiKeyAuth(), # None => PUBLIC endpoint
health_check=HttpHealthCheck(path="/health", port=80),
spot=False, # True = interruptible replicas
)
print(dep.urls) # endpoint URL(s); no .status property — use replica counts
print(dep.running_replicas, dep.pending_replicas, dep.failing_replicas)
r = dep.get(path="/health") # convenience HTTP; auto-auths only for ApiKeyAuthpython
from lightning_sdk import Deployment, Machine
from lightning_sdk.api.deployment_api import (
ApiKeyAuth, BasicAuth, TokenAuth, AutoScaleConfig, Env, Secret,
RollingUpdateReleaseStrategy, HttpHealthCheck,
)
dep = Deployment("my-api", teamspace="my-org/my-teamspace") # owner/teamspace, same as the CLI; org=/user= are deprecated
dep.start(
image="nginx:latest",
machine=Machine.CPU,
ports=[80], # REQUIRED for non-model deployments; a LIST, not an int
autoscale=AutoScaleConfig(min_replicas=0, max_replicas=4, metric="CPU", threshold=90),
env=[Env("MODE", "prod"), Secret("HF_TOKEN")], # or a plain dict for env vars
auth=ApiKeyAuth(), # None => PUBLIC endpoint
health_check=HttpHealthCheck(path="/health", port=80),
spot=False, # True = interruptible replicas
)
print(dep.urls) # endpoint URL(s); no .status property — use replica counts
print(dep.running_replicas, dep.pending_replicas, dep.failing_replicas)
r = dep.get(path="/health") # convenience HTTP; auto-auths only for ApiKeyAuthchanging image/command/env/entrypoint/spot/... creates a NEW RELEASE and needs a release strategy
changing image/command/env/entrypoint/spot/... creates a NEW RELEASE and needs a release strategy
dep.update(image="nginx:1.27", release_strategy=RollingUpdateReleaseStrategy(max_surge=1, max_unavailable=0))
dep.update(min_replicas=1, max_replicas=8) # scaling: no new release, no strategy needed
dep.stop() # scales to 0; blocks until replicas reach 0
HuggingFace model serving via SDK: `dep.start(model="meta-llama/...", machine=Machine.L40S, ports=[8000], hf_token_secret="...")` — `image`/`studio` must be None; still pass `ports`.dep.update(image="nginx:1.27", release_strategy=RollingUpdateReleaseStrategy(max_surge=1, max_unavailable=0))
dep.update(min_replicas=1, max_replicas=8) # scaling: no new release, no strategy needed
dep.stop() # scales to 0; blocks until replicas reach 0
通过 SDK 部署 HuggingFace 模型服务:`dep.start(model="meta-llama/...", machine=Machine.L40S, ports=[8000], hf_token_secret="...")` —— `image`/`studio` 必须为 None;仍需传入 `ports` 参数。Private container images
私有容器镜像
A deployment whose image needs registry credentials must set
to the name of a
secret in the teamspace. Neither the CLI nor exposes
this, and the server rejects an unpullable image with a bare
— — which names neither the image nor
the missing credential. Until the SDK exposes it, patch the field onto the
create request:
V1JobSpec.image_secret_refSECRET_TYPE_DOCKER_REGISTRYDeployment.start()500Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 500.python
from lightning_sdk.lightning_cloud.openapi.api import jobs_service_api
_create = jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info
def _with_image_secret(self, body, project_id, **kwargs):
if getattr(body, "spec", None) is not None:
body.spec.image_secret_ref = "MY_REGISTRY_SECRET"
return _create(self, body, project_id, **kwargs)
jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info = _with_image_secretList the teamspace's secrets, and their types, with:
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId')
lightning api "/v1/projects/${PROJECT_ID}/secrets" | jq -r '.secrets[] | "\(.name) \(.type)"'Diagnosing this 500 is much faster if you bisect against a public image
first (): if that create succeeds with otherwise identical
arguments, the image is the variable, not the machine, port, replica counts or
cloud account.
nginx:latestTrap: does NOT delete the deployment — it is an HTTP helper like / and sends an HTTP request to the deployed service's endpoint. To delete the deployment resource use or the raw API.
dep.delete()dep.get()dep.post()DELETElightning deployment delete NAME --yes如果部署的镜像需要仓库凭证,必须将 设置为工作区中类型为 的密钥名称。CLI 和 都没有暴露该参数,服务器无法拉取镜像时只会返回一个简单的 错误————既不会说明是哪个镜像,也不会说缺少凭证。在 SDK 支持该参数之前,可以通过打补丁的方式在创建请求中添加该字段:
V1JobSpec.image_secret_refSECRET_TYPE_DOCKER_REGISTRYDeployment.start()500Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 500.python
from lightning_sdk.lightning_cloud.openapi.api import jobs_service_api
_create = jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info
def _with_image_secret(self, body, project_id, **kwargs):
if getattr(body, "spec", None) is not None:
body.spec.image_secret_ref = "MY_REGISTRY_SECRET"
return _create(self, body, project_id, **kwargs)
jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info = _with_image_secret使用以下命令列出工作区的密钥及其类型:
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId')
lightning api "/v1/projects/${PROJECT_ID}/secrets" | jq -r '.secrets[] | "\(.name) \(.type)"'如果先使用公共镜像()进行对比排查,诊断这个500错误会快得多:如果其他参数完全相同的情况下公共镜像部署成功,那么问题出在镜像上,而不是机器、端口、副本数或云账号。
nginx:latest陷阱: 不会删除部署资源——它是和 / 类似的 HTTP 辅助方法,会向部署的服务端点发送 HTTP 请求。要删除部署资源,请使用 或原始 API。
dep.delete()dep.get()dep.post()DELETElightning deployment delete NAME --yesExample workflows
示例工作流
Prompts this skill handles: "deploy this docker image behind an API", "serve Llama 3.1 8B on lightning", "scale my deployment down at night", "roll out the new image version".
Deploy a container, verify it responds, then clean up:
bash
lightning auth whoami # `user` → --api-key-auth is callable by you; `scoped-api-key` → use --token-auth
lightning deployment create hello-api --teamspace my-org/my-teamspace \
--image nginx:latest --machine CPU --port 80 --min-replicas 0 --max-replicas 1 --api-key-auth
lightning deployment inspect hello-api --teamspace my-org/my-teamspace # JSON: status, endpoint URL
URL=<endpoint URL from the inspect output above>
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LIGHTNING_API_KEY" "$URL/" # verify the ENDPOINT, not the Studiopython
from lightning_sdk import Deployment
dep = Deployment("hello-api", teamspace="my-org/my-teamspace")
print(dep.urls) # e.g. ['https://80-dep-<id>-d.cloudspaces.litng.ai'] — available even while scaled to zero
print(dep.get(path="/").status_code) # sends the caller's Lightning API key for ApiKeyAuthbash
lightning deployment delete hello-api --teamspace my-org/my-teamspace --yesServe an open-weights LLM (vLLM, built server-side — no Dockerfile needed):
bash
lightning deployment create qwen-served --teamspace my-org/my-teamspace \
--model Qwen/Qwen2.5-7B-Instruct --machine L40S --min-replicas 0 --max-replicas 1 --api-key-auth --dry-run本技能处理的提示词示例:"把这个docker镜像部署成API"、"在lightning上部署Llama 3.1 8B服务"、"晚上把我的部署缩容"、"发布新的镜像版本"。
部署容器、验证响应,然后清理:
bash
lightning auth whoami # `user` → --api-key-auth is callable by you; `scoped-api-key` → use --token-auth
lightning deployment create hello-api --teamspace my-org/my-teamspace \
--image nginx:latest --machine CPU --port 80 --min-replicas 0 --max-replicas 1 --api-key-auth
lightning deployment inspect hello-api --teamspace my-org/my-teamspace # JSON: status, endpoint URL
URL=<endpoint URL from the inspect output above>
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LIGHTNING_API_KEY" "$URL/" # verify the ENDPOINT, not the Studiopython
from lightning_sdk import Deployment
dep = Deployment("hello-api", teamspace="my-org/my-teamspace")
print(dep.urls) # e.g. ['https://80-dep-<id>-d.cloudspaces.litng.ai'] — available even while scaled to zero
print(dep.get(path="/").status_code) # sends the caller's Lightning API key for ApiKeyAuthbash
lightning deployment delete hello-api --teamspace my-org/my-teamspace --yes部署开源大语言模型服务(vLLM,服务端构建——无需 Dockerfile):
bash
lightning deployment create qwen-served --teamspace my-org/my-teamspace \
--model Qwen/Qwen2.5-7B-Instruct --machine L40S --min-replicas 0 --max-replicas 1 --api-key-auth --dry-runreview the model spec it prints, then re-run without --dry-run; if the real create reports
review the model spec it prints, then re-run without --dry-run; if the real create reports
unacknowledged warnings, re-run with --ack <code> (repeatable) or --force
unacknowledged warnings, re-run with --ack <code> (repeatable) or --force
lightning deployment logs qwen-served --teamspace my-org/my-teamspace -f # watch it come up
The endpoint serves an OpenAI-compatible API on port 8000 — point any OpenAI client's `base_url` at `dep.urls[0]`.
**Operate: scale, update, roll back traffic costs:**
```bash
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 1 --max-replicas 4 # keep warm
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 0 --max-replicas 0 # stop (scale to zero)
lightning deployment update hello-api --teamspace my-org/my-teamspace --image nginx:1.27 # new release, rolling by defaultlightning deployment logs qwen-served --teamspace my-org/my-teamspace -f # watch it come up
该端点在8000端口提供兼容 OpenAI 的 API——将任何 OpenAI 客户端的 `base_url` 指向 `dep.urls[0]` 即可。
**运维操作:扩缩容、更新、回滚与流量成本:**
```bash
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 1 --max-replicas 4 # keep warm
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 0 --max-replicas 0 # stop (scale to zero)
lightning deployment update hello-api --teamspace my-org/my-teamspace --image nginx:1.27 # new release, rolling by defaultRaw API fallback
原始API备用方案
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId' | head -1)
lightning api "/v1/projects/${PROJECT_ID}/deployments" -q '.deployments[].name' # plain GET; -f/-F without -X would send a POST
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}"
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}" -X DELETEbash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId' | head -1)
lightning api "/v1/projects/${PROJECT_ID}/deployments" -q '.deployments[].name' # plain GET; -f/-F without -X would send a POST
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}"
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}" -X DELETEGotchas
常见陷阱
- Deployments bill for as long as a replica is up; confirm the spend with the user before creating one. Get an explicit go-ahead before expensive GPU machines (A100/H100/H200/B200), before (which bills continuously, with no scale-to-zero), and before raising
--min-replicas >= 1, since the ceiling is what the bill can reach. Price the machine first — see the--max-replicasskill.lightning-cost-estimation - Omitting all auth flags creates a publicly reachable endpoint — confirm that's intended.
- Check before picking an auth flag.
lightning auth whoamigates the endpoint on a Lightning user key (it serializes to--api-key-auth), so anuserApiKey: truecaller gets 401 on every request to an endpoint that is otherwise healthy — replicas running, seconds billed. UseAuth type: scoped-api-keywhen a scoped key, CI job or agent is what will call the endpoint;--token-auth <token>when humans with their own Lightning logins will. Verified against a control: a second, pre-existing healthy deployment returned an identical 401 to the same scoped key.--api-key-auth - Verify a deployment against its public URL from — never against the app running inside the source Studio. The Studio proves the image; only the endpoint proves the service. A deployment can show a healthy replica and bill normally while 401-ing every request.
deployment inspect - is not reliably cheaper — check the price before you pass it. On GCP the L4 is $0.48/hr on-demand but $0.727/hr spot: interruptible costs 51% more and you take preemption risk for the privilege. Nothing warns you at create time. Fetch both rates from the accelerator catalog first (see the
--interruptibleskill) and takelightning-cost-estimation; spot is only reliably cheaper on some SKUs and clouds.min(cost, spotPrice) - enables scale-to-zero and genuinely stops billing at zero — a teamspace credit balance stays flat while a deployment sits at zero replicas. The next request is served transparently: the edge accepts the connection and holds it open until a replica is ready (~6 minutes on a GPU image) and then returns a normal 200 — it does not return 503 or refuse, so any client without a short timeout just waits.
min_replicas=0bills continuously — flag this cost to the user.min_replicas >= 1 - The idle window before scale-to-zero is SDK-only. has no equivalent flag on
AutoScaleConfig(idle_threshold_seconds=...)ordeployment create, so from the CLI you get the default and cannot tune how long a replica lingers.deployment update - A health check is SDK-only too. exists in Python but there is no
HttpHealthCheck(path=..., port=...)/--health-check-pathflag, so a CLI-created deployment has nothing stopping traffic reaching a replica that is up but not ready.--readiness-probe - Changing image, command, env, entrypoint, path mappings, health check, cloud account, spot, quantity or max runtime forces a new release (switching the machine alone does not). The SDK raises if the deployment has no release strategy, neither passed nor already stored; the CLI auto-adds a rolling update.
RuntimeError - on an existing deployment silently becomes an update/restart — it won't error on a name collision.
Deployment.start() - A port is mandatory for non-model deployments (otherwise);
ValueErrordefaults to 8000 and requires a GPU machine. In the SDK--modelmust be a list — passing an int fails withportsraised from inside the SDK, which does not mentionTypeError: 'int' object is not iterable.ports - accepts no
AutoScaleConfigat construction, thenmetricraisesstart(). Always passValueError: The autoscaling metric is required. Currently supported metrics are ['GPU', 'CPU', 'RPM']; the traceback points atmetric=, not at the config object.start() - A private image needs , which no CLI flag or SDK argument sets — see Private container images. The failure is an opaque
image_secret_ref.500 - serializes with an empty
Secret("NAME")field, sonameshowsdeployment inspect. That is not a bug: the platform resolves the reference and names the environment variable after the secret. Verified by reading{"from_secret": "NAME", "name": "", "value": ""}inside a running replica.env - Deployment logs are readable by anyone with teamspace access. A container that echoes its environment on boot will therefore leak secret values into — do not debug secret injection that way on a real secret.
deployment logs - Deleting is destructive and the CLI prompts unless ; confirm with the user first.
--yesis not "gone" — the deployment keeps appearing inDeployment deleted(anddeployment list) for 90 seconds to ~2.5 minutes afterwards, sometimes inlist --allwith a stale replica count andPENDING. Don't treat a post-delete listing as a failed delete, and don't polldeletedAt: Noneto confirm teardown; re-issuelistonly if it is still there after a few minutes.delete - without
deployment logscan hang indefinitely. A bounded read (-f, no--tail 25) has been observed producing no output at all and never returning, so a scripted log fetch needs its own timeout.--followworks.-f - There is no cost surface for a deployment. exposes
deployment inspect, but it readstotal_costeven after0.034 minutes of L4 GPU time ($0.40 of real spend), and stays 0 for the deployment's whole life — unlike jobs, whosedoes populate. No billing/usage endpoint resolves either. The only way to see what a deployment cost is to difference the teamspace credit balance before and after, and that field flips between full float precision and 2-decimal rounding between consecutive calls — so difference over a window long enough that the rounding is noise.total_cost - deployments may return validation warnings on create (
--model).Deployment has unacknowledged warnings: ... Re-run with --ack <code> (repeatable) or --force.never calls the server, so it never shows those warnings: it only echoes the model spec you set (--dry-run,served_model_nameand any vLLM flags you passed), not the machine, image variant or replica config.weight_source - may be gated on your account, and the refusal is an opaque
--model.403can fail withdeployment create ... --model <hf-id>— no server message, andException: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 403.adds a traceback but still no reason. It is an entitlement, not a bad argument, so don't debug the model id or flags. Fall back to deploying a vLLM container image directly (LIGHTNING_DEBUG=1), which needs no entitlement.--image - In the Python SDK, takes
teamspace=like the CLI. The separate"owner/teamspace"/org=arguments are deprecated (user=), and passing both forms raisesDeprecationWarning. On SDKs older than 2026.7.31, the combined form failed with "Teamspace owner/name does not exist"; upgrade rather than splitting it.ValueError
- 只要有副本在运行,部署就会产生费用;创建部署前请与用户确认费用支出。 在使用昂贵的 GPU 机器(A100/H100/H200/B200)、设置 (会持续计费,无法缩容到0)以及提高
--min-replicas >= 1之前,都需要获得用户明确同意,因为上限就是账单可能达到的金额。请先查询机器定价——参考--max-replicas技能。lightning-cost-estimation - 省略所有认证标志会创建一个公开可访问的端点——请确认这是预期行为。
- 选择认证标志前请先执行 。
lightning auth whoami会使用 Lightning 用户 密钥验证端点(序列化为--api-key-auth),因此调用方类型为userApiKey: true时,即使端点本身正常(副本运行中、正常计费),所有请求也都会返回 401。如果调用方是受限密钥、CI 任务或 Agent,请使用Auth type: scoped-api-key;如果是使用个人 Lightning 账号登录的用户,则使用--token-auth <token>。已通过对照组验证:另一个预先存在的健康部署对同一个受限密钥也返回了相同的401错误。--api-key-auth - 请通过 返回的公共 URL 验证部署,绝不要通过源 Studio 中运行的应用验证。 Studio 只能证明镜像可用;只有端点才能证明服务正常。部署可能显示副本健康且正常计费,但所有请求都返回401。
deployment inspect - 不一定更便宜——使用前请先查询价格。 在 GCP 上,L4 实例的按需价格是 $0.48/小时,而竞价型(spot)价格是 $0.727/小时:可中断实例反而贵51%,还要承担被抢占的风险。创建时不会有任何警告。请先从加速器目录中获取两种价格(参考
--interruptible技能),取lightning-cost-estimation;竞价实例仅在部分 SKU 和云平台上确实更便宜。min(按需价格, 竞价价格) - 会启用缩容到零功能,且副本数为零时确实停止计费——当部署副本数为零时,工作区的信用余额不会变化。下一个请求会被透明处理:边缘节点接受连接并保持连接打开,直到副本准备就绪(GPU 镜像大约需要6分钟),然后返回正常的200响应——不会返回503或拒绝连接,因此没有设置短超时的客户端会一直等待。
min_replicas=0会持续计费——请向用户说明该成本。min_replicas >= 1 - 缩容到零前的空闲窗口仅支持 SDK 设置。 在
AutoScaleConfig(idle_threshold_seconds=...)或deployment create中没有对应的命令行参数,因此使用 CLI 时只能用默认值,无法调整副本保留的空闲时长。deployment update - 健康检查也仅支持 SDK 设置。 Python 中有 ,但 CLI 中没有
HttpHealthCheck(path=..., port=...)/--health-check-path参数,因此 CLI 创建的部署无法阻止流量发送到已启动但未就绪的副本。--readiness-probe - 修改镜像、命令、环境变量、入口点、路径映射、健康检查、云账号、竞价实例、数量或最大运行时间都会强制创建新版本(仅切换机器不会)。如果部署没有发布策略(既没有传入也没有预先存储),SDK 会抛出 ;CLI 会自动添加滚动更新策略。
RuntimeError - 对已存在的部署调用 会静默变为更新/重启操作——不会因为名称冲突而报错。
Deployment.start() - 非模型部署必须指定端口(否则会报 );
ValueError部署默认端口为8000,且需要 GPU 机器。在 SDK 中,--model必须是一个列表——传入整数会在 SDK 内部抛出ports,错误信息不会提到TypeError: 'int' object is not iterable参数。ports - 在构造时不要求传入
AutoScaleConfig,但metric会抛出start()。请务必传入ValueError: The autoscaling metric is required. Currently supported metrics are ['GPU', 'CPU', 'RPM']参数;报错栈会指向metric=而不是配置对象。start() - 私有镜像需要设置 ,但 CLI 和 SDK 都没有对应的参数——参见私有容器镜像。失败时只会返回一个含义不明的
image_secret_ref错误。500 - 序列化后
Secret("NAME")字段为空,因此name会显示deployment inspect。这不是 bug:平台会解析该引用,并以密钥名称作为环境变量名。已通过在运行中的副本内读取{"from_secret": "NAME", "name": "", "value": ""}验证。env - 任何有工作区访问权限的人都可以读取部署日志。因此,如果容器在启动时打印环境变量,会将密钥值泄露到 中——不要用这种方式调试真实密钥的注入问题。
deployment logs - 删除操作是不可逆的,除非使用 参数,否则 CLI 会提示确认;请先与用户确认。显示
--yes不代表部署已经完全消失——删除后的90秒到2.5分钟内,部署仍然会出现在Deployment deleted(以及deployment list)中,有时状态为list --all,副本数是旧值,且PENDING。不要把删除后的列表结果当作删除失败,也不要轮询deletedAt: None来确认删除完成;只有在几分钟后仍然存在时,才重新执行删除操作。list - 不带 的
-f可能会无限期挂起。 有观测显示,有限读取(deployment logs,不带--tail 25)可能没有任何输出且永远不返回,因此脚本化获取日志需要自行设置超时。--follow参数可以正常工作。-f - 部署没有成本查询界面。 会返回
deployment inspect字段,但即使运行了约34分钟的 L4 GPU(实际花费约$0.40),该值仍然是total_cost,且在部署的整个生命周期中都保持为0——这与任务(jobs)不同,任务的0.0会正常填充。也没有可用的计费/使用量 API。查看部署成本的唯一方法是对比前后的工作区信用余额,但该字段在连续调用时会在完整浮点精度和两位小数四舍五入之间切换——因此需要在足够长的时间窗口内对比,以忽略四舍五入的误差。total_cost - 部署在创建时可能会返回验证警告(
--model)。Deployment has unacknowledged warnings: ... Re-run with --ack <code> (repeatable) or --force.不会调用服务器,因此永远不会显示这些警告:它只会回显你设置的模型规格(--dry-run、served_model_name以及你传入的任何 vLLM 参数),不会显示机器、镜像变体或副本配置。weight_source - 功能可能受账号权限限制,拒绝时只会返回含义不明的
--model。403可能会失败并报错deployment create ... --model <hf-id>——没有服务器消息,即使设置Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 403.也只会增加栈跟踪,仍然没有原因说明。这是权限问题,不是参数错误,因此不要调试模型 ID 或参数。可以改为直接部署 vLLM 容器镜像(LIGHTNING_DEBUG=1),该方式不需要权限。--image - 在 Python SDK 中,参数和 CLI 一样接受
teamspace=格式。单独的"owner/teamspace"/org=参数已弃用(会有user=),同时传入两种格式会抛出DeprecationWarning。在 2026.7.31 之前的 SDK 版本中,组合格式会报错“Teamspace owner/name does not exist”;请升级 SDK,而不是拆分参数。ValueError