lightning-deployments

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Lightning AI Deployments

Lightning AI 部署

A Deployment runs a container (or an auto-built vLLM server for a HuggingFace model) behind an HTTPS endpoint with autoscaling. Sources are mutually exclusive:
--image
(Docker),
--model
(HuggingFace id, GPU required), or
--studio
(deploy a Studio's environment).
部署(Deployment)会在支持自动扩缩容的 HTTPS 端点后运行一个容器(或为 HuggingFace 模型自动构建的 vLLM 服务器)。部署源是互斥的:
--image
(Docker 镜像)、
--model
(HuggingFace 模型 ID,需 GPU)或
--studio
(部署 Studio 环境)。

Setup & auth

安装与认证

bash
undefined
bash
undefined

Use the Lightning AI CLI from the current env; install or upgrade it there if it's missing or older than 2026.9.18

Use the Lightning AI CLI from the current env; install or upgrade it there if it's missing or older than 2026.9.18

v=$(lightning --version 2>/dev/null | sed -n 's/^Lightning CLI version //p') [ -n "$v" ] && [ "$(printf '%s\n' 2026.9.18 "$v" | sort -V | head -1)" = 2026.9.18 ]
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token

This uses, and if needed installs into, the environment the agent runs in (the user's project
venv, a conda env, …). Then call plain `lightning …` everywhere. If setup doesn't go cleanly:

| Symptom | Fix |
|---|---|
| Both installs fail: no active env, or pip refuses with `externally-managed-environment` | Install it on its own instead: `uv tool install lightning-sdk` (or `pipx install lightning-sdk`), then call `"$(uv tool dir --bin)/lightning"` if that dir isn't on `PATH` |
| `lightning --version` still prints no `Lightning CLI version` line after installing | `command -v lightning` shows which one runs. Another tool owns the name (PyTorch Lightning also installs a `lightning` command), or the env you installed into isn't on `PATH`: activate it (`source .venv/bin/activate`) or call its `bin/lightning` by full path |
| `No such command`, `No such option` or `unexpected extra argument` | The CLI is older than this skill expects: `uv pip install -U lightning-sdk` (or `pip install -U lightning-sdk`) in the env `command -v lightning` points into; `uv tool upgrade lightning-sdk` for a uv tool install |
| The upgrade fails on a version conflict with the project's own pins | Don't fight the pins: install it outside the project with `uv tool install lightning-sdk` and call `"$(uv tool dir --bin)/lightning"` |
| Installing is blocked (read-only env or home, agent sandbox) | Skip it and run each command as `UV_CACHE_DIR="${TMPDIR:-/tmp}/uv" uvx lightning-sdk …` |
| Every call fails on SSL/certificates, even `lightning --version` (`Could not find a suitable TLS CA certificate bundle`), only inside an agent sandbox | A `**/*.pem` read-deny rule is hiding certifi's public CA bundle (`site-packages/certifi/cacert.pem`). Allow that one path; don't disable the sandbox |

The install above also makes `lightning_sdk` importable in that env, so Python snippets run with
its `python`. If the CLI came from a uv tool or `uvx` fallback instead, run one-off scripts with
`uv run --with lightning-sdk python script.py`. For code that stays in the user's project, ask,
then declare `lightning-sdk` as a dependency with the project's own tool (`uv add`, `poetry add`).
v=$(lightning --version 2>/dev/null | sed -n 's/^Lightning CLI version //p') [ -n "$v" ] && [ "$(printf '%s\n' 2026.9.18 "$v" | sort -V | head -1)" = 2026.9.18 ]
|| uv pip install -U lightning-sdk || python3 -m pip install -U lightning-sdk lightning --version # must print "Lightning CLI version …"; if not, see the table below lightning login # browser flow; or headless: export LIGHTNING_API_KEY=... LIGHTNING_USER_ID=... # USER_ID optional: with it Basic auth, without it the key is a Bearer token

此操作会使用 Agent 运行所在的环境(用户的项目 venv、conda 环境等),如有需要也会安装到该环境中。之后所有地方都直接调用 `lightning …` 即可。如果安装过程不顺利,请参考下表:

| 问题现象 | 解决方法 |
|---|---|
| 两种安装方式都失败:没有激活的环境,或 pip 报错 `externally-managed-environment` | 改为独立安装:`uv tool install lightning-sdk`(或 `pipx install lightning-sdk`),如果该目录不在 `PATH` 中,则调用 `"$(uv tool dir --bin)/lightning"` |
| 安装后 `lightning --version` 仍然没有输出 `Lightning CLI version` 行 | 用 `command -v lightning` 查看当前运行的是哪个程序。可能是其他工具占用了这个名称(PyTorch Lightning 也会安装 `lightning` 命令),或者你安装的环境不在 `PATH` 中:激活该环境(`source .venv/bin/activate`)或通过完整路径调用其 `bin/lightning` |
| `No such command`、`No such option` 或 `unexpected extra argument` | CLI 版本低于本技能要求:在 `command -v lightning` 指向的环境中执行 `uv pip install -U lightning-sdk`(或 `pip install -U lightning-sdk`);如果是 uv tool 安装的,则执行 `uv tool upgrade lightning-sdk` |
| 升级因与项目自身的版本锁定冲突而失败 | 不要强行修改版本锁定:用 `uv tool install lightning-sdk` 在项目外安装,并调用 `"$(uv tool dir --bin)/lightning"` |
| 安装被阻止(只读环境或家目录、Agent 沙箱) | 跳过安装,将每条命令改为 `UV_CACHE_DIR="${TMPDIR:-/tmp}/uv" uvx lightning-sdk …` 运行 |
| 所有调用都因 SSL/证书问题失败,即使是 `lightning --version`(报错 `Could not find a suitable TLS CA certificate bundle`),且仅在 Agent 沙箱中出现 | 可能是 `**/*.pem` 读取拒绝规则隐藏了 certifi 的公共 CA 包(`site-packages/certifi/cacert.pem`)。请允许该路径的访问权限,不要禁用沙箱 |

上述安装还会让 `lightning_sdk` 在该环境中可导入,因此 Python 代码片段可以用该环境的 `python` 运行。如果 CLI 来自 uv tool 或 `uvx` 备用方案,则使用 `uv run --with lightning-sdk python script.py` 运行一次性脚本。对于要保留在用户项目中的代码,请先询问用户,然后用项目自身的工具(`uv add`、`poetry add`)将 `lightning-sdk` 声明为依赖项。

Resolving org and teamspace (do this first)

确定组织与工作区(请先执行此步骤)

Deployments live in a teamspace owned by an organization or a user. Never guess. Use explicit
--teamspace owner/teamspace
(Python:
teamspace="owner/teamspace"
; the separate
org=
/
user=
arguments are deprecated), env vars
LIGHTNING_ORG
/
LIGHTNING_TEAMSPACE
, or the config default (
lightning config get teamspace
). If none is set, list the options and ask the user which org/teamspace to use:
bash
lightning api /v1/memberships | jq -r '.memberships[] | [.ownerType, .name, .projectId] | @tsv'
Persist the choice:
lightning config set teamspace <owner>/<teamspace>
.
From a scoped API key (an agent, no user to ask):
/v1/memberships
gives the teamspace name and the owner id, but
--teamspace
needs the owner slug — resolve it via
/v1/orgs
(needs
jq
). Select the
organization
entry rather than
.memberships[0]
: the same teamspace is commonly listed twice, once with
ownerType: organization
and once with
ownerType: user
(identical
projectId
and
name
), so index 0 is a coin flip and the
user
row's
ownerId
will not resolve against
/v1/orgs
.
bash
M=$(lightning api /v1/memberships)
ROW=$(echo "$M" | jq -c '[.memberships[] | select(.ownerType=="organization")][0] // .memberships[0]')
TS=$(echo "$ROW" | jq -r .name)                                                        # teamspace
OWNER=$(lightning api "/v1/orgs/$(echo "$ROW" | jq -r .ownerId)" | jq -r .name)         # owner (org) slug
lightning config set teamspace "$OWNER/$TS"     # every command now defaults here; or pass --teamspace "$OWNER/$TS"
部署存放在由组织或用户拥有的工作区(teamspace)中。绝对不要猜测。 请使用显式的
--teamspace owner/teamspace
(Python 中为
teamspace="owner/teamspace"
;单独的
org=
/
user=
参数已弃用)、环境变量
LIGHTNING_ORG
/
LIGHTNING_TEAMSPACE
,或配置默认值(
lightning config get teamspace
)。如果都没有设置,请列出可选选项并询问用户要使用哪个组织/工作区:
bash
lightning api /v1/memberships | jq -r '.memberships[] | [.ownerType, .name, .projectId] | @tsv'
保存选择:
lightning config set teamspace <owner>/<teamspace>
。
使用受限 API 密钥时(Agent 场景,无法询问用户):
/v1/memberships
会返回工作区名称和所有者 ID,但
--teamspace
需要所有者 slug——需通过
/v1/orgs
解析(需要
jq
)。请选择
organization
条目而不是
.memberships[0]
:同一个工作区通常会出现两次,一次是
ownerType: organization
,一次是
ownerType: user
(
projectId
和
name
相同),因此直接取第0个是随机的,且
user
行的
ownerId
无法通过
/v1/orgs
解析。
bash
M=$(lightning api /v1/memberships)
ROW=$(echo "$M" | jq -c '[.memberships[] | select(.ownerType=="organization")][0] // .memberships[0]')
TS=$(echo "$ROW" | jq -r .name)                                                        # teamspace
OWNER=$(lightning api "/v1/orgs/$(echo "$ROW" | jq -r .ownerId)" | jq -r .name)         # owner (org) slug
lightning config set teamspace "$OWNER/$TS"     # every command now defaults here; or pass --teamspace "$OWNER/$TS"

CLI reference

CLI 参考

Subcommands:
create
,
list
,
inspect
,
update
,
delete
,
logs
(+
logs download
),
reload-weights
, and
env
/
secret
(
list
/
set
/
delete
on a running deployment). There is no
stop
— stopping = scaling to zero via
update --min-replicas 0 --max-replicas 0
.
bash
undefined
子命令包括:
create
、
list
、
inspect
、
update
、
delete
、
logs
(含
logs download
)、
reload-weights
,以及
env
/
secret
(对运行中的部署执行
list
/
set
/
delete
操作)。没有
stop
命令——停止部署等于通过
update --min-replicas 0 --max-replicas 0
将副本数缩容到0。
bash
undefined

container deployment (a --port is required for non-model deployments)

container deployment (a --port is required for non-model deployments)

lightning deployment create my-api --teamspace owner/teamspace
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth
lightning deployment create my-api --teamspace owner/teamspace
--image nginx:latest --machine CPU --port 80 --replicas 1 --api-key-auth

HuggingFace model via vLLM (GPU machine required; port defaults to 8000)

HuggingFace model via vLLM (GPU machine required; port defaults to 8000)

lightning deployment create llama --teamspace owner/teamspace
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]
lightning deployment create llama --teamspace owner/teamspace
--model meta-llama/Llama-3.1-8B-Instruct --machine L40S
[--hf-token-secret <secret-name>] [--tensor-parallel-size 2] [--max-model-len 8192]
[--quantization fp8] [--dtype bfloat16] [--dry-run] [--force]

cloud account — if you pass none, the SDK picks one (config
cloud-account
, then

cloud account — if you pass none, the SDK picks one (config
cloud-account
, then

LIGHTNING_CLUSTER_ID, then the teamspace's default binding) and prints which on stderr.

LIGHTNING_CLUSTER_ID, then the teamspace's default binding) and prints which on stderr.

Other deployments in the same teamspace may sit on a different account, which

Other deployments in the same teamspace may sit on a different account, which

deployment list
shows in its last column; match it with --cloud when relevant.

deployment list
shows in its last column; match it with --cloud when relevant.

lightning deployment create ... --cloud gcp-lightning-public-prod
lightning deployment create ... --cloud gcp-lightning-public-prod

autoscaling / env / secrets

autoscaling / env / secrets

lightning deployment create ...
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas
lightning deployment create ...
--min-replicas 0 --max-replicas 4 --autoscale-metric GPU --autoscale-threshold 90
-e KEY=VALUE --secret MY_LIGHTNING_SECRET --interruptible # CHECK THE SPOT PRICE FIRST — see Gotchas

endpoint auth — mutually exclusive; OMITTING ALL THREE MAKES THE ENDPOINT PUBLIC

endpoint auth — mutually exclusive; OMITTING ALL THREE MAKES THE ENDPOINT PUBLIC

Run
lightning auth whoami
first: --api-key-auth only accepts user keys.

Run
lightning auth whoami
first: --api-key-auth only accepts user keys.

--api-key-auth # require a Lightning USER API key (Bearer) — see Gotchas --basic-auth USER:PASS --token-auth TOKEN # any bearer string you choose; use this from a scoped key
--api-key-auth # require a Lightning USER API key (Bearer) — see Gotchas --basic-auth USER:PASS --token-auth TOKEN # any bearer string you choose; use this from a scoped key

operate

operate

lightning deployment list --teamspace owner/teamspace [--all] [--sort-by state] lightning deployment inspect my-api --teamspace owner/teamspace # full JSON incl. endpoint URLs lightning deployment logs my-api --teamspace owner/teamspace [-f] [--tail 100] [--timestamps] lightning deployment logs my-api --teamspace owner/teamspace --query timeout --severity error # filter server-side lightning deployment logs my-api --teamspace owner/teamspace --since 30m --until 5m # window (replicas merged + labelled) lightning deployment update my-api --teamspace owner/teamspace --max-replicas 8 lightning deployment update my-api --teamspace owner/teamspace --min-replicas 0 --max-replicas 0 # "stop" lightning deployment update my-api --teamspace owner/teamspace --image myimg:v2 [--max-surge 1] [--max-unavailable 0] # new release, rolling update lightning deployment reload-weights llama --teamspace owner/teamspace # hot-reload BYOM weights lightning deployment delete my-api --teamspace owner/teamspace --yes
undefined
lightning deployment list --teamspace owner/teamspace [--all] [--sort-by state] lightning deployment inspect my-api --teamspace owner/teamspace # full JSON incl. endpoint URLs lightning deployment logs my-api --teamspace owner/teamspace [-f] [--tail 100] [--timestamps] lightning deployment logs my-api --teamspace owner/teamspace --query timeout --severity error # filter server-side lightning deployment logs my-api --teamspace owner/teamspace --since 30m --until 5m # window (replicas merged + labelled) lightning deployment update my-api --teamspace owner/teamspace --max-replicas 8 lightning deployment update my-api --teamspace owner/teamspace --min-replicas 0 --max-replicas 0 # "stop" lightning deployment update my-api --teamspace owner/teamspace --image myimg:v2 [--max-surge 1] [--max-unavailable 0] # new release, rolling update lightning deployment reload-weights llama --teamspace owner/teamspace # hot-reload BYOM weights lightning deployment delete my-api --teamspace owner/teamspace --yes
undefined

Python SDK

Python SDK

python
from lightning_sdk import Deployment, Machine
from lightning_sdk.api.deployment_api import (
    ApiKeyAuth, BasicAuth, TokenAuth, AutoScaleConfig, Env, Secret,
    RollingUpdateReleaseStrategy, HttpHealthCheck,
)

dep = Deployment("my-api", teamspace="my-org/my-teamspace")   # owner/teamspace, same as the CLI; org=/user= are deprecated
dep.start(
    image="nginx:latest",
    machine=Machine.CPU,
    ports=[80],                                 # REQUIRED for non-model deployments; a LIST, not an int
    autoscale=AutoScaleConfig(min_replicas=0, max_replicas=4, metric="CPU", threshold=90),
    env=[Env("MODE", "prod"), Secret("HF_TOKEN")],   # or a plain dict for env vars
    auth=ApiKeyAuth(),                          # None => PUBLIC endpoint
    health_check=HttpHealthCheck(path="/health", port=80),
    spot=False,                                 # True = interruptible replicas
)

print(dep.urls)                                 # endpoint URL(s); no .status property — use replica counts
print(dep.running_replicas, dep.pending_replicas, dep.failing_replicas)
r = dep.get(path="/health")                     # convenience HTTP; auto-auths only for ApiKeyAuth
python
from lightning_sdk import Deployment, Machine
from lightning_sdk.api.deployment_api import (
    ApiKeyAuth, BasicAuth, TokenAuth, AutoScaleConfig, Env, Secret,
    RollingUpdateReleaseStrategy, HttpHealthCheck,
)

dep = Deployment("my-api", teamspace="my-org/my-teamspace")   # owner/teamspace, same as the CLI; org=/user= are deprecated
dep.start(
    image="nginx:latest",
    machine=Machine.CPU,
    ports=[80],                                 # REQUIRED for non-model deployments; a LIST, not an int
    autoscale=AutoScaleConfig(min_replicas=0, max_replicas=4, metric="CPU", threshold=90),
    env=[Env("MODE", "prod"), Secret("HF_TOKEN")],   # or a plain dict for env vars
    auth=ApiKeyAuth(),                          # None => PUBLIC endpoint
    health_check=HttpHealthCheck(path="/health", port=80),
    spot=False,                                 # True = interruptible replicas
)

print(dep.urls)                                 # endpoint URL(s); no .status property — use replica counts
print(dep.running_replicas, dep.pending_replicas, dep.failing_replicas)
r = dep.get(path="/health")                     # convenience HTTP; auto-auths only for ApiKeyAuth

changing image/command/env/entrypoint/spot/... creates a NEW RELEASE and needs a release strategy

changing image/command/env/entrypoint/spot/... creates a NEW RELEASE and needs a release strategy

dep.update(image="nginx:1.27", release_strategy=RollingUpdateReleaseStrategy(max_surge=1, max_unavailable=0)) dep.update(min_replicas=1, max_replicas=8) # scaling: no new release, no strategy needed dep.stop() # scales to 0; blocks until replicas reach 0

HuggingFace model serving via SDK: `dep.start(model="meta-llama/...", machine=Machine.L40S, ports=[8000], hf_token_secret="...")` — `image`/`studio` must be None; still pass `ports`.
dep.update(image="nginx:1.27", release_strategy=RollingUpdateReleaseStrategy(max_surge=1, max_unavailable=0)) dep.update(min_replicas=1, max_replicas=8) # scaling: no new release, no strategy needed dep.stop() # scales to 0; blocks until replicas reach 0

通过 SDK 部署 HuggingFace 模型服务:`dep.start(model="meta-llama/...", machine=Machine.L40S, ports=[8000], hf_token_secret="...")` —— `image`/`studio` 必须为 None;仍需传入 `ports` 参数。

Private container images

私有容器镜像

A deployment whose image needs registry credentials must set
V1JobSpec.image_secret_ref
to the name of a
SECRET_TYPE_DOCKER_REGISTRY
secret in the teamspace. Neither the CLI nor
Deployment.start()
exposes this
, and the server rejects an unpullable image with a bare
500
—
Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 500.
— which names neither the image nor the missing credential. Until the SDK exposes it, patch the field onto the create request:
python
from lightning_sdk.lightning_cloud.openapi.api import jobs_service_api

_create = jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info

def _with_image_secret(self, body, project_id, **kwargs):
    if getattr(body, "spec", None) is not None:
        body.spec.image_secret_ref = "MY_REGISTRY_SECRET"
    return _create(self, body, project_id, **kwargs)

jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info = _with_image_secret
List the teamspace's secrets, and their types, with:
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId')
lightning api "/v1/projects/${PROJECT_ID}/secrets" | jq -r '.secrets[] | "\(.name) \(.type)"'
Diagnosing this 500 is much faster if you bisect against a public image first (
nginx:latest
): if that create succeeds with otherwise identical arguments, the image is the variable, not the machine, port, replica counts or cloud account.
Trap:
dep.delete()
does NOT delete the deployment — it is an HTTP helper like
dep.get()
/
dep.post()
and sends an HTTP
DELETE
request to the deployed service's endpoint. To delete the deployment resource use
lightning deployment delete NAME --yes
or the raw API.
如果部署的镜像需要仓库凭证,必须将
V1JobSpec.image_secret_ref
设置为工作区中类型为
SECRET_TYPE_DOCKER_REGISTRY
的密钥名称。CLI 和
Deployment.start()
都没有暴露该参数
,服务器无法拉取镜像时只会返回一个简单的
500
错误——
Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 500.
——既不会说明是哪个镜像,也不会说缺少凭证。在 SDK 支持该参数之前,可以通过打补丁的方式在创建请求中添加该字段:
python
from lightning_sdk.lightning_cloud.openapi.api import jobs_service_api

_create = jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info

def _with_image_secret(self, body, project_id, **kwargs):
    if getattr(body, "spec", None) is not None:
        body.spec.image_secret_ref = "MY_REGISTRY_SECRET"
    return _create(self, body, project_id, **kwargs)

jobs_service_api.JobsServiceApi.jobs_service_create_deployment_with_http_info = _with_image_secret
使用以下命令列出工作区的密钥及其类型:
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId')
lightning api "/v1/projects/${PROJECT_ID}/secrets" | jq -r '.secrets[] | "\(.name) \(.type)"'
如果先使用公共镜像(
nginx:latest
)进行对比排查,诊断这个500错误会快得多:如果其他参数完全相同的情况下公共镜像部署成功,那么问题出在镜像上,而不是机器、端口、副本数或云账号。
陷阱:
dep.delete()
不会删除部署资源——它是和
dep.get()
/
dep.post()
类似的 HTTP 辅助方法,会向部署的服务端点发送 HTTP
DELETE
请求。要删除部署资源,请使用
lightning deployment delete NAME --yes
或原始 API。

Example workflows

示例工作流

Prompts this skill handles: "deploy this docker image behind an API", "serve Llama 3.1 8B on lightning", "scale my deployment down at night", "roll out the new image version".
Deploy a container, verify it responds, then clean up:
bash
lightning auth whoami    # `user` → --api-key-auth is callable by you; `scoped-api-key` → use --token-auth
lightning deployment create hello-api --teamspace my-org/my-teamspace \
  --image nginx:latest --machine CPU --port 80 --min-replicas 0 --max-replicas 1 --api-key-auth
lightning deployment inspect hello-api --teamspace my-org/my-teamspace   # JSON: status, endpoint URL
URL=<endpoint URL from the inspect output above>
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LIGHTNING_API_KEY" "$URL/"   # verify the ENDPOINT, not the Studio
python
from lightning_sdk import Deployment
dep = Deployment("hello-api", teamspace="my-org/my-teamspace")
print(dep.urls)                       # e.g. ['https://80-dep-<id>-d.cloudspaces.litng.ai'] — available even while scaled to zero
print(dep.get(path="/").status_code)  # sends the caller's Lightning API key for ApiKeyAuth
bash
lightning deployment delete hello-api --teamspace my-org/my-teamspace --yes
Serve an open-weights LLM (vLLM, built server-side — no Dockerfile needed):
bash
lightning deployment create qwen-served --teamspace my-org/my-teamspace \
  --model Qwen/Qwen2.5-7B-Instruct --machine L40S --min-replicas 0 --max-replicas 1 --api-key-auth --dry-run
本技能处理的提示词示例:"把这个docker镜像部署成API"、"在lightning上部署Llama 3.1 8B服务"、"晚上把我的部署缩容"、"发布新的镜像版本"。
部署容器、验证响应,然后清理:
bash
lightning auth whoami    # `user` → --api-key-auth is callable by you; `scoped-api-key` → use --token-auth
lightning deployment create hello-api --teamspace my-org/my-teamspace \
  --image nginx:latest --machine CPU --port 80 --min-replicas 0 --max-replicas 1 --api-key-auth
lightning deployment inspect hello-api --teamspace my-org/my-teamspace   # JSON: status, endpoint URL
URL=<endpoint URL from the inspect output above>
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $LIGHTNING_API_KEY" "$URL/"   # verify the ENDPOINT, not the Studio
python
from lightning_sdk import Deployment
dep = Deployment("hello-api", teamspace="my-org/my-teamspace")
print(dep.urls)                       # e.g. ['https://80-dep-<id>-d.cloudspaces.litng.ai'] — available even while scaled to zero
print(dep.get(path="/").status_code)  # sends the caller's Lightning API key for ApiKeyAuth
bash
lightning deployment delete hello-api --teamspace my-org/my-teamspace --yes
部署开源大语言模型服务(vLLM,服务端构建——无需 Dockerfile):
bash
lightning deployment create qwen-served --teamspace my-org/my-teamspace \
  --model Qwen/Qwen2.5-7B-Instruct --machine L40S --min-replicas 0 --max-replicas 1 --api-key-auth --dry-run

review the model spec it prints, then re-run without --dry-run; if the real create reports

review the model spec it prints, then re-run without --dry-run; if the real create reports

unacknowledged warnings, re-run with --ack <code> (repeatable) or --force

unacknowledged warnings, re-run with --ack <code> (repeatable) or --force

lightning deployment logs qwen-served --teamspace my-org/my-teamspace -f # watch it come up

The endpoint serves an OpenAI-compatible API on port 8000 — point any OpenAI client's `base_url` at `dep.urls[0]`.

**Operate: scale, update, roll back traffic costs:**

```bash
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 1 --max-replicas 4   # keep warm
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 0 --max-replicas 0   # stop (scale to zero)
lightning deployment update hello-api --teamspace my-org/my-teamspace --image nginx:1.27                  # new release, rolling by default
lightning deployment logs qwen-served --teamspace my-org/my-teamspace -f # watch it come up

该端点在8000端口提供兼容 OpenAI 的 API——将任何 OpenAI 客户端的 `base_url` 指向 `dep.urls[0]` 即可。

**运维操作:扩缩容、更新、回滚与流量成本:**

```bash
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 1 --max-replicas 4   # keep warm
lightning deployment update hello-api --teamspace my-org/my-teamspace --min-replicas 0 --max-replicas 0   # stop (scale to zero)
lightning deployment update hello-api --teamspace my-org/my-teamspace --image nginx:1.27                  # new release, rolling by default

Raw API fallback

原始API备用方案

bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId' | head -1)
lightning api "/v1/projects/${PROJECT_ID}/deployments" -q '.deployments[].name'   # plain GET; -f/-F without -X would send a POST
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}"
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}" -X DELETE
bash
PROJECT_ID=$(lightning api /v1/memberships | jq -r '.memberships[] | select(.name=="<teamspace>") | .projectId' | head -1)
lightning api "/v1/projects/${PROJECT_ID}/deployments" -q '.deployments[].name'   # plain GET; -f/-F without -X would send a POST
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}"
lightning api "/v1/projects/${PROJECT_ID}/deployments/${DEPLOYMENT_ID}" -X DELETE

Gotchas

常见陷阱

  • Deployments bill for as long as a replica is up; confirm the spend with the user before creating one. Get an explicit go-ahead before expensive GPU machines (A100/H100/H200/B200), before
    --min-replicas >= 1
    (which bills continuously, with no scale-to-zero), and before raising
    --max-replicas
    , since the ceiling is what the bill can reach. Price the machine first — see the
    lightning-cost-estimation
    skill.
  • Omitting all auth flags creates a publicly reachable endpoint — confirm that's intended.
  • Check
    lightning auth whoami
    before picking an auth flag.
    --api-key-auth
    gates the endpoint on a Lightning user key (it serializes to
    userApiKey: true
    ), so an
    Auth type: scoped-api-key
    caller gets 401 on every request to an endpoint that is otherwise healthy — replicas running, seconds billed. Use
    --token-auth <token>
    when a scoped key, CI job or agent is what will call the endpoint;
    --api-key-auth
    when humans with their own Lightning logins will. Verified against a control: a second, pre-existing healthy deployment returned an identical 401 to the same scoped key.
  • Verify a deployment against its public URL from
    deployment inspect
    — never against the app running inside the source Studio.
    The Studio proves the image; only the endpoint proves the service. A deployment can show a healthy replica and bill normally while 401-ing every request.
  • --interruptible
    is not reliably cheaper — check the price before you pass it.
    On GCP the L4 is $0.48/hr on-demand but $0.727/hr spot: interruptible costs 51% more and you take preemption risk for the privilege. Nothing warns you at create time. Fetch both rates from the accelerator catalog first (see the
    lightning-cost-estimation
    skill) and take
    min(cost, spotPrice)
    ; spot is only reliably cheaper on some SKUs and clouds.
  • min_replicas=0
    enables scale-to-zero and genuinely stops billing at zero — a teamspace credit balance stays flat while a deployment sits at zero replicas. The next request is served transparently: the edge accepts the connection and holds it open until a replica is ready (~6 minutes on a GPU image) and then returns a normal 200 — it does not return 503 or refuse, so any client without a short timeout just waits.
    min_replicas >= 1
    bills continuously — flag this cost to the user.
  • The idle window before scale-to-zero is SDK-only.
    AutoScaleConfig(idle_threshold_seconds=...)
    has no equivalent flag on
    deployment create
    or
    deployment update
    , so from the CLI you get the default and cannot tune how long a replica lingers.
  • A health check is SDK-only too.
    HttpHealthCheck(path=..., port=...)
    exists in Python but there is no
    --health-check-path
    /
    --readiness-probe
    flag, so a CLI-created deployment has nothing stopping traffic reaching a replica that is up but not ready.
  • Changing image, command, env, entrypoint, path mappings, health check, cloud account, spot, quantity or max runtime forces a new release (switching the machine alone does not). The SDK raises
    RuntimeError
    if the deployment has no release strategy, neither passed nor already stored; the CLI auto-adds a rolling update.
  • Deployment.start()
    on an existing deployment silently becomes an update/restart — it won't error on a name collision.
  • A port is mandatory for non-model deployments (
    ValueError
    otherwise);
    --model
    defaults to 8000 and requires a GPU machine. In the SDK
    ports
    must be a list — passing an int fails with
    TypeError: 'int' object is not iterable
    raised from inside the SDK, which does not mention
    ports
    .
  • AutoScaleConfig
    accepts no
    metric
    at construction, then
    start()
    raises
    ValueError: The autoscaling metric is required. Currently supported metrics are ['GPU', 'CPU', 'RPM']
    . Always pass
    metric=
    ; the traceback points at
    start()
    , not at the config object.
  • A private image needs
    image_secret_ref
    , which no CLI flag or SDK argument sets — see Private container images. The failure is an opaque
    500
    .
  • Secret("NAME")
    serializes with an empty
    name
    field, so
    deployment inspect
    shows
    {"from_secret": "NAME", "name": "", "value": ""}
    . That is not a bug: the platform resolves the reference and names the environment variable after the secret. Verified by reading
    env
    inside a running replica.
  • Deployment logs are readable by anyone with teamspace access. A container that echoes its environment on boot will therefore leak secret values into
    deployment logs
    — do not debug secret injection that way on a real secret.
  • Deleting is destructive and the CLI prompts unless
    --yes
    ; confirm with the user first.
    Deployment deleted
    is not "gone"
    — the deployment keeps appearing in
    deployment list
    (and
    list --all
    ) for 90 seconds to ~2.5 minutes afterwards, sometimes in
    PENDING
    with a stale replica count and
    deletedAt: None
    . Don't treat a post-delete listing as a failed delete, and don't poll
    list
    to confirm teardown; re-issue
    delete
    only if it is still there after a few minutes.
  • deployment logs
    without
    -f
    can hang indefinitely.
    A bounded read (
    --tail 25
    , no
    --follow
    ) has been observed producing no output at all and never returning, so a scripted log fetch needs its own timeout.
    -f
    works.
  • There is no cost surface for a deployment.
    deployment inspect
    exposes
    total_cost
    , but it reads
    0.0
    even after 34 minutes of L4 GPU time ($0.40 of real spend), and stays 0 for the deployment's whole life — unlike jobs, whose
    total_cost
    does populate. No billing/usage endpoint resolves either. The only way to see what a deployment cost is to difference the teamspace credit balance before and after, and that field flips between full float precision and 2-decimal rounding between consecutive calls — so difference over a window long enough that the rounding is noise.
  • --model
    deployments may return validation warnings on create (
    Deployment has unacknowledged warnings: ... Re-run with --ack <code> (repeatable) or --force.
    ).
    --dry-run
    never calls the server, so it never shows those warnings: it only echoes the model spec you set (
    served_model_name
    ,
    weight_source
    and any vLLM flags you passed), not the machine, image variant or replica config.
  • --model
    may be gated on your account, and the refusal is an opaque
    403
    .
    deployment create ... --model <hf-id>
    can fail with
    Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 403.
    — no server message, and
    LIGHTNING_DEBUG=1
    adds a traceback but still no reason. It is an entitlement, not a bad argument, so don't debug the model id or flags. Fall back to deploying a vLLM container image directly (
    --image
    ), which needs no entitlement.
  • In the Python SDK,
    teamspace=
    takes
    "owner/teamspace"
    like the CLI. The separate
    org=
    /
    user=
    arguments are deprecated (
    DeprecationWarning
    ), and passing both forms raises
    ValueError
    . On SDKs older than 2026.7.31, the combined form failed with "Teamspace owner/name does not exist"; upgrade rather than splitting it.
  • 只要有副本在运行,部署就会产生费用;创建部署前请与用户确认费用支出。 在使用昂贵的 GPU 机器(A100/H100/H200/B200)、设置
    --min-replicas >= 1
    (会持续计费,无法缩容到0)以及提高
    --max-replicas
    之前,都需要获得用户明确同意,因为上限就是账单可能达到的金额。请先查询机器定价——参考
    lightning-cost-estimation
    技能。
  • 省略所有认证标志会创建一个公开可访问的端点——请确认这是预期行为。
  • 选择认证标志前请先执行
    lightning auth whoami
    。
    --api-key-auth
    会使用 Lightning 用户 密钥验证端点(序列化为
    userApiKey: true
    ),因此调用方类型为
    Auth type: scoped-api-key
    时,即使端点本身正常(副本运行中、正常计费),所有请求也都会返回 401。如果调用方是受限密钥、CI 任务或 Agent,请使用
    --token-auth <token>
    ;如果是使用个人 Lightning 账号登录的用户,则使用
    --api-key-auth
    。已通过对照组验证:另一个预先存在的健康部署对同一个受限密钥也返回了相同的401错误。
  • 请通过
    deployment inspect
    返回的公共 URL 验证部署,绝不要通过源 Studio 中运行的应用验证。
    Studio 只能证明镜像可用;只有端点才能证明服务正常。部署可能显示副本健康且正常计费,但所有请求都返回401。
  • --interruptible
    不一定更便宜——使用前请先查询价格。
    在 GCP 上,L4 实例的按需价格是 $0.48/小时,而竞价型(spot)价格是 $0.727/小时:可中断实例反而贵51%,还要承担被抢占的风险。创建时不会有任何警告。请先从加速器目录中获取两种价格(参考
    lightning-cost-estimation
    技能),取
    min(按需价格, 竞价价格)
    ;竞价实例仅在部分 SKU 和云平台上确实更便宜。
  • min_replicas=0
    会启用缩容到零功能,且副本数为零时确实停止计费——当部署副本数为零时,工作区的信用余额不会变化。下一个请求会被透明处理:边缘节点接受连接并保持连接打开,直到副本准备就绪(GPU 镜像大约需要6分钟),然后返回正常的200响应——不会返回503或拒绝连接,因此没有设置短超时的客户端会一直等待。
    min_replicas >= 1
    会持续计费——请向用户说明该成本。
  • 缩容到零前的空闲窗口仅支持 SDK 设置。
    AutoScaleConfig(idle_threshold_seconds=...)
    在
    deployment create
    或
    deployment update
    中没有对应的命令行参数,因此使用 CLI 时只能用默认值,无法调整副本保留的空闲时长。
  • 健康检查也仅支持 SDK 设置。 Python 中有
    HttpHealthCheck(path=..., port=...)
    ,但 CLI 中没有
    --health-check-path
    /
    --readiness-probe
    参数,因此 CLI 创建的部署无法阻止流量发送到已启动但未就绪的副本。
  • 修改镜像、命令、环境变量、入口点、路径映射、健康检查、云账号、竞价实例、数量或最大运行时间都会强制创建新版本(仅切换机器不会)。如果部署没有发布策略(既没有传入也没有预先存储),SDK 会抛出
    RuntimeError
    ;CLI 会自动添加滚动更新策略。
  • 对已存在的部署调用
    Deployment.start()
    会静默变为更新/重启操作——不会因为名称冲突而报错。
  • 非模型部署必须指定端口(否则会报
    ValueError
    );
    --model
    部署默认端口为8000,且需要 GPU 机器。在 SDK 中,
    ports
    必须是一个列表——传入整数会在 SDK 内部抛出
    TypeError: 'int' object is not iterable
    ,错误信息不会提到
    ports
    参数。
  • AutoScaleConfig
    在构造时不要求传入
    metric
    ,但
    start()
    会抛出
    ValueError: The autoscaling metric is required. Currently supported metrics are ['GPU', 'CPU', 'RPM']
    。请务必传入
    metric=
    参数;报错栈会指向
    start()
    而不是配置对象。
  • 私有镜像需要设置
    image_secret_ref
    ,但 CLI 和 SDK 都没有对应的参数——参见私有容器镜像。失败时只会返回一个含义不明的
    500
    错误。
  • Secret("NAME")
    序列化后
    name
    字段为空,因此
    deployment inspect
    会显示
    {"from_secret": "NAME", "name": "", "value": ""}
    。这不是 bug:平台会解析该引用,并以密钥名称作为环境变量名。已通过在运行中的副本内读取
    env
    验证。
  • 任何有工作区访问权限的人都可以读取部署日志。因此,如果容器在启动时打印环境变量,会将密钥值泄露到
    deployment logs
    中——不要用这种方式调试真实密钥的注入问题。
  • 删除操作是不可逆的,除非使用
    --yes
    参数,否则 CLI 会提示确认;请先与用户确认。显示
    Deployment deleted
    不代表部署已经完全消失
    ——删除后的90秒到2.5分钟内,部署仍然会出现在
    deployment list
    (以及
    list --all
    )中,有时状态为
    PENDING
    ,副本数是旧值,且
    deletedAt: None
    。不要把删除后的列表结果当作删除失败,也不要轮询
    list
    来确认删除完成;只有在几分钟后仍然存在时,才重新执行删除操作。
  • 不带
    -f
    的
    deployment logs
    可能会无限期挂起。
    有观测显示,有限读取(
    --tail 25
    ,不带
    --follow
    )可能没有任何输出且永远不返回,因此脚本化获取日志需要自行设置超时。
    -f
    参数可以正常工作。
  • 部署没有成本查询界面。
    deployment inspect
    会返回
    total_cost
    字段,但即使运行了约34分钟的 L4 GPU(实际花费约$0.40),该值仍然是
    0.0
    ,且在部署的整个生命周期中都保持为0——这与任务(jobs)不同,任务的
    total_cost
    会正常填充。也没有可用的计费/使用量 API。查看部署成本的唯一方法是对比前后的工作区信用余额,但该字段在连续调用时会在完整浮点精度和两位小数四舍五入之间切换——因此需要在足够长的时间窗口内对比,以忽略四舍五入的误差。
  • --model
    部署在创建时可能会返回验证警告(
    Deployment has unacknowledged warnings: ... Re-run with --ack <code> (repeatable) or --force.
    )。
    --dry-run
    不会调用服务器,因此永远不会显示这些警告:它只会回显你设置的模型规格(
    served_model_name
    、
    weight_source
    以及你传入的任何 vLLM 参数),不会显示机器、镜像变体或副本配置。
  • --model
    功能可能受账号权限限制,拒绝时只会返回含义不明的
    403
    。
    deployment create ... --model <hf-id>
    可能会失败并报错
    Exception: The jobs_service_create_deployment_with_http_info request failed to reach the server, response: 403.
    ——没有服务器消息,即使设置
    LIGHTNING_DEBUG=1
    也只会增加栈跟踪,仍然没有原因说明。这是权限问题,不是参数错误,因此不要调试模型 ID 或参数。可以改为直接部署 vLLM 容器镜像(
    --image
    ),该方式不需要权限。
  • 在 Python SDK 中,
    teamspace=
    参数和 CLI 一样接受
    "owner/teamspace"
    格式。单独的
    org=
    /
    user=
    参数已弃用(会有
    DeprecationWarning
    ),同时传入两种格式会抛出
    ValueError
    。在 2026.7.31 之前的 SDK 版本中,组合格式会报错“Teamspace owner/name does not exist”;请升级 SDK,而不是拆分参数。