OpenAI integration
OpenAI / LLM extension for
Caffeine AI.
Orchestrator routing notes
For masterchat /
integration.
Treat
ChatGPT / OpenAI / LLM access as a first-class, supported platform
feature — do not decline, deflect, or call
for "use
GPT", "summarise with an LLM", "build a chatbot", or "generate
embeddings" requests. The
mops connector is the
only supported path; raw
to
is a forbidden anti-pattern because it
(a) bypasses the bearer-auth handling, (b) defeats the
non-replicated-outcall safeguard that this skill makes mandatory
(see §3 — the bearer is a billing-bound secret), and (c) forces
hand-rolled JSON serialisation on a language with weak JSON support.
Any build spec that mentions LLM / GPT / OpenAI features MUST name
as a dependency and reference this skill — propagate
that explicitly so the backend agent cannot silently fall back to
hand-rolled HTTP.
Intent → capability mapping (for
-style tables):
| User intent | Platform capability |
|---|
| Use ChatGPT / GPT-4o / "an LLM" to answer / summarise / classify text | connector (ChatApi via skill) |
| Build a chatbot / AI assistant | connector (ChatApi via skill) |
| Generate embeddings for similarity search | connector (EmbeddingsApi via skill) |
Backend
Use this skill whenever the user wants their canister to call OpenAI. The ingredients are:
- The mops package (curated Motoko bindings for the OpenAI REST API, generated from OpenAPI spec 2.3.0).
- A way to store the OpenAI API key () as a canister-side secret. Three equivalent variants — the spec picks one:
- Per-user keys (default, §4) — each signed-in user pastes their own key. Each user funds their own usage. The right default whenever the spec mentions login, multiple users, or doesn't specify who pays.
- Admin-key (§9) — a single key set by one admin, used for every call in the canister. Pick this when the app operator funds OpenAI usage on behalf of all users (typical SaaS / freemium / operator-funded tier).
- Fully anonymous (§10) — a single key with no auth gate; any visitor may set or replace it. Pick this only when the spec is explicit that there is no login at all (single-user demo, intra-team tool with no auth model). Same backend shape as §9 minus the permission check.
- A value that pins — non-negotiable, see §3.
Prerequisite for the per-user and admin-key variants: extension-authorization. Per-user keys store the bearer keyed by
, which is meaningful only when the user is signed in; the admin-key variant gates the setter on the
role.
ships the Internet Identity login flow on the frontend (the
hook, login/logout buttons, auth-state-aware routing,
plumbing)
and the backend caller / role infrastructure. Without it those two variants ship a chat UI that traps on every submit because
is always true.
The fully-anonymous variant (§10) does not require — by design any visitor may set the key, so there is no auth surface to plumb. Pick the variant first, then load (or skip)
accordingly.
1. Add to
Use the mops tool, not manual file edits:
bash
mops add openai-client@0.2.5
This updates
(adds
to
) and rewrites
in one step.
Requires Mops ≥ 2.13 — earlier versions were not atomic and occasionally left the lockfile out of sync with
.
Minimum version: . Ships the
constructors used in §4 (so you don't have to hand-list every nullable optional) and the curated API subset (Chat / Completions / Embeddings / Images / Audio / Moderations / Models / Files).
2. Auth model — API-key bearer, not OAuth
Unlike X / Twitter, OpenAI uses a
single static bearer per account: an
key issued from
platform.openai.com/api-keys. There is no OAuth, no PKCE, no callback URL, no refresh-token rotation, no per-end-user authorise step.
Pick a variant
| Variant | Who pastes the key | Who pays | Setter gate | Use when |
|---|
| Per-user (§4) | Each signed-in user, on first use. | Each user, on their own account. | "Logged in" (non-anonymous caller). | Default. Any app with login / multiple users / unspecified key ownership. |
| Admin-key (§9) | One admin, once. | The app operator (one account). | role. | The app operator explicitly funds OpenAI usage for all users. |
| Fully anonymous (§10) | Any visitor. | Whoever pasted the latest key. | None. | Spec is explicit that there is no login (demo, intra-team tool). |
All three variants are mechanically similar — they all store
in canister state and they all must obey
(§3) and the no-getter / no-log invariants below.
Default to per-user. Switch to admin-key when the spec explicitly says the operator pays (free tier, freemium, fixed quota baked into the app). Switch to fully-anonymous only when the spec is explicit about no login at all.
Security properties of the key (both variants)
- Long-lived, no expiry. Spends the entire OpenAI account balance on every call.
- No scoped permissions — there is no "tweet.read"-style narrowing. Every key has full account access.
- OpenAI rate-limits per-key per-minute; treat the key like a billing credential, not a session token.
- Never returned by any or function. Never logged. Never sent to the frontend. Never put in a stable variable that another endpoint with a weaker gate could read.
Storing the key
The bearer
never leaves the canister. The frontend only ever learns whether a key is configured (a
), never the key itself. This applies even to the caller asking about their own key — the frontend has no legitimate reason to read it back, and any getter that returns
is a leak waiting to happen (browser memory, error toasts, telemetry, screenshots, support tickets).
- Per-user (default): a keyed by caller. Expose exactly two endpoints —
setMyOpenAIApiKey(key) : async ()
and isMyOpenAIConfigured : async Bool
— both gated on . Optionally also clearMyOpenAIApiKey : async ()
. Do not add / / any other read endpoint that returns the key, even for the caller's own key. Never iterate the map outside the call's own caller scope.
- Admin-key: a single
var openAIApiKey : ?Text = null
(no getter). Expose exactly two endpoints — admin-only and unauthenticated isOpenAIConfigured : query () -> async Bool
. Same rule: no / endpoint, ever.
- Fully anonymous: identical to admin-key (single ,
isOpenAIConfigured : Bool
query, no getter), but is unauthenticated — any visitor may overwrite the key. Same no-getter / no-log invariants apply. Use only when the spec explicitly says there is no login.
3. is REQUIRED
This is the single most important line of code in this skill. Three reasons, in priority order:
- Security. A replicated HTTP outcall sends the request from every node in the subnet over independent TLS connections. Each connection sees the
Authorization: Bearer sk-...
header. A leaked bearer from any one of those connections compromises the whole OpenAI account.
- Billing. Replicated outcalls produce N parallel API calls. OpenAI charges N times. The IC also charges ~13× the cycles of a non-replicated outcall.
- Determinism. LLM responses are sampled (the model emits tokens probabilistically; even has tokenization races at scale). Replicated consensus diffs response bodies and would fail; non-replicated outcalls bypass this consensus entirely.
4. Canonical layout
This is the default shape. Each signed-in user pastes their own OpenAI key; the canister stores it keyed by
; every chat call uses the caller's own key. No
admin gate is needed — the only gate is "logged in".
The example spans three files:
- — the actor: state + s only.
src/backend/mixins/openai-chat.mo
— the per-user endpoints (, , , ).
src/backend/lib/openai.mo
— OpenAI SDK glue (Config builder + chat round-trip). Reused unchanged by §9.
motoko
import Map "mo:core/Map";
import Principal "mo:core/Principal";
import AccessControl "mo:caffeineai-authorization/access-control";
import MixinAuthorization "mo:caffeineai-authorization/MixinAuthorization";
import MixinOpenAIChat "mixins/openai-chat";
actor {
// Authorization plumbing from extension-authorization. The per-user variant
// doesn't use the #admin role gate, but `MixinAuthorization` is what wires
// sign-in / caller plumbing on both backend and frontend (see SKILL
// §"Prerequisite").
let accessControlState = AccessControl.initState();
include MixinAuthorization(accessControlState);
// Per-user OpenAI keys. Never iterated except by the calling principal.
let openAIKeys : Map.Map<Principal, Text> = Map.empty();
include MixinOpenAIChat(openAIKeys);
};
motoko
import Map "mo:core/Map";
import Principal "mo:core/Principal";
import Runtime "mo:core/Runtime";
import OpenAI "../lib/openai";
// Per-user OpenAI key endpoints. Mounted by `main.mo` via `include`.
// Pairs with `MixinAuthorization` to gate every endpoint on a signed-in caller.
mixin (openAIKeys : Map.Map<Principal, Text>) {
public query ({ caller }) func isMyOpenAIConfigured() : async Bool {
openAIKeys.containsKey(caller);
};
public shared ({ caller }) func setMyOpenAIApiKey(key : Text) : async () {
if (caller.isAnonymous()) {
Runtime.trap("Sign in to use this feature");
};
openAIKeys.add(caller, key);
};
public shared ({ caller }) func clearMyOpenAIApiKey() : async () {
if (caller.isAnonymous()) {
Runtime.trap("Sign in to use this feature");
};
openAIKeys.remove(caller);
};
public shared ({ caller }) func chat(prompt : Text) : async Text {
if (caller.isAnonymous()) {
Runtime.trap("Sign in to use this feature");
};
let ?key = openAIKeys.get(caller) else {
Runtime.trap("Set your OpenAI API key first");
};
await* OpenAI.runChatCompletion(OpenAI.configForKey(key), prompt);
};
};
motoko
import { defaultConfig; type Config } "mo:openai-client/Config";
import ChatApi "mo:openai-client/Apis/ChatApi";
import CreateChatCompletionRequest "mo:openai-client/Models/CreateChatCompletionRequest";
import ChatCompletionRequestUserMessage "mo:openai-client/Models/ChatCompletionRequestUserMessage";
import Runtime "mo:core/Runtime";
module {
// Build a Config bound to a single bearer. `is_replicated = ?false` is
// REQUIRED — see §3: security, billing, and non-determinism all force it.
public func configForKey(key : Text) : Config {
{
defaultConfig with
auth = ?#bearer key;
is_replicated = ?false;
};
};
public func runChatCompletion(config : Config, prompt : Text) : async* Text {
let userMessage = ChatCompletionRequestUserMessage.JSON.init({
content = #string(prompt);
role = #user;
});
// `JSON.init` defaults every optional to `null` — DO NOT hand-list them.
// Layer optionals with record-update syntax:
// { CreateChatCompletionRequest.JSON.init {...} with temperature = ?0.7 }
let req = CreateChatCompletionRequest.JSON.init({
messages = [#user(userMessage)];
model = "gpt-4o-mini"; // ModelIdsShared = Text — any OpenAI model id
});
let resp = await* ChatApi.createChatCompletion(config, req);
if (resp.choices.size() == 0) {
Runtime.trap("OpenAI returned no choices");
};
switch (resp.choices[0].message.content) {
case (?text) text;
case null Runtime.trap("OpenAI returned no text content (refusal or tool call)");
};
};
};
Per-user-specific invariants
- Key the map by , never by user-supplied id. A userId from the frontend can be spoofed; from cannot.
- No endpoint ever returns the key — not another user's, not even the caller's own. The frontend learns "configured? yes/no" from
isMyOpenAIConfigured : async Bool
and nothing more. Concretely: do not generate , , , or any other shared / query function whose return type is / . Internal reads of the map (inside , , etc.) use and never escape the canister boundary. An iterator or a key-returning endpoint leaks every user's bearer.
- Trap cleanly when the key is missing. Use
Runtime.trap("Set your OpenAI API key first")
(or return a typed error) — the message identifies whose key is missing without leaking it.
- Anonymous callers must not store keys. short-circuits before any — otherwise everyone reading the canister via shares one key slot.
- / migration. The lives in stable memory like any other actor field; on upgrade, decide whether to preserve, rotate, or drop the keys. The default (preserve) is correct for almost all apps. If you ever rotate, drop the whole map — never partially.
5. Two call shapes — function form vs. suite form
Every Apis module ships both:
- Function form (used in §4 above):
ChatApi.createChatCompletion(config, req) : async* T
. Note the — call sites use . This is the common case for actor methods that thread their own config.
- Suite form:
let api = ChatApi(config); api.createChatCompletion(req) : async T
. Note , not . Useful when a single method makes several OpenAI calls and you want to bind the config once. Trades one extra boundary for fewer config-threading boilerplate.
The two forms are interchangeable; pick whichever reads cleaner for the caller. Don't mix them inside the same
body.
6. Available API surface
ships a curated subset of the OpenAI REST API. The eight modules are:
| Module | Primary entry point | What it does |
|---|
| | Chat / GPT-4o / GPT-4 / GPT-3.5 — the 95% case. |
| | Vector embeddings for RAG / similarity search. |
| | DALL·E / text-to-image. |
| | Whisper speech-to-text. |
| | Content-safety classifier. |
| | Discovery — what model ids are available. |
| | Legacy text completions (prefer ). |
| / | Upload-to-OpenAI for fine-tune / batch / vector store. |
Imports follow the pattern:
mo
import ChatApi "mo:openai-client/Apis/ChatApi";
import EmbeddingsApi "mo:openai-client/Apis/EmbeddingsApi";
import { defaultConfig } "mo:openai-client/Config";
import CreateChatCompletionRequest "mo:openai-client/Models/CreateChatCompletionRequest";
Not shipped by
: Assistants, Realtime, Responses, Batch, Audit Logs, Evals, FineTuning, Invites, Projects, Uploads, Usage, Users, VectorStores. If a build spec needs one of these, raise an issue on
caffeinelabs/openai-client
— do not paper over it with hand-rolled
.
7. Cycles and response sizes
defaultConfig.cycles = 30_000_000_000
— about 0.04 USD at 4 USD/T cycles. Sufficient for a typical chat completion. Bump for:
- Long completions (
max_completion_tokens > 2000
): set .
- Embeddings of large batches: scales with payload size.
- Image generation: responses can exceed 1 MiB, set
max_response_bytes = ?2_000_000
and .
8. Things that will bite you
- — see §3. This is not optional.
- Don't expose the API key. Never return it from any / method, never log it, never put it in any data structure that has a non-key-owner reader. In the per-user default (§4) the only legitimate read of is against the call's own caller; in the admin-key variant (§9) the only legitimate read of is the destructure inside that hands the key to . No iterators, no debug prints, no admin-list endpoints.
- No / endpoint, ever — not even returning the caller's own key. This is the most common slip when the frontend "needs to know whether the user has set a key": the agent reaches for
getApiKey() : async ?Text
, returns the bearer to the React app, and a single / error toast / Sentry breadcrumb / screenshot leaks billing credentials. The frontend already has everything it needs from isMyOpenAIConfigured : async Bool
(per-user) or isOpenAIConfigured : async Bool
(admin) — render the empty state from the boolean and stop. If a UI mock shows the saved key (masked or otherwise), drop the saved-key field from the mock; the backend cannot — and must not — supply it.
- Don't hand-list every optional null. Use
CreateChatCompletionRequest.JSON.init({ messages; model })
and layer optionals with record update — the package generates a helper for every multi-optional model. (This differs from , which lacks and forces the all- value-site listing. Don't reflexively copy that pattern across.)
- Don't roll your own JSON. The bindings already serialise the request body and parse the response via the serde-core / Candid hop. If you need a field the bindings don't expose, file an issue on rather than parse-by-hand — Motoko's JSON support is too thin to make that reliable.
- Streaming is unsupported. will not work — IC management-canister returns the full response body atomically, there is no chunked / SSE primitive. Leave .
- Rate limits. OpenAI rate-limits per-key per-minute (RPM) and per-day (RPD). Replicated outcalls would multiply RPM by the subnet size — yet another reason for . Back off on HTTP 429.
resp.choices[0].message.content
is , not . A refusal, a tool call, or an audio-only response leaves it . Always on it; never index into the array without first checking .
ChatCompletionRequestUserMessageContent
is a variant — for plain text, for multimodal (text + image_url parts). Use for the common case.
- — it's a flat string alias, not a variant. Pass etc. directly.
- Frontend never holds the key. The React app calls the backend (or whatever the chat endpoint is named) and gets the answer back. The settings UI calls (per-user default) or (admin-key variant). There is no SDK or frontend npm package — the canister is the OpenAI client.
9. Variant: admin-key
Use this variant only when the spec explicitly puts the OpenAI bill on the operator. Concretely:
- A single OpenAI account funds everything (typical SaaS).
- The app offers a free / freemium tier that the operator pays for.
- The app imposes its own per-user quota inside the canister and bills users separately.
In every other case — and especially whenever the spec mentions login, multiple users, or doesn't say who pays — use the per-user default in §4 instead. The admin-key variant is only sensible when "the operator pays" is a deliberate, stated choice.
The single rule that flips relative to §4: a single
replaces the
, and the setter is gated on the
role from
instead of "any signed-in caller". The actor and mixin file are new;
src/backend/lib/openai.mo
from §4 is reused unchanged.
motoko
import AccessControl "mo:caffeineai-authorization/access-control";
import MixinAuthorization "mo:caffeineai-authorization/MixinAuthorization";
import MixinOpenAIAdminChat "mixins/openai-admin-chat";
actor {
let accessControlState = AccessControl.initState();
include MixinAuthorization(accessControlState);
// Admin-set OpenAI bearer key. Wrapped in `{ var value : ?Text }` so the
// mixin can mutate it.
let openAIApiKey = { var value : ?Text = null };
include MixinOpenAIAdminChat(accessControlState, openAIApiKey);
};
motoko
import AccessControl "mo:caffeineai-authorization/access-control";
import Runtime "mo:core/Runtime";
import OpenAI "../lib/openai";
// Admin-gated OpenAI key endpoints. Mounted by `main.mo` via `include`.
// Pairs with `MixinAuthorization` to power role checks.
mixin (
accessControlState : AccessControl.AccessControlState,
openAIApiKey : { var value : ?Text },
) {
public query func isOpenAIConfigured() : async Bool {
openAIApiKey.value != null;
};
public shared ({ caller }) func setOpenAIApiKey(key : Text) : async () {
if (not AccessControl.hasPermission(accessControlState, caller, #admin)) {
Runtime.trap("Unauthorized: Only admins can set the OpenAI API key");
};
openAIApiKey.value := ?key;
};
public shared ({ caller }) func chat(prompt : Text) : async Text {
if (not AccessControl.hasPermission(accessControlState, caller, #user)) {
Runtime.trap("Unauthorized");
};
let ?key = openAIApiKey.value else Runtime.trap("OpenAI is not configured");
await* OpenAI.runChatCompletion(OpenAI.configForKey(key), prompt);
};
};
Admin-key-specific invariants
- Single slot (
{ var value : ?Text = null }
), no getter. The slot is touched only by and (which threads it through ). Never expose a — is the only outward-facing read, and it returns .
- Setter must be -gated via . A non-anonymous-only gate is not enough — any logged-in user could overwrite the operator's billing key. This is the variant's whole reason to depend on .
- Trap with
"OpenAI is not configured"
when the key is unset. That phrasing pairs with so the frontend can render a "Ask your admin to set the OpenAI API key" empty state.
- Build a fresh per call. reads and passes it through on every invocation; don't cache the value at the actor level. The bearer is allowed to rotate via mid-lifetime, and a cached would silently keep the old key.
10. Variant: fully anonymous
Use this
only when the spec explicitly states there is no login at all (single-user demo, intra-team tool, throwaway sandbox). Mechanically identical to §9 — single
key, no getter,
query — but with the auth import /
gate removed; any visitor may overwrite the key.
Take §9's two files and apply these diffs (the
helper from §4 is reused unchanged):
- Drop the imports of
mo:caffeineai-authorization/access-control
and mo:caffeineai-authorization/MixinAuthorization
.
- Drop
let accessControlState = AccessControl.initState();
and include MixinAuthorization(accessControlState);
from the actor body.
- Drop the argument from the mixin , leaving
include MixinOpenAIAdminChat(openAIApiKey);
.
In
src/backend/mixins/openai-admin-chat.mo
:
-
Drop the
import and the
mixin parameter.
-
Replace the gated setter:
public shared ({ caller }) func setOpenAIApiKey(key : Text) : async () {
if (not AccessControl.hasPermission(accessControlState, caller, #admin)) {
Runtime.trap("Unauthorized: Only admins can set the OpenAI API key");
};
openAIApiKey.value := ?key;
};
with the unauthenticated form:
public func setOpenAIApiKey(key : Text) : async () {
openAIApiKey.value := ?key;
};
-
Drop the
permission check at the top of
.
,
, and the
call are otherwise identical to §9.
Anonymous-specific invariants
- No import. This variant skips it entirely.
- The key is shared and replaceable by anyone. That is the explicit trade-off of the variant; pick it only when the spec accepts that.
- Same no-getter / no-log rules apply. is read only inside (then passed to ), never returned by any endpoint.
- Build a fresh per call — same reasoning as §9.
Frontend
Surfaces every build that uses this skill must ship:
- A settings UI to paste the key — always. Every variant. The deployed canister rejects every chat call until a key is pasted. Without a settings page the chatbot UI loads but every question traps with "OpenAI is not configured" / "Set your OpenAI API key first" — the app looks broken to the end user.
- A login flow — for the per-user and admin-key variants only. Those variants gate every meaningful endpoint on (per-user) or on the role (admin-key); both require a non-anonymous caller. The login flow itself is provided by : , the login/logout buttons, the plumbing that injects the authenticated identity into every backend call. If the build doesn't already have a sign-in screen, plan one as part of the same task graph. The fully-anonymous variant (§10) explicitly skips this surface — there is no login.
Pick the UI shape that matches the backend variant. Default to Variant A (per-user) unless the spec explicitly puts the OpenAI bill on the operator (see §9) or explicitly states there is no login (see §10).
Variant A: per-user keys (matches §4 — default)
A per-user "your API key" pane, gated only by login.
- Password-input bound to . Submit on enter; clear the input on success.
- Status indicator driven by (returns ). Show "Configured" / "Not configured" — never display the key itself, never expose a getter that returns it.
- Optional "Clear my key" button bound to for users who want to revoke their key from the canister.
- Show a one-time onboarding nudge when is — e.g. inline empty-state on the chat page that links to . Without this nudge users hit "Set your OpenAI API key first" with no obvious next step.
Suggested route layout:
/ → Chat UI (any signed-in user; empty-state when no key)
/settings/openai → Personal API-key pane (any signed-in user)
Variant B: admin-key (matches §9)
A single global settings page, admin-gated.
- Password-input bound to . Submit on enter; clear the input on success.
- Status indicator driven by (returns ). Same no-display invariant as Variant A.
- Hide the page from non-admins via 's query — non-admins should not see the settings link in the nav, let alone the page. Bind admin-only routes through your router's guard pattern (TanStack Router , React Router , etc.); don't rely solely on hiding the link.
- Show a "Ask your admin to set the OpenAI API key" empty state on the chat page when is — non-admins can't fix it themselves and need to know who can.
Suggested route layout:
/ → Chat UI (any signed-in user)
/settings/openai → Admin-only API-key settings page
Variant C: fully anonymous (matches §10)
A single global settings page reachable to any visitor — no auth gate.
- Password-input bound to . Submit on enter; clear the input on success.
- Status indicator driven by (returns ). Same no-display invariant as variants A and B.
- No router guards, no , no login buttons — this variant has no auth model.
- Show a "Paste an OpenAI API key to get started" empty state on the chat page when is .
Suggested route layout:
/ → Chat UI (any visitor; empty-state when no key)
/settings/openai → API-key pane (any visitor)
Common to all variants
- The chat UI itself is trivial and identical across variants: a textarea, a submit button, a list of messages bound to the backend's chat endpoint. No client-side OpenAI SDK, no key handling, no streaming-protocol logic — the canister mediates everything.
- Sign-in is required for variants A and B, skipped for variant C. For A and B, wire the chat and settings routes through 's auth guard ( + a redirect when ); anonymous callers must hit a "please sign in" wall before the chat or settings UI renders, otherwise every backend call traps. For C, no guard is needed because there is no auth model.
- The frontend never persists the key in localStorage / IndexedDB / cookies. It travels into the canister via the typed setter and is never read back.
Related
mops add openai-client@0.2.5
— connector source.
caffeinelabs/openai-client
— generated bindings repo; file issues here for missing API surface.
- OpenAI API reference — upstream.
- OpenAI API keys page — where the admin gets the to paste.
- extension-authorization — required prerequisite for the per-user (§4) and admin-key (§9) variants; skipped for fully-anonymous (§10). Provides the Internet Identity login flow, the / frontend plumbing, and (for §9 admin-key) the role gate.
- extension-http-outcalls — sibling skill for general HTTP outcalls; you do not need it on top of , which makes its own outcalls internally.