ofa_server

OpenAI-compatible HTTP server for ofa (BYOK shim for VS Code, etc.).

Exposes:

GET /v1/models — advertises the ofa-* model IDs POST /v1/chat/completions — OpenAI-format chat with SSE streaming GET /healthz — liveness probe (no auth)

What each /v1/chat/completions request does:

  1. Authenticate via Authorization: Bearer <token> (token read from a keyfile created on first –serve run).

  2. Pick the ofa “mode” from the model field:

    ofa-openfoam, ofa-hpc, ofa-code, ofa-amrex, ofa-marbles, ofa-quantum-computing, ofa-vasp, ofa-reframe

  3. Build the system prompt via ofa_main.load_system_prompt(mode) so long-term prefs + lessons are injected exactly the same way the CLI does.

  4. Run RAG on the user’s last message via the same retriever the CLI uses for that mode, fence the result via _fence_rag, and prepend it to the user message.

  5. Stream the assistant response via ofa_main.chat_stream and repackage each chunk as an OpenAI SSE data: { … } line.

What we deliberately do NOT do in v1:

  • Tool-calling. We advertise toolCalling: false in the BYOK config so VS Code uses ofa as a smart chat backend, not as an agent that expects OpenAI function-call JSON.

  • /v1/embeddings. Out of scope; Copilot or another provider can handle embeddings if VS Code needs them.

Concurrency: ThreadingHTTPServer with a per-request handler. Ollama itself serialises generation under the hood; our handler is mostly I/O forwarding, so threads are cheap and prevent one slow request from blocking the readiness probe.

Functions

load_or_create_api_key(path)

Return the bearer token stored at path, creating it (0o600) if the file doesn't exist.

serve([host, port, api_key_file, no_auth, ...])

Start the BYOK server.
ofa_server.load_or_create_api_key(path)[source]

Return the bearer token stored at path, creating it (0o600) if the file doesn’t exist. A missing parent directory is created too.

Parameters:

path (str)

Return type:

str

ofa_server.serve(host='0.0.0.0', port=None, api_key_file=None, no_auth=False, local_port=None, enable_tools=False, quiet=False)[source]

Start the BYOK server. Blocks until Ctrl+C.

Parameters:

  • host (str) – Address to bind. Default 0.0.0.0 so that an ssh -L through Kestrel’s login node can reach the compute-node socket (127.0.0.1 is unreachable across that hop). The bearer token keeps this safe on Kestrel’s internal network. Pass 127.0.0.1 only when client + server run on the same machine.

  • port (int | None) – TCP port on the server side. None (default) uses a per-user-stable random port in 40000-49999 (persisted to $OFA_SCRATCH/.ofa_serve_port so the BYOK URL stays valid across restarts). Kestrel compute nodes can host up to 4 users (quarter-node GPU allocations), so picking from a 10000-port range keeps the collision probability near-zero. Pass 0 to let the OS pick (different port each restart) or a specific integer to pin.

  • api_key_file (str | None) – Path to the bearer-token file. Created with mode 0o600 on first run. Defaults to $OFA_SCRATCH/.ofa_api_key.

  • no_auth (bool) – Skip the Authorization check. ONLY for local testing.

  • local_port (int | None) – Suggested laptop-side port for the printed ssh -L line and BYOK URL. None (default) uses a per-user-stable random port in the 49200-64200 range (persisted to scratch so the VS Code config doesn’t have to change between runs).

  • enable_tools (bool) – Forward OpenAI-format tools/tool_choice from incoming requests to Ollama and translate tool_calls responses back to OpenAI SSE format. Lets VS Code’s Agent mode chain file edits / terminal commands through one approval gate instead of click-per-block. Off by default because local 31B models can emit malformed JSON for VS Code’s complex tool schemas.

  • quiet (bool)

Return type:

None