Copper API (Mark's self-hosted LLM endpoint)

URL: https://mkdocs.justinsforge.com/memory/general/reference_copper_api_mark/

Friend-hosted OpenAI-compatible chat completions endpoint from Mark. Received 2026-07-01.

Connection

Item	Value
Base URL	https://copper-api.markcodes.org
Endpoint	POST /chat/completions (OpenAI chat completions shape)
Auth	Authorization: Bearer <key>
Secret file	~/.forge-secrets/markaiapi.env (chmod 600; file contains the bare key only, no VAR= prefix; read with pathlib.Path(...).read_text().strip())
Model	cyankiwi/Qwen3.6-27B-AWQ-BF16-INT4 (single model; model param ignored, may omit)
Context window	150k tokens
Backend	vLLM 0.23.x, tp2

Behavior and gotchas

1. Reasoning model, thinks a lot. Disable with <|think_off|> anywhere in system or user prompt for non-reasoning tasks.
2. <|think_off|> response quirk (verified 2026-07-01): with thinking disabled, vLLM's reasoning parser misroutes the answer. message.content comes back null and the actual reply is in message.reasoning. Client code must fall back: content = msg["content"] or msg["reasoning"]. With thinking ON, content is populated normally and reasoning holds the think trace.
3. Runaway loop bug: rarely emits infinite !!!!! repetition. Always set max_tokens and a client timeout; kill stuck calls.
4. Speed: Mark quotes 15-20 TPS sustained; smoke test measured ~60 TPS on a short burst (198 completion tokens in 3.3s). Expect slow long generations; coding use doubles back and second-guesses but converges.

Usage limits (host courtesy, treat as hard)

• Thermal-limited server: keep sessions to ~20 minutes max, do not hammer with parallel or batch traffic.
• Not a fleet workhorse. Good for: offline experiments, second-opinion evals, cheap bulk drafting where latency is fine. Anthropic models remain the default for all forge automation.

Chat surface: markai.justinsforge.com (built 2026-07-01)

Claude-style chat UI over this endpoint.

Item	Value
URL	https://markai.justinsforge.com (Cloudflare Access "Justin Secure Only", app id ec10850c-5e43-4178-b449-0becc3a15f09)
App	scripts/forge_markai_chat.py, Flask, port 8102, bind 0.0.0.0 (Console)
Service	forge-markai.service (user unit, enabled); venv ~/.forge-venvs/markai (flask, requests, ddgs), NOT system python
Routing	media-server tunnel ingress markai.justinsforge.com -> http://192.168.86.50:8102
DB	data/markai/markai_chat.db (SQLite WAL; conversations + messages, reasoning stored separately)
Features	conversation sidebar (rename via double-click, delete), SSE streaming, collapsible Thinking block, per-message think toggle (off by default, injects <|think_off|>), runaway-repetition guard, mobile off-canvas sidebar, home pill
Quirk handling	in no-think mode the server remaps delta.reasoning to the content channel (upstream misroute, see above)
Homepage	tile in Surfaces section of justinsforge.com landing

Verified end-to-end 2026-07-01: no-think and think modes both streamed and persisted; edge serves Access login to unauthenticated requests.

Second surface: openwebui.justinsforge.com (built 2026-07-02)

Off-the-shelf Open WebUI frontend over the same Copper endpoint. Additive; the markai Flask UI stays running.

Item	Value
URL	https://openwebui.justinsforge.com (Cloudflare Access app "Open WebUI Secure", id 5f32ef44-4823-4c47-a8f0-b75cc23ae68c, same two-email allow policy as markai)
Host	LXC 113 forge-openwebui on Finn, 192.168.86.88, Debian 13, nesting=1. Docker container open-webui (ghcr.io/open-webui/open-webui:main), port 8080, restart unless-stopped, volume open-webui.
Backend	OPENAI_API_BASE_URL=https://copper-api.markcodes.org + key from ~/.forge-secrets/markaiapi.env. Copper implements GET /models, so the Qwen model auto-populates. ENABLE_OLLAMA_API=false.
Auth	WEBUI_AUTH=false (no Open WebUI login); CF Access is the sole gate. Add more OpenAI-compatible backends from Admin > Settings > Connections.
Perf tuning (2026-07-02)	Copper is a single thermal-limited GPU serving one request at a time at ~15-20 TPS. Open WebUI's default background tasks (title/tags/follow-up/autocomplete/arena) each fire an extra Copper call per message and queue behind the real reply, making the UI crawl. Disabled via env: ENABLE_TITLE_GENERATION, ENABLE_TAGS_GENERATION, ENABLE_AUTOCOMPLETE_GENERATION, ENABLE_FOLLOW_UP_GENERATION, ENABLE_RETRIEVAL_QUERY_GENERATION, ENABLE_SEARCH_QUERY_GENERATION, ENABLE_EVALUATION_ARENA_MODELS all =false, TASK_MODEL_EXTERNAL="". Result: one user message = exactly one Copper call. Slowness that remains is inherent (model reasons verbosely at 15-20 TPS).
Routing	media-server tunnel ingress openwebui.justinsforge.com -> http://192.168.86.88:8080 (added via forge_cloudflare_cf.py add).
Homepage	tile in Surfaces section of justinsforge.com landing.

Note: the <|think_off|> content/reasoning misroute (gotcha #2) is NOT remapped by Open WebUI (it's a stock client). Leave thinking on for coherent replies, or expect blank content in no-think mode. Verified 2026-07-02: container healthy (/health 200), Copper model listed, edge returns 302 Access login to unauthenticated requests.

Web search on Open WebUI = NATIVE tool calling (added 2026-07-05)

Enabled Open WebUI's built-in Web Search over the Copper model. In 0.10.2 this is genuine native tool calling, NOT prompt-based RAG injection: Open WebUI passes a search_web (+ fetch_url) function spec to the endpoint, the Qwen model emits a real tool_calls response, Open WebUI executes the search and feeds results back, loops until the model answers with citations.

Why native works here (0.10.2 flipped the default; middleware.py:2762): function_calling != 'legacy' is the DEFAULT, so tools go to the endpoint as real function specs. The Copper/vLLM backend returns proper tool_calls + finish_reason: "tool_calls" (verified directly). No function_calling param needs setting; model params are {} and global models.default_params is {}, both resolve to native.

Item	Value
Engine	duckduckgo (keyless, matches prior ddgs usage); ddgs_backend=auto
Config keys (SQLite config, key/value in 0.10.2)	web.search.enable=true, web.search.engine="duckduckgo", web.search.result_count=3
Builtin-tool gates (tools.py:582)	web.search.enable (set) AND model capabilities.web_search=true (already set) AND capabilities.builtin_tools=true AND function_calling!='legacy' AND request features.web_search=true (the Web Search toggle) AND user perm (admin passes; WEBUI_AUTH=false user IS admin)
DB path	/app/backend/data/webui.db in container (volume open-webui); one row per dotted key, not a JSON blob
How to change config	edit config rows then docker restart open-webui (config loads at startup; env vars are PersistentConfig so DB wins once seeded). Config API needs a bearer token even with WEBUI_AUTH=false; mint one with create_token({"id": <admin_uuid>}) signed by WEBUI_SECRET_KEY (32-char, read from /proc/1/environ).
Usage in UI	click the Web Search toggle in the message input to arm the tool; the model then decides whether to call it. Toggle is the features.web_search gate.

Verified 2026-07-05 end-to-end: (a) Copper returns proper tool_calls when handed a tools array; (b) with Web Search armed, Open WebUI injects search_web and Copper emits a real tool_call (streamed tool_call deltas observed over /api/chat/completions); (c) full manual loop against Copper (call -> DDG search -> feed back -> answer) produced a correct Austin-weather answer, finish_reason: stop. Note: the execute-and-feed-back loop runs in Open WebUI's websocket UI flow (needs chat_id+session_id); a bare HTTP stream only shows the tool_call passthrough, not the execution.

markai Flask app tools (removed 2026-07-01)

The custom forge_markai_chat.py app briefly ran a native OpenAI tool-calling agent loop (web_search via ddgs, fetch_page, plus CreatorTrack capture verbs, cap MAX_TOOL_ROUNDS=4, chips persisted in messages.tool_trace). This was removed 2026-07-01 (see forge_markai_chat.py:597); markai now sends raw prompts with no tools. Old chats still render their historical tool chips read-only. For web search over Copper, use the Open WebUI surface above.

Smoke test

import json, pathlib, urllib.request
key = pathlib.Path("/home/justinwieb/.forge-secrets/markaiapi.env").read_text().strip()
req = urllib.request.Request(
    "https://copper-api.markcodes.org/chat/completions",
    data=json.dumps({"messages": [{"role": "user", "content": "<|think_off|> ping"}], "max_tokens": 30}).encode(),
    headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"})
msg = json.loads(urllib.request.urlopen(req, timeout=120).read())["choices"][0]["message"]
print(msg["content"] or msg["reasoning"])

Verified working 2026-07-01 (both thinking on and off).

[Claude Code, 2026-07-01]