Skip to content
C-Kernel-Engine docs Antshiv Open Source

C-KERNEL-ENGINE · GUIDE

Configure the text server

Start with a short reply on loopback. Once it works, change one setting at a time. Run the commands from your CKE checkout after Linux setup.

Choose weights, address and budget

Examplebash
.venv/bin/python -m pip install -r server/requirements.txt
MODEL=hf://unsloth/gemma-3-270m-it-GGUF/gemma-3-270m-it-Q5_K_M.gguf
RUN="$HOME/cke-runs/gemma3-270m-1k"
version/v8/scripts/cks-v8-run serve "$MODEL" --run "$RUN" \
  --host 127.0.0.1 --port 8080 --model-name cke-local \
  --context-len 1024 --max-tokens 64 --temperature 0 \
  --request-output-cap 128 --request-prompt-byte-cap 8192
  • Which model? The positional argument picks the weights. --run gives this build its own directory. Once the bundle works, add --no-build to reopen it. Keep separate directories for different checkpoints, context builds and serving variants.
  • Which address? --host and --port select the listener. The local development service defaults to 127.0.0.1:8080 and has no built-in authentication. An agent on the same host can use that address. An agent in another container has its own loopback.
  • Which name? --model-name sets the ID the client sends. Choose a descriptive name when running multiple endpoints. A friendly name alone cannot identify the exact artifact.
  • How much fits? --context-len is the token budget. The formatted prompt includes instructions, history, role markers and tool definitions. The server needs room for that input plus the requested reply. Reopening a compiled bundle cannot increase its capacity.
  • How much may it answer? --max-tokens defaults the HTTP reply allowance to 512 if omitted. A client can request a different allowance with max_output_tokens. --request-output-cap clamps that allowance. --request-prompt-byte-cap limits rendered UTF-8 input bytes independently. A multibyte character, a byte and a model token are different units.

Choose sampling and stop conditions

--temperature 0 picks the highest-scoring next token. For sampling, --temperature defaults to 0.7 and --top-p to 1.0. Clients can override these with temperature and top_p. Keep the model, template, prompt and settings fixed when comparing outputs. The launcher accepts several compatibility flags, including --top-k and --repeat-penalty, and warns that its native HTTP sampling path ignores them.

Use repeatable --stop-on-text for a decoded delimiter specific to your application. A reply can also finish early at the model's end token. Check the response status and token usage before increasing the limit.

Templates, tools and concurrent clients

Start with the checkpoint's chat template. --serving-variant tools selects a declared tools variant at build time for models such as Nemotron. --chat-template-file supplies an explicit replacement for a matching model. A wrong template can produce plausible text with incorrect role or tool boundaries. Check the model and harness matrix before configuring a tool client.

The default path handles one active generation at a time. Exact-prefix reuse (--prefix-checkpoint-cache-bytes) and experimental two-slot HTTP (--batch-http-extra-bytes) require suitable generated bundles and budgets. Check the artifact-specific evidence before enabling either for an agent. Use --no-stats to disable console metrics or --no-viz to disable the live request view.

Examplebash
version/v8/scripts/cks-v8-run serve --help

Check the running endpoint · CKE server flags at the reviewed revision