CI/CD Guide
How a VSKI deployment is shipped and kept converged from CI. The pattern is platform-agnostic: any CI system (the examples use GitHub Actions syntax, which Forgejo Actions runs unchanged) and any deployment platform (a VM with docker, Kubernetes, a PaaS, anything that runs a container image).
The contract
Section titled “The contract”- CI converges, it never improvises. Everything the deployment is
(databases, collections, graphs, plugin instances, pipelines, cron,
secrets) is declared in IaC and converged with
vski apply. CI does not create resources with one-off API calls; if a change is needed, the YAML changes and a push re-converges the deployment. - CI authenticates with an API key, never a login. Admin login JWTs
expire after 24 hours and are for humans. CI holds a key minted in the
deployment by a human (
vski keys create ci --roles admin), stored as a repository secret. CI never mints keys and never creates admins beyond the documented first-admin bootstrap below. - Two stages. The deploy job ships the artifact and whatever the
platform needs around it. The bootstrap job runs after it and converges
the deployment state with the vski CLI from a python image. On GitHub
Actions the stages can also be two files (the second triggered by
workflow_run); Forgejo Actions has noworkflow_run, so the portable form is two jobs in one file withneeds:.
flowchart LR P["push to main"] --> D["deploy job<br/>(platform: volume, TLS,<br/>runtime env, image)"] D -->|"needs:"| B["bootstrap job<br/>(python image, vski CLI)"] B -->|"admins ensure, iac apply,<br/>pipeline kick"| V["VSKI deployment"] CR["cron"] -.->|"stays converged between pushes"| VWhat the deploy stage provides
Section titled “What the deploy stage provides”The deploy job is the platform-specific half: build the image and roll it out with whatever your platform needs. VSKI does not care how, but the result must provide:
- One container with one persistent volume, mounted at
/app/data, created before the first rollout. The whole state (the SQLite database plus any clone/crawl workspaces) lives there; a container restart must never lose it. - Exactly one instance. SQLite has a single writer and there is no
leader election, so never scale past one replica. On Kubernetes that
means a StatefulSet with
replicas: 1and a ReadWriteOnce PersistentVolumeClaim (StatefulSets); elsewhere it is the default single container. To scale out, deploy the replicated cluster instead (a master and read replicas behind tango, see Replication); raisingreplicason a single StatefulSet only gives you independent masters with separate data. - TLS endpoints for the public service port (what agents call) and the headless API port (what dashboards and CI call). The same certificate covers both when they share a domain.
- The runtime environment:
JWT_SECRET, the API key the deployed service authenticates with (commonly the sameCI_API_KEYthe pipeline uses, written by the deploy job once the secret exists), and nothing plugin-related: plugins and their secrets are configured from IaC by the bootstrap job. - The image: built from the repository and rolled out on every push.
One runner note: actions/checkout is a JavaScript action and needs node
in the job container (or on the host). A python image or a minimal runner
does not have it; on such runners clone the repository directly, as the
bootstrap job below does (a public repo needs no token).
The CLI verbs CI runs on
Section titled “The CLI verbs CI runs on”| Command | Role in the pipeline |
|---|---|
vski health --wait 120 | Block until /health answers. Unauthenticated; safe before any credentials exist. |
vski admins ensure --email ... --password ... | Create the first admin if none exists, no-op otherwise. Uses only the public has-admins / init endpoints and discards the JWT init returns. --json prints {"hasAdmins": ..., "created": ...} for the pipeline to branch on. |
vski iac plan | Diff the IaC directory against the deployment. Exit code 2 on drift: usable as a CI gate or a scheduled drift alarm. |
vski iac apply vski --yes | Converge the deployment to the YAML (creates the pipeline secrets, see below). |
vski pipelines run NAME | Kick a pipeline, typically the initial population/index run after a fresh apply. |
vski plugins instances start NAME | Start a plugin instance explicitly after the first apply (normally the reconcile loop already did). |
Secrets and credentials
Section titled “Secrets and credentials”Three different things travel differently. Getting this split right is most of the work.
| Kind | Example | Path |
|---|---|---|
| Runtime environment of the service | JWT_SECRET | Repository secret, written into the platform’s runtime environment by the deploy job. Not IaC. |
| Values the deployment’s pipelines need | provider API keys used by plugins | Repository secret, read by vski iac apply through secrets.yaml (fromEnv), stored encrypted in the deployment, resolved as {{secrets.NAME}} in step params at run time. |
| Credentials minted in the deployment | the CI key itself, service keys | A human runs vski keys create once. CI gets them as repository secrets it authenticates with; a deployed service can share the CI key or get its own, written by the deploy job either way. Rotate by re-minting and updating the secret. |
Pipeline secrets never appear in YAML files, in the runtime environment, or in CI logs. The physical path is:
repository secret -> job env -> iac apply (fromEnv) -> encrypted store -> pipeline executor ({{secrets.NAME}}) -> plugin paramDeclare them in vski/secrets.yaml (values never live in the file):
version: 1database: docs
secrets: embedApiKey: fromEnv: EMBED_API_KEYand consume them in a pipeline step (vski/pipelines.yaml):
steps: - id: index type: plugin target: docs-indexer/index_all params: apiKey: "{{secrets.embedApiKey}}"If the fromEnv variable is not set at apply time, the secret is skipped
with a message (not an error), so a partial CI environment never overwrites
an existing secret with garbage.
The bootstrap job
Section titled “The bootstrap job”This is the vski-specific half, identical on every platform. It runs after the deploy job, in a python image, and owns the deployment’s state:
bootstrap: needs: deploy runs-on: my-runner # The full python image: git is included, and the script builds its # own venv, so nothing is assumed about the runner. container: image: python:3.12 steps: - name: Checkout run: | cd "$GITHUB_WORKSPACE" rm -rf src git clone --depth 1 --branch main \ https://your-host/your-org/your-repo.git src
- name: Bootstrap (health, admins, IaC) env: VSKI_URL: ${{ secrets.VSKI_URL }} ADMIN_EMAIL: ${{ secrets.ADMIN_EMAIL }} ADMIN_PASSWORD: ${{ secrets.ADMIN_PASSWORD }} CI_API_KEY: ${{ secrets.CI_API_KEY }} PROVIDER_API_KEY: ${{ secrets.PROVIDER_API_KEY }} # fromEnv inputs run: | cd "$GITHUB_WORKSPACE/src" python3 -m venv "$RUNNER_TEMP/vski-cli" "$RUNNER_TEMP/vski-cli/bin/pip" install --quiet \ --extra-index-url https://vski.sh/api/packages/x/pypi/simple/ \ "vski-sdk[cli]>=0.12,<0.13" V="$RUNNER_TEMP/vski-cli/bin/vski"
"$V" health --wait 120
created=$("$V" --json admins ensure \ --email "$ADMIN_EMAIL" --password "$ADMIN_PASSWORD" \ | python3 -c 'import json, sys; print(json.load(sys.stdin)["created"])')
if [ "$created" = "True" ]; then echo "::notice::First deployment: admin created from the repo secrets." echo "Mint the CI key, set the CI_API_KEY secret, push again." exit 0 fi
if [ -z "${CI_API_KEY:-}" ]; then echo "::error::Admins exist but CI_API_KEY is not set." exit 1 fi
export VSKI_API_KEY="$CI_API_KEY" VSKI_DB="$DB_NAME" "$V" iac apply vski --yesFirst run: the admin is created and the job exits green with instructions;
everything after the gate is skipped because no key exists yet. Second and
later runs: iac apply converges the whole deployment, and any
post-apply kick (a population pipeline, a plugin start) belongs right
after it in the same job.
First deployment runbook
Section titled “First deployment runbook”- Set the repository secrets:
VSKI_URL(the deployment’s API URL as reachable from CI),ADMIN_EMAIL/ADMIN_PASSWORD(the default admin, used only while none exists),JWT_SECRET, and every provider key the IaC declares throughfromEnv. - Prepare the platform side: DNS for the service domain, the persistent volume, and whatever the deploy job expects.
- Push to main. Deploy ships the image; Bootstrap creates the first admin and stops green.
- Once, as the human:
vski login, thenvski keys create ci --roles adminand set it as theCI_API_KEYrepository secret. The deployed service can share that key (the deploy job writes it into the runtime environment once the secret exists) or get its own if you want the credentials separated. - Push (or re-run). Bootstrap applies the IaC and the deployment is live. Every later push re-converges; the cron schedules keep data fresh between pushes.
Drift alarm (optional)
Section titled “Drift alarm (optional)”Between pushes, a scheduled workflow can assert the deployment still
matches the IaC; vski iac plan exits 2 on drift, which fails the job:
on: schedule: - cron: "0 6 * * *"Worked example
Section titled “Worked example”The docs MCP repository runs exactly this
shape: a platform deploy job, the bootstrap job above, an embedApiKey
pipeline secret, and the docs-reindex pipeline kicked after every apply.
See Tutorial: MCP Server for what it serves and
how it is built.