Skip to content

CI/CD Guide

How a VSKI deployment is shipped and kept converged from CI. The pattern is platform-agnostic: any CI system (the examples use GitHub Actions syntax, which Forgejo Actions runs unchanged) and any deployment platform (a VM with docker, Kubernetes, a PaaS, anything that runs a container image).

  • CI converges, it never improvises. Everything the deployment is (databases, collections, graphs, plugin instances, pipelines, cron, secrets) is declared in IaC and converged with vski apply. CI does not create resources with one-off API calls; if a change is needed, the YAML changes and a push re-converges the deployment.
  • CI authenticates with an API key, never a login. Admin login JWTs expire after 24 hours and are for humans. CI holds a key minted in the deployment by a human (vski keys create ci --roles admin), stored as a repository secret. CI never mints keys and never creates admins beyond the documented first-admin bootstrap below.
  • Two stages. The deploy job ships the artifact and whatever the platform needs around it. The bootstrap job runs after it and converges the deployment state with the vski CLI from a python image. On GitHub Actions the stages can also be two files (the second triggered by workflow_run); Forgejo Actions has no workflow_run, so the portable form is two jobs in one file with needs:.
flowchart LR
P["push to main"] --> D["deploy job<br/>(platform: volume, TLS,<br/>runtime env, image)"]
D -->|"needs:"| B["bootstrap job<br/>(python image, vski CLI)"]
B -->|"admins ensure, iac apply,<br/>pipeline kick"| V["VSKI deployment"]
CR["cron"] -.->|"stays converged between pushes"| V

The deploy job is the platform-specific half: build the image and roll it out with whatever your platform needs. VSKI does not care how, but the result must provide:

  • One container with one persistent volume, mounted at /app/data, created before the first rollout. The whole state (the SQLite database plus any clone/crawl workspaces) lives there; a container restart must never lose it.
  • Exactly one instance. SQLite has a single writer and there is no leader election, so never scale past one replica. On Kubernetes that means a StatefulSet with replicas: 1 and a ReadWriteOnce PersistentVolumeClaim (StatefulSets); elsewhere it is the default single container. To scale out, deploy the replicated cluster instead (a master and read replicas behind tango, see Replication); raising replicas on a single StatefulSet only gives you independent masters with separate data.
  • TLS endpoints for the public service port (what agents call) and the headless API port (what dashboards and CI call). The same certificate covers both when they share a domain.
  • The runtime environment: JWT_SECRET, the API key the deployed service authenticates with (commonly the same CI_API_KEY the pipeline uses, written by the deploy job once the secret exists), and nothing plugin-related: plugins and their secrets are configured from IaC by the bootstrap job.
  • The image: built from the repository and rolled out on every push.

One runner note: actions/checkout is a JavaScript action and needs node in the job container (or on the host). A python image or a minimal runner does not have it; on such runners clone the repository directly, as the bootstrap job below does (a public repo needs no token).

CommandRole in the pipeline
vski health --wait 120Block until /health answers. Unauthenticated; safe before any credentials exist.
vski admins ensure --email ... --password ...Create the first admin if none exists, no-op otherwise. Uses only the public has-admins / init endpoints and discards the JWT init returns. --json prints {"hasAdmins": ..., "created": ...} for the pipeline to branch on.
vski iac planDiff the IaC directory against the deployment. Exit code 2 on drift: usable as a CI gate or a scheduled drift alarm.
vski iac apply vski --yesConverge the deployment to the YAML (creates the pipeline secrets, see below).
vski pipelines run NAMEKick a pipeline, typically the initial population/index run after a fresh apply.
vski plugins instances start NAMEStart a plugin instance explicitly after the first apply (normally the reconcile loop already did).

Three different things travel differently. Getting this split right is most of the work.

KindExamplePath
Runtime environment of the serviceJWT_SECRETRepository secret, written into the platform’s runtime environment by the deploy job. Not IaC.
Values the deployment’s pipelines needprovider API keys used by pluginsRepository secret, read by vski iac apply through secrets.yaml (fromEnv), stored encrypted in the deployment, resolved as {{secrets.NAME}} in step params at run time.
Credentials minted in the deploymentthe CI key itself, service keysA human runs vski keys create once. CI gets them as repository secrets it authenticates with; a deployed service can share the CI key or get its own, written by the deploy job either way. Rotate by re-minting and updating the secret.

Pipeline secrets never appear in YAML files, in the runtime environment, or in CI logs. The physical path is:

repository secret -> job env -> iac apply (fromEnv) -> encrypted store
-> pipeline executor ({{secrets.NAME}}) -> plugin param

Declare them in vski/secrets.yaml (values never live in the file):

version: 1
database: docs
secrets:
embedApiKey:
fromEnv: EMBED_API_KEY

and consume them in a pipeline step (vski/pipelines.yaml):

steps:
- id: index
type: plugin
target: docs-indexer/index_all
params:
apiKey: "{{secrets.embedApiKey}}"

If the fromEnv variable is not set at apply time, the secret is skipped with a message (not an error), so a partial CI environment never overwrites an existing secret with garbage.

This is the vski-specific half, identical on every platform. It runs after the deploy job, in a python image, and owns the deployment’s state:

bootstrap:
needs: deploy
runs-on: my-runner
# The full python image: git is included, and the script builds its
# own venv, so nothing is assumed about the runner.
container:
image: python:3.12
steps:
- name: Checkout
run: |
cd "$GITHUB_WORKSPACE"
rm -rf src
git clone --depth 1 --branch main \
https://your-host/your-org/your-repo.git src
- name: Bootstrap (health, admins, IaC)
env:
VSKI_URL: ${{ secrets.VSKI_URL }}
ADMIN_EMAIL: ${{ secrets.ADMIN_EMAIL }}
ADMIN_PASSWORD: ${{ secrets.ADMIN_PASSWORD }}
CI_API_KEY: ${{ secrets.CI_API_KEY }}
PROVIDER_API_KEY: ${{ secrets.PROVIDER_API_KEY }} # fromEnv inputs
run: |
cd "$GITHUB_WORKSPACE/src"
python3 -m venv "$RUNNER_TEMP/vski-cli"
"$RUNNER_TEMP/vski-cli/bin/pip" install --quiet \
--extra-index-url https://vski.sh/api/packages/x/pypi/simple/ \
"vski-sdk[cli]>=0.12,<0.13"
V="$RUNNER_TEMP/vski-cli/bin/vski"
"$V" health --wait 120
created=$("$V" --json admins ensure \
--email "$ADMIN_EMAIL" --password "$ADMIN_PASSWORD" \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["created"])')
if [ "$created" = "True" ]; then
echo "::notice::First deployment: admin created from the repo secrets."
echo "Mint the CI key, set the CI_API_KEY secret, push again."
exit 0
fi
if [ -z "${CI_API_KEY:-}" ]; then
echo "::error::Admins exist but CI_API_KEY is not set."
exit 1
fi
export VSKI_API_KEY="$CI_API_KEY" VSKI_DB="$DB_NAME"
"$V" iac apply vski --yes

First run: the admin is created and the job exits green with instructions; everything after the gate is skipped because no key exists yet. Second and later runs: iac apply converges the whole deployment, and any post-apply kick (a population pipeline, a plugin start) belongs right after it in the same job.

  1. Set the repository secrets: VSKI_URL (the deployment’s API URL as reachable from CI), ADMIN_EMAIL / ADMIN_PASSWORD (the default admin, used only while none exists), JWT_SECRET, and every provider key the IaC declares through fromEnv.
  2. Prepare the platform side: DNS for the service domain, the persistent volume, and whatever the deploy job expects.
  3. Push to main. Deploy ships the image; Bootstrap creates the first admin and stops green.
  4. Once, as the human: vski login, then vski keys create ci --roles admin and set it as the CI_API_KEY repository secret. The deployed service can share that key (the deploy job writes it into the runtime environment once the secret exists) or get its own if you want the credentials separated.
  5. Push (or re-run). Bootstrap applies the IaC and the deployment is live. Every later push re-converges; the cron schedules keep data fresh between pushes.

Between pushes, a scheduled workflow can assert the deployment still matches the IaC; vski iac plan exits 2 on drift, which fails the job:

on:
schedule:
- cron: "0 6 * * *"

The docs MCP repository runs exactly this shape: a platform deploy job, the bootstrap job above, an embedApiKey pipeline secret, and the docs-reindex pipeline kicked after every apply. See Tutorial: MCP Server for what it serves and how it is built.