sqi Development Guide¶

This document covers local setup, the test and lint workflow, code layout conventions, a step-by-step walkthrough for adding a new REST endpoint, and guides for extending the worker.

Prerequisites¶

Tool	Purpose	Install
Go ≥ 1.26 (the `go` directive in `go.mod` pins 1.26.3)	Build and test	go.dev/dl
Node.js ≥ 24 with npm ≥ 11 (see `.nvmrc` and `web/package.json` `engines`)	Build the web UI bundle embedded in `sqi-server` (`make build` runs it)	nodejs.org or `nvm use`
`gofumpt`	Stricter formatter (superset of `gofmt`)	`go install mvdan.cc/gofumpt@latest`
`goimports`	Import organiser	`go install golang.org/x/tools/cmd/goimports@latest`
`golangci-lint`	Linter suite	golangci-lint.run/usage/install
`lefthook`	Git hook runner	`go install github.com/evilmartians/lefthook@latest`
`pkgsite`	Local pkg.go.dev docs server	`go install golang.org/x/pkgsite/cmd/pkgsite@latest`
Docker	Build and run the container image	docs.docker.com

gofumpt, goimports, and golangci-lint are required at commit time via pre-commit hooks. Install them before running make hooks.

First-time setup¶

git clone https://github.com/uberware/sqi.git
cd sqi

# Install git hooks (gofumpt, goimports, go vet, golangci-lint on every commit)
make hooks

# Build both binaries
make build

# Start sqi-server with default config (SQLite: sqi.db, NATS: 0.0.0.0:4222)
make run

The server is now reachable at http://localhost:8080. Confirm it is healthy:

curl -sf http://localhost:8080/healthz && echo OK
curl -sf http://localhost:8080/readyz  && echo OK

Makefile targets¶

Run make (no arguments) to see all available targets with descriptions.

Target	Description
`make build`	Build `sqi-server` and `sqi-worker` into `./bin/` (builds the web UI first)
`make build-server`	Build `sqi-server` only (builds the web UI first)
`make build-web`	Build the web UI bundle into `web/dist/` (`npm ci` runs only when npm manifests change)
`make run`	Build then run `sqi-server` with default config
`make run-worker`	Build then run a single `sqi-worker` with default config
`make run-workers`	Build then run N `sqi-worker` instances locally (`N=3` default); Ctrl-C stops all
`make test`	Run all tests with the race detector enabled
`make test-cover`	Run tests and print coverage; fails below 70% (the `COVERAGE_MIN` default in the Makefile)
`make test-cover-html`	Open an HTML coverage report in the browser
`make test-integration`	Run integration tests (build tag `integration`)
`make bench`	Run benchmarks
`make lint`	Run `golangci-lint`
`make lint-fix`	Run `golangci-lint --fix`
`make fmt`	Format all Go files with `gofumpt` and `goimports`
`make vet`	Run `go vet ./...`
`make docs`	Serve Go package docs at `localhost:8080` via `pkgsite`
`make changelog`	Regenerate `CHANGELOG.md` from Conventional Commits via `git-cliff` (`VERSION=x.y.z` tags the pending release)
`make hooks`	Install git hooks via `lefthook`
`make clean`	Remove build artifacts and `coverage.out`
`make ci`	Run the full CI sequence (fmt-check, vet, lint, test-cover)

Override the race detector: make test RACE=off

Override the coverage threshold: make test-cover COVERAGE_MIN=50

Running tests¶

# All tests, race detector on (default)
make test

# Specific package
go test -race ./internal/scheduler/...

# Single test function
go test -race -run TestScheduler_LicenseGating ./internal/scheduler/...

# With verbose output
go test -race -v ./internal/openjd/...

# Integration tests (require the integration build tag)
make test-integration

# Fuzz targets (run for 30 seconds each)
go test -fuzz=FuzzParse         -fuzztime=30s ./internal/openjd/...
go test -fuzz=FuzzRESTPayloads  -fuzztime=30s ./internal/api/...

Code layout¶

sqi/
├── cmd/
│   ├── sqi-server/        Entry point, Cobra subcommands (serve, migrate, backup, config, version)
│   └── sqi-worker/        Worker entry point
├── internal/
│   ├── api/               HTTP router, REST handlers, WebSocket upgrade, OpenAPI spec
│   ├── bus/               Typed NATS JetStream client wrapper
│   ├── config/            Typed config struct, layered loader
│   ├── discovery/         mDNS _sqi._tcp responder
│   ├── health/            /healthz and /readyz handlers
│   ├── log/               slog helpers
│   ├── metrics/           Prometheus metric definitions
│   ├── middleware/         HTTP middleware (logging, metrics, versioning)
│   ├── openjd/            OpenJD parser, validator, parameter-space expansion
│   ├── scheduler/         Assignment loop, worker registry, heartbeat sweep
│   ├── server/            Process boot, graceful shutdown orchestration
│   ├── store/             Store interface + SQLite implementation + migrations
│   ├── ui/                Embeds web/dist; SPA fallback handler
│   ├── version/           Build metadata (version, commit, date, Go version)
│   ├── worker/            sqi-worker binary internals (executor, lease, heartbeat, capabilities, …); only worker/protocol is shared with the server
│   └── ws/                WebSocket hub, subscription management, scheduler-driven fanout
├── pkg/                   Public Go API (currently empty; see pkg/doc.go)
├── api/                   Source-of-truth specs: OpenAPI 3.1, JSON schemas
├── web/                   Frontend source; web/dist is embedded
├── config/                Example config files
├── deploy/                Docker and infrastructure manifests
├── docs/                  Human-readable documentation
├── scripts/               Utility scripts (SBOM, signing, etc.)
└── test/                  Integration test harness

Key conventions¶

No cross-imports between internal packages at the same level. The dependency direction is: cmd → internal/server → internal/api, internal/scheduler → internal/store, internal/bus

Interfaces over concrete types at package boundaries. Handlers receive the store.Store interface, not *sqlite.Store, so tests can inject a fake.

One file per route group. internal/api/jobs.go, internal/api/tasks.go, internal/api/workers.go, etc. Each file owns its handler struct, wire-format types, and route-mounting helper.

Table-driven tests. Tests use []struct{ name, input, want } slices with t.Run(tc.name, ...) rather than separate test functions per case.

SPDX header on every source file.

// SPDX-License-Identifier: AGPL-3.0-or-later

Conventional Commits. The commit-msg hook enforces the format: type(scope)?: description. Valid types: feat fix docs style refactor test chore build ci perf revert.

Changelog. CHANGELOG.md is a generated artifact — do not edit it by hand. git-cliff derives it from the Conventional Commit history (config in cliff.toml), so the way to change an entry is to change the commit message. feat/fix/perf/refactor/build/ci/docs/test commits become changelog entries grouped by type; chore and release commits are omitted. A commit that doesn't parse as a Conventional Commit is skipped entirely, so it never reaches the changelog. Regenerate with:

make changelog                 # refresh, leaving pending commits under [Unreleased]
make changelog VERSION=0.3.0   # roll pending commits into a dated 0.3.0 section

Install git-cliff first (brew install git-cliff, or see https://git-cliff.org/docs/installation). The release workflow also runs git-cliff automatically at tag time and bundles the result into the release archives — but that copy is not committed back, so the tracked CHANGELOG.md is refreshed by hand during release prep (see docs/release-runbook.md).

Adding a new REST endpoint¶

This walkthrough adds a hypothetical GET /api/v1/jobs/{id}/steps endpoint as a concrete example. Follow the same pattern for any new route.

Step 1 — Add the handler method¶

Open (or create) the relevant file in internal/api/. For a new route on an existing resource, add a method to the existing handler struct. For a wholly new resource, create internal/api/<resource>.go following the pattern in jobs.go.

// internal/api/steps.go

// SPDX-License-Identifier: AGPL-3.0-or-later

package api

// GET /api/v1/jobs/{id}/steps
func (h *jobHandler) listSteps(w http.ResponseWriter, r *http.Request) {
    jobID := chi.URLParam(r, "id")
    ctx := r.Context()

    steps, err := h.store.ListStepsForJob(ctx, jobID)
    if err != nil {
        if errors.Is(err, store.ErrNotFound) {
            writeError(w, r, http.StatusNotFound, "job not found")
            return
        }
        h.logger.ErrorContext(ctx, "list steps", slog.String("job_id", jobID), slog.Any("err", err))
        writeError(w, r, http.StatusInternalServerError, "internal error")
        return
    }

    writeJSON(w, http.StatusOK, stepListResponse{
        Items: stepsToResponse(steps),
        Total: len(steps),
    })
}

Step 2 — Wire the route in the router¶

Open internal/api/router.go and find the /api/v1 sub-router block. Add the new route alongside the related routes:

// inside the r.Route("/api/v1", ...) block:
r.Get("/jobs/{id}/steps", jobs.listSteps)

Step 3 — Add or extend the Store interface¶

The Store interface is composed from per-aggregate sub-interfaces. Each sub-interface lives in its own file under internal/store/ (e.g. job.go, task.go, step.go). Add the new method to the relevant sub-interface file:

// internal/store/step.go — add inside the StepStore interface
// ListStepsForJob returns all steps for the given job, ordered by step_order.
// Returns ErrNotFound when no job with that ID exists.
ListStepsForJob(ctx context.Context, jobID string) ([]Step, error)

Then implement it in internal/store/sqlite/step.go using a prepared statement, and add a corresponding stub to the in-memory fake in internal/store/fake/store.go so existing tests keep compiling.

Not every new store method is REST-triggered. The auto-retry + failure-limit feature added RecordTaskFailure, RequeueTaskForRetry (internal/store/sqlite/task.go), and ParkJob (internal/store/sqlite/job.go) purely for the scheduler's own internal use (internal/scheduler/failure.go's handleTaskFailed) — no handler in internal/api/ calls them directly. They still follow the same shape as REST-triggered methods: declared on the relevant sub-interface (TaskStore/JobStore), implemented in internal/store/sqlite/, and stubbed in internal/store/fake/ so scheduler tests can inject the fake. Layered policy resolution that isn't itself a store method — e.g. picking the effective retry policy from Job → Queue → Farm → server default — is a plain package-level helper next to its caller rather than a store method: see resolveRetryPolicy in internal/scheduler/retrypolicy.go. Follow this pattern for scheduler-internal state changes that don't need a REST surface of their own.

Every terminal non-success must leave a reason. task_attempts.message is the per-attempt reason (next to exit_code); tasks.failure_reason denormalizes the latest terminal reason onto the task (mirroring unschedulable_reason), cleared on retry. Two TaskStore methods write it: SetTaskFailureReason(ctx, id, reason) (unconditional) and SetTaskFailureReasonIfEmpty(ctx, id, reason) (a no-op, not an error, when the task already carries a reason). FailureReasonSummary(ctx, jobID) aggregates a job's failed tasks by reason (FailedCount, DominantReason, DistinctReasons) for the job-detail failure banner. The rule for new code: any new code path that drives a task to a terminal failed or canceled must call one of the two setters — reach for SetTaskFailureReasonIfEmpty whenever a more-specific reason may already be set by another path (the pattern cascade-cancel and user-cancel use so cascade always wins regardless of ordering), and the unconditional SetTaskFailureReason only when your path is authoritative. See the durable-failure-reason table for every existing path and its reason string.

Step 4 — Update the OpenAPI spec¶

Add the new path to internal/api/openapi.yaml:

  /jobs/{id}/steps:
    parameters:
      - $ref: "#/components/parameters/id"
    get:
      tags: [jobs]
      operationId: listSteps
      summary: List steps for a job
      responses:
        "200":
          description: Ordered list of steps with status.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/StepList"
        "404":
          $ref: "#/components/responses/NotFound"

Step 5 — Write tests¶

Add a test file (or extend an existing one) in internal/api/:

// internal/api/steps_test.go

func TestListSteps(t *testing.T) {
    cases := []struct {
        name   string
        jobID  string
        want   int // expected HTTP status
    }{
        {"job exists",     existingJobID, http.StatusOK},
        {"job not found",  "nonexistent", http.StatusNotFound},
    }

    for _, tc := range cases {
        t.Run(tc.name, func(t *testing.T) {
            rec := httptest.NewRecorder()
            req := httptest.NewRequest(http.MethodGet, "/api/v1/jobs/"+tc.jobID+"/steps", nil)
            router.ServeHTTP(rec, req)
            if rec.Code != tc.want {
                t.Fatalf("got %d, want %d", rec.Code, tc.want)
            }
        })
    }
}

Run the tests: go test -race ./internal/api/...

Step 6 — Run lint and format¶

make fmt
make lint

Fix any issues before committing.

Adding a product¶

Products are thin catalog entries that wrap an OpenJD template. See docs/products.md for the full concept and REST surface.

Built-in products¶

Built-ins live in internal/product/builtins/*.yaml and are compiled into the binary via //go:embed in internal/product/builtins.go. At process init, loadBuiltins reads every .yaml file in that directory, calls ParseDefinition on each (which re-serializes the inline template and validates it via openjd.Parse + openjd.ValidateWithOptions), and panics if any file is malformed. To add a new built-in:

Create internal/product/builtins/<name>.yaml using the definition file format — metadata fields (name, title, description, category, version) plus an inline template: key containing a full jobtemplate-2023-09 OpenJD template.

Stamp the file with the SPDX comment header:

# SPDX-License-Identifier: AGPL-3.0-or-later

Run go test ./internal/product/... — the TestBuiltins_LoadValidateAndStamp test validates every built-in definition and will catch template errors.
Built-in names are reserved; POST /api/v1/products rejects names that shadow a built-in.
New default/built-in product or reference preset requiring a software capability tag → add a matching built-in detector in internal/worker/capabilities/builtins/; the TestBuiltinDetectors_CoverPresets invariant must stay green (see Adding a built-in DCC detector below).

Custom products¶

Custom products are created at runtime through the REST API and stored in SQLite (the products table). They are mutable (PUT/DELETE) and appear in GET /api/v1/products merged with the built-ins.

curl -X POST http://localhost:8080/api/v1/products \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "my-render",
    "title": "My Renderer",
    "version": "1.0.0",
    "template": "specificationVersion: jobtemplate-2023-09\nname: MyRender\nsteps:\n  - name: Run\n    script:\n      actions:\n        onRun:\n          command: render\n",
    "format": "yaml"
  }'

The server validates the template (via product.ValidateTemplate) before writing; a malformed template returns 400 Bad Request.

Local docs¶

Serve Go package docs locally with pkgsite:

make docs
# Opens http://localhost:8080 with pkg.go.dev-style rendering

Environment variables for local overrides¶

Variable	Purpose
`SQI_HTTP_ADDR`	Override the listen address (e.g. `127.0.0.1:9090`)
`SQI_STORE_SQLITE_PATH`	Point at a specific database file
`SQI_LOG_LEVEL`	Set to `debug` for verbose output
`SQI_LOG_FORMAT`	Set to `text` for human-readable logs
`SQI_HTTP_ENABLE_PPROF`	Set to `true` to enable profiling endpoints

Example — run with debug logging against a test database:

SQI_LOG_LEVEL=debug SQI_LOG_FORMAT=text SQI_STORE_SQLITE_PATH=/tmp/test.db \
  ./bin/sqi-server serve

Pre-commit hooks¶

The hooks installed by make hooks run automatically on every git commit:

gofumpt — formats staged Go files.
goimports — organises imports (stdlib / external / internal groups).
go vet — basic correctness checks.
golangci-lint — full linter suite with auto-fix.

A failing hook blocks the commit and prints the fix command. To bypass for a WIP commit: git commit --no-verify.

To skip the slow golangci-lint step locally while keeping the others, add a lefthook-local.yml (not committed):

pre-commit:
  commands:
    golangci-lint:
      skip: true

Troubleshooting¶

Build fails with "toolchain not found" — the go.mod file pins an exact Go toolchain. Install the matching version from go.dev/dl or run go install golang.org/dl/go1.26.3@latest && go1.26.3 download.

Tests fail with "database is locked" — SQLite WAL mode is used, but concurrent tests that share the same file can still conflict. Ensure each test that opens a database uses a unique temporary path:

db := t.TempDir() + "/test.db"

golangci-lint not found — install it following the official instructions. The go install golangci-lint method is not supported by the project.

gofumpt or goimports not found after installing — ensure $(go env GOPATH)/bin is on your $PATH:

export PATH="$PATH:$(go env GOPATH)/bin"

sqi-worker development¶

Running a worker locally against a dev server¶

Start sqi-server in one terminal:

make run
# or: ./bin/sqi-server serve

The server exposes NATS on 0.0.0.0:4222 (all interfaces) by default. In a second terminal, start the worker pointing at it with debug logging:

SQI_WORKER_NATS_URL=nats://127.0.0.1:4222 \
SQI_WORKER_DISCOVERY_ENABLE_MDNS=false \
SQI_WORKER_LOG_FORMAT=text \
SQI_WORKER_LOG_LEVEL=debug \
  ./bin/sqi-worker start

The worker logs its worker ID, connected NATS URL, and detected capabilities at startup. It will appear under Workers in the web UI at http://localhost:8080 within a few seconds.

To validate configuration without connecting:

SQI_WORKER_NATS_URL=nats://127.0.0.1:4222 \
  ./bin/sqi-worker start --dry-run

To run both components together in a single shell session:

# Terminal 1
make run

# Terminal 2
make build && \
SQI_WORKER_NATS_URL=nats://127.0.0.1:4222 \
SQI_WORKER_DISCOVERY_ENABLE_MDNS=false \
SQI_WORKER_LOG_FORMAT=text \
  ./bin/sqi-worker start

Running multiple workers locally¶

To simulate a multi-worker farm on one machine — for exercising the scheduler, capability matching, or queue routing — use the run-workers target. With sqi-server running (make run in another terminal), start N workers:

make run-workers          # 3 workers (the default)
make run-workers N=5      # 5 workers

Each instance gets a distinct identity so they don't collide: a unique name (worker-1, worker-2, …), its own data dir under ./.run/workers/worker-<i> (holding a separate worker.id UUID), and its own metrics port (9091, 9092, …). Ctrl-C stops them all. Inherited SQI_WORKER_* env vars still apply, so you can layer on shared config — e.g. point them at an explicit NATS URL instead of mDNS discovery:

SQI_WORKER_NATS_URL=nats://127.0.0.1:4222 make run-workers N=5

Tunable Makefile variables: N (or WORKERS), WORKER_METRICS_BASE_PORT, and WORKER_DATA_ROOT. See Running multiple workers on one host for the underlying settings and a manual (non-Make) equivalent.

Worker tests:

# Unit tests for all worker packages
go test -race ./internal/worker/...

# Specific package
go test -race ./internal/worker/executor/...

# Integration test: boots a full server + worker binary
make test-integration

Writing a new executor type¶

The task executor lives in internal/worker/executor. It is wired into the pull loop via the pull.TaskDispatcher interface and into the heartbeat publisher via heartbeat.StateSource. To change how tasks execute, you implement a new type that satisfies these interfaces.

The relevant interfaces (defined in internal/worker/pull/pull.go and internal/worker/heartbeat/heartbeat.go):

// pull.TaskDispatcher — called by the pull loop for each incoming assignment.
type TaskDispatcher interface {
    Dispatch(ctx context.Context, msg *protocol.AssignMsg) error
}

// heartbeat.StateSource — queried by the heartbeat publisher on each tick.
type StateSource interface {
    ActiveTaskCount() int
    ActiveTaskIDs() []string
    LastAssignmentAt() *time.Time
}

Steps to add a new executor type:

Create the executor package — add a new file or sub-package under internal/worker/executor/, e.g. internal/worker/executor/container/container.go.
Implement the interfaces — your type must implement at minimum pull.TaskDispatcher and heartbeat.StateSource. It also needs to implement cancel.TaskCanceler if you want NATS-driven cancellation:

// cancel.TaskCanceler
type TaskCanceler interface {
    Cancel(taskID string) bool
}

Publish status messages — inject *status.Publisher and call Running, Terminal, and (on shutdown) ShutdownFailed. Match the existing executor's publish points:
Running immediately after the workload starts.
Terminal ("succeeded", "failed", "canceled") after it exits.
Wire it in cmd/sqi-worker/start.go — replace the executor.New(...) call with your constructor. Wire the same statusPub, sessionMgr, metrics, and logPub dependencies.
Add unit tests — create executor_test.go asserting at minimum:
Dispatch returns an error when at capacity.
Status messages are published for success and failure.
DrainAndShutdown (or equivalent) blocks until all goroutines exit.

The existing bare-metal executor (internal/worker/executor/) is the canonical reference implementation.

Adding a new capability tag to auto-detection¶

Auto-detected tags are produced by the internal/worker/capabilities package. Detection is abstracted behind the Probe interface, with platform-specific implementations in probe_linux.go, probe_darwin.go, probe_windows.go, and probe_other.go.

Steps to add a new auto-detected tag:

Add a method to the Probe interface in internal/worker/capabilities/capabilities.go:

type Probe interface {
    OS() string
    OSVersion() string
    CPUCount() int
    RAMMb() int
    GPUInfo() GPUInfo
    // Add your new method:
    IsNVLink() bool
}

Implement it on *defaultProbe in the appropriate platform file(s):
If the detection works identically on all platforms (e.g., a runtime package call), add a single implementation in probe_default.go.
If it is platform-specific, add the real implementation in the relevant file and a false/zero stub in the others:

// probe_linux.go
func (*defaultProbe) IsNVLink() bool { return linuxIsNVLink() }

// probe_darwin.go, probe_windows.go, probe_other.go
func (*defaultProbe) IsNVLink() bool { return false }

Consume the result in Detect in capabilities.go. If the tag is a simple presence flag, add it to c.Tags:

func Detect(p Probe) Capabilities {
    // ... existing fields ...
    if p.IsNVLink() {
        c.Tags["nvlink"] = ""
    }
    return c
}

For a key=value tag:

if v := p.SomeString(); v != "" {
    c.Tags["some_key"] = v
}

Update the test probe in detect_test.go — add the new method to the fakeProbe struct used in table-driven tests:

type fakeProbe struct {
    // ... existing fields ...
    isNVLink bool
}
func (f fakeProbe) IsNVLink() bool { return f.isNVLink }

Add table rows covering the true and false cases.

Document the new tag in docs/worker-capabilities.md under the "Auto-detected tags" section.
Run and format:

go test -race ./internal/worker/capabilities/...
make fmt && make lint

Adding a built-in DCC detector¶

This is a separate, declarative detection path from the Probe-based hardware tags above — see docs/worker-capabilities.md for the full picture (how detection runs, the tag/version model, precedence) and the Writing custom detectors reference for the detector schema.

Paired-change convention: every new default/built-in product or reference preset that requires a software capability tag ships a matching built-in detector in internal/worker/capabilities/builtins/. For presets/sqi/*.yaml specifically, this is test-enforced: TestBuiltinDetectors_CoverPresets in internal/worker/capabilities/builtins_test.go cross-checks every attr.worker.tag.<x> required by presets/sqi/*.yaml against the tags emitted by the built-in detectors, so a new reference preset cannot ship without a matching detector or the test fails. (The test currently only scans presets/sqi/, not internal/product/builtins/; apply the same convention by hand to a new built-in product until that coverage is extended.)

Steps to add a new built-in detector:

Create internal/worker/capabilities/builtins/<name>.yaml with the detector schema — tag, checks (each check exactly one of exe/path_glob/env/registry, with an optional per-check os gate), and an optional version.from regex. Use the existing built-ins (maya.yaml, nuke.yaml, houdini.yaml, blender.yaml) as reference — they use the identical schema as a custom detector.

Stamp the file with the SPDX comment header:

# SPDX-License-Identifier: AGPL-3.0-or-later

Run go test ./internal/worker/capabilities/... — TestBuiltinDetectors_CoverPresets and the built-in load/validate tests will catch a malformed detector or a still-uncovered preset tag.
make fmt && make lint.

sqi-sdk (Python) development¶

The Python client lives at clients/python/ (import name sqi_client). It is a separate toolchain from the Go code — it has its own virtualenv, dependencies, and Make targets — so its commands do not overlap with the Go ones above.

First-time setup¶

# Create clients/python/.venv and install sqi-sdk editable with all extras
# (yaml, ws, dev — ruff, mypy, pytest, pytest-cov, pytest-timeout, respx,
# websockets):
make py-install

Day-to-day commands¶

make py-fmt              # format with ruff
make py-lint             # ruff lint
make py-typecheck        # mypy (src targets 3.9, tests target 3.13)
make py-test             # unit tests with coverage gate
make py-check            # full check-only gate: ruff format --check, ruff check, mypy, pytest
make py-build            # build sdist + wheel into clients/python/dist/

make py-check is the pre-PR gate; it matches what CI runs for the client. Run it (not bare pytest) from the repo root, or cd clients/python first — pytest must load clients/python/pyproject.toml for the markers, coverage gate, and addopts to apply.

Integration tests against locally-built binaries¶

The @pytest.mark.integration suite boots the real sqi-server and sqi-worker binaries as subprocesses (skipped by default; coverage off because the subset doesn't meet the unit gate):

# Builds the binaries first, then runs pytest -m integration:
make py-test-integration

# Or, if the binaries are already current, run directly:
cd clients/python && .venv/bin/pytest -m integration --no-cov

The harness finds the binaries at <repo-root>/bin/sqi-server and bin/sqi-worker by default; override with SQI_SERVER_BIN / SQI_WORKER_BIN. If a binary is missing the whole marker is skipped with a clear message.

Adding a new endpoint wrapper¶

To wrap a new server endpoint in the client, mirror the existing layering:

Transport — reach for the shared helpers in client.py rather than calling httpx directly: _request / _request_json (path joining under /api/v1, None-param dropping, typed-error mapping, GET-only retry), parse_page for paginated responses, and the generic _CrudResource for standard create/get/update/delete resources.
Model — add or extend a frozen dataclass in models.py matching the OpenAPI component schema field-for-field, with a tolerant from_dict. Export it from __init__.py's __all__.
Method — add the public method to SqiClient (keyword-only options, fully type-annotated, Google-style docstring). Accept status filters as either the enum or its wire string via _enum_value.
Tests — add respx-mocked unit tests in tests/ (one file per resource group), and extend the integration suite if it exercises a real execution path.
Docs — document the method in docs/python-client.md with a short runnable example.

Run make py-check before opening the PR. The authoritative request/response shape is always the OpenAPI spec (internal/api/openapi.yaml); when it and the server diverge, treat the spec as authoritative and flag the discrepancy.

The sqi-submitter package¶

clients/submitter/ (import name sqi_submitter, distribution sqi-submitter) is a second, separate Python package depending on sqi-sdk — its own virtualenv, pyproject.toml, and gate, independent of both the Go build and clients/python. Full user-facing reference: docs/dcc-submitters.md.

clients/submitter/
  pyproject.toml            sqi-submitter; requires-python >=3.9; deps: sqi-sdk
  src/sqi_submitter/
    core/                   no Qt, no DCC imports — session, form model, pre-fill, HostAdapter
    qt/                      Qt only, imported lazily (PySide6, falls back to PySide2)
    hosts/{maya,houdini,nuke,blender}/   adapter + launch glue per host
  tests/                     pytest; tests/integration/ is env-gated (see below)
presets/sqi/                 the six reference preset YAML fixtures, validated by
                              internal/product/dccpresets_test.go
presets/testing/             the four test/QA preset fixtures (test-render, test-steps;
                              bash + PowerShell), validated by
                              internal/product/testingpresets_test.go

Gate command line (from clients/submitter, with sqi-sdk resolvable — pip install -e ../python first if not already installed):

ruff format --check . && ruff check . && mypy src && pytest -q

tests/integration/ is skipped unless SQI_TEST_SERVER_URL points at a live sqi-server; the Blender render smoke inside it additionally requires SQI_TEST_BLENDER=1 and a local Blender install.

Adding a DCC adapter (a fifth Python-based host) is a HostAdapter subclass plus launch glue — walked through in docs/dcc-submitters.md §6, Writing a new adapter.