Skip to content

Helm: ingestor subchart that submits runs to jobs-manager #86

@saadqbal

Description

@saadqbal

Parent: #85

What

A thin Helm subchart that lets a customer ingest a dataset with helm install. The chart's only job is to package an ingest.yaml and submit it to the cluster's jobs-manager (Ticket client-runtime#21), which owns Job creation.

Scope

Layout

tracebloc-helm-charts/ingestor/ (or client/charts/ingestor/, decide during PR):

  • Chart.yaml, values.yaml
  • templates/configmap-ingest-config.yaml — holds the YAML body
  • templates/post-install-job.yaml — runs once at install time, POSTs the YAML to jobs-manager's /internal/submit-ingestion-run endpoint, exits.
  • No ingestor Job template, no token Secret template. jobs-manager owns those.

Values

```yaml
ingestConfig: | # the ingest.yaml body, schema-validated by jobs-manager
apiVersion: tracebloc.io/v1
kind: IngestConfig
...
image:
digest: sha256:... # pinned, no :latest, no floating tag
serviceAccount:
create: true # ingestor SA, granted ingestion rights via the chart's authz config
jobsManager:
endpoint: http://jobs-manager.tracebloc.svc.cluster.local
authzConfigMapRef: tracebloc-ingest-authz # SA → table mapping, owned by jobs-manager release
```

Customer UX

```bash
helm install my-dataset tracebloc/ingestor -f ingest.yaml
```

(Helm install renders the post-install hook → jobs-manager validates and creates the Job → ingestion runs.)

Cleanup

  • Helm uninstall removes the ConfigMap and post-install hook.
  • The Job created by jobs-manager has its own lifecycle and is not owned by the Helm release. Document this clearly to avoid the docs/MIGRATIONS.md class of surprise. Deleting the Helm release does not delete the run history or the local data.

Acceptance criteria

  • helm install on a valid ingest.yaml results in a running ingestor Job (created by jobs-manager) within ~30s.
  • helm install on an invalid ingest.yaml fails at install time with the validation error from jobs-manager visible in the install output.
  • helm uninstall removes the ConfigMap and hook artifacts but leaves the Job and ingested data alone.
  • Documentation in the chart README explicitly calls out the ownership boundary (Helm owns config + hook; jobs-manager owns Job + token + data).

Depends on

  • client-runtime#21 (jobs-manager endpoint)
  • data-ingestors#45 (image digest to pin in values)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions