paul
0aeb893c0f
- Added Makefile for orchestrating domain instantiation and management. - Created deploy README for guiding the setup process of the Financial Crime domain. - Introduced domain configuration file for mapping metadata to the deployment template. - Implemented pre-commit hook for validating metadata before commits. - Developed instantiation script to manage the lifecycle of the Financial Crime domain. - Added Salesforce CRM source with associated metadata and transformation files. - Added SAP Fraud Management source with associated metadata and transformation files. - Added Temenos Payment source with associated metadata and transformation files. - Removed obsolete payment event and parties transformation files.
Deploy — Financial Crime domain (Phase 1)
Phase 1 stands up a running, stable Financial Crime data domain from this repo's
MD-DDL metadata plus the semprini-data-domain template, and registers it into the
ecosystem (Keycloak, CoreDNS, Uptime-Kuma, Prometheus, OpenMetadata).
Everything is driven from the metadata + domain.config.yml by
instantiate.py. This repo acts as the template's data root: the
domain is scaffolded into ./domains/, and a few platform-owned pieces are bridged
in via repo-local symlinks (all gitignored).
Prerequisites
- Docker running, and the shared external networks present:
semprini_bus,semprini_internal,semprini_proxy(created by the core + bus stacks). - semprini-core running (Keycloak, CoreDNS, Kuma, Prometheus, step-ca) and
semprini-data management zone (OpenMetadata, Apicurio). These live at
../semprini-coreand../semprini-data. ../semprini-data/.envpopulated with admin creds (KEYCLOAK_ADMIN*,KUMA_API_KEY,KUMA_URL, OpenMetadata config). The driver symlinks this in as./.envand appends the domain's own secrets (generated, never committed).- VPN (headscale/tailscale) to reach the
*.financial-crime.data.internalhosts. - Python deps:
pyyaml(already in.venv).
One-command stand-up
make instantiate # preflight → wire → secrets → scaffold → build → up → ducklake → register → activate → verify
make verify # re-run health checks
make teardown # unregister + stop (reverses everything; core/mgmt left clean)
Individual steps (useful for debugging): make wire secrets scaffold build up ducklake register activate. See python3 deploy/instantiate.py --help-style step
list at the top of instantiate.py.
Metadata gate
make preflight # validate the MD-DDL domain
make install-hooks # install a pre-commit hook that runs preflight on every commit
This project is not on GitHub, so the gate is local (a git pre-commit hook). If the
local Gitea grows Actions later, mirror the same preflight.py financial_crime call.
Naming note
The MD-DDL metadata folder is financial_crime (underscore). new-domain.sh requires
a lowercase + hyphen name, so the infra domain name is financial-crime. Both
uppercase to the same env prefix FINANCIAL_CRIME.
Known blocker (as of 2026-06-14)
make build currently fails on the postgres-ducklake image: the template's
postgres/Dockerfile downloads pg_ducklake / pg_duckpipe .deb packages from
GitHub releases that return 404 — duckdb/pg_ducklake has no releases, and the
real project relytcloud/pg_ducklake / relytcloud/pg_duckpipe have zero published
releases. The ariadne image builds fine.
This is a template (submodule) dependency issue, not a metadata issue — the submodule
is not edited here; it should be raised upstream against semprini-data-domain. Once a
valid artifact source (or prebuilt semprini-postgres-ducklake:16 image) is available,
make instantiate completes the live stand-up unchanged.