paul
0aeb893c0f
- Added Makefile for orchestrating domain instantiation and management. - Created deploy README for guiding the setup process of the Financial Crime domain. - Introduced domain configuration file for mapping metadata to the deployment template. - Implemented pre-commit hook for validating metadata before commits. - Developed instantiation script to manage the lifecycle of the Financial Crime domain. - Added Salesforce CRM source with associated metadata and transformation files. - Added SAP Fraud Management source with associated metadata and transformation files. - Added Temenos Payment source with associated metadata and transformation files. - Removed obsolete payment event and parties transformation files.
65 lines
3.1 KiB
Markdown
65 lines
3.1 KiB
Markdown
# Deploy — Financial Crime domain (Phase 1)
|
|
|
|
Phase 1 stands up a *running, stable* Financial Crime data domain from this repo's
|
|
MD-DDL metadata plus the `semprini-data-domain` template, and registers it into the
|
|
ecosystem (Keycloak, CoreDNS, Uptime-Kuma, Prometheus, OpenMetadata).
|
|
|
|
Everything is driven from the metadata + [domain.config.yml](domain.config.yml) by
|
|
[instantiate.py](instantiate.py). This repo acts as the template's *data root*: the
|
|
domain is scaffolded into `./domains/`, and a few platform-owned pieces are bridged
|
|
in via repo-local symlinks (all gitignored).
|
|
|
|
## Prerequisites
|
|
|
|
- **Docker** running, and the shared external networks present: `semprini_bus`,
|
|
`semprini_internal`, `semprini_proxy` (created by the core + bus stacks).
|
|
- **semprini-core** running (Keycloak, CoreDNS, Kuma, Prometheus, step-ca) and
|
|
**semprini-data** management zone (OpenMetadata, Apicurio). These live at
|
|
`../semprini-core` and `../semprini-data`.
|
|
- `../semprini-data/.env` populated with admin creds (`KEYCLOAK_ADMIN*`,
|
|
`KUMA_API_KEY`, `KUMA_URL`, OpenMetadata config). The driver symlinks this in as
|
|
`./.env` and appends the domain's own secrets (generated, never committed).
|
|
- **VPN** (headscale/tailscale) to reach the `*.financial-crime.data.internal` hosts.
|
|
- Python deps: `pyyaml` (already in `.venv`).
|
|
|
|
## One-command stand-up
|
|
|
|
```bash
|
|
make instantiate # preflight → wire → secrets → scaffold → build → up → ducklake → register → activate → verify
|
|
make verify # re-run health checks
|
|
make teardown # unregister + stop (reverses everything; core/mgmt left clean)
|
|
```
|
|
|
|
Individual steps (useful for debugging): `make wire secrets scaffold build up
|
|
ducklake register activate`. See `python3 deploy/instantiate.py --help`-style step
|
|
list at the top of [instantiate.py](instantiate.py).
|
|
|
|
## Metadata gate
|
|
|
|
```bash
|
|
make preflight # validate the MD-DDL domain
|
|
make install-hooks # install a pre-commit hook that runs preflight on every commit
|
|
```
|
|
|
|
This project is not on GitHub, so the gate is local (a git pre-commit hook). If the
|
|
local Gitea grows Actions later, mirror the same `preflight.py financial_crime` call.
|
|
|
|
## Naming note
|
|
|
|
The MD-DDL metadata folder is `financial_crime` (underscore). `new-domain.sh` requires
|
|
a lowercase + hyphen name, so the **infra** domain name is `financial-crime`. Both
|
|
uppercase to the same env prefix `FINANCIAL_CRIME`.
|
|
|
|
## Known blocker (as of 2026-06-14)
|
|
|
|
`make build` currently fails on the **postgres-ducklake** image: the template's
|
|
`postgres/Dockerfile` downloads `pg_ducklake` / `pg_duckpipe` `.deb` packages from
|
|
GitHub releases that return **404** — `duckdb/pg_ducklake` has no releases, and the
|
|
real project `relytcloud/pg_ducklake` / `relytcloud/pg_duckpipe` have **zero published
|
|
releases**. The `ariadne` image builds fine.
|
|
|
|
This is a template (submodule) dependency issue, not a metadata issue — the submodule
|
|
is not edited here; it should be raised upstream against `semprini-data-domain`. Once a
|
|
valid artifact source (or prebuilt `semprini-postgres-ducklake:16` image) is available,
|
|
`make instantiate` completes the live stand-up unchanged.
|