feat: Implement initial setup for Financial Crime domain
- Added Makefile for orchestrating domain instantiation and management. - Created deploy README for guiding the setup process of the Financial Crime domain. - Introduced domain configuration file for mapping metadata to the deployment template. - Implemented pre-commit hook for validating metadata before commits. - Developed instantiation script to manage the lifecycle of the Financial Crime domain. - Added Salesforce CRM source with associated metadata and transformation files. - Added SAP Fraud Management source with associated metadata and transformation files. - Added Temenos Payment source with associated metadata and transformation files. - Removed obsolete payment event and parties transformation files.
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
# Deploy — Financial Crime domain (Phase 1)
|
||||
|
||||
Phase 1 stands up a *running, stable* Financial Crime data domain from this repo's
|
||||
MD-DDL metadata plus the `semprini-data-domain` template, and registers it into the
|
||||
ecosystem (Keycloak, CoreDNS, Uptime-Kuma, Prometheus, OpenMetadata).
|
||||
|
||||
Everything is driven from the metadata + [domain.config.yml](domain.config.yml) by
|
||||
[instantiate.py](instantiate.py). This repo acts as the template's *data root*: the
|
||||
domain is scaffolded into `./domains/`, and a few platform-owned pieces are bridged
|
||||
in via repo-local symlinks (all gitignored).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Docker** running, and the shared external networks present: `semprini_bus`,
|
||||
`semprini_internal`, `semprini_proxy` (created by the core + bus stacks).
|
||||
- **semprini-core** running (Keycloak, CoreDNS, Kuma, Prometheus, step-ca) and
|
||||
**semprini-data** management zone (OpenMetadata, Apicurio). These live at
|
||||
`../semprini-core` and `../semprini-data`.
|
||||
- `../semprini-data/.env` populated with admin creds (`KEYCLOAK_ADMIN*`,
|
||||
`KUMA_API_KEY`, `KUMA_URL`, OpenMetadata config). The driver symlinks this in as
|
||||
`./.env` and appends the domain's own secrets (generated, never committed).
|
||||
- **VPN** (headscale/tailscale) to reach the `*.financial-crime.data.internal` hosts.
|
||||
- Python deps: `pyyaml` (already in `.venv`).
|
||||
|
||||
## One-command stand-up
|
||||
|
||||
```bash
|
||||
make instantiate # preflight → wire → secrets → scaffold → build → up → ducklake → register → activate → verify
|
||||
make verify # re-run health checks
|
||||
make teardown # unregister + stop (reverses everything; core/mgmt left clean)
|
||||
```
|
||||
|
||||
Individual steps (useful for debugging): `make wire secrets scaffold build up
|
||||
ducklake register activate`. See `python3 deploy/instantiate.py --help`-style step
|
||||
list at the top of [instantiate.py](instantiate.py).
|
||||
|
||||
## Metadata gate
|
||||
|
||||
```bash
|
||||
make preflight # validate the MD-DDL domain
|
||||
make install-hooks # install a pre-commit hook that runs preflight on every commit
|
||||
```
|
||||
|
||||
This project is not on GitHub, so the gate is local (a git pre-commit hook). If the
|
||||
local Gitea grows Actions later, mirror the same `preflight.py financial_crime` call.
|
||||
|
||||
## Naming note
|
||||
|
||||
The MD-DDL metadata folder is `financial_crime` (underscore). `new-domain.sh` requires
|
||||
a lowercase + hyphen name, so the **infra** domain name is `financial-crime`. Both
|
||||
uppercase to the same env prefix `FINANCIAL_CRIME`.
|
||||
|
||||
## Known blocker (as of 2026-06-14)
|
||||
|
||||
`make build` currently fails on the **postgres-ducklake** image: the template's
|
||||
`postgres/Dockerfile` downloads `pg_ducklake` / `pg_duckpipe` `.deb` packages from
|
||||
GitHub releases that return **404** — `duckdb/pg_ducklake` has no releases, and the
|
||||
real project `relytcloud/pg_ducklake` / `relytcloud/pg_duckpipe` have **zero published
|
||||
releases**. The `ariadne` image builds fine.
|
||||
|
||||
This is a template (submodule) dependency issue, not a metadata issue — the submodule
|
||||
is not edited here; it should be raised upstream against `semprini-data-domain`. Once a
|
||||
valid artifact source (or prebuilt `semprini-postgres-ducklake:16` image) is available,
|
||||
`make instantiate` completes the live stand-up unchanged.
|
||||
Reference in New Issue
Block a user