Skip to content

Customization Handbook

Reshape the dashboards onto your own backend without rewriting the visual layer. Currently rendered against Your Institution (spec_example).

This handbook is for the developer or product owner dropping the four shipped dashboards (L1 Reconciliation, L2 Flow Tracing, Investigation, Executives) onto their own data — not the Data Integration ETL engineer loading the two base tables (that's the Data Integration Handbook).

The product is built around a small, deliberate set of customer-mutable surfaces. Swap the SQL behind a dataset, swap the colors on a theme, point the dashboards at a different database or extend the metadata contract — each happens in ONE place, with one test that catches the regression. The visual, filter and drill layer above the data binds to a stable column contract; you change WHAT fills the contract, not HOW the visuals consume it.

What stays stable

These are the surfaces this handbook documents. They're the parts of the product that are deliberately small and don't churn under new persona work or dashboard redesigns:

  • Two base tables — transactions + daily_balances. Every app reads from these. Adding a new persona or a new exception check doesn't add a new base table; it adds a new dataset SQL view over the same two tables.
  • DatasetContract — column name + type list per dataset. The SQL query is ONE implementation; you can swap the SQL while preserving the contract and the visual layer keeps working untouched.
  • metadata JSON column — the per-app extension point. Add keys without schema migrations; read them with portable JSON_VALUE syntax.
  • Theme presets — color tokens, fonts, naming prefix. Your brand drops in via one preset registration.
  • config.yaml + CLI — database URL, table prefix, theme, all configurable from one file (or env vars). The CLI itself (recon-gen dashboards / studio for serving the apps, schema apply / data apply for the demo DB, audit apply for the regulator PDF) is the customer-facing surface and won't change shape without a major version bump.

What this handbook does NOT cover

  • Per-visual customization. Each shipped app's visuals evolve as the L1/L2 model + Investigation/Executives stories iterate. Document specific visuals once they stabilize.
  • Per-dataset SQL enumeration. Each dataset's SQL is in apps/<app>/datasets.py (e.g. apps/l1_dashboard/datasets.py, apps/investigation/datasets.py); read it as the source of truth. The pattern for REPLACING it is documented here once.
  • Per-sheet layout. Sheet structure is part of the active product surface and may shift under integrator-driven redesigns.

How filters work

A filter — a date range, a status dropdown, a sigma-threshold slider — is a parameter substituted into the dataset's SQL WHERE clause, NOT a UI-side filter applied after the fact. The dataset query already carries a <<$paramName>> placeholder (or, for the universal date range, a {date_filter} slot the build_dataset helper fills); when the analyst moves a control, the renderer translates that placeholder into a :param_name bind, runs the narrowed query and the database returns the narrowed set.

Two consequences worth knowing if you're swapping SQL behind a dataset:

  1. Keep the placeholders. If the dataset you're replacing has a {date_filter} slot or a <<$pSomething>> in its WHERE, your replacement SQL needs the equivalent — that's how the date control / dropdown / slider stays wired. The DatasetContract still locks the projection (column names + types); the placeholders are the parameter contract on top of it.
  2. It narrows at the database. A filtered view fetches fewer rows, not "all rows then hide some" — relevant when you're sizing matview/index work for your own data volumes.

This is internal renderer architecture (a v9.0.0 change). It does NOT touch your config.yaml or your L2 instance YAML — the theme: block, the optional persona: block, your rails / chains / accounts / limit schedules are all unchanged. If you carried a customization across the v9.0.0 line, your YAML didn't need to move; only the generated dashboard definitions did.

Setup

Get the dashboards running against your data

How do I map my production database to the two base tables?

Pattern-level mapping from your source system to transactions + daily_balances. The first walkthrough a new product owner reads.

How do I serve the dashboards against my data?

recon-gen dashboards for just the apps, recon-gen studio for the editor + implementation tools on top — both read your config.yaml database and L2 instance, no deploy step.

Reskinning + extending

Make the product fit your environment

How do I reskin the dashboards for my brand?

Theme preset registry, color tokens (accent / primary_fg / link_tint), font sizes, the analysis_name_prefix for demo-vs-prod naming.

How do I swap the SQL behind a dataset without breaking the visuals?

The DatasetContract binding contract, the contract test that locks projection-vs-contract, when SQL swap is safe and when it forces a contract change.

How do I add an app-specific metadata key?

Reading metadata from dataset SQL, when to surface a key as a column vs. a filter, cross-link to the ETL-side walkthrough for the write path.

How do I extend the schema with a new rail_name or account_role?

Adding to the canonical value lists, downstream impact on filter dropdowns, why no new tables are needed.

How do I brand my handbook prose?

The optional persona: YAML block — institution name + acronym, stakeholders, GL account labels, merchants, flavor literals. Substitutes via vocab Jinja references at mkdocs render. Skip the block to keep neutral fallback prose.

Testing your customization

Catch regressions before they ship

How do I run the test suite against my customized dataset SQL?

pytest layout, the DatasetContract assertion pattern, when to add an e2e test vs. a unit test for your custom SQL.

Optional ETL extensions

A small set of feed columns are OPTIONAL — leave them NULL and the downstream views fall back to a sensible default; populate them when you can give the dashboard rail-accurate signal:

  • expected_complete_at (TIMESTAMP on transactions) — when your ETL knows the rail's settlement window (instant: same-day; ACH: T+2; cards: T+3), set it per leg. The dashboard's data-driven is_late predicate fires off this column with a posting + INTERVAL '1 day' fallback when it's NULL. Adopt one rail at a time; until then, every row uses the one-day default. Full contract: Lateness as data in the schema doc, plus the expected_complete_at ETL section in the ETL handbook.
  • metadata (JSON TEXT on transactions and daily_balances) — the per-app extension column. Add app-specific keys without schema migrations; the How do I add an app-specific metadata key? walkthrough above is the read/write contract.

The L2-fed pattern (M.2b)

The sections above cover the per-surface customization path — swap a dataset's SQL, register a theme, point the dashboards at your database. The L2-fed pattern is the recommended approach going forward: declare your institution as an L2 instance YAML once and the L1 dashboard renders against it generically.

1. Write your L2 instance YAML

Mirror tests/l2/spec_example.yaml for shape. The L2 declares:

  • Accounts + roles, scopes (internal/external), parents
  • Account templates (role classes that materialize at runtime)
  • Rails — one-leg / two-leg / aggregating; per-rail aging caps (max_pending_age, max_unbundled_age)
  • Transfer templates — multi-leg shared transfers with closure
  • Chains — transfer-of-transfers ordered flows; XOR groups
  • LimitSchedules — per-(parent_role × rail_name) daily caps
  • A description field on every primitive (surfaces as TextBox prose on the dashboard)

Rich descriptions matter — a prose seam pulls them straight into the dashboard's Getting Started, Drift, Limit Breach and L1 Exceptions text boxes. Switching the L2 instance switches the prose without touching dashboard code.

2. Apply the prefixed schema

from recon_gen.common.l2 import emit_schema, load_instance

instance = load_instance("path/to/myorg.yaml")
sql = emit_schema(instance, prefix="myorg")  # Z.C — prefix is now a kwarg
# Pipe to psql, or:
import psycopg
conn = psycopg.connect(your_db_url)
with conn.cursor() as cur:
    cur.execute(sql)

Every table, view and matview in the emitted DDL is prefixed by the prefix= value you pass (typically cfg.db.table_prefix), producing e.g. myorg_transactions, myorg_drift, myorg_stuck_pending. Multiple deployments of the same L2 instance coexist in one database via distinct prefixes.

3. Refresh the matviews after every load

The L1 invariant views are MATERIALIZED for dashboard performance. After every batch insert into spec_example_transactions or spec_example_daily_balances, refresh the dependent matviews:

from recon_gen.common.l2 import refresh_matviews_sql
sql = refresh_matviews_sql(instance)
# base-table ANALYZEs, then a REFRESH + ANALYZE pair per matview in
# dependency order — stats cascade with the refreshes so no matview
# plans against an unanalyzed upstream

4. Serve the L1 dashboard against your instance

Point the dashboards server at your config + your instance YAML:

recon-gen dashboards -c run/config.yaml --l2 path/to/myorg.yaml
# → http://127.0.0.1:8765/dashboards

Want the editor + implementation tools (unified diagram, L2 editor, data-shaping panel) on top? Run recon-gen studio instead — same flags.

5. Verify with the test runner

The ./run_tests.sh up_to=db --variants=sp_pg_lo chain (or sp_or_lo for Oracle) applies the schema, plants the canonical seed scenarios, refreshes matviews and asserts each L1 invariant view returns rows for every planted scenario via the e2e harness tests. For your own instance, write a sibling myorg_seed.py declaring your scenarios via the generic plant primitives (DriftPlant, OverdraftPlant, LimitBreachPlant, StuckPendingPlant, StuckUnbundledPlant, SupersessionPlant); register it as the L2 instance in your config, then run the chain to PASS-gate your customization before touching the dashboard.

For the full L1 invariant inventory (what each spec_example_* view returns + its SHOULD-constraint motivation), see L1 Invariants.

Reference