Skip to main content

CM-DAP

Cross-Module Data Access Policy

ZAYAZ Platform Architecture

1. Purpose​

ZAYAZ is a module-based platform, but it also relies on a shared data foundation. This policy defines when modules may access each other’s data directly via tables and when they must use APIs, events, or exported views instead.

The goal is to:

  • preserve modularity and ownership
  • avoid tight coupling
  • keep analytics and engines efficient
  • make evolution safe at scale

2. Core principle (non-negotiable)​

Modules may freely reference shared platform tables. Module-owned tables must not be directly referenced across modules unless explicitly exported.

This creates a clear boundary between:

  • platform data (shared, stable, foundational)
  • module internals (private, evolving, owned)

3. The three data zones in ZAYAZ​

3.1. Shared Platform Data (direct table access allowed)​

These tables form the relational backbone of ZAYAZ.

Characteristics

  • Stable semantics
  • Used by many modules and engines
  • Low churn
  • Strong consistency requirements

Schemas

  • core_metadata
  • core_signals
  • core_dim
  • core_relations

Typical prefixes

  • ref_
  • sig_
  • dim_
  • rl_

Examples

  • Tenants and identity
  • Standards frameworks and criteria
  • Compute method registries
  • Signal definitions
  • Canonical dimensions

Policy

  • ✅ Other modules may reference these tables directly
  • ✅ Foreign keys and joins are allowed
  • ✅ These tables are considered part of the “platform contract”

3.2. Module-Owned Data (no direct cross-module access)​

These tables represent internal state or behavior of a module.

Characteristics

  • Owned by a single module
  • Subject to frequent change
  • Semantics tied to module logic
  • Not intended as a platform contract

Schemas

  • core_module
  • parts of core_engine (raw engine outputs)

Typical prefixes

  • mod_
  • eng_

Examples

  • Module UI state
  • Workflows and approvals
  • Engine intermediate results
  • Internal caches or staging outputs

Policy

  • ❌ Other modules must not reference these tables directly
  • ❌ No foreign keys from other modules
  • ❌ No assumptions about table shape outside the owning module

3.3. Exported / Consumption Data (controlled sharing)​

When module data must be consumed by other modules, it must be explicitly exported.

Characteristics

  • Designed for reading, not mutation
  • Stable schema
  • Clear ownership and versioning
  • Documented as a contract

Schemas

  • core_marts
  • core_aggregates
  • occasionally core_metadata (for promoted outputs)

Typical prefixes

  • mrt_
  • agg_
  • (sometimes promoted ref_ tables)

Examples

  • Curated engine outputs
  • Aggregated KPIs
  • Cross-module reporting views

Policy

  • ✅ Other modules may read these tables
  • ❌ Other modules may not write to them
  • ✅ Relationships must be documented in relationship_registry

4. API and Event Boundaries (when tables are not enough)​

Modules must use APIs or events instead of direct table access when:

  • data lives in a different database or storage technology
  • the data is operational or transactional
  • the producing module needs freedom to evolve
  • only a subset of information is required
  • latency, caching, or access control matters

Typical use cases

  • Commands (“enable framework”, “run computation”)
  • User actions
  • Workflow transitions
  • Notifications and signals
  • Asynchronous coordination

Rule

If another module depends on the behavior of data, not just its structure, use an API or event.

5. Decision matrix (practical guidance)​

ScenarioDirect table accessAPI / Event
Reading reference data✅❌
Joining on tenants / frameworks✅❌
Reading module internal state❌✅
Triggering behavior❌✅
Cross-module analytics✅ (via exports)❌
Engine execution control❌✅
High-churn internal data❌✅

6. Relationship registry enforcement​

All cross-table dependencies must be registered in relationship_registry.

Allowed cross-module relationships

  • Shared platform tables → any module
  • Exported tables → consumers

Disallowed relationships

  • Module-owned tables → other modules (unless explicitly exported)

If a relationship violates this policy, it must be:

  • redesigned as an API/event interaction, or
  • promoted to an explicit export table

7. Naming and schema rules reinforce this policy​

The policy is reinforced structurally:

  • ref_, sig_, dim_, rl_ → safe to reference
  • mod_, eng_ → private unless exported
  • mrt_, agg_ → read-only cross-module access

If the prefix and schema are correct, the access rule is usually obvious.

One-sentence rule for contributors

If another module needs to depend on the structure, use an API or event. If it only needs to join on stable platform keys, shared tables are fine.

8. Why this policy matters​

Following this policy:

  • keeps modules independently evolvable
  • prevents accidental coupling
  • makes ownership clear
  • keeps analytics fast
  • avoids “giant-ball-of-mud” schemas
  • enables safe automation and documentation

Violating it creates long-term operational risk.

9. Cross-Module Access Matrix (Green / Yellow / Red)​

Below is an explicit Green / Yellow / Red mapping of table_prefix + db_schema combinations for cross-module access in ZAYAZ. All tables in dbdiagram.io should be marked with the correct color.

Correct colors can be seen the the table_overview table.

Legend

  • Green (OK): Other modules may read/join and may define FKs (when same RDBMS boundary).
  • Yellow (Controlled): Read allowed only if explicitly exported/contracted; writes are restricted; FKs usually discouraged.
  • Red (No): Do not reference across modules. Access must be via API/event or an export table.

9.1. Green: Shared Platform Data (Cross-module access OK)​

These combinations are considered part of the platform contract.

db_schemaprefixesAccessNotes
core_metadataref_🟢 GreenCanonical registries and reference definitions (frameworks, criteria, method registries, “latest pointers”).
core_signalssig_🟢 GreenSignal registry is globally shared by design.
core_dimdim_🟢 GreenDimensions are reusable and safe to join from many modules.
core_relationsrl_🟢 GreenPure join tables that link platform concepts (tenant↔framework, etc.).

Green rule: If it’s in Green, it’s eligible for direct joins and (where appropriate) FK constraints.

9.2. Yellow: Controlled Sharing (Allowed only as a deliberate contract)​

These are common sources of accidental coupling—so they’re allowed only with explicit intent.

db_schemaprefixesAccessNotes
core_martsmrt_🟡 YellowRead-optimized, consumer-facing “contract tables”. Cross-module reads encouraged; writes restricted to owner/pipeline.
core_aggregatesagg_🟡 YellowKPI rollups. Read across modules is OK; don’t anchor business logic to their exact shape without versioning.
core_datafact_, ref_ (rare), dim_ (rare)🟡 YellowOnly Green if explicitly declared “platform entity”. Otherwise, treat as controlled to avoid coupling module internals.
core_engineeng_, fact_🟡 YellowEngine outputs may be shared only via curated exports (mrt_/agg_) or explicit “public engine output” designation.
silver / gold*🟡 YellowAnalytics layers: OK to read cross-module, but treat as consumption/BI. Don’t build transactional dependencies.

Yellow rule: If it’s Yellow, cross-module reads require explicit designation (e.g., is_exported=true, contract_version, or “public output” flag in table_overview). No cross-module writes.

9.3. Red: No Direct Cross-Module Access​

These combinations are module-internal or transient by nature.

db_schemaprefixesAccessNotes
core_modulemod_🔴 RedModule-owned state. Must be accessed via API/events or published exports.
core_stagingstg_🔴 RedRaw ingestion data is unstable; never depend on it cross-module.
core_intermediateint_🔴 RedPipeline intermediates; shapes change; not a contract.
core_temptmp_🔴 RedTemporary tables; strictly internal to pipelines.
core_telemetryfact_ (telemetry), data_🔴 RedOperational logs. Expose via observability tooling, not joins in module logic.
bronze*🔴 RedRaw lakehouse layer; never a dependency. Read only for ingestion/processing pipelines.
any schemadata_🔴 RedLegacy/raw general. Allowed only for migrations or deprecation cleanup.

Red rule: If it’s Red, you must not reference it across modules. Use API/event patterns or export a curated table.

9.4. Exceptions and explicit overrides (controlled and auditable)​

Sometimes we’ll need an exception. That’s fine—but it must be explicit.

“Promoted to Platform” override (Yellow → Green)

A table in core_data or core_engine can be treated as Green if:

  • it is declared platform_shared=true in table_overview
  • it has an owner + stability guarantees
  • it has a documented contract (versioned)
  • relationships are registered

“Exported Contract” override (Red → Yellow)

A module may publish data from Red zones by:

  • creating mrt_* or agg_* contract tables
  • optionally materializing from module internals
  • documenting in relationship_registry as derives_from / feeds

9.5. Practical enforcement rules (what reviewers should check)​

When a PR adds a cross-module relationship:

  1. Look up prefix + db_schema for both sides
  2. Apply the matrix:
  • Green → allowed
  • Yellow → allowed only if exported/contracted
  • Red → reject; require export or API/event

9.6. Fields supporting Yellow/Green enforcement​

Operationally fields in table_overview:

  • is_exported (boolean)
  • contract_version (string, e.g. v1)
  • access_scope (enum: platform_shared, module_private, export_contract)
  • owner_module

This prevents “Yellow” from becoming ambiguous.



GitHub RepoRequest for Change (RFC)