Skip to main content
Jira progress: loading…

AIIL-INFO

General Information

1. Retrieval-Augmented Generation (RAG) Architecture​

1.1. Purpose & Role​

The Retrieval-Augmented Generation (RAG) architecture is the foundation of trust in ZAYAZ AI. Unlike general-purpose LLMs that rely on statistical recall, ZAYAZ employs RAG to guarantee that all outputs are:

  • Grounded in authoritative standards, disclosures, and validated data.
  • Traceable with citations that link every statement to a specific framework version and source hash.
  • Aligned with jurisdiction-specific requirements (e.g., ESRS, ISSB, SEC).

The role of this layer is to ensure the AI cannot fabricate knowledge. If no relevant material is found, the system explicitly responds with “insufficient evidence” rather than speculating. This design upholds auditability and regulatory defensibility — essential in the ESG reporting domain.

1.2. Ingestion Pipeline​

  • Source types: ESRS delegated acts, annexes, GRI standards, IPCC datasets, corporate disclosures, verifier attestations.
  • Chunking strategies: hierarchical (section/paragraph/requirement), tuned for regulatory clauses.
  • Embedding & indexing: dual index approach (semantic embeddings + keyword inverted index) for hybrid search.
  • Metadata enrichment:
    • Framework ID (ESRS, ISSB, SEC, etc.)
    • Version ID & date
    • Jurisdiction
    • Hash of source text (for immutability & provenance).

1.3. Retrieval Layer​

  • Hybrid Retrieval: semantic similarity + BM25 keyword weighting.
  • Re-ranking: domain-specific reranker prioritizing standards citations and regulatory clauses over narrative text.
  • Access Control Filtering: apply ACLs before retrieval (customer sees only what they are entitled to).
  • Multi-framework support: retrieval constrained by regulatory_alignment.framework from the Go/No-Go layer.

1.4. Generation Layer (LLM interface)​

  • Prompt structure: context blocks injected with provenance; refusal rules if insufficient context.
  • Citations enforced: all generated statements linked to {framework_id, version, anchor, hash}.
  • Version pinning: RAG always retrieves from a fixed versioned index (e.g., “ESRS v2025-07” vs. “Delegated Acts 2023”).

1.5. Observability & Governance​

  • Retrieval metrics: precision@K, coverage of DR/AR.
  • Trace logging: store retrieval sets, ranks, and hashes per request for audit.
  • Drift detection: re-index and re-evaluate when standards or taxonomies update.
  • Fail-safe modes: if retrieval index is unavailable, block AI responses instead of fallback hallucination.

1.6. Other Features​

  • Cross-lingual retrieval (auto-translate queries, retrieve in local language, map back to EN index).
  • Knowledge graph enrichment (link disclosures with NACE codes, supply chain data, verifier attestations).
  • Computation-linked retrieval (retrieval triggers Monte Carlo / LCA modules automatically when numeric disclosures are queried).

2. Behavioral Calibration​

2.1. Purpose & Role​

The Behavioral Calibration layer governs how ZAYAZ AI communicates once content is retrieved and grounded. While RAG ensures factual accuracy and provenance, calibration ensures the form, tone, and compliance alignment of outputs. Its purpose is to make every response regulator-ready, consistent across frameworks, and adaptable to audience needs (e.g., auditors vs. board members).

This layer acts as a policy adapter between raw model output and stakeholder expectations. It enforces disclosure hierarchies, tone constraints, refusal protocols, and jurisdiction-specific behavior.

2.2. Calibration Principles​

Framework Fidelity​

  • Responses must follow the structure and terminology of the target framework:
    • ESRS → Disclosure Requirement (DR), Application Requirement (AR), Non-Mandatory Illustrative Guidance (NMIG).
    • ISSB → Disclosure Objective, General Requirements, Industry-Specific Guidance.
    • SEC → Itemized rules (e.g., 1500 series).
  • Framework packs ensure that terminology, sectioning, and disclosure logic are correct per jurisdiction.

Tone & Style Enforcement​

  • Default tone: precise, neutral, regulator-ready.
  • Forbidden: speculative language, promotional spin, unverified claims.
  • For investor/board summaries, calibrated styles are available, but they must preserve compliance safeguards.

Refusal & Escalation Policy​

  • If retrieval lacks sufficient evidence, the system must refuse with a structured response:
  • “Insufficient evidence found in {framework/version}; manual review required.”
  • For sensitive domains (e.g., labor rights, litigation exposure), the system flags and routes for human-in-the-loop review.

2.3. Calibration Architecture​

  • Behavior Adapter: A middleware that wraps the LLM output and applies formatting, style enforcement, and refusal injection.
  • Policy Packs: YAML/JSON policies per framework (ESRS, ISSB, SEC), defining allowed section headers, tone parameters, refusal templates, and mandatory citation structures.
  • Adapter Registry: Version-controlled store of behavior adapters, linked to framework version IDs.
  • Feature Flags: Customers can be onboarded incrementally (e.g., enable only ESRS adapters at first).

2.4. Workflow Integration​

  • RAG retrieves context (e.g., ESRS E1 DR2 paragraphs).
  • LLM generates draft answer.
  • Behavior Adapter applies calibration:
    • Enforce DR/AR/NMIG ordering.
    • Insert mandatory citations.
    • Remove speculative phrasing.
    • Add refusal if evidence missing.
  • Calibrated output logged with adapter version for audit.

2.5. Observability & Governance​

  • Behavior Metrics:
    • % of responses adhering to structure (DR/AR/NMIG).
    • % of refusals triggered correctly.
    • Tone/style compliance scores (linting with regex + embeddings).
  • Audit Trails:
    • Every output tagged with behavior adapter ID and policy version.
    • Stored alongside retrieval logs for complete explainability.
  • Change Management:
    • New adapters tested against gold datasets.
    • Rollback supported via feature flags.

2.6. Other Features​

  • Dynamic Audience Calibration: Switch tone/format automatically for regulators, investors, or employees, while preserving compliance guardrails.
  • Multilingual Calibration: Guarantee that tone, formality, and refusal policies carry over consistently across languages.
  • Contextual Escalation: Auto-route sensitive disclosures (e.g., biodiversity impacts, supply chain human rights risks) to human reviewers.
  • Adaptive Policy Learning: Feedback from auditors and customers can tune calibration policies over time, while remaining regulator-aligned.

3. Standards Packs & Jurisdiction Routing​

3.1. Purpose & Role​

As ZAYAZ expands beyond ESRS, the AI must adapt to different jurisdictional reporting standards without losing consistency, auditability, or maintainability. The Standards Packs & Jurisdiction Routing layer enables this by:

  • Modularizing framework-specific rules, disclosure structures, and terminologies into standards packs.
  • Dynamically routing queries and AI behaviors based on customer profile, self-assessment, and framework applicability.
  • Ensuring one core AI system (RAG + Behavioral Calibration) works globally, with framework overlays applied only as needed.

This design allows ZAYAZ to support multi-framework ESG reporting while avoiding “forked” architectures.

3.2. Standards Packs​

Definition: A Standards Pack is a self-contained policy module for a given framework (e.g., ESRS, ISSB, SEC). Each pack contains:

  • Disclosure hierarchy (e.g., DR → AR → NMIG for ESRS; Itemized rules for SEC).
  • Terminology map (e.g., “Scope 3” vs “value chain GHG emissions”).
  • Citation format rules (framework ID, version ID, anchor references).
  • Behavior adapter config (tone, refusal templates, mandatory disclaimers).
  • Validation criteria (critical/blocker rules for Go/No-Go).
  • Crosswalk entries to the Global Disclosure Ontology (GDO).

Examples:

  • ESRS Pack → DR/AR/NMIG hierarchy, July 2025 amendments, EU taxonomy integration.
  • ISSB Pack → Single-materiality lens, cross-industry metrics, interoperability with SASB industry guidance.
  • SEC Pack → Filer categories (LAF/AF/SRC), liability emphasis, climate rule itemization.
  • GRI Pack → Topic-specific disclosures, materiality matrix emphasis.

3.3. Jurisdiction Routing​

The Jurisdiction Router determines which pack(s) to apply at runtime. It uses:

  • Customer Profile:
    • Incorporation country, exchange listing, size thresholds.
    • Self-assessment results (from Go/No-Go engine).
  • Framework Applicability:
    • regulatory_alignment.framework from assessment payload.
    • Multi-framework flag if customer must report under >1 system (e.g., EU subsidiary of US multinational).
  • Routing Logic:
    • Single framework → direct routing to pack.
    • Multi-framework → sequential application (e.g., ESRS baseline + ISSB overlay).
    • Unsupported → refuse + recommendation for manual review.

3.4. Integration with RAG & Behavioral Calibration​

Standards packs integrate seamlessly with the AI Intelligence Layer:

  • RAG: Retrieval scoped to pack-specific corpora and versioned indices.
  • Behavioral Calibration: Enforced via pack-specific adapters (ESRS style vs SEC style).
  • Decision Engine: Critical/blocker rules loaded from the framework’s criteria table.
  • Auditability: Every AI response logs which pack(s) and version(s) were applied.

3.5. Observability & Governance​

  • Pack Version Control: Each pack version is tracked in Git/DB, with effective dates and deprecation notices.
  • Change Management: Updates to packs (e.g., new ESRS delegated acts) trigger re-indexing, re-calibration tests, and CI gates.
  • Usage Metrics: Track % of queries per framework, dual-reporting overlaps, and adapter effectiveness.
  • Compliance Assurance: Logs include pack ID, jurisdiction ID, and version ID for every AI output.

3.6. Other Features​

  • Multi-pack Fusion: AI can generate integrated outputs (e.g., ESRS + ISSB reconciliation tables) via crosswalks in the Global Disclosure Ontology (GDO).
  • Automated Jurisdiction Detection: Based on filings, NACE codes, or exchange rules, ZAYAZ can auto-suggest applicable frameworks.
  • Regional Expansions: Add packs for Singapore SGX, Japan FSA, Brazil CVM, South Africa JSE, etc.
  • Customer-Specific Packs: Strategic customers can have internal reporting overlays (e.g., custom KPIs, voluntary frameworks).

With Standards Packs and Jurisdiction Routing, ZAYAZ achieves global applicability without architectural drift: one core AI system, multiple compliance overlays, and transparent routing decisions.

4. Security & Data Residency​

4.1. Purpose & Role​

The Security & Data Residency layer ensures that all AI operations within ZAYAZ comply with regulatory, contractual, and ethical requirements for sustainability reporting.

Its purpose is to:

  • Guarantee that sensitive ESG data (e.g., emissions, supply chain disclosures, workforce metrics) is protected by design.
  • Enforce jurisdiction-specific residency rules, especially EU data sovereignty under CSRD and GDPR.
  • Prevent data leakage or unauthorized inference, ensuring ZAYAZ outputs remain auditable and trustworthy.

This layer integrates with every other component of the AI Intelligence Layer — ingestion, retrieval, calibration, and jurisdiction routing.

4.2. Security Principles​

  • Data Minimization

    • Only ingest data necessary for disclosure or reporting.
    • Strip PII (personally identifiable information) unless explicitly required (e.g., workforce disclosures).
    • Apply automatic redaction for sensitive free-text fields.

  • Access Control & Segmentation

    • Every document indexed in RAG is tagged with an ACL (Access Control List) before embedding.
    • Retrieval applies ACL filters before semantic or keyword search.
    • Multi-tenant isolation ensures no customer data leaks across indices.

  • Encryption & Transport Security

    • All data in transit secured with TLS 1.3.
    • At-rest encryption with AES-256, with key rotation policies.
    • Optional customer-managed keys for strategic clients.

  • Audit & Logging

    • Every retrieval and generation request logs: customer ID, framework, version, adapter, and retrieval set hashes.
    • Immutable audit logs stored in append-only ledger for compliance assurance.

4.3. Data Residency​

  • EU Pinning (default)

    • All ESRS-related data (including embeddings, indices, adapters, and audit logs) reside in EU data centers.
    • This complies with GDPR and CSRD sovereignty requirements.

  • Jurisdiction-Aware Routing

    • Customers outside the EU can select regional residency zones (e.g., US-East for SEC reporting, APAC-Singapore for ISSB/GRI).
    • Residency policy is applied at both storage (databases, indices) and compute (AI inference endpoints).

  • Hybrid & On-Prem Deployments

    • For high-sensitivity sectors (finance, defense, healthcare), ZAYAZ can run on-premise AI deployments where data never leaves customer infrastructure.
    • Hybrid models (central RAG + local inference) supported for EU-critical clients.

4.4. Threat Models & Controls​

  • Cross-Tenant Data Leakage: Mitigated by ACL-filtered retrieval and strict tenant isolation.
  • Prompt Injection Attacks: Guardrails in Behavioral Calibration block attempts to override compliance constraints.
  • Data Poisoning in Ingestion: Cryptographic hashing + validator checks ensure only verified sources enter indices.
  • Model Supply Chain Risks: Models pinned to digests; adapter versions signed and auditable.
  • Regulatory Drift: Residency rules reviewed quarterly to align with new CSRD/ISSB/SEC guidance.

4.5. Observability & Governance​

  • Security Metrics

    • % of documents with ACL tags.
    • % of retrieval requests filtered correctly.
    • Data residency compliance checks (by region).

  • Audit Readiness

    • Logs aligned to ISO 27001 & SOC 2 standards.
    • Residency proofs exportable for auditor review.

  • Incident Response

    • Kill switch for AI inference in case of data breach.
    • Escalation protocols defined per jurisdiction.

4.5. Other Features​

  • Zero-Knowledge Proofs for Residency: Cryptographic attestation that AI queries never leave EU zones.
  • Federated Retrieval: Queries answered across distributed indices without moving raw data.
  • Differential Privacy: Protects sensitive workforce and supply chain metrics while enabling analytics.
  • Continuous Compliance Monitoring: Automated checks against evolving rules (e.g., EU Data Act, Digital Services Act, SEC climate mandates).

With Security & Data Residency, ZAYAZ AI becomes bulletproof for regulatory-grade reporting: secure, sovereign, and defensible under scrutiny from auditors, regulators, and stakeholders.

5. Observability & SLOs​

5.1. Purpose & Role​

The Observability & Service Level Objectives (SLOs) layer ensures that ZAYAZ AI operates with transparency, reliability, and continuous assurance. Its purpose is to:

  • Provide real-time visibility into retrieval, calibration, and generation pipelines.
  • Define measurable SLOs that reflect regulatory and business requirements.
  • Enable auditors and SRE teams to trace every AI decision from input to output.

This chapter formalizes the “how we know the AI is behaving” part of the intelligence layer.

5.2. Observability Principles​

  • End-to-End Traceability

    • Every query generates a trace ID linking ingestion → retrieval → calibration → output.
    • Traces include: input query, retrieved passages, model prompts, adapter version, citations, and final output.

  • Structured Logging

    • Logs are structured JSON with key fields:
      • customer_id, framework_id, version_id
      • retrieval_hashes, adapter_id
      • decision (GO / NO-GO / CONDITIONAL)

Logs are append-only and tamper-evident for compliance assurance.

  • Metrics Collection

    • Retrieval quality (precision@k, coverage of DRs).
    • Behavioral compliance (% of outputs with correct DR/AR/NMIG structure).
    • Security and residency (% of requests processed within jurisdiction).

  • Auditability by Design

    • All traces stored with audit retention policies (e.g., 7 years for EU financial disclosures).
    • Outputs linked to source hashes, enabling cryptographic re-verification.

5.3. Core SLOs​

ZAYAZ defines AI-specific SLOs that go beyond uptime and latency:

CategoryMetricTarget SLO
Accuracy% of answers with valid citations≥ 99%
Precision@5 for retrieval≥ 95%
Compliance Behavior% of outputs structured DR/AR/NMIG≥ 98%
% of unsupported answers refused≥ 99%
Security & Residency% of queries processed in correct zone100 %
AvailabilityAI inference latency < 2s (P95)99.9% uptime, 2s P95 latency
Auditability% of outputs with complete trace logs100 %

5.4. Monitoring Architecture​

  • Metrics Pipeline

    • Export metrics to Prometheus / OpenTelemetry.
    • Dashboarding in Grafana with compliance overlays (ESRS/SEC readiness views).

  • Tracing

    • Distributed tracing via OpenTelemetry.
    • Retrieval, calibration, and generation spans all tagged with framework and adapter IDs.

  • Alerts

    • Drift in retrieval precision triggers automatic re-index.
    • Adapter failures (e.g., refusal rules not firing) trigger circuit-breaker.
    • Residency violations trigger immediate block and escalation.

5.5. Governance & Audit Readiness​

  • SLO Review Cycle: Quarterly with input from engineering, compliance, and client success teams.
  • Audit Exports: On-demand export of trace logs, retrieval sets, and adapter IDs for regulators/auditors.
  • Incident Reports: Structured reports aligned to ISO 27001 and SOC 2, including AI-specific metrics.
  • Regulatory Assurance: All SLOs are mapped to CSRD Article 19a transparency requirements and equivalent ISSB/SEC assurance expectations.

5.6. Other Features​

  • Regulator Dashboards: Secure portals where auditors/regulators can review ZAYAZ SLO performance in near real-time.
  • Self-Healing AI Pipelines: Automatic retraining or adapter rollback if SLOs breached.
  • Continuous Benchmarking: Gold-standard test queries run daily to measure retrieval quality and calibration adherence.
  • Explainability Enhancements: Integration of SHAP/attention heatmaps to visualize why certain disclosures were retrieved or emphasized.

AI Intelligence Layer - End-To-End Architecture​



GitHub RepoRequest for Change (RFC)