EEP

Engineering Evidence Pipeline

Purpose and Scope

The Engineering Evidence Pipeline (EEP) defines how ZAYAZ captures, stores, indexes, and links engineering reality (what was built, shipped, and operated) with documentation and intent (what was designed, described, and communicated).

This pipeline is foundational for:

• Support & service ("what is actually running?")
• Upgrades & migrations ("what changed, and when?")
• Compliance & auditability ("prove it existed and behaved as claimed")
• Drift detection ("idea vs. implementation")
• Ask ZARA’s system-wide reasoning over docs + code + artifacts

EEP is not a source-code hosting strategy. Source code remains in Git repositories. EEP captures evidence artifacts produced by CI/CD and runtime systems and stores them in AWS S3 as immutable, queryable records.

---

1. Architectural Principles

1.1 Separation of Concerns

Layer	Responsibility	System
Intent & Narrative	Human-reviewed design, rationale, examples	Docusaurus + GitHub
Curated Examples	Pedagogical / illustrative code	GitHub (code/associated-files)
Engineering Evidence	What was built, shipped, configured, or run	AWS S3
Intelligence & Reasoning	Cross-linking, Q&A, drift analysis	Ask ZARA

1.2 Determinism

Every evidence artifact must be traceable to:

• a Git repository
• a commit SHA
• a build or deployment context

EEP artifacts are append-only. Nothing is overwritten.

1.3 Latest-by-Default

Ask ZARA operates on a single consolidated index (latest per component).

Historical versions are preserved in S3, but default retrieval always targets:

the most recent successfully built and indexed artifact per component.

---

2. What Goes into S3 (Engineering Evidence)

EEP stores evidence artifacts, not raw source trees.

2.1 Mandatory Artifact Types

• API contracts (OpenAPI / AsyncAPI / GraphQL schema)
• Data contracts (JSON Schema, Avro, Protobuf descriptors)
• Database migrations + generated schema snapshots
• Configuration schemas + defaults
• SBOMs (CycloneDX / SPDX)
• Build provenance (SLSA-style attestations)
• Runbooks & operational playbooks
• Generated registries (engine registry, signal registry, etc.)

2.2 Optional / Advanced Artifacts

• Container digests and metadata
• Lambda / function package hashes
• Performance benchmarks
• Policy evaluation outputs
• Runtime constraint reports

---

3. S3 Evidence Layout (Canonical)

All evidence is stored under a deterministic prefix structure:

s3://zayaz-evidence/
  org=viroway/
  env=prod|staging|dev/
  domain=ai-intelligence-layer|comp-hub|infra|...
  system=zayaz/
  repo=<repo-name>/
  component=<service-or-package>/
  version=<semver-or-tag>/
  sha=<git-sha>/
  artifact_type=openapi|schema|sbom|runbook|build|config|migration|...
  filename

3.1 GitHub Structural Alignment (Logical Mirroring)

To enable deterministic linking between documentation, curated examples, and engineering evidence, ZAYAZ applies a logical path mirroring strategy between GitHub (/code/associated-files) and S3.

This mirroring is semantic, not physical.

GitHub (Intent & Examples) Curated example files live under:

code/associated-files/
  <doc-logical-path>/
    <example-file>

These files:

• Represent intended usage, contracts, and pedagogical examples
• Are human-reviewed
• Are referenced from MDX using <Snippet file="…" />

S3 (Engineering Evidence) Engineering evidence artifacts are stored in S3 under deterministic prefixes that preserve the same logical path:

s3://zayaz-evidence/
  ...
  component=<component>/
  version=<version>/
  sha=<git-sha>/
  artifact_type=<type>/
  <doc-logical-path>/
    <artifact-file>

Example If a documentation page contains:

<Snippet file="gdo-api-contract-response.json" />

and the page has:

source_file: /ai-intelligence-layer/ai-technical-implement/api-contracts.mdx

Then the logical key is:

ai-intelligence-layer/ai-technical-implement/api-contracts/gdo-api-contract-response.json

This key can be resolved to:

• GitHub (curated example) code/associated-files/ai-intelligence-layer/ai-technical-implement/api-contracts/gdo-api-contract-response.json
• S3 (engineering evidence) s3://zayaz-evidence/.../<artifact_type>/ai-intelligence-layer/ai-technical-implement/api-contracts/gdo-api-contract-response.json

This guarantees:

• Stable cross-layer navigation
• Traceability from idea → example → shipped artifact
• Safe comparison and drift detection

Ask ZARA uses this shared logical path as a join key when correlating documentation, examples, and engineering evidence across GitHub, S3, and the embedded search index.

3.2 Why This Matters

• Enables prefix-based queries (S3, Athena, Glue)
• Natural join key for docs, examples, and Ask ZARA
• Tenant- and environment-safe for future white-labeling

---

4. Evidence Manifest (Schema v0.1)

Each build must emit a manifest describing all uploaded artifacts.

4.1 Location

.../sha=<git-sha>/manifest.json

4.2 Manifest Schema (v0.1)

manifest-schema.jsonGitHub ↗

{
  "schema_version": "0.1",
  "org": "viroway",
  "system": "zayaz",
  "domain": "ai-intelligence-layer",
  "repo": "zayaz-hecate",
  "component": "pef-me",
  "version": "1.4.0",
  "git_sha": "a13f9c2",
  "build_timestamp": "2026-01-21T14:32:10Z",
  "artifacts": [
    {
      "artifact_type": "openapi",
      "s3_uri": "s3://zayaz-evidence/.../openapi/pef-me.openapi.yaml",
      "content_hash": "sha256:...",
      "media_type": "application/yaml",
      "origin": {
        "repo_path": "services/pef-me/openapi.yaml",
        "commit": "a13f9c2"
      },
      "toolchain": {
        "generator": "openapi-generator",
        "version": "7.3.0"
      }
    }
  ]
}

4.3 Contractual Guarantees

• manifest.json is immutable
• content_hash must match artifact content
• origin always points back to Git

---

5. Index Generator Contract (Docs & Ask ZARA)

5.1 Purpose

Transform distributed S3 evidence into a single consolidated index used by:

• Ask ZARA (/app/data/index.jsonl)
• Docusaurus evidence links
• Drift detection tooling

5.2 Input

• Latest manifest.json per (domain, component)
• Optional allowlist of artifact types

5.3 Output

A JSON Lines file:

/app/data/index.jsonl

Each line represents a retrievable knowledge unit:

retrievable-knowledge-unit.jsonGitHub ↗

{
  "id": "pef-me:openapi:latest",
  "type": "engineering_evidence",
  "domain": "ai-intelligence-layer",
  "component": "pef-me",
  "artifact_type": "openapi",
  "version": "1.4.0",
  "git_sha": "a13f9c2",
  "s3_uri": "s3://zayaz-evidence/...",
  "content_hash": "sha256:...",
  "doc_keys": [
    "ai-intelligence-layer/ai-technical-implement/api-contracts"
  ]
}

5.4 Resolution Rules

• Only latest successful manifest per component is indexed by default
• Older versions remain queryable by explicit filters (future extension)

---

6. Linking Docs, Snippets, and Evidence

6.1 MDX as the Join Surface

Docusaurus MDX files define intent and structure.

Example:

<Snippet file="gdo-api-contract-response.json" />

From this, the system can derive:

• GitHub curated example
• S3 evidence artifact (via index)
• Drift status (hash comparison)

6.2 UI / UX Expectations

Each snippet may expose:

• GitHub ↗ (idea / curated example)
• Evidence ↗ (what shipped)
• Drift badge (MATCH ✅ / DRIFT ⚠️ / UNKNOWN ⏳)

---

7. Ask ZARA Integration

Ask ZARA operates on a single embedded semantic index loaded from:

/app/data/index.jsonl

7.1 Retrieval Strategy

Ask ZARA retrieves and reasons over:

• Documentation sections (MDX)
• Snippet-backed examples
• Engineering evidence artifacts

All answers should preserve provenance:

• Doc reference
• Example reference
• Evidence reference (S3 manifest + artifact)

7.2 Default Semantics

• Latest evidence per component
• Human-readable explanation first
• Evidence-backed assertions second

---

8. Governance & Next Extensions

Planned evolutions:

• Version-scoped Ask ZARA queries (@version:1.3.2)
• Evidence diffing & visual drift reports
• Verifier access for auditors (read-only S3 views)
• White-label evidence namespaces per tenant

---

EEP is the backbone that turns ZAYAZ from "documented" into "provable".