The Integrity Framework

Case study · v1.0 · published 2026-04-28

FieldLedger, the first portfolio audit.

Second case study under The Integrity Framework v1.0. First internal portfolio audit. The framework, applied to a real Startvest product, with two interesting consequences.

Where the Delve case study walks an external public failure through the framework, this case walks a real product through it. Source documents are public: FieldLedger's INTEGRITY.md and audits/tif-compliance.md. This page summarizes; both files have the full evidence.

What was audited

FieldLedger is a Startvest LLC product. Operations and compliance tooling for federal contractors and construction businesses — DCAA cost accounting, Davis-Bacon certified payroll, FAR 31.2 indirect rates, ICE I-9 tracking, audit log. Tier-priced subscriptions. Customers pay for the tooling; third parties (DCAA, DOL, prime contractors, IPAs) verify outputs. FieldLedger does not issue any third-party certifications.

Audit conducted 2026-04-28. Methodology: a qualitative pass by Claude Code against FieldLedger's code, INTEGRITY.md, and methodology page; a mechanical pass by integrity-cli (base manifest) against the public repo. The two passes are cross-checked: where the CLI flagged a finding, the qualitative pass verified it against the actual code. Where the qualitative pass flagged a finding, the CLI was checked for whether it should have caught it.

Headline result

20 dimensions · Layer 1 vetoes (6) + Layer 2 constraints (7) + Layer 3 guardrails (7)

14

Pass

5

Partial / Needs update

1

Deferred

0

Fail

Vendor scorecard (TIF v1.0): 4 YES / 0 PARTIAL / 2 NO. Open scorecard rows: MSA refund-on-failure clause (drafted, not rolled in) and annual independent audit (deferred pending external funding, honestly classified).

Layer 1 is fully clean — six pre-build vetoes pass without exception. Layer 2 has three open items, two of them pre-Plus-tier (auditor-readonly role, 7-year DCAA retention policy) and one pre-launch (UI badging on customer-attested fields). Layer 3 has two needs-update items (refund clause, formal whistleblower channel) plus one honestly-deferred item (annual third-party audit, funding-blocked).

The FieldLedger team's self-assessment was honest. The audit confirmed the INTEGRITY.md as written; no surprises in the product. The interesting findings were elsewhere.

The framework auditing itself, twice

The most important outcome of this audit was not a product finding. It was two findings against the framework's own rule set.

The first integrity-cli run flagged three failures. Two of them — the most important ones — were false negatives in the rules, not gaps in the product. They tripped because the base manifest's filename-based globs targeted a hypothetical layered architecture that didn't match FieldLedger's actual code layout (libraries that compute output, with persistence and markers attached at the schema and route layers).

A rule that produces a false negative is silent-pass at the meta level — exactly the failure mode the framework prohibits at the product level.

So the rules were revised, not the product. The fix: a new content-based co-occurrence check kind that looks for required markers anywhere in the corpus, rather than requiring them to appear in specific filename patterns. Two manifest revisions, each with a changelog entry that points back to this audit.

CRIT-SV-AI-REVIEW-GATE

CLI initial verdict

cpars-drafting.ts lacks generatedByModel marker

Manual verification

generatedByModel is present at src/lib/entity-schemas.ts:916 (TypeORM column on CparsResponse), at the route write path, and in two public pages. The library returns { text, model }; persistence and the marker attach at the schema/route layer. Substantively, Layer 2 Constraint 2 is satisfied.

Resolution

Promoted to base manifest v1.3.0 as a content-based co-occurrence rule: trigger on AI SDK imports / messages.create / chat.completions.create; require any of the customer-attested or AI-review markers somewhere in the corpus. Re-run: PASS.

CRIT-SV-NO-PRE-POPULATED-ATTESTATION

CLI initial verdict

globs matched cpars-drafting.ts and certified-payroll.ts; no customer-submitted markers found

Manual verification

Both files are pure libraries. The customer-submitted markers (RateSnapshotJson, EmployeeRowsJson, generatedByModel, etc.) live on the persisted entities. Substantively, Layer 2 Constraint 3 is satisfied at the schema layer; the UI badging gap is separate and pre-existing.

Resolution

Revised in base manifest v1.4.0 to use co-occurrence: trigger on attestation-bearing entity table names (CertifiedPayrolls, RateLetters, CparsResponse, *Attestation, *Statement, *Certification) or attestation-emitting function names (submit*Response, generate*Letter, etc.); require any customer-submitted marker somewhere in the corpus. Re-run: PASS.

The third initial finding — INFO-FL-TRUST-PRINCIPLES-LINK — was a real Layer 3 discoverability gap on the FieldLedger privacy page. Not a rule issue. Tracked as a P2 remediation ticket.

The framework was calibrated against zero real products. First contact with reality found two filename-based blind spots. They are now closed in base manifest v1.4.0, and the lesson carries forward to every subsequent product audit.

Open items at audit close

The audit produced a remediation backlog. Eight items, three priorities:

P0
Layer 3 G1

Finalize MSA refund-on-failure clause and roll into standard template

Draft exists. Blocks vendor scorecard row 2 → YES. Required before first paid customer.

P1
Layer 2 C3

Add explicit 'customer-attested' UI badge to SDVOSB and veteran status fields

Schema-layer isolation already passes. UI-layer visual isolation is the remaining gap. Pre-launch blocker.

P1
Layer 2 C5

Document and enforce 7-year DCAA evidence retention policy

No offboarding code yet — rule passes vacuously. Activates the moment offboarding lands. Plus-tier prerequisite.

P1
Layer 2 C6

Add auditor_readonly role to access control before Plus tier launch

Independent verification hooks constraint. Required for the auditor-can-verify-without-the-product-mediating mode.

P1
Layer 3 G5

Provision and document integrity@startvest.ai whistleblower channel + external counsel SLA

Channel exists; SLA documentation does not. Quarterly external counsel review is the intent.

P1
Layer 3 G3

Annual independent audit — secure funding, sign engagement letter

Honestly classified as DEFERRED. Engagement cost (CPA / security firm) is currently unfunded. Moves to PARTIAL once funding lands and an engagement letter is signed; YES only after a completed cycle with public findings.

P2
Discoverability

Add startvest.ai/trust-principles link to privacy page

Clears INFO-FL-TRUST-PRINCIPLES-LINK. Trivial.

P2
CLI rule hygiene

Rename CRIT-SV-EVIDENCE-CHAIN → CRIT-FL-EVIDENCE-CHAIN

The SV namespace is reserved for the base manifest under CRIT-SV-NO-BASE-ID-OVERRIDE. Naming hygiene only; no code-substance change.

All items are tracked in the FieldLedger repo under the tif-compliance label. The matrix at audits/tif-compliance.md cross-references every item to the specific code path or contract clause that closes it.

What this teaches

  1. The framework adapts when reality finds a gap. Two manifest revisions in one audit cycle, both pointing back to this case study, is not the framework failing. It's the framework working — failure transparency applied to itself.
  2. Filename-based rules don't survive first contact. The original Layer 2 rules assumed a particular code layout. Real products have other layouts. The new co-occurrence pattern is more robust because it asks “does the marker exist anywhere in the corpus that needs it” instead of “does the marker exist in this specific filename pattern.”
  3. Honest self-assessment + qualitative + mechanical is the right combination. FieldLedger's INTEGRITY.md was honest before the audit. The qualitative pass verified it. The mechanical pass found two cases where the rule set was wrong. Any one of those three alone would have missed the rule-set bug. Together they caught it.
  4. The remaining open items are the kind a framework should produce. MSA refund clause, 7-year retention policy, UI badging, formal whistleblower SLA, annual third-party audit funding. None of them are technically heroic; all of them are the kind of work that gets put off without a forcing function. The framework IS the forcing function.

Reproducibility

The audit is mechanically reproducible.

# from a clone of the integrity-cli repo
node bin/integrity.mjs check ../FieldLedger --format=json \
  > ../FieldLedger/audits/tif-compliance.cli-output.json

Re-run on every release. Update Last reviewed in INTEGRITY.md and regenerate this matrix when remediation lands. The CLI output JSON is committed to the FieldLedger repo so an external auditor can compare runs across time without re-running anything.

Changelog