What an AI Project Audit Examines

Your AI coding agent can help ship a feature quickly. The next question is harder: can you explain why the product works this way, what might break when it changes, and which evidence supports the decision to release it?

A working screen answers only part of that question. A project can look healthy at the surface while its data rules, access boundaries, release process, or technical decisions remain unclear. Before a major change, handoff, investment decision, or production release, those gaps deserve attention.

An AI Project Audit is designed to turn that concern into a bounded, evidence-linked diagnosis. It starts with the decision you need to make, the project scope you can legitimately share, and the sources available for review. It then asks which questions apply, what the evidence shows, and where the answer is still uncertain. The areas below are ways to investigate a project, not a fixed checklist that every project must pass.

Start with the decision and the evidence

Suppose you are preparing to add payments. The important question is not simply whether the checkout screen works. You may need to know whether an order can be charged twice, whether a failed payment leaves a clear state, who can see payment records, and how a release could be reversed. A project without payments needs a different emphasis. A prototype and a production service also require different evidence.

Useful scope names the product or workflow, the upcoming decision, the source owners, and the point in time the evidence represents. Evidence might include requirements, decision records, repository structure, tests and their results, deployment records, logs, access rules, or a demonstration of a user flow. A diagram or an agent's assertion can guide an investigation, but it does not by itself prove that the running system follows the diagram. Missing, stale, conflicting, and inaccessible sources should remain visible in the result.

The ten areas pictured on the AI Project Audit page help organize possible engineering questions. Some overlap. Their value is in following a material issue across the boundaries it touches.

1. Architecture and module boundaries

Question to investigate: Where should a change live, and do the actual dependencies follow that boundary? A payment rule spread across a screen, API handler, and background job may be hard to change safely even if each piece works alone.

Evidence to examine: module and service maps, imports and call paths, API contracts, deployment topology, architecture decisions, and a small trace through a consequential workflow. The review compares intended boundaries with code and runtime evidence. An architecture diagram that is no longer current is a finding about evidence quality too; it should not be treated as proof of the present design.

Decision it can inform: whether to make the next feature in place, clarify a boundary first, or narrow the change to avoid spreading a rule further. The right answer depends on the project's stage and the cost of changing its current design.

2. Data ownership and consistency

Question to investigate: Which component owns a fact, and what happens when two steps update it at different times? If an order, invoice, and payment each carry their own status, a failed or repeated event can leave them disagreeing.

Evidence to examine: data models, write paths, transaction or event-handling rules, idempotency behavior, migration history, and tests for retries or partial failures. A sample record can show a problem, but a few healthy records do not establish that every concurrent path is safe. The review should separate observed behavior from an untested assumption.

Decision it can inform: whether a workflow needs a single owner, explicit state transitions, stronger retry handling, or a focused consistency test before more features depend on it.

3. Security and access control

Question to investigate: Who can read or change which project data, and where is that rule enforced? In a product with separate customer workspaces, a correct-looking UI filter does not prove that an API or background job applies the same boundary.

Evidence to examine: actor and role definitions, authorization checks at execution points, data isolation rules, secret-handling practices, and negative tests for unauthorized access. The review must be authorized and limited to the agreed project scope. It should identify what was inspected and what could not be inspected. OWASP's Application Security Verification Standard provides a separate web-application verification reference; following it or passing an audit area does not by itself certify this product.

Decision it can inform: whether a new customer-facing workflow can be released as planned, needs a narrower permission model, or requires a specialist security review. A project-health diagnosis cannot replace that review or grant permission to change sensitive systems.

4. Reliability and failure handling

Question to investigate: What happens when a dependency times out, a job runs twice, or the user abandons a workflow halfway through? The happy path rarely explains how the system behaves when an external service or internal component fails.

Evidence to examine: error paths, retry and timeout rules, incident records, logs or traces, queue behavior, and tests for partial failure. If observability is absent, that is a limitation on the conclusion. A lack of recorded incidents does not establish reliable behavior when no one can see failures.

Decision it can inform: which failures need safe recovery before a release, what an operator must be able to detect, and whether a workflow should be simplified until its failure states are understandable.

5. Testing and regression protection

Question to investigate: What would show that the next change preserved the behavior people rely on? A green test run is useful only when the tests cover the relevant workflow and their assumptions still match the product.

Evidence to examine: unit and integration tests, end-to-end flows, test data, recent CI results, known flaky tests, and manual acceptance evidence for consequential behavior. The review looks for meaningful coverage of rules, failures, access boundaries, and past regressions. A test count alone says little about whether the next release is protected.

Decision it can inform: which regression cases to add before changing a sensitive area, what remains a manual review, and where a passing suite leaves uncertainty.

6. Performance and scale

Question to investigate: Will the important workflow still work at the load and data size the project expects? A fast local demonstration may conceal a slow query, large payload, or dependency bottleneck.

Evidence to examine: representative workload assumptions, measurements, query plans, latency and resource signals, and tests at realistic data sizes where available. The evidence should identify its environment and date. A benchmark from a small fixture should not be extrapolated into a capacity promise.

Decision it can inform: whether to measure the real bottleneck before optimizing, set a practical performance boundary for the next release, or defer scale work because the current decision does not require it.

7. Dependencies and integrations

Question to investigate: Which external systems and packages can change the behavior of this product, and who owns that risk? An integration may be correct today yet depend on an undocumented callback, token lifetime, or vendor error path.

Evidence to examine: dependency manifests and lockfiles, integration contracts, version and update records, test doubles versus live-test evidence, service ownership, and failure or rate-limit handling. GitHub's dependency review documentation explains one narrower tool for spotting dependency changes and known vulnerabilities; that signal does not establish whether an integration fulfills the product workflow.

Decision it can inform: whether an upcoming release needs a contract test, dependency update, fallback behavior, or a named owner for an external failure.

8. Deployment and recovery

Question to investigate: Can the change reach production in a controlled way, and can the team recover when it does not behave as expected? A successful build is only one step in a release.

Evidence to examine: build and migration steps, environment differences, release approvals, rollback paths, backups and restore exercises, deployment logs, and post-release verification. A documented recovery plan without an exercise is a plan, not proven recovery capacity. The assessment should say whether the evidence comes from a real run, a rehearsal, or a proposal.

Decision it can inform: whether to release, rehearse recovery, split a risky migration, or assign an owner to verify the exact deployed revision.

9. Maintainability

Question to investigate: Can another developer or agent change this part of the product without rediscovering hidden rules? Complexity is not automatically bad, but unclear ownership and duplicated logic increase the chance that the next change misses a constraint.

Evidence to examine: code organization, repeated rules, change history, review friction, documentation, ownership, and the effort needed to trace one important behavior. An unfamiliar codebase will always take time to learn; the question is whether its reasoning and responsibilities are recoverable from current sources.

Decision it can inform: whether to clarify one rule, simplify a specific boundary, improve a test or decision record, or accept the current structure because it still supports the next change.

10. Technical decisions

Question to investigate: Can the team explain the choices that shape the system today? When an agent suggests a new approach, it needs to know which constraints are deliberate and which are leftovers from an earlier stage.

Evidence to examine: decision records, rejected alternatives, current requirements, ownership and approval history, and code that either follows or contradicts the recorded choice. An old decision may have been sound at the time but stale now. The review should distinguish a missing decision, a changed context, and an implementation that diverged from an agreed rule.

Decision it can inform: whether to reaffirm, revise, or retire a choice before asking an agent or team to build on it.

Engineering questions need product context

The same technical observation can matter differently depending on what the product is trying to do. A missing backup approver may be minor during an internal prototype and material before a high-stakes release. A complex module boundary may be acceptable if no consequential change touches it soon. The audit therefore also asks what outcome the next change should serve, which requirements are current, who can approve trade-offs, and what evidence would count as acceptance.

That wider context helps prevent two mistakes. One is treating every possible engineering concern as equally urgent. The other is treating a passing code scan or green quality gate as an answer to a product decision. GitHub describes CodeQL code scanning as a way to find vulnerabilities and errors in code. It is a useful narrower signal, while a project decision also needs requirements, architecture, operations, ownership, and evidence that the selected workflow behaves as intended.

Evidence quality is part of the diagnosis, not a footnote. If the source was unavailable, stale, out of scope, or contradictory, the result should say so. A finding should make its observation traceable, state what is uncertain, explain why the issue could affect the pending decision, and propose a next step a person can accept or reject. A recommendation is guidance; it does not authorize code changes or deployment.

For example, imagine an order workflow whose API rejects duplicate payment requests, while the background retry path has no recent test or runtime trace. The evidence may support a narrow conclusion about the API. It does not establish that the whole payment workflow prevents a second charge. The useful next step could be a focused retry test and an owner decision about release scope. That is more honest than calling the area healthy or broken from one code path.

The intended complete Project Health Dashboard brings those findings, coverage, limits, supporting sources, and decision sequence into one working view. Its first audit result is intended to become a baseline you can revisit; returning to that baseline would show the same evidence cutoff rather than silently run a new assessment. A later reassessment would be a separate decision and action.

You can explore the fictional Project Health Dashboard example to see how evidence-linked findings and next steps might be presented. Online audit access is not open yet; the example is not a diagnosis of your project.