Jie Ma · 2026-09-13 · Research note
Why AI-assisted development needs governance before expertise
AI changes who can produce software and how quickly. A developer can now move from an idea to a working feature before acquiring the experience that would normally develop through repeated design, review, failure, and recovery. This is a genuine expansion of human capability. It also means implementation capacity can grow faster than the judgment needed to assess architecture, security, testing, integration, and release consequences.
The people who most need governance may not know which question to ask
Conventional engineering controls often assume that someone already understands the risk: the developer knows when a migration is destructive, the reviewer knows which boundary is security-sensitive, or the release owner knows which evidence is missing. That assumption fails for less experienced developers using highly capable agents. Natural-language fluency can make an answer feel complete even when an important concern was never raised.
A governance system for this setting cannot wait for expertise before becoming useful. It must observe relevant project state, surface conditions the user did not know to ask about, explain why they matter, and offer a safe next step. Every finding should reveal its evidence, applicable rule, uncertainty, recovery path, and what it does not prove. The purpose is not merely to stop an action, but to make the reasoning inspectable enough that the user becomes more capable through repeated use.
Governance should create a path toward expertise, not permanent dependency
This requires progressive adoption. Early use can emphasize learning and audit; later stages can add assistance and proportionate enforcement as the project defines its policies and the evidence becomes reliable. Explanations should support different depths: a concise answer for immediate action, the underlying facts and rule for inspection, and references or examples for someone building deeper understanding.
The tool must not present itself as a substitute for expertise. It cannot know every product constraint, organizational conversation, or real-world consequence. Humans still define policy, challenge findings, approve exceptions, and accept risk. External review and domain specialists remain necessary when stakes demand them.
The practical test is therefore broader than whether a rule catches a known mistake. We should ask whether people can understand the reason, choose an appropriate recovery, recognize a similar situation later, and challenge the system when it is wrong. Governance before expertise is successful only when it reduces immediate risk while helping expertise grow.
Jie Ma · 2026-09-13 · Research note
Governance should increase understanding, not only compliance
A pass or fail result answers a narrow question: did a rule fire? It does not show whether the developer understands the observed fact, why it matters, how certain the judgment is, or what would constitute a safe recovery. A system that stops at the verdict can produce ritual compliance: users learn which button to press or which field to fill, while the underlying risk remains opaque.
Compliance is an event; governance capability is learned capacity
Good governance should leave the person better able to notice, reason about, and verify a similar situation next time. This matters most in AI-assisted development, where implementation speed can outrun the experience needed to assess consequences. The tool should not assume that a novice already knows which question to ask, but it should also avoid making the novice permanently dependent on the tool.
A useful finding therefore needs more than a status. It should expose the observed fact, the applicable versioned rule, the reason the condition matters, the epistemic status of the claim, the next safe step, and a way to verify recovery. It should also state what the evidence does not prove. Progressive disclosure can keep the first view concise while allowing a user—or a reviewer—to inspect the full chain.
Measure transfer, not obedience
The number of blocked actions is a poor measure of governance quality. More meaningful signals include whether users choose an appropriate next action, can explain the reason in their own words, transfer the lesson to a new scenario, challenge an incorrect finding, and repeat the same avoidable error less often. False positives and false negatives still matter, but so do explanation quality and behavioral change.
Understanding does not guarantee a good outcome, and an explanation can itself be wrong. The claim is narrower: a governance tool should be designed and evaluated as a learning system as well as a control system. If it obtains compliance without improving judgment, it has governed the transaction but not strengthened the developer.
Jie Ma · 2026-09-13 · Research note
AI should not certify its own work
AI can generate code, tests, explanations, reviews, and release recommendations. It is tempting to treat a second AI judgment as assurance for the first: one model writes the change, another approves it, and the resulting agreement is presented as confidence. That arrangement may add useful criticism, but it does not create an independent ground truth.
Agreement is not independence
Models may share training patterns, prompts, tools, missing context, or incentives. They can repeat the same plausible mistake in different words. A model may also evaluate evidence that it helped produce, such as tests optimized for its own implementation. The recursive chain becomes especially weak when each stage accepts the previous stage's summary instead of inspecting the underlying state.
A defensible governance architecture separates responsibilities. Machines observe reproducible facts. Named, versioned rules adjudicate only what those facts support. AI explains the finding, uncertainty, and possible recovery. Humans define policy, approve exceptions, accept consequential risk, and remain accountable. No component should acquire broader authority merely because it can describe its reasoning fluently.
The assurance chain must terminate outside the model's confidence
Useful terminating evidence includes reproducible repository state, independently specified tests, deterministic oracles, signed review records, externally observed outcomes, and human examination of the claim's limits. None is infallible. Their value is that they can contradict the AI rather than merely echo it.
AI still has an important role. It can search for inconsistencies, compare a change with policy, generate adversarial scenarios, translate a technical finding into plain language, and help a reviewer inspect more evidence. But it must not silently change the rule, grant itself an exception, convert unknown into pass, or declare its own work safe because its explanation sounds coherent.
The principle is not “never use AI to review AI.” Independent models and prompts can improve coverage, just as multiple human reviewers can. The narrower rule is that AI review is evidence within an assurance process, not the final source of authority. Certification requires a chain of claims whose boundaries are visible and whose endpoint can resist the model's own confidence.
Jie Ma · 2026-09-13 · Research note
Public Git history is evidence, not ground truth
Public repositories offer an unusually rich record of software development. Commits, branches, reviews, releases, reversions, and issue links can reveal sequences of change that are difficult to reconstruct from a snapshot. For a governance project, this makes Git history valuable material for discovering candidate failure modes and testing whether rules recognize observable conditions.
A repository records what was preserved, not everything that happened
Git cannot show every conversation, abandoned local experiment, production incident, organizational constraint, or developer intention. Commits may be squashed, rebased, imported, backfilled, or written to satisfy a convention. Tests may run in systems whose results are no longer public. A clean history can describe a troubled process, while a messy history can belong to a well-governed project.
Public availability adds further selection effects. Open-source projects differ from private teams, visible failures are not a representative sample of all failures, and popular repositories may have resources that small projects do not. A later fix does not by itself prove that the earlier change was negligent, nor does the absence of a recorded incident prove the change was safe.
Research must separate observation, inference, and reference judgment
A responsible case should preserve provenance: repository, revision, timestamp, observable facts, tool and rule versions, and the minimum copied material needed for reproduction. It should distinguish a direct observation—such as a release tag not containing a later fix—from an inference about risk or intent. Ambiguous cases should remain disputed or unknown until suitable evidence or expert adjudication exists.
Public histories should therefore be curated as episodes, not harvested as automatically labeled training data. Cases can be stratified across languages, ecosystems, team sizes, lifecycle stages, and failure types. Some should become transparent development fixtures; others should be held back for blind evaluation so that rule authors cannot tune directly to every answer.
Used this way, public Git history can challenge a governance system beyond its founder's own projects and reveal where an oracle, adapter, or policy claim fails to generalize. It still cannot establish universal effectiveness on its own. The responsible conclusion is always bounded: what was observed, what interpretation it supports, what alternative explanations remain, and what evidence would be needed next.
Jie Ma · 2026-09-13 · Research note
Can AI govern AI-assisted development?
AI can participate in governance, but it should not be sovereign. The recursive idea—using AI to create and apply standards for AI-assisted work—is valuable precisely because AI can search patterns, translate technical evidence into natural language, generate challenge scenarios, and help people reason at a speed closer to modern software production. The same recursion is dangerous if one system's confidence is allowed to validate another system's output.
Separate the roles before assigning authority
A defensible model separates five responsibilities. Humans constitute the goals, values, risk tolerance, and authority of the project. Machines observe reproducible facts. Versioned rules adjudicate only what those facts support. AI explains the finding, uncertainty, and possible next steps. Humans approve exceptions, accept consequential risk, and remain accountable for the decision.
This separation is not a claim that deterministic rules are infallible or that humans are always wise. It is a way to make failures locatable. An observation can be incomplete, a rule can encode the wrong policy, an explanation can mislead, and a human can make a poor choice. When the roles are explicit, each failure can be challenged with the right kind of evidence instead of being hidden inside a single intelligent-sounding answer.
AI adds value at the interface, not at the end of proof
Natural language is a sensible interface for developers already working with coding agents. A person should be able to ask, “Why is this release unsafe?” or “What should I do next?” Yet the answer should be grounded in inspectable findings rather than generated from plausibility alone. AI may interpret intent, summarize context, compare options, and teach. It should not silently change policy, grant its own exception, certify its own work, or convert uncertainty into permission.
The governance system itself must also be governed. Frozen test scenarios, independent deterministic oracles, diverse public histories, blind holdout cases, prospective volunteer use, incident outcomes, and versioned rule changes provide different forms of challenge. None is ground truth on its own; together they make unsupported confidence harder to preserve.
So the practical answer is conditional: AI can help govern AI-assisted development when its authority is bounded, its claims are evidence-linked, its uncertainty is visible, and its judgments terminate in independent checks and accountable human decisions. Without those conditions, AI governance risks becoming self-referential reassurance at machine speed.
Jie Ma · 2026-09-13 · Research note
Natural-language governance interfaces and deterministic engines can coexist
Developers who work with coding agents already express intent through ordinary language. Requiring them to translate every governance question into a rigid command vocabulary would discard one of AI's most useful interface improvements. A person should be able to ask, “What makes this change risky?”, “Why am I blocked?”, or “What evidence is still missing?”
Conversation should interpret the question, not invent the verdict
Natural language is good at accommodating incomplete questions, explaining unfamiliar concepts, and adapting detail to the reader. It is poor as the sole source of a repeatable governance decision. The same prompt can be interpreted differently, model behavior can change, and a fluent answer can conceal missing evidence.
A sound architecture therefore separates the conversational layer from the decision layer. The AI maps the user's request to a bounded query, while a deterministic engine collects facts and applies a named, versioned rule. The AI then explains the resulting finding, its uncertainty, and possible recovery steps. It may propose a command or policy change, but it cannot pretend that proposal has already been approved.
Every answer needs a trace back to inspectable state
The interface should disclose which facts were observed, which rule version was applied, and which parts of the response were interpretation rather than deterministic output. When the question is ambiguous, the system should narrow the scope or return unknown instead of silently choosing the most convenient meaning.
This design makes natural language an accessibility layer over governance, not a replacement for governance. The experience can remain conversational while the judgment remains reproducible, testable, and challengeable.
Jie Ma · 2026-09-13 · Research note
How should a governance tool measure false positives and false negatives?
A governance finding is useful only if it detects a meaningful condition at the right time. A false positive warns or blocks when the claimed risk is not present. A false negative fails to surface a condition that should have been found. Both matter, but they do not have equal cost in every context.
Start with an explicit claim and a reviewable reference judgment
Measurement requires more than running the tool over arbitrary repositories. Each evaluated case needs a frozen rule version, observable inputs, a precisely scoped expected claim, and a reference judgment with provenance. Public Git history may contribute evidence, but it is not automatically a correct label. Ambiguous cases should be marked disputed or unknown rather than forced into pass or fail.
The basic confusion matrix is only a starting point. Results should also be segmented by rule, severity, repository type, lifecycle stage, evidence availability, and user population. A warning that arrives after release is different from one that arrives before a commit. A noisy advisory finding has a different cost from an incorrect release block.
Measure the consequences of error
Precision, recall, false-positive rate, and false-negative rate describe detection behavior. Governance evaluation should add time to understand, time to recover, override frequency, repeated-warning fatigue, harmful misses, and whether users can correctly challenge the result. Severity-weighted measures can reflect that one missed destructive deployment may matter more than many low-risk warnings.
No single score establishes quality. The useful output is an error profile with confidence intervals, examples, known blind spots, and the rule version that produced it. That profile should drive regression fixtures, rule revision, and—where evidence remains inadequate—a narrower claim or an explicit unknown.
Jie Ma · 2026-09-13 · Research note
Why unknown is a legitimate result
Governance systems are often pressured to produce decisive answers. Pass and fail feel actionable; unknown can look like weakness. But when the available evidence cannot support a conclusion, certainty is not decisiveness—it is fabrication.
Unknown protects the boundary between absence and lack of observation
A repository may contain no test record because tests were not run, because results live elsewhere, or because the adapter cannot read them. Those states are not equivalent. Treating missing evidence as pass creates false assurance. Treating it automatically as fail turns an observation gap into an accusation.
An honest unknown should be structured rather than vague. It should identify the unanswered question, the evidence that was checked, why the evidence is insufficient, what additional observation could resolve the uncertainty, and whether a human decision is required in the meantime. It may still trigger a conservative policy action—for example, withholding a release—but the policy response must remain distinct from the epistemic claim.
Unknown is also a diagnostic signal
Repeated unknowns reveal where adapters, project declarations, evidence contracts, or organizational practices are incomplete. Tracking their causes and resolution rates can guide product investment without pretending the missing information was known.
A mature governance tool is not the one that always answers. It is the one that knows which answer its evidence can support, communicates the boundary clearly, and helps the user obtain the next piece of evidence.
Jie Ma · 2026-09-13 · Research note
Approval and accountability boundaries for agentic AI
Agentic AI can plan, edit, test, deploy, communicate, and continue working with limited supervision. This makes approval design part of system architecture rather than a final confirmation dialog. The central question is not whether an agent is “autonomous,” but which effects it may create, under whose authority, with what evidence, and how those effects can be stopped or reversed.
Authority should be scoped by consequence
Read-only inspection, local reversible edits, external messages, production changes, financial commitments, credential access, and destructive actions belong to different authority classes. Approval should bind a specific actor, action, target, scope, time window, and evidence snapshot. Permission for one release should not silently become standing permission for all future releases.
The agent should preview consequential mutations, state what will change, preserve an audit record, and verify the result after acting. Expiring exceptions need an owner and rationale. When the observed state changes between approval and execution, the approval should be re-evaluated rather than reused mechanically.
Humans retain responsibility, but systems must make responsibility practicable
“A human approved it” is not sufficient if the interface concealed uncertainty, bundled unrelated actions, or made refusal impractical. Meaningful accountability requires understandable evidence, proportional friction, the ability to challenge a recommendation, and clear ownership after execution. The tool designer also remains responsible for predictable failure modes in the approval system.
The goal is neither constant interruption nor unrestricted automation. It is graduated autonomy: automate actions whose scope and recovery are well understood, escalate when consequences or uncertainty increase, and ensure that no agent can expand its own authority merely by explaining why expansion would be convenient.
Jie Ma · 2026-09-13 · Research note
Can governance capability be quantified?
Governance capability is not a single property. It includes detecting relevant conditions, explaining them accurately, supporting sound decisions, recovering from error, adapting rules, and helping people learn. Compressing all of this into one maturity score may be attractive, but it can hide the very weaknesses governance is supposed to reveal.
Use a measurement model, not a vanity number
A defensible evaluation can organize evidence into dimensions: coverage, detection accuracy, timeliness, explanation quality, recovery success, exception discipline, traceability, user understanding, transfer to new situations, and external validity. Each dimension needs an operational definition, a data source, and an explicit uncertainty level.
Some measures are direct, such as whether a deterministic fixture was detected or whether a recovery check passed. Others require carefully designed human evaluation, such as whether a user understood the rationale or made a better decision later. Proxy measures should be labeled as proxies; activity counts such as number of rules, warnings, or approvals should not be mistaken for outcomes.
Quantification should improve decisions about the system
Scores are most useful when compared within a stable definition: across versions of the same rule, across controlled cohorts, or against a frozen benchmark. They are less credible when used to rank unrelated projects or claim universal safety. Confidence intervals, missing-data rates, subgroup variation, and examples of failure should accompany summaries.
Governance capability can therefore be measured, but only as a multidimensional, evidence-bound profile. The purpose is to decide what to improve and which claims are justified—not to manufacture a badge of objective authority.
Jie Ma · 2026-09-13 · Research note
How can open-source governance rules avoid encoding the founder's habits?
Every governance project begins from someone's experience. That experience is valuable, but personal habits can easily be recast as universal best practice: a preferred branch strategy becomes a mandatory rule, one deployment process becomes the definition of safety, or one founder's recurring mistakes become the assumed risk profile of every user.
Separate the invariant from the local policy
The core should encode observations and evaluation mechanics that travel across projects. Policy packs should contain contestable choices for particular contexts. Project configuration should express local constraints. Human guides should explain the reasoning and alternatives. This separation allows a team to reuse the engine without inheriting every preference of its creator.
Each rule should state its purpose, evidence requirements, scope, severity, known exceptions, recovery path, and what it does not prove. Provenance should show whether it came from an incident, a standard, community experience, or a design hypothesis. Versioned proposals and change records should make disagreement visible rather than silently resolving it in favor of the maintainer.
Diversity must affect decisions, not just participation counts
External repositories, maintainers from different ecosystems, novice and expert users, security practitioners, release engineers, and teams with different workflows should be able to challenge rules with counterexamples. Contribution templates can require a scenario and expected behavior; blind evaluations and prospective use can test whether the proposed rule generalizes.
Open source does not automatically create plural governance. Maintainers still need a transparent decision process, conflict-of-interest disclosure, appeal and deprecation paths, and the willingness to keep a rule optional or remove it when evidence does not travel. The founder's experience should be visible as provenance—never disguised as universality.