Haute Lumière
Commerce · VI.10 · MMXXVI · daylight
TEN CONCEPT BRIEFS · Chapter VI.10 — Auditing a Governance System
One page each. A reader who reads only these ten pages has the chapter.
BRIEF 1 — Description Is Not Governance
The idea. A constitution tells you what a body has agreed to do. An audit tells you what it does. These are different documents about different things, and the first is not evidence for the second.
This is the same move Elinor Ostrom made about commons. Her contribution was never the claim that commons work; it was eight design principles specifying the conditions under which they hold — and the empirical record behind them, which Chapter VI.02 sets out, is a record of people going and coding cases. Nobody established the principles by reading charters.
Worked example. Two housing cooperatives have the same model rules, purchased from the same federation, differing by no clause. One holds four general meetings a year at which twenty people speak; the other holds four at which the same three people speak and the rest ratify. The rulebooks are identical. The governance is not remotely the same.
Why it matters. It tells you where to look. Every measure in this chapter is computed from a record of behaviour — minutes, registers, work orders, payments — and none from the governing document, except the one measure whose whole job is to compare the two.
You already know this because you have read a company's values statement and then worked there.
BRIEF 2 — Weighted Participation, Not Counted
The idea. The number of people who attended is a count of attendance. Participation is how much of the decision they actually held, and the two can differ by two orders of magnitude.
Use the effective number, which political science has used since Laakso and Taagepera proposed it in 1979 for counting parties:
N_eff = 1 / Σ sᵢ²where sᵢ is person i's share of the decision weight. An assembly of equals returns its headcount. A body where a small tier holds the deciding vote returns roughly the size of that tier, however full the hall.
Worked example. Porto Alegre's participatory budget at its height: about 20,000 attendees a year in a city of 1,360,590, roughly a thousand delegates, and a council of 44 — sixteen regions and six thematic assemblies returning two each. Weight the tiers 60/30/10 and the effective number of participants is 121. Counted over effective: 165×.
Now the other direction, which matters just as much. The city chamber beside it has 36 seats, so the participatory process ran at 3.4× the effective participation of the elected body, and annually rather than once in four years.
Why it matters. It is the one participation figure that cannot be improved by convening another meeting. And it is honest in both directions at once: it deflates the headline and it still says the thing worked.
You already know this because you have been in a meeting of thirty where four people decided, and you did not need a formula to know how many were really in the room.
BRIEF 3 — Publish the Weights With the Number
The idea. The weights in N_eff are a parameter, not a measurement. Nobody observed that the council holds sixty percent of the decision weight; somebody chose it. So print the choice beside the result, the way a confidence interval is printed beside a mean.
This is Chapter II.10's parameter register applied to your own arithmetic. A parameter that has never been varied is a parameter nobody has examined.
Worked example. The same Porto Alegre data at three weightings:
councillor-heavy 0.80 / 0.15 / 0.05 N_eff = 69
central case 0.60 / 0.30 / 0.10 N_eff = 121
assembly-heavy 0.40 / 0.40 / 0.20 N_eff = 263A 3.84× swing on three numbers no dataset contains. Meanwhile the delegate count — a real quantity somebody could go and verify — barely matters: move it from 800 to 1,400 and the central answer moves from 121 to 121.
Why it matters. It tells you where to spend effort. Arguing about the delegate roll is wasted; arguing about the weights is the actual disagreement, and once the weights are on the page that argument can finish.
You already know this because you have seen two teams produce opposite business cases from the same spreadsheet by using different discount rates that neither of them printed.
BRIEF 4 — The Calendar Floor
The idea. Decision latency is the time from a demand's first external trace to money moving. Most of it is usually the calendar, not the governance — so report excess over the floor, and name the floor.
Worked example. A demand raised at a Porto Alegre first-round assembly on 1 April reaches an enacted budget in 258 days and a completed execution year 639 days — about 1.75 years. That looks damning. Then ask: a conventional capital item raised with a councillor on the same afternoon enters the same budget law and the same execution year. Latency attributable to the participatory process: zero days.
Nearly all 639 days are the statutory fiscal cycle. Publish the raw figure and you have audited the constitution of Brazil.
Why it matters. Every governing body sits inside a cycle it did not choose — a budget year, a term, a harvest, a licensing round. The auditable quantity is the part that is not the cycle, and computing it usually exonerates the body rather than condemning it.
You already know this because you have waited eleven weeks for something and discovered that ten of them were procurement.
BRIEF 5 — Measure Non-Implementation, Not Reversal
The idea. Reversal rate is the most gameable measure in the set, because reversing is the one form of failure that requires a motion. A body that never formally rescinds anything can still abandon half its decisions: let them lapse, re-scope them under a new title, or simply not do them.
So measure non-implementation: the share of decisions with no downstream artefact within twelve months.
Worked example. A cooperative board reports a reversal rate of two percent and is proud of it. The audit matches each decision against procurement records, work orders and payments — systems the board does not control — and finds that of sixty decisions in the year, nineteen have no downstream trace at all. The governance record is spotless. The implementation record is not, and only the second one was checkable against somebody else's system.
Why it matters. It catches every variant of the game, including the honest ones, because it is computed from documents the governing body did not author. Report silent reversal beside it: decisions revisited under a different title, found by matching subject rather than wording.
You already know this because you have watched a project be cancelled without anybody ever cancelling it.
BRIEF 6 — The Exception Share, and Stating the Denominator
The idea. Two numbers, always together. What share of decisions were taken outside the standing process — and what share of the organisation's decisions the standing process covers at all.
The second is the one everybody omits, and omitting it is how a numerator gets published as a rate.
Worked example. The Orçamento Participativo governed the investment budget. Capital investment is a minority of municipal spending, so 80 to 95 percent of the money was decided elsewhere, by people the process never touched. Porto Alegre never claimed otherwise; the omission happens downstream, in the retelling.
Why it matters. This is the denominator rule from the architecture doctrine, in governance form: a green check with unstated coverage is worse than no check. An audit that reports a wonderful participatory process governing four percent of the money has not audited the organisation.
The gaming to watch is reclassification: as soon as the exception share is targeted, exceptional decisions start being minuted as routine. The defence is the same as everywhere else in this chapter — date and classify from the first external trace, which was written before anybody was being measured.
You already know this because you have seen a "standard exception" and recognised it immediately as a contradiction that somebody needed.
BRIEF 7 — The Effective Number of Voices
The idea. Take a year of minutes and count recorded interventions by name. Then compute three things: the top decile's share, the number of people appearing exactly once, and the effective number of voices — the same inverse-Simpson formula as Brief 2, applied to speech instead of votes.
Worked example. A worked distribution of 24 names and 498 interventions over twelve months gives a top decile — two people — holding 33.9 percent, four people appearing exactly once (16.7 percent of the named), and an effective number of voices of 9.4: 39.2 percent of the headcount.
The headcount says twenty-four. The room has about nine voices in it.
Why it matters. It is the measure a chair can act on this month, without anyone's permission, and it is almost never hostile: the people at the top of the distribution are usually the most conscientious members, not the most domineering. The singleton count is the operative one. A person who spoke once in a year attended; they did not participate, and somebody should find out why before assuming.
You already know this because you can already name, without looking, the three people who speak most in any group you belong to.
BRIEF 8 — Audit the Archive, Not the Room
The idea. The intuition says go and watch meetings. The arithmetic says the opposite, twice.
First, one meeting is not an instrument. If a body takes one decision in five by exception, a visitor to a meeting of eight items sees no exception at all 16.8 percent of the time. See two in eight and the Wilson interval on the true rate runs 7 to 59 percent — 52 points wide.
Second, the observer effect is real and a field audit cannot measure it. Compare a year of observed meetings with a year of unobserved ones and the minimum detectable difference in exception rate is 16.2 points. Anything smaller is invisible to you.
Worked example. The Hawthorne illumination experiments gave the observer effect its name. When Levitt and List went back to the original archival data in 2011, the canonical effect largely disappeared. The observer effect's own founding evidence did not survive an audit of the records.
Why it matters. The archive is the only sample nobody performed for. It is larger, it is cheaper, and it was written before you arrived. Attend a meeting to check that the minutes resemble the meeting — a validation sample, declared as such — and do the audit on paper.
You already know this because you have watched a team behave differently on the day the visitor came, and known perfectly well that you were seeing the visit rather than the team.
BRIEF 9 — Decisions in One Meeting Are Not Independent Draws
The idea. Sampling arithmetic assumes independent observations. Decisions taken in the same meeting share a chair, an agenda, a mood and an evening, so they are not independent — and ignoring that is the commonest way an audit reports a precision it does not have.
The correction is Kish's design effect:
DEFF = 1 + (m − 1) · ICCwhere m is the cluster size and ICC the intra-meeting correlation.
Worked example. To estimate a proportion to ±5 points you need 385 decisions. At eight decisions a meeting and an ICC of 0.15, DEFF is 2.05, so you need 788. A body taking eight decisions at twelve meetings a year produces 96 a year, so one year supports ±14.3 points, five years of archive ±6.4, and reaching ±5 points from one body alone would take 8.2 years.
Why it matters. It changes the design, not just the caveat. The clustering lives in the meeting, so sampling four comparable bodies for two years buys precision that sampling one body for eight years cannot. Governance audits should be comparative by construction, and this is the arithmetic reason.
You already know this because you know that asking eight people at the same table is not the same as asking eight people.
BRIEF 10 — A Profile, Never a Score
The idea. Six measures. Never one number.
Jacopo Baggio and colleagues examined 69 commons cases configurationally and found that no single Ostrom design principle is necessary and none is sufficient. They work in combinations, and different combinations work in fisheries, forests, pastures and irrigation.
Worked example. Six of eight principles present is not "three-quarters of a working commons". It may be a viable configuration or a broken one depending entirely on which six. Sum them and you have added terms that do not add.
The same holds for this chapter's six measures. High effective participation with high non-implementation is a body that consults widely and does nothing — a different illness from low participation with excellent implementation, which is a competent oligarchy. A single index gives both the same score.
Why it matters. It is also a defence. A profile is much harder to target than a score, because there is no one number to move; and Chapter II.10's warning holds exactly — composites communicate by hiding the weights, and a hidden weight will not be argued with.
You already know this because you would never accept a single number summarising a person's health, and you would ask what it left out.