La Bourse / Volume VI / Nº VI.10 / Quiz, reflection, essays

Four colleagues standing in conversation by a wall of windows, the sun low behind them. — Plate VI.10 · Quiz, reflection, essaysFive Years of Tuesdays.Nobody wrote these to be compared with each other. That is exactly why they can be.

ASSESSMENT · Chapter VI.10 — Auditing a Governance System

Three instruments: a ten-point quiz, eight reflection questions, five essay prompts. The quiz checks comprehension rather than recall. The reflections are private and first-person. The essays are arguable from more than one side.

THE QUIZ — ten points

Four on recall.

1. Name the six measures of a governance audit.

Weighted participation; decision latency; reversal rate; the share of decisions taken by exception; concentration in the minutes over a year; and the gap between the written rule and the observed practice. One mark for the six, one for noting that they are reported as a profile and never summed.

2. Write the effective-number formula and say what sᵢ is.

N_eff = 1 / Σ sᵢ², where sᵢ is person i's share of the total decision weight. It returns the headcount for an assembly of equals and roughly the size of the deciding tier when one small tier holds the vote. The formula is Laakso and Taagepera's, from counting political parties.

3. What did Baggio and colleagues find about Ostrom's design principles, and what does it forbid?

Across sixty-nine cases, no single principle is necessary and none is sufficient; they operate in configurations. It forbids summing them into a score — six of eight is not three-quarters of a working commons.

4. Why is reversal rate replaced by non-implementation rate?

Because reversing requires a motion and abandoning does not. A body can drop half its decisions by letting them lapse or re-scoping them, with a spotless reversal record. Non-implementation is computed from procurement, work orders and payments — systems the governing body does not control.

Four on application.

5. A community land trust reports 340 people at its annual meeting, up from 190, and calls participation "nearly doubled". Twelve trustees hold every vote between meetings. Diagnose the claim.

It is a count of attendance presented as a measure of participation. Under N_eff the answer is dominated by the twelve, so the reported near-doubling moves the effective number very little. The stronger answer adds two things: that the right response is to compute N_eff from the decision record rather than policing the attendance sheet, and that the finding is not that the trust is undemocratic — twelve trustees chosen by 340 members may be a perfectly good arrangement — only that the sentence as written is not supported by the number cited.

6. An auditor proposes attending six board meetings to estimate how often the board decides by exception. What do you tell her, and what do you propose instead?

Six meetings of eight items is forty-eight decisions, which supports an estimate to roughly ±14 points before any clustering correction — too coarse to distinguish a well-run body from a poor one. Propose the archive: five years of existing minutes is ten times the sample, costs less, and carries no observer effect. Use one or two visits as a validation sample to check that the minutes resemble the meeting, and say so in the boundary statement.

7. A cooperative's chair says: "Our governance is excellent — our rulebook was drafted by the federation's lawyers and we follow it." What single measure tests the claim, and what are its two directions?

Rule–practice divergence. Direction one: provisions creating a right or duty with no trace in the archive in twelve months — dormancy. Direction two: standing practices with no written basis at all. Full marks note that the second direction is Freeman's point in "The Tyranny of Structurelessness": an unwritten rule cannot be contested.

8. Your participation figure rises 40 percent in a year and your protected holdout — names appearing in the minutes who hold no leadership role — does not move. What have you learned, and what must you not do next?

That most of the reported rise is in the measure rather than in the thing the measure stands for; the divergence is your estimate of the gaming component. What you must not do is question the holdout or fold it into anybody's objectives, which is the natural response and converts one corrupted series into two.

Two that cannot be answered without doing the arithmetic.

9. A body takes eight decisions a meeting and meets twelve times a year. You want its exception rate to ±5 percentage points at 95 percent confidence. Decisions inside a meeting have an intra-meeting correlation of 0.15. How many years of one body's minutes do you need, and what is the better design?

Base sample: n = z²·0.25/d² = 1.96² × 0.25 / 0.05² = 384.1, so 385 decisions. Design effect: DEFF = 1 + (m−1)·ICC = 1 + 7 × 0.15 = 2.05. Corrected: 385 × 2.05 ≈ 788 decisions. The body produces 12 × 8 = 96 a year, so 788 / 96 = 8.2 — more than eight years. The better design is comparative: four similar bodies for two years reaches the same sample in two calendar years, because the clustering is in the meeting, not the year. One mark for 385, one for 8.2 and the comparative design.

10. A lender offers a 25 basis point margin step on a £5,000,000 facility against three verified governance targets. Audit and independent verification cost £11,500 a year. Does the covenant clear, what is the breakeven step, and what do you do if the facility were £3,000,000?

Margin saving: £5,000,000 × 0.0025 = £12,500. Cover: 12,500 / 11,500 = 1.09× — it clears, narrowly. Breakeven step: 11,500 / 5,000,000 × 10,000 = 23.0 bp. At £3,000,000 the saving is £7,500 against £11,500`, a cover of 0.65×, so the ratchet costs more than it saves. Run the audit anyway without the ratchet: the six measures earn their keep at £11,500 whether or not a lender is watching, and the first pass is the one that finds the dormant provisions. Full marks require the last sentence — dropping the audit because the ratchet does not clear is the wrong inference.

EIGHT REFLECTION QUESTIONS

Private, first-person, and not for a room.

Think of a body you belong to. Without looking anything up, how many voices does it really have — and how close is that to its headcount?

When did you last change your mind in a meeting and say so out loud? What made that possible that day, and what would make it possible more often?

Which of the rules you operate under have you never seen used? What would it cost you to ask about one of them?

Where in your own working life is there a practice with no written basis that you rely on — and who does it advantage?

Recall a decision you were part of that was never implemented and never reversed. What happened to it, and who, if anyone, noticed?

If somebody counted your interventions in every meeting you attended this year, what would the distribution look like, and how would you feel about the number?

Which measure of your own work would you least like to see computed? What does that reaction tell you about where the useful information is?

If you were asked to name one number about your body that nobody may ever target, which would you choose — and who would object?

FIVE ESSAY PROMPTS

Each is arguable from more than one side. Each requires a source the chapter cites and a source it does not.

1. "An audit of governance is itself an act of governance." Michael Power's The Audit Society argues that auditing constitutes the thing it measures rather than merely reporting on it. Take a position on whether the six-measure audit in this chapter escapes that critique or is an instance of it. You may not resolve it by asserting that this audit is better designed; show the mechanism.

2. Porto Alegre's participatory budget shows 165× between counted and effective participation, and 3.4× more effective participation than the elected chamber beside it. Argue which of those two figures a city should put in its annual report, and why. Then argue the other side as strongly.

3. This chapter claims that three of its six measures are gameable and proposes engineering rather than disclosure as the response. Assess whether that is sufficient, drawing on Campbell's law and at least one documented gaming case outside the commons literature. Is there a fourth measure here that is gameable and the chapter has not admitted it?

4. Ostrom's design principles were found broadly supported by Cox and colleagues and configural by Baggio and colleagues. Argue whether a framework that cannot be summed can still function as a practical standard for a lender, a regulator or a federation — or whether the demand for a score is so strong that a profile will always be collapsed into one by somebody downstream.

5. The chapter argues for auditing the archive rather than the room, partly on the grounds that the Hawthorne effect largely disappeared when Levitt and List reanalysed the original data. Construct the strongest case against that position: when is direct observation not merely useful but the only honest instrument, and what kind of governance failure is invisible in any minute book?