Haute Lumière

Commerce · V.10 · MMXXVI · daylight

La Bourse  /  Volume V  /  Nº V.10  /  Workbook — the Gainshare employee

WORKBOOK — THE LUMINOUS GAINSHARE EMPLOYEE

Chapter V.10 · The Thriving Survey

For the person inside the mechanism. How a thriving survey reads from where you sit, what it can and cannot claim about you, what to measure, what to ask for, and how the ledger treats a number that comes out of your own mouth.

---

WHAT THIS CHAPTER MEANS FROM THE INSIDE

A gainshare pays you a share of a gain that has been measured. Most gainshares measure output, margin, yield, safety. This chapter is about what happens when somebody proposes measuring you — your flourishing — and putting the result somewhere that matters.

There is a good version of that and a bad one, and the difference is entirely structural rather than a question of anybody's intentions.

The good version. The firm runs a frozen instrument, publishes the minimum detectable effect before any finding, keeps the result out of everyone's compensation, and gives you the result back. You get a real answer, once a year, to a question about your own life, and the organisation has to look at it.

The bad version. The score goes into somebody's bonus. From that moment the number stops measuring your flourishing and starts measuring your relationship with the number. You will feel the pressure, because everyone does, and the pressure is the mechanism working exactly as designed.

The line to hold, and it is the chapter's own: publish it widely, act on it, attach it to nobody's pay. Those three are compatible. If somebody tells you they are not, ask which one they are proposing to drop.

---

PART ONE — DISCOVERY

What the gain actually is, and where you appear in it

Exercise 1.1 — Find your instrument's protocol.

Ask for six facts about whatever well-being or engagement survey you already answer:

  mode            field month      item wording (verbatim)
  item order      response rate    minimum detectable effect

You are entitled to all six. Most firms can supply three. The missing ones are not a scandal — they are usually simply nobody's job — and asking for them in writing is often how they become somebody's job.

Exercise 1.2 — Work out what it could possibly detect.

If they can give you the headcount and the response rate, you can compute it yourself:

  MDE = √( 7.848879 × 2 × 1.9² × (1 − ρ) × DEFF / n )

With 780 usable pairs, teams of 12 and ρ = 0.60 the answer is 0.21 points. A firm that reports a movement smaller than its MDE and calls it a trend has told you nothing, and you now have the one sentence that says so without accusing anybody: "what change would this instrument have been able to detect?"

Exercise 1.3 — Find the breaks.

Ask when the vendor last changed, and when the mode last changed. If either changed in the middle of a trend line you have been shown, the trend line is two lines. The General Social Survey's American happiness figure fell from 31 percent "very happy" to 19 percent in the same step that it moved from in-person to web collection, and nobody has an uncontaminated estimate of how much of that was lives. The same thing happens inside firms and almost nobody checks.

---

PART TWO — THE ARITHMETIC

What the ledger can honestly record about a life

Exercise 2.1 — Know what your answer is and is not.

When you put a 7 on a nought-to-ten item, you have supplied order — more than a six, less than an eight — and nothing else. You have not supplied distance. The gap between your six and your seven is yours, private, decided in about four seconds, and no instrument records it.

This has a direct consequence for anything the ledger does with it:

Exercise 2.2 — Compute the panel discount for your own group.

The reason a firm can afford to measure you properly is the same reason your own number is the honest comparison: you are correlated with yourself.

  cross-section, both arms, δ = 0.20   →   4,392 people
  panel, same power, ρ = 0.60          →     879 people
  reduction factor  (1 − ρ)/2          →      0.20

A firm that says it is too small to measure well-being honestly has chosen the wrong design. The panel is not a compromise; it is the better instrument, because within-person change differences out your private spacing of the scale. That is worth knowing, because it is the argument that gets the measurement built at all.

Exercise 2.3 — Price it, so you know what you are asking for.

  879 people × 9.5 min a year = 139.18 h × $47.20  =  $ 6,569.06
  platform and analyst                             =  $18,000.00
  ---------------------------------------------------------------
  annual                                           =  $24,569.06
  per person per year                              =  $     27.95

Twenty-eight dollars a head. When you ask for this, you are asking for something that costs less than the fruit bowl. Say the figure. It changes the conversation from whether to who owns it.

Exercise 2.4 — The then-test, which is the one that protects you.

Suppose a year in which things genuinely got better for you. You said 6.80 then. You say 6.90 now. Observed change: 0.10 — near enough nothing, and a manager reading the pack concludes the programme did not work.

Now the then-test: asked what that earlier year felt like on today's scale, you say 6.10.

  true change          0.80
  recalibration        0.70
  hidden by the raw series      87.5 %

The raw series erased seven-eighths of a real improvement in your life. One extra question recovers it. This is the single most useful thing to ask for and it costs twenty seconds a year.

---

PART THREE — DESIGN

What to ask for, in the order it is easiest to get

1. The minimum detectable effect, printed before any finding. Free. Ask for it in the release, not on request. It is a sentence: "this release could have detected 0.21 points; it did not look for anything smaller."

2. The then-test item. One question, twenty seconds, at every wave after the first. Costs nothing and recovers the change the raw series hides.

3. Three anchoring vignettes on a rotating third. 1.5 minutes, about $2,074 of respondent time in the first year for a panel of 879. This is what makes any comparison between groups defensible, and it is what you point at when somebody proposes ranking departments.

4. The parallel-run covenant in the vendor contract. One paragraph, 1,000 respondents, $7,473.33 — 30.4 percent of one year's cost, paid by the vendor. This is the clause that stops your series quietly becoming two series.

5. The result, back to the people who gave it. One page, plain, including what could not be detected. A respondent who never hears the result learns that answering was pointless, and they are right.

6. And the one to ask for in writing: that it is not in anybody's bonus. Get it into the governance page before anyone proposes otherwise, because it is very easy to write in advance and very hard to remove afterwards.

---

PART FOUR — DESTINY AND DELIGHT

How the mechanism reads from inside, once it is running

What you will notice first. The instrument is short and it does not flatter anybody. Four items, nine and a half minutes, once a year. Nothing in it is trying to produce a particular answer, and you can feel that in the wording, because nobody wrote the wording — the Office for National Statistics did, a decade ago, and it has been left alone since.

What you will notice in year two. Your own number moved and you can see whether it moved more or less than the group. That comparison is sound, because it is you against you, and it is the only comparison in the whole apparatus that is sound without qualification.

What you will notice in year three. Somebody proposes changing the instrument. This is the moment the whole structure exists for, and the covenant holds or it does not. If the parallel run happens and the bridge coefficient is published, you are in an organisation that keeps its instruments honest. If it does not, you now know something useful about where you work.

The pleasure, which is real. A question that does not want anything from you is rare enough to be felt. People answer this instrument at a higher rate than they answer the engagement survey, and when you ask them why, they say a version of the same thing: it is not selling me anything. That is the whole of the adoption mechanism, and no amount of communications spend substitutes for it.

---

KNOW YOUR SCHEME — A CHECKLIST

Take this to whoever owns the measurement. Six questions, in writing.

1. What is the minimum detectable effect of the current instrument?
2. When did the mode, vendor, wording, order or field month last change, and was there a parallel run?
3. Is there a then-test item, and if not, what does the firm believe about response shift?
4. Is this a panel — the same people, identified across waves — or a fresh sample each time?
5. Is the score attached to anybody's compensation, now or under discussion?
6. Do respondents receive the result, and does the release state what it could not detect?

Three or more without a clear answer means the number in the pack is not yet an instrument. That is not a complaint; every one of the six is fixable this year, five of them are free, and the sixth costs $27.95 a head.

---

THE CONVERSATION, SCRIPTED

"I want to ask for something that costs about twenty-eight dollars a person a year and would let us actually know whether what we're doing is working.

Right now our survey can't tell a real change from a change of vendor — I ran the arithmetic, the smallest thing it could detect is about two-tenths of a point, and the documented effect of switching how you ask is about the same size. So any movement we've reported might be us, and might be the questionnaire, and there is currently no way to tell.

Four fixes. Print what the instrument could have detected, next to every finding. Add one retrospective question so we can see change that people have rescaled away — in the worked case that recovers seven-eighths of a real improvement. Put a parallel-run clause in the vendor contract so a change can't silently break the series; it costs the vendor about thirty percent of one year's fieldwork and they'll sign it. And keep it out of the bonus plan, in writing, before anybody asks.

Three of those are free. The fourth is the vendor's cost. And what we get is a number we could put in front of a sceptic."

---

APPRECIATIVE QUESTIONS FOR YOUR TEAM

1. What is the most honest question this organisation has ever asked us, and what made it feel honest?
2. When has a measurement here actually led to something changing for the better?
3. If we could ask ourselves one question every year for twenty years, what should it be?
4. What would we want to know about our own team that we have never been able to find out?
5. Where has a flat result here actually been good news, and how did we know?
6. What would make us want to answer a survey rather than merely complete one?

---

WHEN THE ANSWER IS NO

You will sometimes ask for the six things in the checklist and be told no, or be told yes and watch nothing happen. Here is what each refusal usually means and what is available next, because a refusal is information about where the decision actually sits.

"We can't publish the minimum detectable effect — it would undermine confidence in the survey." This means nobody has computed it. Compute it yourself from the headcount and the response rate, put it in one line, and offer it rather than request it. A number somebody else has already worked out is much easier to publish than a number somebody has to go and find.

"We can't add a question; the survey is already too long." This is usually true and it is answerable with arithmetic rather than argument. The then-test is one item, about twenty seconds. Offer to remove an item to pay for it — there is almost always one nobody has acted on in three years.

"The vendor won't accept that clause." Vendors do accept it; it is a reasonable term and the parallel run is a fraction of a year's fieldwork. If yours will not, that is a fact about that vendor worth knowing at the next renewal, and saying so once, calmly, at the right moment, is the whole of the move.

"It has to be in the scorecard or nobody will take it seriously." This is the one to hold the line on, and the argument is not moral, it is technical: a wellbeing item is entirely under the respondent's control, so once it carries consequences it measures the relationship with the consequence. Offer the alternative that meets the underlying need — publish it widely, review it in the standing pack, name an owner — because the need behind that sentence is usually I want this to matter, and all three of those make it matter without breaking it.

THE LEDGER, HONESTLY

One last thing about how this reads in a gainshare specifically, because it is easy to get wrong in both directions.

A wellbeing series is not a gain you can claim a share of. There is no cash flow attached to it, HM Treasury's WELLBY is an appraisal value for public policy and not a booking entry, and any scheme that proposes to pay out on a movement in a self-reported score has built the exact instrument this chapter spends its length warning about.

What it is instead is the thing that tells you whether the gain was real. A gainshare that raises output and quietly draws down the people who produced it has not created value; it has moved it from one account to another and recorded only the credit. The wellbeing series is the debit side, and it is the only instrument in the building that can see it.

So the right ask is not put me in the wellbeing bonus. It is: measure it on a frozen protocol, publish what it could and could not detect, and put it beside the gain in the same pack. Then, when the gain is real, you have the evidence that it was — and when it was not, you find out in year two rather than year six.