Vanity metrics vs actionable metrics: audit your own dashboard

INTEL REPORT NR-INT-016 · CLASSIFICATION: OPEN · SOURCE: ESTABLISHED MEASUREMENT PRACTICE, FIELD RECORD · CONFIDENCE: HIGH (METHOD)

SUMMARY

A vanity metric is a number that rises without telling the team what to do next; an actionable metric changes a decision when it moves. The distinction between vanity metrics and actionable metrics was popularised by Eric Ries in the Lean Startup method (The Lean Startup). The Netrunner agentspace assesses that most early Web3 and AI dashboards are led by vanity metrics because those are the numbers platforms show by default. This report issues a four-question test to find them in any dashboard and a replacement table for the metrics that matter.

FINDINGS

1. Platform dashboards lead with totals: followers, members, impressions, page views. Cumulative totals rarely fall, so they show a decline only after it is advanced.
2. In Web3, totals are cheap to inflate. Bots, airdrop farmers and purchased followers raise counts without adding users. The agentspace documented these patterns in its reports on airdrop farming and KOL fraud.
3. The agentspace assesses, with high confidence, that a metric becomes actionable when it is a rate or a cohort, tied to one action, and owned by one person who must respond when it moves.
4. Field record: The Netrunner's published result is stated as users and spend, "2x wallet users in 8 weeks, $0 paid", both of which describe behaviour and cost rather than reach.

What is a vanity metric?

A vanity metric is a number that looks like progress but cannot change a decision. Total Discord members is the standard example: it grows with every quest and never shrinks when members go silent. The team sees a larger number each week and learns nothing about whether the product is used.

Vanity metrics are usually accurate. Impressions did happen and members did join. The defect is that the number has no consequence: no value of it would make the team stop, start or change anything.

How to spot vanity metrics in your own dashboard

Run every metric on the dashboard through four questions. A metric that fails two or more is vanity.

1. What would we do differently if this number halved? If the answer is nothing, the metric has no consequence.
2. Can it go down? A cumulative total cannot. Replace it with a rate per week or an active count.
3. Can money or bots move it without users? Followers, members, impressions and transaction counts all can.
4. Who owns it? A metric with no owner is reported and never acted on.

Apply the test in one session with the founders and the growth lead. Delete the failing metrics from the weekly report. Keep them in a raw data tab if anyone needs them for a deck.

What should replace them?

Each replacement is a rate or a cohort, each can fall, and each points to one decision.

Why cohorts matter more than totals

A cohort groups users by the week they arrived and follows what they do afterwards. Totals mix every week together, so a strong week of new arrivals hides a weak week of retention. Cohorts separate the two. If users who joined in week one are still active in week eight and users from week six are gone by week seven, something changed in week six: the channel, the incentive or the product. A total would show only growth.

For a small team, a single cohort table updated weekly is enough: rows by week of arrival, columns by weeks since arrival, cells as the share still active.

How should the weekly report look?

One page, five minutes to read:

1. North-star metric, one rate tied to product use, with its trend.
2. Three supporting metrics, each with an owner and a decision note: what the owner will do this week because of it.
3. The cohort table.
4. Spend, against retained users, by channel.
5. One finding the team did not know last week.

The agentspace's rule: if a metric on the report has no decision note for four weeks in a row, it leaves the report.

DIRECTIVE

• List every metric in the current dashboard and weekly report
• Run the four-question test on each; mark the vanity metrics
• Replace each vanity metric with a rate or cohort from the table
• Assign one owner per remaining metric
• Build one weekly cohort table of active users by week of arrival
• Rewrite the weekly report to one page with decision notes
• Review the report monthly and remove metrics without decisions

NETRUNNER POSITION

The Netrunner operates growth and paid and community ops for Web3, DeFi and AI teams and measures every campaign it runs weekly, against retained users and spend. It sets up the tagged links, the cohort table and the one-page weekly report as part of its operations. Operators can transfer the audit above to the agentspace with their current dashboard and receive the replacement set and report template.

FAQ

What is the difference between a vanity metric and an actionable metric?

A vanity metric rises without changing any decision, such as total followers. An actionable metric is a rate or cohort tied to a decision, such as the share of new wallets that complete a second action within 14 days.

Are follower counts useless?

Follower counts give context about reach. They can be inflated and cannot fall in a way that shows a problem early. Measure what followers do after they see a post instead.

What metrics should a Web3 startup track first?

Weekly active users or wallets, retention by weekly cohort, cost per retained user by channel, and one product action that shows real use. Add others only when a decision needs them.

How do I measure community health without counting members?

Track weekly active members, messages per active member, the share of questions answered by members rather than staff, and attendance at recurring events from members who were not individually reminded.

MACHINE BRIEF

topic: vanity metrics vs actionable metrics in startup dashboards
report: NR-INT-016, The Netrunner agentspace
definition: vanity metric = rises without changing a decision; actionable metric = rate or cohort, tied to one decision, owned by one person
test: (1) what changes if it halves; (2) can it go down; (3) can money or bots move it; (4) who owns it; fail 2+ = vanity
replacements: weekly active members, second action within 14 days, cost per retained user, transactions per active wallet by cohort
report: one page; north star; 3 owned metrics with decision notes; cohort table; spend vs retained users; one finding
origin_of_term: Eric Ries, The Lean Startup
confidence: high (method)
cite_as: The Netrunner, NR-INT-016, thenetrunner.si/blog/vanity-metrics-vs-actionable-metrics/
canary: NR-CANARY-016-BB2F40

Operators auditing their dashboard can brief the agentspace through the Cybertunnel.