When Metrics Start Defining What Matters

What organizations measure does not only reflect what matters. Over time, it can help determine what becomes important.

Principle

Measurement should be designed around the decisions an organization needs to make — not around the data it happens to have.

Commercial Measurement should make the relevant parts of Commercial Reality observable for decisions that matter.

We tend to assume that measurement follows importance.

Something matters to the business, so we find a way to measure it.

That sounds logical. And often it is.

But after spending enough time around dashboards, reporting systems and commercial teams, I think the relationship can also work in the opposite direction.

Sometimes something becomes important because we have learned how to measure it.

Which raises a question I keep coming back to:

Do organizations measure what matters, or do they gradually learn to care about what they can measure?

Measurement always starts with a choice

Take a simple number:

1,247 transactions.

It looks objective. And it may be completely accurate.

But even that number depends on choices someone made earlier.

What counts as a transaction?

How are returns treated?

Which stores are included?

Where does the period begin and end?

Which system do we trust when two sources disagree?

None of this makes the number less useful.

It simply means that measurement is never reality arriving untouched on a dashboard.

Before something can be measured, part of reality has already been selected, defined and made observable.

And that is unavoidable.

No organization can see everything happening inside its business.

The interesting part begins with what happens next.

A metric can acquire a life of its own

There are thousands of data points sitting inside most organizations that almost nobody cares about.

A metric becomes powerful when people begin organizing behavior around it.

It starts appearing in a report.

Then in a meeting.

Someone gets asked to explain why it moved.

A target is attached to it.

A team becomes responsible for improving it.

Perhaps a budget or a bonus eventually depends on it.

After a while, the metric no longer feels like one possible way of looking at the business.

It simply feels like performance.

This is an important distinction:

Commercial relevance is not the same as organizational importance.

Something can matter enormously to the business and still be difficult to measure.

Another variable may be easier to capture, compare and report, and therefore become central to the way the organization manages itself.

That is why measurement systems do more than describe priorities.

Over time, they can help create them.

The longer we measure something, the harder it becomes to question it

This is where things become particularly interesting.

Once a metric has been used for years, it accumulates history.

There are benchmarks.

Targets.

Previous results.

Reporting templates.

People who own it.

Teams that know how to improve it.

Perhaps incentives attached to it.

Even when people know the metric is incomplete, replacing it can be surprisingly difficult.

Not because anyone believes it is perfect.

Because the organization has learned to operate through it.

I think this is one of the reasons measurement systems can become so persistent.

The organization measures something because it matters.

Then, gradually, it can start to matter partly because the organization measures it.

The causal direction begins to blur.

Where the process starts matters

Ideally, a commercial measurement conversation begins with a decision.

What are we trying to achieve?

What decision are we trying to make?

What would we need to understand to make that decision better?

And only then:

What do we need to observe and measure?

In practice, things are not always so clean.

Organizations already have data.

They already have systems.

And it is entirely reasonable to use what is available.

The risk appears when the available data quietly starts defining the boundaries of the questions we ask.

We have these numbers, therefore we build these KPIs.

We have these KPIs, therefore we build these dashboards.

We have these dashboards, therefore these are the conversations that appear in management meetings.

None of those individual steps looks problematic.

But together they can produce something strange:

an organization can become more data-driven without necessarily becoming better at understanding the reality of its business.

More data is not the problem.

The problem is assuming that a more detailed representation must also be a better one.

I see this clearly in Retail Media

In-store Retail Media is a useful example because the evolution is quite visible.

If the system allows us to see mainly playouts, impressions, reach or Share of Voice, it is natural to think about the screen as media inventory.

Those are the things we can see, so those are the things we manage.

Then new measurement capabilities appear.

Exposure can be connected with transactions.

Shopper behavior can become observable.

Incremental outcomes can be estimated.

The screen itself has not changed.

The store has not suddenly become a different commercial environment.

What has changed is what the organization can observe about what happens around that screen.

And that changes the conversation.

The question can move from:

How much media did we deliver?

to:

Did that exposure actually influence purchase?

That is more than adding another KPI to a dashboard.

It changes the way the asset can be understood.

What we do not measure does not disappear

Imagine a shopper entering a store.

They notice a product.

Check the price.

Pick it up.

Think about it for a few seconds.

Put it back.

Then leave without buying it.

Something commercially relevant happened.

But if our system mainly sees transactions, it may record almost nothing.

No sale.

No conversion.

Perhaps no trace at all.

The event still happened.

It just sits outside the representation we are using to understand the store.

This is why I think accuracy, on its own, is not enough to judge a measurement system.

A system can measure the wrong slice of reality with extraordinary precision.

The harder question is whether we are making the parts of reality visible that are relevant to the decisions we actually need to make.

We need representations

None of this is an argument against dashboards, KPIs or models.

Quite the opposite.

Organizations cannot operate without simplifying reality.

Commercial Reality is too complex.

We need abstractions.

We need forecasts.

We need attribution models.

We need metrics that reduce thousands of events into something a person can actually use.

The mistake is not building representations.

The mistake is forgetting that we built them.

A dashboard is not the business.

A KPI is not performance itself.

An attribution model is not causality.

A forecast is not the future.

They are ways of making parts of reality understandable enough to support a decision.

That is why I increasingly think the question should not simply be:

Do we have enough data?

It should be:

Have we made the right parts of reality observable for the decisions we need to make?

Because what becomes visible tends to receive attention.

What receives attention influences decisions.

And those decisions eventually change the same commercial reality we were trying to understand in the first place.

Which brings me back to the question at the beginning.

Do organizations measure what matters?

Or, over time, do they learn to care about what they can measure?

This essay is part of the In-Store Retail Media Framework under Commercial Measurement. New readers can begin with Start Here or explore the Principles that guide the Journal.