And before we start, a call out to Roger Martin for the wonderful phrase "What would have to be true"

Modelling with what was to hand

On a recent engagement, we were replacing a manufacturer's distributor portal. It was a complicated environment, and I needed something to demonstrate the ecosystem. I went with a service blueprint style and looked for the business services the new portal would provide. The client had no architecture practice, no repository or capability model or the usual information. What we had was whatever was to hand: several hundred workshop requirements, a partial journey list, prototype screenshots, core information objects retrofitted from those screenshots, a mapping of the core system's data, the client's value stream, and notes from interviews. It was full of inconsistencies and not a model.

Decisions were required, and it was action before analysis, so I modelled as we went. I used IAF's ways into business services, through objects, roles, goals, events and processes, and drew candidate services from each source.

I placed the services on the value stream, added the back-end systems behind each one, and rated each service for risk against high-risk unknown areas and for desirability based on evidence of user pain. It reminded me that a model is always worth having, even if it is an incomplete one.

Going back to being an EA

It also sent me back to the framework I trained in. IAF is usually met as a large method for documentation, with a repository to fill and deliverables to produce, and used that way it drives a mass collection of data that few people read. What it does better is improve the quality of thinking, because underneath the deliverables it is a short set of questions.

Down the side are four: why we are doing this, what that requires, how it should be structured, and with what it will be delivered. Across the top are the areas those questions are asked of: business, information, information systems and technology. Read downwards, the grid is a chain of decisions; each level answers the one above, so a platform choice can be traced back to a reason. Read across, it shows dependency, with the business need shaping the information, which shapes the systems. Read upwards from what exists, it becomes an assessment of what today's estate supports.

The questions apply to every organisation, whether or not it has an architecture practice, and they always will. Every organisation has reasons, needs, some structure and real things that deliver it; the only difference is whether anyone has written them down. That is why the grid worked on the portal with nothing but workshop notes and screenshots. I didn't need to fill it, only to ask each question of what I had, and the places where the chain broke were the useful part: a system with no need above it, or a goal with nothing below it.

A capability isn't a single cell; it's a slice through the grid, from the outcome the business wants at the top to what delivers it at the bottom, and one slice per capability fits on a page.

An empty stage

One stage of the value stream, handover to the end customer, had nothing behind it. None of the sources said so, because each of them described what was there. A requirements list can't contain what nobody asked for, and a data mapping can't map a process that doesn't exist.

The model could show the gap because it had a place for something that wasn't there. That answered the question about whether it was worth having, at least for this decision, and it is the argument of this piece in small.

However little model we keep, we should keep one habit: before acting on a pattern, write down what we think causes it and what would show we're wrong.

Where the profession is heading

At an EA conference this autumn, I listened to a keynote on adaptive architecture. It made me think of change at pace, the ability to pivot, and less weight of process and models, but not at the expense of quality thinking.

During sessions on organisations established models I noted that many organisations have no architecture to work with: housing associations that don't see themselves as complex enough, ERP-based organisations that assume one platform is enough, and investor-owned businesses that can't see how architecture drives revenue. I could probably go on, and the key in all of those is how it drives value. Those organisations still make decisions, and each one rests on some idea of what causes what; the process of architecture goes on whether or not anyone calls it that.

So, the habit doesn't depend on having a practice, a repository or a team. Writing down the claim and what would show it wrong can be done with whatever is to hand, and it makes the reasoning visible to the people who must live with the decision.

The end of theory

One line from my conference notes was that being data-led can mean being bias-led. The clearest statement of the opposite view is from 2008, when Chris Anderson, then editor of Wired, published "The End of Theory: The Data Deluge Makes the Scientific Method Obsolete".

He argued that with enough data you no longer need a model of why things happen; correlation is enough to act on. Geoffrey West quotes it in Scale to disagree on the grounds that data tells you what has happened but not what will happen when conditions change.

Read now, Anderson's piece is an accurate description of a temptation, written by someone who thought he was describing progress. Each of the shifts I hear in my work pushes towards it. Pace strengthens the bias for action and shortens the time to test anything; AI pattern-finding invites us to treat what it finds as the answer; and just enough model can quietly become no model, unless what remains still makes a claim about cause.

The model doesn't disappear when we stop drawing it, though. Nobody acts on a correlation without an explanation in their head, and under pressure to act, that explanation is usually the one they already held.

So, people pick the correlations that fit and call the result data-driven (after all, they stated the requirements for the dashboard!).

The real choice is between a model that's written down and can be checked, and one that isn't, which is why the habit matters more as the model gets lighter.

Where ideas come from

Several conversations at the conference were about repositories: which tool, how to populate it, how to keep it current. I want to be a little contrary about this. Inspiration doesn't come from the repository.

In science, the idea comes from a question. Karl Popper once began a lecture to physics students by telling them to take pencil and paper, observe carefully and write down what they saw. They asked, reasonably, what they were supposed to observe. Data can prompt a question, but usually because it contradicts an expectation, and an anomaly needs a model before it can be one. Scientists then collect data to test the idea, and at the edge of what is known they often have to work hard, building new instruments, to get it at all.

That doesn't make data unimportant. It supports the whole journey of a change: it prompts questions when something doesn't fit, it tests the idea, and once the change is live it carries the organisation through running, sustaining and eventually retiring what was built. Building architecture has reached the same place with Building Information Modelling, where much of the case for a rich data model of a building rests on constructing it and the long years of running it, rather than on where the design came from.

Most of the data in an EA repository answers run questions: how many applications we have, what they cost, when they go out of support. A repository answers those well, and they matter. But they are questions about keeping the organisation going rather than moving it forward. In IAF terms, the repository mostly fills the bottom row, with what, and a question about change starts at the top, with why.

So, a repository can't supply the hypothesis. It can help test one, once there is a question to put to it, and it carries what we decide through the rest of its life. Collecting more in the hope that direction will emerge from it is EA's own version of the end of theory.

What the data can't see

On the portal, one of the capabilities was keeping distributors informed about their orders. Written as a hypothesis, it was: if we can keep distributors informed of fulfilment, status calls will fall. That makes a claim about cause, that distributors call because they don't know where their orders are.

For it to hold, three things had to be true: carrier data had to exist, the core system had to be able to send updates when an order changed, and someone had to own communication with distributors. A fourth was needed to test it at all: someone had to be counting status calls.

The first two were technical questions. The last two were not, and neither was true; nobody owned distributor communications, and nobody counted status calls. The feature could have been built well without anyone knowing whether it worked, and a data-led approach would have had nothing to read, because the number that mattered didn't exist. Writing the claim down is what brought both gaps into view.

Data has two other limits that a written claim helps with. A correlation holds only while the conditions behind it stay the same, as West argues, and a change programme is a change of conditions, so what we sense must be checked for still being true.

Data only sees what is measured, while most of the risk in change sits in the seams, especially between people, in things like slack, trust and the roles that hold work together.

The pull towards change that moves a dashboard is one everyone feels, me included. A written claim at least makes it obvious when the dashboard is all we have to go on.

How science adapted

Science has been under the same pressure to move faster and lean on large data sets, and it kept the hypothesis by changing its form. Preregistration asks researchers to state the hypothesis, the method and what would count against it before they see the results, so an explanation can't be fitted afterwards. Adaptive trials let a trial change as it runs, dropping or adjusting arms as evidence comes in, but only by rules set in advance.

Neither returns to the slow, complete form of the classic study. Both keep the part that matters, which is saying in advance what would have to be true. Preregistration is the habit this piece is asking for, in another field's language.

What to keep

The ask, then, is small: before acting on a pattern, write down what you think causes it and what would show you're wrong. On the portal the capability slice did this for the distributor hypothesis. Its gaps were the four conditions above, the things that would have to be true, in Roger Martin's phrase.

From there the method is short. Pick at most three questions as the cheapest evidence that would confirm or kill the hypothesis, stop once the next decision is clear, and set a point, such as the sprint boundary, where it will be revisited and could be shown wrong. Stating the hypothesis first is the preregistration step, and revisiting it at a point fixed in advance is the adaptive trial's rule; it is the same discipline at the scale of a portal project.

Framing the hypothesis as a capability matters. Change doesn't necessarily build capability; the distributor feature could have shipped and worked without the business getting any better at keeping distributors informed, and a dashboard can move for reasons that have nothing to do with what the organisation is able to do. Holding the hypothesis at the capability level, where things change slowly, gives the work a direction, and the sprint boundary is where course is corrected at the level of change, where things move quickly. That is one way to have the pace the conference asked for without giving up the reason for moving.

Two things follow from the ask. The first is to check the claim against what the data can't tell you: whether it is still true, and what it leaves out.

The second is to teach the habit rather than guard it. Good leaders already do architecture without mentioning architecture (the first rule of architecture club is….), and AI now does parts of it, so our value can't rest on owning the method. In an organisation with no architecture, a written claim is often the first piece of architecture it has.

None of this is an argument for the complete model. Backing a model means committing to what is enough for the next decision, not defending a settled picture of the whole. But it does mean committing. A model you stand behind is one you can be wrong about, personally and in front of people, and part of the appeal of being data-led is that nobody has to be. I'd rather be wrong in a way someone can check.