The uncomfortable question worth asking
After a few years of enthusiastic AI and marketing technology spending, a quieter question is starting to be asked in boardrooms and trustee meetings alike. Is any of this actually working?
The honest answer, for most organisations, has been no. A widely reported study from MIT found that 95% of organisations investing in generative AI had seen no measurable return on that investment, despite tens of billions of pounds poured in across the economy [1]. A 2026 survey of 1,800 executives by Boston Consulting Group reached a similar conclusion, finding that only around a quarter had generated meaningful financial value from their AI [2]. These are not arguments against the technology. They are arguments against buying it on faith and never checking whether it paid off.
For a small business or charity, where there is no spare budget to waste, that check matters even more. The good news is that measuring whether your marketing technology is working is far simpler than the jargon suggests.
Why so much marketing technology fails to pay off
The reasons are rarely technical. The same MIT research found that the barrier to getting value was not infrastructure, regulation, or talent. It was that tools were never properly woven into how people actually work, and that success was judged by the wrong things. A tool that sits to one side of the real workflow, however clever, quietly delivers nothing.
There is also a measurement problem hiding in plain sight. A great deal of marketing spending goes on tools chosen for what they do, not for the outcome they are meant to produce. When nobody defined what success looked like at the start, nobody can say whether it arrived.
Activity is not the same as impact
This is the heart of it. Clicks, opens, impressions, followers, hours of tool usage, even the raw volume of content produced, are all measures of activity. They are easy to count, which is exactly why they are so often counted. None of them is the same as impact.
The technology chief at one of the world’s largest companies made the point bluntly to his own staff recently, reminding them that heavy usage of a tool is not a measure of progress of any kind. A team can be extremely busy with a shiny new system and produce nothing of value. The only questions that matter are whether the work is bringing in more enquiries, more donations, better retention, or genuinely freeing up time, and whether that gain is worth what you are paying.
What good measurement actually looks like
You do not need a data team to measure this well. You need a little discipline at the start and a willingness to be honest later.
- Name the outcome first. Before you buy or switch on a tool, decide what business result it is meant to improve. More qualified enquiries. Higher supporter retention. Ten hours a week given back to the team. Be specific.
- Take a baseline. Write down where that number sits today. Without a before, there is no after, and you will be left guessing.
- Count the full cost, not the sticker price. Include the subscription, the usage charges, the setup time, the training, and the hours spent keeping it running. A cheap tool that eats a week of someone’s time is not cheap.
- Compare honestly. Set the value created, whether that is revenue gained, costs saved, or time returned to the mission, against that full cost. If it does not stack up, that is useful information, not a failure.
Metrics worth your attention, and metrics worth ignoring
Worth tracking are the ones tied to a real outcome. Cost per enquiry or per donation. Conversion from interest to action. Supporter or customer retention. Revenue or funds the activity genuinely influenced. Time saved, measured properly and reinvested somewhere that matters.
Worth treating with suspicion are the numbers that feel good and prove little. Impressions and reach on their own. Follower counts. Engagement that never leads anywhere. Volume of output. How many times a tool was used. These have their place as early signals, but they are the beginning of a question, not the answer to one.
A simple example
Picture a small charity that brings in an AI writing tool to speed up its supporter newsletters and appeals. The subscription is 40 pounds a month, and with the time spent learning it and tidying its output, the true cost in the first few months is nearer 80 pounds a month once someone’s hours are counted.
The outcome it is meant to improve is twofold, namely the time the communications lead spends drafting, and the performance of the appeals themselves. Before starting, the charity notes that drafting a newsletter takes about six hours, and that recent appeals raised an average of a given amount. Three months in, drafting takes two hours, freeing four hours a fortnight for donor relationships, and appeal income has nudged up because messages are going out more often and more consistently.
That is a measurable return. The four hours have a value, the extra income is real, and both comfortably exceed 80 pounds a month. Crucially, the charity can now say so with confidence, rather than hoping the tool is helping. Had the numbers gone the other way, it would know that too, and could stop before the waste compounded.
Give it time, but set a checkpoint
None of this means judging a tool in its first week. New systems take time to bed in, and pulling the plug too early is its own kind of waste. The trick is to decide, in advance, when you will honestly review it. Put a date in the diary, three or six months out, and hold yourself to it.
The small minority of organisations that get real value from AI share one habit above all. They evaluate by business outcomes rather than by how impressive the tool is, and they are willing to stop what is not working. Analysts expect a significant share of AI projects begun in recent years to be abandoned for poor returns [3]. A deliberate review on your own timetable is a great deal better than an abrupt cancellation when the budget review comes around.
Our take
Measuring whether your marketing technology works is not bureaucracy, and it is not a lack of faith in the tools. It is simply how you tell the difference between an investment and an expense. Name the outcome, take a baseline, count the true cost, and review it honestly when you said you would.
Do that, and you join the minority who can actually point to what their technology is doing for them. If you would like help working out whether your marketing is earning its keep, a marketing audit is often the clearest place to start, and we would be glad to talk it through.
