Big Data LDN 2025: Where Does All the Data Go?
When autumn is coming to London, so is Big Data LDN, the annual London expo for big data, analytics, and AI. As I did in 2022 and again in 2023, I want to share some observations about what has changed and what the vibe in the industry is.
This is not a product review, but more of a social visit. As a slightly cynical IT veteran, I am well aware that I am not the target audience for most of the companies, though I have in the past found useful services there. What fascinates me is to find out what the marketing buzz of the season is.
On the bright side, the Olympia in Hammersmith is always a nice place to visit, so much better than the dark industrial Excel London in the east, which feels more like meeting in the parking garage of a shopping mall.
Observations
My first impression this year was that a certain type of startup has now all but disappeared from the expo: the small, ambitious startups that create technology from scratch (like Memgraph, whom I met three years ago), want to challenge big tech services, and connect to SMEs. They have been almost exclusively replaced by better-funded startups that mainly build on existing products from Snowflake, Microsoft, or OpenAI.
While AI was still exotic in 2022, “AI-powered” is now everywhere, even if sometimes this appears a bit tagged on. This year, “agentic” had obviously joined most offerings, justified or not. “In the scramble for agentic systems,” read the very first keynote, “the question has to be asked, are we ready?” I am not sure we are. At least the event organizers stayed away from offering their own chatbot app like last year (which was pretty bad and dysfunctional in my opinion).
Everyone has a house at the data lake now
The “lakehouse,” which combines the much criticized data lake and the data warehouse, is now omnipresent. Combined with the “AI-native platform” that tries to distinguish itself from AI as an add-on, though frankly, in some cases, it is.
But overall the services offered by various companies appear quite homogeneous and struggle to distinguish themselves from the competition, which is odd for an industry that likes to praise itself for its innovations.
“What,” I asked at one stand, “does your service do in detail?” “Well,” was the answer, “you put all your data in, and we generate a dashboard.” That is vague but pretty much sums up how 90% of the services I saw operate:Mostly unstructured data is fed into the pipeline, something happens on the service side, and graphs and charts come out.
Where are the quality assurance services (to validate the outcome, not to clean the data)? Where are the test suites? The Lakehouse might work like magic, but at the cost of transparency and explainability. The absence of open-source applications was also notable (with the exception of Neo4j) in this context.
There is a lot missing though: What about the fine-grained triggers and thresholds we used to work with when dealing with data streams? A lot of the precision appears to have been lost. Many at the expo will, of course, argue that this is the inevitable consequence of dealing with vast amounts of data, and the probabilistic approach of LLMs is the only way.
Also, when I say data, we mostly talk marketing, customer, and business data, or at least that is the impression from demos and presentations. I am aware that this is a selling point made to middle and upper management, but it often feels limited.
All this raises the question of how customers pick the service, and, if everything just looks and feels the same, how many of those companies will survive. More likely the future market will coalesce around the big guys: Amazon, Microsoft, Google, and Oracle.
Educational and an anachronism, but a welcome one
If the IT landscape is really changing as rapidly as some claim, then AI literacy and competence will play a critical role. It was good to see more companies working in this space, such as the Data Literacy Academy or DataCamp.
Talking about education, I was happy to see The Royal Statistical Society (RSS) here, which was founded in 1834 as the Statistical Society of London in “Charles Babbage’s house.” I wish other organizations like the Ada Lovelace Institute or other research institutes would attend expos like this.
Worried, us?
Some companies appear to be aware that adoption and reception of AI in the workplace have not been optimal. AI, considered by the industry as the holy grail of productivity, is found to instead sabotage productivity with “workslop” in many cases, a study suggests (Harvard Business Review, 2025).
According to an MIT report, 95% of generative AI pilots at companies are failing. While the report partly blames the companies for the failures, it is extremely bad news for the AI industry that has made a huge bet on IT businesses adopting its tools quickly and paying good money for it. It also remains a question how often management goes ahead after a pilot anyway in the fear of missing out and due to investor pressure to adopt AI at any cost and risk.
How mature are many of the services really? The problem is that boasting that an AI-powered service “works” is maybe not the selling point some think it is. If I am honest, it is the minimum I would expect from an (expensive) enterprise service. Imagine your appliance shop advertising a “washing machine that … actually washes.”
Women in Data, and Data for good
The highlight for me was to see that for the first time at the Big Data LDN Data and AI ethics played a more prominent role, maybe indicating that the importance of the field is starting to be acknowledged.
A series of lectures was dedicated to bias, gender, and discrimination, which was a first for the expo. Women in Data, who presented the results of their study “The Data Delta: The Hidden Reality of Women’s Safety,” also had a large and prominent presence in the main hall.
Special thanks..
.. to Alejandra from Opentext for the chat, and to Elle from Simpson Associates for the chat and the fizzy candy. And a shout out to Multiverse for the beautiful insulated water bottle. Appreciated.
If you are interested in ethical questions in AI and data, follow misaligned or subscribe to the misaligned bits newsletter.
Women in Data Website
The Data Delta: The Hidden Reality of Women’s Safety [PDF]
AI-Generated “Workslop” Is Destroying Productivity. Kate Niederhoffer, Gabriella Rosen Kellerman, Angela Lee, Alex Liebscher, Kristina Rapuano and Jeffrey T. Hancock. Harvard Business Review. 2025
MIT report: 95% of generative AI pilots at companies are failing, Fortune, 2025
Royal Statistical Society, History
All Images by Wolfgang Hauptfleisch, 2025
