Welcome to Nicolas Lalaguna's online archive

I have recently been doing quite a lot of reading around the implementation of AI and the potential pitfalls. The reason for this is two-fold. In my private life I have a friend who whenever we speak invariably gives me an unrehearsed TedTalk on their work with the various Generative AIs (Gen AI)s and why this is the most important thing to ever happen to humanity. Secondly, this has come at a time when I have started to see how large numbers of people in the charity sector are increasingly using Gen AIs, in some cases relying on them to perform complete tasks. I worry that the fear of missing out on Gen AI is putting major charities at risk. 

I bring this up because when I recently applied by email for a role that I was highly qualified for, the employer replied with a “thanks but not thanks” type of email in just under 4 minutes. This seemed awfully quick to read a 2 page C.V. and a 1,200 word covering letter, compare it with the other applications, find enough fault to immediately knock it back and get the email sent out to me. I checked in with a couple of old friends in the sector to ask them what they thought. Without even letting me finish the story they both said that whichever charity it was it would definitely be using AI to shortlist candidates, something that they both told me is increasingly common.

During this last year of acclimatising myself back into the sector the thing that has really jumped out at me is the way that many people, at all levels in the charity sector, are outsourcing aspects of or in some cases entire tasks to Gen AIs like ChatGPT and Microsoft Copilot. I have personally witnessed junior, entry level and management level charity staff using these for writing internal and external communications, strategy documents and reports. And again when I discussed this with friends not one of them was surprised. Some where even using it themselves, all be it as a drafting tool.

There are several reasons why this gives me cause for concern. But I don’t want to get ahead of myself as this is a fairly complicated subject. Firstly there are two key versions of the technology at work here, Generative AI and Reasoning AI.

Generative AI is a software that learns from massive sets of data and then uses that as the framework for allegedly creating something new and unique that feels to a human that it has been created by another human. Whether that is music, artwork or the written word, which in turn could take the form of film scripts, strategy documents, marketing briefs or even press releases. Quite how human the output appears to be is definitely open to debate, but too subjective for the purpose of this article.

And of course there is the other inconvenient complication that almost immediately that Gen AI became available to the general public, it was tasked on a massive scale to develop and disseminate misinformation and disinformation. Demonstrated most clearly in the increasing online trend of deepfake content. Another can of worms I can’t open up now.

Reasoning AI on the other hand is a software which is attempting to mimic how certain people believe the human brain reasons. The process that the AI industry has decided on is that the software analyses data, interprets that data and then, based on predefined criteria, attempts to logically infer a conclusion. For instance this might include using tens of thousands of MRI scans to infer early warning signs for cancer.

While this sounds great, it doesn’t really take into consideration the pre-existing and external factors that are the context for that patient. Take the example of early cancer diagnosis. There are many factors that are impacting early diagnoses over and above not having enough pre-symptomatic early warning signs. Even if we had early warning signs recognisable on an MRI, these would only help if everyone was receiving regular MRI Scans. With the NHS on the cross-party fast track to full privatisation the likelihood of regular MRI scans being on the cards for the poorest 95% is highly unlikely. So while it is a strong argument, it doesn’t take into consideration the environmental reality in which it exists.

So without getting into some of the obvious pitfalls facing blind adoption of the the two AI technologies while ignoring the wider and deeper socio-economic contexts, there are some fundamental questions that need addressing.

However first it is probably worth remembering that we as a species have not definitively reached a scientific consensus on what is human consciousness and how it does what it does. While most people working in disciplines related to this field have a preferred theory, the reality is that most of the specialists at the apex of such related fields are willing to embrace a more transactional interdisciplinary approach than they may have done 5 decades ago.

The reason I bring this up now is because the name itself, “Artificial Intelligence”, has been marketed into this technology with little or no discussion of the fact that we don’t have a universally agreed understanding of what human intelligence is. Surely logic dictates this would be a a precursor to creating an artificial simulacrum of it. Although it is probably best not to ask the AI luminaries to wade in on this. I can’t imagine that what the increasingly far-right leaning tech billionaires think human intelligence is, is likely to sit very well with the rest of us.

All that being said, one of the fundamental questions for me, whenever a technology company or group of companies decide to move fast and break things, is whether there is oversight or safeguards in place? For instance in light of the phenomenal adoption across the world of AIs by companies and organisation, has anyone asked the question how accurate is it? Before we put our lives and livelihoods into its hands it would be nice to know if it makes mistakes? After all, privately owned and controlled AIs are not only being given access to our entire online and digital lives, but we are increasingly being forced to engage with the world through them.

Let’s run a thought experiment. Say that AI’s are making fairly major mistakes. And organisations in pursuit of return on investment are looking to decrease costs (“sacking people”) by allowing cheaper and more inexperienced staff to use these tools to perform the day-to-day tasks that would have previously been delivered by more experienced employees. Would it really be a surprise if those same organisations started to make major mistakes? To be entirely honest, that’s not a thought experiment. They are already making huge mistakes, the kind of huge mistakes that threaten the bottom line.

The fact is that there is an increasing body of evidence showing that both Generative and Reasoning AIs are making large mistakes at a fairly alarming rate. An article in Techradar in May 2025 argued that the two newest Reasoning models from OpenAI, called GPT o3 and o4-mini, were hallucinating at rates of 51% and 79% respectively on general knowledge questions. Similarly OpenAI’s own report warned of hallucination rates of 33% and 48% respectively when answering questions on public figures.

Incidentally, when techies refer to hallucinations in the AI context they are talking about the AI presenting incorrect or misleading information as factual.

In a study earlier in the year of the eight leading Gen AI search engines they were shown to not only be inaccurate on average 60% of the time, but often arguing that the incorrect response that they had given were actually correct, on a few occasions going as far as putting forward false information to support the incorrect response when confronted over it. They are getting it wrong, lying about getting it wrong, and when that doesn’t work making stuff up to support their lies. Sound like anyone you know?

I can give you two anecdotal examples that I have personally experienced of these kind of things happening. In order to get a better understanding of how Gen AIs worked I asked ChatGPT to write a 500 word summary about me as a writer, drawing from my own published works, responses in the comments sections of said work, my own online repository of work and any online reviews it could find. In it’s answer it missed one of my only two published novels, which has been reviewed, and attributed a novel to me that I not only didn’t write, but I can’t find any reference on the internet to anyone having ever written it. I could see the mistake, but very few other people would be able to.

A couple of weeks later I was talking to a retired University Professor about how many academics are increasingly worried about students using Gen AI to do their work for them. And to show them how quick it was to perform such a task I asked Chat GPT, “in 1,500 words can you write an undergraduate level essay on the socio-political history of the relationship between Israel and Iran that led to the Israeli airforce bombing Iran last week”. In Chat GPT’s response the word Iran was never mentioned, instead choosing to produce a 1,500 word essay on the history of Israel and Palestine. I pointed out that I had asked for the essay to be about Israel and Iran, to which it responded I must have meant Palestine because that is the defining geopolitical relationship in that region.

I worry that a large number of organisations are using this clearly untested and barely understood tool to replace people in order to increase net income; people who from personal experience are most of the time happy to be accountable for their actions, care about the individuals they work with and the organisations they work for, and are willing to do their best and get better at their jobs where they can.

The reason I am so troubled by this technology “running fast” is because I am personally seeing who and what things are likely to get broken in my own day-to-day life, and the lives of people I love and care for.

There has been a recent case in the US where $365k in damages has been agreed to be paid out to unsuccessful job applicants who weren’t shortlisted by the AI because of their age. This isn’t an anomaly either. There is yet another class action case currently pending against an HR software company that have developed an AI driven software that screens out people based on skin colour, disability and age.

The reality is that organisations with very little understanding of how this technology works or the potential pitfalls of blindly adopting it are doing exactly that. One can only imagine that this is quite likely to be based on a fear of being left behind. One such charity that I feel very strongly about as I worked at the parent charity for many years and have personally come to experience the importance of their work seems to be desperate to get to the front of the crowd when the Emperor rides past in his all together. In a role they are currently advertising for they proudly claim in the “How to Apply” section of the application “We love innovation (yes, we use AI too!)”.

I can only hope that they are not using an AI to screen the applications, and if they are it isn’t one that has inexplicably decided to start screening applicants based on the prejudices baked into the datasets that they are built on.

I understand that clambering through piles of applications is time consuming. At one of the larger charities I worked for, one of the fundraising teams received over 600 applications for an entry level role. The Head of Fundraising asked all the Senior Managers to pitch in on shortlisting the applications. While it seemed arduous at the time, the candidate that got the role stayed at the charity for over 10 years, and until very recently was still reporting to the same manager, with both of them having been promoted up the organisation.

I was one of the managers that begrudgingly at the time, had to find 3 hours in my own very busy schedule to read through a pile of those applications for another manager’s team, but the Head of the department was right. The person that got the job was and still is an exemplary employee of that charity. The money the charity spent on getting the right candidates on that short list has paid the charity back countless times over.

I know that many of the roles that I have been applying for have been receiving anywhere between 100 to 200 applications for each role. If each application is averaging around 1,500 words, then 150 applications would amount to some 225,000 words. That is the equivalent of reading Bronte’s Wuthering Heights and Pullman’s The Golden Compass.

I get that it is a lot of work, but handing the job over to a sorting machine that you don’t really know how it works or if it even does work can’t be a good idea. In this last year I have seen just over 10% of the vacancies that I have applied for get re-advertised within 2 months after the application date passes. That really isn’t an effective use of a charity’s budget.

Fundraising, once you get past the legal, regulatory, financial and systems specific knowledge required of it, is at its very core a human to human communications specialism. Fundraisers know how to communicate, they know how to tell stories, they know how to engender sympathy and empathy in others, and while the medium by which those messages are conveyed may change, it takes people to raise funds from other people.

These aren’t things that can be cleverly systematised. We don’t really know why some people are good at communicating, or why some people are good at maths, or why some people are good at managing other people. What we do know for certain is that while we can’t really explain the how and why of it, we recognise it when we see it. Gen AI’s are now threatening the good will and trust of donors, while risking letting down the beneficiaries that desperately need these charities to deliver. All because of the fear of missing out on Gen AI?