Could AI Do What Our Moderators Do? We Tested It To Find Out.
Over the last year, AI-generated content has become impossible to ignore.
It can write emails, social media posts and customer service responses in seconds. For many organisations, that sounds like an opportunity to save time and resources.
But when it comes to charity moderation, we believe something important gets lost.
At Social AF, our moderators respond to thousands of supporters every day. People share their fundraising achievements, but they also share grief, illness, diagnoses, financial worries, loneliness and some of the most difficult moments of their lives.
Those conversations deserve more than a computer generated response.
They deserve a truly human one, that has taken the time and thought before replying.
To demonstrate why, we recently ran an experiment comparing AI-generated responses with responses written by our own experienced moderators.
Two examples stood out in particular.
Example 1: Supporting Someone Through Grief
Participant post
“I’ve had a really emotional day today. I think it’s finally hitting me after losing <name> in January. I contacted our counselling service through work and the first support for miscarriages they mentioned was Tommy’s. As hard as today has been, I’m so proud to be taking part in this challenge.”
AI-generated response
“What you’ve shared is so raw and brave. Grief has a way of arriving in waves, especially after something as heartbreaking as losing <name>, and it makes perfect sense that today felt heavy. Reaching out for support is such a powerful act of self-care.
Despite everything you’re navigating, it’s wonderful that you’re taking part in this challenge and finding the strength to keep moving forward. Be kind to yourself, take things one day at a time and know that every step you take is an achievement.”
Moderator response
Thank you for trusting us with this and for sharing about Bean. I’m so deeply sorry for your loss in January. It makes so much sense that some days the weight of it all feels heavier, as grief can come in and feel more intense. Today sounds like one of those days. Please be gentle with yourself and take things moment by moment
I’m really glad you reached out for counselling. Seeking support is a brave, loving step, and I hope that connection with Tommy’s continues to help you feel steady and safe as you move through this. I’m just going to pop in our links and numbers for you again so that they are there for you
To speak to someone, you can contact the team at midwife@tommys.org. You can also call them for free on 0800 014 7800 (Monday to Friday, 9am to 5pm) or our link with the relevant support and info on https://www.tommys.org/baby-loss-support
We are so very proud of you as you wear your t-shirt. We’re right here with you – holding space for you and supporting you as you take the next small step. You don’t have to do any of this alone
Thank you for being with us and we are always with you for every step of the way
What stood out?
The AI response sounds thoughtful on first glace, bu when you read it, it it feels performative. Phrases like “raw and brave” and “powerful act of self-care” are common AI language patterns and can feel detached from the reality of what someone has shared.
The moderator’s response feels more natural. They acknowledges the participant’s loss, references specific details they shared, signposts support appropriately and end by recognising their achievement within the challenge.
Most importantly, it sounds like a real person.
Example 2: Celebrating a Challenge Finisher
Participant post
“My 10th run completed. 53.3 miles done and nearly £300 raised, which is far more than I anticipated.”
AI-generated response
“53.3 miles across 10 runs is not only an incredible physical achievement, but also a powerful reflection of dedication, resilience, consistency and the strength of the human spirit.
Raising nearly £300 is equally impressive, especially when it’s exceeded your original expectations. Every mile you’ve completed and every pound you’ve raised will help make a meaningful difference to people who rely on the charity’s support and services.
You should feel incredibly proud of everything you’ve accomplished. Thank you”
Moderator response
Well, well, well…what do we have here? We have a FINISHER!
Let’s start with the most important thing, saying a HUGE well done for completing the challenge. How are your legs feeling after moving that many miles in one month?
Alongside our incredible scientific research, empowering patients and people with prostate cancer is at the fore of what we do, spearheaded by our incredible patients projects team who connect patients across the country together and with our work. Patients like Ray, who shared his treatment journey and choices with us – “Overall, at the time with PCR’s help, I believe I made the right decision for my partner my family and myself.”
Your fundraising has been truly epic, almost £300 is phenomenal, thank you so much, but now you have completed the challenge, if you could share your fundraising page one last time we would be forever grateful so that we can continue to support people like Ray
Thank you so much again! Have a great day and remember…
YOU
ARE
AMAZING
What stood out?
The AI response sounds impressive on the surface but quickly becomes generic. It wasn’t able to know that this person had finished their challenge, or to check things like their fundraising totals
“A powerful reflection of dedication, resilience and the strength of the human spirit” could be attached to almost any achievement.
The moderator response does something different. Celebrating the participant’s success, connecting their achievement back to the cause they’re supporting and asking a question to continue the conversation. More importantly, it’s hugely personal, not generic.
The Patterns We Keep Seeing
When we review AI-generated moderation responses, the same patterns appear time and time again:
They sound polished
Often they are far too polished. People don’t naturally speak the way AI writes.
They use repetitive language
Terms such as these are over-used:
- valid
- journey
- healing
- self-care
- community
- strength
- resilience
They can feel emotionally hollow
The response appears empathetic, but doesn’t feel genuinely connected to the individual.
They lack context
AI doesn’t understand the charity, challenge, systems, processes or safeguarding considerations involved in moderation.
Why This Matters
Charity supporters trust that there is a real person behind the screen.
When someone shares a story about losing a baby, living with cancer, struggling financially or feeling isolated, they aren’t looking for a beautifully constructed paragraph.
They’re looking for somebody who has listened. They’re looking for authenticity and ultimately, connection.
At Social AF, that’s exactly what our charity partners trust us to provide.
Our moderators aren’t paid to copy and paste responses. They’re paid for their judgement, empathy, experience and ability to build genuine relationships with supporters.
That’s one of the reasons we’re trusted by the UK’s leading charities.
AI is an incredibly powerful tool, and it undoubtedly has a place in helping organisations work more efficiently. But moderation isn’t simply about generating a response. It’s about understanding the person behind the message.
When somebody shares a fundraising milestone, a cancer diagnosis, the loss of a loved one or a moment of self-doubt, they’re not looking for perfectly structured words. They’re looking for connection – someone who understands the context, recognises what matters in that moment and responds in a way that feels authentic.
AI can imitate empathy, but human moderators create genuine human connection.