How we're using AI (and why that's likely to change)

The way we build software, and what that software does, has changed more over the past year than in any other in our 22-year history. And it feels like we're only at the start of this change.

AI is behind much of the shift. It brings a knotty mix of huge opportunities and tricky questions, both for us and for our clients. So we want to share how we've been using AI so far, what's gone well and what’s gone awry, and how we're thinking about what comes next. 

Rather than being a set position, this is a snapshot of where we are now. And how, as a team, we're working through this gnarly challenge.

Where we started

We started using Claude, an AI assistant made by Anthropic, around 18 months ago. A few of us had experimented with ChatGPT and OpenAI before that. Today, everyone at Supercool uses AI in some way, to varying degrees.

That doesn't mean we all feel the same about it. Some of the team are excited by what it makes possible. Others have been firmly against it, with legitimate concerns about its impact on people – and the planet. Several have moved positions along the way, in both directions.

We see that range of views as a strength. It means that we pause to ask questions, rather than mindlessly following the hype.

What hasn't gone well

Our adoption of AI certainly hasn't all been smooth sailing. We've had false starts and wasted some time, and we've encountered a few results we'd rather forget. Some challenges are still very much live:

• Trust: Knowing when an AI answer is actually correct, rather than just 'sounding' correct. (Because AI generally sounds very confident!)
• Over-reliance: Making sure people keep their own skills and judgement sharp.
• Low-quality output: The filler and "slop" that can creep into AI responses and documentation.
• Pace: Pressure and fatigue, with AI pushing humans to work ever faster.

Being open about these downsides matters because it's how we, and our clients, make better decisions about where AI genuinely helps – and where it doesn't.

The sustainability question

Sustainability is central to how we work, so most of the team have asked how AI fits within that. And there's no easy answer.

On the one hand, AI has helped us to make certain areas of our code 'leaner' – the less code that needs to be processed, the less energy is needed to run that process. Sustainability win! 

On the other hand, those gains rely on huge data centres, and the energy-hungry training of new AI models. 🙁

The lack of transparency from AI companies can make their impact on the environment hard to judge too. Anthropic, the company behind Claude, haven't published a sustainability report or emissions data in the way Google and Microsoft do. (We have asked them about this but not had a meaningful response.)

Questions and concerns about sustainability are something we take very seriously. But we don't think the best response to those questions and concerns is to turn away. AI is reshaping our industry, and we want to be part of shaping how it's used – and of working out how it can be used responsibly and ethically.

So, we're balancing two things: optimism about what's possible, with a healthy curiosity and caution about the risks.

How we use AI in technical work

We use AI as a tool to help improve our efficiency when building new features, improve our existing code, and to review and test what we've built.

Some of our biggest wins have come from our own developer tools (i.e. for diagnosis, verifying  and testing). Content migrations can now take hours rather than days. Optimising code has improved our accessibility scores and our code's sustainability. We've also built standalone products that rely more heavily on AI-written code.

Throughout all of this, however, people stay in charge. AI can help us increase efficiency and speed up certain areas of work, but responsibility for quality stays with us humans.

How we use AI to help us run projects

On the project side, we use AI for research and analysis. It also helps us to complete certain rote and/or time-consuming tasks (such as reviewing documentation) much more quickly. This helps us to cut admin time, and keep projects super-efficient.

We're looking at building automated reporting across all the websites we host, to help keep them fast and compliant. And we've just started exploring how AI-powered tools inside Craft CMS could make our clients' day-to-day work easier.

Our team checks what AI produces, applies their own experience and judgement, and stays fully accountable for the result.

We've built a strong reputation for exceptional customer service – and have published client ratings to back that up. Our team's care and judgement are what build long-lasting relationships with our clients. That's not something you can automate.

The rules we work by (for now)

We have policies in place to protect data (ensuring sensitive data is kept out of LLMs) and to keep the code we put live to a high standard.

We've set up 'AI Labs' – a Slack channel where anyone at Supercool can share their AI experiments: the wins, the near-misses, and the glorious flops. Nothing is too small or too awkward to post. Learning from what didn't work is just as valuable as celebrating what did. 

And it helps us to keep an honest, open conversation going across the whole team.

This year, we started an internal AI Working Group. It gives us dedicated time to discuss, review, and steer how we use AI across the whole agency. Our guardrails, ways of working and skills are still evolving, and they'll keep changing as the technology does.

In essence, we use AI to reduce time spent on rote tasks, so we have more time to do the work that adds real value to clients – using good old-fashioned human brainpower and creative thinking.

What's next?

In the New Year, we'll review which AI coding tools and models we use – with environmental impact a key factor in that decision.

We'll continue to share what we learn openly, including what works and what doesn't – similar to what we do with our Sustainability Working Group. (Think of it as a public version of our AI Labs channel; that same honesty we encourage inside the team, shared ad hoc with our clients and the wider cultural sector.)

AI moves faster than any technology we've worked with. New models, tools, and evidence about their impact appear every few months. So what we've described here is our current position, not a static one. Some of today's practices will change, and we'll adapt them as we learn.

We don't claim to have all the answers. And what we've shared here is likely to become out of date – possibly even quicker than we're expecting! 

What we can say for sure is that we'll keep making mindful, considerate choices about AI – and helping our clients to do the same.