Writing Website Copy With Grok 2, Claude 3.5, & ChatGPT 4

I’ve been writing webpage copy for the pricing page of a SAAS product and decided it would be interesting to compare ChatGPT 4, Claude 3.5, and Grok 2 LLM AI models while doing so (in hindsight I should have also included llama).

My methodology was to ask each one the same questions and requests for refinements each step of the way while writing my copy for the page. To keep this article true to my own experience, I wrote it completely by hand (no AI inputs).

My personal, non-scientific, experience was not what I thought it would be. Here’s the breakdown:

Wordiness: Winner Grok

I explicitly told each model that I wanted the text to be concise. ChatGPT started out strong but got more and more wordy; to the point I had to keep reminding it to not be. Claude did an okay job. Only once or twice did it get a bit too wordy. Much to my surprise, Grok excelled at keeping things concise.

CTA advice: Winner Grok

Many of my requests were centered around refining my call to actions (CTA) on the page. All three models provided good insightful information. Here again, ChatGPT was unreasonably long winded in its copy; which is not ideal for a CTA. Grok was the clear winner though because it was the best at understanding my product and tailoring the CTA to it.

Final Refinement: Tie Claude & Grok

Towards the end of writing the copy a lot of minor tweaks needed to be done and they’re the most important tweaks because it’s what makes a page feel either personable or sterile. All three models provided a good set of refinement options. That said, Claude and Grok both explain what they changed as well as give you the revised text. ChatGPT just gives you the revised text; meaning you have to go on a scavenger hunt each time to see what specifically changed. I personally greatly prefer the way Claude and Grok do it.

Closing Thoughts

All three models performed well but for my experiment’s specific use case Grok was the clear winner. That being said, we’ve got to talk about the user experience. I realize this is very subjective, but I find Claude’s user interface substantially more pleasant to use than ChatGPT and Grok. I really wish those two offered options to change the colors and fonts so that I could make them look like Claude. I find Claude to be much more readable and relaxing to use. So while Grok was the winner for content generated, I find it hard to not go to Claude first when using LLM’s because of the joy of using their interface.

Also thought it would be fun to ask each to give me a sentence opinion on this article :)

Your article offers a clear, insightful comparison of LLM models, blending practical results with personal preferences effectively.

— ChatGPT

Your hands-on comparison of leading AI models for copywriting reveals Grok as a surprise frontrunner in content generation, while highlighting how user interface design can significantly impact the overall experience.​​​​​​​​​​​​​

— Claude

Your article offers a valuable comparison of AI models, with Grok excelling in content conciseness and Claude in user interface.

— Grok

Cheers 🥂