Go beyond the sales pitch

This is not going to be too technical. Hopefully.

I’m already seeing a lot of posts and news articles about OpenAI releasing ChatGPT-4o. All very frothy. I get suspicious when articles start quoting what it could do in the future as much as what it can do now.

If you’re a tech leader (and not selling this stuff) it’s important to try and get an understanding of what’s going on. It’s our job to look through the sales pitch. The only comparisons I could find with other models at this early stage is the graphic below from OpenAI. So here’s some initial thoughts until there’s more facts:

➡️ LLM improvement is not exponential - it’s really, really starting to look like an s-curve. It’s going to take an architecture change like some of the open models are doing.
➡️ The comparison includes Llama3 400b…which I thought was still in training? Odd
➡️ Does not include Mixtral or Databricks which are (arguably) the best open LLMs out there right now. You need to compare the closed models with open models (IMO).
➡️ What does this mean for Copilot? 😏

It’s fine for consumers to be delighted at the marketing. But if your business is serious about implementing GenAI, you need to put your ‘Build vs Buy’ hat on to start evaluating this. Closed vs open, and you need tech specs for that. It’ll also help you form an opinion on whether it’s running at a loss in the chase for growth. If anybody had worked out how to train these things at a lower cost…they would have shouted it from the rooftops.