“Open-source models must get smaller over time…For everything else, the major LLM providers will dominate.”
I’ve been racking my brain lately trying to figure out where this leads. Open-source can find niches, I think - but man oh man the major LLMs are just so smart (for whatever definition you choose for that word). And smartness kicks ass.
I use both OpenAI (as a paying subscriber) and I also host multiple open source LLMs locally.
The privacy aspect of the local models is definitely nice, but if you use OpenAI also, you’re unavoidably blown away at the sheer speed of the responses. It’s hard to beat.
This assumes that OpenAI's pricing is sustainable, and not a land/developer grab. OpenAI wants you to abandon doing it yourself by subsidizing infrastructure (for now). That will end. Not to say that AWS is the cheapest way to do this - but when the industry is committed to losing billions of dollars to stake out turf, this is what you'll see. You definitely can't extrapolate this over time.
Because OpenAI's cost of running the actual infrastructure isn't going to be that much different than AWS'.
I also think if you upped the parameters significantly, it's a less rosy picture.
That, or the $11.3 billion in funding through 7 funding rounds from 14 investors they've got lets them light a nearly limitless amount of money on fire in the hopes that in a few years all these other companies offering GaaS (GPT-as-a-Service) will be in too deep to stop paying when they jack up the price.
Smaller specialized LLMs running locally may be the future both for companies and individuals who prefer to keep IP and other information a secret. This message written by a human.
Also, pre-empting the haters. OpenAI is positive margin. It's expensive to serve inference, but you can still make money.
"per-token inference costs for fine-tuned GPT-3.5 is 10x more expensive than GPT-3.5 it is still 10x cheaper than GPT-4!"
this is off.
fintuned-GPT3.5 is 3x cheaper than GPT4 -- $0.0120 / 1K tokens vs $0.03 / 1K tokens
“Open-source models must get smaller over time…For everything else, the major LLM providers will dominate.”
I’ve been racking my brain lately trying to figure out where this leads. Open-source can find niches, I think - but man oh man the major LLMs are just so smart (for whatever definition you choose for that word). And smartness kicks ass.
I use both OpenAI (as a paying subscriber) and I also host multiple open source LLMs locally.
The privacy aspect of the local models is definitely nice, but if you use OpenAI also, you’re unavoidably blown away at the sheer speed of the responses. It’s hard to beat.
This assumes that OpenAI's pricing is sustainable, and not a land/developer grab. OpenAI wants you to abandon doing it yourself by subsidizing infrastructure (for now). That will end. Not to say that AWS is the cheapest way to do this - but when the industry is committed to losing billions of dollars to stake out turf, this is what you'll see. You definitely can't extrapolate this over time.
Because OpenAI's cost of running the actual infrastructure isn't going to be that much different than AWS'.
I also think if you upped the parameters significantly, it's a less rosy picture.
That, or the $11.3 billion in funding through 7 funding rounds from 14 investors they've got lets them light a nearly limitless amount of money on fire in the hopes that in a few years all these other companies offering GaaS (GPT-as-a-Service) will be in too deep to stop paying when they jack up the price.
Smaller specialized LLMs running locally may be the future both for companies and individuals who prefer to keep IP and other information a secret. This message written by a human.
OpenAI squeezing the middle from both sides. The best capabilities and the cheapest to try your first experiment with.
Keep the back of the envelop math for LLMs coming!