Open Source Models Feel Dumb, But Not As Dumb As Me

August 31, 2026   I’m going to sound dumber than usual today. I tried to download an open source model to try it out and…I literally can’t figure it out.

I went to Google and asked for open source models. This stupid AI summary response gave me instructions. Note the callout that this is the “Easiest Method”...save that little tidbit for later…

I went to the Ollama website. I guess I was expecting a more dark web kinda feel, but it’s the usual marketing website stuff. Two download buttons to make it easy for guys like me - we’re off to a good start!

(Quick sidequest…on the right hand side, that’s a video that plays automatically. Zoom in a bit and I saw an interaction that feels very similar to a lot of AI demos vendors have been showing me. “Hey, we’re model agnostic! Look how cool this is! Your team can seamlessly switch to the most efficient model after we build this for you.” If it’s so hard to do, how come a generic LLM website shows it on the landing page? This only reinforces my queasy feeling that these vendors are smoke and mirror parlor tricks laid on top of Claude/ChatGPT models…but I digress)

Being the tech forward guru I am, I clicked “download” and waited for the magic to happen!

A few seconds later, this is what I got - the cute little guy in the top right hand corner of my screen. Nothing else. Nothing launched, no prompts to do stuff. I tried clicking on him to no avail.

So I ran to…ChatGPT.

Ignoring the irony that I am asking ChatGPT how to download an open source model to replace it, I opened the terminal and copied and pasted that exact prompt. We are finally off to the rac…

Nope, my IT security blocked it. This is becoming quite the chore.

Who needs IT anyway…

Turns out, it didn’t matter anyway. Nothing happened. Remember, this is the easiest method.

Let’s stop here before it gets any more embarrassing. At this point, and probably sooner, I eject and make someone on my team do this instead. You will too. Comparative advantage and all.

But how much am I going to save anyway? The AI models are beating their own pricing down as they race towards IPOs. Here’s an image from the LoanBoss billing console inside of Claude. Ben and Tom are the top two users of Claude each month.

• Ben runs the LoanOps team, so all loan abstracts flow through his account

The “unlimited” in this image doesn’t mean unlimited usage like a cell phone might have. This means there is no dollar cap when they go over their monthly allotment.

For both Ben and Tom, the dollar figure you see is their monthly overage. Dmitry and Eugune both show $0, but that doesn’t mean they haven’t used Claude, it means they haven’t gone over the monthly allotment. Here’s where

Here’s where I messed up, but it actually provides some valuable insight into usage. I should have made Tom a Premium user, but I hadn’t (Ben already was). So one of my power users was exceeding his monthly allotment much faster than he should have been. Remember from last week Claude’s pricing model:

• Standard seats: $25/mo
• Premium seats: $125/mo
  
  

But this makes for a relatively straightforward cost analysis. I pay $100 extra per month for Ben to be a Premium user, but his overage is about $1,000 cheaper than Tom’s. Assuming their usage is the same, I spent $100 to save $1,000, so I net $900 to the good. 

Claude doesn’t tell me how much usage each seat gets, so I can’t quantify it. But I can ballpark reverse engineer and see that an extra $100 provides quite a bit of usage. Abstracting with 99% accuracy hundreds of loan docs a month and automating thousands of emails and doc handoffs and customer support workflows is basically costing me: 

$125/mo + $1,000 overage = $1,250 per month

These two teammates then cost me $2,500 per month in a Claude bill to run their workflows, but save me over $35,000 per month in personnel expenses. I don’t want to diminish the personal side of that equation, but that is the cold hard math.

Which brings me back to the open source part of the equation…is open source really worth it?

You can’t just click a button and start doing magical things like you can with Claude and ChatGPT because the open source models are just…models.

I want a car, but open source models are just the car engine. I complain, “this can’t drive me to work!” Ford jumps in and says, “Yeah dummy, that’s what we do. We provide the car, not just the parts.” Same thing with the LLM models. They aren’t just providing a single part, but the entire product. 

I will keep investigating open source options, but in the interim I don’t feel like I am in a race against spiraling AI costs. My worst month jumped to $16k and we investigated and dialed it way back without losing functionality (“stop uploading hundreds of images!”).

Claude must be on to me, because I just got this offer from them. “Switch to usage based pricing and we’ll give you $225,000 in free credits over the next year”. In other words, Ben and Tom don’t have some baked in usage for $125 per month.

Since I don’t know how much usage they are getting for that Premium seat, I can’t predict what a straight usage bill would look like. I’d be trading out a flat $125 + overage for some unknown.

But once someone starts offering me $225k for free, I start thinking there’s something in it for them. Maybe my bill is 10x a year from now when the $225k credits are gone…

For now, we keep plugging away with Claude and investigating open source options. My best guess is we might eventually incorporate open source for things that don’t require every latest model update (how much better do I need 99% accuracy on loan abstracts to be?), but I am not losing sleep over the monthly Claude bills right now.

Our energy is better spent on optimizing operations and rolling out cool stuff for our clients, not trying to squeeze $125/month down to $0.

Do I want to tinker with the engine on weekends? Or drive laps around Monaco?