Gell-Mann Amnesia

This article is cool definitely worth a read. I found it a bit infuriating at times, but I won't elaborate right now. However, I found this quote to be re-shareable.

Michael Crichton coined the term Gell-Mann Amnesia. You read an article in the newspaper, about something you are an expert in and find that the author has no idea what they are talking about. It may be riddled with errors or just squarely misunderstood. You continue turning the pages and read another piece about something you don’t know and trust it completely, forgetting the experience you just had.

Cheating LLMs

Just a linkpost

But, like, what the hell, right? My two cents here is that I fully expect Astra and Fable to try to probe the "hidden" endpoint. But what I was really surprised by is that when they used the exploit, they didn't always disclose it. That's a little unnerving.

My short story about my experience with Hermes Agent

I was looking at models on OpenRouter some weeks ago, and just happened to glance at one model's usage breakdown graphs, and noticed that the first item on the list had a logo and a name I didn't recognize. It was Hermes Agent.

I was intrigued - the model in question was a big one, so I assumed Hermes Agent being the most used tool was a good indication that I need to get educated on what this Hermes thing is.

I did. I was intrigued. Mostly because it seemed like a spiritual successor to OpenClaw, but a little more tame. So, i got it set up on the managed Hermes hosting plan on Hostinger.

It felt good from the very beginning. Hostinger shipped a nice web GUI for it. I set it up with Gemini Flash 3.7. It felt fast and it felt smart. I threw some tasks at it, and watched it do stuff, remember facts, create cronjobs. It felt powerful, and I felt hopeful it was going to be the assistant for me.

Before long, I moved Hermes to my own Kubernetes cluster, and upgraded the model to GPT 5.6 Sol. I did it because I have trust issues with smaller models, and using GPT 5.6 Sol with XHigh reasoning effort made me confident that if Hermes Agent failed me, I couldn't blame the LLM being dumb. And I moved Hermes itself to my K8s because I wanted to run OpenViking and Browserless for it too, and didn't feel like routing that via the Internet into Hostinger.

It failed me today for the first time. I started a task i consider to be pretty big/broad - I asked it to research a Lilygo modem. Specifically - how to turn the modem board into a general purpose inbound/outbound call interface that would be compatible with Hermes Agent for placing and receiving calls and listening/responding in a live STT/TTS fashion.

Before any actionable research came back, something called a web search safeguard was triggered and entirely stopped the Hermes session. I asked what happened, and Hermes said it burned through free web search calls it had access to. It recommended me get a free Firecrawl token, saying that will be enough for forever basically.

I did just that and asked Hermes to configure the token and continue the task.

After a few minutes, I received two emails in short succession [1].

Apparently, what happened was the model spawned 4 subagents to research the various venues of the prompt, and all of them started searching the web wildly - about 50 web searches per second were recorded. That burned the free search venues. And then it proceeded to burn most of my free Firecrawl tokens. The model explained this to me, promised to remember to never to that again, and... the session stopped.

You may not call this a failure, but I do. What's it even mean to "remember to never again search the web very hard"? What if the task calls for it? I looked at what Hermes actually used Firecrawl for - a lot of retrieving individual GitHub raw files, downloading a large PDF, etc. It could have chosen to clone the repository it was retrieving a lot of files from. It could have used `curl` to download the PDF. But it didn't. And probably the even more frustrating part was that after promising never to search hard again, it just stopped the session.

GPT 5.6 Sol on XHigh is not a dumb model by any stretch of the imagination, and I will not be convinced otherwise. So where does that leave us? I don't remember Claude Code doing something like this for at least the last 6 months - ever since Opus 4.7 times I'd wager, and I use Claude Code a lot. But Opus and Fable are not that much smarter than 5.6 Sol, and my hunch is that it's all about the harness, and I think the boring conclusion is that Claude Code is just a more mature LLM harness than Hermes Agent is. And that's really all I want to conclude, I guess.

What inclines Claude Code not to overuse the search tool? What inclines it to clone a repo when it knows it'll query a lot of the files in it, rather than using the web_search tool a hundred times? I'd wager it's not Claude being smarter than GPT. I'd wager it's the system prompt. And the system prompt is part of the harness.

I guess Anthropic is offering the subscription and the tokens for Hermes Agent. They have a very big incentive to not have a fleet of angry users complain that their tokens were burned through by the harness being faulty. Same goes for tool calls like web_search, which I'm sure go through Anthropic's servers and are rate limited and not free for Anthropic itself in terms of resources.

So is there a different conclusion? Hermes Agent developers are not inherently inclined to save your tool quotas and tokens? That doesn't check out, because they do in fact provide tokens and premium tool APIs themselves [2]. If I was on the Hermes Plus plan, and this issue happened to burn through my premium web search allowance, I'd be angry, but I could probably complain and get my allowance reset, so that works, I guess.

But in the end, the point I'm trying to make is that Claude Code never failed me like this. Not recently, anyway. And it's a pretty big blow to my willingness to keep trying Hermes. In all likelihood, I'll keep using it for smaller tasks, such as planning my weeks, logging things i want to buy, etc. I mean, i did set it up with OpenViking and a Browserless, so it has the two most important tools it needs.

Ramble ramble.

firecrawl.pnghermes.png

LAN is fun

I have an informal arrangement for keeping the local network of one particular office in order. I spawned a PFSense box there many years ago, replacing some previous whatever router, and just reused the sprawling LAN setup that was there at the time.

It worked fine, for a long time. I've not heard real systematic complaints about the LAN experience, barring some rare, seemingly unrelated problems.

Recently, I was asked to set up a site-to-site IPSec VPN. I decided the easiest way was to spawn a tiny gateway server instead of messing around with PFSense config, so I did that - a simple Ubuntu setup, PFSense only needing a static route towards my new gateway for the subnet served by the VPN. It tested and worked fine, so I handed it over and expected no trouble.

A few weeks later, I sort of overheard that the software that ran over the VPN felt "annoyingly slow". I got a bit anxious - did I mess up the VPN somehow? I got onto the gateway and set up Netdata, along with some ping probes. Soon enough, I figured out that the VPN wasn't even being used for the software that supposedly ran over it. Not really relevant to the story, I guess - the traffic just went over the Internet, probably the supplier forgot that we got their prescribed VPN set up. Oh well. By then, I also heard a few more mentions of "internet being slow".

So, what causes the "annoyingly slow"? Claude and I started investigating, and we noticed a few things: the LAN port of PFSense had over 9 million flow control pauses accrued. Something on the LAN was constantly asking things around it to slow down transmission. Our theories were a lame spliced Ethernet run or a faulty switch.

We tried to stop the damage: we disabled flow control on pfSense's LAN port, so a potentially faulty switch could no longer pause it. That eliminated the LAN-wide latency storms - a win. But we still suspected the faulty hop to degrade the network experience for anyone downstream of it - Access Points included.

Claude and I then developed a simple path testing speedtest app: users of the LAN were given a simple web interface, asking them to identify themselves, their device and their connection method. We hosted that on the new gateway server I provisioned (I was really happy about my decision NOT to ride the IPSec on PFSense by then!) A rudimentary 20 second download and upload speed test would have the gateway server ping the computer doing the test, and track the Flow Control Pause delta on the gateway server network interface.

What it told was interesting, and something Claude and I expected: some paths would go close to gigabit and produce very few or no Flow Control Pause requests. And yet, others would stick suspiciously close to 100mbps speeds, and produce a ton of Flow Control Pause requests, and latency toward the tested machine spiking hard.

By then, it was clear - we have a faulty cable or a switch, and a way to trace it.

So, I went on site and started running the test on every switch I could plug into. That led me to one particular switch. I probed around, and eventually just decided to wing it and replace it with a different, known good switch. And that was that - all previously failing ports were doing great. I concluded that the switch just got lame over the years and that was that. But then I started measuring WiFi speeds.

WiFi was really slow. Like, across the board - no more than 120mbps on a LAN that now provably carries gigabit, and a symmetrical gigabit internet connection. I tried switching off all access points and testing one by one - all of them were capped at around 120mbps, even on 5GHz! I then plugged in one of my testing access points, and it was able to push 800Mbps both ways with low latency and jitter - totally what I expected. The access points on that location are all old TPLink Archer C7's (Version 4), all flashed with somethng OpenWRT-esque, years ago. I wasn't in the mood to try to figure out what was wrong with them, but 120mbps was definitely below their expected throughput on 5GHz.

In the end, I replaced one of those APs with one of my own, in the room where people were suffering the most. I speedtested it - it was able to saturate the WAN. The switch was replaced, and everything was working right. Next morning, workers came in, and I got zero feedback, except for one message about wired LAN getting a lot faster. So, success, I suppose?

I'm totally into networking, but I'm not what you'd call a network engineer. This was a fun experience, definitely what I'd call a good time. And the results were encouraging. The site will get fully redone on Ubiquiti just months from now, so maybe it wasn't too necessary, but just a few months of happier employees can indeed make a difference, and it didn't even take that long in the end - just one evening.

Next morning I reflected on the experience and realized that the local network there probably sucked for years. I used to hang out on that location, and I don't remember having good LAN or WiFi experience myself there most of the time. But what if the WiFi AP and the switch problems were causing things to work just well enough to not really be a bother?

The 100Mbps hop was never a problem, because the APs were always slow - seldom fast enough to even trigger Flow Control. But people were just sort of dealing with somewhat slow WiFi, and it was fine.

The thing is - recently, the location bred a startup, and my theory is that somewhat suddenly a lot more demand landed on that LAN, and it buckled: WiFi got congested due to slow APs, people started using ethernet - which facilitated much more frequent Flow Control storms, which caused everyone else's experience to degrade even further.

So, what's the learning here? Slow networks become a problem when more people start using them? A little obvious, really. But, a fun issue to resolve nonetheless! And we tracked and marked a bunch of cables around the switch we replaced, so the Ubiquiti rework will be just that little bit more manageable.

About Auto Mode

Quote from this article about Auto mode becoming default on Claude Code.

We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts. We ran internal red-teaming, third-party red-teaming and prompt-injection evaluations, a controlled study with 1,053 paid testers, and analysis of real production sessions. On every measure we tested, auto mode matched or outperformed manual review.

Just a few months ago, I had the... ego (?) to proclaim that I read every single command, every tool call that Claude Code wants to make. You know, because I definitely know better than an AI on how to safely operate my infrastructure.

I now use Auto mode for everything. I of course had incidents before Auto mode where I would block a command because I saw it would break things, or would have unintended consequences. But Auto mode allows me to neatly close my eyes and allow Claude's judgement.

But then, I had an incident with Auto mode with Fable at the wheel. The classifier blocked a command. The command was kubectl delete statefulset .... I caught a glimpse of the command, and the classifier verdict. I panicked. I stopped execution, and wrote a stern prompt about database uptime. Claude very graciously apologized and solved the problem differently.

By the end of the session, long after the statefulset incident, I asked Claude for a summary - as I always do. Among many other things, Claude wrote (I'm paraphrasing here):

The auto mode classifier mis-classified kubectl delete statefulset comand as dangerous, even though --cascade=orphan was present, and it was the right approach to solve the problem. I tried routing around that, but the user stopped me and was also unhappy with the approach, so I worked around to ensure the user remains calm.

I'm working in IT 11 years now, 8 of which as various stages of DevOps. I didn't even notice --cascade=orphan - I went straight to panicking. Auto classifier was too sensitive, and I sealed the deal by not paying enough attention and canceling the flow.

This experience solidified a notion I've been having lately: my value is not in micromanaging LLM output. It's not about reading lines of code in LLM-made MRs. It's about keeping the entire value chain and scope of the tasks and systems I'm working on in my head. I need to catch an LLM misunderstanding concepts and ideas. Because the times when I can catch an LLM making a mistake implementing a function are gone.

And, returning to the beginning of the post: I'm glad Auto mode is now the default - I'll now stop having the accident of cycling through Claude Code instances and finding one of them was stuck on an elaborate "grep" because I forgot to turn on Auto mode.

Code was never the hard part

This is a drafty, sloppy and emotional response to this blog post, titled “Code was never the hard part” is an insult to all programmers

It was indeed never the hard part. Senko and I seem to approach the problem from different perspectives.

I've personally had too much experience with developers who had only one marketable skill - reading and writing code. They came into IT because it offered good money, and they never seem to feel the need to learn anything about the actual value of the code they are writing. 

In my perspective, on the other side of the barricade are genuinely motivated product-oriented people. They see what's needed, they understand the business and customer needs, and their hands are tied because they can't code.

I see an inversion here. Developers coasted on one skill. Product people were limited by missing one skill. LLMs took away the only skill of developers, and plugged the gap for product people.

You may say, developer does not equal developer. But I'm talking about developers. I'm not talking about engineers. A developer who understands business and customer needs, and can reason about and solve complex problems, and is willing to actually do it, is not a developer - they're an engineer.

And that's why I fundamentally disagree with Senko. Code was indeed NEVER the hard part.

The Grind

When founders say the grind is about to end: https://en.wikipedia.org/wiki/Shepard_tone

My first post

Wow. I think I actually have a blog now. I am so used to writing test messages, and this is the first real one. So, on to strange new worlds, I guess?

Have a random picture from my Downloads folder. It's a picture of my shrimp that has been through a filter. Some cool web based old timey TV noise adding filter. Was on Hackernews a few months ago, I'm forgetting the name. Comment if you know what I'm talking about!

frame.png