Here's a question for you: if you wrote extra bitchen skills, prompts, and instructions, could you make Haiku act like Opus or Sonnet?
I'm prepping my first cohort session for tomorrow and there's still room to join us. I'm going to go deep on this stuff, to see how and where we can extract the best performance and quality from our LLM tools of choice. For me, this experiment is critical! Do we always need to use frontier models to write code? Or can we hand that off to something cheaper? If you want to find out with us, and contribute what you know, please do!
Anyway: Haiku, if you didn't know, is Claude's "ugly stepchild" model that's fast and, well, kinda dopey. By "dopey" I mean it will give you something that's totally 2025 in terms of AI, when you felt good about your job and that AI was just a toy.
So I'll lead with that: Haiku is just a toy. I proved it to myself today, and I'm about to share that with you.
OK Haiku, You Stay Here and Guard the Door
I created a 57 line CLAUDE.md to start with, which basically was this:
I got into coding conventions, testing, my preferred stack (Bun/Hono/SQLite/Drizzle, AWS, Cloudflare) and basic code styles. This was step 1, and I knew step 2 was coming.
The prompt (remember this, it's the one I used for every model and config in this experiment) is basic, which means I wanted to see how "creative" the model could be, given a problem that's pretty basic:
You are an online store expert familiar with digital commerce and delivery. You are creating an order processing and digital delivery pipeline for me as described in the instructions.
I want to prove that you, Haiku, can create an amazing application as long as I give you the rules by which to do so. Code quality is of paramount importance here, as is testing what you're doing.
Show me something that blows my mind. Create my store. This is a one-shot deal, not interactive. I'm going to the beach, do your best.
Yes, I'm aware that I'm asking for trouble here, but stay with me.
Extreme Failure
When the build was finished I upped the model to Opus in a new client window and then told it, basically:
I just created an app using Haiku with this prompt: [CLIPPED]. Evaluate how well it did. Review the code. Give me a quality score based on SOLID and good patterns
Not surprisingly, the entire effort was a massive waste of tokens:
Security issues, things that were claimed as finished that clearly weren't... SINGLETONS!?!
That's when I thought "I just need to get creative".
No Seriously Make Haiku Work!
After reading the review, which was really unexpected, I told Opus this:
Help me create a set of skills that will make Haiku do a better job, specifically: verifying that it did in fact do what was asked. Review itself, and so on. What do you think? Is there anything I missed? I'm going to rerun this exercise and I want to see if I can bump up the code quality.
I find that giving your LLM a challenge seems to add some extra "sauce" to the effort, making it try extra hard.
The result was a series of skills that amounted to "try real hard not to suck", including verification steps, self-review, and how to test properly. I also got a warning:
The core failure wasn't code quality. It was claim/reality decoupling. Haiku wrote a RUN.md asserting "Avoided Singleton," "Full SOLID adherence," "Zero technical debt," "Ready for production" — while the code did the opposite. It generated the report it wished were true...
Basically Opus was telling me that it's little bro was... not exactly capable. But I figured why not give the kid a chance?
Wow. What Year Is This?
Claude updated my CLAUDE.md and added all the skills I needed. I cleared up both the global project memory and the local session (creating a completely new workspace) and gave it the exact same prompt as before.
This time, however, CLAUDE.md had this:
I was warned about this addition, too. Opus told me, basically: "just because this is here, it doesn't mean it will be followed."
Yep. Completely accurate. Wild how it was totally ignored!
Long story shorter, I ran the exact same review (using Opus to evaluate this new Haiku effort) and ... well...
Haiku just... ignored everything. So much for that idea!
What About... Plugins?
OK so trying to get Haiku to do the heavy lifting is silly, but what about Your Favorite Plugin? There are quite a few out there that are designed to do the needful without producing "garbage"... or are they?
I used my exact same CLAUDE.md, but with slightly different prompts. For instance, I used the same prompt as above (with the same CLAUDE.md), and I added "use the superpowers plugin to build this.
It took about 15 minutes, and then I asked Opus to review. It was... suboptimal:
This isn't a superpowers problem, to be clear. I was trying to see the main differences between Haiku with guardrails and these plugins.
I did exactly the same with Oh-My-ClaudeCode using the autopilot skill. Here's the review for OMC, and I must admit I was happily surprised:
The tests were outstanding too, and followed my demands reasonably. I was honestly impressed!
But Then Again...
We don't one-shot code, I know that. There's no orchestration here and superpowers was the only test that had actual Git commits and a plan to follow.
It's not about the code only is it? It's all the things that support the effort, building now and for the future.
That said, this litte afternoon of experimenting taught me something important: you can't make a lower model do something it's not trained to do. Haiku is fine for things that don't require reasoning (or adhering to reality), but yeah, if you're going to be writing code, frontier models are where you probably want to be.
How does this stack up against my groovy skill library? Well I'm going to get into that with my cohort, which starts tomorrow at 1100 PDT, so if you want to join us come on! We're starting off with deep dives everywhere, flexing what we know, and making sure that level of experience and skill is reflected in our very own library of skills and commands.
Other than that - thanks for reading as always. This is fun stuff :).
Rob