Cactus (YC S25) Just Distilled Gemini Into a 26M Model. Should YC F26 Founders Train Their Own Model or Wrap an API?

Today the top story on Hacker News, at 667 points, is a Show HN from Cactus (YC S25): they distilled Gemini's tool-calling ability into a 26-million-parameter model called Needle. It quantizes to a 14MB INT4 file, runs at roughly 1,200 tokens/second of decode on a consumer device, and is MIT-licensed. A function-calling model small enough to run on a smartwatch, trained by a Summer 2025 batch company, sitting at #1 on HN.

If you're writing a YC F26 application right now, that single post should reframe a question almost every AI founder is quietly avoiding: do you need to train your own model, or is wrapping someone else's API enough?

What Cactus actually did (and why it matters for your application)

Needle isn't a frontier model. It does one thing: parse a user request and emit the right structured function call ("set a timer for 9am," "text Mom I'm late"). Cactus generated a 2-billion-token synthetic dataset of function calls using Gemini, then post-trained their small architecture on it in about 45 minutes. The pretraining was heavier (16 TPUs, 27 hours), but the point stands: they took a narrow, high-value capability out of a giant model and shrank it until it fit on hardware the giant model can't touch.

That is the move. Not "build a better Gemini." Take one capability that matters, and make it run somewhere the incumbent structurally cannot follow you, which is on the device, offline, in 14MB, at zero marginal cost.

For a YC application, this is the difference between a project and a wedge.

So should you train your own model for YC F26?

The honest answer for most applicants: no, not at first, and YC does not expect you to. A huge share of recent batches are teams building on top of OpenAI, Anthropic, and Google APIs. YC funds wrappers constantly because a wrapper that owns a workflow, a dataset, and a set of users is a real business. "We wrap GPT" is not a disqualifier.

But "we wrap GPT" is also not an answer to the defensibility question, and that question is now the most predictable thing in the entire 10-minute interview. The partner across the table has seen four hundred AI wrappers this cycle. When they ask "what stops OpenAI from doing this in a weekend," a shrug ends the meeting.

What Needle illustrates is that there is a middle path between "we trained a foundation model" (you didn't, and they won't believe you) and "we call an API" (so does everyone). The middle path is: you distilled, fine-tuned, or specialized a small model around proprietary data or a constraint the big labs ignore.

When wrapping an API is the right call

Wrap an API when your edge is the workflow, the distribution, or the data you accumulate, not the intelligence itself. If you're building vertical software for dental offices, your moat is dental offices, not your model. Use the best API, ship fast, and put your application's energy into showing usage: how many real dentists use it weekly, and what happens to that number when you turn the AI off.

That last test, often called the "unplug test," is what YC partners actually probe. If your product is just a thin prompt over GPT, unplugging the model unplugs your company.

When you should consider training or distilling

Consider owning a model when one of these is true: you operate where APIs can't reach (offline, on-device, privacy-locked, latency-critical, or regulated), or you have proprietary data the labs will never see. Cactus lives in the first category. Edge inference is a place a cloud API physically cannot win, because the whole point is that there's no network call.

Notice Cactus didn't train Needle from a blank slate using their own money and intuition. They used Gemini as a teacher. Distillation let a small team capture a frontier capability cheaply and then deploy it where they have the structural advantage. That's a fundable story: not "we're smarter than Google," but "we put Google's capability where Google's product can't go."

How to write the defensibility paragraph YC actually wants

Whether you train or wrap, your application needs one tight paragraph that survives the "why won't a lab crush you" question. The structure that works:

1. Name the capability you depend on (e.g., reliable tool calling, document extraction, voice).
2. State where you run it and why that location is defensible (on-device, inside a customer's VPC, on data only you have).
3. Show the compounding asset that grows with usage (a fine-tuning dataset, a feedback loop, an integration surface).

For Cactus, that paragraph writes itself: tool calling, on-device in 14MB, getting better as more on-device interactions feed the next distillation run. For you, the same three slots need real answers, not adjectives.

The failure mode is hand-waving slot two and three with words like "proprietary" and "defensible" while having neither. YC partners discount those words to zero. They want the mechanism.

The trap: training a model to look impressive

There's an opposite mistake worth naming. Some technical founders, especially strong ML researchers, train a model because they can, not because the business needs it. That reads as "resume-driven development" and partners catch it fast. If wrapping an API gets you to real users faster and the model isn't your moat, training your own is a distraction that burns the runway and the calendar before Demo Day.

The question is never "can we train a model." It's "does owning this model let us do something for users that we otherwise couldn't, and does it compound?" Cactus passes that test. Make sure you can say the same before you write "we trained our own model" in your application.

Before you submit

Read your own application and find the sentence where you answer "why can't a big lab do this." If that sentence leans on a buzzword instead of a mechanism, fix it before a partner does it for you in the interview.

This is exactly the kind of thing a second pair of eyes catches and you can't, because you're too close to it. YC Roaster connects you with founders who've actually been through YC and will tell you, bluntly, whether your defensibility paragraph holds up or whether you're one "so what stops OpenAI?" away from a rejection. Get that read before the partners give you theirs.

The wrappers that get in aren't the ones pretending to be model companies. They're the ones who know exactly which capability they own, where they own it, and why it compounds. Cactus just gave you a clean example. Use it.