A speculative proposal. I’m looking for feedback (especially reasons this may be dead-on-arrival).
Here’s the state of things, as I see it:
- LLM providers (OpenAI, Anthropic, Google, Meta, xAI, etc) want more people to use their APIs
- Developers want to develop cool open projects that use LLMs
But LLMs are expensive, so developers face a choice:
- Let user provide their own API keys (BYOK): this is common, esp among smaller projects. However, the developer doesn’t make any money, and so has a smaller incentive to build this kind of software.
- Charge a subscription: this is the dominant model. However, this doesn’t work well
- For projects that have non-Gaussian usage patterns (sporadic, power law, etc)
- For projects that are very simple; it’s a pain for both dev and user to setup a billing system
Overall, this lends the ecosystem towards fewer open, small, composable tools, and more towards larger walled gardens behind a subscription barrier.
So, I propose a simple solution to unify this gap and encourage proliferation of lots of small, useful open source projects: API requests should include a referral header.
How this works (analogous to e.g. Amazon Affiliate Program):
- Developer gets a referral code from an LLM Provider
- Within the open-source project, developer places code in LLM API request header
- End user uses the project with their own API key
- End user is charged the normal $ amount for their usage
- Developer receives credits (for API usage or cash) for some small % of the usage, e.g. 1%.
This seems aligned with incentives of all parties:
- LLM API providers get more traffic
- User experience is unaffected
- Developers of “small but mighty” projects make some $
Largest issues:
- Fraud prevention: while other affiliate/referral programs do exist, it might not work out as well here, or just be too much of a distraction for LLM providers to spend time on.
- Unit economics: the incentive may be too small to meaningfully impact developer choices.
FAQs:
Q: Won’t people build cool tools for the “love of the game” without any expectation of reward?
A: Some people will, but any open-source founder will tell you there is a tension with monetization, which leads many companies to either introduce more limiting licenses, or adopt an open-core approach (with premium features closed).
Q: Why hasn’t this been done with other APIs before?
A: LLM APIs are new in the sense that 1) they are expensive + 2) BYOK is common.
Q: Don’t LLM providers already have a ton of traffic?
A: They are clearly in competition, and many providers are trying to catch up to OpenAI/Anthropic in terms of API use. This proposal would give smaller LLM providers another mechanism to encourage adoption.
Q: Don’t most tools let the end-user choose between models, so the competition point is moot?
A: There are non-zero switching costs (tuning prompts, etc) between models, so this proposal would give stronger incentives for developers to build for providers with high referral credits.
Q: Don’t LLM providers operate at very low gross margins?
A: It does seem like that, but even a small reward could be a strong incentive for developers.
Q: Really? Is this enough money to actually make a difference to developers?
A: For popular projects, yes (hopefully). Users can spend a surprising amount on token-hungry applications - see this user spending $6k/month on Cursor.
Q: How do you prevent fraud/abuse?
A: While it’s easy to detect the obvious case (referral code on same account as API key), it’s difficult to prevent header injection/override, referral rings, etc. It may be necessary to do some combination of 1) eligibility filters (KYC + public repos only), 2) cryptographic signing, 3) heuristics (delayed settlement windows, etc.), or 4) end-user opt-in at the key-level (adds friction, not ideal).