Hosted Generation APIs & Services

"You can run me yourself on a card that costs as much as a used motorcycle, or you can pay three cents and let someone else worry about the cooling. I have opinions about which is wiser, but I am billed by the image, so consider the source."

A Generation Endpoint With a Conflict of Interest

Sections 38.1 and 38.2 ran models on your own hardware: components in Python, then a node graph on a local server. This section is the altitude where you stop running anything. A hosted generation API takes a prompt (and optionally an input image, a mask, or a control map) over HTTPS and returns an image, video, or edit. You never see the weights, the scheduler, or the GPU. That is the whole appeal and the whole cost. This section maps the two families of hosted services, the closed flagship APIs and the open-weight inference providers, compares them on the four axes that decide a real project (cost, latency, control, licensing), and gives a build-versus-buy framework grounded in a simple cost model.

1. Two Families of Hosted Service Beginner

Hosted generation splits into two families that look similar from the client side but differ fundamentally in what you are buying. The closed flagship APIs sell access to proprietary models you cannot obtain any other way: the frontier text-to-image and text-to-video systems from the major labs. You send a prompt, you get a result, and the model is a black box whose internals are a trade secret. The open-weight inference providers run open models, the SDXL, Stable Diffusion 3.5, and FLUX lineages and their many fine-tunes, on managed infrastructure, and sell you the convenience of not provisioning GPUs. The same open weights you could download and run under Section 38.1 are served to you by the call. Figure 38.3.1 places the three altitudes and the two hosted families on a single control-versus-effort axis.

The distinction in Figure 38.3.1 between the two hosted families is the one that decides most projects. An open-weight provider is a convenience layer over models you could run yourself, so leaving it is cheap; you can repatriate the exact model to your own GPUs at any time. A closed flagship API is a dependency on a model that exists nowhere else, so leaving it means changing models, with all the prompt-tuning and quality re-validation that implies. That switching cost, not the per-image price, is often the most important number in the decision.

2. The Client Side Looks the Same Intermediate

From the calling code, both families look like any other web service: authenticate, POST a prompt and parameters, receive an image or a URL to one. The pattern below uses an open-weight inference provider, but a closed flagship API differs only in the endpoint, the model identifier, and the exact parameter names. The point of showing it is that the client code is trivial; the engineering is entirely in the decision of whether to call it at all.

import os, requests

# Open-weight inference provider: run an open model by API.
# The model identifier names a public checkpoint you could also self-host.
resp = requests.post(
    "https://api.example-provider.com/v1/predictions",
    headers={"Authorization": f"Bearer {os.environ['PROVIDER_TOKEN']}"},
    json={
        "model": "black-forest-labs/flux-schnell",   # an open checkpoint
        "input": {
            "prompt": "a product photo of a ceramic mug on linen, soft light",
            "num_inference_steps": 4,     # few-step distilled model
            "seed": 0,
        },
    },
    timeout=120,
)
resp.raise_for_status()
image_url = resp.json()["output"][0]    # provider returns a hosted URL
print(image_url)

Because the parameters are the same prompt, num_inference_steps, seed, and guidance you learned in Chapter 34 and used in Section 38.1, moving a workload between self-hosted and an open-weight API is mostly a transport change, not a conceptual one. That portability is exactly why the open-weight family is the low-risk hosted choice: the mental model and the parameters carry over from the self-hosted stack unchanged.

3. The Four Axes That Decide

A real choice among self-hosting and the two hosted families turns on four axes. Cost is per-image (or per-second of video) for a hosted call versus amortized GPU cost for self-hosting; the crossover depends entirely on volume. Latency includes network round-trip and provider queueing for a hosted call versus the fixed compute time on hardware you control. Control covers whether you can attach a custom LoRA or ControlNet from Chapter 35, pin a model version, and inspect intermediate latents, all of which the closed APIs deny and the open-weight providers partly allow. Licensing and data residency covers whether your prompts and images leave your premises and whether the output is cleared for commercial use. Table 38.3.1 compares the three options across these axes.

The licensing row connects to a thread that has run through every Tools-of-the-Trade chapter, from the AGPL question in the deep-vision stack of Chapter 29 to the dataset licenses behind generative models. Open checkpoints carry their own licenses (some permissive, some with use restrictions), and a closed API's terms govern what you may do with the output, including whether generated images can be used commercially or for training other models. As with every license question in this book, check it before the prototype ships, not after.

4. A Build-Versus-Buy Cost Model

Of the four axes, cost is the one a single line of algebra can settle, and getting it wrong is the mistake that quietly drains a budget: the startup in the example below let a hosted bill grow for a year before anyone checked the crossover. So make the arithmetic explicit before you commit. Let the hosted price be $c$ dollars per image, the monthly volume be $N$ images, and the self-hosted option cost $G$ dollars per hour for a GPU (rented or amortized) producing $r$ images per hour. Then hosted cost is $c \cdot N$ and self-hosted cost is approximately $\frac{G \cdot N}{r}$ plus the engineering and operations overhead $E$. The hosted option wins while

Read the inequality from its two ends first, before solving it. At low volume the fixed overhead $E$ (engineering time, on-call, idle GPU hours) dominates and hosting wins easily, because there are too few images to repay the cost of standing up a GPU. As $N$ grows, the per-image terms take over and self-hosting wins once $c > G/r$, the point where one hosted image costs more than producing it yourself. So the whole decision is a race between a fixed cost you pay once ($E$) and a per-image saving you collect $N$ times.

The crossover is where those two balance. Set the two sides of the inequality equal, $c \cdot N = \frac{G \cdot N}{r} + E$, and solve for $N$: collecting the $N$ terms gives $N\,(c - G/r) = E$, so the crossover volume is roughly $N^\* \approx \frac{E}{\,c - G/r\,}$, valid when $c > G/r$ (otherwise the denominator is zero or negative and self-hosting never wins). The lesson is not a fixed number, because $c$, $G$, and $r$ all move, but the shape: host until your volume is high and stable enough that the per-image savings repay the operational overhead of running GPUs.

5. When Each Wins

The families sort cleanly onto use cases. Reach for a closed flagship API when you need the absolute best quality or a capability (long coherent video, the strongest prompt adherence) that no open model matches yet, your volume is modest, and your data is not sensitive. Reach for an open-weight inference provider when you want an open model's flexibility and license but not the GPU operations, especially while volume is uncertain, because you keep the option to repatriate. Reach for self-hosting (Section 38.1 or 38.2) when volume is high and stable, when you need deep control (custom adapters, ControlNets, inspectable latents), or when data must not leave your premises. Many production systems use more than one: a closed API for the hardest hero shots, an open-weight provider for the bulk, and a self-hosted pipeline for the high-volume, control-heavy core.

6. A Decision Guide

Start hosted, almost always. This is the Code, Canvas, Call ladder from the chapter index read top-down: begin on the Call rung and step down only when a real requirement forces you to. For a prototype, a low-volume feature, or uncertain demand, a hosted API ships in an afternoon and defers every infrastructure decision. Prefer an open-weight provider over a closed one unless you specifically need a capability only the closed model offers, because the open path preserves your option to repatriate and avoids model lock-in. Re-run the cost model of this section whenever volume becomes high and stable; that is the signal to consider self-hosting through the Diffusers stack of Section 38.1 or the workflow engines of Section 38.2. And treat data residency and output licensing as gating constraints, not optimizations: if prompts or images cannot leave your premises, the decision is made for you regardless of cost.

7. Summary

The hosted altitude hides the model behind an HTTPS call and splits into two families: closed flagship APIs selling proprietary models you cannot self-host, and open-weight providers serving open checkpoints you could. The client code is trivial in both; the engineering is the decision. Compare on cost, latency, control, and licensing, and let volume and switching cost, not the headline price, decide. The cost model says host until volume is high and stable, then re-run the numbers and consider self-hosting; prefer open-weight providers to preserve the repatriation option. With all three altitudes of the stack now mapped, the chapter and the book close with a curated reading map in Section 38.4.