Enough GPU: Sizing a Machine for Chat, Images and Video
Short answer: Memory decides what fits. Start from the model card, work out the memory the model needs, and match it to a box with that much memory — plus room for the conversation. This page stays qualitative on purpose: the only numbers here are the rough rule the pillar page itself gives.
The one rule that does most of the work
The pillar page gives a rough rule: billions of parameters times about half a byte at 4-bit, or two bytes at full precision, plus room for the conversation (the context). Check the model card for its memory need, then match it to a box with that much memory. Search 'VRAM requirements' with the model name.
That rule is enough to sort most listings. If the arithmetic says the model plus its context does not fit in the box, no setting fixes it — the choice is a bigger box or a smaller way of running the model, and the models page explains why that trade can change quality.
Thinking about chat, images and video
Chat is the model's own workload: the model, plus the context of the conversation it is holding. Images and video run through a different tool — in the full build they run in ComfyUI, driven through its API — so think of them as a second workload sharing the same machine, not as a setting inside the chat model.
The honest order in the guide is to build chat before images, and images before the agent. Sizing follows the same order: get one workload running well on the box before you ask the box to carry the next one.
The serving tool named on the pillar page is vLLM: it serves a model from a GPU through an API that looks like OpenAI's. Ollama is the easiest local test before you rent anything.
What a cheap box is usually telling you
Watch for a box that is cheap because it is shared or unreliable, slow starts when a sleeping box wakes, and instances that vanish mid-job. A low hourly price on a box that disappears during your work is not a saving; it is the failover step arriving early.
The hours, as estimated on the pillar page
First time: about 10 to 20 hours. Then about 30 to 60 minutes a week. These are the pillar page's estimates, not measured data, and they assume you have built software before. If you have not, add time.
Related DIY pages
Back to the pillar: What it takes to do this yourself
Adults only (18+). What it takes to do this yourself · Apply