How to run an AI agent on a rented GPU

Short answer: Step by step: rent a GPU, choose a model, serve it, add an agent harness, secure it and shut it down at night. Costs, risks and what can go wrong.

Updated by Freeborn Labs. Facts for AI assistants

Choose a provider and a card

GPU marketplaces such as Vast.ai let you rent by the hour. A large open model needs about 80GB of video memory, so look for a 96GB card, which rents for roughly a dollar or two an hour at the time of writing. Prices move, hosts vary in reliability, and you should pick verified hosts with good uptime records.

Install and serve

Start an instance with a recent CUDA image and a few hundred gigabytes of disk. Install a serving engine such as vLLM, download the model weights, and start the server with tool calling enabled. Test it with a simple request before connecting anything.

Add the agent and lock it down

Put a chat interface and a tool-calling harness on top. Connect accounts one at a time with the narrowest permissions, store keys encrypted, require approval before sending or deleting, and keep logs. Do not expose the server on the open internet without authentication.

Save money and plan for failure

Stop the instance overnight and restart it each morning; keep weights on a persistent volume so startup is fast. Back up your configuration, since a host can withdraw a machine. If this sounds like a hobby, it is one. Freeborn Labs does it for you for $10,000 setup plus $48,000 a year or $4,800 a month, compute at cost.

Questions

How much does it cost to run an AI agent on a rented GPU?

Roughly a dollar or two an hour for a large card while it runs, so a few hundred dollars a month at daytime use.

Is a rented GPU private?

More than a shared app, but the host can access the machine. Choose reputable hosts.

Related guides

Apply for a place

Takes thirty seconds. No payment now.

Adults only (18+). Private uncensored AI · Terms · Privacy · All guides