Rent an NVIDIA L40S for production-grade FP8 70B inference on Ada silicon. 48 GB ECC + Ada FP8 tensor cores — Llama-3 70B FP8 single-card serving at ~720 tok/s with 16-request KV, Hunyuan-Video FP8 at 4.5 min per 5-second clip, LLaVA-1.6 multimodal serving with vision encoder resident. Billed per-minute, paid in BTC, USDT/USDC or CLORE. The default 70B serving target when datacenter HBM supply is tight.
L40S is what happens when Nvidia takes 4090-class silicon, doubles the VRAM, adds ECC, locks the clocks for 24/7 operation, and ships it in a passive datacenter form factor.
FP8 quant fits Llama-3 70B in 48 GB with KV cache room — and the L40S delivers it at typically 40–60% of an H100's rental price. The pragmatic pick for production inference teams who don't need HBM3 bandwidth or NVLink fabric, just consistent token throughput.
SDXL, Flux, Stable Video Diffusion, HunyuanVideo. RT cores 3rd-gen + OptiX make it the fastest non-HBM card for diffusion.
13B–34B QLoRA on a single card. 48 GB ECC means stable long runs without the corruption risk of consumer GDDR6X.
L40S delivers Hopper-class FP8 inference at a fraction of the H100 hourly rate. The right card when latency matters but you're not training from scratch.
// prices are spot-market lows · refreshed every 60 s
Every server is priced by its host. These are the live floors across the marketplace — you'll see hundreds of variants once you're in.
No sales call. No quota request. No three-week procurement. The first four commands are all you need.
Filter the marketplace by L40S 48GB, country, GPU count, reliability score, network speed.
Choose a Docker image — PyTorch, vLLM, ComfyUI, Blender — or paste your own.
You get a public endpoint, an SSH key, and Jupyter on port 8888 in under 90 s.
Per-minute billing rounds to the second. Stop the instance and the meter stops with it.
For inference, often yes — FP8 throughput on Llama-3 70B is competitive at a fraction of the rental price. For training, the H100's HBM3 bandwidth and NVLink fabric still win. Pick L40S for serving, H100 for pretraining.
An L40S serving Llama-3 70B FP8 with vLLM and continuous batching pushes roughly 3,000-4,500 output tokens/second at batch saturation. At a $0.78/hr spot rate, that lands near $0.05-$0.07 per million output tokens before the 2.5% spot fee. PoH staking knocks the fee in half; reserved spot floors land you closer to $0.04/M. Numbers vary with prompt length and batch shape - benchmark on your traffic.
Yes. The inference tier (T4, L4, L40S, A10) is exactly what vLLM's PagedAttention and continuous batching are tuned for. L40S handles 70B FP8 single-card with KV-cache headroom; A10 and L4 serve 7B-13B at high throughput; T4 covers Whisper, embeddings, and 7B INT8. Pull the official vLLM Docker image, point it at your model, expose port 8000.
L40S has Ada FP8 tensor cores - the same architecture as H100 for inference math, at a fraction of the rental price. L4 also supports FP8. T4 and A10 are pre-FP8 but have INT8 (T4 added INT8 in Turing, A10 in Ampere) and excel at quantized 7B-13B serving. Pick L40S when FP8 throughput matters; pick A10 or T4 when $/request matters more.
On A10 or L4 with vLLM and batch-1, time-to-first-token for a 7B FP16 model lands around 80-150 ms; p99 inter-token latency is 25-40 ms. L40S with FP8 cuts both roughly in half. T4 doubles them. Real numbers depend on prompt length and concurrent batch size - low-batch interactive serving is fastest, high-batch saturation maximizes throughput.
MIG (Multi-Instance GPU) is supported on A100, A30, and H100/H200 - not on L4, L40S, T4, or A10. For consumer-tier multi-tenancy on the inference tier, run multiple model replicas inside a single Docker container or use container-level resource limits. If you need hardware-isolated MIG slices, rent A100 40GB and partition into up to 7 instances.
48 GB ECC + Ada FP8 + 350 W — the production substitute for H100 inference when supply is tight.
FP8 quant + 48 GB fits 70B with room for KV cache — typically 40–60% the price of an H100 for inference workloads.
Read the guide →FP8 path nearly doubles Hunyuan throughput vs A6000 at the same VRAM — production-grade gen-video card.
Read the guide →Vision-language SaaS pipeline — 48 GB holds vision encoder + Llama backbone + 16-batch KV simultaneously.
Read the guide →Side-by-side specs across the inference tier. Click any row to see that GPU.
Step-by-step guides verified on CLORE.AI hardware. Pick a workload, copy the docker image, ship in minutes.
Per-minute payouts in BTC, USDT, USDC or CLORE. No listing fee, no contracts, withdraw any time.
Hosts around the world are accepting workloads right now. Sign up, top up your wallet, and the next hour is yours.