How to download a file in Python
Stream the response with httpx.stream, write to disk in chunks. Never load a big download fully in memory.
import httpx
url = "https://example.com/big.zip"
with httpx.stream("GET", url, timeout=30) as res:
res.raise_for_status()
with open("big.zip", "wb") as f:
for chunk in res.iter_bytes(chunk_size=64 * 1024):
f.write(chunk)httpx.stream returns a context manager that keeps the response body streaming instead of loading it all into memory. res.iter_bytes yields chunks — 64KB is a reasonable size (large enough to amortise syscall cost, small enough to stay in L1 cache).
Mode 'wb' opens the file in binary write mode — required for anything that isn't text. Never use 'w' for a download; text mode will corrupt any file with byte sequences that aren't valid UTF-8.
For resumable downloads (long files, flaky connection), pass a Range header once you know how many bytes you already have on disk.
Variations
Small file, one-shot
import httpx
res = httpx.get(url, timeout=30)
res.raise_for_status()
open("file.pdf", "wb").write(res.content)OK for small files (< 50MB). Loads the full body into memory.
With a progress bar
from tqdm import tqdm
with httpx.stream("GET", url) as res:
total = int(res.headers.get("content-length", 0))
with open("out.zip", "wb") as f, tqdm(total=total, unit="B", unit_scale=True) as bar:
for chunk in res.iter_bytes(chunk_size=64 * 1024):
f.write(chunk)
bar.update(len(chunk))Requires pip install tqdm.
Resumable download
from pathlib import Path
start = Path("file.zip").stat().st_size if Path("file.zip").exists() else 0
headers = {"Range": f"bytes={start}-"}
with httpx.stream("GET", url, headers=headers) as res:
with open("file.zip", "ab") as f:
for chunk in res.iter_bytes():
f.write(chunk)Server must support Range requests (most CDNs do).
Get one Python lesson + one career idea every Friday
No spam, no "buy our course now". Three bullets, every Friday. Unsubscribe with one click.