Python requests JSON: GET, POST, Headers, and Response Parsing

Q: How do I parse a JSON response with Python requests?

Call response.json() on the response object; it internally calls json.loads(response.text) and raises ValueError if the content is not JSON. Always call response.raise_for_status() first to check for HTTP errors before parsing. For extra safety also check response.headers.get('Content-Type', '').startswith('application/json') before parsing — some APIs return error HTML with a 200 status code. The response.json() method also respects the charset declared in the Content-Type header, making it slightly more correct than manually calling json.loads(response.text) for non-UTF-8 encoded responses.

Q: How do I send JSON data with Python requests?

Use the json= parameter in requests.post()/put()/patch(): requests.post(url, json=payload) — this automatically serializes the dict to JSON and sets Content-Type: application/json. Do NOT use data= with json.dumps() — this sends the data as form-encoded and won't set the Content-Type header correctly. Use data= only for form-encoded data (like HTML form submissions). The json= parameter is the correct and idiomatic way to send JSON in requests.

Q: What is the difference between requests.get().json() and json.loads(response.text)?

response.json() also considers the response encoding (charset from Content-Type header) when decoding bytes; json.loads(response.text) always uses the text as-is after requests decodes the bytes. They are functionally equivalent for UTF-8 responses (the vast majority of JSON APIs). response.json() is slightly more correct because it handles non-UTF-8 encodings declared in Content-Type — for example, if an API declares charset=iso-8859-1, response.json() will decode correctly while json.loads(response.text) may also work because requests.text uses the detected encoding.

Q: Should I always call raise_for_status() before response.json()?

Yes. raise_for_status() raises HTTPError for 4xx/5xx responses. Without it, a 404 or 500 response body will parse as JSON if the API returns error details in the body — and you will accidentally treat an error body as success data. The correct pattern is: response.raise_for_status() then response.json(). Some APIs return structured JSON error bodies ({"error": "not found"}) on 4xx responses; you can catch HTTPError and then call response.json() inside the except block to parse the error details.

Q: How do I set a timeout in requests?

Pass timeout=seconds or timeout=(connect, read) tuple. Connect timeout is how long to wait for a TCP connection to be established. Read timeout is how long to wait between bytes of a response — not the total response time. Always set a timeout — without it, requests hangs forever on slow or unresponsive servers. The requests documentation recommends timeout=(3.05, 27): 3.05 seconds to connect (slightly above 3 to avoid race conditions with TCP retransmission) and 27 seconds to read. For large downloads, set a higher read timeout.

Q: How do I add authentication to a requests JSON API call?

Bearer token: headers={'Authorization': f'Bearer {token}'}. Basic auth: auth=('user', 'pass') or auth=HTTPBasicAuth('user', 'pass'). API key header: headers={'X-API-Key': key}. API key query param: params={'api_key': key}. For OAuth 2.0, use the requests-oauthlib library. Set headers on a Session object to avoid repeating them on every call: session.headers.update({'Authorization': f'Bearer {token}'}). Never hardcode tokens in source code — use environment variables or a secrets manager.

Q: What is a requests.Session and when should I use it?

Session reuses TCP connections (connection pooling) for faster successive requests to the same host. It also stores default headers, auth, and cookies across requests so you don't have to repeat them. Use Session when making 3 or more requests to the same API. A context manager (with requests.Session() as s:) closes all connections on exit, preventing connection leaks. Connection reuse reduces latency by approximately 30-50ms per request versus creating new connections. You can also mount retry adapters on a Session to add automatic retry logic for transient failures.

Q: How do I validate JSON responses from requests with Pydantic?

Call User.model_validate(response.json()) — Pydantic validates the dict structure and types at runtime. It catches missing required fields, wrong types, and invalid enum values, and returns a typed Python object with IDE autocomplete. Wrap in try/except ValidationError to handle API shape changes gracefully. Pydantic is safer than TypedDict (compile-time only) for production API clients because it validates at runtime when the actual API response arrives. Define a model for the full response shape including nested objects and lists.

Written and reviewed by the Jsonic editorial team — every guide is verified against the official spec or runtime before publication.

Last updated: May 19, 2026

The Python requests library is the de facto standard for making HTTP calls in Python. It has first-class support for JSON: sending JSON payloads, parsing JSON responses, and handling authentication and errors. This guide covers the full workflow from a basic GET to production patterns with sessions, retries, Pydantic validation, and pagination.

GET and Parse JSON Response

import requests

# Basic GET request
response = requests.get('https://api.example.com/users/1')
response.raise_for_status()   # raises HTTPError for 4xx/5xx
user = response.json()         # → Python dict/list

print(user['name'])   # → "Alice"
print(user['email'])  # → "alice@example.com"

# With timeout (always set one!)
response = requests.get(
    'https://api.example.com/users',
    timeout=(3.05, 27),   # (connect timeout, read timeout) in seconds
)
response.raise_for_status()
users = response.json()

# Full error handling pattern
try:
    response = requests.get('https://api.example.com/data', timeout=10)
    response.raise_for_status()
    data = response.json()
except requests.exceptions.Timeout:
    print('Request timed out')
except requests.exceptions.HTTPError as e:
    print(f'HTTP error: {e.response.status_code}')
except requests.exceptions.RequestException as e:
    print(f'Network error: {e}')
except ValueError:
    print('Response is not valid JSON')

POST JSON Data

import requests

# Use json= parameter — requests handles serialization + Content-Type header
payload = {
    'name': 'Alice',
    'email': 'alice@example.com',
    'role': 'admin',
}

response = requests.post(
    'https://api.example.com/users',
    json=payload,   # ← sets Content-Type: application/json automatically
    timeout=10,
)
response.raise_for_status()
created = response.json()   # → {"id": 42, "name": "Alice", ...}
print(f"Created user #{created['id']}")

# PUT / PATCH
response = requests.put(
    f"https://api.example.com/users/{created['id']}",
    json={'name': 'Alice Smith', 'role': 'editor'},
    timeout=10,
)
response.raise_for_status()

# PATCH (partial update)
response = requests.patch(
    f"https://api.example.com/users/{created['id']}",
    json={'role': 'viewer'},
    timeout=10,
)

# DELETE
response = requests.delete(
    f"https://api.example.com/users/{created['id']}",
    timeout=10,
)
response.raise_for_status()   # 204 No Content — no body to parse

Authentication and Headers

import requests

# Bearer token auth (most common for JSON APIs)
headers = {
    'Authorization': f'Bearer {access_token}',
    'Accept': 'application/json',
    'X-Request-ID': '550e8400-e29b-41d4-a716-446655440000',
}

response = requests.get(
    'https://api.example.com/protected',
    headers=headers,
    timeout=10,
)

# Basic auth
response = requests.get(
    'https://api.example.com/data',
    auth=('username', 'password'),   # HTTPBasicAuth
    timeout=10,
)

# API key in header
response = requests.get(
    'https://api.example.com/data',
    headers={'X-API-Key': api_key},
    timeout=10,
)

# API key as query param
response = requests.get(
    'https://api.example.com/data',
    params={'api_key': api_key, 'limit': 100},  # ?api_key=...&limit=100
    timeout=10,
)

Session for Multiple Requests

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# Session reuses TCP connections — faster for multiple requests to same host
session = requests.Session()
session.headers.update({
    'Authorization': f'Bearer {access_token}',
    'Accept': 'application/json',
    'User-Agent': 'MyApp/1.0',
})

# Auto-retry on transient errors
retry_strategy = Retry(
    total=3,
    status_forcelist=[429, 500, 502, 503, 504],  # retry these status codes
    backoff_factor=1,          # wait 1s, 2s, 4s between retries
    raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount('https://', adapter)
session.mount('http://', adapter)

# All requests reuse auth + retry config
users = session.get('https://api.example.com/users', timeout=10).json()
product = session.get('https://api.example.com/products/1', timeout=10).json()

# Context manager (closes connections automatically)
with requests.Session() as session:
    session.headers['Authorization'] = f'Bearer {token}'
    data = session.get('https://api.example.com/data', timeout=10).json()

Validate JSON Response with Pydantic

import requests
from pydantic import BaseModel, ValidationError
from typing import Optional

class User(BaseModel):
    id: int
    name: str
    email: str
    role: str = 'viewer'   # default
    created_at: Optional[str] = None

def get_user(user_id: int) -> User:
    response = requests.get(
        f'https://api.example.com/users/{user_id}',
        headers={'Authorization': f'Bearer {token}'},
        timeout=10,
    )
    response.raise_for_status()

    try:
        return User.model_validate(response.json())
    except ValidationError as e:
        print(f"API returned unexpected shape: {e}")
        raise

# Paginated response
class PaginatedUsers(BaseModel):
    data: list[User]
    total: int
    page: int
    per_page: int

def list_users(page: int = 1) -> PaginatedUsers:
    response = requests.get(
        'https://api.example.com/users',
        params={'page': page, 'per_page': 20},
        timeout=10,
    )
    response.raise_for_status()
    return PaginatedUsers.model_validate(response.json())

Upload and Download JSON Files

import requests
import json

# Upload a JSON file as multipart/form-data
with open('data.json', 'rb') as f:
    response = requests.post(
        'https://api.example.com/upload',
        files={'file': ('data.json', f, 'application/json')},
        headers={'Authorization': f'Bearer {token}'},
        timeout=30,
    )
response.raise_for_status()

# Upload JSON as raw body (when API expects raw JSON, not form-data)
with open('data.json') as f:
    data = json.load(f)

response = requests.post(
    'https://api.example.com/bulk',
    json=data,  # sends file contents as JSON body
    timeout=30,
)

# Download large JSON file with streaming
response = requests.get(
    'https://api.example.com/export/users.json',
    stream=True,
    timeout=60,
)
response.raise_for_status()

with open('users.json', 'wb') as f:
    for chunk in response.iter_content(chunk_size=8192):
        f.write(chunk)

Advanced Patterns: OAuth, Pagination, and Rate Limiting

import requests
import time

# Automatic pagination — fetch all pages
def fetch_all_pages(url: str, headers: dict) -> list:
    all_items = []
    page = 1
    while True:
        response = requests.get(url, params={'page': page, 'per_page': 100},
                                headers=headers, timeout=10)
        response.raise_for_status()
        data = response.json()
        items = data.get('data') or data.get('results') or data  # handle varying shapes
        if not items:
            break
        all_items.extend(items)
        if not data.get('next'):  # cursor or next URL missing = last page
            break
        page += 1
    return all_items

# Handle rate limits (429 Too Many Requests)
def request_with_backoff(url: str, headers: dict, max_retries: int = 5) -> dict:
    for attempt in range(max_retries):
        response = requests.get(url, headers=headers, timeout=10)
        if response.status_code == 429:
            retry_after = int(response.headers.get('Retry-After', 2 ** attempt))
            print(f"Rate limited. Waiting {retry_after}s...")
            time.sleep(retry_after)
            continue
        response.raise_for_status()
        return response.json()
    raise RuntimeError(f"Failed after {max_retries} attempts")

FAQ

How do I parse a JSON response with Python requests?

Call response.json() on the response object; it internally calls json.loads(response.text) and raises ValueError if the content is not JSON. Always call response.raise_for_status() first to check for HTTP errors before parsing. For safety, also check response.headers.get('Content-Type', '').startswith('application/json') before parsing — some APIs return error HTML with a 200 status code. The response.json() method also respects the charset declared in the Content-Type header, making it slightly more correct than manually calling json.loads(response.text) for non-UTF-8 encoded responses.

How do I send JSON data with Python requests?

Use the json= parameter in requests.post()/put()/patch(): requests.post(url, json=payload) — this automatically serializes the dict to JSON and sets Content-Type: application/json. Do NOT use data= with json.dumps() — this sends the data as form-encoded and won't set the Content-Type header correctly. Use data= only for form-encoded data (like HTML form submissions). The json= parameter is the correct and idiomatic way to send JSON in requests.

What is the difference between requests.get().json() and json.loads(response.text)?

response.json() also considers the response encoding (charset from Content-Type header) when decoding bytes; json.loads(response.text) always uses the text as-is after requests decodes the bytes. They are functionally equivalent for UTF-8 responses, which is the vast majority of JSON APIs. response.json() is slightly more correct because it handles non-UTF-8 encodings declared in Content-Type. In practice, use response.json() — it is the idiomatic choice and handles edge cases transparently.

Should I always call raise_for_status() before response.json()?

Yes. raise_for_status() raises HTTPError for 4xx/5xx responses; 2xx and 3xx responses pass through silently. Without it, a 404 or 500 response body will parse as JSON if the API returns error details in the body — and you will accidentally treat an error body as success data. The correct pattern is: response.raise_for_status() then response.json(). Some APIs return structured JSON error bodies ({"error": "not found"}) on 4xx responses; you can catch HTTPError and then call response.json() inside the except block to parse those error details.

How do I set a timeout in requests?

Pass timeout=seconds or timeout=(connect, read) tuple. Connect timeout is how long to wait for a TCP connection to be established. Read timeout is how long to wait between bytes of a response — not the total response time. Always set a timeout — without it, requests hangs forever on slow or unresponsive servers. The requests documentation recommends timeout=(3.05, 27): 3.05 seconds to connect (slightly above 3 to avoid race conditions with TCP retransmission) and 27 seconds to read. For large file downloads, set a higher read timeout.

How do I add authentication to a requests JSON API call?

Bearer token: headers={'Authorization': f'Bearer {token}'}. Basic auth: auth=('user', 'pass') or auth=HTTPBasicAuth('user', 'pass'). API key header: headers={'X-API-Key': key}. API key query param: params={'api_key': key}. For OAuth 2.0, use the requests-oauthlib library. Set headers on a Session object to avoid repeating them on every call: session.headers.update({'Authorization': f'Bearer {token}'}). Never hardcode tokens in source code — use environment variables or a secrets manager.

What is a requests.Session and when should I use it?

Session reuses TCP connections (connection pooling) for faster successive requests to the same host. It also stores default headers, auth, and cookies across requests so you don't have to repeat them. Use Session when making 3 or more requests to the same API. A context manager (with requests.Session() as s:) closes all connections on exit, preventing connection leaks. Connection reuse reduces latency by approximately 30–50ms per request versus creating new connections. You can also mount retry adapters on a Session to add automatic retry logic for transient failures.

How do I validate JSON responses from requests with Pydantic?

Call User.model_validate(response.json()) — Pydantic validates the dict structure and types at runtime. It catches missing required fields, wrong types, and invalid enum values, and returns a typed Python object with IDE autocomplete. Wrap in try/except ValidationError to handle API shape changes gracefully. Pydantic is safer than TypedDict (compile-time only) for production API clients because it validates at runtime when the actual API response arrives. Define a model for the full response shape including nested objects and lists for complete safety.

Definitions

response.json(): requests method that decodes the response body as JSON; calls json.loads() internally; raises ValueError if the body is not valid JSON; respects the charset from the Content-Type header when decoding bytes.
raise_for_status(): requests method that raises HTTPError if the response status code is 4xx or 5xx; 2xx and 3xx responses pass through silently; essential before parsing the response body to avoid treating error responses as success data.
requests.Session: a requests object that persists settings (headers, auth, cookies) and TCP connections across multiple requests; provides ~30–50ms speedup per request for the same host via connection reuse; supports mounting retry adapters.
timeout tuple: (connect_timeout, read_timeout) passed to requests methods; connect timeout is how long to wait for a TCP handshake; read timeout is how long to wait between bytes of a response; always required to prevent hanging on unresponsive servers.
json= parameter: requests kwarg for POST/PUT/PATCH methods that serializes a Python dict to JSON and sets Content-Type: application/json automatically; equivalent to manually calling json.dumps() and setting the header, but cleaner and less error-prone.