Python Tutorial / Understand HTTP APIs, JSON Responses and Request Failures

An HTTP API accepts requests and returns a status and response data. A Python client should separately check request success, network wait limits, response size and the structure of decoded JSON.

Separate the protocol, address and data format

HTTP is the request–response protocol; a URL identifies the resource; JSON is one possible body format. A successful status does not prove the returned data meets the application’s expectations. External services additionally require their documented HTTPS, authentication and request-limit rules.

This exercise uses a local server so that the response is under your control. It demonstrates the client workflow rather than a production server deployment.

For the relevant APIs and platform requirements, see Understand HTTP APIs, JSON Responses and Request Failures documentation.

Prepare a local response and server

In a fresh folder, save expenses.json with this complete content:

[{"id":"e1","name":"coffee","amount":3500},{"id":"e2","name":"lunch","amount":8000}]

In the first terminal, enter that folder and run:

python -m http.server 8000 --bind 127.0.0.1

Binding to 127.0.0.1 restricts the server to the same computer. If port 8000 is occupied, choose another port and update the request URL to match.

Run the complete client

Save the following as fetch_expenses.py:

import json
import sys
from urllib.request import urlopen
from urllib.error import HTTPError, URLError

url = sys.argv[1]
try:
    with urlopen(url, timeout=5) as response:
        if response.status != 200:
            raise ValueError("unexpected status")
        raw = response.read(1_000_001)
        if len(raw) > 1_000_000:
            raise ValueError("response too large")
        data = json.loads(raw.decode("utf-8"))
except (HTTPError, URLError, TimeoutError, UnicodeError, ValueError) as error:
    raise SystemExit(f"request failed: {type(error).__name__}")
if not isinstance(data, list):
    raise SystemExit("expected a list")
total = 0
for row in data:
    if not isinstance(row, dict) or type(row.get("amount")) is not int or row["amount"] < 0:
        raise SystemExit("invalid record")
    total += row["amount"]
print("records:", len(data))
print("total:", total)

In a second terminal using the project environment, run:

python fetch_expenses.py http://127.0.0.1:8000/expenses.json

Expect records: 2 and total: 11500. Stop the local server with Ctrl+C when finished. The client needs only the standard library.

Distinguish network failures from invalid data

A missing path can produce HTTPError; connection failures can produce URLError. A timeout bounds individual waits rather than guaranteeing one total end-to-end deadline. The example also stops for an oversized response, invalid UTF-8, malformed JSON or an invalid record.

Retries need bounded counts and delays plus an understanding of request side effects. Repeating a payment request indiscriminately can duplicate work. Manage authentication tokens according to the service’s policy and keep them out of code, URLs and logs.

1. Request a misspelled filename to trigger an HTTP failure.
2. Change an amount to a string and check data rejection.
3. Stop the server and distinguish connection failure from a successful response.

Restore valid input before comparing results. The API collection project adds transactional storage to this request-and-validation flow.