🌐

requests Internals — What requests.get() Actually Does

The call chain from Session to PreparedRequest to HTTPAdapter to urllib3

response = requests.get('https://api.example.com/data')

What happens behind this one line:

Call Chain

requests.get(url)
  → Session().request()   # Temporary Session
    → PreparedRequest     # Assemble headers, cookies, encoding
      → HTTPAdapter.send() # Select adapter (https:// → HTTPS)
        → urllib3.PoolManager # Get connection from pool
          → socket.connect()  # Actual TCP

Why Know This

Calling requests.get() 100 times in a loop creates 100 new TCP connections. Using Session directly reuses the connection pool.

urllib3 — The Engine

requests delegates all HTTP protocol handling to urllib3. Connection pooling, Keep-Alive, retries, SSL — all urllib3 level. requests just adds a "user-friendly API" on top.

Fine Control with HTTPAdapter

Control pool size, retry count/interval, max concurrent connections.

Key Points

1

requests.get() internally creates a temporary Session

2

PreparedRequest assembles headers/cookies/encoding

3

HTTPAdapter → urllib3.PoolManager → socket.connect() for actual TCP

4

Creating Session directly reuses connection pool for better performance

Use Cases

API clients — Session + HTTPAdapter for retry/timeout/pool config Crawlers — Session maintains cookies/headers across multiple page requests