vsay

Writing

The Journey of an HTTP Request

what happens after typing google.com into a browser and pressing Enter

12 min read

The Journey of an HTTP Request

This blog follows a laptop running Linux making an HTTP request to http://google.com, starting with DHCP network configuration, then DNS resolution, routing and ARP, sockets and ports, TCP connection establishment, IP packet creation, link-layer framing, router forwarding, NAT, the journey across the Internet, the HTTP request and response, and finally the browser rendering the page.

Before the laptop is able to make a request to google.com, or even communicate with another device, it needs a network configuration; this is where DHCP gets involved in the chain.

The Dynamic Host Configuration Protocol (DHCP) is a protocol used to assign IP addresses and other communication parameters to devices connected to the network.

The router implements the DHCP server, while the laptop implements the DHCP client.

At the beginning, the laptop does not have an IP address yet, so the first DHCP messages are broadcast on the local network.

A simplified DHCP request starts with:

Source IP        → 0.0.0.0
Destination IP   → 255.255.255.255
Source UDP port  → 68
Destination port → 67

The router replies with the network configuration the laptop needs, such as an IP address, subnet mask, default gateway, and DNS server.

Once this configuration is in place, the laptop has the basic information, for example:

IP address        → 192.168.1.5
Subnet            → 192.168.1.0/24
Default gateway   → 192.168.1.1
DNS server        → 192.168.1.1

Here:

  • 192.168.1.5 is the laptop’s local IP address.
  • 192.168.1.0/24 represents the laptop’s local network.
  • 192.168.1.1 is the default gateway.
  • 192.168.1.1 is also the DNS server in this example.

The default gateway is used when the destination is outside the local network. The router then forwards those packets toward other networks, potentially across the Internet.

Now the laptop has the necessary network configuration required to communicate using IP addresses.

But google.com is a human-readable string, not an IP address, and this is where DNS enters the chain.

The Domain Name System (DNS) maps human-readable names into IP addresses.

The laptop may first check things such as the browser cache and local hosts configuration.

If the IP address is not already known, the laptop needs to ask its configured DNS resolver:

192.168.1.1

The DNS query is sent using UDP to destination port 53.

But before the laptop can send anything to 192.168.1.1, it first needs to know how to reach that address on the local network.

This is where the routing table and ARP get involved.

Routing Table

A routing table tells Linux where to send network traffic based on its destination.

The laptop has an entry similar to:

192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.5

which tells the kernel that 192.168.1.0/24 is directly reachable through wlan0.

It also has:

default via 192.168.1.1 dev wlan0

which tells Linux to use 192.168.1.1 as the default gateway when the destination is outside the local network.

The DNS server is:

192.168.1.1

which belongs to:

192.168.1.0/24

so Linux knows that the DNS server is directly reachable through wlan0.

The laptop knows the DNS server’s IP address, but it still needs a link-layer destination address before it can send data on the local network.

This is where ARP gets involved.

ARP

The Address Resolution Protocol (ARP) is used to find the MAC address associated with an IPv4 address on the local network.

The laptop needs to know which MAC address belongs to:

192.168.1.1

It sends an ARP request onto the local network.

The request is broadcast because the laptop does not know which device owns the address yet.

The router replies with its MAC address, for example:

192.168.1.1 is at aa:bb:cc:dd:ee:ff

Linux stores this information in its neighbor table.

It can be viewed with:

ip neigh

Now the laptop knows:

DNS server IP   → 192.168.1.1
DNS server MAC  → aa:bb:cc:dd:ee:ff
Outgoing device → wlan0

The DNS query can now leave the laptop.

Conceptually:

Link-layer frame
└── IP packet
    └── UDP datagram
        └── DNS query for google.com

The IP destination is:

192.168.1.1

and the UDP destination port is:

53

Since the DNS server is the router in this example, the frame is sent to the router’s MAC address.

If there is a wireless access point or switch between the laptop and router, it forwards the local traffic toward the router.

The router receives the DNS query, the DNS resolver processes it, and a DNS response comes back.

Assume the result is:

google.com → 203.0.113.10

203.0.113.10 is only an example.

Now the laptop has Google’s IP address.

The next question is:

How should packets reach 203.0.113.10?

Linux checks the routing table again.

The destination:

203.0.113.10

is not inside:

192.168.1.0/24

therefore Linux selects:

default via 192.168.1.1 dev wlan0

So:

Final destination → 203.0.113.10
Next hop           → 192.168.1.1
Interface          → wlan0

The MAC address of 192.168.1.1 is already known from ARP.

Now Linux has enough information to start sending traffic toward Google.

TCP/IP 4-Layer Model

The TCP/IP model used here has four layers:

Application
    HTTP, DNS

Transport
    TCP, UDP

Internet
    IP

Network Access
    Ethernet, Wi-Fi, ARP

For the HTTP request in this example, the path looks roughly like:

HTTP
    ↓
TCP
    ↓
IP
    ↓
Wi-Fi

The browser does not build IP packets or Wi-Fi frames itself.

The browser uses sockets, while Linux handles TCP, IP, routing, ARP, and the network interface.

Transport Layer

The browser needs a way to communicate with the remote service.

Linux exposes this through a socket.

For this request, the browser opens a TCP socket and connects it to:

203.0.113.10:80

Here:

203.0.113.10 → destination IP
80           → destination port

Port 80 is used for HTTP.

Linux also chooses a temporary source port for the laptop, for example:

192.168.1.5:52734 → 203.0.113.10:80

So there are two endpoints:

Laptop
192.168.1.5:52734

Google
203.0.113.10:80

Before HTTP data can be sent over TCP, TCP establishes the connection with the three-way handshake:

Laptop                         Google

SYN          -------------------->

             <-------------------- SYN-ACK

ACK          -------------------->

The details of sequence numbers, retransmission, flow control and congestion control are outside this journey.

The important point is that the first SYN is already a TCP segment that needs to leave the laptop.

That TCP segment moves down to the Internet layer.

Internet Layer

Linux puts the TCP segment inside an IP packet.

Conceptually:

IP packet
└── TCP segment

The packet contains information similar to:

Source IP       → 192.168.1.5
Destination IP  → 203.0.113.10
Protocol        → TCP

Inside it is the TCP segment:

Source port       → 52734
Destination port  → 80

So the structure looks like:

IP packet
├── Source IP       → 192.168.1.5
├── Destination IP  → 203.0.113.10
│
└── TCP segment
    ├── Source port       → 52734
    ├── Destination port  → 80
    └── SYN

The IP address and port solve different problems.

IP address
    → identifies where the packet is going

Port
    → identifies which service or connection should receive it

Linux already knows the route:

203.0.113.10
        ↓
default via 192.168.1.1
        ↓
wlan0

The IP packet still has Google’s IP address as its destination.

The router is only the next hop.

Now the packet needs to cross the local network.

This is where the Network Access layer gets involved.

Network Access Layer

Linux now has an IP packet:

IP packet
├── Source IP       → 192.168.1.5
├── Destination IP  → 203.0.113.10
│
└── TCP segment
    ├── Source port       → 52734
    ├── Destination port  → 80
    └── SYN

The routing table already told Linux that the next hop is:

192.168.1.1

and ARP already told Linux its MAC address:

192.168.1.1 → aa:bb:cc:dd:ee:ff

Now Linux can prepare the packet for the local network.

The IP packet is placed inside a link-layer frame:

Link-layer frame
└── IP packet
    └── TCP segment

Wrapping data from one TCP/IP layer inside the data of the layer below it is called encapsulation.

At this point:

Link-layer destination → router
IP destination         → Google

The router is only the next hop.

Google is still the final destination.

Since the laptop uses wlan0, the packet is sent through the wireless network interface.

If there is a wireless access point between the laptop and router, it carries the local traffic toward the router.

The first TCP SYN can now leave the laptop.

The Router Receives the Packet

The router receives the frame and removes the local link-layer information.

This exposes the IP packet inside:

Link-layer frame
        ↓
IP packet
    └── TCP segment

Removing the outer layer is called decapsulation.

The router now checks the destination IP address:

203.0.113.10

and looks for that destination in its forwarding table.

A forwarding table tells the router which next hop should be used for a destination.

For example:

Destination         Next hop

203.0.113.0/24      Router B
0.0.0.0/0           Router C

Since:

203.0.113.10

belongs to:

203.0.113.0/24

the router chooses Router B.

The router then prepares the IP packet for the next link by putting it inside a new link-layer frame.

So:

Laptop → Router A

Link-layer destination → Router A
IP destination         → Google

becomes:

Router A → Router B

Link-layer destination → Router B
IP destination         → Google

The link-layer information changes from one link to another.

The destination IP remains Google.

The router also decreases the packet’s TTL before forwarding it:

Laptop sends → TTL 64
Router A     → TTL 63
Router B     → TTL 62

TTL prevents packets from circulating forever if a routing loop happens.

NAT

The packet currently has:

Source IP       → 192.168.1.5
Destination IP  → 203.0.113.10

But 192.168.1.5 is a private IP address and cannot be used directly on the public Internet.

Before forwarding the packet outside the local network, the router changes the source IP address to its public IP address.

It usually also translates the source port.

For example:

Before NAT:

192.168.1.5:52734 → 203.0.113.10:80

becomes:

After NAT:

Public-IP:62001 → 203.0.113.10:80

The router keeps this mapping.

Later, when the response comes back to:

Public-IP:62001

the router knows that the traffic actually belongs to:

192.168.1.5:52734

This is called Network Address Translation (NAT).

It also allows several devices on the local network to share one public IPv4 address.

Crossing the Internet

The Internet is a network of networks.

After leaving the local router, the packet may pass through the ISP, other networks, routers, fiber-optic cables, and other networking infrastructure before reaching Google’s network.

At this point it is almost like magic:

Laptop
    ↓
Router
    ↓
Internet
    ↓
Google

There is no single path that can describe every request.

Different networks have different infrastructure, different routes, and different connections between them.

For this journey, the useful thing to know is that routers keep forwarding the IP packet toward its destination until it eventually reaches Google’s network.

The TCP Connection Reaches Google

The TCP SYN eventually reaches the server side.

The destination is:

203.0.113.10:80

The operating system on the server side sees that the packet is for TCP port 80, where the HTTP service is listening.

Google sends back a SYN-ACK.

The response travels back through the network toward the laptop.

The laptop replies with the final ACK.

Now the TCP connection is established.

The browser can start sending HTTP data.

Sending the HTTP Request

Now the browser can finally send the HTTP request.

Conceptually the request is similar to:

GET / HTTP/1.1
Host: google.com

A real browser sends more headers, but the important meaning here is:

GET /
from google.com

The browser writes the HTTP request through the TCP socket.

The data moves down the same layers:

HTTP
    ↓
TCP
    ↓
IP
    ↓
Network Access

At the laptop the structure is now:

Link-layer frame
└── IP packet
    └── TCP data
        └── HTTP request

The frame reaches the router.

The router decapsulates the IP packet, forwards it toward the next hop, and the process continues across the Internet.

Eventually the request reaches Google’s infrastructure.

Google Processes the Request

At some point the HTTP service receives the request and creates an HTTP response.

An HTTP response contains three important parts:

status
headers
body

For example:

HTTP/1.1 200 OK
Content-Type: text/html

<html>
    ...
</html>

Here:

200 OK              → status
Content-Type        → response header
<html> ... </html>  → response body

The exact response can be different, but this is enough for this journey.

The response is sent back through the existing TCP connection.

The Response Travels Back

The response moves back through:

HTTP
    ↓
TCP
    ↓
IP
    ↓
Network Access

Routers keep forwarding the packets toward the laptop.

Eventually the response reaches the local router’s public side.

The destination at that point may look like:

Public-IP:62001

The router checks the NAT mapping created earlier:

Public-IP:62001
        ↓
192.168.1.5:52734

The router translates the destination back to the laptop’s private address and port.

The packet can then be sent onto the local network toward:

192.168.1.5

The laptop receives the link-layer frame through its network interface.

Linux then removes the layers in the opposite direction:

Link-layer frame
        ↓
IP packet
        ↓
TCP data
        ↓
HTTP response

This is decapsulation on the receiving side.

Linux identifies the TCP connection using the IP addresses and ports and delivers the data to the correct socket:

192.168.1.5:52734
        ↕
203.0.113.10:80

The browser reads the HTTP response from that socket.

The Browser Builds the Page

Receiving the HTML does not mean the page is finished.

The browser parses the HTML and builds the DOM.

The HTML may reference other resources:

CSS
JavaScript
images
fonts
API endpoints

Those resources can create more HTTP requests.

Some may reuse an existing connection.

Others may require another DNS lookup or another connection if they come from a different hostname.

The browser parses CSS, executes JavaScript, calculates layout, paints the page, and composites the final result.

A simplified view is:

HTML
    ↓
DOM

CSS
    ↓
styles

DOM + styles
    ↓
layout
    ↓
paint
    ↓
compositing
    ↓
page

At this point the page becomes visible.

The Whole Chain

The complete journey can now be reduced to:

Laptop gets network configuration through DHCP
        ↓
DHCP uses UDP 68/67 and local broadcast
        ↓
Laptop needs to resolve google.com
        ↓
Linux checks the route to the DNS server
        ↓
ARP resolves the DNS server/router MAC address
        ↓
DNS query is sent using UDP port 53
        ↓
google.com is resolved to an IP address
        ↓
Linux checks the routing table for Google
        ↓
Default gateway is selected
        ↓
Browser opens a TCP socket
        ↓
Linux selects a source port
        ↓
TCP handshake
        ↓
TCP segment
        ↓
IP packet
        ↓
Link-layer frame
        ↓
Router
        ↓
Router forwarding table
        ↓
NAT
        ↓
Internet: network of networks
        ↓
Google
        ↓
HTTP request
        ↓
HTTP response
        ↓
Packets travel back
        ↓
Reverse NAT
        ↓
Laptop receives the frame
        ↓
Decapsulation
        ↓
Linux delivers data to the correct socket
        ↓
Browser parses the response
        ↓
Page is rendered

A request that looks like:

google.com

actually involves several different systems and protocols working together.

DHCP gives the laptop its network configuration.

DNS maps the hostname to an IP address.

The routing table tells Linux where traffic should go.

ARP maps a local IPv4 address to a MAC address.

Sockets and ports connect the browser to TCP.

TCP provides the connection.

IP carries packets between networks.

The Network Access layer moves those packets across each local link.

Routers use forwarding tables to move packets toward the destination.

NAT translates between the laptop’s private address and the router’s public address.

HTTP carries the request and response.

Finally, the browser turns the response into the page shown on the screen.