Computer Networks

Communication Models

OSI Seven-Layer Model

Application layer: provides network services for user applications. Common application layer protocols include HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), SMTP (Simple Mail Transfer Protocol), DNS (Domain Name System), etc. Various network applications used directly by users (such as browsers and email clients) all run at the application layer.
Presentation layer: handles the representation and conversion of data, ensuring that different systems can correctly understand and process data. It includes functions such as data encryption, decryption, compression, decompression, and data format conversion (such as conversion between ASCII and EBCDIC).
Session layer: responsible for establishing, managing, and terminating sessions. It allows applications on different hosts to conduct sessions and provides session synchronization and recovery mechanisms.
Transport layer: provides end-to-end reliable or unreliable communication services. There are two main protocols: Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).
Network layer: responsible for transmitting data packets from the source node to the destination node, handling routing and addressing in the network. It uses IP addresses to identify devices in the network and forwards data packets between different networks through routers. Common network layer protocols are IPv4 and IPv6.
Data link layer: encapsulates the bit stream received from the physical layer into frames, handling frame transmission and error detection. It is divided into the Logical Link Control (LLC) sublayer and the Media Access Control (MAC) sublayer. The MAC sublayer controls access to the physical medium and identifies devices in the network via MAC addresses; the LLC sublayer provides error control and flow control.
Physical layer: responsible for transmitting the bit stream, defining the electrical, mechanical, and functional characteristics of physical media (such as cables, optical fibers, wireless), including voltage, signal encoding, connector specifications, etc.

Data Transmission Process

The application layer message is delivered to the transport layer
In the simplest case, the transport layer receives the message and attaches additional information; this header will be used by the transport layer on the receiving end
The application layer message together with the transport layer header information forms the transport layer segment. The additional information may include: information allowing the receiving transport layer to deliver the message up to the appropriate application, as well as error-detection bit information. This information lets the receiving end determine whether the bits in the message have been altered on the way
The transport layer then passes the segment to the network layer, which adds network layer header information such as source and destination system addresses, generating the network layer datagram
The network layer datagram is then passed to the link layer, where the sender's MAC address and the receiver's MAC address are added to the packet before it is encapsulated into a data frame
At the physical layer, the data frame is encapsulated into a bit stream and then transmitted to the peer through the transmission medium
The peer then unwraps the encapsulation step by step to obtain the transmitted data

TCP/IP Five-Layer Model

The TCP/IP five-layer protocol is a simplification of the OSI seven-layer protocol in practical applications. It merges the session layer and presentation layer of the OSI model into the application layer, forming a five-layer structure

Application layer: contains the functions of the application layer, presentation layer, and session layer of the OSI model, providing network interfaces for user applications and supporting various network applications.
Transport layer: provides end-to-end communication services, including the TCP and UDP protocols, ensuring reliable or fast data transmission.
Network layer: its main function is routing and addressing of data packets, using the IP protocol to enable communication between different networks.
Data link layer: same as the data link layer of the OSI model, encapsulating the bit stream into frames, handling frame transmission and error detection, and controlling access to the physical medium.
Physical layer: same function as the physical layer of the OSI model, responsible for transmitting the bit stream and handling physical media and signal transmission.

TCP/IP Four-Layer Model

The TCP/IP four-layer protocol is another simplified network model. Based on the five-layer model, it merges the physical layer and data link layer into the network interface layer, forming a four-layer structure

Application layer: provides network services for user applications, containing various application layer protocols such as HTTP, FTP, SMTP, etc., supporting users in using various network applications.
Transport layer: provides end-to-end communication services, achieving reliable or unreliable data transmission through the TCP or UDP protocol, ensuring data transmission between applications.
Internet layer: its main function is routing and addressing of data packets, using the IP protocol to enable communication between different networks and ensuring that data packets can be transmitted from the source host to the destination host.
Network Interface Layer: responsible for encapsulating IP packets into frames suitable for transmission over the physical network and transmitting them through the physical medium. It covers the functions of the physical layer and data link layer of the OSI model, handling different types of network interfaces and physical media.

Differences

OSI seven-layer model: mainly used for theoretical research and teaching, providing a standard reference model for the design and analysis of network protocols. However, due to its excessive complexity, it is rarely used directly in real networks.
TCP/IP five-layer model: widely used in practical network development and teaching. It both retains the layered thinking of the OSI model and simplifies it, making it easy to understand and implement.
TCP/IP four-layer model: the network model actually used by the Internet. It places more emphasis on network practicality and efficiency, and is widely used in various network devices and operating systems.

DNS (Domain Name System) Protocol

DNS is used to map domain names to different IP addresses. It is generally divided into the root domain ., top-level domains com, cn, second-level domains cctv.com, and so on

DNS Query Process

Recursive Query

The local DNS requests the root name server
The root name server requests the top-level name server
The top-level name server requests the authoritative name server
The authoritative name server sends the IP address corresponding to the domain name to the top-level name server
The top-level name server returns it to the root server
The root server returns it to the local DNS

Iterative Query

Similar to recursive query, but the root name server and top-level name server only return the address of the server for the next stage. For example, the root name server returns the address of the top-level name server, and the local DNS itself requests the top-level name server to query the authoritative name server address

Domain Name Caching

Browser cache: for performance reasons, the browser caches DNS itself, so it does not need to query the operating system every time
Operating system cache: the operating system internally caches a domain name/IP mapping table, so DNS does not need to be re-queried every time

Domain Name Resolution Process in the Browser

Search the browser's cached DNS for whether the corresponding IP is stored; if so, return it directly
Request the local DNS; the local DNS checks whether the corresponding IP is cached; if so, return it to the browser
Perform a DNS iterative query
Cache the query result in the operating system, and also send it to the browser
The browser receives the resolution result from the local DNS and caches it in the browser's DNS resolution table

UDP (User Datagram Protocol)

No matter how long a message the application layer hands to UDP, it sends it all, one message at a time, without splitting or merging

UDP provides no complex control mechanisms, using IP to provide connectionless communication services
If packet loss occurs during transmission, UDP is not responsible for retransmission
When packets arrive out of order, UDP has no function to correct it.
Moreover, it is a mechanism that sends data from the application to the network immediately, as is, at the moment it is received. Even when network congestion occurs, UDP cannot perform flow control or other behaviors to avoid network congestion

TCP (Transmission Control Protocol)

TCP fully implements various control functions during data transmission: it can perform retransmission control when packets are lost, and can also perform order control on packets that arrive out of order. None of these exist in UDP.
In addition, as a connection-oriented protocol, TCP only sends data when it confirms that the communication peer exists, thereby controlling the waste of communication traffic.
With these mechanisms of TCP, highly reliable communication can be achieved even on a connectionless network like IP ( mainly implemented through mechanisms such as checksum, sequence numbers, acknowledgment, retransmission control, connection management, and window control )

Three-Way Handshake

1. client to server (SYN)

Process: the client sends a SYN segment ( SYN=1 ), carrying the initial sequence number ISN(c) ( Sequence Number ).
Purpose:
- Initiate the connection: the client expresses to the server its intention to establish a connection.
- Synchronize sequence numbers: tells the server the starting number of subsequent packets, used for reassembly and acknowledgment.
Consequences if missing:
- The server cannot perceive the client's connection request, and the connection cannot be established.
- The client cannot send data (has not entered the connected state).

2. server answer client (SYN-ACK)

Process: the server returns a SYN-ACK segment ( SYN=1, ACK=1 ), containing its own initial sequence number ISN(s), and acknowledges the client's SYN segment via ACK=ISN(c)+1.
Purpose:
- Accept the connection: the server agrees to connect and sends its own sequence number.
- Confirm the client's capability: by replying with ACK, it proves the server can receive data normally.
Consequences if missing:
- The client receives no response and will trigger timeout retransmission of the SYN segment, which may lead to a SYN flood attack (server resources exhausted by incomplete half-open connections).
- The two sides' sequence numbers cannot be synchronized, and subsequent communication becomes chaotic.

3. client answer server (ACK)

Process: the client sends an ACK segment ( ACK=1 ) with acknowledgment number ISN(s)+1, completing the handshake.
Purpose:
- Confirm the server's capability: the client proves it can receive the server's data.
- Prevent interference from historical connections: if the client receives an old SYN-ACK segment (such as a retransmission caused by network delay), it will ignore it or send an RST to terminate the invalid connection.
Consequences if missing:
- The server keeps waiting for ACK, occupying resources to maintain a half-open connection, which may lead to resource exhaustion.
- The server cannot confirm the client is ready and may mistakenly send data, causing packet loss or duplicate transmission.

Why Must It Be Three Handshakes?

The Risk of Two Handshakes

If an old SYN segment sent by the client reaches the server due to network delay, the server will mistake it for a new connection, directly allocate resources, and send SYN-ACK. At this point the client may already be closed, causing server resource waste (the "historical connection problem").

It cannot ensure that both sides' sending and receiving capabilities are normal (for example: the server has not confirmed that the client can receive data).

The Reliability of Three Handshakes

Through bidirectional confirmation (client → server → client → server), it ensures that both sides' sequence numbers are synchronized and that both have sending and receiving capabilities, avoiding resource waste and data chaos.

Summary

The three-way handshake solves the following problems through synchronizing sequence numbers and bidirectional capability verification:

Prevent invalid historical connections from occupying resources.
Ensure both sides' sending and receiving capabilities are normal.
Negotiate the initial sequence numbers, laying the foundation for reliable transmission.

Missing any step will make the connection unreliable, waste resources, or cause chaotic communication.

Four-Way Wave

1. client to server (FIN)

Process: the actively closing party (such as the client) sends a FIN segment (FIN=1) and enters the FIN_WAIT_1 state.
Purpose:
- Initiate the close: notify the other party (the passive closer) that no more data will be sent.
- Terminate the sending direction: close the client → server data transmission channel.
Consequences if missing:
- The passive closer cannot perceive the close request, and the connection keeps occupying resources.
- The client cannot terminate normally and may repeatedly send data.

2. server to client (ACK)

Process: after the passive closer (such as the server) receives the FIN, it returns an ACK segment (ACK=1) and enters the CLOSE_WAIT state. After the active closer receives the ACK, it enters the FIN_WAIT_2 state.
Purpose:
- Acknowledge the close request: tell the active party that the termination request has been received.
- Allow continued data reception: the passive party can still send remaining data (such as an unfinished response).
Consequences if missing:
- The active party stays blocked in the FIN_WAIT_1 state for a long time, and resources cannot be released.
- The passive party may mistakenly believe the connection is normal and keep sending data, causing packet loss.

3. client to server (FIN)

Process: after the passive closer finishes sending data, it sends a FIN segment (FIN=1) and enters the LAST_ACK state.
Purpose:
- Terminate the reverse channel: close the server → client data transmission channel.
- Request final acknowledgment: wait for the active party to confirm the connection termination.
Consequences if missing:
- The active party stays in the FIN_WAIT_2 state for a long time, unable to release connection resources.
- The client cannot perceive that the server has closed and may repeatedly send ACKs.

4. server to client (ACK)

Process: after the active closer receives the FIN, it sends an ACK segment (ACK=1), enters the TIME_WAIT state (waiting for 2MSL time), and finally closes the connection. The passive party closes immediately upon receiving the ACK.
Purpose:
- Confirm the termination request: ensure the passive party can safely release resources.
- Handle delayed segments: wait for old segments to die out via TIME_WAIT, preventing the new connection from receiving historical data.
Consequences if missing:
- The passive party retransmits the FIN segment because it did not receive the ACK, staying in the LAST_ACK state for a long time.
- The new connection may receive old data, causing data chaos.

Why Are Four Needed?

Independent closing of a full-duplex connection:

A TCP connection is full-duplex; each direction needs to be closed independently.
- The active party closes the sending channel (FIN) → the passive party acknowledges (ACK).
- The passive party closes the sending channel (FIN) → the active party acknowledges (ACK).
Prevent data loss:

Ensure that both sides have no remaining data to transmit before fully closing.

https

The HTTPS Workflow

Connection establishment and certificate verification: the client initiates an HTTPS request, and the server returns a digital certificate (containing the public key, issuing authority, validity period, and other information). The client verifies the legitimacy of the certificate (such as the trustworthiness of the issuing authority, domain name matching, validity period, etc.).
Key negotiation: after verification passes, the client generates a random symmetric key (session key), encrypts it with the server's public key, and sends it to the server. The server uses its private key to decrypt and obtain the session key.
Symmetric encrypted communication: both parties transmit data encrypted symmetrically based on the negotiated session key. Symmetric encryption algorithms (such as AES) are used for subsequent communication due to their efficiency, while asymmetric encryption (such as RSA) is only used for the initial key exchange.

HTTPS Authentication Core Flow

1. Establishing the Connection via TCP Three-Way Handshake

Purpose: provide a stable transmission channel for subsequent encrypted communication.
Flow:
1. The client sends a SYN request (synchronize sequence numbers).
2. The server returns SYN + ACK (synchronize + acknowledge).
3. The client sends ACK (final acknowledgment), completing the connection establishment.

2. Certificate Verification Stage

Core goal: verify the legitimacy of the server's identity to prevent man-in-the-middle attacks.
Detailed steps:
1. Server Hello: the server sends the SSL/TLS certificate (containing the public key, domain name, issuing authority, and other information).
2. Certificate chain verification:
  - Check whether the certificate is issued by a trusted CA (verify the integrity of the certificate chain).
  - Verify the validity of the signature through the CA root certificates built into the operating system/browser.
3. Validity checks:
  - Whether the certificate has expired.
  - Whether the certificate domain matches the accessed domain.
  - Query the certificate revocation list (CRL/OCSP) to confirm it has not been revoked.

3. Key Negotiation and Exchange

Purpose: generate a symmetric encryption session key to improve the efficiency of subsequent communication.
Flow:
1. Client generates a random key: generates a random key and encrypts it with the server's public key.
2. Server Key Exchange: the server decrypts with its private key to obtain the random key.
3. Handshake complete: both parties confirm the encryption algorithm and switch to encrypted communication mode.

4. Encrypted Data Transmission

Mechanism: use the negotiated symmetric key to encrypt all subsequent communication.
Features:
- Use high-efficiency symmetric encryption algorithms such as AES.
- Ensure data integrity through MAC (Message Authentication Code).

Appendix: Mutual Authentication Flow (Extended)

Applicable scenarios: scenarios with high security requirements such as enterprise-level applications.
Additional steps:
1. The server requires the client to provide a certificate.
2. The client sends the certificate and the server verifies it (similar to step 2).
3. Both parties hold each other's public key, achieving mutual identity authentication.

---
title: Mutual Authentication Flow Diagram
---
sequenceDiagram
    participant Client
    participant Server
    participant CA

    Client->>Server: Initiate HTTPS connection request
    Server->>Client: Send SSL certificate and certificate chain
    Client->>CA: Request verification of certificate validity
    CA-->>Client: Return verification result
    alt Certificate valid
        Client->>Client: Generate random session key
        Client->>Server: Send session key encrypted with public key
        Server->>Server: Decrypt with private key to obtain session key
        Client->>Server: Start encrypted communication
    else Certificate invalid
        Client->>Client: Terminate connection and warn
    end

SSL Encryption Flow

The client accesses the server via a URL to establish an SSL connection
After receiving the client request, the server transmits a copy of the website's supported certificate information (the certificate contains the public key) to the client
The client and server begin negotiating the security level of the SSL connection, that is, the level of information encryption
The client's browser establishes a session key according to the security level agreed by both parties, then encrypts the session key with the website's public key and transmits it to the website
The server uses its own private key to decrypt the session key
The server uses the session key to encrypt the communication with the client

The Difference Between SSL and TLS

SSL was released by Netscape; TLS was released after standardization by the IETF.

TLS is the successor of SSL, with enhanced security, fixed SSL vulnerabilities, and the introduction of new encryption methods and key exchange mechanisms

Optimizing HTTPS Request Time

Session resumption: reuse previously negotiated session keys via TLS Session Resumption (Session ID or Session Tickets), avoiding a full handshake.
OCSP Stapling: the server proactively provides certificate status information, reducing the latency of the client querying the CA for certificate revocation status.
Protocol and algorithm optimization:
- Adopt TLS 1.3 (reduces handshake RTT, supports 0-RTT mode).
- Prefer efficient cipher suites (such as AES-GCM, ChaCha20-Poly1305).
Certificate optimization: use ECC certificates instead of RSA, shortening key length and improving verification speed.
CDN and HTTP/2: shorten physical distance through CDN, and use HTTP/2 multiplexing to reduce connection overhead.

How HTTPS Ensures Security

Server security

Verify the authenticity of the server's identity through a digital certificate (issued by a CA authority), establishing a trusted communication foundation. During the SSL/TLS handshake phase, the server sends a CA certificate containing the public key, and the client uses the pre-installed CA root certificate to verify the validity of the certificate chain, including:
- Verify whether the certificate issuing authority is trusted
- Check the certificate validity period and domain name matching
- Verify certificate integrity (via digital signature)
Key negotiation mechanism
- Asymmetric encryption to transmit the session key: after the client generates a random symmetric key, it is transmitted encrypted with the server's public key (such as the RSA algorithm)
- Symmetric encryption to transmit business data: subsequent communication is encrypted with the negotiated symmetric key (such as AES-256); a per-session key ensures forward secrecy
Data integrity protection
- Digital signature: the sender's private key encrypts the message digest (such as SHA-256), and the receiver decrypts it with the public key and compares the digest value
- Message Authentication Code (HMAC): a MAC algorithm involving the session key generates an authentication tag, ensuring the data has not been tampered with

HSTS ( HTTP Strict-Transport-Security )

This is a web security policy mechanism that can force pages to use HTTPS

After using the HSTS policy, within the cache time, requests to the corresponding domain will be automatically converted to the secure HTTPS method. If the browser accesses via HTTP, the browser will internally redirect to HTTPS access via 307 and add the response header Non-Authoritative-Reason: HSTS. And the 200 response header field returns strict-transport-security: max-age=127800; [includeSubDomains;] [preload]

If the site the user visits is not a secure HTTPS website, the browser will indicate that the website is not secure and block access (if strict security is not enabled, the user can continue by ignoring the danger)

HSTS only takes effect when accessed via HTTPS and cached by the browser.

Before caching, if the user declares the use of the HTTP protocol, a 301 redirect jumps to HTTPS. At this point there is a risk of man-in-the-middle hijacking: the redirect information can be hijacked and the 301 redirect address modified, thereby attacking the user.

This is where the HSTS preload feature is needed: adding the specified domain to the preload list, so the browser can only access the site via https, avoiding man-in-the-middle attacks, and also saving one 301 redirect when accessing via http.

SSE (Server-Sent Events)

Strictly speaking, the HTTP protocol cannot make the server actively push information. However, there is a workaround: the server declares to the client that what follows is streaming information ( streaming ).

In other words, what is sent is not a one-time data packet, but a data stream that is sent continuously. At this point, the client will not close the connection and will keep waiting for the new data stream from the server; video playback is such an example. Essentially, this kind of communication completes one very long download in the form of streaming information.

SSE uses this mechanism to push information to the browser using stream information. It is based on the HTTP protocol and is supported by mainstream browsers.

Advantages

SSE uses the HTTP protocol, which existing server software all supports. WebSocket is an independent protocol.
SSE is lightweight and simple to use; the WebSocket protocol is relatively complex.
SSE supports automatic reconnection by default; WebSocket requires you to implement it yourself.
SSE is generally only used to transmit text; binary data needs to be encoded before transmission, while WebSocket supports transmitting binary data by default.
SSE supports custom message types for sending.

Client Example

// Check whether SSE is supported
if ('EventSource' in window) {
  const source = new EventSource(url, {
    // For cross-origin, use this option to make SSE requests carry cookies
    withCredentials: true,
  })

  // Check the current connection state
  switch (source.readyState) {
    case EventSource.CONNECTING:
      // EventSource.CONNECTING = 0
      // Means the connection has not been established yet, or it is reconnecting after a disconnection.
      break
    case EventSource.OPEN:
      // EventSource.OPEN = 1
      // Means the connection has been established and data can be received.
      break
    case EventSource.CLOSED:
      // EventSource.CLOSED = 2
      // Means the connection is broken and will not reconnect.
      break

    default:
      break
  }

  // When the connection is established
  source.onopen = function (event) {
    // Callback when the connection is established
  }
  // Another way to write the callback
  source.addEventListener('open', (event) => {
    // Callback when the connection is established
  }, false)

  // Handle messages
  source.onmessage = function (event) {
    const data = event.data
    // handle message
  }
  // Another way to handle messages; if the server sends custom events, you must use this form
  // and replace message with the custom event name
  source.addEventListener('message', (event) => {
    const data = event.data
    // handle message
  }, false)

  // Handle errors
  source.onerror = function (event) {
    // handle error event
  }
  // Another way to handle errors
  source.addEventListener('error', (event) => {
    // handle error event
  }, false)

  // Actively close the connection
  source.close()
}

Server Example

The response must carry the following response headers

Content-Type: text/event-stream  # The MIME type must be specified as event-stream
Cache-Control: no-cache
Connection: keep-alive

# Each piece of information sent consists of several messages, separated by \n\n.
# Each message consists of several lines internally, each line in the following format.

: This is a single-line comment; different message types can be distinguished by different prefixes, and ID uniquely identifies the data
: When the request is interrupted, the ID is sent to the server in the `Last-Event-ID` request header, used to rebuild the connection
id: msg1\n
data: some text\n\n

: Use event to declare that the following messages should trigger the foobar event; the data content is what is declared in data
event: foobar\n
data: another message\n
data: {"username": "bobby", "time": "02:33:48"}\n\n

: Tell the client to try reconnecting after the specified interval
retry: 10000\n

HTTP

Common Status Codes

1xx Means the Request Has Been Received and Processing Should Continue

100 (the client should continue sending the request; this is a temporary response): this temporary response is used to notify the client that part of its request has been received by the server and has not yet been rejected. The client should continue sending the rest of the request, or, if the request is already complete, ignore this response. The server must send a final response to the client after the request is complete
101: the server switches protocols according to the client's request, mainly used for WebSocket or HTTP2 upgrades

2xx Means the Request Has Been Successfully Received, Understood, and Accepted by the Server

200 (OK): the request has succeeded; the desired response headers or data body will be returned with this response
201 (Created): the request succeeded and the server created a new resource
202 (Accepted): the server has received the request but has not yet processed it
203 (Non-Authoritative Information): the server successfully processed the request, but the returned information may come from another source
204 (No Content): the server successfully processed the request but did not return any content
205 (Reset Content): the server successfully processed the request but did not return any content
206 (Partial Content): the server successfully processed a partial request

3xx Means Further Action Is Needed to Complete the Request

300 (Multiple Choices): for the request, the server can perform multiple operations. The server can choose one operation based on the requester (user agent), or provide a list of operations for the requester to choose from
301 (Moved Permanently): permanent redirect
302 (Found): temporary redirect, must not be cached
303 (See Other): temporary redirect, allows changing the method, must not be cached
304 (Cache hit): negotiated caching, telling the client that a cache exists and to use the cached data directly; only header information is returned for the page, without the content part
305 (Use Proxy): the requester can only access the requested web page through a proxy. If the server returns this response, it also indicates that the requester should use a proxy
307 (Temporary Redirect): temporary redirect, does not allow changing the method, must not be cached
308 (Permanent Redirect): permanent redirect, does not allow changing the method

4xx Means the Client Appears to Have Made an Error, Hindering the Server's Processing

400 (Bad Request): the server does not understand the syntax of the request
401 (Unauthorized): the request requires authentication. For web pages that require login, the server may return this response.
403 (Forbidden): the server refused the request
404 (Not Found): the server cannot find the requested web page
405 (Method Not Allowed): the method specified in the request is disabled
406 (Not Acceptable): the requested web page cannot be responded to with the requested content characteristics
407 (Proxy Authentication Required): this status code is similar to 401 (Unauthorized), but specifies that the requester should authorize the use of a proxy
408 (Request Timeout): a timeout occurred while the server was waiting for the request

5xx Means the Server Cannot Complete an Apparently Valid Request. These Status Codes Represent Errors or Abnormal States Occurring While the Server Processes the Request

500 (Internal Server Error): the server encountered an error and cannot complete the request
501 (Not Implemented): the server does not have the functionality to complete the request. For example, the server may return this code when it cannot recognize the request method
502 (Bad Gateway): the server, acting as a gateway or proxy, received an invalid response from the upstream server
503 (Service Unavailable): the server is currently unavailable (due to overload or downtime for maintenance)
504 (Gateway Timeout): the server, acting as a gateway or proxy, did not receive the request from the upstream server in time
505 (HTTP Version Not Supported): the server does not support the HTTP protocol version used in the request

Common Request Headers

Field name	Description	Example
`Accept`	The acceptable response content types (`Content-Types`)	`Accept: text/plain`
`Accept-Charset`	The acceptable character sets	`Accept-Charset: utf-8`
`Accept-Encoding`	The list of acceptable encoding methods	`Accept-Encoding: gzip, deflate`
`Accept-Language`	The list of acceptable natural languages for the response content	`Accept-Language: en-US`
`Authorization`	Authentication information used for HTTP authentication	`Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==`
`Cache-Control`	Used to specify directives that all caching mechanisms in this request/response chain must obey	`Cache-Control: no-cache`
`Connection`	The connection type the browser prefers to use	`Connection: keep-alive`
`Cookie`	An HTTP Cookie sent by the server via Set-Cookie	`Cookie: $Version=1; Skin=new;`
`Content-Length`	The length of the request body expressed as an octet array (8-bit bytes)	`Content-Length: 348`
`Content-Type`	The multimedia type of the request body	`Content-Type: application/x-www-form-urlencoded`
`Date`	The date and time the message was sent	`Date: Tue, 15 Nov 1994 08:12:31 GMT`
`Expect`	Indicates that the client requires the server to perform a specific behavior	`Expect: 100-continue`
`Host`	The server's domain name (used for virtual hosting), and the TCP port number the server is listening on	`Host: en.wikipedia.org:80` `Host: en.wikipedia.org`
`If-Match`	The corresponding operation is only performed when the entity provided by the client matches the corresponding entity on the server. Its main purpose is to be used in methods like PUT, updating a resource only if it has not been modified since the user last updated it	`If-Match: "737060cd8c284d8af7ad3082f209582d"`
`If-Modified-Since`	Allows returning 304 Not Modified if the corresponding content has not been modified	`If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT`
`If-None-Match`	Allows returning 304 Not Modified if the corresponding content has not been modified	`If-None-Match: "737060cd8c284d8af7ad3082f209582d"`
`If-Range`	If the entity has not been modified, send me the one or more parts I am missing; otherwise, send the entire new entity	`If-Range: "737060cd8c284d8af7ad3082f209582d"`
`Range`	Requests only part of an entity	`Range: bytes=500-999`
`User-Agent`	The browser's identity string	`User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/21.0`
`Origin`	Initiates a request for cross-origin resource sharing	`Origin: http://www.example-social-network.com`

Differences Between Similar Request Headers

Request header	Usage scenario	Field content	Required	Security
`Host`	The target server domain name of the request, automatically filled by the browser based on the request URL	Domain + port	Required in HTTP/1.1	No sensitive path information, but may be used in attacks
`Origin`	Indicates the "origin" of the request (protocol + domain + port), used for access control of cross-origin requests (such as CORS)	Protocol + domain + port	Only present for cross-origin or non-simple requests	Safer than Referer (no path information)
`Referer`	The full URL of the page the current request comes from (including the path)	Protocol + domain + port + path (no hash or user info)	Not required, automatically generated by the browser	May expose user privacy (such as URL parameters)

HTTP 1.0

In HTTP 1.0, the browser and server only maintain a short-lived connection; each request requires establishing a TCP connection with the server

The server disconnects the TCP connection immediately after completing request processing; the server neither tracks each client nor records past requests

Simply put, every interaction with the server requires opening a new connection, and every request requires a new handshake and wave

If a persistent connection is needed, a non-standard Connection field must be set: Connection: keep-alive

HTTP 1.1

In HTTP1.1, persistent connections are supported by default (Connection: keep-alive), meaning multiple HTTP requests and responses can be transmitted over one TCP connection, reducing the cost and latency of establishing and closing connections

One connection is established, and multiple requests are completed over this connection

At the same time, HTTP 1.1 also allows the client to send the next request without waiting for the result of the previous one, but the server must send back the response results in the order it received the client requests, so that the client can distinguish the response content of each request. This also significantly reduces the time required for the entire download process

Also, on the basis of HTTP1.0, HTTP1.1 adds more request headers and response headers to improve functionality, as follows:

Introduced more cache control strategies, such as If-Unmodified-Since, If-Match, If-None-Match and other cache headers to control caching strategies
Introduced range, allowing requesting only a certain part of a resource
Introduced host, enabling multiple virtual WEB sites to be created on one WEB server using different host names on the same IP address and port number

And additional request methods were added: put, delete, options

HTTP/2

Compared with previous versions, HTTP/2 has greatly improved performance, adding the following features:

Greatly reduced amount of transmitted data
- Transmission in binary
- Header compression: during transmission, the first message carries the complete request headers, and the second message only carries the request headers that differ from the previous message
Multiplexing and related features:
- Message priority: each stream has a priority ( 1-256 ), and streams can have dependencies between them
Server push:
- Push messages in parallel

Core Concepts

Connection: a connection. One TCP connection. Contains one or more Streams
Stream: a data stream. A bidirectional communication data stream, containing multiple Messages
Message: a message. Corresponds to a request or response in HTTP1, containing one or more Frames
Frame: a data frame. The smallest unit, storing HTTP1 content in a binary compressed format

Multiplexing

HTTP/2 multiplexes TCP connections: within one connection, both the client and the browser can send multiple requests or responses at the same time, and they do not need to correspond one-to-one in order, thus avoiding "head-of-line blocking"

Head-of-line blocking: TCP sends packets in segments, and thanks to sliding window control, multiple packets can be sent at once. The receiver receives and processes them by number; if the packet with the head number is lost, subsequent packets are blocked and can only be processed after the head packet is retransmitted.

Binary Framing

The frame is the smallest unit of information in HTTP/2 communication

HTTP/2 transmits data in binary format rather than the text format of HTTP 1.x, making parsing more efficient

Request and response data are divided into smaller frames, and they are binary-encoded

In HTTP/2, all communication under the same domain is completed over a single connection, which can carry any number of bidirectional streams

Each stream is sent in the form of messages, and a message consists of one or more frames. Multiple frames can be sent out of order and reassembled according to the stream identifier in the frame header, which is also the implementation condition for multiplexing to send data simultaneously

Header Compression

HTTP/2 uses a "head table" on both the client and server sides to track and store previously sent key-value pairs; identical data is no longer sent with every request and response

The head table persists for the lifetime of the HTTP/2 connection and is progressively updated jointly by the client and server

Server Push

HTTP/2 introduces server push, allowing the server to push resources to the client

The server pushes some resources the client needs along the way; for example, when responding to a page request, other resources of the page can be pushed along with it

This saves the client from creating another connection and sending a request to the server to fetch them

This approach is very suitable for loading static resources

HTTP3

Reasons for Abandoning the TCP Protocol

High connection establishment latency: HTTPS initial connection ( TCP handshake + TLS handshake ) requires 3 RTTs to establish
Head-of-line blocking problem: taking HTTP2 as an example, all streams on one TCP connection (streams, the data units transmitted by HTTP/2) must be transmitted in order. If one stream is lost, subsequent streams are blocked until the lost data is retransmitted.
Protocol ossification problem: as a protocol that has been running for nearly 40 years, many middleboxes (such as firewalls and routers) already rely on certain implicit rules, making it unrealistic to patch or push updates to the TCP protocol.

Introduction to QUIC

QUIC is a reliable data transfer protocol implemented based on UDP, offering lower latency and higher throughput

QUIC Features

Supports connection migration

When the network changes from WiFi to 4G, TCP uses the four-tuple (source IP, source port, destination IP, destination port) as the identifier, while QUIC uses the Connection ID as the identifier.

When the source IP changes due to network changes, the QUIC Connection ID does not change, so HTTP1/1.1/2 using the TCP protocol all need to reconnect, while HTTP3 does not.

Low-latency connection

QUIC integrates the TLS security protocol internally, so there is no need to go through the TCP three-way handshake and then the TLS handshake before data transmission can start. QUIC's initial connection only needs 1 RTT to start data transmission.

TCP + TLS1.2 ( 2 RTT ) + HTTP = 3 RTT ( the TCP RTT and the first RTT of TLS1.2 can overlap )

TCP + TLS1.3 + HTTP = 2 RTT ( the TCP RTT and the TLS1.3 RTT can overlap )

QUIC + HTTP2 = 1 RTT ( the QUIC RTT and the HTTP RTT can overlap )

Pluggable congestion control

TCP has a built-in sliding window for congestion control; modifying the algorithm is tied to the operating system kernel, and may require upgrading the operating system kernel version. QUIC supports flexibly "plugging in" different congestion control algorithms, such as Cubic, BBR, PCC, allowing engineers to flexibly adjust reliable transmission mechanisms and congestion control strategies without deep kernel development. For example, Cloudflare's open-source QUIC implementation quiche provides the setSendAlgorithm method, so engineers can directly choose an appropriate congestion control algorithm without going through the operating system kernel.

Reduced impact of packet loss

If a TCP packet belonging to Stream2 is lost, the transmission of subsequent packets will be blocked. This is the problem often referred to in the industry as "head-of-line blocking". In contrast, QUIC designs an independent control mechanism for each Stream, with no sequential dependency between Streams. This means that if a UDP packet belonging to Stream2 is lost, it only affects the processing of Stream2 and does not block the transmission of Stream1 and Stream3. This design effectively avoids the head-of-line blocking problem in the TCP protocol.

QUIC's Challenges

Server side: not only does the QUIC protocol need to be adapted, but compatibility with the TCP protocol must also be ensured. In addition, TCP has been deeply optimized over many years; can QUIC's actual performance match TCP's?
Client side: cost trade-offs need to be made between adaptation and benefits. Clients adopting the QUIC protocol must have downgrade and fault-tolerance capabilities, and be prepared to maintain both the new and old network libraries simultaneously for a long time.

CDN (Content Delivery Network)

The biggest role of a CDN is to allocate the nearest resources based on the user's location, which can reduce the data transmission distance and make requests respond faster

Load Balancing System

During the DNS resolution phase, a CDN does not return an IP address, but returns a CNAME.

Since no IP address is returned, the local DNS sends another request to the load balancing system, entering the CDN's global load balancing system for intelligent scheduling:

Look at the user's IP address, look up the geographic location in a table, and find the relatively nearest edge node
Look at the user's carrier network and find an edge node on the same network
Check the load of edge nodes and find a lightly loaded node
Others, such as the node's "health status", service capacity, bandwidth, response time, etc.

Combining the above factors, the most suitable edge node is obtained and then returned to the user, so the user can access the CDN's cache proxy nearby

Cache Proxy

The caching system is another key component of a CDN; the caching system selectively caches the most frequently used resources

There are two metrics for measuring CDN service quality:

Hit rate: the resource the user accesses happens to be in the caching system and can be returned directly to the user; the ratio of hits to all accesses
Origin-pull rate: the resource is not in the cache and must be fetched from the origin via a proxy; the ratio of origin pulls to all accesses

The caching system can also be divided into levels: first-level cache nodes and second-level cache nodes. First-level caches have higher configurations and connect directly to the origin; second-level caches have lower configurations and connect directly to users

When pulling from the origin, second-level caches only ask first-level caches, and only when the first-level cache misses does it go to the origin, which can effectively reduce real origin pulls

Modern commercial CDNs all have hit rates above 90%, equivalent to amplifying the origin's service capacity by more than 10 times

WebSocket

Full-Duplex

Communication allows data to be transmitted in both directions simultaneously; in capability it is equivalent to the combination of two simplex communication methods, and the server can actively push data to the client.

Binary Frames

It adopts a binary frame structure whose syntax and semantics are completely incompatible with HTTP. Compared with HTTP/2, WebSocket focuses more on "real-time communication", while HTTP/2 focuses more on improving transmission efficiency, so the frame structures of the two also differ greatly

It does not define streams like HTTP/2, so features such as multiplexing and priority do not exist

It is itself full-duplex, so server push is not needed either

Protocol Names

ws and wss are introduced to represent the plaintext and ciphertext WebSocket protocols respectively, and the default ports are 80 or 443, almost identical to HTTP

Handshake

Sending the Request

Connection: must be set to Upgrade, indicating that the client wants to upgrade the connection
Upgrade: must be set to WebSocket, indicating the desire to upgrade to the WebSocket protocol
Sec-WebSocket-Key: a base64-encoded ciphertext sent by the client, used as a simple authentication key. It requires the server to return a correspondingly encrypted Sec-WebSocket-Accept response; otherwise the client will throw an error and close the connection
Sec-WebSocket-Version: indicates the supported WebSocket version

Example request:

GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13

Returning the Response

Response line: indicates that the server accepts the client's WebSocket protocol connection
Sec-WebSocket-Accept: verifies the client request message, also to prevent mistaken connections. The specific approach is to take the value of Sec-WebSocket-Key in the request header, append a dedicated UUID, and then compute a digest

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat

Advantages

Less control overhead: the protocol header of data packets is small, unlike HTTP which needs to carry the complete headers with every request
Stronger real-time capability: compared with HTTP, where the server can only respond after waiting for the client to initiate a request, the latency is significantly lower
Maintains connection state: after the communication is created, state information can be omitted, unlike HTTP which needs to carry authentication with every request
Better binary support: binary frames are defined, better handling binary content
Supports extensions: users can extend the WebSocket protocol and implement some custom sub-protocols
Better compression: with appropriate extension support, WebSocket can reuse the context of previous content, significantly improving the compression ratio when transmitting similar data

Disadvantages

Data fragments are transmitted in order, so multiplexing is not supported, and the head-of-line blocking problem of HTTP1.1 still exists
Data compression is not supported

GET/POST Differences

GET

Requests a representation of a specified resource; requests using GET should only be used to retrieve data
Harmless when the browser goes back
The URL can be bookmarked
Requests are actively cached by the browser
Request parameters can only be URL-encoded
Request parameters are fully preserved in the browser history
Parameters transmitted in the URL have a length limit; the entire URL length generally does not exceed 2083 (2048 + 35)
Parameters only accept ASCII characters
Parameters are directly exposed in the URL, so sensitive data cannot be transmitted
Parameters are passed via the URL
Request parameters are generally sent together with the header

POST

Submits an entity to the specified resource, usually causing a state change or side effects on the server
The request is resubmitted when the browser goes back
The URL generally cannot be bookmarked
Requests are not actively cached by the browser; it must be set manually
Request parameters support multiple encoding methods
Request parameters are not preserved by the browser
Request parameters have no length limit, but overly large parameters may cause the request to time out
Parameters have no restrictions; they can be binary, text, images, etc.
Parameters require packet capture to see and do not appear in the URL, so sensitive data can be transmitted
Parameters are passed via the request body
The header may be sent first, the server responds with 100 continue, and then the data is sent