HTB Academy - Web Requests

🎯 Objective

Keep a crisp, practical explainer of HTTP requests, responses, headers, and methods/status codes—with raw examples and copy‑paste cribs. This is the version I actually reference during testing.

🌐 Big Picture

HTTP exchanges are a client request (browser/cURL) and a server response (web server/app). The request carries the target resource (URL/path/params), headers, and optionally a body. The server processes it and replies with a status code, headers, and often a response body.

HTTP/1.x is text‑based (newline‑delimited). HTTP/2 is binary/framed (HPACK, streams), but concepts—methods, status, headers—carry over.

📤 HTTP Request (anatomy)

Example request (GET a login page):

Request line has three space‑separated fields:

Then headers (name: value), then an empty line, then (for methods like POST/PUT/PATCH) an optional body.

📥 HTTP Response (anatomy)

Example raw response:

Status line: HTTP/1.1 200 OK → protocol + status code + reason.
Followed by headers and an optional body (HTML, JSON, images, PDFs, etc.).

🏷️ HTTP Headers (types)

General headers

Describe the message rather than its content (used on requests and responses).

Entity headers

Describe the content entity (commonly in responses, or POST/PUT bodies).

Request headers

Used by the client; not about the message body itself.

Response headers

Used by the server; not about the body content.

Security headers

Policies to harden browser behavior.

Note: Apps can emit custom headers as needed.

📨 Methods (verbs)

Common ones you’ll see/testing basics:

Many modern apps use mostly GET/POST; RESTful APIs often use PUT/DELETE/PATCH for updates.

🔢 Status codes (families)

• 1xx: Informational (rare in browsing).
• 2xx: Success (e.g., 200 OK).
• 3xx: Redirects (e.g., 302 Found).
• 4xx: Client errors (e.g., 400 Bad Request, 403 Forbidden, 404 Not Found).
• 5xx: Server errors (500 Internal Server Error).

Quick examples:

• 200 OK – Success; body contains the resource.
• 302 Found – Temporary redirect (often after login).
• 400 Bad Request – Malformed request (missing newline, bad JSON).
• 403 Forbidden – Authenticated but not authorized (or input flagged).
• 404 Not Found – Resource doesn’t exist.
• 500 Internal Server Error – App/server blew up handling it.

📎 Copy‑paste crib

See exactly what cURL sends (HTTP/1.1):

curl -v http://example.com/ -H 'Accept: text/html'

Send JSON (with correct content type):

curl -sS -X POST 'https://api.example.com/items'   -H 'Content-Type: application/json'   -d '{"name":"demo","enabled":true}'

HEAD only (check size/headers without body):

curl -I https://example.com/big.iso

Trace raw over TLS (handy when debugging headers):

openssl s_client -connect example.com:443 -servername example.com
# then paste an HTTP/1.1 request:
# GET / HTTP/1.1
# Host: example.com
# Connection: close
#

OPTIONS to see allowed methods:

curl -i -X OPTIONS https://api.example.com/resource

🧪 Troubleshooting

• 415/400 → Content‑Type mismatch or malformed JSON.
• CORS in browsers → Use server‑side or cURL (CORS is a browser policy).
• Auth woes (401/403) → Recheck Authorization/cookies; capture via DevTools and replay.
• Chunked vs length → Servers may use Transfer-Encoding: chunked instead of Content-Length.
• HTTP/2 gotchas → Intermediaries may coalesce headers; use --http1.1 if behavior differs.

✅ TL;DR

• Request = method + path + version + headers + body (optional).
• Response = version + status + headers + body (optional).
• Know the header categories, the core verbs, and the status families.
• When stuck, capture in DevTools and replay with cURL—then iterate.