Skip to content

Instantly share code, notes, and snippets.

@sebdeckers

sebdeckers/README.md

Created July 29, 2019 10:10
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save sebdeckers/b9d0f720bb250512011d5b8c4ac4aaba to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save sebdeckers/b9d0f720bb250512011d5b8c4ac4aaba to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. sebdeckers created this gist Jul 29, 2019.
    289 changes: 289 additions & 0 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,289 @@
    # HTTP/2, HTTP/3, and Beyond

    A brief overview of the web's latest transport protocols and an exploration of what's coming soon-ish.

    - Date: 2019-07-26
    - Speaker: Sebastiaan Deckers
    - Location: Zendesk, Singapore

    ## Introduction

    Joined Zendesk 1 month ago (Team Groot, GraphQL API)

    https://app.slack.com/client/T024F4EL1/C1LQ214T1/user_profile/UJSFRTB7C

    Part-time freelance with startups in SG for past ~10 years

    https://www.linkedin.com/in/sebastiaandeckers/

    Full-time DIY FOSS for past 3 years

    https://twitter.com/sebdeckers

    https://gitlab.com/sebdeckers

    Contributed to HTTP/2 in Node.js

    https://github.com/nodejs/node

    https://github.com/nodejs/node/pull/14239

    Commons Host: FOSS CDN

    https://commons.host

    - Static sites

    - DNS over HTTPS (DoH)

    https://hub.docker.com/r/commonshost/dohnut

    - Gopher over HTTPS

    https://gopher.commons.host

    Web perf tooling

    - Unbundle: resource tracing for HTML, JS, CSS

    https://gitlab.com/sebdeckers/unbundle

    - Server Push Manifest & Cache Diary

    https://help.commons.host/manifest/

    https://help.commons.host/server/configuration/server/#push

    ## HTTP/1.1

    Published in ~~1997~~ 1999

    https://tools.ietf.org/html/rfc2068

    https://tools.ietf.org/html/rfc2616

    Text based protocol over TCP/IP sockets.

    https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#/media/File:Http_request_telnet_ubuntu.png

    Simple bytestream, easy to parse by looking for `\r\n` line endings.

    Open socket, write request, read response headers, read response body, close socket. GG EZ.

    ## HTTPS: HTTP over TLS

    Simple idea: Take HTTP/1.1 and stuff it inside a TLS socket.

    https://tools.ietf.org/html/rfc2818

    - HTTP/1.1 RFC: 176 pages
    - TLS/1.0 RFC: 80 pages
    - HTTPS RFC: 7 pages

    It's called TLS. Calling it SSL makes Baby Groot sad. SSL is a Netscape technology from the mid-90's and not an IETF standard. TLS is the actual standard. It has been around since 1999.

    Still just a blackbox bytestream. Treat TLS like TCP. It just works.

    ## Hacks to make HTTP scale

    https://hpbn.co

    ### Data concatenation

    Image slicing, inlined base64 URIs, font icons, JavaScript/CSS bundles, etc.

    ### Parallel Connections

    Multiple TLS+TCP sockets

    Independent slow start, difficult to reach full speed with not enough data per socket

    ## HTTP/2: Not as new as you might think

    https://en.wikipedia.org/wiki/SPDY

    First proposed as SPDY by Google in 2012. Standard published as RFC in 2015. Implemented in Node.js since 2017.

    https://http2.github.io

    What's new:

    - Header compression (HPACK)

    https://http2.github.io/http2-spec/compression.html

    State held on both server and client. Start with predefined common header field names and values, then use Huffman coding to update table state for duration of the session.

    Cheap to send lots of headers if they are duplicated across requests & responses within the same session.

    Avoid table churn or exhaustion from huge headers like changing cookies.

    - Server Push

    https://http2.github.io/http2-spec/#PUSH_PROMISE

    Server sends `PUSH_PROMISE` frame with the headers of a "fake" request.

    Server sends `HEADERS` frame with the headers of the response.

    Server sends `DATA` frame(s) with the payload of the response.

    Client avoids making the round-trip-time (RTT) of making requests.

    Only half of a feature: Server has no idea what the client already has cached. Rarely used in real world today. Can even be bad when there is over-push, i.e. push redundant data for responses the client already has cached. Cache Digest proposal tried to fix this problem but browser implementors not currently interested.

    - Streams with multi-plexing

    https://developers.google.com/web/fundamentals/performance/http2/

    Many request/response streams within a session.

    Single, long-lived TCP/IP socket. Only need to do the multiple round trip TCP & TLS handshake once.

    Avoids HTTP request & response head-of-line (HoL) blocking. No need for parallel connections. Err, just pushes HoL down to the single TCP/IP socket.

    Supports prioritisation but this turned into a big mess and not really utilised IRL.

    - Always encrypted

    Runs on port 443/tcp like HTTPS

    Uses `https:` protocol scheme

    Spec allows plaintext H2 but impractical to deploy at Internet scale.

    - DoH

    Used by DNS over HTTP as a transport. Get the benefit of encrypting DNS, without changing the payload. Easy to implement and deploy on familiar infrastructure.

    ## Middleboxes

    https://en.wikipedia.org/wiki/Middlebox

    Routers, switches, load balancers, accelerators, etc.

    Equipment at the edges is easy to update. Browsers auto-update daily/weekly/monthly. Server apps deployed with CI/CD.

    Middleboxes are *never* updated. Deployed for years until EOL.

    Behaviour is ossified. Unknown protocols or extensions are dropped. Impossible to roll out new Internet protocols.

    Middleboxes make too many assumptions about plaintext HTTP on port 80.

    Solution is to use TLS extension ALPN to select HTTP/2 instead of HTTP/1.1 for the encrypted data.

    https://datatracker.ietf.org/doc/rfc7413/

    E.g. TCP Fast Open (TFO) when deployed ends up dropped by some % of middleboxes, requires retransmission, often actually slower. Sadpanda.

    We're stuck with TCP...?

    Middleboxes mostly ignore UDP. Free to carry any payload.

    ## QUIC: "TCP/2" over UDP/IP

    https://quicwg.org

    QUIC implements TCP-like mechanism, and a lot of the cool newer stuff, over UDP.

    https://tools.ietf.org/html/rfc793

    TCP RFC793:

    - 85 pages
    - September 1981

    https://tools.ietf.org/html/rfc768

    UDP RFC768:

    - 3 pages
    - August 1980

    TCP is old and smart. Ossified AF.

    UDP is older and dumb. Nothing for middleboxes to break.

    Caveat: Sporadic UDP blocking and rate limiting in the wild, ~3-7% failure by Google measurements.

    https://docs.google.com/spreadsheets/d/1D0tW89vOoaScs3IY9RGC0UesWGAwE6xyLk0l4JtvTVg/edit#gid=1965785440

    19 implementations at IETF Hackathon interop testing.

    https://www.github.com/nodejs/quic

    WIP in Node.js

    ## HTTP/3: You're probably already using it<sup>*<sup>

    Live in Chrome browser

    Live at Google/Youtube, Facebook, Cloudflare, etc.

    <sup>*</sup> Google QUIC ~= IETF QUIC+HTTP/3

    gQUIC is one big spec. iQUIC has split into many specs.

    - HTTP/3: HTTP over QUIC. HTTP/3 is conceptually roughly the same as HTTP/2. Mapping of HTTP streams to QUIC streams. Stream priority may or may not get dropped or redesigned.

    - Invariants: Avoid ossification using greasing of future versions

    - QPACK: Redesigned header compression. HPACK is now QPACK.

    - TLS in QUIC: Modular instead of layered design. Allows for 0RTT (careful: replay attacks)

    - Spin Bit: Most discussed bit in history.

    ## Future of QUICv2, QUICv3, etc

    Current focus is entirely on QUICv1 and HTTP/3 specs.

    https://datatracker.ietf.org/wg/quic/documents/

    Still many design issues being discussed. Hopefully concensus soon, deferring many features. Expect short interval iterations for future versions of QUIC and extensions.

    - Multi-path: Connections across different networks. E.g. switching between Wi-Fi and cellular

    https://datatracker.ietf.org/doc/draft-deconinck-quic-multipath/

    - Unreliable datagrams: Allow mixing reliable and unreliable streams inside QUIC (aka DTLS for QUIC)

    https://martinthomson.github.io/quic-dtls/

    Other protocols will explore using QUIC as their transport to get al this cool free stuff. E.g. WebRTC, WebSocket, DNS, etc. Same for proprietary appliation protocols like databases or message queues or MUX.

    ## Deployment & Discovery of HTTP/3

    First connection is still HTTP/2

    Check for any `Alt-Svc` header indicating QUIC support. Cache for future connections.

    Reconnect using QUIC and HTTP/3.

    https://tools.ietf.org/html/rfc7838

    https://datatracker.ietf.org/meeting/105/materials/slides-105-httpbis-sessb-httpssvc-record-00

    Proposals to bootstrap this over DNS using `HTTPSSVC` or `ALTSVC` DNS resource record (RR).

    ## Get Involved

    https://nodejs.org/api/http2.html

    Use HTTP/2 in your projects

    Contribute to QUIC

    https://www.ietf.org/how/lists/

    Subscribe to IETF mailing lists

    https://www.youtube.com/watch?v=wPGErSxTYD8&list=PLC86T-6ZTP5j2xKSoqW0_ajvdr58Fau6g

    Watch live streams or recordings of IETF meetings

    https://ietf.org/how/meetings/106/

    Attend IETF 106 in Singapore November 2019. Hackathon (interop event) is free of charge.

    EOF