diff --git a/DEVELOP.md b/DEVELOP.md index 6bb292f..bc7385f 100644 --- a/DEVELOP.md +++ b/DEVELOP.md @@ -78,6 +78,12 @@ Before pushing, verify: - [ ] No `unwrap()` outside of tests - [ ] Config values are externalized (not hardcoded) +## References + +- [Intra Comparison](docs/intra-comparison.md) — Architectural comparison + with Jigsaw's Intra, explaining why DNS forwarding alone can't bypass + censorship and what's needed (TLS splitting, TUN interception, etc.) + ## Milestone Tracking See [docs/plan.md](docs/plan.md) for the full project plan and milestone checklist. diff --git a/README.md b/README.md index 0a46802..426a2af 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,18 @@ A lightweight DNS-over-HTTPS (DoH) forwarder written in Rust. -Accepts DNS queries over HTTPS ([RFC 8484](https://www.rfc-editor.org/rfc/rfc8484)) and forwards them to upstream DoH resolvers (Cloudflare, Google, Quad9, or custom). +Accepts DNS queries over traditional DNS (UDP/TCP) and HTTP DoH ([RFC 8484](https://www.rfc-editor.org/rfc/rfc8484)), then forwards them upstream via DoH (HTTPS POST) to configurable resolvers. + +## Why? + +See [docs/story.md](docs/story.md) for the full context — why this exists, what already works, and where this project is headed. ## Features +- **Dual-protocol listener** — traditional DNS (UDP/TCP) + HTTP DoH endpoints - **RFC 8484 compliant** — GET (`?dns=base64url`) and POST (`application/dns-message`) - **Multiple upstreams** — configure several DoH resolvers -- **Selection strategies** — round-robin or failover -- **In-memory cache** — TTL-aware with LRU eviction (configurable) +- **Selection strategies** — round-robin or failover with automatic retry - **Environment overrides** — override config via env vars - **Graceful shutdown** — handles SIGINT and SIGTERM - **Health endpoint** — `GET /health` @@ -20,7 +24,7 @@ Accepts DNS queries over HTTPS ([RFC 8484](https://www.rfc-editor.org/rfc/rfc848 # Build cargo build --release -# Run with default config +# Run with default config (DoH on :8800, DNS on :53530) ./target/release/doh-forwarder # Run with a custom config @@ -29,94 +33,111 @@ cargo build --release ## Configuration -Create a TOML config file (see [config/default.toml](config/default.toml)): +See [config/default.toml](config/default.toml): ```toml [server] -listen = "0.0.0.0:3000" +listen = "0.0.0.0:8800" + +[dns] +listen = "0.0.0.0:53530" [upstream] resolvers = [ { name = "cloudflare", url = "https://cloudflare-dns.com/dns-query" }, - { name = "google", url = "https://dns.google/dns-query" }, - { name = "quad9", url = "https://dns.quad9.net/dns-query" }, ] strategy = "round-robin" # or "failover" - -[cache] -enabled = true -max_entries = 10000 -max_ttl_secs = 3600 ``` ### Environment Variable Overrides | Variable | Description | Example | |---|---|---| -| `DOH_LISTEN` | Override server listen address | `0.0.0.0:8080` | +| `DOH_LISTEN` | Override DoH server listen address | `0.0.0.0:8080` | | `DOH_UPSTREAM_STRATEGY` | Override upstream strategy | `failover` | | `DOH_UPSTREAM_RESOLVERS` | Comma-separated `name=url` pairs | `google=https://dns.google/dns-query,cf=https://cloudflare-dns.com/dns-query` | | `RUST_LOG` | Log level filter | `debug`, `doh_forwarder=trace` | -## API +## Usage -### POST `/dns-query` - -Send a raw DNS wire-format query in the request body. +### Point your system DNS at the forwarder ```bash -# Using a pre-built DNS query (hex-encoded) -curl -X POST http://localhost:3000/dns-query \ +# /etc/resolv.conf +nameserver 127.0.0.1 +# Then run the forwarder with [dns] listen = "127.0.0.1:53" +``` + +Or use `resolvectl` with systemd-resolved: + +```bash +resolvectl dns eth0 127.0.0.1 +resolvectl dns-over-tls no # we handle encryption upstream +``` + +### DoH API + +**POST `/dns-query`** — raw DNS wire-format in body: + +```bash +curl -X POST http://localhost:8800/dns-query \ -H "Content-Type: application/dns-message" \ -H "Accept: application/dns-message" \ --data-binary @query.bin ``` -### GET `/dns-query` - -Send a base64url-encoded DNS query as a query parameter. +**GET `/dns-query`** — base64url-encoded query parameter: ```bash -# Encode a DNS query and send via GET -QUERY=$(python3 -c " -import base64, struct -# Minimal A record query for example.com -q = b'\\x12\\x34' # ID -q += b'\\x01\\x00' # flags -q += b'\\x00\\x01' # 1 question -q += b'\\x00\\x00' # 0 answers -q += b'\\x00\\x00' # 0 authority -q += b'\\x00\\x00' # 0 additional -q += b'\\x07example\\x03com\\x00' # name -q += b'\\x00\\x01' # type A -q += b'\\x00\\x01' # class IN -print(base64.urlsafe_b64encode(q).rstrip(b'=').decode()) -") -curl "http://localhost:3000/dns-query?dns=$QUERY" +curl "http://localhost:8800/dns-query?dns=" ``` -### GET `/health` - -Returns `ok` if the server is running. +**GET `/health`** — returns `ok`: ```bash -curl http://localhost:3000/health -# ok +curl http://localhost:8800/health ``` -## Upstream Strategies +### Upstream Strategies | Strategy | Behavior | |---|---| -| `round-robin` | Distributes queries across resolvers in order | +| `round-robin` | Distributes queries across resolvers in rotation | | `failover` | Tries resolvers in order; moves to the next on failure | +## Project Structure + +``` +doh-forwarder/ +├── Cargo.toml +├── config/ +│ └── default.toml # Default configuration +├── docs/ +│ ├── plan.md # Project plan & milestones +│ └── story.md # Why this project exists +├── src/ +│ ├── main.rs # Entrypoint, server setup, graceful shutdown +│ ├── lib.rs # Public API re-exports +│ ├── config.rs # TOML config + env overrides +│ ├── routes.rs # HTTP route handlers (/dns-query, /health) +│ ├── dns/ +│ │ ├── mod.rs +│ │ ├── query.rs # DNS query parsing +│ │ ├── response.rs # DNS response building (SERVFAIL) +│ │ └── listener.rs # UDP/TCP DNS listener +│ └── upstream/ +│ ├── mod.rs +│ └── doh_client.rs # DoH forwarding client (retry, strategies) +├── tests/ +│ └── integration.rs # HTTP integration tests +└── DEVELOP.md # Development rules & milestone tracking +``` + ## Development -```bash -# Prerequisites -rustup show # Rust 1.80+ +See [DEVELOP.md](DEVELOP.md) for development rules, coding standards, and milestone tracking. +```bash # Run with auto-reload cargo install cargo-watch cargo watch -x run @@ -129,32 +150,6 @@ cargo clippy -- -D warnings cargo fmt --check ``` -See [DEVELOP.md](DEVELOP.md) for full development rules and milestone tracking. - -## Project Structure - -``` -doh-forwarder/ -├── Cargo.toml -├── config/ -│ └── default.toml # Default configuration -├── src/ -│ ├── main.rs # Entrypoint, server setup -│ ├── lib.rs # Public API re-exports -│ ├── config.rs # TOML config + env overrides -│ ├── cache.rs # DNS response cache (TTL + LRU) -│ ├── dns/ -│ │ ├── mod.rs -│ │ ├── query.rs # DNS query parsing -│ │ └── response.rs # DNS response building -│ ├── upstream/ -│ │ ├── mod.rs -│ │ └── doh_client.rs # DoH forwarding client -│ └── routes.rs # HTTP route handlers -└── tests/ - └── integration.rs # Integration tests -``` - ## License MIT diff --git a/docs/intra-comparison.md b/docs/intra-comparison.md new file mode 100644 index 0000000..934ba2a --- /dev/null +++ b/docs/intra-comparison.md @@ -0,0 +1,266 @@ +# doh-forwarder vs Intra — Architectural Comparison + +## Overview + +This document compares **doh-forwarder** (this project) with +[Intra](https://github.com/Jigsaw-Code/Intra) by Jigsaw (Google) to +understand why Intra can unblock YouTube and other censored sites while +doh-forwarder can only resolve their DNS. + +## The Core Problem + +DNS resolution is only the first layer of censorship. Even when DNS is +encrypted and returns the correct IP, the **actual HTTPS connection** to +the target server can be blocked by Deep Packet Inspection (DPI) at the +TCP/TLS level. + +``` +doh-forwarder flow: + Browser → "youtube.com?" → doh-forwarder → DoH → returns IP ✅ + Browser → HTTPS to YouTube IP:443 ──────────── BLOCKED by DPI ❌ + +Intra flow: + Browser → HTTPS to YouTube IP:443 → TUN → Intra split/retry → YouTube ✅ +``` + +## Architectural Differences + +### doh-forwarder — DNS-only forwarder + +``` + +---------------------+ + | DNS Client | + | (browser, app, OS) | + +--------+------------+ + | + +------------+-------------+ + | | + HTTPS (POST/GET) UDP/TCP (port 53) + | | + v v + +-------------------+ +-------------------+ + | Axum HTTP Server | | DnsListener | + | (port 8800) | | (UDP + TCP) | + +--------+----------+ +--------+----------+ + | | + +-----------+-------------+ + | + v + +-------------------+ + | DohClient | + | (reqwest HTTP) | + +--------+----------+ + | + +-----------+-----------+ + | | | + v v v + Cloudflare Google Quad9 + (DoH) (DoH) (DoH) +``` + +- Accepts DNS queries on port 53 (UDP/TCP) and port 8800 (DoH) +- Forwards queries to upstream DoH resolvers over HTTPS +- Returns DNS responses to clients +- **Does not touch any non-DNS traffic** + +### Intra — Full traffic interception + circumvention + +``` + +-----------------------------------------------------+ + | TUN Device | + | (captures ALL device traffic) | + +---------------------------+--------------------------+ + | + +---------------+----------------+ + | | + DNS packets (port 53) TCP packets (port 443) + | | + v v + +------------------+ +------------------------+ + | DNS Interception | | intraStreamDialer | + | → DoH query via | | → DialWithSplitRetry() | + | HTTP POST | | → TLS hello splitting | + +------------------+ +------------------------+ + | + +--------+--------+ + | | + Success? Blocked? + | | + v v + Normal TLS Close + retry with + handshake split Client Hello +``` + +- Creates a **TUN device** that captures all device traffic +- Intercepts DNS packets → sends via DoH (like doh-forwarder) +- Intercepts TCP connections to port 443 → applies circumvention +- Uses **TLS Client Hello splitting** to evade DPI + +## Feature Comparison + +| Feature | doh-forwarder | Intra | +|---------|:------------:|:-----:| +| DNS-over-HTTPS forwarding | ✅ | ✅ | +| Multiple upstream resolvers | ✅ | ✅ | +| Round-robin / failover | ✅ | ❌ | +| UDP + TCP DNS listener | ✅ | ✅ (via TUN) | +| HTTP DoH endpoint | ✅ | ❌ | +| TUN device (all traffic) | ❌ | ✅ | +| TCP stream interception | ❌ | ✅ | +| TLS Client Hello splitting | ❌ | ✅ | +| Retry on censorship detection | ❌ | ✅ | +| SNI-based error reporting | ❌ | ✅ | +| EDNS padding | ❌ | ✅ | +| IP map with confirmation tracking | ❌ | ✅ | +| Servfail hangover (rate limiting) | ❌ | ✅ | +| Written in | Rust | Go + Java | +| Platform | Linux (server) | Android (VPN) | + +## The Missing Piece: TLS Client Hello Splitting + +The critical technique Intra uses to bypass censorship is **TCP segment +splitting** on the TLS Client Hello. Here's how it works: + +### How DPI censorship works + +When your browser connects to `https://www.youtube.com`, the first thing +it sends is a **TLS Client Hello** packet. This packet contains the +**SNI (Server Name Indication)** field in plaintext — the domain name +`www.youtube.com`. A DPI system reads this field and blocks the +connection if the domain is on a blocklist. + +### How splitting defeats DPI + +Intra's `splitHello()` function splits the TLS Client Hello into **two +TCP segments** at a random offset (6–64 bytes): + +``` +Normal Client Hello (one TCP segment): + [TLS Header (5 bytes)] [SNI: www.youtube.com ... rest of hello] + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + DPI reads this → finds "youtube.com" → BLOCKS + +Split Client Hello (two TCP segments): + Segment 1: [TLS Header (5 bytes)] [first N bytes...] + Segment 2: [remaining bytes... SNI: www.youtube.com ...] + + DPI can't reassemble fast enough → PASSES THROUGH ✅ +``` + +For TLS Client Hello specifically, Intra also **properly fragments the +TLS record header** — it adjusts the record length field in each segment +so the TLS layer itself remains valid: + +```go +// First segment: adjust record length to split point +pkt := hello[:splitLen] +binary.BigEndian.PutUint16(pkt[3:5], uint16(recordSplitLen)) + +// Second segment: copy TLS header, set remaining length +pkt = hello[splitLen-5:] +copy(pkt, hello[:5]) +binary.BigEndian.PutUint16(pkt[3:5], recordLen-uint16(recordSplitLen)) +``` + +### The retry mechanism + +`DialWithSplitRetry()` in `retrier.go`: + +1. Opens a normal TCP connection to the target +2. Sends the TLS Client Hello +3. Sets a read deadline (1200ms + 2×RTT) +4. If the connection is dropped or times out → **censorship detected** +5. Closes the connection +6. Opens a **new** TCP connection +7. Replays the Client Hello as **split segments** +8. If this succeeds → the connection proceeds normally + +## Why doh-forwarder Can't Unblock YouTube + +doh-forwarder operates at the **DNS layer only**. It has no visibility +into or control over: + +- TCP connections to resolved IPs +- TLS handshakes +- HTTP requests + +The traffic flow is: + +``` +1. Browser asks OS: "What's the IP for youtube.com?" +2. OS sends DNS query to doh-forwarder (port 53) +3. doh-forwarder forwards via DoH → gets correct IP (e.g., 142.250.80.46) +4. Browser connects directly to 142.250.80.46:443 +5. Browser sends TLS Client Hello with SNI=www.youtube.com +6. ISP's DPI reads SNI → blocks connection +7. Browser shows "connection timed out" or "connection reset" +``` + +Step 4–7 happen entirely outside doh-forwarder's scope. + +## What Would Be Needed + +To achieve what Intra does, doh-forwarder would need to evolve from a +DNS forwarder into a **traffic interception and circumvention proxy**: + +### Option A: SOCKS5 Proxy with Split Transport (M6 path) + +``` +Browser → SOCKS5 proxy (doh-forwarder) → split TCP → target server +``` + +- Configure browser to use SOCKS5 proxy +- Proxy applies TLS splitting on outbound connections +- Lighter than full TUN, requires browser config + +### Option B: Transparent Proxy with TUN (Intra-like) + +``` +All traffic → TUN device → doh-forwarder → DNS: DoH / TCP: split → target +``` + +- Zero browser configuration +- Requires root/CAP_NET_ADMIN for TUN +- Significantly more complex + +### Option C: Hybrid + +- DNS forwarding via DoH (current functionality) +- Optional SOCKS5 proxy for TCP circumvention +- Optional TUN mode for transparent interception +- User chooses based on their environment + +## Lessons from Intra's Codebase + +### 1. IP Map with Confirmation Tracking + +Intra maintains an `ipmap` that tracks which IPs of a DoH resolver +actually work. If a confirmed IP starts failing, it's disconfirmed and +alternatives are tried. This provides resilience against partial IP +blocking. + +### 2. EDNS Padding + +Intra pads DNS queries to standard sizes to prevent traffic analysis +that could identify specific domains by query size. + +### 3. Servfail Hangover + +When a DoH server returns errors, Intra enters a 10-second "hangover" +where it stops sending queries (returns SERVFAIL immediately). This +rate-limits queries to misconfigured or blocked servers. + +### 4. SNI Reporter + +Intra reports which SNIs are being censored (via the Choir library) to +contribute to censorship monitoring without revealing individual user +activity. + +## References + +- [Intra source code](https://github.com/Jigsaw-Code/Intra) +- [Outline SDK](https://github.com/Jigsaw-Code/outline-sdk) — the + transport library Intra uses +- [RFC 8484 — DNS over HTTPS](https://tools.ietf.org/html/rfc8484) +- [TLS Client Hello splitting](https://gfw.report/talks/syria2016/en/) + — research on DPI evasion diff --git a/docs/story.md b/docs/story.md new file mode 100644 index 0000000..5366b07 --- /dev/null +++ b/docs/story.md @@ -0,0 +1,140 @@ +# The Story — Why This Project Exists + +## The Problem + +In countries with internet censorship (Iran, China, Russia, and others), DNS is +one of the first things controlled. ISPs intercept plain DNS queries (UDP/TCP +port 53) and: + +- Return fake IP addresses for blocked domains +- Log every domain you look up +- Inject ads or redirect to government portals + +DNS-over-HTTPS (DoH) and DNS-over-TLS (DoT) solve this by encrypting DNS +queries, making them invisible to the ISP's DNS layer. + +## What Already Exists + +This is not a new problem. Mature tools already solve local DNS → DoH forwarding: + +| Tool | What it does | Maintained by | +|---|---|---| +| `cloudflared` | Local DNS → DoH proxy (`cloudflared proxy-dns`) | Cloudflare | +| `dnscrypt-proxy` | DoH + DoT + anonymized relays, blocklists | Frank Denis | +| `stubby` | Local DNS → DoT proxy | getdns team | +| `systemd-resolved` | Built into systemd, supports DoT | systemd team | +| `AdGuard Home` | DNS proxy + DoH/DoT + web UI + blocklists | AdGuard | + +For simple "local DNS → DoH" forwarding, **`cloudflared proxy-dns` does exactly +what this project does in one command:** + +```bash +cloudflared proxy-dns --port 5353 --upstream https://cloudflare-dns.com/dns-query +``` + +## The Real Obstacle + +The harder problem — and the one most relevant to censored regions — is that +**upstream DoH/DoT endpoints themselves get blocked:** + +| Technique | What gets blocked | How | +|---|---|---| +| SNI inspection | `cloudflare-dns.com`, `dns.google` | TLS ClientHello reveals the domain | +| IP blocklist | Known resolver IPs (1.1.1.1, 8.8.8.8) | Direct IP blocking | +| TLS fingerprinting | DoH/DoT traffic patterns | JA3/JA4 fingerprint analysis | +| Deep packet inspection | HTTPS traffic to known DNS resolvers | Pattern matching on domain + path | + +A local forwarder that sends `POST https://cloudflare-dns.com/dns-query` will +work in most of the world, but **will likely fail in Iran** because the +connection to Cloudflare itself is blocked. + +## What Actually Works in Censored Environments + +Tools that solve the **upstream blocking** problem: + +1. **Warp / WireGuard** — tunnel all traffic (including DNS) through an + encrypted VPN. The ISP sees WireGuard traffic but can't inspect it. + +2. **GoodbyeDPI / zapret** — packet manipulation to evade DPI without a VPN. + Modifies TCP flags, splits packets, fakes SNI. + +3. **dnscrypt-proxy with anonymized relays** — DNS queries go through + intermediate relays. The resolver never sees your IP, and the ISP never + sees the resolver's IP. + +4. **Tor** — route DNS through Tor exit nodes. Slow but highly resistant. + +5. **Encrypted Client Hello (ECH)** — hides the SNI field in TLS handshakes. + Still emerging; not widely supported. + +## So Is This Project Necessary? + +**Honestly: no, not for its current functionality.** + +If the goal is "use encrypted DNS from Linux in a censored region," install +`cloudflared` or `dnscrypt-proxy` + a Warp/WireGuard tunnel. These are +battle-tested, maintained by teams, and packaged in every distro. + +## Where This Project Becomes Useful + +This project becomes **genuinely valuable** if it evolves beyond a basic +forwarder. Directions that matter: + +### 1. Upstream obfuscation (the real differentiator) + +Route upstream DoH traffic through a proxy (SOCKS5, HTTP CONNECT, or +WireGuard tunnel) instead of direct HTTPS. This bypasses SNI and IP +blocking: + +``` +Local DNS → this forwarder → SOCKS5 proxy → DoH resolver +``` + +No existing lightweight Rust tool does this well. `dnscrypt-proxy` has +anonymized relays but no SOCKS5 support for DoH. + +### 2. Pluggable transport layer + +Support multiple upstream transports: +- **Direct DoH** (current — works in free internet) +- **DoH over SOCKS5** (works behind blocks) +- **DoH over WireGuard tunnel** (works behind aggressive DPI) +- **DNS-over-TLS** (for resolvers that support it) + +The user picks the transport based on their environment. + +### 3. Smart upstream selection + +Don't just round-robin — **probe** upstreams for latency and availability. +Auto-switch when a resolver becomes unreachable. This matters when resolvers +get blocked intermittently. + +### 4. Built-in blocklist support + +Ad-blocking and malware-blocking at the DNS level. `dnscrypt-proxy` and +`AdGuard Home` do this, but a lightweight Rust binary with this built-in +is appealing for embedded/resource-constrained systems. + +### 5. Learning + +Even if you never deploy it, building this teaches: +- DNS wire format (RFC 1035) +- DoH protocol (RFC 8484) +- Rust async (Tokio, Axum) +- Network programming +- Error handling patterns + +That alone justifies the project. + +## Summary + +| Question | Answer | +|---|---| +| Does this solve local DNS → DoH? | Yes, but `cloudflared` already does this | +| Does this bypass censorship? | No — upstream DoH endpoints get blocked | +| Is the project pointless? | No — it's a foundation for something better | +| What makes it worth building? | SOCKS5/proxy upstream support, pluggable transports, learning | + +The project is a **starting point**, not a finished solution. The first 80% +(local DNS → DoH forwarding) is solved by existing tools. The last 20% +(bypassing upstream blocking) is where this project can differentiate. diff --git a/pkg/arch/PKGBUILD b/pkg/arch/PKGBUILD index 6161eb7..06e8dbe 100644 --- a/pkg/arch/PKGBUILD +++ b/pkg/arch/PKGBUILD @@ -4,12 +4,12 @@ pkgver=0.1.0 pkgrel=1 pkgdesc="A lightweight DNS-over-HTTPS forwarder" arch=('x86_64') -url="https://gitea.logi.camp/LogiCrew/doh-forwarder" +url="https://git.logicamp.dev/mohamad/doh-forwarder" license=('MIT') depends=('gcc-libs') makedepends=('rust' 'cargo') backup=('etc/doh-forwarder/config.toml') -source=("$pkgname-$pkgver.tar.gz::https://git.logicamp.dev/LogiCrew/doh-forwarder/archive/v${pkgver}.tar.gz" +source=("$pkgname-$pkgver.tar.gz::https://git.logicamp.dev/mohamad/doh-forwarder/archive/v${pkgver}.tar.gz" 'doh-forwarder.service' 'doh-forwarder.sysusers' 'doh-forwarder.conf')