The goal is to implement a realistic HTTP HTTPS proxy server, and there are currently two ways to implement a proxy

Plain Proxy : This proxy plays the role of a middleman, for the client it is the server, for the server it is the client, it is responsible for passing HTTP messages back and forth in the middle

Tunnel Proxy : It is a proxy that is done through the HTTP body, which is a TCP-based application layer proxy that uses the CONNECT method of HTTP to establish a connection.

This is an HTTP request, with \r\n for line feed, and the request data after two consecutive \r\ns are encountered, divided into the following parts

  1. request line
  2. request header
  3. empty line
  4. request data (body)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

curl -Lv http://baidu.com

> GET / HTTP/1.1
> Host: baidu.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 25 Jun 2020 12:12:32 GMT
< Server: Apache
< Last-Modified: Tue, 12 Jan 2010 13:48:00 GMT
< ETag: "51-47cf7e6ee8400"
< Accept-Ranges: bytes
< Content-Length: 81
< Cache-Control: max-age=86400
< Expires: Fri, 26 Jun 2020 12:12:32 GMT
< Connection: Keep-Alive
< Content-Type: text/html
<
<html>
<meta http-equiv="refresh" content="0;url=http://www.baidu.com/">
</html>

Realization

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

package main

import (
    "io"
    "log"
    "net"
    "net/http"
    "time"
)

func handleTunneling(w http.ResponseWriter, r *http.Request) {
    destConn, err := net.DialTimeout("tcp", r.Host, 10*time.Second)
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    w.WriteHeader(http.StatusOK)
    hijacker, ok := w.(http.Hijacker)
    if !ok {
        http.Error(w, "Hijacking not supported", http.StatusInternalServerError)
        return
    }
    clientConn, _, err := hijacker.Hijack()
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
    }
    go transfer(destConn, clientConn)
    go transfer(clientConn, destConn)
}

func transfer(destination io.WriteCloser, source io.ReadCloser) {
    defer destination.Close()
    defer source.Close()
    io.Copy(destination, source)
}

func handleHTTP(w http.ResponseWriter, req *http.Request) {
    resp, err := http.DefaultTransport.RoundTrip(req)
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    defer resp.Body.Close()
    copyHeader(w.Header(), resp.Header)
    w.WriteHeader(resp.StatusCode)
    io.Copy(w, resp.Body)
}

func copyHeader(dst, src http.Header) {
    for k, vv := range src {
        for _, v := range vv {
            dst.Add(k, v)
        }
    }
}

func main() {
    server := &http.Server{
        Addr: ":8888",
        Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            if r.Method == http.MethodConnect {
                handleTunneling(w, r)
            } else {
                handleHTTP(w, r)
            }
        }),
        // Disable HTTP/2.
        TLSNextProto: make(map[string]func(*http.Server, *tls.Conn, http.Handler)),
    }

    log.Fatal(server.ListenAndServe())
}

The above code is not applicable to production environment

The above code implements two methods, a normal proxy and a tunnel proxy, to distinguish between different processing logic by determining whether the requesting party uses the CONNECT method to request a proxy server.

1
2
3
4
5
6
7

http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    if r.Method == http.MethodConnect {
        handleTunneling(w, r)
    } else {
        handleHTTP(w, r)
    }
})

The logic based on the normal proxy is easy to understand, here is a look at how to implement the tunnel-based proxy, handleTunneling The first step is to establish a connection with the target server

1
2
3
4
5
6

destConn, err := net.DialTimeout("tcp", r.Host, 10*time.Second)
if err != nil {
    http.Error(w, err.Error(), http.StatusServiceUnavailable)
    return
 }
 w.WriteHeader(http.StatusOK)

The next step is to hijack the connection maintained by the HTTP service, of type net.Conn

1
2
3
4
5
6
7
8
9

hijacker, ok := w.(http.Hijacker)
    if !ok {
        http.Error(w, "Hijacking not supported", http.StatusInternalServerError)
        return
    }
    clientConn, _, err := hijacker.Hijack()
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
    }

After Hijacker interface gets the connection, the caller maintains the connection and the HTTP standard library is not responsible for managing it

At this point, two TCP connections have been established (client -> proxy, proxy -> target server), and the last step is to forward their messages to each other

1
2

go transfer(destConn, clientConn)
go transfer(clientConn, destConn)

Test

Use Chrome:

1

Chrome --proxy-server=https://localhost:8888

Use Curl:

1

curl -Lv --proxy https://localhost:8888 --proxy-cacert server.pem https://baidu.com