Help with HTTP Proxy Interceptor in Ruby?
I have the beginning of an HTTP interceptor written in Ruby:
require 'socket' # Get sockets from stdlib
server = TCPServer.open(8080) # Socket to listen on port 8080
loop { # Servers run forever
Thread.start(server.accept) do |client|
puts "** Got connection!"
@output = ""
@host = ""
@port = 80
while line = client.gets
line.chomp!
if (line =~ /^(GET|CONNECT) .*(\.com|\.net):(.*) (HTTP\/1.1|HTTP\/1.0)$/)
@port = $3
elsif (line =~ /^Host: (.*)$/ && @host == "")
@host = $1
end
print line + "\n"
@output += line + "\n"
# This *may* cause problems with not getting full requests,
# but without this, the loop never returns.
break if line == ""
end
if (@host != "")
puts "** Got host! (#{@host}:#{@port})"
out = TCPSocket.open(@host, @port)
puts "** Got destination!"
out.print(@output)
while line = out.gets
line.chomp!
if (line =~ /^<proxyinfo>.*<\/proxyinfo>$/)
# Logic is done here.
end
print line + "\n"
client.print(line + "\n")
end
out.close
end
client.close
end
}
This simple proxy I made parses the assignment from the HTTP request, then reads the HTTP response and executes the logic based on custom HTML tags. The proxy works for the most part, but seems to have problems with binary data and HTTPS connections.
How can I fix these problems?
a source to share
First, you are probably better off building an existing Ruby HTTP proxy implementation. One of them is already available in the Ruby standard library, namely WEBrick :: HTTPProxyServer . See for example this related question for an implementation based on this same class: Webrick transparent proxy .
As far as HTTPS proxying goes, you can't do much more than just transfer raw bytes. Because HTTPS is cryptographically secure, you cannot validate content at the HTTP protocol level. It's just an opaque stream of bytes.
a source to share
WEBrick is blocking I / O ... This means that it is not capable of transmitting a response. For example, if you go to a YouTube page to see a video, the stream will not be redirected to your browser until the proxy has downloaded all the video quotes. If you want the video to play in your browser while it is loading, you need to look for a non-blocking I / O solution like EventMachine. For HTTPS, the solution is a bit tricky as you need to evolve the person into an average proxy.
a source to share
This was an old question, but there is another answer here for completeness.
I have implemented an HTTP / HTTPS interception proxy in Ruby, the project is hosted on github .
The HTTP case is obvious, HTTPS interception is done through an HTTPS server that acts as a reverse proxy (and handles the TLS handshake). I.e.
Client(e.g. Browser) <--> Proxy1 <--> HTTPS Reverse Proxy <--> Target Server
As Valko pointed out, when a client connects to an HTTPS server through a proxy server, you will see a stream of encrypted bytes (since SSL provides end-to-end encryption). But not everything is encrypted, the proxy needs to know who needs to send the byte stream, so the client issues a request CONNECT host:port
(which is the body of the SSL stream request).
The trick here is that the first proxy redirects this request to the HTTPS Reverse Proxy
actual target server instead. This reverse proxy will handle the SSL negotiation with the client, access the decrypted requests, and send copies (optionally modified versions) of those requests to the real target server, acting like a normal client. It will receive responses from the target server (optional), modify the responses, and send them back to the client.
a source to share