Although we say that “Magic and unicorns are at the heart of R.O.B.E.R.T.”, that’s technically not true. In reality there are no unicorns, but there is some magic.
R.O.B.E.R.T. is a custom DNS proxy written in Go. The R.O.B.E.R.T. binary is deployed to every VPN server, and it sits in front of the recursive DNS server, handing DNS queries on behalf of all connected users.
Here is a simplified chart of how it works, I’ll discuss each phase in more detail below.
Phase 1 — Connection to the VPN server
When you press the ON button in our apps, or initiate the connection to a VPN server manually, a set of credentials are supplied with the connection attempt. The credentials are verified to make sure you are allowed to access the server. An RFC1918 IP is assigned to each connected user.
Phase 2 — Fetch Settings
The connection daemon makes a request to the API, fetching the R.O.B.E.R.T. settings for the newly connected user. Once the settings are received, R.O.B.E.R.T. is notified that a new client connected, and it applies the settings + custom rules to the specific LAN IP address which was assigned in Phase 1. The app should now be connected.
The rules are stored in R.O.B.E.R.T.’s memory to allow for high speed look-ups, since a look-up will need to be done for every single DNS query, and every microsecond counts. We’ve been able to achieve ~20,000 queries per second, with an average look-up time of 5 microseconds from a single instance of R.O.B.E.R.T.
Phase 3 — Operation
You request to load reddit.com. Your computer needs to know the IP address of the website, so it sends a DNS query over the tunnel to R.O.B.E.R.T.
From R.O.B.E.R.T.’s perspective, all DNS queries originate from random LAN IP addresses which are assigned when a user connects, not to be confused with your actual WAN source IP, which is not a factor, since R.O.B.E.R.T. doesn’t have access to it.
R.O.B.E.R.T. looks up the in-memory blocklist settings to see if there are rules for this domain. If there is a BLOCK rule, R.O.B.E.R.T. spoofs the response and returns 0.0.0.0 and ::/0 (ipv6 equivalent). Your browser tries to connect to the IP address, which fails. The block is enforced!
If there is no matching rule for this domain, in the house block lists, custom block lists, or you chose to whitelist it, R.O.B.E.R.T. proxies the DNS query upstream, to the DNS resolver (BIND) running on the same machine. It recursively resolves the domain from the authoritative servers, and responds with the IP address(es) of the domain. Your OS/browser then initiates the connection to the IP and downloads the content.
Phase 4 — Live setting updates
When you makes changes to your R.O.B.E.R.T. settings, they get stored in our database. Any subsequent connection attempt will fetch them, and apply them during the connection process. But how do you update them in real time, for all your connections at that exact movement? This is where the magic comes in.
If you recall our Privacy Policy, you remember that we store the number of parallel connections you have at any given movement (this does not include your source IP). This data is used to prevent abuse, like account sharing, but this also allows us to do some pretty neat things, like instantly update your R.O.B.E.R.T. settings for all your existing connections in real time, globally.
After you make the changes on our website, a footer bar (it’s actually called a snackbar for some reason) appears asking you if you want to apply the changes immediately. This hits our API, which sends a request to R.O.B.E.R.T. on all the servers you’re currently connected to and supplies the new settings. R.O.B.E.R.T. updates its internal in-memory store and begins enforcing the new rules. This process takes anywhere between 50ms and 300ms, depending on how far away the server is from our central API.
