Zola's Blog Where I share everything that Im interested in Rss | Home

RawLink: Building a VPN Protocol From Scratch

I always wanted to give making a new VPN/proxy protocol a shot to understand what happens at the packet level to evade censorship or make a secure tunnel to a server, and to see what it takes to actually design and implement one from scratch.

What I wanted to make was something very simple, something without too much overhead and complication that would still result in a reliable encrypted tunnel that could bypass DPI (deep packet inspection) firewalls and be used as a censorship circumvention tool.

But little did I know how much more complicated it would get than I initially intended. The goal was simple, though: have a simple client-server model in which the user would connect to the client through a SOCKS5 proxy, and the client would send encrypted packets through the established session, encrypting everything with a pre-shared key to keep things simple.

So, with the help of AI (a GLM 5.3-flash model, if you’re wondering), I got to coding, tweaking my designs as I implemented them.

The project became RawLink.

What started as a simple encrypted TCP tunnel eventually grew into a protocol with packet framing, multiplexed streams, authenticated encryption, replay protection, header masking, traffic obfuscation, SSH protocol mimicry, backpressure, connection pooling, and failure detection.

Here, I wanted to write a blog post to explain what’s going on with the protocol and how it works, in case anyone ends up using it, and to share my experience developing something with the assistance of AI.

The original idea

At its core, RawLink follows a client-server model.

The client runs a local SOCKS5 proxy. Applications connect to it as they would connect to any other SOCKS5 proxy, and the client forwards those connections through an encrypted tunnel to a remote server. The server then opens the requested connections and relays data in both directions.

Browser / Application
        |
        | SOCKS5
        v
   RawLink Client
        |
        | Encrypted TCP Tunnel
        |
   RawLink Server
        |
        | Ordinary TCP Connections
        v
   Destination Servers

I chose SOCKS5 because it’s very simple. Using it, the client doesn’t need to understand HTTP or the application-layer protocols being transported. It simply knows where a connection should go and how to move bytes between the local socket and the remote server.

I chose TCP as the carrier protocol because it already handles reliable delivery, ordering, retransmission, and congestion control. Implementing those mechanisms myself would lead to something really messy, unreliable, and overcomplicated, and it wouldn’t be as good.

But it still came with trade-offs, of course. TCP introduces head-of-line blocking, and tunnelling multiple connections through a single TCP connection can make the effects of packet loss much worse.

Beyond encryption

Well, we all know that encryption alone is not enough to bypass sophisticated censorship firewalls. Nowadays, firewalls can identify patterns in encrypted traffic and block it with ease.

Encryption should be there to protect the contents of a connection, though, but it doesn’t automatically hide the fact that a particular protocol is being used. A DPI system still checks packet sizes, timing, framing, handshake patterns, and other characteristics without decrypting a single byte.

For the encryption, I went with AES-256-GCM since most hardware has built-in hardware acceleration for it, and the GCM mode provides authentication for encrypted packets without me having to implement it from scratch.

So, I knew that I needed something beyond just encryption, something to either make the packets look like nothing in particular or make them look like something ordinary (like TLS or, in this case, SSH).

And I ended up with an optional obfuscation layer, but I decided to keep it separate in the architecture. RawLink’s core protocol would handle streams, packets, encryption, and session management, while the optional obfuscation layer would change how those packets are carried over the network, allowing me to add more obfuscation schemes later.

Designing the packet format

The next step was deciding what a packet should actually look like.

I needed enough information in each packet to identify the session, determine the packet’s position in the sequence, and decrypt the payload. At the same time, I wanted to keep the fixed overhead small.

What I ended up with was a packet format that uses a 17-byte header followed by an encrypted body and an authentication tag.

The header contains a protocol version, a key-phase bit, reserved bits, a session identifier, and a packet number. The encrypted body contains the actual protocol frames.

The version field allows incompatible protocol revisions to reject each other. Rather than letting different implementations interpret the same bytes in different ways, a peer that doesn’t understand the version can reject the packet.

The session identifier serves two purposes. It lets the server identify the session, and it contributes to deriving the session’s cryptographic keys. This means that two tunnels using the same PSK don’t automatically end up using identical session keys.

The packet number increments with every packet sent in a particular direction. It gives the receiver a way to track packet progression and provides the input needed to construct a unique nonce for each packet, which is needed for GCM encryption.

At first, I had to keep the header readable because the receiver needs some of this information before it can decrypt the body. That decision would eventually create another problem, which I’ll get to later.

Authenticated encryption

As I said, I ended up with AES-256-GCM for encrypting the packets.

What matters here is the GCM mode, which provides authenticated encryption, meaning the receiver can verify that the ciphertext and the authenticated metadata have not been modified. This is very important for preventing tampering when transferring data.

The header is supplied as additional authenticated data (AAD). Although AAD isn’t encrypted by the AEAD operation, it is bound to the authentication tag. Changing an authenticated header field therefore causes verification to fail.

RawLink also derives separate keys for client-to-server and server-to-client traffic using HKDF-SHA256, with the session identifier incorporated into the derivation.

I kept the directions separate to avoid sharing the same key and nonce space between both sides. This is really important with AES-GCM, where reusing a key-nonce pair can compromise security.

The nonce is generated from a per-direction salt and the packet number. Since packet numbers advance monotonically, the protocol can generate nonces without negotiating a new random nonce for every packet.

Why use a pre-shared key?

I just didn’t want to get into complicated handshakes, key derivation, and certificates.

A 32-byte PSK gives the client and server a shared secret from which session-specific keys can be derived. It keeps configuration straightforward and avoids introducing certificate infrastructure just to establish the tunnel.

Well… It wasn’t that great, and it had its downsides, since key distribution now becomes the user’s responsibility. It also meant no forward secrecy, which kinda sucks, but I’m okay with it.

Multiplexing connections with frames

A browser doesn’t make just one connection when loading a website. It may open multiple connections to fetch HTML, images, scripts, and other resources.

Creating a separate tunnel for every application connection would be wasteful. Instead, RawLink multiplexes multiple logical streams over one established TCP connection.

To do that, I introduced a framing layer. This way, each frame is a small, typed message containing a type, a body length, and a body. Different frame types represent different protocol events, including opening a stream, transmitting data, closing a stream, resetting a stream, requesting a remote TCP connection, and sending keepalives.

This way, I could fit multiple frames into one packet, allowing the sender to batch several operations together rather than paying the fixed packet overhead for every small piece of data. Like so:

Application Connections
        |
      Streams
        |
       Frames
        |
      Packets
        |
  Carrier / Obfuscation
        |
       TCP

In this model, each layer does a specific thing. The stream layer manages application connections, the frame layer represents protocol operations, the packet layer handles encryption and authentication, and the carrier layer handles transmission over TCP.

This model would also make debugging easier. If a stream fails to close correctly, the problem can be investigated at the stream and frame layers, narrowing down the area that needs debugging.

Why stream offsets?

RawLink’s DATA frames carry a stream identifier, an offset, and a payload.

TCP already guarantees ordered delivery, so at first glance, explicit offsets might seem unnecessary. But offsets provide a way to track how much data has been transmitted and received on each logical stream independently.

A close frame includes the final offset, allowing the receiver to check whether the expected amount of stream data has arrived before considering the stream complete.

This gives the stream protocol an explicit notion of completeness rather than relying only on the underlying socket’s lifecycle.

Limits on the DATA frame

During implementation, I had to limit individual DATA frames to 16 KiB.

When I was testing, a large transfer would continuously produce data, while another connection would only need to send a few small messages. If bulk data dominated the sender’s work, smaller operations could experience unnecessary delays. This was what was happening, and it was really, really annoying to figure out what was causing it.

Splitting bulk transfers into smaller frames gave the session more opportunities to process other queued work between chunks.

And 16 KiB wasn’t a magical number, either. It just gives the implementation a unit of work rather than allowing a single enormous data frame to dominate all the processing.

The header problem

At this point, RawLink had authenticated encryption, separate directional keys, session-specific key derivation, and a framing protocol, and it was kind of working fine.

But the header was still readable.

The packet number seemed a bit problematic to me. The first packet in a direction would carry one number, the next would carry the next, and so on. A DPI system could potentially recognize that progression even though the packet body looked like random ciphertext.

I needed to hide the header without breaking the receiver’s ability to process packets.

The solution I came up with was to add a header-masking layer using an XChaCha20 keystream.

Instead of sending the header directly, RawLink would XOR-mask its bytes before putting the packet on the wire. The receiver reverses the operation before attempting to decrypt the packet body.

This didn’t provide any security benefits for the protocol, but it changed how the header would appear to someone observing it.

It wasn’t all that simple, though. The server needs the session identifier to derive the session-specific masking keys, but the session identifier is itself inside the header that must be unmasked. The server can’t use information it hasn’t recovered yet.

The fix I came up with was a two-stage key schedule. The first client packet uses masking material derived from the PSK alone. Once the server unmasks that initial packet and recovers the session identifier, subsequent packets can use masking keys derived from the PSK, session identifier, and direction.

This lets the protocol move from an initial state into a session-specific state without adding a separate handshake round trip.

The session’s deadlock problem

Encrypting the packets was one piece of the puzzle in building a reliable tunnel. The session implementation also needed to behave sensibly when many streams were active, buffers filled up, or one side stopped responding.

To achieve this, RawLink uses two main threads per session: a receiver and a sender.

The receiver reads incoming data, processes the carrier framing, unmasks and decrypts packets, checks replay protection, and dispatches frames to the appropriate streams.

The sender, meanwhile, owns outbound work. It collects queued stream commands, constructs frames and packets, encrypts them, and writes them to the carrier.

The sender sleeps until work becomes available and is woken through a condition variable rather than continuously polling.

Dividing the work this way keeps the receive path from having to perform unrelated send-side operations. It also gives outbound packet construction and socket writes a single owner, reducing the number of places that need to coordinate access to session state.

Backpressure

Each stream has a limited send buffer, with a maximum of 4 MiB of unsent data. When the buffer fills, the application writing into that stream experiences backpressure instead of allowing memory usage to grow indefinitely.

I had to go with this model to avoid implementing a separate acknowledgement-window protocol (which, at this point, I didn’t even want to think about) just to regulate how much application data can accumulate in userspace. At the same time, I had to fix the annoying reliability issues I was having when testing with real data.

The point was that the application shouldn’t be allowed to enqueue unlimited data while the network or remote endpoint is stalled.

Bounded queues make overload visible instead of turning it into infinite memory growth.

Knowing when a connection has died

A tunnel can fail without either endpoint receiving a clean close notification.

A cable can be unplugged. A firewall can silently discard packets. The TCP socket might remain apparently open even though communication is no longer possible.

Without some form of liveness check, a dead tunnel can remain around indefinitely.

I had to introduce PING and PONG frames to detect this situation. While no other traffic is being received, the sender schedules probes at randomized intervals between 1.2 and 3.6 seconds.

The session considers the connection dead when there has been ten seconds of receive silence and at least one probe remains unanswered.

The implementation also uses randomized keepalive timing and variable padding. This avoids making every idle connection send an identical, perfectly periodic message.

Making the client recover automatically

I didn’t want users to have to manually reconnect every time the remote server restarted or a temporary network failure occurred. And temporary network failures were occurring a lot during testing, which was pretty annoying.

So I came up with a tunnel manager responsible for maintaining the pool of outbound sessions.

When a tunnel dies, the manager attempts to establish another one using backoff with increasing delays. The delay increases after repeated failures, while randomization prevents many clients from reconnecting at exactly the same time after a shared outage.

A session that stays alive for a while resets the backoff to its minimum, so a temporary failure doesn’t leave the client waiting unnecessarily long during the next recovery attempt.

The manager can also open additional tunnels when existing ones approach their stream limits. New streams are routed to the least-loaded available tunnel, and surplus tunnels can be retired after a period of inactivity.

This way, streams are the units of application work, while tunnels are the units of transport failure.

What happens when strangers connect?

When exposing any port to the internet, people WILL try to connect to it. And not all of them will be legitimate users. They might be scanners or people just trying to exhaust resources.

RawLink limits how many unauthenticated connections can be held simultaneously and requires a connection to provide an authenticated packet within a short admission window. Authenticated sessions are subject to a separate concurrency limit.

This matters because cryptographic validation isn’t free, and maintaining a large number of incomplete connections consumes resources even if none of them successfully authenticate.

If a wrong PSK causes an immediate close while an idle connection remains open for several seconds, an observer can potentially distinguish those cases. That behavior can give away a lot of information about the server’s authentication process.

Because of that, RawLink attempts to make admission failures close within a randomized time window, including failures involving timeouts, malformed data, incorrect keys, and carrier errors.

This one might be overkill, and it probably is, but it’s what I ended up with.

The censorship problem

At this point, RawLink was connecting and forwarding traffic, but that still didn’t make it a circumvention tool, so I had to design and implement the obfuscation layer.

And I ended up with two completely different modes to do so.

Making traffic look random

The first mode, rand, was designed to make the connection look like a stream of random bytes rather than a recognizable custom protocol.

It uses a handshake built around fresh random values and a message authentication code derived from the PSK. The handshake has a fixed size, including when the server responds to an invalid probe.

After the handshake, the carrier uses a masked framing header and a per-connection keystream. Frames can also contain random padding, so the size of a transmitted frame doesn’t directly reveal the exact size of its application payload.

I also included optional dummy traffic during idle periods. Without it, a quiet tunnel might produce no packets for a while and then suddenly resume transmitting application data. Depending on the surrounding traffic patterns, that behavior can itself provide a useful signal to an observer.

One issue with this was that padding and dummy traffic consume bandwidth, and randomness alone doesn’t make a protocol indistinguishable from every other protocol.

This mode as a whole was just an attempt to reduce protocol fingerprinting, but it still wasn’t something I would be happy with. An observer may still classify a connection based on packet-size distributions, directionality, timing, or other signals.

Mimicking SSH

With this mode, rather than making the connection look like noise, I tried to make the initial exchange resemble an ordinary OpenSSH session.

A custom protocol that looks like an entirely new transport stands out, whereas a connection that follows a familiar protocol’s handshake has a more plausible explanation for its existence.

The implementation mimics the SSH identification exchange and key-exchange message sequence, including the transition into the encrypted phase. The RawLink handshake then takes place in the post-key-exchange portion of the connection.

Here, the design goal is to reproduce the observable structure that an SSH-aware observer expects to see, rather than merely putting an SSH-looking string at the beginning of an otherwise custom connection.

Also, mimicking SSH is only useful where SSH-like traffic is permitted and sufficiently common to blend into the surrounding network. A network that blocks SSH, restricts it to approved servers only, or uses behavioral analysis may still identify or block the tunnel.

Well, it’s nothing special now that I look at it, but it was still something interesting to build, even if AI was involved.

I don’t usually generate code with AI because sometimes it’s easier to write code than to read someone else’s code, and for a good part of the project, it felt like that. But I eventually got the hang of it.

This project became much larger than I intended, but it surely taught me a lot about networks and how censorship systems work. But a large project is not necessarily a good one, and this project is no exception. The security of the protocol is not proven yet, the encryption hasn’t been audited by anyone besides me, and it lacks a lot of features, like perfect forward secrecy and so on.

If you’re interested in seeing how it turned out, I’ve published it as free and open-source software on GitHub, and I appreciate any feedback or improvements to the project: