While preparing another blog post, I realized that the networking part of the
code was quickly becoming larger than the part of the code that was meant to
illustrate the topic of the post. So I decided to write one blog post
entirely about how to send data from process A to process B over a plain
Who needs sending things at TCP/IP level?
Granted, many, if not most, scenarios, undoubtedly do better with a higher-level
network protocol that hides all the technical details beneath a fancy API.
And there are already plenty to choose from, depending on the needs:
Message queue protocols, gRPC, protobuf, FlatBuffers, RESTful Web API’s,
WebSockets, and so on.
However, in some situations–especially with small projects–, any approach
you choose may look like completely oversized, not to mention the additional
package dependencies that you’d have to introduce.
Luckily, creating simple network communication with
is not as difficult as it may seem.
Simplification #1: connections are io streams
net.Conn interface implements the
interfaces. Hence you can use a TCP connection
I know what you think – “Ok, so I can send strings or byte slices over a
TCP connection. That’s nice but what about complex data types? Structs and such?”
Simplification #2: Go knows how to encode complex types efficiently
(Did you also just read “God knows…”? I think it happens to me almost every
other time I read this text.)
When it comes to encoding structured data for sending over the net, JSON comes
readily to mind. But wait – Go’s standard
encoding/gob package provides
a way of serializing and deserializing Go data types without the need for
adding string tags to structs, dealing with JSON/Go incompatibilities, or waiting
for json.Unmarshal to laboriously parse text into binary data.
Gob encoders and decoders work directly on
io streams – and this fits just
nicely into our simplification #1 – connections are
Let’s put this all together in a small sample app.
The sample app’s goal
The app shall do two things:
- Send and receive a simple message as a string
- Send and receive a
The first part–sending simple strings–shall demonstrate how easy it is
to send data over a TCP/IP network without any higher-level protocols.
The second part goes a step further and sends a complete struct over the
network, with strings, slices, maps, and even a recursive pointer to the
Thanks to the
gob package, this requires no efforts. The following
animation shows how gob data gets from a client to a server, and when
this looks quite unspectacular, it’s because using
gob is unspectacular.
It’s not much more than that!
Basic ingredients for sending string data over TCP
On the sending side
Sending strings requires three simple steps.
- Open a connection to the receiving process
- Write the string
- Close the connection
net package provides a couple of methods for this.
ResolveTCPAddr() takes a string representing a TCP address (like, for example,
[::1]:80, which all represent port #80 on
the local machine) and returns a
net.TCPAddr (or an error if the string
cannot be resolved to a valid TCP address).
DialTCP() takes a
net.TCPAddr and connects to this address. It returns
the open connection as a
net.TCPConn object (or an error if the connection
If we don’t need much fine-grained control over the Dial settings, we can use
net.Dial() instead. This function takes an address string directly and
returns a general
net.Conn object. This is sufficient for our test case;
however, if you need functionality that is only available on TCP connections,
you have to use the “TCP” variants (
After successful dialing, we can treat the new connection like any other
input/output stream, as mentioned above. We can even wrap the connection into
bufio.ReadWriter and benefit from the various
ReadWriter methods like
** Remember that buffered Writers need to call
Flush() after writing,
so that all data is forwarded to the underlying network connection.**
Finally, each connection object has a
Close() method to conclude the
A couple of tuning options are also available. Some examples:
Dialer interface provides these options (among others):
Timeoutoptions for timing out an unsuccessful dial;
KeepAliveoption for managing the life span of the connection
Conn interface also has deadline settings; either for the connection as
a whole (
SetDeadLine()) or specific to read or write calls (
Note that the deadlines are fixed points in (wallclock) time. Unlike timeouts,
they don’t reset after a new activity. Each activity on the connection must
therefore set a new deadline.
The sample code below uses no deadlines, as it is simple enough so that we can
easily see when things get stuck.
Ctrl-C is our manual “deadline trigger
On the receiving side
The receiver has to follow these steps.
- Start listening on a local port.
- When a request comes in, spawn a goroutine to handle the request.
- In the goroutine, read the data. Optionally, send a response.
- Close the connection.
Listening requires a local port to listen to. Typically, the listening
application (a.k.a. “server”) announces the port it listens to, or if it
provides a standard service, it uses the port associated with that service.
For example, Web servers usually listen on port 80 for HTTP requests and
on port 443 for HTTPS requests. SSH daemons listen on port 22 by default,
and a WHOIS server uses port 43.
The core parts of the
net package for implementing the server side are:
net.Listen() creates a new listener on a given local network address. If
only a port ist passed, as in “:61000”, then the listener listens on
all available network interfaces. This is quite handy, as a computer usually
has at least two active interfaces, the loopback interface and at least one
real network card.
Accept() method waits until a connection request comes in.
Then it accepts the request and returns the new connection to the caller.
Accept() is typically called within a loop to be able to serve multiple
connections simultaneously. Each connection can be handled by a goroutine,
as we will see in the code.
Instead of just pushing a few bytes around, I wanted the code to demonstrate
something more useful. I want to be able to send different commands with
different data payload to the server. The server shall identify each
command and decode the command’s data.
So the client in the code below sends two test commands: “STRING” and “GOB”.
Each are terminated by a newline.
The STRING command includes one line of string
data, which can be handled by simple read and write methods from
The GOB command comes with a
struct that contains a couple of fields,
including a slice, a map, and a even a pointer to itself. As you can see when
running the code, the
gob package moves all this through our network
connection without any fuss.
What we basically have here is some sort of ad-hoc protocol, where the client
and the server agree that a command is a string followed by a newline followed
by some data. For each command, the server must know the exact data format
and how to process the data.
To achieve this, the server code takes a two-step approach.
Step 1: When the
Listen() function accepts a new connection, it spawns
a new goroutine that calls function
handleMessage(). This function reads
the command name from the connection, looks up the appropriate handler
function from a map, and calls this function.
Step 2: The selected handler function reads and processes the command’s data.
Here is a visual summary of this process.
Keep these pictures in mind, they help reading the actual code.