Recent posts:
Blog index

Brendan Gregg's Blog


Linux bcc/BPF tcplife: TCP Lifespans

30 Nov 2016

"i really wish i had a command line tool that would give me stats on TCP connection lengths on a given port"

# ./tcplife -D 80
PID   COMM       LADDR           LPORT RADDR           RPORT TX_KB RX_KB MS
27448 curl   54146  80        0     1 263.85
27450 curl   20618   80        0     1 243.62
27452 curl   11480   80        0     1 231.16
27454 curl   31382     80        0     1 249.95
27456 curl   33416   80        0     1 545.72
27458 curl   16406    80        0     1 222.29
27460 curl   11634   80        0     1 217.52
27462 curl   25660   80        0     1 250.81

That's tracing destination port 80, you can also trace local port (-L), or trace all ports (default behavior).

The quote and good idea is from Julia Evans on twitter, and I added it as a tool to the Linux BPF-based bcc open source collection.

The output of tcplife, short for TCP lifespan, shows not just the duration (MS == milliseconds) but also throughput statistics: TX_KB for Kbytes transmitted, and RX_KB for Kbytes received. It should be useful for performance and security analysis, and network debugging.

I am NOT tracing packets

The overheads can cost too much to examine every packet, especially on servers that process millions of packets per second. To do this I'm not using tcpdump, libpcap, tethereal, or any network sniffer.

So how'd I do it? There were four challenges:

1. Measuring lifespans

2. Fetching addresses and ports

3. Fetching throughput statistics

4. Showing task context

That's how tcplife works right now. It the future it may change and improve, especially if more TCP tracepoints become available. I'm also not the first to try this kind of TCP event tracing: Facebook already have their own BPF TCP event tool, although I think their implementation is a little different.

Here's the full USAGE message for tcplife:

# ./tcplife -h
usage: tcplife [-h] [-T] [-t] [-w] [-s] [-p PID] [-L LOCALPORT]
               [-D REMOTEPORT]

Trace the lifespan of TCP sessions and summarize

optional arguments:
  -h, --help            show this help message and exit
  -T, --time            include time column on output (HH:MM:SS)
  -t, --timestamp       include timestamp on output (seconds)
  -w, --wide            wide column output (fits IPv6 addresses)
  -s, --csv             comma seperated values output
  -p PID, --pid PID     trace this PID only
                        comma-separated list of local ports to trace.
                        comma-separated list of remote ports to trace.

    ./tcplife           # trace all TCP connect()s
    ./tcplife -t        # include time column (HH:MM:SS)
    ./tcplife -w        # wider colums (fit IPv6)
    ./tcplife -stT      # csv output, with times & timestamps
    ./tcplife -p 181    # only trace PID 181
    ./tcplife -L 80     # only trace local port 80
    ./tcplife -L 80,81  # only trace local ports 80 and 81
    ./tcplife -D 80     # only trace remote port 80

And there is more example output in bcc, along with a man page. For more about bcc and BPF (eBPF), see the collection of documents on my Linux performance page.

The only catch is that tcplife does need newer kernels (say, 4.4).

You can comment here, but I can't guarantee your comment will remain here forever: I might switch comment systems at some point (eg, if disqus add advertisements).

comments powered by Disqus