Metadata-Version: 2.4
Name: bwninja
Version: 1.3.0
Summary: A modern replacement for iftop
Author-email: Tom Flanagan <tom@zkpq.ca>
Project-URL: Homepage, https://bwninja.zkpq.ca
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console :: Curses
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Telecommunications Industry
Classifier: License :: Free For Educational Use
Classifier: License :: Free For Home Use
Classifier: License :: Free for non-commercial use
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python
Classifier: Topic :: Internet
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Networking :: Monitoring
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: bcc!=0.1.10,>=0.26.0
Requires-Dist: tomli; python_version < "3.11"

# Bandwidth Ninja (bwninja)

A modern replacement for `iftop`.

Why?

- **Colors.** Local, private, global, and multicast or broadcast addresses have their own color. Ports have a color per service name (`https` is one color over TCP and UDP). A port without a name has the color of its range: orange below 1024 (system), white to 32767 (registered), grey above (ephemeral). Rates have a color per magnitude (`Bps`, `KiBps`, `MiBps`, ...) too.
- **One screen for all interfaces.** Each row shows the interface it was seen on.
- **Low overhead.** An eBPF program counts bytes in the kernel. No packet is copied to user space.

```
    Network flows, Bps over the past 30s. Rows 1-5 of 5
                                                  TOTAL 6054       Bps -->  <--         53.17     KiBps
⮞  1 ▾ ens18      [host.ens18] 10.87.1.31              1284       Bps -->  <--         51.18     KiBps one.one.one.one
   2                                          :43210   1204       Bps -->  <--         51.10     KiBps :https
   3                                          :43211     80.00    Bps -->  <--            80.00    Bps :https
   4 ▸ ens18      [host.ens18] 10.87.1.31                80.00    Bps -->  <--             0          10.87.1.1
   5   wg0        [host.wg0] 10.10.0.2:51820           3410       Bps -->  <--           941.0     Bps 10.10.0.1:51820
    q: quit    up/down/pgup/pgdn: select    enter: ports    n: names [pretty]    b: unit [Bps]    </>: sort [left address]    l: listen
```

The **total row** under the header sums the rates of every row. It stays
in place while the table scrolls. A packet seen on two interfaces counts
twice, as it does in the rows.

Each **host row** is two addresses on one interface, summed over all ports.
Both directions of a connection are in the same row. The rate next to an
address is what that address **sends**: the left number flows `-->` to the
right address, the right number flows `<--` to the left address. Rates are averaged over the past 30 seconds and shown with 4
significant digits. Bytes per second use binary prefixes (`Bps`, `KiBps`,
`MiBps`, ...). The `b` key cycles to bits per second with decimal prefixes
(`bps`, `kbps`, `Mbps`, ...), the network convention, and to packets per
second (`Pps`, `kPps`, ...). The digits line up by
the magnitude of the whole value: the integer part is right-aligned, each
prefix step moves that column 3 places left, and the decimals trail. A zero
rate shows as a dim `0` without a unit.

In the default `pretty` style, an address shows its reverse DNS name when
known, and a port shows its service name from `/etc/services` (`:https`) when
there is one. The colon and the port text are right-justified to six
characters (`  :ntp`, ` :8080`, `:27346`), so the ports line up down the
table. Multicast and broadcast addresses have no reverse DNS name, so
`bwninja` names them from the IANA multicast registries (`mdns`, `ssdp`,
`all-nodes`, `solicited-node`, ...) and names the broadcast addresses of the
local subnets `broadcast`. The `n` key cycles to `address:port` numbers, and
to both.

A host row with several port pairs has a fold marker (`▸` or `▾`). Enter
expands the selected host row. One **port row** per pair of ports appears
below the host row, with the ports next to the rates. Enter on an expanded
host row, or on one of its port rows, collapses the host row. A host row with
a single port pair shows the ports on the row itself and has no marker. A
host row with several port pairs shows the port that every pair has on a
side (`one.one.one.one:https`), or a dim `  :...` where the pairs differ.
A flow that is not TCP or UDP has no ports. The protocol name from
`/etc/protocols` stands in for both (`8.8.8.8:ICMP`), or `proto:` and the
protocol number in the `address:port` style (`8.8.8.8:proto:1`), so each
protocol between two hosts is its own port row. TCP and UDP flows on the
same port number are separate port rows too.

The **info panel** above the footer shows the selected row. Under the `Flow
info` bar, the details of the left endpoint and the details of the right
endpoint meet in the middle, four lines each: the host name with the protocol
and the service names of the ports of the row (`example.com TCP https`), the
address with the protocol number and the port numbers (`1.2.3.4 proto:6 443`),
the organization (AS) from [ipinfo.io](https://ipinfo.io), and the city,
region, and country.
An address of this host and a private address have no organization or place.
Below, a graph shows the rate of the row, one column per screen update, the
newest on the right. The two directions share a zero line: `rx`, what the
left address receives, grows up from the line, and `tx`, what the left address
sends, hangs down from it. Both have the same scale, so the proportion of the
directions is visible: the lowest 1, 2, or 5 of a prefix that covers the
largest rate, shown next to `±` as the rate of a full column. The rates next
to `rx` and `tx` are those of the last update, the rightmost column, not the
30 second average of the row. The `i` key
hides and shows the panel. The panel hides itself on a terminal too short for
it.

```
    Flow info
                                          home2 40950    ? https
                                     10.87.1.13 40950    162.159.140.220 443
                                     this host, vmbr0    AS13335 Cloudflare, Inc.
                                                         San Francisco California US
     ±  200 KiBps         ▂▂▂███▃▃▃▁▁▁▆▆▆▁▁▁▃▃▃▁▁▁███▂▂▂▁▁▁███▁▁▁▃▃▃▆▆▆
                       ▄▄▄██████████████████████████████████████████████
    rx 173.3 KiBps     ████████████████████████████████████████████████████
    tx  1182 Bps       ▀
```

The lookups at ipinfo.io run in the background and every answer is kept in
`~/.cache/bwninja/ipinfo.db`, so an address is looked up once. Without a
token the service allows about 50000 lookups a month. A failed lookup is kept
for a day, so an outage or a rate limit does not make `bwninja` hit the server
again and again.

The name style, the rate unit, the sort column, and the panel are saved in
`~/.config/bwninja.toml` and
are the start values of the next run. `bwninja` writes the file 15 seconds
after the last change, or when it quits. The file belongs to the user who
started `bwninja`, not to root. `XDG_CONFIG_HOME` moves the file.

```toml
names = "address"    # pretty, address, both
rates = "bits"       # bytes, bits, packets
sort = "send"        # iface, left, send, recv, right
info = true          # the info panel
```

The `l` key (listen) leaves the table and shows the traffic of the selected
row: the two addresses, and the two ports on a port row. Ctrl-c stops the
capture and returns to the table. The output stays in the terminal scrollback
after `bwninja` quits. When the name style is `address`, the capture tool gets
`-n` and looks up no names.

The capture tool is `tshark` when it is on the path, else `tcpdump`:

| Tool | Per packet |
|---|---|
| `tshark --color -P -O <protocols> -x` | a summary line in the Wireshark colors, one line each for the frame, ethernet, IP, and transport layers, every field of the protocol above them (DNS, TLS, HTTP, DHCP, ...), and the contents in hex and ASCII |
| `tcpdump -vvv -X` | the full `tcpdump` decode and the contents in hex and ASCII |

The protocols that `tshark` expands are in `TSHARK_PROTOCOLS` in `tui.py`.
`tshark` only prints colors on a terminal, and only when `COLORTERM` is
`truecolor`. `sudo` drops that variable, so `bwninja` sets it for `tshark`
when it is not set.

Rows are sorted by the left address: the addresses of this host first, then
other hosts from near (link-local, private) to far (global), then addresses
that are not one host (unspecified, multicast, broadcast). In a pair, the
address that sorts first is on the left, so a multicast group is always on the
right. A row keeps its place on screen while the rates change.

The `<` and `>` keys move the sort to the column on the left or on the right:
the interface, the left address, the left rate, the right rate, or the right
address. A rate column sorts the largest rate first, in the unit on screen, so
`Pps` sorts by packets. The port rows under an expanded host row follow the
same sort. The footer shows the sort column.


## Install

```sh
pip install bwninja
```

### bcc

`bwninja` needs the `bcc` python package. The version on PyPI is not usable,
so install the package of your distribution. On Debian and Ubuntu:

```sh
sudo apt install python3-bpfcc linux-headers-$(uname -r)
```

Proxmox VE ships its own kernel. On a Proxmox host, install the Proxmox
headers package instead of `linux-headers`:

```sh
sudo apt install python3-bpfcc proxmox-headers-$(uname -r)
```

For other systems see https://github.com/iovisor/bcc/blob/master/INSTALL.md

Requirements:

| Requirement | Why |
|---|---|
| Linux kernel >= 5.9 | batch lookup-and-delete on BPF hash maps |
| kernel headers for the running kernel | bcc compiles the eBPF program at start |
| `iproute2` (`ip` command) | lists the addresses of this host |
| `tshark`, or `tcpdump` as the fallback | the `l` key (listen) only |
| root | attaches the eBPF program to raw sockets |


## Run

```sh
bwninja                  # all interfaces with an address, except lo
bwninja -i eth0 wg0      # only these interfaces
bwninja -t 1             # sample every second (default 0.3)
python3 -m bwninja.tui   # same as bwninja, from a checkout
```

`bwninja` needs root. When started as a user it re-runs itself under `sudo`
with the same python and the same package paths, so a user install or a
checkout works without a root install.

Keys:

| Key | Action |
|---|---|
| `q` | quit |
| up / down | select the row above or below |
| page up / page down | move the selection by half of the table |
| enter | expand or collapse the ports of the selected row |
| `n` | cycle how endpoints are shown: `name:service`, `address:port`, or `name:service (address:port)` |
| `b` | cycle the rates through bytes, bits, and packets per second |
| `<` / `>` | move the sort column left or right: interface, left address, left rate, right rate, right address |
| `i` | hide or show the info panel |
| `l` | listen: show the traffic of the selected row with `tshark` (or `tcpdump`) until ctrl-c |
| click | select the row. A click on the selected row expands or collapses its ports |
| wheel | move the selection by one row per report |

`bwninja` takes the mouse while the table is on screen, so a plain drag does
not select text. Most terminals select text with a shift-drag instead.

Print raw flows without the UI, once per second:

```sh
sudo python3 -m bwninja.net -i eth0
sudo python3 -m bwninja.net -i eth0 --printk   # show kernel trace messages
```


## How it works

```
 kernel                          user space (root)
 ┌──────────────────────┐        ┌─────────────────────────────┐
 │ raw socket per iface │        │ net.Monitor.read()          │
 │   └─ net.c           │ drain  │   └─ Flow(iface, src, dst,  │
 │ flows4 / flows6 maps │──────► │           ports, bytes)      │
 └──────────────────────┘        │ tui.FlowTable  -> pairs      │
                                 │ tui.tree_rows  -> host rows  │
                                 │ tui.NetTui     -> screen     │
                                 └─────────────────────────────┘
```

- `bwninja/net.c` is a socket-filter eBPF program. It reads the IP and TCP/UDP
  headers of every packet and adds the packet size to a hash map entry keyed
  by interface index and 5-tuple. IPv4 and IPv6 have separate maps.
- `bwninja/net.py` compiles `net.c` once with bcc, attaches it to a raw
  socket on every monitored interface, and drains the maps on each read.
  Each raw socket puts its interface in promiscuous mode, as tcpdump does.
  A bridge (`vmbr0`, `docker0`) passes the frames that it forwards between
  its ports (VM taps, the physical port) up to the bridge interface only
  while the bridge is promiscuous. The mode ends when bwninja exits.
- `bwninja/tui.py` reads the flows on a background thread every `-t`
  seconds and keeps the recent samples. The screen thread sums them into
  pairs, folds the pairs into host rows, and draws the table after every
  sample and every key. The reads go on while listen runs under the
  `l` key, so the table is complete when the table returns.
- `bwninja/term.py` is the terminal: escape sequences, the 256-color
  palette, the width of styled text, the screen modes, and the decoding
  of keys and mouse reports from stdin. `bwninja` needs an xterm-like
  terminal with 256 colors, and no terminal library.
- `bwninja/config.py` reads and writes the settings file, with the
  debounce, and knows the places of the files of bwninja in the home of
  the user.
- `bwninja/ipinfo.py` looks up the details of an address (AS, city,
  country, ...) at ipinfo.io and keeps every answer, and every failure,
  in `~/.cache/bwninja/ipinfo.db` (`dbm.sqlite3`, python >= 3.13),
  `~/.cache/bwninja/ipinfo.dbm` (`dbm.gnu`, the python3-gdbm package), or
  `~/.cache/bwninja/ipinfo.dumb.dat` and `.dir` (`dbm.dumb`).

Limits:

- Each map holds 16384 flows between two reads. More flows in one interval
  are dropped until the next read.
- IPv6 packets with extension headers before TCP/UDP are counted with port 0.
- Packets that are not TCP or UDP have no ports. The table shows the
  protocol name from `/etc/protocols` in place of the port (`8.8.8.8:ICMP`),
  or `proto:` and the number in the address style (`8.8.8.8:proto:1`), and
  keeps one port row per protocol under a host row.
- Byte counts are IP packet sizes, without the link-layer header.


## Development

```sh
make run                 # python3 -m bwninja.tui from the checkout
make test                # unit tests for the pure parts (needs bcc importable)
make install-editable
make publish             # build and upload to PyPI
```

Changes are recorded in `CHANGELOG.md`.

The `exp/` directory holds old experiments (an sqlite recorder and a web
view). They are not part of the package and do not run against the current
`net.c`.
