pmacct Local Traffic Monitoring

What pmacct does

pmacct is the flow accounting toolkit bundled with the Network Probe. It ships as several daemons, and Energy Logserver uses them in two distinct roles:

  • Collector (nfacctd, sfacctd): receives NetFlow, IPFIX, or sFlow that network devices export to the probe over UDP, and forwards it into the collection pipeline. This is the default role, covered in Bundled Services and Technical Configuration.

  • Generator (pmacctd): captures packets directly from a host interface in promiscuous mode, aggregates them into flows, and exports those flows as NetFlow. This chapter covers the generator role.

Use the generator when you want to account for the traffic of a host that does not export NetFlow on its own, most commonly the Energy Logserver server itself. It complements Suricata IDS and Zeek: where those inspect content and protocols, pmacctd answers who talked to whom, over which protocol and port, and how much data moved.

How pmacctd fits the Network Probe

pmacctd does not write to Energy Logserver directly. It produces NetFlow and hands it to the collector that already runs on the probe:

Host interface (promiscuous, libpcap)
        │
        ▼
pmacctd  ── aggregates packets into flows, nfprobe plugin exports NetFlow v9
        │  UDP 127.0.0.1:2055
        ▼
nfacctd  ── the bundled NetFlow collector (plugins: kafka)
        │  Kafka topic "netflow"
        ▼
Network Probe pmacct pipeline  ── /etc/logserver-probe/conf.d/pmacct/
        │
        ▼
Index  stream-netflow-pmacct-*

The right-hand side of this chain already exists on a probe that collects NetFlow: nfacctd listens on UDP 2055, publishes to the Kafka netflow topic, and the pmacct pipeline writes the enriched flows (GeoIP, DNS, bad-IP lookups) to stream-netflow-pmacct-*. What you configure here is only pmacctd, so it captures on the correct interface and exports to the port the collector is already listening on.

This is why there is no One Click step for the generator. One Click installs the Energy Logserver side of a pipeline; here the Energy Logserver side is the existing NetFlow collection path, and the work is entirely host-side.

Note

The hand-off from pmacctd to the collector is a UDP protocol, not a file. pmacctd sends flows and receives no acknowledgement. That shapes what goes wrong: if flows do not arrive, the question is whether pmacctd is exporting and whether the collector is listening on UDP 2055, not whether a file is readable.

Why manual configuration is needed

The pmacct package ships the pmacctd binary but no generator configuration and no service unit. Energy Logserver configures nfacctd/sfacctd for the collector role, not pmacctd. To run the generator you create its configuration file and a systemd unit yourself, described below.

Requirements

  • The pmacct package, installed with the Network Probe (rpm -qa | grep pmacct).

  • A running NetFlow collector on the probe: nfacctd listening on UDP 2055, with its pmacct pipeline active, so the generated flows have somewhere to go. See Bundled Services.

  • Root access on the host you want to account for.

  • The interface that carries the traffic you want to measure.

Step 1: Create the configuration and log directories

The package ships no generator directories, so create them:

mkdir -p /etc/pmacct
chmod 755 /etc/pmacct

mkdir -p /var/log/pmacct
chmod 755 /var/log/pmacct

Step 2: Identify the capture interface

Find the interface that carries the traffic you want to account for. You will write its name into the configuration in the next step.

ip -br addr show

Note the interface name (for example ens192, eth0, enp0s3). The interface does not need an address of its own to be captured, but it must be UP.

Step 3: Create the pmacctd configuration

Create /etc/pmacct/pmacctd.conf. Replace INTERFACE_NAME with the interface from Step 2:

! pmacctd generator configuration for {{ project }}
! Local traffic accounting with NetFlow v9 export to the local collector

! General
daemonize: true
pidfile: /var/run/pmacctd.pid
syslog: daemon

! Capture
pcap_interface: INTERFACE_NAME
promisc: true

! Export flows as NetFlow v9 to the local collector on UDP 2055
plugins: nfprobe
nfprobe_version: 9
nfprobe_receiver: 127.0.0.1:2055
nfprobe_timeouts: tcp=120:maxlife=3600

! Aggregation key
aggregate: src_host, dst_host, src_port, dst_port, proto, tos

Note

Do not add imt_mem_pools_size or imt_buckets here. Those tune the in-memory table of the memory plugin. With plugins: nfprobe there is no memory table, so pmacctd accepts the keys without error but they do nothing. The memory plugin is only useful if you query flows locally with the pmacct client (pmacct -s), which this setup does not.

Step 4: Validate the configuration

Run pmacctd in the foreground with debug output to confirm it parses the file and loads the plugins:

pmacctd -f /etc/pmacct/pmacctd.conf -d 2>&1 | head -20

The output lists the loaded plugins (core and nfprobe) and echoes the interface and receiver you set. Stop it with Ctrl+C. If it reports a parse error, check the interface name and that every comment line starts with !.

Step 5: Create the systemd service

pmacctd ships without a unit. Create /etc/systemd/system/pmacctd.service:

[Unit]
Description=pmacctd - Promiscuous Mode Accounting Daemon
Documentation=man:pmacctd(8)
After=network-online.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/var/run/pmacctd.pid
ExecStart=/usr/sbin/pmacctd -f /etc/pmacct/pmacctd.conf
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10
CapabilityBoundingSet=CAP_NET_RAW CAP_NET_ADMIN
AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN

[Install]
WantedBy=multi-user.target

Reload systemd and enable the service:

systemctl daemon-reload
systemctl enable pmacctd

Step 6: Start pmacctd

systemctl start pmacctd
systemctl status pmacctd --no-pager

A healthy start reports active (running) with two tasks: the core process and the NetFlow probe plugin.

Step 7: Verify pmacctd is exporting

Confirm the process is up and the log shows flows leaving for the collector:

journalctl -u pmacctd -n 20 --no-pager

The log line Exporting flows to [127.0.0.1]:iop confirms export; iop is the service-name notation for port 2055.

Verify end to end with test traffic

pmacctd exports a flow only when the flow expires, not on every packet. Generate traffic and wait for the expiry interval before concluding anything is wrong:

ping -c 10 8.8.8.8 > /dev/null 2>&1
curl -s https://example.com > /dev/null 2>&1

# wait for flow expiry before the count updates
sleep 90

Then confirm documents are landing in the collector’s index:

curl -s -u admin:PASSWORD \
  "http://localhost:9200/_cat/indices/stream-netflow-pmacct-*?v&h=index,docs.count,store.size"

A growing docs.count on today’s stream-netflow-pmacct-* index confirms the whole chain: capture, export, collection, and indexing.

Explore in Discover

The generated flows join the same index pattern as all collected NetFlow. Open Discover, select the stream-* index pattern, and set a recent time range. To isolate the generator’s own flows, filter on the index stream-netflow-pmacct-*.

Useful fields:

  • netflow.src.ip, netflow.dst.ip: the endpoints of each flow.

  • netflow.src.port, netflow.dst.port: the ports.

  • netflow.protocol_name: TCP, UDP, ICMP.

  • netflow.bytes, netflow.packets: the volume of each flow.

  • netflow.exporter: the address that exported the flow (127.0.0.1 for the local generator).

Troubleshooting

pmacctd will not start

Read the service log for the cause:

journalctl -u pmacctd -n 50 --no-pager

pcap_open_live(): INTERFACE_NAME: No such device means the interface name is wrong. Either the placeholder from Step 3 was never replaced, or the name does not exist on this host. List the real interfaces with ip -br link show and correct pcap_interface.

pcap_open_live(): Permission denied means the process lacks CAP_NET_RAW. Run pmacctd as root, or confirm the CapabilityBoundingSet and AmbientCapabilities lines are present in the unit.

No flows reach the index

pmacctd exports over UDP, which is fire-and-forget: it reports no error even when nothing receives the flows. If the collector is not listening, the kernel discards every packet silently while pmacctd still logs Exporting flows.

Confirm something is bound to the collector port:

ss -ulnp | grep 2055

If nothing is listening, the NetFlow collector is not running. Start it and confirm the pmacct pipeline is active on the probe (see Bundled Services). If the collector is listening but the index stays empty, generate traffic and wait for flow expiry, then re-check, since flows are exported only when they expire.

High CPU use

pmacctd builds an aggregation key for every packet and keeps a flow entry for each unique combination. The more fields in the key, the more distinct flows exist, and the more CPU each packet costs. At high traffic volumes a fine-grained aggregate directive is usually what drives the load.

Reduce the aggregation granularity first. Check what you currently aggregate on:

grep aggregate /etc/pmacct/pmacctd.conf

Drop the fields you do not query on. Source and destination IP, port, and protocol produce far fewer flows than adding ToS, interfaces, and MAC addresses on top. Restart and measure again:

systemctl restart pmacctd
top -p "$(pgrep -d, pmacctd)"

If trimming the key is not enough, sample the traffic:

sampling_rate: 100  ! analyse 1 packet in 100

Warning

Sampling trades accuracy for CPU. With sampling_rate: 100, byte and packet counters become estimates scaled by 100, and short flows can be missed entirely. That is acceptable for traffic trends and top-talker views, but not for accounting or forensic use where individual flows matter. Trim the aggregation key first; sample only if the sensor still cannot keep up.

Reducing data volume

To lower the number of exported flows and the storage they consume, raise the export timeouts so flows are batched over longer intervals:

nfprobe_timeouts: tcp=300:maxlife=7200

This reduces the volume of NetFlow records and downstream documents, not the per-packet CPU cost of capture and aggregation. It also delays when data appears: with a long maxlife, an active session may not reach Energy Logserver until it ends or the timeout elapses. Use it to control data volume, and keep the timeouts short if you need flows to surface quickly.

Flows land under the wrong date

NetFlow indices are date-based (stream-netflow-pmacct-YYYY.MM.DD), and the index a flow lands in follows its timestamp. If the host clock is wrong, flows are written to an index for another date and do not appear under a normal time range in Discover. Confirm the clock and enable time synchronisation:

timedatectl
timedatectl set-ntp true

List the indices that actually exist to find where the flows went:

curl -s -u admin:PASSWORD "http://localhost:9200/_cat/indices/stream-netflow-pmacct-*?v"

Missing GeoIP data

GeoIP enrichment is applied by the Network Probe pmacct pipeline, not by pmacctd. If flow records have no netflow.geoip fields, confirm the GeoIP databases are installed and current on the probe, and check the probe log for GeoIP errors. Local flows between private addresses have no GeoIP by design.

Definition of terms

  • pmacctd: the pmacct generator daemon. Captures packets from an interface with libpcap and exports flows. Distinct from nfacctd/sfacctd, which receive flows over UDP.

  • nfprobe: the pmacctd plugin that formats flows as NetFlow and exports them to a receiver.

  • Flow: an aggregate of packets that share the same key (source and destination address, ports, protocol) over a time window.

  • Flow expiry: the point at which pmacctd stops accumulating a flow and exports it, either when the connection ends or when a timeout elapses.

  • Promiscuous mode: an interface mode in which the card accepts all packets it sees, not only those addressed to it. Required to account for traffic that passes the interface.