Troubleshooting Netconsole Configuration
Common Problems๐
Itโs important to be aware of some special cases when configuring Netconsole, as setup may not be trivial in certain environments.
Keep in mind that Netconsole does not send data via regular sockets. Instead, it uses low-level kernel mechanisms to transmit logs directly over the network. This means that typical user-space tools or firewall settings might not apply or may interfere unexpectedly.
MAC Address of the Device๐
In most simple cases, you can skip this parameter โ Netconsole should automatically detect the appropriate server. However, if you donโt see any messages on the receiving server, and both the Netconsole host and receiver are on the same network, you can manually specify the MAC address of the receiverโs network interface.
In cases where Netconsole is behind a NAT, you must specify the MAC address of the NAT gateway instead.
Network Interfaces๐
Netconsole should only be started after the network interface is fully initialized.
If you’re using bonding or other virtual interfaces, they may take additional time to become ready. Before starting Netconsole, make sure that:
- The network interface is in the UP state
- Broadcasting is available
Failing to wait for interface readiness may result in Netconsole silently failing to send logs.
Log Level๐
The dmesg debug level determines which kernel messages are allowed to be sent.
To check the current level, use the command:
cat /proc/sys/kernel/printk
The preferred level is 8, which ensures that all messages, including debug-level ones, are captured and forwarded to the Troubleshooting Server.
To change the dmesg logging level, run:
dmesg -n 8
This setting helps ensure that no important logs are missed during an incident.
Firewall Restrictions๐
Make sure that network traffic from the Troubleshooting Server is allowed and not blocked by firewall rules.
To verify connectivity, itโs useful to run a simple UDP server using netcat on the receiving side:
nc -u -l 5556
Then try sending a test message from the Netconsole host. If the message is not received, double-check firewall settings and network accessibility between the nodes.
To use Netconsole, the kernel must be built with the following options enabled:
- CONFIG_NETCONSOLE=y โ builds Netconsole directly into the kernel
- CONFIG_NETCONSOLE_DYNAMIC=y โ allows dynamic configuration of Netconsole at runtime
Test Netconsole manually๐
Start Netconsole๐
Check if the Netconsole module is active
lsmod | grep netconsole
Expected output if the module is loaded:
- If there is no output, it means Netconsole is not active.
- If the module is already loaded, you should unload it before reconfiguring:
modprobe -r netconsole
You can manually start Netconsole with the following command:
modprobe netconsole netconsole=5555@192.168.0.1/enp1s0,5555@192.168.0.2/52:54:00:00:00:00
After starting, check dmesg logs. You should see a message like:
dmesg
[ 7.626968] netpoll: netconsole: local port 5555
[ 7.627424] netpoll: netconsole: local IPv4 address 192.168.0.1
[ 7.628102] netpoll: netconsole: interface 'enp1s0'
[ 7.628648] netpoll: netconsole: remote port 5555
[ 7.629131] netpoll: netconsole: remote IPv4 address 192.168.0.2
[ 7.629822] netpoll: netconsole: remote ethernet address 52:54:00:00:00:00
[ 7.630688] printk: console [netcon0] enabled
[ 7.631132] netconsole: network logging started
Autostart With Systemd๐
There are several ways to configure Netconsole to start automatically. The simplest method is to use systemd.
[Service] Type=oneshot ExecStart=/sbin/modprobe netconsole netconsole=5555@192.168.0.1/enp1s0,5555@192.168.0.2/52:54:00:00:00:00 ExecStop=/sbin/rmmod netconsole RemainAfterExit=true
[Install] WantedBy=multi-user.target
After that:
systemctl daemon-reload
systemctl enable netconsole.service
systemctl start netconsole.service
Autoload module๐
Another way to automatically load Netconsole at startup is to use a modprobe configuration file. Add to the file /etc/modprobe.d/netconsole.conf
Add netconsole to the modules list /etc/modules-load.d/netconsole.conf Add to the file /etc/modules-load.d/netconsole.conf
netconsole
Autostart with Bootloader๐
You can also configure Netconsole to load automatically at boot time via the GRUB bootloader.
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-60ff1bb1-77d5-4c40-9ff1-67c0345f167c' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root 26a6a4af-3c42-4bbb-8d26-94c804ee9b4e
linux /vmlinuz-6.8.0-60-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro console=ttyS0,115200n8 tempesta_dbmem=1280M netconsole=5555@192.168.0.1/enp1s0,5555@192.168.0.2/52:54:00:00:00:00
initrd /initrd.img-6.8.0-60-generic
}
โฆand donโt forget to set Netconsole parameters to load via the bootloader as shown above.
โ ๏ธ Important: This method requires the kernel to be built with the following configuration:
CONFIG_NETCONSOLE=y
If Netconsole is built as a module (CONFIG_NETCONSOLE=m), kernel parameters will have no effect โ in that case, use modprobe or systemd-based autoloading instead.