Auto-reboot Linux after a kernel panic

Linux is a robust and stable operating system kernel, but there are instances where it can panic, be it due to bad hardware or bad software. It does not happen often, but it can happen.

If you’re running a server or some other always-on system that you may not have easy access to, a kernel panic typically means an inconvenient trip to reboot a system or a phone call to inconvenience someone else. You can, however, configure Linux to automatically reboot on a kernel panic by making a small modification to /etc/sysctl.conf, a configuration file that tweaks many kernel operating parameters.

Add the following to /etc/sysctl.conf:

kernel.panic = 20

This tells the kernel that if it encounters a panic, it is to reboot the system after a 20 second delay. By default, the kernel will never reboot when it encounters a panic, but with the above setting you can force it to.

Of course, if you enable this, make sure you are using swatch or some other means of observing log files to make sure you aware of the fact when the system panics so you can take appropriate steps to correct the problem.

On local systems, it is also convenient to be able to reboot the system with a key-press in the case of a panic. Instead of having the system reboot automatically on a local system, consider using the magic SysRq keys to reboot your system if X locks up or keyboard entry is being ignored.

To enable magic SysRq support, you must again edit /etc/sysctl.conf; some Linux distributions have this enabled by default whereas others do not.

kernel.sysrq = 1

If the time comes when the SysRq keys are required, use the magic SysRq combination, which is: [ALT]+[SysRq]+[COMMAND], where the [SysRq] key is the “print screen” key and [COMMAND] is one of the following:

  • b - reboot immediately without syncing or unmounting disks
  • e - sends a SIGTERM to all running processes, except for init
  • o - shut down system
  • s - attempt to sync all mounted filesystems
  • u - attempt to remount all mounted filesystems as read-only

These keys need to be pressed together simultaneously to take effect.

Auto-rebooting is great for remote systems, and the magic SysRq combo is very useful for local systems.

No comments: