How to interact with Linux Kernel through procfs and sysfs

What is procfs (/proc)

Linux 'procfs' is a pseudo filesystem that provides an interface to kernel in memory data structures. It is commonly mounted at '/proc'. Typically, it is mounted automatically by the system, but can also be mounted manually using a command such as:

mount -t proc proc /proc

Most of the files inside the '/proc' filesystem are read-only, but some are
writable, allowing kernel variables to be changed.

Procfs content overview

$ ls /proc/
1     117   1581  1847  20    28   4    570  8    985        diskstats    kallsyms     meminfo       self           uptime
10    12    1582  1865  200   29   417  573  80   99         dma          kcore        misc          slabinfo       version
1017  13    1586  1868  2003  30   445  574  81   acpi       driver       keys         modules       softirqs       version_signature
1040  14    16    1869  201   302  447  575  82   buddyinfo  execdomains  key-users    mounts        stat           vmallocinfo
1049  15    17    188   21    303  452  593  83   bus        fb           kmsg         mtrr          swaps          vmstat
1062  1533  170   19    22    31   453  6    9    cgroups    filesystems  kpagecgroup  net           sys            zoneinfo
1067  154   172   1928  23    339  454  7    90   cmdline    fs           kpagecount   pagetypeinfo  sysrq-trigger
1075  155   18    1942  24    34   455  755  904  consoles   interrupts   kpageflags   partitions    sysvipc
1077  156   1844  1948  25    340  456  774  911  cpuinfo    iomem        loadavg      sched_debug   thread-self
1096  157   1845  199   27    35   457  78   927  crypto     ioports      locks        schedstat     timer_list
11    1570  1846  2     278   36   483  79   939  devices    irq          mdstat       scsi          tty

For infos about each files/directories purposes use: man procfs

What is sysfs (/sys)

Linux 'sysfs' is a pseudo filesystem that provides information about available devices on the system and hardware attributes they are using. It is commonly mounted at '/sys'. Typically, it is mounted automatically by the system, but can also be mounted manually using a command such as:

mount -t sysfs sysfs /sys

Sysfs content overview

$ ls /sys/
block  bus  class  dev  devices  firmware  fs  hypervisor  kernel  module  power

• block: provides info on connected block devices
• bus: provides info on available bus (drivers, connected devices, parameters...)
• class: provides info on devices, organized by devices class
• dev: provides info on initialized device nodes, by device nodes types
• devices: provides info on devices with a categorized presentation
• firmware: firmware info
• fs: provides info on filesystem types used by block devices and their
  associated parameters and statistics
• (...)

Getting infomation about the system hardwares

To ease the process of getting information about devices from the sysfs pseudo filesystem, the 'udevadm' command can be used. 'udevadm info --help' for details. Example:

# Get all types of info about the /dev/sda device
udevadm info --query=all --name=/dev/sda

Here is a list of other tools we can use to get hardwares information from sysfs:

• lspci: get info about devices connected to the PCI bus. Use the '-k' option to get info about the associated kernel modules in use for the listed hardwares
• lsusb: get info about devices connected to the USB bus.
• dmidecode: get BIOS related information and detailed computer related info coming from the firmware. Use 'man dmidecode' for more info about the command and the structure of the printed information
• dmesg: show the content of the kernel ring buffer, containing some info about recent hardware events (plugging/unplugging an hardware, etc). Each printed line starts with a timestamp in seconds corresponding to the number of seconds since the booting of the system.
• lsscsi: get info about what is connected to the SCSCI bus
• lsblk: get info about block devices
• lsdev: get info about devices
• lscpu: get info about CPU hardware

Updating kernel runtime parameters

Kernel runtime parameters can be seen in the '/proc/sys' directory. Inside that directory, we will find subdirectories that categorize those parameters:

$ ll /proc/sys/
total 0
dr-xr-xr-x   1 root root 0 Dec 27 08:50 ./
dr-xr-xr-x 191 root root 0 Dec 27 08:50 ../
dr-xr-xr-x   1 root root 0 Dec 27 08:52 abi/
dr-xr-xr-x   1 root root 0 Dec 27 09:07 debug/
dr-xr-xr-x   1 root root 0 Dec 27 08:52 dev/
dr-xr-xr-x   1 root root 0 Dec 27 08:50 fs/
dr-xr-xr-x   1 root root 0 Dec 27 08:50 kernel/
dr-xr-xr-x   1 root root 0 Dec 27 08:50 net/
dr-xr-xr-x   1 root root 0 Dec 27 09:07 user/
dr-xr-xr-x   1 root root 0 Dec 27 08:50 vm/

For info about each of the subdirectories or sometimes parameters files they contain and what they are used for, use 'man procfs' and search for the subdirectory or parameters files absolute path. For instance, to get info about the 'abi' subdirectory, you will search for: \/proc\/sys\/abi.

We can also get descriptions about some of the kernel parameters by looking at the Linux kernel manual for the underlying module or protocol. For instance, to know what the '/proc/sys/net/ipv4/ip_forward' or 'net.ipv4.icmp_echo_ignore_all' do, we can use:

man 7 ip # and search for 'ip_forward'
man 7 icmp # and search for 'icmp_echo_ignore_all'

Now let's have a look at the content of one of the parameters files present inside '/proc/sys'. This one is used to enable (1) or disable (0) packet forwarding for IPv4:

$ cat /proc/sys/net/ipv4/ip_forward
1

To enable or disable packet forwarding for IPv4, we simply have to update the value inside the parameter file with 1 or 0:

# Enable packet forwarding for IPv4
echo "1" > /proc/sys/net/ipv4/ip_forward

# Disable packet forwarding for IPv4
echo "0" > /proc/sys/net/ipv4/ip_forward

We can also use the 'sysctl' tool for that:

sysctl -w net.ipv4.ip_forward=1 # or
sysctl -w net.ipv4.ip_forward=0

The changes we made that way will not persist after system reboot.

Make kernel parameters changes persist after reboot

To make kernel parameters updates persistent, we have to set the parameters inside '/etc/sysctl.conf' or inside a new drop-in file in '/etc/sysctl.d/'. The drop-in file must have the '.conf' extension. Each line inside the configuration file is in the form: 'param = value'.

Now let's override the IPv4 packet forwarding parameter using our new '/etc/sysctl.d/local.conf' configuration file. We can use the following to find the parameter to use inside the configuration file:

$ sysctl -a | grep "ip_forward"
net.ipv4.ip_forward = 1 # this is the one we are looking for
net.ipv4.ip_forward_update_priority = 1
net.ipv4.ip_forward_use_pmtu = 0

# or use the --pattern to search for a specific
# pattern across available parameters
$ sysctl -a --pattern ip_forward$
net.ipv4.ip_forward = 1

We can now put the following configuration line inside the '/etc/sysctl.d/local.conf' configuration file:

net.ipv4.ip_forward = 1 # To enable IPv4 packet forwarding

# or
net.ipv4.ip_forward = 0 # To disable IPv4 packet forwarding

After updating the configuration file, we need to run the following command to make the kernel re-read the configuration files and update parameters values accordingly:

systemctl restart procps.service # or
service procps reload # or
deb-systemd-invoke restart procps.service