The oldest and most widely accepted way of doing this is through the /etc/hosts file. This file contains information (in a line by line) about IP addresses and their names in the following format:

127.0.0.1	localhost
127.0.1.1	zephyrus-ubuntu

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Copy

This file however, contains only local mappings and is best reserved for mappings that are required during boot. Other mappings can be found using DNS or LDAP.

On Linux, the configuration file for the DHCP client can be found at /etc/dhcp/dhclient.conf.

A network interface is just hardware that can connect to a network. Machines with multiple ethernet ports, for example, will have different network interfaces controlling each port. Every network interface (hardware, not virtual) has a unique MAC address (also called hardware address) (which is chosen at time of manufacture) to identify itself.

Every machine has the lo network interface, which is virtual and is a loopback. A loopback interface is used by a machine to connect to itself. The other interfaces are hardware dependent. On my laptop for example:

wahid@zephyrus-ubuntu:~$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 50:eb:f6:e1:84:26 brd ff:ff:ff:ff:ff:ff
    altname enp2s0
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether b4:8c:9d:5b:1d:6b brd ff:ff:ff:ff:ff:ff
Copy

There is the before mentioned lo interface. Then there is a eno1 interface that controls the ethernet port and a wlp3s0 wireless network interface.

These network interfaces can be turned on and off using

ip link set <device> <on|off>
Copy

When a network packet bound for some other host arrives, then the packet's destination IP is searched for in the kernel's routing table. From here, one of two things can happen:
(To understand how the packet is sent from a device to the router via Ethernet : ARP)

• The destination IP is found in the kernel's routing table; that is, the destination address is either local or can be reached via a local gateway. Then the packet is forwarded to the appropriate destination.
• The destination IP is not found in the kernel's routing table. The default route (0.0.0.0) is invoked.
  An example kernel routing table:

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 wlp3s0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 wlp3s0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 wlp3s0
Copy

When we use ssh to connect to a remote host, that is not recognized by our machine (a remote host that we have previously not connected to), the server sends a hash of its public key called the key fingerprint. If we choose to accept this "unknown" remote server's fingerprint, then the fingerprint is added to ~/.ssh/known_hosts.

These are stored at ~/.ssh. However, if this directory doesn't have the permissions set to 0700, it is ignored. The SSH protocol can make use of public key cryptography (it also has other means of authentication) for authentication.

Here is how SSH is usually configured an used:

1. The user creates a private-public key pair with ssh-keygen
   1. ssh-keygen can be used to generate specific types of keys with the -t flag.
   2. ssh-keygen -t rsa -b 4096 generates an RSA key with 4096 bits.
   3. We are also prompted to set an optional passphrase for the keys.
2. The private and public keys are stored as ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub respectively.
3. Now, if the server has our public key listed in its ~/.ssh/authorized_keys, then we are allowed to login to this account via SSH.
   Important

   It is necessary that our remote user's ~/.ssh directory have the permssions set as 0700 and that ~/.ssh/authorized_keys (which contains our public key) have the permissions set to 0600.

ssh-agent is a daemon, that caches private keys. When using ssh, the private keys that are cached are used automatically. It is also possible to have multiple private keys cached.

• A private key can be loaded using the ssh-add command as follows - ssh-add ~/.ssh/id_rsa
• And currently loaded private keys can be listed with ssh-add -l
• To remove a private key, we do ssh-add -d ~/.ssh/id_rsa (This one is weird, it apparently requires that the public key be present in the same directory as the key we are trying to remove. If we don't have the public key, it can be extracted by doing ssh-keygen -yf ~/.ssh/id_rsa)
• ssh-add -D purges all cached keys
  Arguably, the most important feature of ssh-agent, is that cached keys can be forwarded to remote machines. This means, we don't have to copy over our private keys to jump around remote hosts. The cached keys are taken forward with us, whenever we ssh, provided we use ssh -A (A stand for agent forwarding).

Local port forwarding is done with the -L flag like this -

Alright, this is kinda fucky to read, but whats happening is basically

Basically, the host tries to access the remote service instead of us, and just forwards us the data on local_port. And also, whatever we send to local_port gets forwarded to remote_address:remote_port

We can also specify -R for remote port forwarding. This is where we allow a remote server to access our local ports.
Example
If we have a webserver running on port 80 on our machine, and we have done ssh -R 9000:localhost:80 user@host. Then any requests to the host at port 9000 will be forwarded to localhost on port 80.

These are just a blob of data, a series of bytes. They could be text, libraries, executables and so on.

A directory contains a list of references (hardlinks) to files. ., .. are special directories that mean current and parent directory. The name of a file is stored in the directory it is in, and not the file itself. So, it is possible for multiple directories to contain a reference to the same file.

A directory stores the name of a file and its inode number.
Related: Directory permission

Device files act as a gateway between the filesystem and device drivers. When a request is made to a device file, it just forwards the request to the appropriate device driver. Each device file has a major and minor device number, to represent which driver and which unit of that driver to refer to.
Example
/dev/tty0 and /dev/tty1 would have the same major number but different minor numbers.

Sockets allow processes to communicate. Also called Unix Domain Sockets (UDS), these are specifically used for communication between processes running on the same host OS. Unlike TCP packets which are used for communication between processes running on a network.
So, because UDS know they're on the same host, they can avoid a few checks that TCP/IP packets go through, and therefore be lighter and faster.^[1]
A socket is also bidirectional, and can be used by multiple processes simultaneously.

Named pipes / FIFOs are basically super primitive local domain sockets. They have nothing to do with a network, are not bidirectional. We would require two pipes, one for read and one for write, for each client process that would want to communicate.^[2]

Symlinks or softlinks are a pointer to a file. If the file is moved or is removed, then the symlink becomes invalid.

The ext4 filesystem is what is called a journaling filesystem. Journaling is somewhat like transactions in SQL. Whatever fs operation is to be performed is written to a journal, and then it is commited. Only then is the actual modification made.
If for some reason the actual operation failes, the filesystem can just look at the journal and safely reapply the last commit.

BTRFS and ZFS follow what is called Copy On Write. With COW, the entire filesystem moves from one consistent state to another. Instead of modifying data in place on the disk, an in-memory copy is modified and then written to some vacant block. And whatever was pointing to the original data is also rewritten to point to the new block. And so on, the parent's parent is rewritten up until the topmost level.

Sometimes an empty package is listed with a bunch of dependencies, so that all these dependencies can be installed under a single alias. For example, gnome-desktop-environment is not actually an existing software package, but is a dummy package with a list of dependencies that are required to run GNOME.

The configuration file for apt is located at /etc/apt/sources.list. This file contains information about where to find packages.
Each line is supposed to follow this format:

• Type: Can be deb, deb-src, rpm, rpm-src
• A URL that points to a file or an http server or an ftp server from which to fetch packages
• A release name (that is often called a "distribution")
• A list of components that are basically categories of packages. universe is one such component that has lots of open-source software. main, multiverse are some other examples

apt is actually a wrapper for a bunch of other low-level commands like apt-get

• update forces apt to download and update package information from configured repositories
• upgrade installs available upgraded packages. If an older dependency is no longer required, it will NOT be removed
• autoremove removes all orphan dependencies. If however, this orphan dependency is something that is useful (it could be a standalone package that we manually installed), then it can be marked with apt-mark for autoremove to exclude it
• install searches for a given package name and installs it
• remove just removes an installed package but leaves behind configuration files
• purge is remove but also removes config files. NOT the config files in /home. purge also works on already remove'd packages

SUID: When set on an executable file, the file is executed with the permissions of the user (owner).
SGID: When set on an executable file, the file is executed with the permissions of the group of the user.
Sticky: Was used to retain programs in memory. Is now obsolete.

A directory just stores a filename and the file's inode number. Read permissions on a dir allow for accessing the filenames only. Write permissions allow for adding, renaming and deleting the files in it.^[4]
The "execute" bit is reused as the search permission. When it is set, we are allowed to search for a file's inode. We can only read the files inside a directory, if we have read perms on the file itself and search (execute) perms on the directory. The execute bit is also required for the stat() syscall, which is called during opening and deletion of a file.

SUID: Has no effect.
SGID: Files created in the directory will inherit the group of the directory.
Sticky: If set, only the owner and the super user are allowed to rename or delete files in it.

The chown command is used to change the ownership (owner and group) of a file. Usual syntax is

The chmod command is used to change the permissions of a file. The syntax is pretty memorable(u g o = user group other; r w x s = read write execute uid/gid).

The useradd command is used to add an user. The default config for this command (such as the default shell) is located at /etc/login.defs. Using -D, we can change the default values.
Example
useradd -D -s /bin/tcsh changes the default shell for any users that will be added in the future.
useradd -D lists the default values. On my laptop, these are the defaults (haven't been modified)

adduser is a more user friendly and interactive command (perl script really) that is available by default on Debian/Ubuntu.

Each process is assigned a unique ID by the kernel called a Process ID (PID). When a process wants to call a sub process, it must clone itself, and then exchange itself with a different process. This clone is called the child process, and the original process is the parent process. The child is given an attribute called the Parent PID (PPID) which is the PID of the parent process.

• UID is the ID of the user under which the process was invoked
• EUID is the ID of the user, with whose perms the process is being run. It is usually the same as UID, but in case of sudo'ing a process, its EUID will be root's UID
• GID and UGID are similar to how UID and EUID work

A process's CPU time is determined based on how much CPU time it has most recently used, and how long it waited, and something called the nice value or niceness. The higher the niceness, the higher the priority.

We can observe two values - PR (priority) and (NI) niceness, when we use a process explorer like top. PR and NI are related as - ^[5]
PR = 20 + NI

• The kernel allows for 140 priority values from -100 to 39, both included. The priorites from -100 to -1 (100 total) are called real time priorites, and these take priority over the normal processes.
• As for the niceness values, the range is -20 to 19, both included. So, the priority level for a nice'd process maps from 0 to 39, which are the rest of the 140 priorities.
  We can set a process' niceness before it starts with the nice command as show here -

We can also set niceness to a running process with renice as follows -

(The -p flag is to specify that we will pass in PIDs).

We CAN set niceness to a negative value to make a process realtime, but this would require higher perms (sudo or root user). Real time processes will have negative PRs, like discussed earlier, but some of them might also have rt as their PR, which means -100, or highest priority.

Processes with a known PID can be stopped using the kill command. This command also takes in a signal as an argument that is sent to the process.
^C sends a SIGINT signal which is an interrupt signal, and the program can handle this interrupt; in most cases it aborts.

Processes can be stopped usually with a SIGINT, if however that fails, we can use -

• kill -9 <PID> to send a SIGKILL. If this fails for some reason, the only way to kill it properly is to reboot. For other possible signals, try kill -l
• killall <name> is very similar to kill, but instead of PIDs it takes names and kills all processes by that name. Default signal is SIGTERM but we can change it just like with kill.
• pgrep or process grep, basically greps processes by name and lists their PIDs. pkill kills them instead of listing them. We can also pkill by owner of the process using the -u flag and specifying user.

^Z sends a SIGTSTP signal which suspends a process.
Suspended processes can be viewed using the jobs command, and can be resumed to foreground or background with the fg and bg commands followed by the index that the jobs command provides.

Processes can be monitored using the ps command. This command's output is highly configurable and the most important and common usage is -

• ps aux, a for all processes, x for including processes without a control terminal, u for user oriented output
• One drawback of the u flag is that it converts all the UIDs to usernames, which might be very slightly computationally expensive. So, a "more" efficient way is ps lax which lists the output in (l) long format
• Additionally add ww to the args to allow for wrapping
  How can a process not have a control terminal?

  Processes that are started by the system's init system (systemd or init ...) don't have a control terminal.
  Daemons startup by forking a child process and then the parent process is killed. The daemon is now an orphan and is adopted by the init system. Therefore, a daemon will also not have a control terminal.

strace can be used to attach to a process by doing strace -p <PID> and it outputs all syscalls made by that process.

cron is used to schedule tasks. It is configured by modifying the cron table (crontab). When a cron service is active, it will schedule all tasks in the crontab. The actual location of crontabs depends on the implemenation that is being used.

systemd handles more than just processes. It handles network connections, kernel logs, logins. It has all the features that traditional init did, but adds on to it so much more and unifies them.

systemd manages entities that it calls units. A unit can be a lot of things, so I won't list all of them here, but some of the important ones are a service, a socket, a device, a mount point.
We can differentiate them with their "extension". A service, for example would end with .service.
Example unit file

[Unit] 
Description=fast remote file copy program daemon ConditionPathExists=/etc/rsyncd.conf 

[Service] 
ExecStart=/usr/bin/rsync --daemon --no-detach 

[Install] 
WantedBy=multi-user.target
Copy

Unit files can be found at -

• /usr/lib/systemd/system
• /lib/systemd/system
• /etc/systemd/system (Highest priority)

Every process has atleast three channels available - STDIN, STDOUT, STDERR. Which are assigned numbers called file descriptors, in this case 0, 1, 2 respectively. We are allowed to redirect these channels among processes in a number of ways by using a few symbols. ^[7]

• < connects STDIN to an existing file
• > and >> write and append to a file respectively. They connect STDOUT to a file.
• >& is used to redirect both STDOUT and STDERR to the same location.
• 2> is used to redirect only STDERR
  • A common usecase for this is to dump all errors to /dev/null
• 1>&2 redirects STDOUT to STDERR
• 2>&1 redirects STDERR to STDOUT
• | connects the STDOUT of one command to the STDIN of another
• && executes the second command if the first one succeeds
• || executes the second command if the first one fails
• ; can be used to write a bunch of commands in one line

Variables are assigned as var='text'. But they are referenced, prefixed with a $ sign, like $var. When a variable has to be referenced inside a string, we additionally surround the variable name in {}, like ${var}.
Anything that is wrapped in backticks is treatead like a command, and the output of the command is substituted.

When a process starts, it receives all command line arguments along with these "pre-existing" variables called environment variables. Based on environment variables, a program can decide to change its behaviour.
Example
When the MOZ_ENABLE_WAYLAND environment variable is set to 1. Firefox will try to use wayland instead of x11.

^P in an active terminal session brings up the last used command. By using the fc command, we can save our current command to a file, in our preferred editor (the editor is set with an env variable).

A shell script (sh) can consist of nothing but commands. The first line has what is called a shebang - #! followed by the location of the interpreter that should be used to execute the script.

• #!/bin/sh would execute the script using sh
• We can also use other interpreters or commands to run our script, provided they're in PATH by doing #!/usr/bin/env python, for example
  We will also have to make the script executable by setting its execute bit. It is bad practice to use setuid bits for managing security, and instead sudo should be used.
  If the shebang is set, we can invoke the script directly by its name to run it. Or we can pass the script path as argument to sh or bash or whatever interpreter we are using.

• fdisk is a safe bet, as most linux distros ship with it. Basic usage is fdisk /dev/sdx
• fdisk has a bunch of commands. These also have their shorthand notations, usually as one letter
  • g : GPT partition table
  • n : Create new partition
  • And so on, a complete list can be viewed by using help as a command
• When creating a new partition, it asks for
  1. Partition number
  2. First sector
  3. Last sector
• It is smart enought to figure out the partition number and auto increment for successive partitions. It is also smart enought to figure out the first sector based on the partitions that have already been made. The last sector part is what we really need to focus on.
• If the last sector is input to be +20G for example, the last sector will be set to first_sector + 20GBytes. Basically, +size will create a new partition starting at the first sector we provide, that is as big as the size we provide.