Linux Deep Dive — Target some unknown unknowns

We can’t escape Linux, and there’s no need to. Once you know the efficacy with which we can perform tasks on this Operating System, you are sure to love it.

Let’s see some linux concepts that are crucial to be known but often missed, how many did you find new?

Photo by Gabriel Heinzer on Unsplash

The Linux Directory Structure

Root Directory (/)

The root directory is the topmost level in the Linux filesystem hierarchy. All other directories and files stem from this point. It’s analogous to the C:\ drive in Windows or the root folder in macOS.

Check file stats like size, modified on, and birth date

stat filename

Essential Directories

• /bin: Contains binary executables required for system operation, available to all users.
• /sbin: Similar to /bin, but contains system administration binaries.
• /usr: Houses user-related programs and data, excluding those needed for booting the system.
• /lib: Stores shared libraries required by the system and applications.
• /boot: Contains files necessary for booting the system, such as the GRUB bootloader configuration, I came across GRUB when installing kali on a VM locally.
• /home: Holds user-specific directories, where individual users store their personal files and configurations.
• /var: Used for storing variable data, such as system logs, databases, and print jobs.
• /tmp: A temporary storage area for files created by the system or users.
• /mnt and /media: Used for mounting removable media and external filesystems, will be useful when we see NFS and mounting in AWS using EFS.
• /dev: Contains device files, representing hardware devices connected to the system.
• /run: Stores runtime data, such as lock files and sockets.
• /srv: Contains server-specific data, like web pages served by a web server.
• /opt: Used for installing third-party software packages.

Special Directories

• /etc: Configuration files for the system, including user accounts, network settings, and services.
• /proc: A virtual filesystem that presents information about processes and other system information in a hierarchical file-like structure.
• /sys: Another virtual filesystem, providing a mechanism for the kernel to export information about devices, drivers, and low-level hardware components.
• /lost+found: Contains recovered files after a system crash or reboot.

File Types

Linux distinguishes between several types of files:

- `d`: Directory
- `l`: Symbolic link
- `c`: Character device file
- `b`: Block device file
- `s`: Socket
- `p`: Named pipe

Creating and linking directories can streamline navigation:

mkdir -p /opt/one/two/three/test
touch /opt/one/two/three/test/filename.txt
ln -s /opt/one/two/three/test shortcut

Accessing the linked file (`shortcut`) displays the content of `filename.txt`.

Filtering and I/O Redirection

Linux provides powerful tools for filtering text and redirecting input/output:

echo "Hello, World!" > new_file.txt

-grep: Searches for patterns within files.

grep root /etc/passwd

- less, more, head, tail, cut, sed: Offer various ways to view and manipulate text.
- Piping (|): Combines the output of one command as the input to another.

ls | grep logdir

Input redirection

echo "$(< new_file.txt)"

AWK

AWK is a powerful text processing language that comes with Unix/Linux environments. It allows you to perform complex operations on text files, extracting, formatting, and transforming data. The basic syntax of AWK is straightforward:

awk 'pattern {action}' file(s)

Here, pattern specifies the condition that determines which lines to process, and action defines what to do with those lines. Actions can include printing, modifying variables, or executing functions.

Example:

To print lines containing the word “error” from a log file named logfile.txt, you would use:

awk '/error/{print}' logfile.txt

This command searches for the pattern “error” and executes the action {print}, which outputs the matching lines.

FIND

The find command is used to search for files and directories based on various criteria such as name, type, size, modification time, and more. Its syntax is versatile, allowing for detailed specifications of what to search for.

Basic Syntax:

find /path/to/search -name pattern

Options:

• -inum n: Search by inode number.
• -type t: Specify the type of file to search for (e.g., d for directory).
• -user u: Search for files owned by user u.
• -group g: Search for files belonging to group g.

Example:

To find a file named myconfigfile.txt in the home directory of a user named username, you would use:

find /home/username/ -name myconfigfile.txt

This command searches the /home/username/ directory for a file named myconfigfile.txt.

VIM

VIM (Vi IMproved) is a highly configurable text editor built to enable efficient text editing. It is an improved version of the vi editor distributed with most UNIX systems. VIM is often considered more powerful than editors like Nano or Pluma due to its extensive feature set and customization options.

If you hate vim, try Pluma.

Modes:

VIM operates in three main modes:

• Insert Mode (i): Allows you to insert text into the document.
• Command Mode: Enables you to execute commands without inserting text.
• Extended Command Mode (:): Provides access to more advanced commands, often starting with a colon.

Commands:

• gg: Move the cursor to the first line of the document.
• G: Move the cursor to the last line of the document.
• u: Undo the last change.
• w: Save the current file.
• b: Go back to the beginning of the current word.

Users and Groups

Linux supports different types of users and groups, including regular users, service users, and root. Managing these entities involves creating and deleting users and groups, assigning them to specific roles, and setting up their home directories and mailboxes.

Adding and Managing Users

sudo useradd sam
sudo passwd sam
su - sam

Adding and Managing Groups

groupadd opsadmin
usermod -G opsadmin devops

Deleting Users and Groups

sudo userdel -r sam
sudo groupdel opsadmin

File Transfer and Disk Usage

• File Transfer: Utilize `cp` and `rsync`.
  - Disk Usage: Commands like `df`, `fdisk -l`, `du`, and `mount` offer insights into disk space utilization and partition layout.

Processes and System Information

Monitoring running processes and system health is essential:

• Processes: Use `ps`, `pmap`, `top`, `kill`, `fg`, `bg`.
  - System Info: Access `uname`, `uptime`, `hostname`, `cal`, `who`, `finger`.

ps -ef

htop

kill -9 pid (-9 flag for force kill)

To run a command in background, run it with & at end of the command.

Hardware Details and Statistics

- Hardware: Explore `/proc/cpuinfo`, `free -m`, and stress testing tools.
- Statistics: Tools like `top`, `mpstat`, `vmstat`, `iostat` provide performance metrics.

File Permissions and Sudo Privileges

Managing file permissions and sudo access is critical for security:

• Permissions: Modify using `chown`, `chgrp`, `chmod`. We can also use numerals, 4 for read, 2 for write and 1 for execute

chmod 600 rsa_id

• Sudo: Configure via `visudo` for secure privilege escalation.

Cron Jobs

Automate tasks with cron jobs, utilizing `crontab` to schedule scripts or commands to run periodically.

Cron jobs are a powerful feature in Unix/Linux systems that allow users to schedule scripts or commands to run automatically at fixed times, dates, or intervals. This automation capability is particularly useful for routine tasks such as backups, system monitoring, and report generation.

Creating Cron Jobs

1. Open the Crontab Editor: You can edit your own crontab by running crontab -e in the terminal. If you need to edit the root user's crontab, prepend sudo to the command.
2. Understand the Format: A cron job entry consists of five fields specifying when the job should run, followed by the command to execute. The format is:

*     *     *   *    *        command to be executed
-     -     -   -    -
|     |     |   |    |
|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
|     |     |   +------- month (1 - 12)
|     |     +--------- day of month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

• Example Entry: To run a script located at /home/user/myscript.sh every day at 5 PM, you would add the following line:

0 17 * * * /home/user/myscript.sh

• Save and Exit: After adding your cron jobs, save and close the editor. The new cron jobs will take effect immediately.

Some more lesser-known commands: nohup, tr, xargs, and xxd

• nohup: Runs a command in the background and ignores hangup signals. Useful for running long-running processes that should continue even if the user logs out. Syntax: nohup command &
• Example: nohup python myscript.py > output.log 2>&1 &
• tr: Translates or deletes characters. It's used for converting text from one character set to another, removing whitespace, or replacing certain characters. Syntax: tr [options] set1 [set2]
• Example: echo "Hello World" | tr '[:upper:]' '[:lower:]' converts uppercase letters to lowercase.
• xargs: Constructs argument lists and invokes utility. It's useful for building arguments for commands that accept multiple inputs. Syntax: xargs [options] command [arguments]
• Example: echo "arg1 arg2 arg3" | xargs -n 1 echo prints each argument on a separate line.
• xxd: Creates a hex dump of a given file or standard input. It's useful for inspecting binary files. Syntax: xxd [options] [infile] [outfile]
• Example: xxd -l 32 myfile.bin generates a hex dump of myfile.bin with 32 bytes per line.

This deep dive into Linux covers the essentials and delves into advanced topics. Knowing these tools will help me traverse around Linux better, which will prove to be beneficial in my future endeavors.