The Software Developer's Guide to Linux: A practical, no-nonsense guide to using the Linux command line 9781804616925

Table of contents :
Preface
Who this book is for
What this book is not
What this book covers
To get the most out of this book
Get in touch
How the Command Line Works
In the beginning…was the REPL
Command-line syntax (read)
Command line vs. shell
How does the shell know what to run? (evaluate)
A quick definition of POSIX
Basic command-line skills
Unix filesystem basics
Absolute vs. relative file paths
Absolute vs. relative pathname review
Opening a terminal
Looking around – command-line navigation
pwd - print working directory
ls - list
Moving around
cd – change directory
find – find files
Reading files
less – page through a file
Making changes
touch – create an empty file, or update modification time for an existing one
mkdir – create a directory
rmdir – remove empty directories
rm – remove files and directories
mv – move or rename files and directories
Getting help
Shell autocompletion
Conclusion
Working with Processes
Process basics
What is a Linux process made of?
Process ID (PID)
Effective User ID (EUID) and Effective Group ID (EGID)
Environment variables
Working directory
Practical commands for working with Linux processes
Advanced process concepts and tools
Signals
Practical uses of signals
Trapping
The kill command
lsof – show file handles that a process has open
Inheritance
Review – example troubleshooting session
Conclusion
Service Management with systemd
The basics
init
Processes and services
systemctl commands
Checking the status of a service
Starting a service
Stopping a service
Restarting a service
Reloading a service
Enable and disable
A note on Docker
Conclusion
Using Shell History
Shell history
Shell configuration files
History files
Searching through shell history
Exceptions
Executing previous commands with !
Re-running a command with the same arguments
Prepending a command to something in your history
Jumping to the beginning or end of the current line
Conclusion
Introducing Files
Files on Linux: the absolute basics
Plaintext files
What is a binary file?
Line endings
The filesystem tree
Basic filesystem operations
ls
pwd
cd
touch
less
tail
mv
Moving
Renaming
cp
mkdir
rm
Editing files
File types
Symbolic links
Hard links
The file command
Advanced file operations
Searching file content with grep
Finding files with find
Copying files between local and remote hosts with rsync
Combining find, grep, and rsync
Advanced filesystem knowledge for the real world
FUSE: Even more fun with Unix filesystems
Conclusion
Editing Files on the Command Line
Nano
Installing nano
Nano cheat sheet
File handling
Editing
Search and replace
Vi(m)
Vi/vim commands
Modes
Command mode
Normal mode
Tips for learning vi(m)
Use vimtutor
Think in terms of mnemonics
Avoid using arrow keys
Avoid using the mouse
Don’t use gvim
Avoid starting with extensive configuration or plugins
Vim bindings in other software
Editing a file you don’t have permissions for
Setting your preferred editor
Conclusion
Users and Groups
What is a user?
Root versus everybody else
sudo
What is a group?
Mini project: user and group management
Creating a user
Create a group
Modifying a Linux user
Adding a Linux user to a group
Removing a user from a group
Removing a Linux user
Remove a Linux group
Advanced: what is a user, really?
User metadata / attributes
A note on scriptability
Conclusion
Ownership and Permissions
Deciphering a long listing
File attributes
File type
Permissions
Number of hardlinks
User ownership
Group ownership
File size
Modification time
Filename
Ownership
Permissions
Numeric/octal
Common permissions
Changing ownership (chown) and permissions (chmod)
Chown
Change owner
Change owner and group
Recursively change owner and group
Chmod
Using a reference
Conclusion
Managing Installed Software
Working with software packages
Update your local cache of repository state
Search for a package
Install a package
Upgrade all packages that have available updates
Remove a package (and any dependencies, provided other packages don’t need them)
Query installed packages
Caution required – curl | bash
Compiling third-party software from source
Example: compiling and installing htop
Install prerequisites
Download, verify, and unarchive the source code
Configure and compile htop
Conclusion
Configuring Software
Configuration hierarchy
Command-line arguments
Environment variables
Configuration files
System-level configuration in /etc/
User-level configuration in ~/.config
systemd units
Create your own service
Quick note: configuration in Docker
Conclusion
Pipes and Redirection
File descriptors
What do these file descriptors reference?
Input and output redirection (or, playing with file descriptors for fun and profit)
Input redirection:
Use >> to append output without overwriting
Error redirection with 2>
Connecting commands together with pipes (|)
Multi-pipe commands
Reading (and building) complex multi-pipe commands
The CLI tools you need to know
cut
sort
uniq
Counting
wc
head
tail
tee
awk
sed
Practical pipe patterns
“Top X”, with count
curl | bash
Security considerations for curl | sudo | bash
Filtering and searching with grep
grep and tail for log monitoring
find and xargs for bulk file operations
sort, uniq, and reverse numerical sort for data analysis
awk and sort for reformatting data and field-based processing
sed and tee for editing and backup
ps, grep, awk, xargs, and kill for process management
tar and gzip for backup and compression
Advanced: inspecting file descriptors
Conclusion
Automating Tasks with Shell Scripts
Why you need Bash scripting basics
Basics
Variables
Setting
Getting
Bash versus other shells
Shebangs and executable text files, aka scripts
Common Bash settings (options/arguments)
/usr/bin/env
Special characters and escaping
Command substitution
Testing
Testing operators
[[ file and string testing ]]
Useful operators for string testing
Useful operators for file testing
(( arithmetic testing ))
Conditionals: if/then/else
ifelse
Loops
C-style loops
for…in
While
Variable exporting
Functions
Prefer local variables
Input and output redirection
and >>: output redirection
Use 2>&1 to redirect STDERR and STDOUT
Variable interpolation syntax – ${}
Limitations of shell scripts
Conclusion
Citations
Secure Remote Access with SSH
Public key cryptography primer
Message encryption
Message signing
SSH keys
Exceptions to these rules
Logging in and authenticating
Practical project: Set up a key-based login to a remote server
Step 1: Open your terminal on the SSH client (not the server)
Step 2: Generate the key pair
Step 3: Copy the public key to your server
Step 4: Test it out!
Converting SSH2 keys to the OpenSSH format
What we are trying to achieve
How to convert the SSH2-formatted key to OpenSSH
The other direction: Converting SSH2 keys to the OpenSSH format
SSH-agent
Common SSH errors and the -v (verbose) argument
File transfer
SFTP
SCP
Clever examples
Without SFTP or SCP
Directory upload and .tar.gz compression
Tunnels
Local forwarding
Proxying
The configuration file
Conclusion
Version Control with Git
Some background on Git
What is a distributed version control system?
Git basics
First-time setup
Initialize a new Git repository
Make and see changes
Stage and commit changes
Optional: add a remote Git repository
Pushing and pulling
Cloning a repository
Terms you might come across
Repository
Bare repository
Branch
Main/master branch
HEAD
Tag
Shallow
Merging
Merge commit
Merge conflict
Stash
Pull request
Cherry-picking
Bisecting
Rebasing
Best practices for commit messages
Good commit messages
GUIs
Useful shell aliases
Poor man’s GitHub
Considerations
1. Connect to your server
2. Install Git
3. Initialize a repository
4. Clone the repository
5. Edit the project and push your changes
Conclusion
Containerizing Applications with Docker
How containers work as packages
Prerequisite: Docker install
Docker crash course
Creating images with a Dockerfile
Container commands
docker run
docker image list
docker ps
docker exec
docker stop
Docker project: Python/Flask application container
1. Set up the application
2. Create the Docker image
3. Start a container from your image
Containers vs. virtual machines
A quick note on Docker image repositories
Painfully learned container lessons
Image size
C standard library
Production is not your laptop: outside dependencies
Container theory: namespacing
How do we do Ops with containers?
Conclusion
Monitoring Application Logs
Introduction to logging
Logging on Linux can get... weird
Sending log messages
The systemd journal
Example journalctl commands
Following active logs for a unit
Filtering by time
Filtering for a specific log level
Inspecting logs from a previous boot
Kernel messages
Logging in Docker containers
Syslog basics
Facilities
Severity levels
Configuration and implementations
Tips for logging
Keywords when using structured logging
Severity
Centralized logging
Conclusion
Load Balancing and HTTP
Basic terminology
Gateway
Upstream
Common misunderstandings about HTTP
HTTP statuses
Don’t just check for 200 OK
404 Not Found
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout
Introduction to curl: checking HTTP response status
HTTP headers
Case-insensitive headers
Custom headers
Viewing HTTP headers with curl
HTTP versions
HTTP/0.9
HTTP/1.0 and HTTP/1.1
HTTP/2
HTTP/3 and QUIC
Load balancing
Sticky sessions, cookies, and deterministic hashing
Round-robin load balancing
Other mechanisms
High availability
Troubleshooting redirects with curl
Using curl as an API testing tool
Accepting and displaying bad TLS certificates with curl
CORS
Conclusion
Other Books You May Enjoy
Index