System Adminstration
So far, we've looked at hardware, then the OS that sits on top of it; the networks that link everything together so that they can send each other data, and how we secure that data using some basic (and we mean basic) Cybersecurity primitives, technologies and methods.
Now it's time to loop back around to the operating system and actually start managing it.
This is the role of a Systems Administrator. We're responsible for "building" and setting up the OS so that it's configured how we need it to be, or how the business needs it to be, to support the workloads it will operate. These workloads are software that can do virtually anything - sometimes it will listen on a network port, other times it might run on a schedule (a cronjob, for example) and pull data from a remote API. There's no telling what the business might need it to do.
Generally speaking, Systems Administration (SysAd) involves configuring, locking down, monitoring, and maintaining the operating system.
We're going to be focusing only on Linux. Windows Server is extremely popular too, and although Windows Server provides different interfaces and tools than Linux, the end goal is the same, as our the concepts you'll learn here.
Subjects
- Secure SHell (known as SSH) for remote access
- The shell
- Manual pages
- The root user
- File systems
- File management
- File permissions
- Environment variables
- User management
- Software management
- Service management
- Resource utilisation
Curated Materials
The following curated materails will give you a solid grounding in Systems Administration. It doesn't include everything - it doesn't even include all the major operating systems - but it's most certainly enough for you to get started.
Linux all the way...
All of these materials focus on Ubuntu 22.04 LTS. As concepts, they can be applied to any operating system ever invented because users are users on any operating system; a process is a process; files are files; and so on.
When you understand this, and see an example of it being applied in Linux, you'll know what you need to do regardless of the operating system you're faced with.
Secure SHell (known as SSH) for remote access
SSH is how we remotely access Linux systems. By connecting over SSH, we're able to send commands to the remote system using the command line. Everything we send is encrypted. Authentication is handled in one of three ways:
- Passwords
- SSH Keys
- SSH Certificates
Passwords are not recommended, and you should use keys whereever you can. That being said, SSH Certificates are better than SSH keys, but they're not as widely supported as we'd like.
Check out this resource from DigitalOcean to get started.
The shell
Once you've made a connection to a remote Linux system via SSH, or you've opened a terminal up on a local Linux system, you're presented with a command prompt, which in turn is being handled by a "shell". This shell is how you interact with the system, giving it commands to process for you, and it's the shell that gives you feedback after your command has been processed.
The most common shell you'll find installed on Linux is "Bash".
This resource from DigitalOcean is a great starting point.
Optional/additional:
- https://ryanstutorials.net/linuxtutorial/commandline.php
Manual pages
Manual pages, or "man pages", are a Linux (or Unix) systems's documentation. If installed on the system you're currently operating, then man pages are a great way of looking up how-to use a tool or utility. These pages include details about the item you're looking up, such as command line flags, options you can pass in, files that might be important to the operation of the item, and more.
Is there a man for man?
Yes, you can type man man - try it!
They're a useful thing to have around, and learning to use them can save you a lot of time.
Next level detail on manual pages
This video is 40 minutes long, but goes into crazy detail on manual pages. It's abolsutely worth checking out if you've got the time.
The root user, su, and sudo
When you install Linux, Unix or any Linux/Unix variation, you'll get a user called root as a matter of default. On
Windows, this is the Administrator user. This root user can do anything on the system, and besides a few
safeguards that may (may) be in place, you can destroy your entire system using the root user if you get it
wrong. To prevent this, we (usually) create a non-privileged user and use that account for day-to-day operations,
switching to root occasionally for certain tasks.
Speaking of "switching" to root: we have a command for that called su. This literally means "switch user".
If you're a regular, none root account, then you can use su (on its own) to switch to the root user provider you
know the password. If you're already the root user, and you use (for example) su some_user, then you'll simply
switch to that other user without needing their password - hence why the root user can be dangerous.
Another option is sudo, which stands for "switch uuser, do". This is the most widely used option you're
going to see in your career. It's very common to find Linux distributions in the Cloud that not only come with sudo
preinstalled, but also with passwordless sudo enabled, meaning you and SSH into that user and sudo -i over to the
root user without a password!.
Please don't leave that as is
We highly recommend you use a system like Ansible to provision your system configurations. We talk about Ansible
later on. Using whatever system you want to use, disable passwordless sudo usage. It's just not worth the
security implications.
File Structure + File Systems
How Linux lays out all of its files on disk is something worth understanding. Certain files have a certain funtion, and so they're stored in certain places. That being said, each Linux distribution can change things up and store them in different places. It's not a massive between between distribution, but it's enough that you might need to look it up for that particular distribution.
The most common file system on Linux is the EXT, specifically ext4. This is a great resource on the EXT file system
and even explores its history and decendants:
File management
With a file system in place, you now need to manage files. Files make up pretty much everything on an operating system. Almost everything is basically a file at some point, even complex databases write the data to disk in the form of files. TCP/IP sockets are just files written and read in a special way.
Files are pretty important, so it's critical that you know how-to manage them. The following resources teach you everything you need to know, and when it comes to the projects, below, we're going to have plenty of opportunity to test out your skills.
File permissions
Once you know how-to manipulate files, moving them around on disk and move, you must be able to secure them. File permissions look simple on the surface, but a poorly configured configuration file can reveal passwords and other sensitive materials to the wrong types of people.
Environment variables
All operating systems have environment variables. They're like hidden bits of information dotted around the place. In actual fact, they're pretty important and they're not hidden at all - you simply never knew to look for them.
Environment variables are simple KEY=VALUE pieces of information that are setup on different levels within your
system. Software can read from them, define them, or not bother with them at all. And of course there are special
environment variables too, like $PATH.
Time to learn!
User management
User management is one of those things that's very important, but seldom covered by other learning resources, and to be fair, most organisations don't manage many users on a daily basis. Operating system level users aren't something you're going to be configuring daily, but understanding what's going on is still important. Every process runs as a user (except certain special processes and the very first process on a system), so having the correct users on a system is important because you ultimately execute software as that user.
Executing software as a user is when environment variables, file permissions, etc., all come into play. Firstly, users
can have their own environment variables set, as well as those set globally across the system. The main one $PATH,
which helps the shell to find the location of binaries and other executables on the syste. Secondaly, a user has a UID
(User ID) and a GID (Group ID), and those two simple entities have access to files in the form of permissions.
What does this mean? It means users have access to things, like environment variables and files, and so do the processes they run, because those process run as the user, and so they inherit their permissions (and environment!)
Don't get too bogged down in this right now. It becomes more clear as you practice.
Software management
Without installing software on the servers you're managing, what would be the point of even setting them up? We build computers - virtual or otherwise - because their CPUs, and other components, can execute software for us which in turn can do some pretty powerful things, so installing software is kind of important.
Software can do anything you need it to, especially if you can write it yourself, but mostly we install and manage software that provides networking services for various protocols, like HTTPS and SSH.
The following resource will help up get a solid grasp on managing software installations in Ubuntu and Debian.
Service management
After you've installed software on your system, you're going to want two things to happen:
- You want the software to run when you logout of the system
- And you want the software to run if the system reboots
Remember that software runs as a particular user. If you login to a Linux system, run nginx, and then log out again,
that particular instance (process) of nginx will stop running. Obviously, if you're trying to run a HTTPS server 24/7,
then you cannot have it stopped when you log out and you can't stay logged in forever. That's where we configure
services to run and manage the process(es) for us.
We use systemd (on Linux) to handle this for us. We write a Unit file and then have systemd manage it from there.
Let's learn more about service management and systemd.
Resource utilisation
Once your system is configured with users, software, services, etc., you're going to want to make sure that software isn't consuming every resource on your system, from the CPU and RAM, to disk space and network bandwidth. You must be able to determine the resource usage on a given system, especially during its peak operation time (when it's handling the most users), so that you can design and configure systems to operate accordingly.
This is one of those topics, like the ones before it, that can become overwhelmingly complex very easily. That's why
we're just going to look at the top command today and then get an idea of looking at the basics of systems
utilisation.
Once you've got a grasp on top, check out this article from RedHat (who make RedHat Enterprise Linux and manage
CentOS, too):
Log Files
And finally, after you've got everything running on your system, you're going to want to be able to read log files. Log files are just plain text files that software writes to so that it can inform system administrators (or users) what it's doing, send out warnings or error messages about problems it's facing, or just log activities like users of the software doing certain things like logging in or buying products.
Log files are the most common way in which software engineers can "talk" to themselves (in the none crazy sense) about what their running software is doing. For example, it's very common to find that software has a "debug" mode, which (generally) instructs the software to be (extremely) verbose about what it's doing, as it's doing it, whilst it's running and writing to a log file. That log file can then be used by engineers to debug a problem that a user is facing, or discover the root cause of a crash.
Managing log files can be quite important as they can fill up your system's disk space, causing serious problems such as complete outages. In most cases, log files are shipped off to a remote host without ever being written to the local storage on the server. That enables us to analyse logs in real-time on a remote system and determine if there's anything of interest.
Project
Warning
Missing content.
Next
Warning
Missing content.