Notes on The Missing Semester: I. Course Overview and the Shell

Record

• Course motivation: Explore the potential of existing tools and learn to develop more tools

• Course structure: 11 lectures, each focusing on a specific topic

• Course topic: The Shell

• What is a shell: It lets you execute programs, enter input, and obtain some semi-structured output

• Using the shell: The shell is a programming environment, so it has variables, conditionals, loops, and functions. When you execute a command in the shell, you are actually executing a short piece of code that the shell can interpret and run. If you ask the shell to execute an instruction that is not a programming keyword the shell understands, it consults the environment variable $PATH, which lists the paths where the shell searches for programs when it receives an instruction. When we run the echo command, the shell understands that it needs to execute the echo program, then searches through a series of directories in $PATH separated by : to find that program by name. When it finds the program, it executes it (assuming the file is an executable program, which later lectures explain in detail). To determine which specific program a program name refers to, use the which program. We can also bypass $PATH by directly specifying the path of the program we want to execute.

• Navigating in the shell: A path in the shell is a set of separated directories, using / on Linux and macOS and \ on Windows. The path / represents the system root directory; all folders are contained under this path, while on Windows each drive has its own root directory (for example: C:\). We assume you are using a Linux file system while studying this course. If a path starts with /, it is an absolute path; all others are relative paths. A relative path is a path relative to the current working directory, which can be obtained with the pwd command. In addition, use the cd command to change directories. In paths, . means the current directory, while .. means the parent directory.

• Connecting programs: In the shell, programs have two main “streams”: their input stream and output stream. When a program tries to read information, it reads from the input stream; when it prints information, it writes to the output stream. Usually, a program’s input and output streams are your terminal. That is, your keyboard is the input and your display is the output. The simplest redirects are < file and > file. These two commands redirect a program’s input and output streams to files, respectively. You can also use >> to append content to a file. With pipes, we can make better use of file redirection. The operator | lets us connect one program’s output to another program’s input.

• Root user: On most Unix-like systems, one type of user is very special: the root user. You should already have noticed that, in the output above, the root user is almost unrestricted: they can create, read, update, and delete any file on the system. Usually we do not log in to the system directly as root, because doing so could damage the system through mistaken operations. Instead, we use the sudo command when needed. As the name suggests, it lets you perform operations as su (short for super user or root). When you encounter a permission denied error, it is usually because you must be root to perform the operation. However, please double-check that you really want to perform it. One thing you must be root to do is write to sysfs files. The system is mounted under /sys, and sysfs files expose some kernel parameters. Therefore, without any specialized tools, you can easily configure the system kernel at runtime. Note that Windows and macOS do not have this file.

Reduce

• The shell is a programming environment

• The shell parses command programs

• Absolute paths and relative paths

• Creating connections between programs

• Root user

• Commands: echo, which, pwd, cd, ls, mv, cp, mkdir, man, sudo, touch, and chmod

Recite

• The shell is a programming environment with variables, conditionals, loops, and functions

• The shell parses command programs; when you execute a command in the shell, you are actually executing a short piece of code that the shell can interpret and run

• An absolute path starts with /; it is an absolute path, while all others are relative paths

• Links created between programs, such as |, >, and <, are executed by the shell, not by each program individually

• When you encounter a permission denied error, root privileges are usually required to execute the operation

• Recommended commands to master: echo (print arguments), which (determine which specific program a program name refers to), pwd (get the path of the current working directory), cd (change directories), ls (view which files a specified directory contains), mv (rename or move files), cp (copy files), mkdir (create a new folder), man (look up program arguments and input/output information), sudo (execute as su [short for super user or root]), touch (create a new file), and chmod (change permissions)

Reflect

1. Detailed explanation of the echo command

Output a specified string or variable

Additional Notes

The echo command is used in the shell to print the value of shell variables or directly output a specified string. The Linux echo command is extremely common in shell programming, and it is also often used to print a variable value in the terminal, so it is necessary to understand how echo is used. The function of the echo command is to display a piece of text on the screen, generally serving as a prompt.

Syntax

echo(选项)(参数)

Options

-e：启用转义字符。
-E: 不启用转义字符（默认）
-n: 结尾不换行

When using the -e option, if the following characters appear in the string, they are handled specially rather than being output as ordinary text:

• \a emits an alert sound;
• \b deletes the previous character;
• \c produces no further output (characters after \c are not output);
• \f starts a new line while the cursor remains in its original position;
• \n starts a new line and moves the cursor to the beginning of the line;
• \r moves the cursor to the beginning of the line without starting a new line;
• \t inserts a tab;
• \v is the same as \f;
• \\ inserts the \ character;
• \nnn inserts the ASCII character represented by nnn (octal);

Arguments

Variable: specifies the variable to print.

Examples

/bin/echo Hello, world!

In the command above, the two words (Hello and world!) are passed to echo as separate arguments, and echo prints them in order, separated by a space

The next command produces the same output:

/bin/echo 'Hello, World!'

However, unlike the first example, the command above provides the single-quoted string ‘Hello, world!’ as one single argument.

Single quotes reliably protect it from shell interpretation, passing special characters and escape sequences to echo literally.

For example, in the bash shell, variable names are preceded by a dollar sign ($). In the next command, the variable name inside quotes is treated literally; outside the quotes, it is converted to its value.

/bin/echo 'The value of $PATH is' $PATH
# The value of $PATH is
# /home/hope/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Print colored text with the echo command:

Text color:

# 这段命令在我的Mac上没有执行成功
echo -e "\e[1;31mThis is red text\e[0m"
This is red text

• \e[1;31m sets the color to red
• \e[0m resets the color

Color codes: reset=0, black=30, red=31, green=32, yellow=33, blue=34, magenta=35, cyan=36, white=37

# 这段执行成功
echo -e "\x1b[30;1m 0 黑色 \x1b[0m"\
"\x1b[31;1m 1 红色 \x1b[0m"\
"\x1b[32;1m 2 绿色 \x1b[0m"\
"\x1b[33;1m 3 黄色 \x1b[0m"\
"\x1b[34;1m 4 蓝色 \x1b[0m"\
"\x1b[35;1m 5 洋红 \x1b[0m"\
"\x1b[36;1m 6 青色 \x1b[0m"\
"\x1b[37;1m 7 白色 \x1b[0m"

Background color:

# 这段也没执行成功
echo -e "\e[1;42mGreed Background\e[0m"
Greed Background

Color codes: reset=0, black=40, red=41, green=42, yellow=43, blue=44, magenta=45, cyan=46, white=47

Blinking text:

# 这段执行成功
echo -e "\033[37;31;5mMySQL Server Stop...\033[39;49;0m"

Other numeric parameters at the red number position: 0 turns off all attributes, 1 sets high intensity (bold), 4 underlines, 5 blinks, 7 reverses display, 8 conceals

Do not add a newline at the end of the output

echo -n 'hello'

2. Detailed explanation of the touch command

Create a new empty file

Additional Notes

The touch command has two functions: first, it updates the timestamps of existing files to the current system time (the default behavior), leaving their data intact; second, it creates new empty files.

Syntax

touch(选项)(参数)

Options

-a：或--time=atime或--time=access或--time=use  只更改存取时间；
-c：或--no-create  不建立任何文件；
-d：<时间日期> 使用指定的日期时间，而非现在的时间；
-f：此参数将忽略不予处理，仅负责解决BSD版本touch指令的兼容性问题；
-m：或--time=mtime或--time=modify  只更该变动时间；
-r：<参考文件或目录>  把指定文件或目录的日期时间，统统设成和参考文件或目录的日期时间相同；
-t：<日期时间>  使用指定的日期时间，而非现在的时间；
--help：在线帮助；
--version：显示版本信息。

Arguments

File: specifies the list of files whose time attributes are to be set.

Examples

touch ex2

Create an empty file ex2 in the current directory; then, using the ls -l command, you can see that file ex2 has a size of 0, indicating that it is an empty file.

Create files in batches

touch file{1..5}.txt

Create the job1.md file and write job 1 into it

echo "job 1" > job1.md

3. Detailed explanation of the chmod command

Used to change the permissions of files or directories

Synopsis

chmod [OPTION]... MODE[,MODE]... FILE...
chmod [OPTION]... OCTAL-MODE FILE...
chmod [OPTION]... --reference=RFILE FILE...

Main Uses

• Change the permissions of the target file or directory using symbolic combinations.
• Change the permissions of the target file or directory using octal numbers.
• Change the permissions of the target file or directory by referring to another file’s permissions.

Arguments

mode: octal number or symbolic combination.

file: specifies one or more files whose permissions are to be changed.

Options

-c, --changes：当文件的权限更改时输出操作信息。
--no-preserve-root：不将'/'特殊化处理，默认选项。
--preserve-root：不能在根目录下递归操作。
-f, --silent, --quiet：抑制多数错误消息的输出。
-v, --verbose：无论文件是否更改了权限，一律输出操作信息。
--reference=RFILE：使用参考文件或参考目录RFILE的权限来设置目标文件或目录的权限。
-R, --recursive：对目录以及目录下的文件递归执行更改权限操作。
--help：显示帮助信息并退出。
--version：显示版本信息并退出。

Return Value

The return status is success unless an invalid option or invalid argument is provided.

Examples

According to the man chmod documentation’s DESCRIPTION section:

• The u symbol represents the current user.
• The g symbol represents users in the same group as the current user, hereafter called group users.
• The o symbol represents other users.
• The a symbol represents all users.
• The r symbol represents read permission and the octal number 4.
• The w symbol represents write permission and the octal number 2.
• The x symbol represents execute permission and the octal number 1.
• The X symbol means execute permission can be set if the target file is an executable file or directory.
• The s symbol represents setting suid and sgid permissions; use the permission combination u+s to set the file user ID bit, and g+s to set the group user ID bit.
• The t symbol means only the owner of a directory or file may delete files under the directory.
• The + symbol means adding the corresponding permissions for the target users.
• The - symbol means removing the corresponding permissions from the target users.
• The = symbol means adding the corresponding permissions for the target users and removing permissions not mentioned.

linux文件的用户权限说明：

# 查看当前目录（包含隐藏文件）的长格式。
ls -la
  -rw-r--r--   1 user  staff   651 Oct 12 12:53 .gitmodules

# 第1位如果是d则代表目录，是-则代表普通文件。
# 更多详情请参阅info coreutils 'ls invocation'（ls命令的info文档）的'-l'选项部分。
# 第2到4位代表当前用户的权限。
# 第5到7位代表组用户的权限。
# 第8到10位代表其他用户的权限。

# 添加组用户的写权限。
chmod g+w ./test.log
# 删除其他用户的所有权限。
chmod o= ./test.log
# 使得所有用户都没有写权限。
chmod a-w ./test.log
# 当前用户具有所有权限，组用户有读写权限，其他用户只有读权限。
chmod u=rwx, g=rw, o=r ./test.log
# 等价的八进制数表示：
chmod 764 ./test.log
# 将目录以及目录下的文件都设置为所有用户拥有读写权限。
# 注意，使用'-R'选项一定要保留当前用户的执行和读取权限，否则会报错！
chmod -R a=rw ./testdir/
# 根据其他文件的权限设置文件权限。
chmod --reference=./1.log  ./test.log

Notes

1. This command is part of the GNU coreutils package. For related help information, see man chmod or info coreutils 'chmod invocation'.

2. The permissions of symbolic links cannot be changed. If a user changes permissions on a symbolic link, the change applies to the original file being linked to.

3. When using the -R option, be sure to preserve the current user’s execute and read permissions, otherwise an error will occur!

Review

• Master commands such as echo, touch, and chmod
• Review the original example code
• Try writing things by hand

Course

• The Missing Semester of Your CS Education
• Course Overview and the Shell