Unix/Linux编程实践读书笔记

Unix/Linux编程实践教程

作者: Bruce Molay
原作名: Understanding UNIX/LINUX Programming
译者: 杨宗源 / 黄海涛 

第1章  Unix系统编程概述  

　　通过三个方法理解学习系统编程

• 分析程序
• 学习系统调用
• 编程实现

对应三个问题

　　what does it do->how does it work->let's write our own version

more 的实现

　　more的实现版本1 

/**
 * read and print 24 lines then pause for a few special commands
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define PAGELEN 24
#define LINELEN 512
void do_more(FILE*);
int see_more();

int main(int ac, char *av[]) {
  FILE* fp;
  if (ac == 1) {
    do_more(stdin);
  } else {
    while (--ac) {
      if ((fp = fopen(*++av, "r")) != NULL) {
        do_more(fp);
        fclose(fp);
      } else {
        exit(1);
      }
    }
  }
  return 0;
}

/**
 * read PAGELEN lines, then call see_more() for further instructions
 */
void do_more(FILE* fp) {
  char line[LINELEN];
  int num_of_lines = 0;
  int see_more(), reply;
  while (fgets(line, LINELEN, fp)) {
    if (num_of_lines == PAGELEN) {
      reply = see_more();
      if (reply == 0)
        break;
      num_of_lines -= reply;
    }
    if (fputs(line, stdout) == EOF)
      exit(1);
    num_of_lines++;
  }
}

/**
 * print message, wait for response, return # of lines to advance
 * q means no, space means yes, CR means one line
 */
int see_more() {
  int c;
  printf("\033[7m more? \033[m");
  while ((c = getchar()) != EOF) {
    if (c == 'q')
      return 0;
    if (c == ' ')
      return PAGELEN;
    if (c == '\n')
      return 1;
  }
  return 0;
}

　　上述版本存在许多问题，比如输入q和空格后，需要按回车，而且输入是有回显的。

输入重定向问题

cat more01.c  | more

cat more01.c  | ./more01

　　more01的标准输入重定向到了cat命令的标准输出，而程序中使用getchar()，从标准输入读取数据，此时就会出现问题。

解决办法是，从标准输入中读入分页数据，改为从键盘读用户输入。

　　more的实现版本2 

/**
 * read and print 24 lines then pause for a few special commands
 * feature of version 0.2: reads from /dev/tty for commands
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define PAGELEN 24
#define LINELEN 512
void do_more(FILE*);
int see_more(FILE*);

int main(int ac, char *av[]) {
  FILE* fp;
  if (ac == 1) {
    do_more(stdin);
  } else {
    while (--ac) {
      if ((fp = fopen(*++av, "r")) != NULL) {
        do_more(fp);
        fclose(fp);
      } else {
        exit(1);
      }
    }
  }
  return 0;
}

/**
 * read PAGELEN lines, then call see_more() for further instructions
 */
void do_more(FILE* fp) {
  char line[LINELEN];
  int num_of_lines = 0;
  int reply;

  FILE* fp_tty = fopen("/dev/tty", "r");
  if (fp_tty == NULL)
    exit(1);

  while (fgets(line, LINELEN, fp)) {
    if (num_of_lines == PAGELEN) {
      reply = see_more(fp_tty);
      if (reply == 0)
        break;
      num_of_lines -= reply;
    }
    if (fputs(line, stdout) == EOF)
      exit(1);
    num_of_lines++;
  }
}

/**
 * print message, wait for response, return # of lines to advance
 * q means no, space means yes, CR means one line
 */
int see_more(FILE* cmd) {
  int c;
  printf("\033[7m more? \033[m");
  while ((c = getc(cmd)) != EOF) {
    if (c == 'q')
      return 0;
    if (c == ' ')
      return PAGELEN;
    if (c == '\n')
      return 1;
  }
  return 0;
}

　　文件/dev/tty，是键盘和显示器的设备描述文件，向这个文件写相当于显示在用户的屏幕上，读相当于从键盘获取用户的输入。即使程序的输入/输出被重定向，程序仍可通过这个文件同终端交换数据。

第2章  用户、文件操作与联机帮助：编写who命令 

概念

• 联机帮助作用与使用方法；
• 文件操作、建立读写；
• 缓冲：用户级缓冲与内核级缓冲；
• 内核模式、用户模式和系统调用的代价；

命令who

who命令能做些什么？

　　执行who命令，查看手册帮助man who，

localhost ~]$ man who
WHO(1) User Commands WHO(1)

NAME
who - show who is logged on

SYNOPSIS
who [OPTION]... [ FILE | ARG1 ARG2 ]

DESCRIPTION
Print information about users who are currently logged in.

-a, --all
same as -b -d --login -p -r -t -T -u

who命令是如何工作的？

从unix中学习unix

• 阅读联机帮助
• 搜索联机帮助
• 阅读.h文件
• 从参阅部分得到启示

阅读联机帮助

localhost ~]$ man who

...

DESCRIPTION

...

If FILE is not specified, use /var/run/utmp. /var/log/wtmp as FILE is common. If ARG1
ARG2 given, -m presumed: 'am i' or 'mom likes' are usual.

...

　　从描述中可以看到通常使用文件 /var/run/utmp. /var/log/wtmp 

搜索联机帮助

localhost ~]$ man -k utmp
dump-utmp (8) - print an utmp file in human-readable format.
getutmp (3) - copy utmp structure to utmpx, and vice versa
getutmpx (3) - copy utmp structure to utmpx, and vice versa
utmp (5) - login records
endutent (3) - access utmp file entries

　　或可直接使用man utmp

从搜索的联机帮助中看到 utmp (5) - login records，查找该帮助

localhost ~]$ man 5 utmp
UTMP(5) Linux Programmer's Manual UTMP(5)

NAME
utmp, wtmp - login records

SYNOPSIS
#include <utmp.h>

DESCRIPTION
The utmp file allows one to discover information about who is currently using the system.
There may be more users currently using the system, because not all programs use utmp log‐ging.

阅读.h文件

　　上述帮助 man 5 utmp，给出了结构体 utmp 的描述

The file is a sequence of utmp structures, declared as follows in <utmp.h> (note that this
is only one of several definitions around; details depend on the version of libc):

当然也可查找阅读.h源文件

localhost ~]$ sudo find / -name "utmp.h"
find: ‘/run/user/1000/gvfs’: Permission denied
/usr/lib/x86_64-redhat-linux6E/include/bits/utmp.h
/usr/lib/x86_64-redhat-linux6E/include/utmp.h
/usr/include/bits/utmp.h
/usr/include/utmp.h

who的工作原理

　　who 通过读文件获取需要的信息，每个登录的用户信息都记录在文件中。文件中的结构存放用户登录信息，读取信息并显示出来实现who操作。

如何编写who

• 从文件中读取数据结构
• 将结构中的信息以适当的方式显示出来

查找联机帮助获取文件读写方法:man -k file | grep read

...
pwrite64 (2) - read from or write to a file descriptor at a given offset
read (2) - read from a file descriptor
read (3p) - read from a file
readahead (2) - perform file readahead into page cache
...

man 2 read

得到函数原型  ssize_t read(int fd, void *buf, size_t count);

SEE ALSO
close(2), fcntl(2), ioctl(2), lseek(2), open(2), pread(2), readdir(2), readlink(2),
readv(2), select(2), write(2), fread(3)

通过open 获取文件描述符。

通过open帮助，找到对close的引用。

因此整个过程使用open,read,close系统调用。

who1版本

/**
 * open, read UTMP file, and show
 */

#include <stdio.h>
#include <utmp.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>

#define SHOWHOST

void show_info(struct utmp*);

int main() {
  struct utmp current_record;
  int utmpfd;
  int reclen = sizeof(current_record);

  printf("Utmp_file = %s\n\n",UTMP_FILE);
  if ((utmpfd = open(UTMP_FILE, O_RDONLY)) == -1) {
    perror(UTMP_FILE);
    exit(1);
  }

  while (read(utmpfd, &current_record, reclen) == reclen)
    show_info(&current_record);
  close(utmpfd);
  return 0;
}

/**
 * display cotents of the utmp struct in human readable form
 * *note* these sizes should not be hardwired
 */
void show_info(struct utmp* utbufp) {
  printf("%-8.8s", utbufp->ut_user);
  printf(" ");
  printf("%-8.8s", utbufp->ut_line);
  printf(" ");
  printf("%10d", utbufp->ut_time);
  printf(" ");
  #ifdef SHOWHOST
  printf("(%s)", utbufp->ut_host);
  #endif
  printf("\n");
}

其中宏UTMP_FILE  为 /var/run/utmp

grep "UTMP_FILE" /usr/include/utmp.h
#define UTMP_FILE _PATH_UTMP
#define UTMP_FILENAME _PATH_UTMP

grep "_PATH_UTMP" /usr/include/*.h
/usr/include/paths.h:#define _PATH_UTMP "/var/run/utmp"
/usr/include/utmp.h:#define UTMP_FILE _PATH_UTMP
/usr/include/utmp.h:#define UTMP_FILENAME _PATH_UTMP
/usr/include/utmpx.h:# define UTMPX_FILE _PATH_UTMPX
/usr/include/utmpx.h:# define UTMPX_FILENAME _PATH_UTMPX

改进地方：正确显示登录时间，去除空白记录。

去除空白记录

utmp结构体中成员 ut_type 值为7的时候，表示这是一个已经登录的用户，据此修改程序消除空白行。

以可读方式显示登录时间

 man -k time | grep transform

 man -k time | grep convert

 man ctime

 编写cp（读和写） 

• 问题1：cp命令能做些什么 
• 问题2：cp命令是如何创建／重写文件的 
• 问题3：如何编写cp 

代码

/**
 * uses read and write with tunable buffer size
 */
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>

#define BUFFERSIZE 4096
#define COPYMODE 0644

void oops(char *, char *);

int main(int argc, char *argv[]) {
  int in_fd, out_fd, n_chars;
  char buf[BUFFERSIZE];

  if (argc != 3) {
    fprintf(stderr, "usage: %s source destination\n", *argv);
    exit(1);
  }

  if ((in_fd = open(argv[1], O_RDONLY)) == -1)
    oops("Cannot open ", argv[1]);

  if ((out_fd = creat(argv[2], COPYMODE)) == -1)
    oops("Cannot create ", argv[2]);

  // copy files
  while ((n_chars = read(in_fd, buf, BUFFERSIZE)) > 0) {
    if (write(out_fd, buf, n_chars) != n_chars)
      oops("Write error to ", argv[2]);
  }

  if (n_chars == -1)
    oops("Read error from ", argv[1]);

  if (close(in_fd) == -1 || close(out_fd) == -1)
    oops("Error closing files", "");

  return 0;
}

void oops(char *s1, char *s2) {
  fprintf(stderr, "Error: %s ", s1);
  perror(s2);
  exit(1);
}

提高文件I/O效率的方法：使用缓冲 

缓冲区的大小对性能的影响 

为什么系统调用需要很多时间 

在who2中添加缓冲区，每次读取多个结构体数据，减少系统调用次数。

/**
 * utmplib.h - function to buffer read from utmp file
 * functions are
 *   utmp_open(filename) - open file
 *     return -1 on error
 *   utmp_next()         - return pointer to next struct
 *     return NULL on eof
 *   ump_close()         - close file
 *
 * read NRECS per read and then doles them out from the buffer
 */

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <utmp.h>

#define NRECS 16
#define NULLUT ((struct utmp *)NULL)
#define UTSIZE (sizeof(struct utmp))

static char utmpbuf[NRECS * UTSIZE];  // storage
static int  num_recs;                 // num stored
static int  cur_rec;                  // next to go
static int  fd_utmp = -1;             // read from

int utmp_open(char *filename) {
  fd_utmp = open(filename, O_RDONLY);
  cur_rec = num_recs = 0;
  return fd_utmp;
}

int utmp_reload() {
  int amt_read;
  amt_read = read(fd_utmp, utmpbuf, NRECS * UTSIZE);
  num_recs = amt_read / UTSIZE;
  cur_rec = 0;
  return num_recs;
}

struct utmp * utmp_next() {
  struct utmp *recp;
  if (fd_utmp == -1)
    return NULLUT;
  if (cur_rec == num_recs && utmp_reload() == 0)
    return NULLUT;

  recp = (struct utmp *)&utmpbuf[cur_rec * UTSIZE];
  cur_rec++;
  return recp;
}

void utmp_close() {
  if (fd_utmp != -1)
    close(fd_utmp);
}

主函数

/**
 * command 'who' with buffered reads
 * - open, read UTMP file, and show
 * - suppresses empty records
 * - formats time nicely
 * - buffers input (using utmplib)
 */

#include <stdio.h>
#include <utmp.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
#include "utmplib.c"

#define SHOWHOST
void show_time(long);
void show_info(struct utmp*);

int main() {
  struct utmp * utbufp;

  if (utmp_open(UTMP_FILE) == -1) {
    perror(UTMP_FILE);
    exit(1);
  }

  while ((utbufp = utmp_next()) != ((struct utmp *)NULL))
    show_info(utbufp);
  utmp_close();
  return 0;
}

/**
 * display cotents of the utmp struct in human readable form
 * *note* these sizes should not be hardwired
 */
void show_info(struct utmp* utbufp) {
  if (utbufp->ut_type != USER_PROCESS) // user only
    return;

  printf("%-8.8s", utbufp->ut_name);
  printf(" ");
  printf("%-8.8s", utbufp->ut_line);
  printf(" ");
  show_time(utbufp->ut_time);
  #ifdef SHOWHOST
  if (utbufp->ut_host[0] != '\0')
    printf("(%s)", utbufp->ut_host);
  #endif
  printf("\n");
}

/**
 * display time in a format fit for human consumption
 * use ctime to build a string then picks pars out of it
 * Node: %12.12s prints a string 12 chars wide and LIMITS
 * it to 12 chars
 */
void show_time(long timeval) {
  char *cp;
  cp = ctime(&timeval); // looks like: Mon Feb 4 00:46:40 EST 1991
  printf("%12.12s", cp + 4);
}

内核缓冲技术 

　　应用缓冲技术对提高系统效率是明显的，主要思想是一次读入大量的数据存入缓冲区，需要的时候然后从缓冲区读取数据。

　　内核模式和用户模式之间的切换需要时间，相比之下，磁盘的I/O操作消耗更多时间，效率更低，因此也需要缓冲技术。

　　内核将磁盘上的数据块复制到内核缓冲区，当一个用户空间中的进程需要从磁盘上读数据时，内核一般不直接读磁盘，而是将内核缓冲区的数据复制到进程缓冲区。

　　从理论上讲，内核可以在任何时候写磁盘，但不是任意一次write操作都会导致内核的写动作，内核会把要写的数据缓存在内核缓冲区，积累到一定数量后，再一次性写入磁盘。

　　应用内核缓冲技术结果：

• 提高磁盘I/O效率，
• 优化磁盘写操作，
• 需及时的将缓存数据写入磁盘。

 第3章  目录与文件属性：编写Is 

问题1：ls命令能做什么 

问题2：ls是如何工作的 

man -k direct | grep read

readdir (3) - read a directory
readdir_r (3) - read a directory
seekdir (3) - set the position of the next readdir() call in the directory stream.
readdir (2) - read directory entry
readdir (3p) - read a directory
readdir_r (3p) - read a directory
readlinkat (2) - read value of a symbolic link relative to a directory file descriptor

问题3：如何编写ls 

ls算法如下

main

　　opendir

　　while(readdir)

　　　　print dname

　　closedir

实现代码ls1

/**
 * purpose list contents of directory or directories
 * action if no args, user . else list files args
 */

#include <stdio.h>
#include <unistd.h>
#include <dirent.h>

void do_ls(char[]);

int main(int argc, char *argv[]) {
  if (argc == 1) {
    do_ls(".");
  } else {
    while (--argc) {
      printf("%s:\n", *++argv);
      do_ls(*argv);
    }
  }
}

/**
 * list files in directory called dirname
 */
void do_ls(char dirname[]) {
  DIR *dir_ptr;
  struct dirent *direntp;

  if ((dir_ptr = opendir(dirname)) == NULL) {
    fprintf(stderr, "ls1: cannot open %s \n", dirname);
  } else {
    while ((direntp = readdir(dir_ptr)) != NULL) {
      printf("%s\n", direntp->d_name);
    }
    closedir(dir_ptr);
  }
}

　　上述版本还能做什么

　　排序，分栏，隐藏. ..目录，选项 -l等

编写ls -l

  问题1：ls-l能做些什么

  问题2：ls-l是如何工作的 

 man -k file | grep -i information

 man -k file | grep status

 实现ls -l

　　显示文件状态信息代码:显示模式，链接数，文件所有者，组，大小，最后修改时间，文件名

/**
 * - use stat() to obtain and print file properties
 * - some members are just numbers
 */
#include <stdio.h>
#include <unistd.h>
#include <sys/stat.h>

void show_stat_info(char *fname, struct stat *buf);

int main(int argc, char *argv[]) {
  struct stat info;

  if (argc > 1) {
    if (stat(argv[1], &info) != -1) {
      show_stat_info(argv[1], &info);
      return 0;
    }
  } else {
    perror(argv[1]);
  }
  return 1;
}

/**
 * display some info from stat in a name = value format
 */
void show_stat_info(char *fname, struct stat *buf) {
  printf("    mode: %o\n", buf->st_mode);
  printf("   links: %lu\n", buf->st_nlink);
  printf("    user: %d\n", buf->st_uid);
  printf("   group: %d\n", buf->st_gid);
  printf("    size: %ld\n", buf->st_size);
  printf(" modtime: %ld\n", buf->st_mtime);
  printf("    name: %s\n", fname);
}

编译运行结果如下

cc -o fileinfo fileinfo.c
@localhost]$ ./fileinfo fileinfo.c
mode: 100644
links: 1
user: 1000
group: 1000
size: 1152
modtime: 1029894340

将模式字段转化成字符，将用户/组ID转换为字符串

/**
 * purpose list contents of directory or directories
 * action if no args, user . else list files args
 * note: users stat and pwd.h and grp.h
 * BUG: try ls2 /tmp
 */

#include <stdio.h>
#include <unistd.h>
#include <dirent.h>
#include <sys/stat.h>
#include <string.h>
#include <time.h>

void do_ls(char[]);
void do_stat(char *);
void show_file_info(char *, struct stat *);
void mode_to_letters(int, char []);
char * uid_to_name(uid_t);
char * gid_to_name(gid_t);

int main(int argc, char *argv[]) {
  if (argc == 1) {
    do_ls(".");
  } else {
    while (--argc) {
      printf("%s:\n", *++argv);
      do_ls(*argv);
    }
  }
}

/**
 * list files in directory called dirname
 */
void do_ls(char dirname[]) {
  DIR *dir_ptr;
  struct dirent *direntp;

  if ((dir_ptr = opendir(dirname)) == NULL) {
    fprintf(stderr, "ls2: cannot open %s \n", dirname);
  } else {
    while ((direntp = readdir(dir_ptr)) != NULL) {
      do_stat(direntp->d_name);
    }
    closedir(dir_ptr);
  }
}

void do_stat(char *filename) {
  struct stat info;
  if (stat(filename, &info) == -1) {
    perror(filename);
  } else {
    show_file_info(filename, &info);
  }
}

/**
 * display the info about filename.
 * the info is stored in struct at * info_p
 */
void show_file_info(char *filename, struct stat * info_p) {
  char modestr[11];

  mode_to_letters(info_p->st_mode, modestr);

  printf("%s", modestr);
  printf("%4d ", (int)info_p->st_nlink);
  printf("%-8s ", uid_to_name(info_p->st_uid));
  printf("%-8s ", gid_to_name(info_p->st_gid));
  printf("%8ld ", (long)info_p->st_size);
  printf("%.12s ", 4 + ctime(&info_p->st_mtime));
  printf("%s\n", filename);
}

/**
 * utility functions
 */
void mode_to_letters(int mode, char str[]) {
  strcpy(str, "----------");
  if (S_ISDIR(mode)) str[0] = 'd';
  if (S_ISCHR(mode)) str[0] = 'c';
  if (S_ISBLK(mode)) str[0] = 'b';

  if (mode & S_IRUSR) str[1] = 'r'; // 3 bits for user
  if (mode & S_IWUSR) str[2] = 'w';
  if (mode & S_IXUSR) str[3] = 'x';

  if (mode & S_IRGRP) str[4] = 'r'; // 3 bits for group
  if (mode & S_IWGRP) str[5] = 'w';
  if (mode & S_IXGRP) str[6] = 'x';

  if (mode & S_IROTH) str[7] = 'r'; // 3 bits for other
  if (mode & S_IWOTH) str[8] = 'w';
  if (mode & S_IXOTH) str[9] = 'x';
}


#include <pwd.h>

/**
 * returns pointer to username associated with uid, uses getpw()
 */
char * uid_to_name(uid_t uid) {
  struct passwd * getpwuid(), *pw_ptr;
  static char numstr[10];

  if ((pw_ptr = getpwuid(uid)) == NULL) {
    sprintf(numstr, "%d", uid);
    return numstr;
  } else {
    return pw_ptr->pw_name;
  }
}

#include <grp.h>

/**
 * returns pointer to group number gid, used getgrgid(3)
 */
char * gid_to_name(gid_t gid) {
  struct group * grp_ptr;
  static char numstr[10];
  if ((grp_ptr = getgrgid(gid)) == NULL) {
    sprintf(numstr, "%d", gid);
    return numstr;
  } else {
    return grp_ptr->gr_name;
  }
}

小结：

• ls命令：可以列出给定目录的内容；显示给定文件的属性信息。ls的使用方法，各个常用命令行选项。理解ls包括三个方面：

 如何分辨给定的目录还是文件，如何列出目录信息，如何读取并显示文件属性 

• 文件树：Unix下磁盘上的文件和目录被组织成一颗目录树，每个节点是目录或者是文件。 
• ls命令的工作方式与who命令相似，只是who是打开文件、读取信息指导末尾，而ls是打开目录，读取信息直到目录末尾。 
• 目录是什么？目录如utmp一样是一个特殊文件，其内容的每一项记录是文件和目录的名字（其实这一项中还包含其他很多有用的信息），且目录文件永远不为空，因为每个目录中包含有“.”“..”两项，分别表示当前目录和上一级目录。 
• 既然ls命令的工作原理同who相似，那么也有一套与之相应的函数：opendir、readdir、closedir、seekdir、telldir、rewinddir。这些函数与文件操作的函数功能类似。 
• readdir读取的便是目录文件中的记录，返回的是指向当前记录的指针。记录的类型是struct direent，这个结构体定义在/usr/include/dirent.h中。