The Conduit

Generally, the method of exchanging information between processes can only be through fork or exec transmission to open files, or through the file system. And there are other technologies for interprocess communication - IPC (InterProcess Communication)

(Because different processes have different process space, we can not set up a data structure to make different processes accessible, so we need to use the operating system, which can provide us with such a mechanism. IPC)

Pipeline is the oldest form of IPC in UNIX system, and all UNIX systems provide this communication mechanism.

But it has limitations:

They are semi-duplex (i.e., data can only flow in one direction)

They can only be used between processes with common ancestors. (Typically, a pipeline is created by a process, which then calls fork, which can then be applied between parent and child processes.)

Despite these two limitations, semi-duplex pipelines are still the most commonly used form of IPC.

Creating Pipelines

#include <unistd.h>

int  pipe (int fd[2]) ;

This function returns two file descriptors: fd[0] and fd[1]. The former opens to read and the latter opens to write. Provide a one-way (one-way) data stream.

Single-process pipelines are almost useless, usually calling pipe processes and fork, which creates IPC channels from parent to child processes.

Typical uses of pipelines:

In the following way, two different processes (one is the parent process and the other is the child process) are provided with means of inter-process communication.

First, a pipe is created by a process (which will become the parent) and then fork is invoked to derive a copy of itself.

Next, the parent process closes the readout end of the channel and the child process closes the write end of the same channel. This provides a one-way data flow between parent and child processes.

(Pipeline is the data structure in the kernel, so fork does not replicate pipes. But the file descriptor of the pipeline is in the process space, so after fork, the subprocess gets a copy of the file descriptor of the pipeline.

[Attention]

When one end of the pipe is closed:

When reading a pipeline whose write end has been closed, after all data has been read, read returns 0 to indicate that the end of the file has been reached.

(2) When writing a pipeline whose reader has been closed, a signal SIGPIPE is generated, write returns - 1, and errno is set to EPIPE.

Pipeline Implementation: Client-Server Model

Pipelines are in the kernel, so all data from the client to the server and from the server to the client traverses the user-kernel interface twice: once when writing to the pipeline and once when reading from the pipeline.

//Client-server model:
//The client gets the path of the request file from the standard input and pipes it to the server
//After the server gets the file path from the pipeline, it reads the file content and writes the file content to another pipeline.
//Customers obtain file content from the pipeline and display it in standard output
//
//This startup of the parent-child process does not require synchronization.
//If the server starts first, it will block on read (waiting for client to write messages to the pipeline)
//If client starts first, it will block fgets to get user requests
//So whoever starts first will do.
//
//
#include <sys/types.h>
#Include < unistd.h >// include pipe()
#include <sys/wait.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <fcntl.h>
 
#define MAXLINE 1024
 
void client(int, int) ;
void server(int, int) ;
 
int
main(int argc, char** argv)
{
    int   pipe1[2], pipe2[2] ;
    pid_t childpid ;
 
    //Create two pipes
    pipe(pipe1) ;
    pipe(pipe2) ;
 
    if ((childpid = fork()) == 0) //Subprocesses
    {
        //Write Descriptor for Closing Pipeline 1 and Read Descriptor for Pipeline 2
        close(pipe1[1]) ;
        close(pipe2[0]) ;
 
        //Start the server function
        server(pipe1[0], pipe2[1]) ;
        exit(0) ;
    }
 
    //Paternal process
    //Read Descriptor for Closing Pipeline 1 and Write Descriptor for Pipeline 2
    close(pipe1[0]);
    close(pipe2[1]);
 
    //Start the client function
    client(pipe2[0], pipe1[1]) ;
 
    //Waiting for the child process to finish
    waitpid(childpid, NULL, 0) ;
    exit(0) ;
}
 
void
server(int readfd, int writefd)
{
    int    fd ;
    ssize_t n ;
    char   buff[MAXLINE+1] ;
 
    //Read the request message from the client from the pipeline
    if ((n = read(readfd, buff, MAXLINE)) == 0)
        perror("EOF while readingpathname") ;
    buff[n] = '\0' ;
 
    //Open the file requested by the user in a read-only manner
    if ((fd = open(buff, O_RDONLY)) < 0)
    {
        //Failed to open, returning error information to the user
        snprintf(buff + n, sizeof(buff)-n,":can't open, %s\n",
                 strerror(errno)) ;
        n = strlen(buff) ;
        write(writefd, buff, n) ;
    }
    else //Open Success
    {  //Read the file content and write it to the pipeline
        while ((n = read(fd, buff, MAXLINE))> 0)
            write(writefd, buff, n) ;
        close(fd) ;
    }
}
 
//readfd and writefd are used for docking with pipelines
//client reads the message from the readfd of the pipeline and writes the message to the writefd of the pipeline
void
client(int readfd, int writefd)
{
    size_t  len ;
    ssize_t n ;
    char    buff[MAXLINE] ;
 
    //Get the user's request (file path) from standard input
    fgets(buff, MAXLINE, stdin) ;
    len = strlen(buff) ;
    if (buff[len-1] == '\n')
        len-- ;
 
    //Write the file path requested by the user to the pipeline
    write(writefd, buff, len) ;
 
    //Read the server-side response message from the pipeline and output it to standard output
    while ((n = read(readfd, buff, MAXLINE))> 0)
    {
         write(STDOUT_FILENO, buff, n) ;
    }
}

Pipelining: Synchronization between Father and Son Processes

//Pipes can be used to synchronize parent-child processes
//
//
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
 
static int pfd1[2] ; //The parent process blocked on Pipeline 1 waiting for messages from the child process
static int pfd2[2] ; //The child process blocked on Pipeline 2 waiting for messages from the parent process
 
//Initialization synchronization mechanism
void
TELL_WAIT(void)
{
    if (pipe(pfd1) < 0 || pipe(pfd2) < 0)
        perror("pipe error!\n") ;
}
 
//Notify the parent process (that is, writing a message to Pipeline 1 will wake up the parent process waiting on it)
void
TELL_PARENT(pid_t pid)
{
    if (write(pfd1[1], "c", 1) != 1)
        perror("write error!\n") ;
}
 
//Waiting for the parent process (read blocking on Pipeline 2)
void
WAIT_PARENT(void)
{
    char c ;
    if (read(pfd2[0], &c, 1) != 1)
        perror("read error\n") ;
 
    //Check whether the data is from the parent process (to prevent forgery)
    if (c != 'p')
        perror("WAIT_PARENT:incorrectdata\n") ;
}
 
//Notify the child process (that is, writing a message to Pipeline 2 will wake up the child process waiting on it)
void
TELL_CHILD(pid_t pid)
{
    if (write(pfd2[1], "p", 1) != 1)
        perror("write error!\n") ;
}
 
//Waiting for child processes (read blocking on Pipeline 1)
void
WAIT_CHILD(void)
{
    char c ;
    if (read(pfd1[0], &c, 1) != 1)
        perror("read error\n") ;
 
    //Check whether the data is from a child process (to prevent forgery)
    if (c != 'c')
        perror("WAIT_PARENT:incorrectdata\n") ;
}
 
//----------------------------------------------------------------------------------------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
 
int
main(void)
{
    int i = 0 ;
    int cnt = 0 ;
    pid_t pid ;
 
    //Initialization for synchronization mechanism
    TELL_WAIT() ;
 
    if ((pid = fork()) < 0)
        perror("fork error") ;
    else if (pid > 0) //Paternal process
    {
        for (i = 0; i < 3; ++i)
        {
            printf("From parent :%d\n", i) ;
        }
        TELL_CHILD(pid) ;
        WAIT_CHILD(pid) ;
        for (; i < 6; ++i)
        {
            printf("From parent :%d\n", i) ;
        }
        TELL_CHILD(pid) ;
        return 0 ;
    }
    else //Subprocesses
    {
        WAIT_PARENT(getppid()) ;
        for (i = 0; i < 3; ++i)
        {
            printf("From child : %d\n",i) ;
        }
        TELL_PARENT(getppid()) ;
        WAIT_PARENT(getppid()) ;
        for (; i < 6; ++i)
        {
            printf("From child :%d\n", i) ;
        }
        return 0 ;
    }
}
//As long as there are: blocking-waiting mechanism can be considered to achieve inter-process synchronization. (Signals, pipes, etc.)

popen and pclose functions

The operation of these two functions is to create a pipeline, call fork to produce a subprocess, close the unused end of the pipeline, execute a shell to run the command, and then wait for the command to terminate. (Packing this series of operations)

#include <stdio.h>

FILE* popen(constchar* cmdstring,  const char* type) ;

int      pclose(FILE* fp) ;

popen creates a pipeline between the calling process and the specified command. If the parameter type is "r", the returned file pointer is connected to the standard output of cmdstring. (analogy with file open fopen)

These two functions make it easy for us to use shell commands to assist us in implementing the functions of the program and reduce the amount of code we need to write. )

(popen's role is to redirect the input / output of shell commands)

//Client-server model:
//Realization with cat command
//
#include <sys/types.h>
#Include < unistd.h >// include pipe()
#include <sys/wait.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
 
#define MAXLINE 1024
 
int
main(int argc, char** argv)
{
    size_t n ;
    char  buff[MAXLINE] ;
    char  command[MAXLINE] ;
    FILE* fp ;
 
    //Get the user request file path from standard input
    fgets(buff, MAXLINE, stdin) ;
    n = strlen(buff) ;
    if (buff[n-1] == '\n')
        buff[n-1] = '\0' ;
 
    //Compose the file path and cat into shell commands
    snprintf(command, sizeof(command),"cat %s", buff) ;
    fp = popen(command, "r") ;
 
    //Copy the results of executing shell commands to standard output
    while (fgets(buff, MAXLINE, fp) != NULL)
        fputs(buff, stdout) ;
 
    exit(0) ;
}

Collaborative process

When a program generates the input of a filter and reads the output of the filter, the filter becomes a collaborative process. (that is, its standard input/output is redirected to another process, serving another process only)

(Collaborative process mainly emphasizes that we have transformed the original program for input/output into one for our own use. It emphasizes not to modify the code of the original program to make it work for me. In the previous client-server program, both processes were written by us to operate on the agreed pipeline, while the collaborative process is a black box for us, and its code does not know that there are pipelines.)

popen only provides a one-way pipeline connecting standard input or standard output to another process. For a collaborative process, it has two one-way pipelines connecting to another process -- one receiving its standard input and the other from its standard output. We first write the data to its standard input, after processing, and then read the data from its standard output.

[Attention]

Processing functions of collaborative processes for input/output. If it's a low-level I/O function (write, read), no problem. If it is a standard I/O function, some processing is needed.

Because standard input is a pipeline, the default standard I/O of the system is fully buffered (for terminals, default line buffer), and so is standard output.

Thus, when a collaborative process reads blocking from its standard input, the main process reads blocking from the pipeline. Deadlock occurs (because the standard IO function of the collaborative process is fully buffered and data is not written to the pipeline).

Therefore, change the buffer type of the collaborative process.

setvbuf(stdin, NULL, _IOLBF, 0); // Set line buffer

setvbuf(stdout, NULL, _IOLBF, 0) ;

If we cannot change the source code of the collaboration process, we must make the called collaboration process think that its standard input and output are connected to a terminal. This can be achieved with pseudo-terminals. (Brief)

[Example]

//Collaborative processes:
//Read two numbers from standard input, add them up, and output them to standard output.
//
//
#include <pthread.h>
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
 
#define MAXLINE 1024
 
int
main(void)
{
    int n1, n2 ;
    char line[MAXLINE] ;
 
    setvbuf(stdin, NULL, _IOLBF, 0) ; //Setting up line buffer
    setvbuf(stdout, NULL, _IOLBF, 0) ;
 
    //Getting User Input from Keyboard with Standard IO
    while (fgets(line, MAXLINE, stdin) != NULL)
    {
        if (sscanf(line, "%d%d",&n1, &n2) == 2)
            if (printf("%d\n", n1+n2)== EOF)
                perror("printferror\n") ;
    }
 
    exit(0) ;
}
 
//Master Control Process of Collaborative Process
//
//
#include <unistd.h>
#include <signal.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
 
#define MAXLINE 1024
 
void sig_pipe_handler(int signo);
 
int
main(void)
{
    size_t n ;
    pid_t pid ;
    int  fd1[2], fd2[2] ; //Two Pipelines for Connecting Collaborative Processes
   char  line[MAXLINE] ;
 
    //For pipelines, set up a processing program for SIGPIPE signals.
    if (signal(SIGPIPE, sig_pipe_handler) ==SIG_ERR)
        perror("signal error\n") ;
 
    //Create two pipes for communicating with collaborative processes
    if (pipe(fd1) < 0 || pipe(fd2) < 0)
        perror("pipe error\n") ;
 
    if ((pid = fork()) < 0)
        perror("fork error\n") ;
    else if (pid > 0) //Parent process (sending message to pipeline, getting the result of message processing by cooperative process from pipeline)
    {
        //Close unused pipe file descriptors
        close(fd1[0]) ;
        close(fd2[1]) ;
       
        while ((fgets(line, MAXLINE, stdin)) !=NULL)
        {
           n = strlen(line) ;
            //Write user input to the pipeline and pass it to the collaborative process
            if (write(fd1[1], line, n) != n)
                perror("write error frompipe\n") ;
            //Read feedback from collaborative processes from pipes
            if ((n = read(fd2[0], line,MAXLINE)) < 0)
                perror("read error frompipe\n") ;
            if (n == 0)
            {
                perror("child closepipe\n") ;
                break ;
            }
            //Print the results of the collaborative process to the screen
            line[n] = 0 ;
            if (fputs(line, stdout) == EOF)
                perror("fputserror\n") ;
        }
        exit(0) ;
    }
    else //Subprocesses (connecting two pipes to the input/output of a collaborative process, respectively)
    {
        close(fd1[1]) ;
        close(fd2[0]) ;
       
        //Reset the standard input descriptor to fd1[0] of the pipeline
        if (fd1[0] != STDIN_FILENO)
        {  
            if (dup2(fd1[0], STDIN_FILENO) !=STDIN_FILENO) //dup2 for replicating descriptors
                perror("dup2error\n") ;
            close(fd1[0]) ;
        }
 
        //Reset the standard output descriptor to fd2[1] of the pipeline
        if (fd2[1] != STDOUT_FILENO)
        {  
            if (dup2(fd2[1], STDOUT_FILENO) !=STDOUT_FILENO) //dup2 for replicating descriptors
                perror("dup2error\n") ;
            close(fd2[1]) ;
        }
 
        //Start a collaborative process
        if (execl("./add","add", (char*)0) < 0)
            perror("execl error\n") ;
    }
    exit(0) ;
}
 
 
void
sig_pipe_handler(int signo)
{
    printf("SIGPIPE caught\n");
    exit(1) ;
}