Producer and Consumer Model

Posted by ghostrider1 on Wed, 26 Jun 2019 01:26:46 +0200

I. What is the producer-consumer model
In real life, we may encounter the following situations: one module is responsible for generating data, the other module is responsible for processing data. Modules used to generate data become producers, and modules used to process data become consumers. Only producers and consumers are not enough. This model must also have a buffer between producers and consumers as an intermediary. The producer puts the data in the buffer, while the consumer takes the data out of the buffer.

Conditions and uses of producer-consumer models
To illustrate the relationship between producers and consumers in one sentence, that is, the "three, two, one principle" carefully refers to "three relationships, two types of objects, one trading place".
Three relationships: producer and consumer
Two types of objects: three relationships are: producer and consumer (mutual exclusion and synchronization), producer and producer (mutual exclusion), consumer and consumer (mutual exclusion).
A trading place: A trading place is a warehouse for data exchange between producers and consumers, which is equivalent to a buffer. The producer is responsible for putting the data in the buffer, while the consumer is taking the data out of the buffer for processing.
Characteristics between producers and consumers:
1. The producer only cares about the buffer zone, and does not need to care about the specific situation of consumers.
2. For consumers, they don't need to care about the specific producer. They just need to care about whether there is data in the buffer.
3. Consumers can not consume when producers produce, and producers can not produce when consumers consume. It is equivalent to a mutually exclusive relationship, that is, producers and consumers can only have one access buffer at the same time.
4. Buffers cannot be consumed when they are empty.
5. When the buffer is full, production cannot be carried out.
3. Why use the producer-consumer model?
1. Supporting decoupling
Assuming that producers and consumers are two classes, if the producers call a function of consumers directly, there will be dependencies (coupling) between producers and consumers. If the consumer's code changes, it may affect the producer, and if both rely on a buffer, they do not depend directly on each other, and the coupling decreases.
2. Supporting concurrency
One drawback of a method that producers call consumers is that the function calls are synchronized (or blocked) and the producers have to block until the consumer's method does not return. If consumers process data slowly, producers will waste their time. After using the producer-consumer model, producer-consumer is two independent subjects. The producer can produce the next data by putting the produced data in the buffer.
3. Supporting uneven busyness and leisure
Another advantage of buffers is that if data is generated faster or slower, the benefits of buffers are reflected. When the data is manufactured so fast that the data can not be processed, the data can be temporarily stored in the buffer, and can continue to be used when the data production is slow.

4. What are conditional variables
The above is about synchronization and mutually exclusion to achieve the producer and consumer model. We also need to solve a problem: if you go shopping in the mall and find that the shelves are empty, the supermarket does not provide you with any goods to buy. Would you be disappointed? To avoid this, we need conditional variables to help us. When there is a commodity, you will be informed to buy, when there is no commodity, always wait.
Conditional variable:
Conditional variable is a mechanism for synchronizing global variables shared among threads. It mainly includes two actions: one thread waits for the condition of condition variable to be established; the other thread makes the condition to be established (giving condition to signal). So conditional variables allow us to sleep and wait for certain conditions to occur.
To prevent competition, the use of conditional variables is always matched with a mutex.
V. Usage of Conditional Variables
A conditional variable returns 0 if it succeeds, and an error code if it fails.
Data types of conditional variables:

pthread_cond_t

Initialization of conditional variables:
1. Define a global conditional variable directly and initialize it with PTHREAD_COND_INITIALIZER.
2. Initialize by calling the function pthread_cond_init.

pthread_cond_t (pthread_cond_t *restrict cond,
              const pthread_condattr_t *restrict attr);

The first parameter is cond, which is dynamically assigned to us. Unless a condition variable with a non-default attribute is created, the second parameter attr is always NULL.

Note: If you do not want to define conditional variables as global, you must create them dynamically.

pthread_cond_t *cond = (pthread_cond_t*)malloc(sizeof(pthread_cond_t));

Destruction of conditional variables: call the pthread_cond_destroy function to destroy conditional variables
Waiting for conditional variables:
Waiting for conditional variables to become true using pthread_cond_wait or pthread_cond_timewait functions

int pthread_cond_timedwait(pthread_cond_t *restrict cond,pthread_mutex_t *restrict mutex,const struct timespec *restrict abstime);

int pthread_cond_wait(pthread_cond_t *restrict cond,pthread_mutex_t *restrict mutex);

pthread _cond_wait:
The first parameter is a pointer to the conditional variable, and the second parameter is a pointer to the mutex. Conditional variables are always used in conjunction with a mutex. The aim is to prevent competition for resources. The second parameter of the wait function is to release the lock resource in time when the consumer applies for the lock, but the condition variable is false (no lock resource).
There is a mutually exclusive relationship between producers and consumers. We can't access trading venues at the same time, so we add a lock to coordinate the two.
Chestnut:
Suppose there are two consumers A and B waiting for resources (waiting for the condition variable to become true). At this time, the producer applied for the lock and produced a commodity. Then the lock resource was released and a signal was sent to the consumer: "You can buy it."
Consumer A took the lead in snatching the lock and buying the "commodity". After that, consumer B applied for the lock and found that there was no merchandise in the shopping mall. At this time, the condition variable was false. Consumer B waits until the conditional variable is true. But at this time, it is locked on B. If B does not release the lock and waits all the time, the producer can't produce "goods" because he can't apply for the lock. Consumer B waits all the time because he has no goods. This will cause a "deadlock" problem.
6. Examples of Producer-Consumer Model
The example of producer and consumer is that producer produces a data that is stored at the head of a list and placed on the head in turn. Consumers take data from the head of the list every time. The code is as follows:

#include<stdio.h>
#include<stdlib.h>
#include<pthread.h>
#include<assert.h>
#include<unistd.h>

typedef struct Node
{
    int data;
    struct Node* next;
}Node,*Node_p,**Node_pp;

Node_p CreatNode(int data)
{
    Node_p _n = (Node_p)malloc(sizeof(Node));
    if(_n == NULL)
    {
        return NULL;
    }
    _n->data = data;
    _n->next = NULL;
    return _n;
}

void Init(Node_pp list)
{
    *list = CreatNode(0);
}

void PushFront(Node_p list,int data)
{
    assert(list);
    Node_p _n = CreatNode(data);
    _n->next = list->next;
    list->next = _n;

}

void DelNode(Node_p node)
{
    assert(node);
    free(node);
    node = NULL;
}

int Isempty(Node_p list)
{
    assert(list);
    if(list->next != NULL)
    {
        return 0;
    }
    {
        return 1;
    }
}

void PopFront(Node_p list,int *data)
{
    assert(data);
    if(Isempty(list))
    {
        printf("The list is empty!\n");
        return;
    }
    Node_p _del = list->next;
    list->next = _del->next;
    *data = _del->data;
    DelNode(_del);
}

void Destory(Node_p list)
{
    int data = 0;
    assert(list);
    while(list->next)
    {
        PopFront(list,&data);
    }
    free(list);
    list = NULL;
    printf("list if destroy..\n");
}

void Print(Node_p list)
{
    assert(list);
    Node_p cur = list->next;
    while(cur)
    {
        printf("%d->",cur->data);
        cur = cur->next;
    }
    printf("\n");
}

pthread_mutex_t mylock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t mycond = PTHREAD_COND_INITIALIZER;

void* Consumer(void *arg)
{
    Node_p head = (Node_p)arg;
    while(1)
    {
        int data = 0;
        pthread_mutex_lock(&mylock);
        if(Isempty(head))
        {
            pthread_cond_wait(&mycond,&mylock);
        }
        PopFront(head,&data);
        printf("consumer: %d\n",data);
        pthread_mutex_unlock(&mylock);
    }
    return NULL;
}

void* Producer(void *arg)
{
    Node_p head = (Node_p)arg;
    while(1)
    {
        usleep(123456);
        int data = rand()%1000;
        pthread_mutex_lock(&mylock);
        PushFront(head,data);
        printf("Producer: %d\n",data);
        pthread_cond_signal(&mycond);
        pthread_mutex_unlock(&mylock);
    }
    return NULL;
}


int main()
{
    Node* head = NULL;
    Init(&head);
    pthread_t tid1,tid2;
    int ret1 = pthread_create(&tid1,NULL,Consumer,(void*)head);
    int ret2 = pthread_create(&tid2,NULL,Producer,(void*)head);
    printf("ret1:%d,ret2:%d\n",ret1,ret2);
    pthread_join(tid1,NULL);
    pthread_join(tid2,NULL);
    pthread_mutex_destroy(&mylock);
    pthread_cond_destroy(&mycond);
    return 0;
}

Seven: Signals
We can use semaphores to implement the producer and consumer models in addition to the above method. Suppose there are only five shelves in the mall where goods are stored. Only one item can be placed in each lattice. These five lattices can be recycled. Think of it as a circular queue.

Under the circular queue, producers and consumers need to follow the following rules: producers first, consumers can never catch up with producers, producers can not exceed consumers. (Imagine a race between two people)
The producer pays attention to lattice resources while the consumer pays attention to commodity resources. We can define two semaphores to represent these two resources respectively.
The data type of semaphore is sem_t, and the header file of function interface is semaphore.h.

#include <semaphore.h>

int sem_init(sem_t *sem, int pshared, unsigned int value);
int sem_destroy(sem_t *sem);
int sem_wait(sem_t *sem);
int sem_trywait(sem_t *sem);
int sem_post(sem_t *sem);

VIII: Semaphore to Realize Producer-Consumer Model
To enable semaphores to implement producer and consumer models instead of mutexes and conditional variables, we need to implement the data now in a ring-to-column.
The code is as follows:

#include<stdio.h>
#include<pthread.h>
#include<semaphore.h>

int ring[64];  //Ring queue
sem_t blank;  //A signal that represents a space.
sem_t datas;  //A semaphore that represents data.

void* Producer(void* arg)
{
    int i = 0;
    while(1)
    {
        sleep(1);
        sem_wait(&blank);
        int data = rand()%10000;
        ring[i] = data;
        i++;
        i %= 64;
        printf("produce: %d\n",data);
        sem_post(&datas);
    }
}

void* Consumer(void* arg)
{
    int i = 0;
    while(1)
    {
        sleep(1);
        sem_wait(&datas);
        int data = ring[i];
        i++;
        i %= 64;
        printf("consumer: %d\n",data);
        sem_post(&blank);
    }
}


int main()
{
    sem_init(&blank,0,64);
    sem_init(&datas,0,0);
    pthread_t producer;
    pthread_t consumer;
    pthread_create(&producer,NULL,Producer,NULL);
    pthread_create(&consumer,NULL,Consumer,NULL);
    pthread_join(producer,NULL);
    pthread_join(consumer,NULL);

    sem_destroy(&blank);
    sem_destroy(&datas);
    return 0;
}

Topics: Attribute