01. Introduction to libcurl

I wrote a blog about the introduction of libcurl. If you don't know, you can turn to it: 2019 configure Http protocol and libcurl third-party library for POST communication

02. libcurl test

After completing the above blog, we can write a small project to test the effect. We take Baidu as an example to test the capture: https://www.baidu.com/ Content for this site.

An instance step of quoting the official website is the original template

/***************************************************************************
 *                                  _   _ ____  _
 *  Project                     ___| | | |  _ \| |
 *                             / __| | | | |_) | |
 *                            | (__| |_| |  _ <| |___
 *                             \___|\___/|_| \_\_____|
 *
 * Copyright (C) 1998 - 2021, Daniel Stenberg, <daniel@haxx.se>, et al.
 *
 * This software is licensed as described in the file COPYING, which
 * you should have received as part of this distribution. The terms
 * are also available at https://curl.se/docs/copyright.html.
 *
 * You may opt to use, copy, modify, merge, publish, distribute and/or sell
 * copies of the Software, and permit persons to whom the Software is
 * furnished to do so, under the terms of the COPYING file.
 *
 * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
 * KIND, either express or implied.
 *
 ***************************************************************************/
/* <DESC>
 * Shows how the write callback function can be used to download data into a
 * chunk of memory instead of storing it in a file.
 * </DESC>
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
#include <curl/curl.h>

//Define a structure to record the content and data length of each callback
typedef struct MemoryStruct {
  char *memory;
  size_t size;
}MemoryStruct;

//Define a global static function to respond to the data of the server
/* Parameter description
*	void* contents: This parameter is the source data of the server response, that is, the data sent by the server (original data)
*	size_t size:  This parameter is the number of blocks sent (similar to the second parameter in the fwrite function)
*	size_t nmemb: This parameter is the number and size of blocks sent (similar to the third parameter in fwrite function)
*	void* userp:  This parameter is a pointer for external interaction, usually (FILE *). Take the actual situation as a reference, see the following usage
*/
static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
  size_t realsize = size * nmemb;  //Total data size of this callback
  struct MemoryStruct *mem = (struct MemoryStruct *)userp; //External application space and internal data addition method
 //Dynamically apply for space for structure member memory
  char *ptr = realloc(mem->memory, mem->size + realsize + 1);
  if(!ptr) {
    /* out of memory! */
    printf("not enough memory (realloc returned NULL)\n");
    return 0;
  }
 
  mem->memory = ptr;
  memcpy(&(mem->memory[mem->size]), contents, realsize);
  mem->size += realsize;
  mem->memory[mem->size] = 0;
 
  return realsize;  //As long as realsize is not 0, the callback will always be called
}

int main(void)
{
  CURL *curl_handle;
  CURLcode res;
 
  struct MemoryStruct chunk;
  struct curl_slist* headers = NULL; //http message header
 
  chunk.memory = malloc(1);  /* will be grown as needed by the realloc above */
  chunk.size = 0;    /* no data at this point */
 
  curl_global_init(CURL_GLOBAL_ALL);
 
  /* init the curl session */
  curl_handle = curl_easy_init();

   
 
  /* specify URL to get */
  curl_easy_setopt(curl_handle, CURLOPT_URL, "http://www.baidu.com/");
 
  /* send all data to this function  */
  curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
 
  /* we pass our 'chunk' struct to the callback function  This parameter is the 4th parameter of callback*/ 
  curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);
 
  /* some servers don't like requests that are made without a user-agent
     field, so we provide one */
  curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
 
  /* get it! */
  res = curl_easy_perform(curl_handle);
 
  /* check for errors */
  if(res != CURLE_OK) {
    fprintf(stderr, "curl_easy_perform() failed: %s\n",
            curl_easy_strerror(res));
  }
  else {
    /*
     * Now, our chunk.memory points to a memory block that is chunk.size
     * bytes big and contains the remote file.
     *
     * Do something nice with it!
     */
     char* buf = chunk.memory;  
 	FILE* fptr = fopen(strFilePath,"wb");
 	if(!fptr)/*strFilePath Is the file path (full and named path)*/
 	{
		printf("fopen is failed......\n");
		return 0;
	}

	size_t bRet = fwrite(buf,1,Chunk.size,fptr);
	if(bRet >0)
	{
		printf("write Content to file is success!");
		return 0;
	}	

	fclose(fptr);
    printf("%lu bytes retrieved\n", (unsigned long)chunk.size);
  }
 
  /* cleanup curl stuff */
  curl_easy_cleanup(curl_handle);
 
  free(chunk.memory);
  chunk.memory = NULL;
  /* we're done with libcurl, so clean it up */
  curl_global_cleanup();
 
  return 0;
}

03. fwrite added mode description

C function fopen can set multiple read-write modes for open files. It is necessary to explain what fwrite's append mode is. Many articles on the Internet are unclear. They only say that a is append and w cannot be appended. In fact, there are many misunderstandings, which need to be clarified here (ok through case test):

1. w: indicates that the fopen file will clear the information of the original file (if any) and write it again. Without fclose file, multiple fwrites are also added to the end of the file and will not overwrite the contents of the previous fwrites.

2. a: indicates that the fopen file will retain the information of the original file (if any) and add it to the end for writing. Each time fwrite is written to the end of the file.

Other instructions:
w: Text write, write only
w +: readable and writable
wb: binary write, write only
wb +: binary write or read, readable and writable
a: Append write to fopen file in text form
ab: append write to fopen file in binary form
ab +: additional writing or reading of fopen files, in binary form

04. Summary

About libcurl, I wrote a basic introduction before, but in the later application, there were many bug s in the use process, resulting in the abnormal shutdown of the program. I always thought that there was a problem in this communication. I was confused for 2 ~ 3 weeks, but I couldn't solve it. Finally, I downloaded the source code and found that the callback function must be a static function outside the class, Otherwise, the callback address will be different every time, leading to an illusion that the address of the fourth parameter in the transfer callback function is changing, resulting in the content is always wrong. I'd like to record my libcurl usage experience in this article.