SSL encountered at work_ Read related issues, for SSL_ There are some questions about the read mechanism of read:
SSL_ What is the difference between read and read and recv?.
SSL_ Does read cause the inconsistency between the data to be read and the data actually returned?. SSL_ What is the behavior of read under blocking and non blocking sockets?.
With these questions, first of all, SSL in man page_ The definition of read is understood.
SSL_read function definition
SSL_read(3SSL) OpenSSL SSL_read(3SSL)
NAME
SSL_read - read bytes from a TLS/SSL connection.
SYNOPSIS
#include <openssl/ssl.h>
int SSL_read(SSL *ssl, void *buf, int num);
DESCRIPTION
SSL_read() tries to read num bytes from the specified ssl into the buffer buf.
NOTES
If necessary, SSL_read() will negotiate a TLS/SSL session, if not already explicitly performed by SSL_connect(3) or SSL_accept(3). If the peer requests a re-negotiation, it will be
performed transparently during the SSL_read() operation. The behaviour of SSL_read() depends on the underlying BIO.
For the transparent negotiation to succeed, the ssl must have been initialized to client or server mode. This is being done by calling SSL_set_connect_state(3) or SSL_set_accept_state()
before the first call to an SSL_read() or SSL_write(3) function.
SSL_read() works based on the SSL/TLS records. The data are received in records (with a maximum record size of 16kB for SSLv3/TLSv1). Only when a record has been completely received, it can
be processed (decryption and check of integrity). Therefore data that was not retrieved at the last call of SSL_read() can still be buffered inside the SSL layer and will be retrieved on
the next call to SSL_read(). If num is higher than the number of bytes buffered, SSL_read() will return with the bytes buffered. If no more bytes are in the buffer, SSL_read() will trigger
the processing of the next record. Only when the record has been received and processed completely, SSL_read() will return reporting success. At most the contents of the record will be
returned. As the size of an SSL/TLS record may exceed the maximum packet size of the underlying transport (e.g. TCP), it may be necessary to read several packets from the transport layer
before the record is complete and SSL_read() can succeed.
If the underlying BIO is blocking, SSL_read() will only return, once the read operation has been finished or an error occurred, except when a renegotiation take place, in which case a
SSL_ERROR_WANT_READ may occur. This behaviour can be controlled with the SSL_MODE_AUTO_RETRY flag of the SSL_CTX_set_mode(3) call.
If the underlying BIO is non-blocking, SSL_read() will also return when the underlying BIO could not satisfy the needs of SSL_read() to continue the operation. In this case a call to
SSL_get_error(3) with the return value of SSL_read() will yield SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE. As at any time a re-negotiation is possible, a call to SSL_read() can also cause
write operations! The calling process then must repeat the call after taking appropriate action to satisfy the needs of SSL_read(). The action depends on the underlying BIO. When using a
non-blocking socket, nothing is to be done, but select() can be used to check for the required condition. When using a buffering BIO, like a BIO pair, data must be written into or retrieved
out of the BIO before being able to continue.
SSL_pending(3) can be used to find out whether there are buffered bytes available for immediate retrieval. In this case SSL_read() can be called without blocking or actually receiving new
data from the underlying socket.
Read the third paragraph carefully to learn about ssl_ Working mechanism of read, ssl_ The read() function is implemented based on ssl/tls records. The data is received into the records. In SSLv3 and TLSv1 protocols, the maximum size of a single record is 16kB. Only after a record is completely read can it be parsed (including decryption and authentication). Therefore, the ssl layer may buffer some data every ssl_ The number of bytes read by read() may not be all the data in a record, but SSL_read reads from the socket with record as the minimum unit. The data not read in the previous record will be read before the ssl_ When read is read, it is cached in the ssl layer and waits for the next SSL_read() is called.
If num (that is, the expected size to be read) is greater than the actual cached byte size, SSL_read() returns the actual size of the bytes that have been cached. SSL if there is no data in the cache_ Read () triggers the TCP stream to be read and generates the next record. SSL is enabled only after a record is completely read and decrypted for authentication_ Read() will return success. Because the size of a single SSL/TLS record may exceed the maximum size of a single packet of a lower layer protocol (such as TCP), in order to maintain the integrity and SSL of a record_ If read() can return successfully, you may need to read multiple packets from the transport layer to ensure.
If the underlying io socket is a blocking socket, SSL_read will be returned only when a complete read operation is completed or an exception occurs. Unless renegotiation occurs, SSL will be returned_ ERROR_ WANT_ Read error. However, this behavior can also be done by calling SSL_CTX_set_mode(3) interface setting SSL_ MODE_ AUTO_ Control with the retry flag.
If the underlying IO is a non blocking socket, SSL_read() will also fail to meet SSL in the underlying bio_ Returned when required by read(). In this case, through SSL_get_error(3) call SSL_ The return value of read() will return SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE. Renegotiation may occur at any time, so SSL is called at this time_ Read() may cause a write operation because the negotiation content needs to be sent. After taking appropriate measures to meet SSL_ After the execution conditions of read (), the calling process needs to be repeated. The specific measures depend on the underlying BIO type. No additional operations are required when using non blocking sockets, but select() can be used to detect suitable matching conditions (SSL_read()). When using IO with buffer, such as BIO pair, data must be written or read from bio before subsequent operations.
It can be seen from the above description that in theory, the data packets returned by ssl are not streaming, but packet. When the client contracts, the packets sent by calling the ssl interface of the corresponding sdk will be thrown out in the form of a complete record when receiving, but the upper layer can be read in a streaming manner.
SSL record screenshot
The data of a single SSL record is shown in the following figure:
The type of the SSL packet is 23, that is, the data type. The protocol is TLS1.2 and the content is 8274 bytes. It is composed of seven TCP packets to form a complete SSL packet. It can be seen from the serial number 2942 that the TCP packet is 1506 bytes, but for the SSL packet, the valid content of the packet is only 669 bytes. It can be seen that the TCP packet contains part of the previous SSL packet and part of the next SSL packet. It can be seen that the TCP packet is indeed completely streaming.
From the above, generally speaking, ssl_ Generally, the read record will be a complete record, because it has been processed by a layer of ssl protocol. Unlike tcp, the content read by recv is purely a stream.
SSL_read source code analysis
Refer to the source code of openssl-1.0.2l to sort out SSL_ The interface calling process of read is as follows:
SSL_ The code implementation of read is as follows:
int SSL_read(SSL *s, void *buf, int num)
{
if (s->handshake_func == 0) {
SSLerr(SSL_F_SSL_READ, SSL_R_UNINITIALIZED);
return -1;
}
if (s->shutdown & SSL_RECEIVED_SHUTDOWN) {
s->rwstate = SSL_NOTHING;
return (0);
}
return (s->method->ssl_read(s, buf, num));
}
Known ssl_read is the function callback pointer of the method struct member. SSL is the input parameter of external initialization, which is controlled by SSL_ The new interface is initialized and defined as follows
SSL *SSL_new(SSL_CTX *ctx);
SSL_new is implemented in SSL_ In lib. C, the method structure members are initialized as follows:
SSL *SSL_new(SSL_CTX *ctx)
{
SSL *s;
if (ctx == NULL) {
SSLerr(SSL_F_SSL_NEW, SSL_R_NULL_SSL_CTX);
return (NULL);
}
if (ctx->method == NULL) {
SSLerr(SSL_F_SSL_NEW, SSL_R_SSL_CTX_HAS_NO_DEFAULT_SSL_VERSION);
return (NULL);
}
...
s->method = ctx->method;
...
}
That is, SSL_ method definition in CTX. SSL_CTX is generally defined in the following form during external initialization,
pSslCtx = SSL_CTX_new(TLSv1_2_client_method());
Its definition is also in SSL_ In lib. C
SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth)
{
SSL_CTX *ret = NULL;
if (meth == NULL) {
SSLerr(SSL_F_SSL_CTX_NEW, SSL_R_NULL_SSL_METHOD_PASSED);
return (NULL);
}
#ifdef OPENSSL_FIPS
if (FIPS_mode() && (meth->version < TLS1_VERSION)) {
SSLerr(SSL_F_SSL_CTX_NEW, SSL_R_ONLY_TLS_ALLOWED_IN_FIPS_MODE);
return NULL;
}
#endif
if (SSL_get_ex_data_X509_STORE_CTX_idx() < 0) {
SSLerr(SSL_F_SSL_CTX_NEW, SSL_R_X509_VERIFICATION_SETUP_PROBLEMS);
goto err;
}
ret = (SSL_CTX *)OPENSSL_malloc(sizeof(SSL_CTX));
if (ret == NULL)
goto err;
memset(ret, 0, sizeof(SSL_CTX));
ret->method = meth;
...
}
It can be seen that the method definition comes from TLSv1_2_client_method() is defined according to the specific protocol type.
Source insight cannot find tlsv1 directly_ 2_ client_ Method function. Tlsv1 can be found by global search_ 2_ client_ The method function is defined by IMPLEMENT_tls_meth_func macro definition, in T1_ In clnt. H
static const SSL_METHOD *tls1_get_client_method(int ver);
static const SSL_METHOD *tls1_get_client_method(int ver)
{
if (ver == TLS1_2_VERSION)
return TLSv1_2_client_method();
if (ver == TLS1_1_VERSION)
return TLSv1_1_client_method();
if (ver == TLS1_VERSION)
return TLSv1_client_method();
return NULL;
}
IMPLEMENT_tls_meth_func(TLS1_2_VERSION, TLSv1_2_client_method,
ssl_undefined_function,
ssl3_connect,
tls1_get_client_method, TLSv1_2_enc_data)
By viewing implement_ tls_ meth_ The definition of func can be seen by comparing with SSL_METHOD method definition SSL_ S->method->ssl_ in read The read function is ssl3_read function
# define IMPLEMENT_tls_meth_func(version, func_name, s_accept, s_connect, \
s_get_meth, enc_data) \
const SSL_METHOD *func_name(void) \
{ \
static const SSL_METHOD func_name##_data= { \
version, \
tls1_new, \
tls1_clear, \
tls1_free, \
s_accept, \
s_connect, \
ssl3_read, \
ssl3_peek, \
ssl3_write, \
ssl3_shutdown, \
ssl3_renegotiate, \
ssl3_renegotiate_check, \
ssl3_get_message, \
ssl3_read_bytes, \
ssl3_write_bytes, \
ssl3_dispatch_alert, \
ssl3_ctrl, \
ssl3_ctx_ctrl, \
ssl3_get_cipher_by_char, \
ssl3_put_cipher_by_char, \
ssl3_pending, \
ssl3_num_ciphers, \
ssl3_get_cipher, \
s_get_meth, \
tls1_default_timeout, \
&enc_data, \
ssl_undefined_void_function, \
ssl3_callback_ctrl, \
ssl3_ctx_callback_ctrl, \
}; \
return &func_name##_data; \
}
For ssl3_ The call flow of read function is as follows:
The core process of the above code process is in two links.
One is ssl3_read_bytes. If there is no data in the record, the data reading from the record will be triggered. The rr here is the content and offset of the latest record stored in SSL *s
int ssl3_read_bytes(SSL *s, int type, unsigned char *buf, int len, int peek)
{
int al, i, j, ret;
unsigned int n;
SSL3_RECORD *rr;
void (*cb) (const SSL *ssl, int type2, int val) = NULL;
...
start:
s->rwstate = SSL_NOTHING;
/*-
* s->s3->rrec.type - is the type of record
* s->s3->rrec.data, - data
* s->s3->rrec.off, - offset into 'data' for next read
* s->s3->rrec.length, - number of bytes.
*/
rr = &(s->s3->rrec);
/* get new packet if necessary */
if ((rr->length == 0) || (s->rstate == SSL_ST_READ_BODY)) {
ret = ssl3_get_record(s);
if (ret <= 0)
return (ret);
}
...
}
Then the data will be copied from record. If the expected reading data n is longer than the length of record, only the length in record will be read. If the expected read data n is less than the length of record, n bytes will be read, then the offset off that has been read in record records will be adjusted. Next time, it will start reading from off.
if (type == rr->type) { /* SSL3_RT_APPLICATION_DATA or
* SSL3_RT_HANDSHAKE */
/*
* make sure that we are not getting application data when we are
* doing a handshake for the first time
*/
if (SSL_in_init(s) && (type == SSL3_RT_APPLICATION_DATA) &&
(s->enc_read_ctx == NULL)) {
al = SSL_AD_UNEXPECTED_MESSAGE;
SSLerr(SSL_F_SSL3_READ_BYTES, SSL_R_APP_DATA_IN_HANDSHAKE);
goto f_err;
}
if (len <= 0)
return (len);
if ((unsigned int)len > rr->length)
n = rr->length;
else
n = (unsigned int)len;
memcpy(buf, &(rr->data[rr->off]), n);
if (!peek) {
rr->length -= n;
rr->off += n;
if (rr->length == 0) {
s->rstate = SSL_ST_READ_HEADER;
rr->off = 0;
if (s->mode & SSL_MODE_RELEASE_BUFFERS
&& s->s3->rbuf.left == 0)
ssl3_release_read_buffer(s);
}
}
return (n);
}
The second is ssl3_get_record, which reads the actual data from the socket, performs decryption and authentication, and then obtains a complete record data. It is known from the code flow that only one record record will be read at most at a time.
According to the above, SSL in openssl_ Through the analysis of the read source code, we can see that the logic of the code implementation is consistent with the phenomenon we observed. If there is still data in the record, this SSL_ The read function called by the read function will only get from the remaining data in the record, and will not re read a new record. The content of the next ssl record will not be parsed until the next read finds that the result in the record cache is 0.