Nginx's stream module provides TCP load balancing functionality. The original stream module was relatively simple, and after nginx-1.11.4 it began to process requests in a phased manner similar to the HTTP module.
Processing phase of stream module
Seven stages of the stream module are defined in ngx_stream.h.As shown below
typedef enum { NGX_STREAM_POST_ACCEPT_PHASE = 0, NGX_STREAM_PREACCESS_PHASE, NGX_STREAM_ACCESS_PHASE, NGX_STREAM_SSL_PHASE, NGX_STREAM_PREREAD_PHASE, NGX_STREAM_CONTENT_PHASE, NGX_STREAM_LOG_PHASE } ngx_stream_phases;
As with the HTTP module, each stage has a corresponding checker check method and handler callback method, and each stage has zero or more ngx_stream_phase_handler_t structures.
typedef struct ngx_stream_phase_handler_s ngx_stream_phase_handler_t; typedef ngx_int_t (*ngx_stream_phase_handler_pt)(ngx_stream_session_t *s, ngx_stream_phase_handler_t *ph); typedef ngx_int_t (*ngx_stream_handler_pt)(ngx_stream_session_t *s); typedef void (*ngx_stream_content_handler_pt)(ngx_stream_session_t *s); struct ngx_stream_phase_handler_s { ngx_stream_phase_handler_pt checker; ngx_stream_handler_pt handler; ngx_uint_t next; };
When the stream module receives the request, it calls ngx_stream_core_run_phases to execute the processing functions of each phase in turn after initializing the connection.
void ngx_stream_core_run_phases(ngx_stream_session_t *s) { ngx_int_t rc; ngx_stream_phase_handler_t *ph; ngx_stream_core_main_conf_t *cmcf; cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module); ph = cmcf->phase_engine.handlers; while (ph[s->phase_handler].checker) { rc = ph[s->phase_handler].checker(s, &ph[s->phase_handler]); if (rc == NGX_OK) { return; } } }
POST_ACCEPT, PREACCESS, ACCESS phases
The checker checking methods of POST_ACCEPT, PREACCESS, and ACCESS stages are ngx_stream_core_generic_phase, which mainly do some work of access control.Because the stream module handles TCP requests, the relationship between stages is simple, and the stage in which the module is mounted only affects the order of execution.
The following is the ngx_stream_core_generic_phase function
ngx_int_t ngx_stream_core_generic_phase(ngx_stream_session_t *s, ngx_stream_phase_handler_t *ph) { ngx_int_t rc; /* * generic phase checker, * used by all phases, except for preread and content */ ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0, "generic phase: %ui", s->phase_handler); rc = ph->handler(s); if (rc == NGX_OK) { s->phase_handler = ph->next; return NGX_AGAIN; } if (rc == NGX_DECLINED) { s->phase_handler++; return NGX_AGAIN; } if (rc == NGX_AGAIN || rc == NGX_DONE) { return NGX_OK; } if (rc == NGX_ERROR) { rc = NGX_STREAM_INTERNAL_SERVER_ERROR; } ngx_stream_finalize_session(s, rc); return NGX_OK; }
Similar to the HTTP module, it is processed based on the RC = ph-> handler (s) result.
- rc for NGX_OK indicates that this stage of work is complete and that the next stage is in progress
- rc for NGX_DECLINED indicates proceeding to the next module processing
- rc for NGX_AGAIN indicates that the current request cannot be completed at this time, returning NGX_OK
- rc for NGX_DONE indicates that the current request is over and will be called again to return NGX_OK
- rc indicates an error for NGX_ERROR, ending the request
- When rc is a different value, processing is complete, and rc is a status code
Note that there is no status code in a TCP request like an HTTP request, where the status code only represents information about connection processing, such as a source timeout status code of 502.
SSL phase
The checker check method for the SSL phase is also ngx_stream_core_generic_phase, where the ngx_stream_ssl_module is mounted and where the SSL handshake is handled when SSL is turned on.
PREREAD phase
This phase is characterized by reading downstream request packages.Call handler callback function processing after reading.The read data is saved in c->buffer.For example, ngx_stream_proxy_module in the CONTENT stage sends the contents of c->buffer to the upstream source server when it is processed.If you read downstream data before the CONTENT phase and want to send it upstream through proxy, you can add data to c->buffer.
The ngx_stream_ssl_preread_module is officially mounted here, which can be used to parse the client hello handshake package when the TCP connection uses SSL communication and get the server_name from the extensions field assigned to the variable $ssl_preread_server_name.This module can only be used when the listening address does not have SSL turned on, but the upstream and downstream are communicating via SSL.
The PREREAD phase checker method is ngx_stream_core_preread_phase.As shown below
ngx_int_t ngx_stream_core_preread_phase(ngx_stream_session_t *s, ngx_stream_phase_handler_t *ph) { size_t size; ssize_t n; ngx_int_t rc; ngx_connection_t *c; ngx_stream_core_srv_conf_t *cscf; c = s->connection; c->log->action = "prereading client data"; cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module); if (c->read->timedout) { rc = NGX_STREAM_OK; } else if (c->read->timer_set) { rc = NGX_AGAIN; } else { rc = ph->handler(s); } while (rc == NGX_AGAIN) { if (c->buffer == NULL) { c->buffer = ngx_create_temp_buf(c->pool, cscf->preread_buffer_size); if (c->buffer == NULL) { rc = NGX_ERROR; break; } } size = c->buffer->end - c->buffer->last; if (size == 0) { ngx_log_error(NGX_LOG_ERR, c->log, 0, "preread buffer full"); rc = NGX_STREAM_BAD_REQUEST; break; } if (c->read->eof) { rc = NGX_STREAM_OK; break; } if (!c->read->ready) { if (ngx_handle_read_event(c->read, 0) != NGX_OK) { rc = NGX_ERROR; break; } if (!c->read->timer_set) { ngx_add_timer(c->read, cscf->preread_timeout); } c->read->handler = ngx_stream_session_handler; return NGX_OK; } n = c->recv(c, c->buffer->last, size); if (n == NGX_ERROR) { rc = NGX_STREAM_OK; break; } if (n > 0) { c->buffer->last += n; } rc = ph->handler(s); } if (c->read->timer_set) { ngx_del_timer(c->read); } if (rc == NGX_OK) { s->phase_handler = ph->next; return NGX_AGAIN; } if (rc == NGX_DECLINED) { s->phase_handler++; return NGX_AGAIN; } if (rc == NGX_DONE) { return NGX_OK; } if (rc == NGX_ERROR) { rc = NGX_STREAM_INTERNAL_SERVER_ERROR; } ngx_stream_finalize_session(s, rc); return NGX_OK; }
CONTENT phase
The check method for the CONTENT phase is ngx_stream_core_content_phase
ngx_int_t ngx_stream_core_content_phase(ngx_stream_session_t *s, ngx_stream_phase_handler_t *ph) { ngx_connection_t *c; ngx_stream_core_srv_conf_t *cscf; c = s->connection; c->log->action = NULL; cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module); if (c->type == SOCK_STREAM && cscf->tcp_nodelay && ngx_tcp_nodelay(c) != NGX_OK) { ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR); return NGX_OK; } cscf->handler(s); return NGX_OK; }
Obviously, there can only be one processing method at this stage, such as ngx_stream_proxy_module required for TCP proxy, where the handler function of the proxy module is mounted
LOG phase
Although the LOG phase is a separate phase, it is called at the end of the connection.At the end of the CONTENT phase, the ngx_stream_finalize_session is called to end the request.This section is in ngx_stream_handler.c
void ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc) { ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0, "finalize stream session: %i", rc); s->status = rc; ngx_stream_log_session(s); ngx_stream_close_connection(s->connection); } static void ngx_stream_log_session(ngx_stream_session_t *s) { ngx_uint_t i, n; ngx_stream_handler_pt *log_handler; ngx_stream_core_main_conf_t *cmcf; cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module); log_handler = cmcf->phases[NGX_STREAM_LOG_PHASE].handlers.elts; n = cmcf->phases[NGX_STREAM_LOG_PHASE].handlers.nelts; for (i = 0; i < n; i++) { log_handler[i](s); } }