The purpose of this study on MiniWeb framework is not to complete the compilation of the framework, but to learn the ideas in the compilation of the framework, mainly to understand the definition and automatic maintenance of routing table, the function of view function, the development idea of separation of front and back ends, and the realization and call of application function in WSGI protocol.
In the past, we mostly completed static web servers, mainly dealing with some data that has been "written dead". Today, let's learn about dynamic data processing.
When it comes to dynamic data, one thing we need to understand is the web framework.
The so-called web framework is simply a py program used to process data or templates.
Then, I will give you a brief introduction of the whole process of a browser accessing dynamic data.
WSGI (Web Server Gateway Interface) is a simple interface defined for python to communicate between server and framework. Its interface principle is to implement application function.
The application function format is as follows:
def application(environ, func): start_response('200 OK', [('Content-Type', 'text/html;charset=utf-8')]) return 'Hello MiniWeb'
The first parameter of the function, the environ type, is a dictionary that receives the request resource path encapsulated in the dictionary.
The function's second argument, func, is a function that receives the function reference passed on the call.
The application function is implemented in the framework, which is the py file for data processing; it is called on the server side.
The py file that implements the application function and completes the data processing can be called the WSGI interface.
First of all, let's take a look at a picture
This picture perfectly illustrates the application from browser to server to web framework.
Next, I will give you a brief introduction:
First of all, you need to make it clear that in this development, the files with. html suffix are dynamic resources, and the rest are static resources
1. The browser sends HTTP request message to the server,
2. The server judges according to the request path of HTTP request message,
If the static resource is requested, the server reads the data of the static resource directly and splices it into an HTTP response message and returns it to the browser;
If a dynamic resource is requested, the server sends the dynamic resource request to the web framework.
3. After receiving the request forwarded by the server, the web framework first judges the request,
If the template is requested, the web framework will search the template in the template. If the template is found, the template data will be returned to the web server, and then the server will return to the browser,
If the request is data, the web framework will go to the database for operation, return the data or return value of query and other operations to the web server, and then the server will return to the browser.
So far, the whole process is finished.
1. Implementation of web server code:
# 1.Import socket Modular import socket import threading import FreamWork # Create server class class HttpServerSocket(object): # Set properties for objects of the server class def __init__(self): # 2.Establish Socket object self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # 3.Set up port multiplexing self.server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True) # 4.Bind port self.server_socket.bind(('', 8000)) # 5.Set listening self.server_socket.listen(128) def start(self): while True: # 6.Waiting for client connection client_socket, ip_port = self.server_socket.accept() # gevent.spawn(self.task, client_socket, ip_port) print("Online.", ip_port) threading.Thread(target=self.task, args=(client_socket, ip_port), daemon=True).start() def task(self, client_socket, ip_port): # 7.receive data recv_data = client_socket.recv(1024).decode('utf-8') print(recv_data) if not recv_data: print("client:%s Offline, port number:%s" % ip_port) return # 8.send data # Determine whether the request resource contains parameters # Request line format: GET /index.html HTTP/1.1 recv_path = recv_data.split()[1] # print("First split",recv_path) # If there is a parameter, use "no"? division if '?' in recv_path: real_recv_path = recv_path.split('?')[0] # print("?division",real_recv_path) else: # Leave the request path unchanged if there are no parameters real_recv_path = recv_path # print("None? division",real_recv_path) # The resource path is not specified in the setting, and it is returned by default index.html if real_recv_path == '/': real_recv_path = '/index.html' # Determine whether the requested resource is static or dynamic if real_recv_path.endswith('.html'): env = {'PATH_INFO': real_recv_path} # Call the application function response_body = FreamWork.application(env, self.start_response) response_line = 'HTTP/1.1 %s\r\n' % self.status response_header = 'Server: PWS/1.0\r\n' # self.response_header The received tuples in the list need to be unpacked response_header += '%s :%s\r\n' % self.response_header[0] send_data = (response_line + response_header + '\r\n' + response_body).encode('utf8') client_socket.send(send_data) client_socket.close() else: # Determine whether the requested resource path exists try: with open(f"static{real_recv_path}", "rb") as file: response_body = file.read() except Exception as e: # 404 if not present response_line = 'HTTP/1.1 404 NOT FOUND\r\n' response_header = 'Server: PWS/1.0\r\n' response_body = 'sorry nor found page!\r\n'.capitalize() send_data = (response_line + response_header + '\r\n' + response_body).encode('utf-8') client_socket.send(send_data) else: # Replace the requested page information if it exists response_line = 'HTTP/1.1 200 OK\r\n' response_header = 'Server: PWS/1.0\r\n' send_data = (response_line + response_header + '\r\n').encode('utf-8') + response_body client_socket.send(send_data) finally: # Disconnect from client client_socket.close() def start_response(self, status, response_header): self.status = status self.response_header = response_header def __del__(self): # Stop the server service when the server program ends self.server_socket.close() def main(): http_socket = HttpServerSocket() http_socket.start() if __name__ == '__main__': main()
Core code (code in red and bold in code snippet) interpretation:
1. Judge whether the request is an html page by the suffix of the decomposed request resource path. If so, the request is a dynamic resource;
2. Encapsulate the dynamic resource path into a dictionary, and pass the reference of the dictionary and function to the application function,
3. The web framework (application function) searches the template for the requested template according to the passed resource path, and if so, reads the template data and returns it;
4. Receive the data returned by the web framework (application function), splice the HTTP response message, send the HTTP response message to the browser through the browser socket, and finally close the connection with the browser.
2.MiniWeb framework code implementation:
import db import json # Define a routing table router_table = {} # Writing interface functions # environ Parameter to receive the request resource path # start_response Parameter to receive the passed function def application(environ, start_response): start_response('200 OK', [('Content-Type', 'text/html;charset=utf-8')]) func = error if environ['PATH_INFO'] in router_table: func = router_table[environ['PATH_INFO']] body = func() return body # Define a decorator to realize automatic maintenance of routing table def router(url): def decorator(func): def inner(*args, **kwargs): body = func(*args, **kwargs) return body router_table[url] = func return inner return decorator # Front and rear end non separation technology @router('/index.html') def index(): db_data = db.db_select() with open('templates/index.html', 'r') as file: body = file.read() row_str = """ <tr> <td>%s</td> <td>%s</td> <td>%s</td> <td>%s</td> <td>%s</td> <td>%s</td> <td>%s</td> <td>%s</td> <td> <input type="button" value="add to" id="toAdd" name="toAdd" systemidvaule="%s"> </td> </tr> """ body_data = '' for data in db_data: body_data += row_str % (data[0], data[1], data[2], data[3], data[4], data[5], data[6], data[7], data[1]) body = body.replace('{%content%}', body_data) return body # Front and rear end separation technology @router('/center.html') def center(): with open('templates/center.html', 'r', encoding='utf-8') as file: body = file.read() return body # Writing interface functions def center_data(): result = db.db_select_interface() # to write JSON data # JSON Type data format:[{"key":"value"},{"key":"value"},{"key":"value"}] data = [] for i in result: dd = {} dd['code'] = i[0] dd['short'] = i[1] dd['chg'] = i[2] dd['turnover'] = i[3] dd['price'] = i[4] dd['highs'] = i[5] dd['info'] = i[6] data.append(dd) # Replace data with JSON Type data # data:String to convert # ensure_ascii Set whether to use ASCII Code parsing, False No, True be for the purpose of josn_data = json.dumps(data,ensure_ascii=False) return josn_data @router('/update.html') def update(): with open('templates/update.html', 'r', encoding='utf-8') as file: body = file.read() return body def error(): return '<h1>Your page is lost...</h1>'
Data format of JSON:
[{"key":"value"},{"key":"value"},{"key":"value"}]
During JSON data conversion, it should be noted that json.dumps() will use ASCII code input decoding conversion by default. If you want to transcode without ASCII code, you need to set the protect ASCII parameter, and set it to False to solve the problem.
Here is a brief introduction to the concept of routing table:
The routing table is a dictionary composed of the Key Value pairs formed by taking the request address as the Key and the reference of the function in the decorator as the Value.
The format is:
{ '/index.html':inner, '/error.html':inner }
For automatic maintenance of routing tables, we can write decorators with parameters to achieve:
router_table = {} def router(url): def decorator(func): def inner(*args, **kwargs): func(*args, **kwargs) router_table[url] = inner return inner return decorator
Add data to the routing table through the @ router('/index.html') syntax sugar.
Main process:
1. Execute router('/index.html') first, and then return the reference of decorator function;
2. Pass the view function to func through @ decorator reference, and save the key value pair composed of inner reference and url to the routing table;
Here I will extend a concept to you. The View function is the V in MVT development mode, that is, View. Its main function is to process data.
3. Implementation of database operation code:
import pymysql # Define a database connection function def db_connect(): db = pymysql.connect(host='localhost', port=3306, user='root', password='mysql', database='stock_db', charset='utf8') # Return database connection object return db # Define a index Functions of page query data def db_select(): db = db_connect() cur = db.cursor() sql = ''' select * from info; ''' cur.execute(sql) result = cur.fetchall() cur.close() db.close() return result # Define a function for interface to query data def db_select_interface(): db = db_connect() cur = db.cursor() sql = '''select i.code, i.short, i.chg, i.turnover,i.price, i.highs, f.note_info from info i inner join focus f on i.id = f.info_id ''' cur.execute(sql) result = cur.fetchall() cur.close() db.close() return result
Finally, summarize the general process of MiniWeb development:
1. Add a request resource judgment based on the original TCP server, that is, whether it is a dynamic resource;
2. If it is a dynamic resource, send the request path of the dynamic resource to the framework,
3. The framework is judging. If the dynamic resource is a template, read out the template data and return it to the server. Finally, the server splices the response message to the client,
4. If the dynamic resource requests data, the framework will return to the database to query the data, and splice it into JSON data format strings, then convert the JSON data format strings into JSON data, finally return it to the server, and finally splice the response message from the server and send it to the client.