Partial function and DBUtils for getting started with Flask

Posted by michalurban on Thu, 31 Oct 2019 15:22:09 +0100

Last article > Message flash, request extension, middleware, blueprint and context management for the introduction to Flask (1)

Supplement

1. Partial function

#!/usr/bin/env python
# coding:utf-8
import functools
def fun(a, b):
    print(a, b)

new_fun = functools.partial(fun, "aaa")
new_fun1 = functools.partial(fun, b="b1b1")
new_fun("bbb")
new_fun1(a='a1a1')

Result:

aaa bbb
a1a1 b1b1

2. Object oriented

When all the functions in object-oriented are implemented, the corresponding methods will be executed when the object does any operation.

#!/usr/bin/env python
# coding:utf-8
class Foo(object):
    def __init__(self, num):
        self.num = num

    def __add__(self, other):
        data = self.num + other.num
        # print(self.num, other.num)
        return Foo(data)

obj1 = Foo(1)
obj2 = Foo(2)
# print(obj1.num, obj1)
# print(obj2.num, obj2)
v = obj1 + obj2
print(v.num, v)

Result:

3 <__main__.Foo object at 0x7fd772bb7278>

3. Value of splicing list (chain)

(1) splicing list
#!/usr/bin/env python
#coding:utf-8
from itertools import chain

li1 = [1, 2, 3]
li2 = [4, 5, 6]
new_li = chain(li2, li1)
for i in new_li:
    print(i)

Result:

4
5
6
1
2
3
(2) list and function
#!/usr/bin/env python
#coding:utf-8

from itertools import chain
def f1(x):
    return x + 1

func1_list = [f1, lambda x:x-1]
def f2(x):
    return x + 10

new_func_list = chain([f2],func1_list)
for fun in new_func_list:
    print (fun)

II. Database connection pool

1. Connection difference

  • Django
    • ORM(pymsql/MySQLdb)
  • Flask
    • Native SQL
      • pymsql (Python 2 / 3, here we use this method)
      • MySQLdb(Python2)
    • SQLAchemy(ORM) (pymsql/MySQLdb)

2. DBUtils module operation

(1) install DBUtils

Download address: https://pypi.org/project/DBUtils/#files

[root@Python ~]# wget https://files.pythonhosted.org/packages/c2/88/4f06764f776a691413ac1508f0c98097faddc076791da2620be6d146b222/DBUtils-1.3.tar.gz
[root@Python ~]# tar xf DBUtils-1.3.tar.gz
[root@Python ~]# cd DBUtils-1.3/
[root@Python DBUtils-1.3]# python setup.py install
(2) install pymysql

Download address: https://pypi.org/project/PyMySQL/#files

[root@Python ~]# wget https://files.pythonhosted.org/packages/da/15/23ba6592920e21cb40eb0fe0ea002d2b6177beb1ca8a4c1add5a8f32754d/PyMySQL-0.9.3.tar.gz
[root@Python ~]# cd PyMySQL-0.9.3/
[root@Python PyMySQL-0.9.3]# python setup.py install
(3) two connection modes of DBUtils

< 1 > mode 1

Create a connection for each thread. Even if the thread calls the close method, it will not be closed, but it will be put back into the connection pool for the thread to use again. The connection is closed when the thread terminates.

import pymysql
from DBUtils.PersistentDB import PersistentDB
POOL = PersistentDB(
    creator=pymysql,    ###Use the module to connect to the database
    maxusage=None,      ###The maximum number of times a link can be reused, None means unlimited
    setsession=[],      ###List of commands executed before starting a session. For example: ["xxx","xxxx"]
    ping=0,             ###Ping the MySQL server to check whether the service is available. Generally, we use 4 or 7.4. For example: 0=None: never,1=default: whenever it is requested, 2: when a cursor is created, 4: when a query is executed, 7: always
    closeable=False,    ###If False, conn.close() is actually ignored for the next use, and the connection will be closed when the thread is closed. If True, conn.close() closes the connection. When pool.connection is called again, an error will be reported.
    threadlocal=None,   ###This thread's exclusive object is used to save the linked object. If the linked object is reset
    host='10.10.10.111',
    port=3306,
    user='root',
    password='1',
    database='pooldb',
    charset='utf8',
)

def fun():
    conn = POOL.connection(shareable=False)
    cursor = conn.cursor()
    cursor.excute('select * from tb')
    res = cursor.fetchall()
    cursor.close()
    conn.close()

fun()

< 2 > mode 2

Create a batch of connections to the connection pool for all threads to share.

#!/usr/bin/env python
#coding:utf-8
import pymysql
from DBUtils.PooledDB import PooledDB
POOL = PooledDB(
    creator=pymysql,    ###Use the module to connect to the database
    maxconnections=6,   ###The maximum number of connections in the connection pool, 0 and None means unlimited
    mincached=2,        ###When initializing, the connection pool creates at least idle links, and 0 and None indicate not to create
    maxcached=5,        ###The most idle links in the connection pool, 0 and None represent unlimited
    maxshared=3,        ###The maximum number of shared links in the connection pool. 0 and None represent all shared links. Since both pymysql and MySQL DB modules threadsafety are 1, the settings have no effect. As a result, all links will be shared.
    blocking=True,      ###If there is no connection available in the connection pool, whether to block waiting. True: wait; False: do not wait and report an error
    maxusage=None,      ###The maximum number of times a link can be reused, None means unlimited
    setsession=[],      ###List of commands executed before starting a session. For example: ["xxx","xxxx"]
    ping=0,             ###Ping the MySQL server to check whether the service is available. Generally, we use 4 or 7.4. For example: 0=None: never,1=default: whenever it is requested, 2: when a cursor is created, 4: when a query is executed, 7: always
    closeable=False,    ###If False, conn.close() is actually ignored for the next use, and the connection will be closed when the thread is closed. If True, conn.close() closes the connection. When pool.connection is called again, an error will be reported.
    threadlocal=None,   ###This thread's exclusive object is used to save the linked object. If the linked object is reset
    host='10.10.10.111',
    port=3306,
    user='root',
    password='1',
    database='pooldb',
    charset='utf8',
)

def fun():
    conn = POOL.connection()
    cursor = conn.cursor()
    cursor.excute('select * from tb')
    res = cursor.fetchall()
    cursor.close()
    conn.close()     ###Put back to connection pool

fun()
(4) use

The following two are all under the same level directory

db_helper.py:

#!/usr/bin/env python
#coding:utf-8
import pymysql
from DBUtils.PooledDB import PooledDB
POOL = PooledDB(
    creator=pymysql,    ###Use the module to connect to the database
    maxconnections=6,   ###The maximum number of connections in the connection pool, 0 and None means unlimited
    mincached=2,        ###When initializing, the connection pool creates at least idle links, and 0 and None indicate not to create
    maxcached=5,        ###The most idle links in the connection pool, 0 and None represent unlimited
    maxshared=3,        ###The maximum number of shared links in the connection pool. 0 and None represent all shared links. Since both pymysql and MySQL DB modules threadsafety are 1, the settings have no effect. As a result, all links will be shared.
    blocking=True,      ###If there is no connection available in the connection pool, whether to block waiting. True: wait; False: do not wait and report an error
    maxusage=None,      ###The maximum number of times a link can be reused, None means unlimited
    setsession=[],      ###List of commands executed before starting a session. For example: ["xxx","xxxx"]
    ping=0,             ###Ping the MySQL server to check whether the service is available. Generally, we use 4 or 7.4. For example: 0=None: never,1=default: whenever it is requested, 2: when a cursor is created, 4: when a query is executed, 7: always
    closeable=False,    ###If False, conn.close() is actually ignored for the next use, and the connection will be closed when the thread is closed. If True, conn.close() closes the connection. When pool.connection is called again, an error will be reported.
    threadlocal=None,   ###This thread's exclusive object is used to save the linked object. If the linked object is reset
    host='10.10.10.111',
    port=3306,
    user='root',
    password='1',
    database='pooldb',
    charset='utf8',
)

class SQLHelper(object):
    @staticmethod
    def fetch_one(sql, args):
        conn = POOL.connection()
        cursor = conn.cursor()
        cursor.excute(sql, args)
        res = cursor.fetchone()
        cursor.close()
        conn.close()
        return res

    @staticmethod
    def fetch_all(sql, args):
        conn = POOL.connection()
        cursor = conn.cursor()
        cursor.excute(sql, args)
        res = cursor.fetchall()
        cursor.close()
        conn.close()
        return res
"""
//Another way of writing:
class SQLHelper(object):
    def __init__(self):
        self.conn = POOL.connection()
        self.cursor = self.conn.cursor()

    def close(self):
        self.cursor.close()
        self.conn.close()

    def fetch_one(self, sql, args):
        self.cursor.excute(sql, args)
        res = self.cursor.fetchone()
        self.close()
        return res

    def fetch_all(self, sql, args):
        self.cursor.excute(sql, args)
        res = self.cursor.fetchall()
        self.close()
        return res

//Must be instantiated when calling:
obj = SQLHelper()
obj.fetch_one()
"""
#!/usr/bin/env python
#coding:utf-8
from flask import Flask
app = Flask(__name__)
from db_helper import SQLHelper
@app.route('/')
def hello():
    SQLHelper.fetch_all('implement SQL')
    return "Hello World"

if __name__ == '__main__':
    app.run(host='10.10.10.111',)

Topics: Python SQL Database MySQL