</resource-ref>
<% String jndiName = "java:comp/env/jdbc/druid-test";## Write jsp Because you need to web Use in environment, if you write directly from a class main Method testing, will always be wrong, currently there is no good way. It's easy to use here jsp Take a test.
InitialContext ic = new InitialContext();
// Get ComboPooledDataSource on JNDI
DataSource ds = (DataSource) ic.lookup(jndiName);
JDBCUtils.setDataSource(ds);
// Create sql
String sql = "select * from demo_user where deleted = false";
Connection connection = null;
PreparedStatement statement = null;
ResultSet resultSet = null;
// Query Users
try {
// Get Connected
connection = JDBCUtils.getConnection();
// Get Statement Object
statement = connection.prepareStatement(sql);
// implement
resultSet = statement.executeQuery();
// Traversing result set
while(resultSet.next()) {
String name = resultSet.getString(2);
int age = resultSet.getInt(3);
System.err.println("User name:" + name + ",Age:" + age);
}
} catch(SQLException e) {
System.err.println("Query User Exceptions");
} finally {
// Release Resources
JDBCUtils.release(connection, statement, resultSet);
}
%>
Packaging project runs on tomcat9, accessing http://localhost:8080/hikari-demo/testJNDI.jsp, the console prints the following:
User name: zzs001,Age: 18 User name: zzs002,Age: 18 User name: zzs003,Age: 25 User name: zzf001,Age: 26 User name: zzf002,Age: 17 User name: zzf003,Age: 18
Example of using JMX to manage connection pools
demand
Turn on the JMX functionality of HikariCP and view it using jconsole.
Modify hikari.properties
Add the following configuration to the example one. This sets the registerMbeans to true, and the JMX function is turned on.
#-------------JMX-------------------------------- #Allow connection pools to be suspended and restored via JMX #Default to false allowPoolSuspension=false #Whether to turn on JMX #Default false registerMbeans=true #Data source name, commonly used in JMX. #Default auto-generation poolName=zzs001
Writing test classes
To see the effect, let the main thread sleep instead of ending.
public static void main(String[] args) throws InterruptedException {
new HikariDataSourceTest().findAll();
Thread.sleep(60 * 60 * 1000);
}
View using jconsole
Run the project, open jconsole, select our project backpoint connection, and you can see our project on the MBAN tab. The configuration can be dynamically modified through PoolConfig (only some parameters allow modifications); the number of connections (active, idle, and all) to the connection pool can be obtained through Pool; the number of threads waiting to connect; connection pools can be suspended and restored; unused connections can be discarded, and so on.
Profile Detailed Writing
HikariCP configuration parameters are simpler than other connection pools, and there are several features to note: HikariCP forces on loan and idle tests, does not turn on recycle tests, and optionally only leak tests.
Database Connection Parameters
Note that several parameters are stitched behind the url to avoid scrambling and time zone errors. Additionally, if you do not want to add parameters to the time zone, you can execute the following command in the mysql command window:set global time_zone='+8:00'.
#------------------------------------------------------------------------------------------------- jdbcUrl=jdbc:mysql://localhost:3306/github_demo?useUnicode=true&characterEncoding=utf8&serverTimezone=GMT%2B8&useSSL=true username=root password=root #Driver implementation class name used by JDBC driver #The default is empty. Will be parsed according to jdbcUrl driverClassName=com.mysql.cj.jdbc.Driver
Connection pool data basic parameters
These two parameters are common and are recommended to be adjusted to specific projects.
#---------------- Connection pool size related parameters------------------------------------- #Maximum number of connection pools #Default is 10. Dynamic modification through JMX maximumPoolSize=10 #Minimum number of idle connections #The default is the same as maximumPoolSize. Dynamic modification through JMX minimumIdle=0
Connection Check Parameters
For connection failure, HikariCP forces on loan and idle tests, does not turn on recycle tests, and optionally only leak tests.
#------------------------------------------------------- #The sql used to detect whether the connection is valid requires a query statement, commonly select'x' #If the driver supports JDBC4, it is not recommended because Connection is called by default. IsValid() method, which is more efficient to detect #Empty by default connectionTestQuery=select 1 from dual #Timeout in milliseconds to detect whether a connection is valid #Minimum allowable value 250 ms #Default 5000 ms. Dynamic modification through JMX validationTimeout=5000 #Minimum time for a connection to remain idle without being expelled. Unit milliseconds. #This configuration will only take effect if the minimumIdle < maximumPoolSize again, with a minimum allowable value of 10000 ms. #The default value is 10000*60 = 10 minutes. Dynamic modification through JMX idleTimeout=600000 #The maximum time a connection object is allowed to "leak". Unit milliseconds #The minimum allowable value is 2000 ms. #Default 0 means leak detection is not enabled. Dynamic modification through JMX leakDetectionThreshold=0 #Maximum connection lifetime. Unit milliseconds #Minimum allowable value 30000 ms #Default 30 minutes. Dynamic modification through JMX maxLifetime=1800000 #Maximum wait time in milliseconds for connection acquisition #Getting more time than this configuration will throw an exception. Minimum allowable value 250 ms #Default 30000 ms. Dynamic modification through JMX connectionTimeout=300000 #Timeout in milliseconds to get a connection before starting the connection pool #When > 0, the connection is attempted. If the fetch time exceeds the specified time, the connection pool is not opened and an exception is thrown #When = 0, an attempt is made to acquire and verify the connection. The pool is not opened if the acquisition succeeds but the validation fails, but it will open if the acquisition fails #When <0, the pool is opened regardless of whether it is acquired or verified successfully. #Default 1 initializationFailTimeout=1
Transaction-related parameters
It is recommended that you leave the default.
#------------------------------------------------------------------------------------------------------------------------------------------------------ #Set autocommit when connection returns to pool #Default to true autoCommit=true #Is connection set to read-only when removed from pool #Default value false readOnly=false #Default TransactionIsolation state for connections created by connection pools #Available values are one of the following: NONE,TRANSACTION_READ_UNCOMMITTED, TRANSACTION_READ_COMMITTED, TRANSACTION_REPEATABLE_READ, TRANSACTION_SERIALIZABLE #The default value is empty, determined by the driver transactionIsolation=TRANSACTION_REPEATABLE_READ #Whether to isolate internal queries in transactions. #autoCommit will not take effect until it is false #Default false isolateInternalQueries=false
JMX parameters
It is recommended that allowPoolSuspension not be turned on, which has a significant impact on performance, and later source analysis will explain why.
#-------------JMX-------------------------------- #Allow connection pools to be suspended and restored via JMX #Default to false allowPoolSuspension=false #Whether to turn on JMX #Default false registerMbeans=true #Data source name, commonly used in JMX. #Default auto-generation poolName=zzs001
Other
Note that the dataSourceJndiName here is not the jdbc/hikariCP-test from the previous example, which is used to create a native connection object and is generally not needed.
#-------------------------------------------------------------------------------------------------------------------------------------------------- #Database Catalog #Default is driven catalog=github_demo #Data source class name provided by JDBC driver #XA data sources are not supported. If not set, DriverManager will be used by default to get the connection object #Note that if driverClassName is set, dataSourceClassName is not allowed to be set again, otherwise an error will be reported #Empty by default #dataSourceClassName= #Data source name for JNDI configuration #Empty by default #dataSourceJndiName= #Initialization statements that need to be executed after each connection is acquired and before it is put into the pool #If execution fails, the connection is dropped #Empty by default #connectionInitSql= #------------------------------------------------------------------------------------------------------------------------------------------------------------------------ #TODO #Empty by default #metricRegistry #TODO #Empty by default #healthCheckRegistry #Data source instance for Hikari wrapping #Empty by default #dataSource #Factory used to create threads #Empty by default #threadFactory= #Thread pool for executing timed tasks #Empty by default #scheduledExecutor=
Source Code Analysis
HikariCP's source code is light and simple, and it doesn't take too much effort to read, so this time it won't analyze the code logic from start to finish, it will analyze more clever design points. Before reading the HiakriCP source code, you need to know: CopyOnWriteArrayList, AtomicInteger, SynchronousQueue, Semaphore, AtomicIntegerFieldUpdater and other tools. Note: Considering length and readability, the following code has been deleted to preserve only the required parts.
Why is HikariCP fast?
Combined with source analysis and references, HikariCP is faster than connection pools such as DBCP and C3P0 for the following reasons:
-
Significantly reduce lock competition between threads through code design and optimization. This is achieved primarily through ConcurrentBag, which is expanded below.
-
More features of JDK have been introduced, especially tools for the concurrent package. DBCP and C3P0 appeared earlier and were developed based on earlier JDKs, which made it difficult to enjoy the benefits of later updates.
-
Using javassistto directly modify the class file to generate dynamic proxy simplifies many unnecessary byte codes and improves the speed of proxy methods. Compared to dynamic proxies for JDK and cglib, Proxy classes generated by directly modifying class files through javassistwill run faster (This is what you find online, but JDK and cglib have been optimized several times so far that they should run as fast as one order of magnitude in the proxy class, so I'll take a moment to test them again.) HikariCP refers to javassist code in the JavassistProxyFactory class, so check it out for yourself;
-
Pay attention to the impact of code details on performance. An example of this is the fastPathPool below, where you can find many similar detailed optimizations when you carefully consider the HikariCP code, in addition to custom collection classes such as FastList.
Next, the above points will be discussed in the process of analyzing the source code.
Architecture of HikariCP
Before analyzing the specific code, here is the overall architecture of HikariCP, which is a bit similar to DBCP2 (it is clear that the performance difference between HikariCP and DBCP2 is not due to the architecture design).
We deal with HikariCP through the following portals:
-
HikariConfigMXBean is called by JMX to dynamically modify the configuration (only some parameters allow modification, as noted in the configuration details);
-
Call HikariPoolMXBean through JMX to get the number of connections (active, idle and all), the number of threads waiting to connect, suspend and resume the connection pool, discard unused connections, etc.
-
Loading a configuration file using HikariConfig, or manually configuring HikariConfig's parameters, generally takes it as an input to construct a HikariDataSource object;
-
Use HikariDataSource to get and discard connection objects. Also, because we inherit HikariConfig, we can configure parameters through HikariDataSource, but configuration files are not supported in this way.
Why does HikariDataSource hold two references to HikariPool
As you can see in the figure, HikariDataSource holds a reference to HikariPool, and students who have seen the source code may ask why there are two HikariPools in the attribute, as follows:
public class HikariDataSource extends HikariConfig implements DataSource, Closeable{
private final HikariPool fastPathPool;
private volatile HikariPool pool;
}
In fact, the different values of the two HikariPool s here represent different configurations. Configuration method 1: fastPathPool and pool are not empty and the same when HikariDataSource is created by constructing a new HikariDataSource(HikariConfig configuration) with parameters; Configuration 2: When a new HikariDataSource() is created by parameterless construction and manually configured, the fastPathPool is empty and the pool is not empty (initialized at the first getConnectionI()), as follows;
public Connection getConnection() throws SQLException
{
if (isClosed()) {
throw new SQLException("HikariDataSource " + this + " has been closed.");
}
if (fastPathPool != null) {
return fastPathPool.getConnection();
}
// The second configuration initializes the pool on the first getConnectionI()
HikariPool result = pool;
if (result == null) {
synchronized (this) {
result = pool;
if (result == null) {
validate();
LOGGER.info("{} - Starting...", getPoolName());
try {
pool = result = new HikariPool(this);
}
catch (PoolInitializationException pie) {
if (pie.getCause() instanceof SQLException) {
throw (SQLException) pie.getCause();
}
else {
throw pie;
}
}
LOGGER.info("{} - Start completed.", getPoolName());
}
}
}
return result.getConnection();
}
For the above two configurations, in fact, one pool can be used to complete, so why are there two? Let's compare the two ways:
private final T t1;
private volatile T t2;
public void method01(){
if (t1 != null) {
// do something
}
}
public void method02(){
T result = t2;
if (result != null) {
// do something
}
}
Of the above two methods, the code executed is almost the same, but method02 will perform slightly worse than method01. Of course, the main problem is not that method02 defines more than one variable, but the volatile nature of t2. Because T2 is modified by volatile, method02 will perform slightly worse than method01 in order to achieve data consistency with unnecessary overhead. The same is true for pool and fastPathPool, so the second configuration is not recommended. As you can see from the above questions, HiakriCP s attach great importance to detail in their pursuit of performance. No wonder they are the fastest connection pool!
HikariPool - Manage connected ponds
HikariPool is a very important class that manages connections and involves a lot of code logic. A brief introduction to this class will help you with the specific analysis of the code below.
Several properties of HikariPool are described below:Attribute type and attribute name Description
In order to understand the meaning of the above fields more clearly, I have drawn a simple picture, which is not very precise. Let's have a look. In this diagram, PoolEntry encapsulates a Connection object, which is better understood as a Connection object. We can see that ConcurrentBag is the core of the entire HikariPool, and other objects operate around it, which is explained separately later. Client threads can call their borrow, requite, and remove methods, and the houseKeepingExecutorService thread can call its remove method, only addConnectionExecutor can do the add operation.
borrow and requite are read-only operations for ConcurrentBag, and addConnectionExecutor only opens one thread to execute tasks, so the add operation is single-threaded, and the only lock competition is the remove method. ConcurrentBag is explained in detail next.
ConcurrentBag - fewer lock conflicts
In HikariCP, ConcurrentBag is used to store a PoolEntry object (encapsulated with a Connection object, the IConcurrentBagEntry implementation class), which can essentially be a resource pool.
Here's a brief description of what the following fields do: Attribute Description
describe
How do these fields work in ConcurrentBag? Let's see how borrow works:
public T borrow(long timeout, final TimeUnit timeUnit) throws InterruptedException
{
// 1. Get the object from the threadList first
// Get the List<Object>object bound to the current thread, and note that the implementation of this collection is usually FastList, which HikariCP implements by itself, as you will see later
final List<Object> list = threadList.get();
// Traversal Combination
for (int i = list.size() - 1; i >= 0; i--) {
// Gets the current element and deletes it from the collection
final Object entry = list.remove(i);
// If weakThreadLocals is set, the WeakReference object is stored, otherwise the PoolEntry object is set for us to start with
@SuppressWarnings("unchecked")
final T bagEntry = weakThreadLocals ? ((WeakReference<T>) entry).get() : (T) entry;
// Change the object state obtained by CAS from unused to in use, and if failure indicates that other threads are using it, elements on the threadList can be "stolen" by other threads.
if (bagEntry != null && bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
return bagEntry;
}
}
// 2. If you haven't, you'll get the object from the sharedList
// Number of threads waiting to get a connection+1
final int waiting = waiters.incrementAndGet();
try {
// Traversing sharedList
for (T bagEntry : sharedList) {
// Change the object state obtained by CAS from unused to in use, and if the current element is in use, it cannot be modified successfully and proceeds to the next cycle
if (bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
// Notify listeners to add package elements. If waiting - addConnectionQueue. Size () >= 0 causes addConnectionExecutor to perform the PoolEntryCreator task
if (waiting > 1) {
listener.addBagItem(waiting - 1);
}
return bagEntry;
}
}
// Notify listeners to add package elements.
listener.addBagItem(waiting);
// 3. If it has not been acquired yet, it will enter the handoffQueue queue from the round training to acquire the connection objects
timeout = timeUnit.toNanos(timeout);
do {
final long start = currentTime();
// Get and delete elements from the handoffQueue queue. This is a blocked queue with no capacity. Insertion needs to block pending deletion, deletion does not need to wait, null is returned if no element is inserted, and wait if timeout is set
final T bagEntry = handoffQueue.poll(timeout, NANOSECONDS);
// There are three things going on here,
// 1. Timeout returns null
// 2. Get the element, but the state is in use, continue execution
// 3. Get element, element state unused, modify unused and return
if (bagEntry == null || bagEntry.compareAndSet(STATE_NOT_IN_USE, STATE_IN_USE)) {
return bagEntry;
}
// Calculate Remaining Timeout
timeout -= elapsedNanos(start);
} while (timeout > 10_000);
// Timeout returns null
return null;
}
finally {
// Number of threads waiting to get a connection -1
waiters.decrementAndGet();
}
}
In the above method, the only possible thread switch is handoffQueue.poll(timeout, NANOSECONDS), except that we don't see any synchronized and lock. The main reasons for this are the following:
1. The introduction of element States and the use of CAS methods to modify states. In ConcurrentBag, elements that represent the state of elements in use, unused, deleted, and retained are used instead of using different collections to maintain different states. The introduction of the concept of element state is critical and provides the basis for the following points. The CAS method is called multiple times in ConcurrentBag's method to determine and modify the state of an element, and no locks are required in this process.
2.Use of threadList. Elements returned by the current thread are bound to ThreadLocal. When the thread retrieves the element again, it can be obtained directly without stolen it, and it does not need to go through the sharedList to retrieve it.
3. Use CopyOnWriteArrayList to store elements. In CopyOnWriteArrayList, read and write use different arrays, avoiding lock competition between the two, and adding ReentrantLock locks for multiple threads to write.
4. Read and write control of sharedList. Both borrow and requite are unlocked for sharedLists, with the disadvantage of sacrificing consistency. User threads cannot add elements, only addConnectionExecutor can, and addConnectionExecutor only opens one thread to perform tasks, so there is no lock competition for add operations. As far as remove is the only way to cause lock competition, I think you can also refer to addConnectionExecutor to mark the state of PoolEntry as deleted before joining the task queue.
In fact, we will find that ConcurrentBag uses more JDK features than design improvements to reduce lock conflicts.
How to load a configuration
In HikariCP, HikariConfig is used to load configurations. The specific code is not complex, but it is easier to load than other projects. We're directly from PropertyElf. The setTargetFromProperties (Object, Properties) method starts with the following:
// This is done by setting the parameters of the properties to HikariConfig
public static void setTargetFromProperties(final Object target, final Properties properties)
{
if (target == null || properties == null) {
return;
}
// Here you will use reflection to get
List<Method> methods = Arrays.asList(target.getClass().getMethods());
// ergodic
properties.forEach((key, value) -> {
// If it is dataSource. * Parameter, directly added to the dataSourceProperties property
if (target instanceof HikariConfig && key.toString().startsWith("dataSource.")) {
((HikariConfig) target).addDataSourceProperty(key.toString().substring("dataSource.".length()), value);
}
else {
### Last
**If you think this is helpful to you, you might as well give me some compliment and pay attention to it.**
**[Free data collection method: click here](https://gitee.com/vip204888/java-p7)**
properties.forEach((key, value) -> {
// If it is dataSource. * Parameter, directly added to the dataSourceProperties property
if (target instanceof HikariConfig && key.toString().startsWith("dataSource.")) {
((HikariConfig) target).addDataSourceProperty(key.toString().substring("dataSource.".length()), value);
}
else {
### Last
**If you think this is helpful to you, you might as well give me some compliment and pay attention to it.**
**[Free data collection method: click here](https://gitee.com/vip204888/java-p7)**
[Outer Chain Picture Transfer in Progress...(img-SBSY6fIe-1628484640460)]
