Concurrency / high concurrency
tomcat default 150 concurrent connection (socket)
RT: corresponding time
QPS: throughput
Hardware
cpu, memory, disk, network
Software
Maximize the use of hardware resources
Number of threads, JVM memory size, network communication mechanism (BIO, NIO, AIO), disk IO
How to increase the number of concurrent threads on the server
What is a thread
Road Lane
Thread is the smallest scheduling unit for cpu execution
The number of parallel threads is determined by the cpu (number of cpu cores or number of cores * 2)
Concurrency and parallelism
Single core cpu also supports multithreading -- > cpu time slice switching
Concurrency: simultaneous connection requests
Parallelism: simultaneous processing of executed requests
Characteristics of multithreading
synchronization
You need to block and wait for other threads to complete processing
asynchronous
There is no need to block the current processing
parallel
Multithreaded simultaneous processing
Threads in Java
- Runnable excuse
- Thread class
- Callable/Future with return value
public class ThreadDemo extends Thread {
@Override
public void run(){
//Instructions executed by threads
}
public static void main(String[] args){
ThreadDemo t = new ThreadDemo();
t.start();//Start thread
}
}
public class CallableDemo implements Callable<String> {
@Override
public String call(){
//call() is equivalent to run(), the instruction executed by the thread
return null;
}
public static void main(String[] args){
ExecutorService exe = Executors.newFixedThreadPool(1);
CallableDemo c = new CallableDemo();
Future<String> f = exe.submit(c);
f.get();//The get() method is blocked
}
}
How to use Java threads
- Network request distribution
- File import
- SMS sending
Thread Foundation
Thread life cycle
Start -- > end of thread
Thread state
block
- TIMED_WAITING
- WAITING
- BLOCKED
public class Demo {
public static void main(String[] args){
//Blocking status TIMED_WAITING
new Thread(() -> {
while(true){
try {
TimeUnit.SECONDS.sleep(100);
} catch (InterruptedException e){
e.printStackTrace();
}
}
}).start();
//Blocking state WAITING
new Thread(() -> {
while(true) {
synchronized (Demo.class) {
try {
Demo.class.wait();
} catch (InterruptedException e){
e.printStackTrace();
}
}
}
}).start();
//Preemptive lock TIMED_WAITING
new Thread(new BlockedDemo()).start();
//Lock not preempted BLOCKED
new Thread(new BlockedDemo()).start();
}
static class BlockedDemo extends Thread {
@Override
public void run() {
while(true) {
synchronized (Demo.class) {
try {
TimeUnit.SECONDS.sleep(100);
} catch (InterruptedException e){
e.printStackTrace();
}
}
}
}
}
}
jps view the Pid of the current thread
jstack pid view the stack information of pid
Thread states in Java: 6
-
new (thread initialization)
-
runnable (CPU is ready before OS scheduling)
-
waiting
sleep(0);
wait();
join();
LockSupport.parkUnitl();
notify
notifyAll
unpark
-
timed_waiting
sleep(long);
wait(long);
join(long);
LockSupport.parkUnitl(xx);
notify
notifyAll
unpark
-
Blocked (lock blocked)
-
Termination (end of thread run)
Thread states at the operating system level: 5
No new status
Thread startup
new Thread().start();//Start thread new Thread().run();//Call instance method
-
Thread. The start method calls the local method start0 to start the thread
-
The JVM calls creation threads at different system levels through the local method start0
-
The OS calls the JVM's thread Run method (OS calls CPU to execute run method instruction through CPU scheduling algorithm)
-
JVM calls thread. In Java run
-
After execution, the JVM destroys the thread
Thread termination
When does a thread terminate
End of run method execution
Thread. The stop method is forced to terminate and is not recommended
Send termination signal to notify termination, interrupt
Thread.currentThread().isInterrupted() indicates the interrupt flag, which is false by default
Thread.interrupt(); Set interrupt = true
Thread.interrupted(); Reset, resume interrupt = false
interrupt(); --> Communication between threads is realized through a shared variable
Set the value of a shared variable to true. volatile jint _interrupted;
Wake up blocked threads
Thread safety
problem
public static int count = 0;
public static incr() {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
count ++;
}
public static void main(String[] args) {
for(int i = 0; i < 1000; i ++) {
new Thread(APP::incr).start();
}
try {
Thread.sleep(1000);
} catch(InterruptedException e) {
e.printStackTrace();
}
}
The result is a random number less than or equal to 1000
Thread safety issues
- Atomicity
- Order
- visibility
Data security problems caused by cpu time slice switching
The problem is count + +, which is not an atomic operation. It is compiled into three instructions:
- Load into register
- accumulation
- Write memory
Lock (Synchronized)
public class SynchronizedDemo {
synchronized void demo1(){}//method
synchronized static void demo2(){}//method
Object o = new Object();
void demo3() {
synchronized(o) {
//Code block
}
}
}
//Lock range
Instance lock / object lock
The scope of the lock is controlled within the object instance
SynchronizedDemo demo = new SynchronizedDemo();
new Thread(() -> {
demo.demo1();
}).start();
new Thread(() -> {
demo.demo1();
}).start();
void demo() {
synchronized(this) {
//Code block
}
}
Class lock
Static method, class object, class lock
synchronized static void demo(){}
void demo() {
synchronized(SynchronizedDemo.class) {
//Code block
}
}
The essence of mutex
shared resource
Achieve exclusivity by preempting shared resources
Storage of locks (object headers)
Object layout in Heap - > OOP hpp ->oopDesc
Object tag markoop_ mark
Class meta information markoop_ mark
Instance data_ metadata
[align fill]
Object tag (markOop.hpp)
hashcode
Generational age
Synchronization lock mark
Deflection lock mark
Lock holder thread ID
Monitor() - > implementation logic of objectmonitor competing for lock
[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-zkaatadw-1639183807603) (/ users / muprince / library / Application Support / typera user images / image-20211124001751204. PNG)]
jol is a tool for viewing the layout of class object header information
-20: - usecompressedoops turns off the JVM object header compression pointer
<dependency> <groupId>org.openjdk.jol</groupId> <artifactId>jol-core</artifactId> <version>0.10</version> </dependency>
System.out.println(ClassLayout.parseInstance(instance).toPrintable());
Bias lock. CAS replaces the thread ID of the object header once. It is closed by default
Lightweight lock, multiple CAS replacement of pointer to stack frame in mark word, spin lock, default 10 times before 1.6, adaptive spin lock after 1.6
Heavyweight lock, thread blocking
Optimistic lock
CAS compares whether the expected data is consistent with the original data. If it is consistent, it will be modified. If it is inconsistent, the modification will fail
object monitor
monitor, implementation of heavyweight lock
Thread communication (wait/notify)
Producer consumer
//consumer
public class Consumer implements Runnable {
private Queue<String> msg;
private int maxSize;
public Consumer(Queue<String> msg, int maxSize) {
this.msg = msg;
this.maxSize = maxSize;
}
@Override
public void run() {
while(true) {
synchronized(msg) {
while(msg.isEmpty()) {//Queue is empty, unable to consume, blocked
try {
msg.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("Consumer message:" + msg.ramove());
msg.notify();//Awaken producer production
}
}
}
}
//producer
public class Producer implements Runnable {
private Queue<String> msg;
private int maxSize;
public Producer(Queue<String> msg, int maxSize) {
this.msg = msg;
this.maxSize = maxSize;
}
@Override
public void run() {
int i = 0;
while(true) {
i ++;
synchronized(msg) {
while(msg.size() == maxSize) {//Queue full, no production, blocking
try {
msg.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("Producer production messages: messages" + i);
msg.add("news" + i);
msg.notify();//Awaken consumer consumption
}
}
}
}
public class App {
public static void main() {
Queue<String> msg = new LinkedList<>();
int maxSize = 10;
new Thread(new Producer(msg, maxSize)).start();
new Thread(new Consumer(msg, maxSize)).start();
}
}
Thread deadlock
What is deadlock
A group of threads competing to share resources wait for each other, resulting in permanent blocking
The livelock thread is not blocked and keeps looping
Conditions for deadlock generation (both satisfied)
- Mutually exclusive. Shared resources X and Y can only be occupied by one thread
- Occupy and wait. Thread 1 has obtained shared resource X. when waiting for shared resource Y, shared resource x is not released
- Cannot preempt. Other threads cannot forcibly preempt the resources occupied by thread 1
- Loop waiting, thread 1 waits for the resources occupied by thread 2, and thread 2 waits for the resources occupied by thread 1
By expanding the granularity of the lock, it destroys possession and waiting
Via reentrantlock Trylock() is broken and cannot be preempted
Cycle waiting is broken by sequential locking
ThreadLocal
Thread isolation mechanism, unique to threads
Each thread holds a ThreadLocalMap
0x61c88647 magic number, golden section, Fibonacci hash
Linear exploration - > a strategy for resolving hash conflicts
- Write to find the nearest free cell where the conflict occurred
- Find, look back from where the conflict occurred
private static final int HASH_INCREMENT = 0x61c88647;
public static void main() {
magicHash(16);
}
public static void magicHash(int size) {
int hashCode = 0;
for (int i = 0; i < size; i ++) {
hashCode = i * HASH_INCREMENT + HASH_INCREMENT;
System.out.print((hashCode & (size - 1)) + " ");
}
}
Thread safety – visibility (volatile)
problem
public class VolatileDemo {
public static boolean stop = false;
//public static volatile boolean stop = false;
//public static volatile int i;
public static void main() {
new Thread(() -> {
int i = 0;
while(!stop) {
i ++;
//System.out.println("rs:" + i); You can end the cycle
//try {
// Thread.sleep(0); You can end the cycle
//} catch (InterruptedException e) {
// e.printStackTrace();
//}
}
System.out.println("rs:" + i);
}).start();
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
stop = true;
}
}
Unable to stop thread, unable to output rs + i
volatile keyword modification can solve the visibility problem
-Djava.compiler=NONE turn off the immediate compiler
The jdk of server version has been deeply optimized (JIT) --just in time
while(!stop){
i ++;
}
//After optimization
if(!stop) {
while(true) {
i ++;
}
}
print can cause the loop to end
Activity failure
print contains the synchronized modifier
The release of the lock forces the write operation of the working memory to be synchronized to the main memory
print is an IO operation
IO operations block threads
Thread.sleep() can end the cycle
Officially, the compiler is free to load this one or more times done
sleep will cause thread switching, invalidate the cache and reload
volatile keyword
hsdis tool for printing assembly instructions
lock assembly instruction – ensure visibility
The lock instruction loads the cache from the cpu core
What is visibility
Thread A is not visible to thread B after modifying the variable
Hardware level
cpu, memory, IO device
Speed CPU > memory > IO
- cpu increase cache
- Operating system, process, thread, cpu time slice switching
- Compiler optimization, more rational use of cpu cache
cpu cache
- L1d data cache, cpu core exclusive
- L1i instruction cache, cpu core exclusive
- L2 cpu core exclusive
- L3 sharing
The existence of cache leads to cache consistency problems
Bus lock
Buffer lock
Does the cpu architecture support
Does the current data exist in the cache row
Cache consistency protocol
MSI,MESI,MOSI...
MESI represents four cache states
- M modify modify
- E exclusive
- S shared
- I invalid
Store Buffer
store buffer is introduced to solve the blocking of cpu notifying cache invalidation
The introduction of store buffer causes instruction reordering
Memory barrier
Instructions are not allowed to be reordered and written to main memory
volatile keywordprevents instruction reordering through a memory barrier
- Read barrier #Load instruction
- Write barrier #Store instruction
- Full screen barrier #Fence command
Prevent instruction reordering
Disable cpu caching
#Lock instruction - > equivalent to memory barrier
Different cpu architectures, X86 is a strong consistency architecture
Software level
Java Memory Model (JMM)
The memory model defines the operation specification of multi-threaded read and write in shared memory
Provides solutions to visibility and ordering problems
JMM encapsulates memory barrier instruction types based on hardware level
- LoadLoad Barriers sequentially loads Load1;LoadLoad;Load2
- LoadStore Barriers read and then write Load1;LoadStore;Store2
- StoreStore Barriers writes store1 in sequence; StoreStore; Store2
- StoreLoad Barriers write first and then read Store1;StoreLoad;Load2
javap -v Xx.class view byte instruction
ACC_VOLATILE
bytecodeInterpreter.cpp
orderAccess.hpp
Happens before model
Program sequence rules (as if serial semantics)
-
The execution result of the program cannot be changed (in the single thread environment, the execution result remains unchanged)
-
Dependency problem. If there is a dependency between two instructions, reordering is not allowed
void test() { int a = 1; int b = 1; int c = a * b; }a happens-before b ; b happens-before c
Transitive rule
a happens-before b ; b happens-before c --> a happens-before c
volatile variable rule
Writes to volatile variables must happen before subsequent reads to volatile variables
Prevent instruction reordering through memory barriers
public class VolatileExample{
int a = 0;
volatile boolean flag = false;
//Thread1
public void writer() {
a = 1; // 1
flag = true; //modify two
}
//Thread2
public void reader() {
if (flag) { //flag = true 3
int i = a; //i = 1 4
}
}
}
1 happens before 2 is established? yes
3 does happens before 4 hold? yes
2 happens before 3 -- > volatile rule
1 happens before 4 I = 1 established
[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-i1az0a4d-1639183807604) (/ users / muprince / library / Application Support / typera user images / image-20211124052551123. PNG)]
Monitor lock rule
int x = 10;
synchronized(this) {
//The value of x read by subsequent threads must be 12
if (x < 12) {
x = 12;
}
}
The release of the lock must happen before the subsequent operation of locking the thread
start rule
public class StartDemo {
int x = 0;
Thread t = new Thread(() -> {
//The value of x read must be 20
if (x == 20){} //It must be established
});
x = 20；
t.start();
}
The operation before thread start must be a happens before thread run operation
Join rule
public class JionDemo {
public static void main() {
int x = 0;
Thread t = new Thread(() -> {
x = 20;
});
t.start();
t.jion();//Ensure visibility of results
//The value of x read after jion must be 20
}
}
Changes to shared variables in the t thread are visible to all operations after jion
wait/notify, Thread instance object lock
The fatally keyword provides rules for memory barriers