JDK1. 8 source code analysis of concurrenthashmap
jdk1.8 container initialization
Source code analysis
- There are five construction methods in the ConcurrentHashMap of jdk8. None of the four construction methods initialize the internal array, but deal with the initial values of some variables
- The array initialization of concurrent HashMap of jdk8 is completed when the element is added for the first time
Constructor I
// There is no operation to maintain any variables. If this method is called, the default length of the array is 16 public ConcurrentHashMap() { }
Constructor II
- Note that the initial capacity obtained by calling this method is different from that of HashMap and concurrent HashMap of jdk7. Even if you pass a power of 2, the initial capacity calculated by this method is still a power of 2 greater than this value
// Pass in an initial capacity. Based on this value, ConcurrentHashMap will calculate a power of 2 greater than this value as the initial capacity // For example, if the initialCapacity passed in is 32, the initial capacity will be 64 public ConcurrentHashMap(int initialCapacity) { if (initialCapacity < 0) throw new IllegalArgumentException(); int cap = ((initialCapacity >= (MAXIMUM_CAPACITY >>> 1)) ? MAXIMUM_CAPACITY : // Pay attention to this place: initialCapacity + (initialCapacity > > > 1) + 1) tableSizeFor(initialCapacity + (initialCapacity >>> 1) + 1)); // sizeCtl: This is the key point, which will be discussed later this.sizeCtl = cap; } private static final int tableSizeFor(int c) { int n = c - 1; n |= n >>> 1; n |= n >>> 2; n |= n >>> 4; n |= n >>> 8; n |= n >>> 16; return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1; }
Constructor III
public ConcurrentHashMap(int initialCapacity, float loadFactor) { this(initialCapacity, loadFactor, 1); }
Constructor IV
//Calculate a capacity value greater than or equal to a given value, which is the power of 2 as the initial capacity public ConcurrentHashMap(int initialCapacity, float loadFactor, int concurrencyLevel) { if (!(loadFactor > 0.0f) || initialCapacity < 0 || concurrencyLevel <= 0) throw new IllegalArgumentException(); if (initialCapacity < concurrencyLevel) // Use at least as many bins initialCapacity = concurrencyLevel; // as estimated threads long size = (long)(1.0 + (long)initialCapacity / loadFactor); int cap = (size >= (long)MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : tableSizeFor((int)size); // sizeCtl: This is the key point, which will be discussed later this.sizeCtl = cap; }
Constructor V
//Build a ConcurrentHashMap based on a Map set //The initial capacity is 16 public ConcurrentHashMap(Map<? extends K, ? extends V> m) { // sizeCtl: This is the key point, which will be discussed later this.sizeCtl = DEFAULT_CAPACITY; putAll(m); }
**sizeCtl meaning interpretation
Note: the above construction methods all involve a variable sizeCtl. This variable is a very important variable and has very rich meanings. Its values are different, and the corresponding meanings are also different. Here, we first explain the meanings of different values of this variable, and then explain it further in the process of source code analysis
- sizeCtl is equal to 0, which means that the array is uninitialized and the initial capacity of the array is 16
- sizeCtl is a positive number. If the array is not initialized, it records the initial capacity of the array; If the array has been initialized, it records the expansion threshold of the array (initial capacity of the array * load factor)
- sizeCtl equals - 1, indicating that the array is being initialized
- sizeCtl is less than 0 and not - 1, indicating that the array is being expanded, indicating that n threads are jointly completing the expansion of the array at this time
jdk1.8 add security
Source code analysis
Add element put/putVal method
public V put(K key, V value) { return putVal(key, value, false); }
final V putVal(K key, V value, boolean onlyIfAbsent) { // If there is a null value or null key, throw the exception directly if (key == null || value == null) throw new NullPointerException(); // The hash value is calculated based on key, and a certain perturbation algorithm is carried out // This value must be a positive number, which is convenient for adding elements later to judge the type of the node int hash = spread(key.hashCode()); // Record the number of elements on a bucket. If there are more than 8, it will be turned into a red black tree int binCount = 0; for (Node<K,V>[] tab = table;;) { Node<K,V> f; int n, i, fh; // If the array is not initialized, initialize the array first if (tab == null || (n = tab.length) == 0) tab = initTable(); // If there is no element in the bucket position calculated by hash, use cas to add the element // (n - 1) & hash): calculate array angle else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) { // cas + spin (and the outer for form a spin cycle) to ensure the safety of element addition // cas judges whether there are elements in the bucket position, and executes the addition operation if there are no elements; If there are elements, discard the addition and return false if (casTabAt(tab, i, null, new Node<K,V>(hash, key, value, null))) break; } // If the hash value of the bucket position element calculated by the hash is MOVED, which proves that the capacity is being expanded, assist in capacity expansion else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); // The bucket position element calculated by hash is not empty, and it is not currently in the capacity expansion operation. Add the element else { V oldVal = null; // Lock the current bucket to ensure thread safety and perform element addition synchronized (f) { // The function of judgment again is to prevent other threads from turning the linked list into a tree, and the position of the bucket node will change if (tabAt(tab, i) == f) { // Common linked list node if (fh >= 0) { binCount = 1; // Loop through linked list for (Node<K,V> e = f;; ++binCount) { K ek; // If the key and hash values of the linked list node and the node to be added are the same // The description is the same node, and the new value overwrites the old value if (e.hash == hash && ((ek = e.key) == key || (ek != null && key.equals(ek)))) { oldVal = e.val; if (!onlyIfAbsent) e.val = value; // Exit traversal break; } // If it is different from the linked list node, it is inserted into the tail of the linked list node (tail insertion method) Node<K,V> pred = e; if ((e = e.next) == null) { pred.next = new Node<K,V>(hash, key, value, null); break; } } } // FH < 0 indicates a tree structure // Tree node to add elements to the red black tree else if (f instanceof TreeBin) { Node<K,V> p; binCount = 2; if ((p = ((TreeBin<K,V>)f).putTreeVal(hash, key, value)) != null) { oldVal = p.val; if (!onlyIfAbsent) p.val = value; } } } } if (binCount != 0) { // If the length of the linked list > = 8 and the length of the array > = 64, turn the linked list into a red black tree if (binCount >= TREEIFY_THRESHOLD) treeifyBin(tab, i); // If it is a duplicate key, return the old value directly if (oldVal != null) return oldVal; break; } } } // New elements are added. Maintain the length of the set and judge whether expansion is necessary addCount(1L, binCount); return null; }
From the above source code, we can see that when we need to add elements, we will lock the bucket corresponding to the current element. On the one hand, we can ensure the safety of multithreading when adding elements. At the same time, locking a bucket will not affect the operation of other buckets, so as to further improve the concurrency efficiency of multithreading
Locking diagram
Array initialization, initTable method
private final Node<K,V>[] initTable() { Node<K,V>[] tab; int sc; // Initialize the cas + thread to ensure the safety of the array while ((tab = table) == null || tab.length == 0) { // If the value (- 1) of sizeCtl is less than 0, it indicates that another thread is initializing at this time if ((sc = sizeCtl) < 0) Thread.yield(); // The current thread cedes cpu usage rights // cas modifies the value of sizeCtl to - 1. If the modification is successful, initialize the array; If it fails, continue to spin else if (U.compareAndSwapInt(this, SIZECTL, sc, -1)) { try { if ((tab = table) == null || tab.length == 0) { // If sizecl is 0, the default length is 16. Otherwise, the value of sizecl is taken int n = (sc > 0) ? sc : DEFAULT_CAPACITY; @SuppressWarnings("unchecked") // Build an array object based on the initial length Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n]; // Assign to member variable table table = tab = nt; // Calculate the expansion threshold and assign it to sc sc = n - (n >>> 2); // Equivalent to (n - 1/4 * n) == (0.75 * n) } } finally { // Assign the expansion threshold value to sizeCtl sizeCtl = sc; } break; } } return tab; }
jdk1.8 expansion safety
Source code analysis
Capacity expansion transfer method
private final void transfer(Node<K,V>[] tab, Node<K,V>[] nextTab) { int n = tab.length, stride; // If it is a multi-core cpu, divide each thread into tasks, and the minimum amount of tasks is the migration of 16 buckets if ((stride = (NCPU > 1) ? (n >>> 3) / NCPU : n) < MIN_TRANSFER_STRIDE) // stride = 16 stride = MIN_TRANSFER_STRIDE; // If it is a capacity expansion thread, the new array is null at this time if (nextTab == null) { try { @SuppressWarnings("unchecked") // Double expansion to create a new array Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n << 1]; nextTab = nt; } catch (Throwable ex) { sizeCtl = Integer.MAX_VALUE; return; } nextTable = nextTab; // transferIndex: current array length // Record the bucket that the thread starts to migrate, and migrate from back to front (corresponding to the following diagram) transferIndex = n; } // Record the end of the new array int nextn = nextTab.length; // Bucket bits that have been migrated will be occupied by this node (the hash value of this node is - 1(MOVED)) ForwardingNode<K,V> fwd = new ForwardingNode<K,V>(nextTab); boolean advance = true; boolean finishing = false; for (int i = 0, bound = 0;;) { Node<K,V> f; int fh; while (advance) { int nextIndex, nextBound; // i record the index value of the bucket currently being migrated // bound records the starting bucket of the next task migration // --I > = bound is true, indicating that the migration task assigned by the current thread has not been completed if (--i >= bound || finishing) advance = false; // No elements need to be migrated -- > subsequently, the number of threads for capacity expansion will be reduced by 1 and whether the capacity expansion is completed will be judged else if ((nextIndex = transferIndex) <= 0) { i = -1; advance = false; } // Calculate the starting bucket of the next task migration and assign this value to transferIndex else if (U.compareAndSwapInt (this, TRANSFERINDEX, nextIndex, nextBound = (nextIndex > stride ? nextIndex - stride : 0))) { bound = nextBound; i = nextIndex - 1; advance = false; } } // If there are no more buckets to be migrated, enter this if if (i < 0 || i >= n || i + n >= nextn) { int sc; // After capacity expansion, save the new array, recalculate the capacity expansion threshold and assign it to sizeCtl if (finishing) { nextTable = null; table = nextTab; // (n << 1) - (n >>> 1) = 2 * n - 1/2 * n = 3/2 * n = 1.5 * n = 0.75 * 2n sizeCtl = (n << 1) - (n >>> 1); return; } // Reduce the number of threads of capacity expansion task by 1 if (U.compareAndSwapInt(this, SIZECTL, sc = sizeCtl, sc - 1)) { // Judge whether all current expansion task threads have completed execution // be careful: // (sc - 2) != resizeStamp(n) << RESIZE_STAMP_SHIFT <=> sc! << RESIZE_STAMP_SHIFT + 2 // And (SC < < resize_stamp_shift + 2) means that a thread is expanding capacity if ((sc - 2) != resizeStamp(n) << RESIZE_STAMP_SHIFT) return; // All capacity expansion threads have completed execution, and the identification ends finishing = advance = true; i = n; } } // There is no element in the currently migrated bucket. Add a fwd node at this location directly else if ((f = tabAt(tab, i)) == null) advance = casTabAt(tab, i, null, fwd); // The current node has been migrated else if ((fh = f.hash) == MOVED) advance = true; // The current node needs to be migrated and locked to ensure the safety of multithreading else { // The migration logic here is the same as the concurrent HashMap of jdk7 and will not be repeated synchronized (f) { if (tabAt(tab, i) == f) { Node<K,V> ln, hn; if (fh >= 0) { int runBit = fh & n; Node<K,V> lastRun = f; for (Node<K,V> p = f.next; p != null; p = p.next) { int b = p.hash & n; if (b != runBit) { runBit = b; lastRun = p; } } if (runBit == 0) { ln = lastRun; hn = null; } else { hn = lastRun; ln = null; } for (Node<K,V> p = f; p != lastRun; p = p.next) { int ph = p.hash; K pk = p.key; V pv = p.val; if ((ph & n) == 0) ln = new Node<K,V>(ph, pk, pv, ln); else hn = new Node<K,V>(ph, pk, pv, hn); } setTabAt(nextTab, i, ln); setTabAt(nextTab, i + n, hn); setTabAt(tab, i, fwd); advance = true; } else if (f instanceof TreeBin) { TreeBin<K,V> t = (TreeBin<K,V>)f; TreeNode<K,V> lo = null, loTail = null; TreeNode<K,V> hi = null, hiTail = null; int lc = 0, hc = 0; for (Node<K,V> e = t.first; e != null; e = e.next) { int h = e.hash; TreeNode<K,V> p = new TreeNode<K,V> (h, e.key, e.val, null, null); if ((h & n) == 0) { if ((p.prev = loTail) == null) lo = p; else loTail.next = p; loTail = p; ++lc; } else { if ((p.prev = hiTail) == null) hi = p; else hiTail.next = p; hiTail = p; ++hc; } } ln = (lc <= UNTREEIFY_THRESHOLD) ? untreeify(lo) : (hc != 0) ? new TreeBin<K,V>(lo) : t; hn = (hc <= UNTREEIFY_THRESHOLD) ? untreeify(hi) : (lc != 0) ? new TreeBin<K,V>(hi) : t; setTabAt(nextTab, i, ln); setTabAt(nextTab, i + n, hn); setTabAt(tab, i, fwd); advance = true; } } } } } }
jdk1.8 multi thread capacity expansion efficiency improvement
The operation of multi thread assisted capacity expansion will be triggered in two places:
- When adding an element, if it is found that the bucket used by the added element pair is a fwd node, it will first assist in capacity expansion, and then add the element
- After adding elements, judge that the current number of elements has reached the capacity expansion threshold. At this time, it is found that the value of sizeCtl is less than 0 and the new array is not empty. At this time, it will assist in capacity expansion
Source code analysis
If the element is not added, assist in capacity expansion first, and then add the element after capacity expansion
final V putVal(K key, V value, boolean onlyIfAbsent) { if (key == null || value == null) throw new NullPointerException(); int hash = spread(key.hashCode()); int binCount = 0; for (Node<K,V>[] tab = table;;) { Node<K,V> f; int n, i, fh; if (tab == null || (n = tab.length) == 0) tab = initTable(); else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) { if (casTabAt(tab, i, null, new Node<K,V>(hash, key, value, null))) break; } // It is found that this is a fwd node. Assist in capacity expansion. After capacity expansion, recycle back and add elements else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); // Omit code
final Node<K,V>[] helpTransfer(Node<K,V>[] tab, Node<K,V> f) { Node<K,V>[] nextTab; int sc; if (tab != null && (f instanceof ForwardingNode) && (nextTab = ((ForwardingNode<K,V>)f).nextTable) != null) { int rs = resizeStamp(tab.length); while (nextTab == nextTable && table == tab && (sc = sizeCtl) < 0) { if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 || sc == rs + MAX_RESIZERS || transferIndex <= 0) break; if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1)) { // For capacity expansion, pass a nextTab that is not null transfer(tab, nextTab); break; } } return nextTab; } return table; }
Add elements first, and then assist in capacity expansion
private final void addCount(long x, int check) { // Omit code if (check >= 0) { Node<K,V>[] tab, nt; int n, sc; // The number of elements reaches the capacity expansion threshold while (s >= (long)(sc = sizeCtl) && (tab = table) != null && (n = tab.length) < MAXIMUM_CAPACITY) { int rs = resizeStamp(n); // If sizecl is less than 0, it indicates that capacity expansion is in progress, then assist in capacity expansion if (sc < 0) { if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 || sc == rs + MAX_RESIZERS || (nt = nextTable) == null || transferIndex <= 0) break; // Add 1 to the capacity expansion thread. After successful expansion, assist in capacity expansion if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1)) // Assist in capacity expansion. newTable is not null transfer(tab, nt); } // No other thread is expanding. After reaching the expansion threshold, it assigns a large negative number to sizeCtl // rs << RESIZE_ STAMP_ Shift: is a very large negative number // (RS < < resize_stamp_shift) + 2)): indicates that a thread is expanding at this time else if (U.compareAndSwapInt(this, SIZECTL, sc, (rs << RESIZE_STAMP_SHIFT) + 2)) // Capacity expansion, newTable is null transfer(tab, null); s = sumCount(); } } }
Note: the code for capacity expansion is in the transfer method, which will not be repeated here
graphic
Accumulation method of set length (key points)
Source code analysis
addCount method
- The counter cell array is not empty. The number of counter cell records in the array is used first
- If the array is empty, try to accumulate baseCount. After failure, execute fullAddCount logic
- If you are adding elements, you will continue to judge whether you need to expand the capacity
private final void addCount(long x, int check) { // ----------------------Maintain set length------------------------------ CounterCell[] as; long b, s; // When the counter cell array is not empty, the number of counter cell records in the array will be used first // Or when the accumulation operation of baseCount fails, the number of counter cell records in the array will be used if ((as = counterCells) != null || // cas determines whether the baseCount in the main memory is consistent with the current baseCount value. If so, add x to the baseCount !U.compareAndSwapLong(this, BASECOUNT, b = baseCount, s = b + x)) { CounterCell a; long v; int m; // Identifies whether there is multithreading contention boolean uncontended = true; // When the as array is empty // Or when as length is 0 // Or the element of as array bucket corresponding to the current thread is empty // Or the bucket of as array corresponding to the current thread is not empty, but the accumulation fails if (as == null || (m = as.length - 1) < 0 || (a = as[ThreadLocalRandom.getProbe() & m]) == null || !(uncontended = U.compareAndSwapLong(a, CELLVALUE, v = a.value, v + x))) { // If any of the above conditions is true, the method will be entered, and the incoming uncontended is false fullAddCount(x, uncontended); return; } if (check <= 1) return; // Calculate the number of elements s = sumCount(); } // -------------------------------------------------------------- // ---------------------Judge whether capacity expansion is required-------------------------- if (check >= 0) { Node<K,V>[] tab, nt; int n, sc; // When the number of elements reaches the capacity expansion threshold // And the array is not empty // And the array length is less than the maximum limit // If all the above conditions are met, the capacity expansion will be implemented while (s >= (long)(sc = sizeCtl) && (tab = table) != null && (n = tab.length) < MAXIMUM_CAPACITY) { // This is a big positive number int rs = resizeStamp(n); // If sc is less than 0, it indicates that a thread is expanding capacity, so it will assist in capacity expansion if (sc < 0) { // The operation ends when the capacity expansion ends or the number of threads reaches the maximum, or the array after capacity expansion is null or there are no more bucket bits to transfer if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 || sc == rs + MAX_RESIZERS || (nt = nextTable) == null || transferIndex <= 0) break; // Add 1 to the capacity expansion thread. After successful expansion, assist in capacity expansion if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1)) // Assist in capacity expansion. newTable is not null transfer(tab, nt); } // No other thread is expanding. After reaching the expansion threshold, it assigns a large negative number to sizeCtl // rs << RESIZE_ STAMP_ Shift: is a very large negative number // (RS < < resize_stamp_shift) + 2)): indicates that a thread is expanding at this time else if (U.compareAndSwapInt(this, SIZECTL, sc, (rs << RESIZE_STAMP_SHIFT) + 2)) // Capacity expansion, newTable is null transfer(tab, null); s = sumCount(); } } // ------------------------------------------------------------ }
fullAddCount method
- When the counter cell array is not empty, the value of the counter cell in the counter cell array is accumulated first
- When the counter cell array is empty, the counter cell array will be created, the default length is 2, and the value of counter cells in the array will be accumulated
- When the array is empty and another thread is creating the array at this time, try to accumulate baseCount. If it succeeds, it will be returned. Otherwise
private final void fullAddCount(long x, boolean wasUncontended) { int h; // Gets the hash value of the current thread if ((h = ThreadLocalRandom.getProbe()) == 0) { ThreadLocalRandom.localInit(); h = ThreadLocalRandom.getProbe(); wasUncontended = true; } // Identify whether there is conflict. If the last bucket is not null, it is true boolean collide = false; for (;;) { CounterCell[] as; CounterCell a; int n; long v; // The array is not empty. The value of CouterCell in the array is accumulated first if ((as = counterCells) != null && (n = as.length) > 0) { // The bucket corresponding to the current thread is null if ((a = as[(n - 1) & h]) == null) { if (cellsBusy == 0) { // Create a CounterCell object and assign x to the value attribute of the CounterCell object CounterCell r = new CounterCell(x); // Use cas to modify the cellBusy status to 1. If successful, put the CounterCell object just created into the array if (cellsBusy == 0 && U.compareAndSwapInt(this, CELLSBUSY, 0, 1)) { boolean created = false; try { CounterCell[] rs; int m, j; // rs array is not empty and its length is greater than 0 if ((rs = counterCells) != null && (m = rs.length) > 0 && // And the bucket corresponding to the current thread is empty rs[j = (m - 1) & h] == null) { // Put the CounterCell object into the array rs[j] = r; // true indicates that the input is successful created = true; } } finally { // Finally, setting cellsBusy to 0 indicates that it is not busy cellsBusy = 0; } if (created) // Successfully exited the loop break; // The bucket has been placed with counter cell object by another thread. Continue the loop continue; } } collide = false; } // If the bucket is not empty, recalculate the thread hash value, and then continue the loop else if (!wasUncontended) wasUncontended = true; // After recalculating the hash value, if the corresponding bucket is still not empty, the value attribute of the counter cell object of the bucket is accumulated // If successful, the cycle ends // Continue the following judgment if it fails else if (U.compareAndSwapLong(a, CELLVALUE, v = a.value, v + x)) break; // If the array is changed by another thread, or the array length exceeds the available cpu size, recalculate the thread hash value, otherwise continue to the next judgment else if (counterCells != as || n >= NCPU) collide = false; // When there is no conflict, change it to conflict, recalculate the thread hash and continue the loop else if (!collide) collide = true; // If the array length of CounterCell does not exceed the number of cpu cores, double the capacity of the array // And continue the cycle else if (cellsBusy == 0 && // cas judges that if cellsBusy is equal to 0, it will change cellsBusy to 1 U.compareAndSwapInt(this, CELLSBUSY, 0, 1)) { try { if (counterCells == as) { // n is 0 by default, and n < < 1 is equivalent to 2 times of capacity expansion, so the initial capacity of rs array is 2 CounterCell[] rs = new CounterCell[n << 1]; for (int i = 0; i < n; ++i) // Copy old array to new array rs[i] = as[i]; counterCells = rs; } } finally { cellsBusy = 0; } collide = false; continue; } // Recalculate the hash value of the current thread h = ThreadLocalRandom.advanceProbe(h); } // The counter cell array is empty, and no thread is creating the array. Modify the tag and create the array else if (cellsBusy == 0 && counterCells == as && U.compareAndSwapInt(this, CELLSBUSY, 0, 1)) { boolean init = false; try { if (counterCells == as) { CounterCell[] rs = new CounterCell[2]; rs[h & 1] = new CounterCell(x); counterCells = rs; init = true; } } finally { cellsBusy = 0; } if (init) break; } // If the array is empty and another thread is creating the array, try to accumulate baseCount, exit the loop if successful, and continue the loop if failed else if (U.compareAndSwapLong(this, BASECOUNT, v = baseCount, v + x)) break; } }
graphic
In the fullAddCount method, when the as array is not empty
jdk1.8 collection length acquisition (key points)
Source code analysis
size method
public int size() { long n = sumCount(); return ((n < 0L) ? 0 : (n > (long)Integer.MAX_VALUE) ? Integer.MAX_VALUE : (int)n); }
sumCount method
final long sumCount() { CounterCell[] as = counterCells; CounterCell a; //Gets the value of baseCount long sum = baseCount; if (as != null) { //Traverse the counter cell array and accumulate the value of each counter cell object for (int i = 0; i < as.length; ++i) { if ((a = as[i]) != null) sum += a.value; } } return sum; }