catalogue
Result 1: use the Serial collector
Result 2: use the ParallelScavenge collector
Execution entry of Parallel Scavenge
Look at the value of [need_full_gc]
Enter the [should_full_GC] method
[padded_average_promoted_in_bytes] is not the average size of previous promotions on the Internet
Recycling from another perspective: conditions for promotion to the old age
Extension 1: finalize method escape collection
Extension 2: Reference Collection
Extension 3: circular reference collection
Guarantee mechanism
Guarantee, with legal awareness, is used in contracts where the creditor requires the debtor to provide guarantee to the creditor in order to ensure the realization of its creditor's rights. The memory allocation guarantee mechanism, literally, is that the new generation (debtor) provides memory for users (creditors). Once the memory cannot be allocated, the old age (contract) will be used as a guarantee to allocate memory.
JVM memory area is divided into heap, stack, local method stack, method area, program counter and out of heap memory. The heap is divided into Cenozoic, old age, and one permanent generation (jdk8 has been removed). The Cenozoic is divided into Eden area (Eden area) and two survivors (survival area), and the survivor is divided into two from areas and to areas of the same size.
JVM allocates memory mainly in the new generation. When the new generation cannot allocate memory, it will use the old age. If the old generation cannot allocate memory, and the new and old generations cannot allocate memory after Full GC, an error will occur, and the famous OOM (OutOfMemoryError) memory overflows.
This guarantee mechanism was only activated by default in jdk6, Parameter is [- XX:+HandlePromotionFailure], the most commonly used version is jdk8, so this parameter can be ignored. The meaning of the parameter HandlePromotionFailure is whether guarantee failure is allowed, because it involves a probability problem, that is, the core judgment condition of guarantee: whether the largest continuous space in the old age is greater than that of the objects promoted to the old age Average size.
If it is greater than, it means that the old generation is likely to be able to load the promotion objects of the new generation. At this time, it is not necessary to execute Full GC, take a risk to execute Youth GC, and then promote the surviving objects of the new generation to the old age. Without this guarantee mechanism, Full GC will be executed directly, which will increase the frequency of performance impact. Of course, if it is less than, it means that the guarantee is likely to fail, and Full GC needs to be executed. Therefore, the guarantee mechanism is to reduce the execution frequency of Full GC and improve the performance of the application.
Next, observe the GC situation through a case. There are some differences between different collectors before using the guarantee mechanism.
Case 1: the new generation is unable to allocate memory, is promoted to the old age, and guarantees success
/**
* <p>
* Set the Java heap size to 20MB, of which 10M is allocated to the new generation and 10M is allocated to the old generation, and it is not scalable
* The proportion of each area in Cenozoic [- XX: SurvivorRatio=8], that is, Eden is 8, and from and to are all 1
* -verbose:gc -Xms20M -Xmx20M -Xmn10M -XX:SurvivorRatio=8 -XX:+HandlePromotionFailure -XX:+PrintGCDetails -XX:+UseSerialGC
* </p>
*
* @author xiaohong
* @since 2021-08-11 09:14
*/
public class Test1 {
private static final int _1MB = 1024 * 1024;
public static void main(String[] args) {
byte[] a1 = new byte[2 * _1MB];
byte[] a2 = new byte[2 * _1MB];
byte[] a3 = new byte[2 * _1MB];
byte[] b1 = new byte[4 * _1MB]; // The Eden area and the survivor area cannot be put here. At this time, start the guarantee mechanism to write 6MB objects to the elderly generation
}
}
Result 1: use the Serial collector
Result 2: use the ParallelScavenge collector
Case 2: Parallel Scavenge promotion condition: when the memory to be allocated is less than or equal to half of the memory in Eden area, it is possible to promote the existing objects in Eden area to the old age
/**
* <p>
* Parallel Scavenge Promotion conditions: when the memory to be allocated is less than or equal to half of the memory in Eden area, it is possible to promote the existing objects in Eden area to the old age
* </p>
*
* @author xiaohong
* @since 2021-08-11 09:14
*/
public class Test2 {
private static final int _1MB = 1024 * 1024;
// Parallel Scavenge
public static void main(String[] args) {
byte[] a1 = new byte[2 * _1MB];
byte[] a2 = new byte[2 * _1MB];
byte[] a3 = new byte[2 * _1MB];
// Judge 3MB (memory to be allocated) < = 6MB / 2 (half of Eden area memory),
// However, Eden district and survivor district cannot continue to allocate. At this time, the guarantee mechanism is started and 6MB objects are written to the elderly generation
byte[] b1 = new byte[3 * _1MB];
}
}
Results
It is found that there is a [Full GC (Ergonomics)]. What is the reason for this gc?
FullGC(Ergonomics)
Ergonomics translates to "Ergonomics", which is a gc mechanism used to better maintain the full gc process.
HotSpot source code download address portal: http://hg.openjdk.java.net/
http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/archive/4735f3031e23.tar.gz
Execution entry of Parallel Scavenge
// This method contains all heap specific policy for invoking scavenge.
// PSScavenge::invoke_no_policy() will do nothing but attempt to
// scavenge. It will not clean up after failed promotions, bail out if
// we've exceeded policy time limits, or any other special behavior.
// All such policy should be placed here.
//
// Note that this method should only be called from the vm_thread while
// at a safepoint!
bool PSScavenge::invoke() {
assert(SafepointSynchronize::is_at_safepoint(), "should be at safepoint");
assert(Thread::current() == (Thread*)VMThread::vm_thread(), "should be in vm thread");
assert(!Universe::heap()->is_gc_active(), "not reentrant");
ParallelScavengeHeap* const heap = (ParallelScavengeHeap*)Universe::heap();
assert(heap->kind() == CollectedHeap::ParallelScavengeHeap, "Sanity");
PSAdaptiveSizePolicy* policy = heap->size_policy();
IsGCActiveMark mark;
const bool scavenge_done = PSScavenge::invoke_no_policy();
const bool need_full_gc = !scavenge_done ||
policy->should_full_GC(heap->old_gen()->free_in_bytes());
bool full_gc_done = false;
if (UsePerfData) {
PSGCAdaptivePolicyCounters* const counters = heap->gc_policy_counters();
const int ffs_val = need_full_gc ? full_follows_scavenge : not_skipped;
counters->update_full_follows_scavenge(ffs_val);
}
// If yes, gc of [_adaptive_size_policy] type will be executed
// You can know by viewing the [const char * gccause:: to_string (gccause:: cause)] method,
// This type is Ergonomics
if (need_full_gc) {
GCCauseSetter gccs(heap, GCCause::_adaptive_size_policy);
CollectorPolicy* cp = heap->collector_policy();
const bool clear_all_softrefs = cp->should_clear_all_soft_refs();
if (UseParallelOldGC) {
full_gc_done = PSParallelCompact::invoke_no_policy(clear_all_softrefs);
} else {
full_gc_done = PSMarkSweep::invoke_no_policy(clear_all_softrefs);
}
}
return full_gc_done;
}
Ergonomics value
Look at the value of [need_full_gc]
const bool need_full_gc = !scavenge_done ||
policy->should_full_GC(heap->old_gen()->free_in_bytes());
Enter the [should_full_GC] method
// If the remaining space in the old age is less than that required for the next [estimated] collection, make a full gc now
// If the remaining free space in the old generation is less that
// that expected to be needed by the next collection, do a full
// collection now.
bool PSAdaptiveSizePolicy::should_full_GC(size_t old_free_in_bytes) {
// A similar test is done in the scavenge's should_attempt_scavenge(). If
// this is changed, decide if that test should also be changed.
bool result = padded_average_promoted_in_bytes() > (float) old_free_in_bytes;
if (PrintGCDetails && Verbose) {
if (result) {
gclog_or_tty->print(" full after scavenge: ");
} else {
gclog_or_tty->print(" no full after scavenge: ");
}
gclog_or_tty->print_cr(" average_promoted " SIZE_FORMAT
" padded_average_promoted " SIZE_FORMAT
" free in old gen " SIZE_FORMAT,
(size_t) average_promoted_in_bytes(),
(size_t) padded_average_promoted_in_bytes(),
old_free_in_bytes);
}
return result;
}
[padded_average_promoted_in_bytes] is not the average size of previous promotions on the Internet
Case parameters: [- verbose:gc -Xms20M -Xmx24M -Xmn10M -XX:SurvivorRatio=8 -XX:+PrintGCDetails -XX:+PrintTenuringDistribution]
result:
If it is changed to [- Xmx30M], there will be no [Ergonomics], which means that the [padded_average_promoted_in_bytes] method predicts more space, rather than simply calculating the average memory value of the previous promotion generations. Moreover, the Ergonomics mechanism will be implemented for the first promotion generations.
size_t padded_average_promoted_in_bytes() const {
return (size_t)avg_promoted()->padded_average();
}
float padded_average() const { return _padded_avg; }
[_padded_avg]
// The weighted average includes the deviation from the average, and its average plus some multiples thereof. This is the best estimate of the upper limit of future unknowns.
// A weighted average that includes a deviation from the average,
// some multiple of which is added to the average.
//
// This serves as our best estimate of an upper bound on a future
// unknown.
class AdaptivePaddedAverage : public AdaptiveWeightedAverage {
private:
float _padded_avg; // The last computed padded average
float _deviation; // Running deviation from the average
unsigned _padding; // A multiple which, added to the average,
// gives us an upper bound guess.
protected:
void set_padded_average(float avg) { _padded_avg = avg; }
void set_deviation(float dev) { _deviation = dev; }
public:
AdaptivePaddedAverage() :
AdaptiveWeightedAverage(0),
_padded_avg(0.0), _deviation(0.0), _padding(0) {}
AdaptivePaddedAverage(unsigned weight, unsigned padding) :
AdaptiveWeightedAverage(weight),
_padded_avg(0.0), _deviation(0.0), _padding(padding) {}
// Placement support
void* operator new(size_t ignored, void* p) throw() { return p; }
// Allocator
void* operator new(size_t size) throw() { return CHeapObj<mtGC>::operator new(size); }
// Accessor
float padded_average() const { return _padded_avg; }
float deviation() const { return _deviation; }
unsigned padding() const { return _padding; }
void clear() {
AdaptiveWeightedAverage::clear();
_padded_avg = 0;
_deviation = 0;
}
// Override
void sample(float new_sample);
// Printing
void print_on(outputStream* st) const;
void print() const;
};
Case 3: Ergonomics
/**
* @author xiaohong
* Configuration: 20M heap, non expandable, 10M for new and old generations respectively, Eden:Survivor==8:1
* -verbose:gc -Xms20M -Xmx20M -Xmn10M -XX:SurvivorRatio=8 -XX:+PrintTenuringDistribution -XX:+PrintGCDetails
* -XX:+UseSerialGC
* Separate test+/-
* -XX:-HandlePromotionFailure
**/
public class OldTest {
private static final int _1MB = 1024 * 1024;
public static void main(String[] args) throws InterruptedException {
// Note the following code, and you will find that after the empty method main is executed, the eden area occupies about 3mb of space
byte[] a1, a2, a3, a4, a5, a6, a7;
a1 = new byte[2 * _1MB];
a2 = new byte[2 * _1MB];
// young: a3，old: a1,a2
a3 = new byte[2 * _1MB]; // gc: empty main occupies 3MB, and gc will occur here. Due to the guarantee mechanism, a1 and a2 enter the old generation
a1 = null;
a4 = new byte[2 * _1MB];
a5 = new byte[2 * _1MB];
// young: a6，old: a2,a3,a4,a5
a6 = new byte[2 * _1MB]; // minor gc + full gc (Ergonomics): the new generation a3, a4 and a5 enter the old generation, and a1 is recycled
a4 = null;
a5 = null;
a6 = null;
// young: a7，old: a2,a3
a7 = new byte[6 * _1MB]; // Memory allocation failed, but there is no youth gc, but full gc (Ethics): a4, a5, a6
}
}
result
For the last gc, instead of youth gc, full gc of Ergonomics type is directly executed. This is the gc debugging mechanism of Parallel Scavenge collector Ergonomics, which can improve execution performance and reduce gc frequency.
Case 4: age determination of dynamic objects: the sum of the sizes of all objects of the same age in the Survivor space is greater than half of the space in the Survivor area. Objects with an age greater than or equal to this age can directly enter the elderly generation without waiting for the age required in MaxTenuringThreshold
/**
* @author xiaohong
* Configuration: 20M heap, non expandable, 10M for new and old generations respectively, Eden:Survivor==8:1
* -verbose:gc -Xms20M -Xmx20M -Xmn10M -XX:+PrintGCDetails -XX:SurvivorRatio=8 -XX:+UseSerialGC -XX:MaxTenuringThreshold=15 -XX:+PrintTenuringDistribution
**/
public class DynamicAgeTest {
private static final int _1MB = 1024 * 1024;
public static void main(String[] args) {
byte[] a1, a2, a3, a4;
// It can also be changed to / 8 here, because the memory of a1, a2 and a3 can fill the from area
a1 = new byte[_1MB / 4];
a2 = new byte[_1MB / 4]; // At this time, the memory (512K) of a1 + a2 occupies more than half of the survivor area space (1M)
// Age is set to 1
a3 = new byte[4 * _1MB]; // The ages of a1, a2 and a3 are the same, and gc occurs in a4. Due to [dynamic object age determination], a3 enters the elderly generation
a4 = new byte[4 * _1MB]; // Trigger Minor GC
a4 = null;
a4 = new byte[4 * _1MB]; // The minor gc is triggered, resulting in the last a4 memory recovery, and the age of the object in the from area reaches 2, which is promoted to the old age
byte[] a5 = new byte[4 * _1MB]; // Trigger minor gc, a4 to enter the elderly generation
}
}
Results
Case 5: reach the promotion age and promote to the old age (code reprinted super God: you're fake stupid)
/**
* @author You're stupid
* -verbose:gc -Xmx200M -Xmn50M -XX:TargetSurvivorRatio=60 -XX:+PrintTenuringDistribution -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:MaxTenuringThreshold=3
**/
public class SurvivorTest {
public static void main(String[] args) throws InterruptedException {
// These two objects will not be recycled. They are used to copy back and forth in s0 and s1 and increase the age until they reach the pretenuesizethreshold and are promoted to old
byte[] byte1m_1 = new byte[1 * 512 * 1024];
byte[] byte1m_2 = new byte[1 * 512 * 1024];
// After YoungGC, byte1m_1 and byte1m_ The age of 2 is 1
youngGc(1);
Thread.sleep(3000);
// After YoungGC again, byte1m_1 and byte1m_ The age of 2 is 2
youngGc(1);
Thread.sleep(3000);
// After the third YoungGC, byte1m_1 and byte1m_ The age of 2 is 3
youngGc(1);
Thread.sleep(3000);
// This time, because byte1m_1 and byte1m_ The age of 2 is already 3 and MaxTenuringThreshold=3, so these two objects will be promoted to the Old area, and after ygc, the objects in s0(from) and s1(to) spaces will be cleared
youngGc(1);
Thread.sleep(3000);
// The objects allocated in the main method will not be recycled. The continuous allocation is to reach the value specified by the TargetSurvivorRatio ratio, that is, 5M*60%=3M(Desired survivor size). Note: 5M is the size of the S area and 60% is specified by the TargetSurvivorRatio parameter. The Desired survivor size can be reached after the following three objects are allocated
byte[] byte1m_4 = new byte[1 * 1024 * 1024];
byte[] byte1m_5 = new byte[1 * 1024 * 1024];
byte[] byte1m_6 = new byte[1 * 1024 * 1024];
// During this ygc, since the s area has occupied 60%(-XX:TargetSurvivorRatio=60), the age of the object promotion will be recalculated. The calculation formula is: min(age, MaxTenuringThreshold) = 1
youngGc(1);
Thread.sleep(3000);
// Since age=1 was calculated in the previous ygc, byte1m is calculated in the next ygc_ 4, byte1m_ 5, byte1m_ 6 will be promoted to the old area without waiting for MaxTenuringThreshold so many times. After this ygc, the objects in s0(from) and s1(to) spaces will be cleared again, and all objects will be promoted to old
youngGc(1);
Thread.sleep(3000);
System.out.println("hello world");
}
private static void youngGc(int ygcTimes) {
for (int i = 0; i < ygcTimes * 40; i++) {
byte[] byte1m = new byte[1 * 1024 * 1024];
}
}
}
Recycling from another perspective: conditions for promotion to the old age
- Guarantee mechanism
- Large objects enter the elderly generation directly
- When the age reaches [- XX: maxtenuringthreshold] (15 by default), you can be promoted to the old age. Every time minor gc still lives in the new generation, the age increases by 1
- Dynamic object age determination: if the sum of the sizes of all objects of the same age in the Survivor space is greater than half of the Survivor space (according to the parameter [TargetSurvivorRatio], the default is 50, that is, 50%, half), objects older than or equal to this age can directly enter the elderly generation without waiting for the age required in MaxTenuringThreshold.
Extension 1: finalize method escape collection
finalize is an Object method that will be executed once when the Object is recycled. This method is used to close other related resources recycled by the Object. However, this method is generally not used because it is possible to escape recycling.
public class FinalizeEscapeGC {
private static FinalizeEscapeGC SAVE_HOOK = null;
private void isAlive() {
System.out.println("yes, i am still alive :)");
}
@Override
protected void finalize() throws Throwable {
super.finalize();
System.out.println("finalize method executed!");
FinalizeEscapeGC.SAVE_HOOK = this;
}
public static void main(String[] args) throws Throwable {
SAVE_HOOK = new FinalizeEscapeGC();
// The object successfully saved itself for the first time
SAVE_HOOK = null;
System.gc();
// Because the finalize method has a low priority, pause for 0.5 seconds to wait for it
Thread.sleep(500);
if (SAVE_HOOK != null) {
SAVE_HOOK.isAlive();
} else {
System.out.println("no, i am dead :( ");
}
// The following code is exactly the same as the above, but this self rescue failed
SAVE_HOOK = null;
System.gc();
// Because the finalize method has a low priority, pause for 0.5 seconds to wait for it
Thread.sleep(500);
if (SAVE_HOOK != null) {
SAVE_HOOK.isAlive();
} else {
System.out.println("no, i am dead :( ");
}
}
}
Extension 2: Reference Collection
Generally speaking, strong references are used by default. In addition to being convenient for writing, another reason is the function. After the function is called, the variables in the function can be recycled. Therefore, soft references, weak applications and virtual references are used less frequently in daily applications. Here is an analysis of the recycling of these applications.
import io.github.darryring.util.learn.test.util.ThreadUtil;
import java.lang.ref.PhantomReference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.SoftReference;
import java.lang.ref.WeakReference;
import java.util.WeakHashMap;
/**
* @author xiaohong
* @explain Reference type: strong, soft, weak, virtual
*/
class Reference1Test {
/**
* The recycled key can be obtained by referencing the queue. Refer to {@ link WeakHashMap#expungeStaleEntries()}
* Virtual reference source code instance reference: {@ link sun.misc.Cleaner}
*/
public static void main(String[] args) {
String s = "Strong reference";
System.out.println(s);
SoftReference<String> softReference = new SoftReference<>("Soft reference"); // Recycle when memory is low
System.out.println(softReference.get());
WeakReference<String> weakReference = new WeakReference<>("Weak reference"); // gc time recovery
System.out.println(weakReference.get());
PhantomReference<Object> phantomReference = new PhantomReference<>("Virtual reference", new ReferenceQueue<>()); // You can get a system notification when recycling
phantomReference = new PhantomReference<>(new Object(), new ReferenceQueue<>()); // If it is a String, the manual gc may not be recycled because the constant pool has its own recycling mechanism
System.out.println(phantomReference.get()); // Always return null, that is, you cannot obtain an instance through [virtual reference]
System.out.println(phantomReference.isEnqueued()); // Judge whether it is recycled
System.gc();
ThreadUtil.waitThreeSeconds(); // Wait for the gc operation to complete
System.out.println(phantomReference.isEnqueued()); // Judge whether it is recycled
}
}
Extension 3: circular reference collection
/**
* @author xiaohong
* -XX:+PrintGCDetails
**/
public class ReferenceCountingGC {
public Object instance = null;
private static final int _1MB = 1024 * 1024;
private byte[] bigSize = new byte[2 * _1MB];
public static void main(String[] args) {
ReferenceCountingGC objA = new ReferenceCountingGC();
ReferenceCountingGC objB = new ReferenceCountingGC();
// Circular reference
objA.instance = objB;
objB.instance = objA;
objA = null;
objB = null;
// Assuming GC occurs in this line, can obja and objB be recycled?
System.gc();
// According to the enumeration root node algorithm, if the objects of the circular reference are null, the connection with the root node will be lost, so it will be recycled.
System.out.println("done"); // This sentence will also take up about 1mb of space for printing
}
}
References
JVM tuning super God: are you stupid
https://blog.csdn.net/kavito/article/details/82292035 (reprinted from Tencent cloud)
https://blog.csdn.net/weixin_43194122/article/details/91526740 (reprinted from Tencent cloud)
https://cloud.tencent.com/developer/article/1082687 (Tencent cloud blog)
https://cloud.tencent.com/developer/column/2258/tag-10164 (Tencent cloud blog)