SecondPass support for unreal engine

1 Preface

At present, the illusory engine provides a material editor for users to realize various rendering effects.

However, some rendering effects require more than one Pass to complete. For example, the cartoon style characters are illuminated and colored on the surface and traced.

Note: of course, the function of tracing can also be realized through post-processing. Here are just examples.

It seems that the function can be easily realized by writing multi Pass shaders in Unity3D, but UE does not support it.

Therefore, the engine source code needs to be modified to realize the function of supporting SecondPass.

2. One Mesh multiple materials and one material multiple Pass

Before introducing the specific implementation, let's first understand the following two concepts:

• One Mesh multiple materials;
• One material has multiple passes;

They are two different concepts.

A Mesh has multiple materials, which means that a Mesh is divided into multiple parts, and the materials of each part are different.

For example, for a car, the body uses the material of Index0 and the tire uses the material of Index1.

Multiple passes for one material means that a Mesh can be drawn multiple times with the same or different materials.

For example, in the cartoon rendering mentioned earlier, the first time, we use the material for lighting calculation on the surface to render, and the second time, we use the stroked material to render the outline of the cartoon character.

3 Mesh Drawing Pipeline

Before introducing the implementation details of SecondPass, we need to understand how the unreal engine collects and renders Mesh.

Mesh Drawing Pipeline Provides a detailed introduction.

The core data structures involved are:

• FPrimitiveSceneProxy；
• FMeshBatch；
• FMeshDrawCommand；
• FMeshProcessor；

Among them, FPrimitiveSceneProxy is the representation of the primitive in the rendering thread, and it is the proxy of the game thread UPrimitiveComponent. During a rendering, the renderer collects FPrimitiveSceneProxy and converts it into FMeshBatch. Then convert FMeshBatach into FMeshDrawCommand through FMeshProcessor. Finally, draw through RHICommandList call.

The above provides the general logic of unreal rendering of Mesh.

The specific logic is also optimized: for example, for static Mesh, when its rendering state does not change, the FMeshDrawCommand (rendering instruction) generated by it will be cached. It can be rendered directly without collecting and converting again.

After the above understanding, we return to the question discussed in the current article, that is, how to support SecondPass?

It is conceivable that the final rendering data comes from FMeshBatch converted by FPrimitiveSceneProxy. You need to add two FMeshBatch in the process of rendering the collected elements. One is the material of FirstPass and the second is the material of SecondPass.

So, let's summarize how to implement it next:

1. Add an interface of SecondPass material to edit and set the material to be rendered;
2. Transfer the material resource to the corresponding fplimitivesceneproxy, and generate FMeshBatch of SecondPass when the renderer collects FMeshBatch;

4. SecondPass of staticmesh

The modified documents involved in the core functions are as follows:

StaticMesh.h/cpp
StaticMeshComponent.h/cpp
StaticMeshResources.h
StaticMeshRender.cpp

StaticMesh.h

Add an array to the UStaticMesh class to set the SecondPass material:

// UE source code location, add after this line of code
TArray<FStaticMaterial> StaticMaterials;
// Author added
/** List of SecondPass materials applied to this mesh. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = StaticMesh)
TArray<class UMaterialInterface*> SecondPassMaterials;

StaticMesh.cpp

In the UStaticMesh::PostLoad function, add:

// Author added
if (SecondPassMaterials.Num() != StaticMaterials.Num())
{
	SecondPassMaterials.SetNumZeroed(StaticMaterials.Num());
}
// UE source code location, add before this line of code
if(BodySetup == NULL)
{
//...
}

Through the modification of the above two files, the setting interface of SecondPass material is added.

StaticMeshComponent.h

In the UStaticMeshComponent class, add the following code:

// Interface used to override all SecondPass materials of all SubMesh of a UStaticMeshComponent
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "StaticMesh", meta = (DisplayName = "Second Pass Material"))
    class UMaterialInterface* SecondPassMaterial = nullptr;
// Get the SecondPass material interface corresponding to the material index
UMaterialInterface* GetSecondPassMaterial(int32 MaterialIndex) const;

StaticMeshComponent.cpp

Implement GetSecondPassMaterial interface

UMaterialInterface* UStaticMeshComponent::GetSecondPassMaterial(int32 MaterialIndex) const
{
	if (StaticMesh && StaticMesh->SecondPassMaterials.IsValidIndex(MaterialIndex) && StaticMesh->SecondPassMaterials[MaterialIndex])
	{
		return StaticMesh->SecondPassMaterials[MaterialIndex];
	}
	return SecondPassMaterial;
}

In the UStaticMeshComponent::GetUsedMaterials function, add:

• Add a valid material to the OutMaterials array;

if( GetStaticMesh() && GetStaticMesh()->RenderData )
{
	//...  Omit part of the source code
    // Author added
    // Second Pass Materials
    for (int32 MatIdx = 0; MatIdx < StaticMesh->SecondPassMaterials.Num(); ++MatIdx)
    {
        if (StaticMesh->SecondPassMaterials.IsValidIndex(MatIdx) && StaticMesh->SecondPassMaterials[MatIdx])
        {
            OutMaterials.Add(StaticMesh->SecondPassMaterials[MatIdx]);
        }
    }	
}
// Author added
if (SecondPassMaterial)
{
    OutMaterials.Add(SecondPassMaterial);
}

StaticMeshResources.h

Add a SecondPass material to FSectionInfo class and modify the constructor:

/** Information about an element of a LOD. */
struct FSectionInfo
{
	/** Default constructor. */
	FSectionInfo()
	: Material(NULL)
#if WITH_EDITOR
	, bSelected(false)
	, HitProxy(NULL)
#endif
	, FirstPreCulledIndex(0)
	, NumPreCulledTriangles(-1)
    // Author added
	, SecondPassMaterial(NULL)
	)
	{}

	/** The material with which to render this section. */
	UMaterialInterface* Material;
   	// Author added
	UMaterialInterface* SecondPassMaterial;
    
	//...  Omit part of the source code
};

StaticMeshRender.cpp

Add in FStaticMeshSceneProxy::FLODInfo::FLODInfo function

// Gather the materials applied to the LOD.
Sections.Empty(MeshRenderData->LODResources[LODIndex].Sections.Num());
for(int32 SectionIndex = 0;SectionIndex < LODModel.Sections.Num();SectionIndex++)
{
	const FStaticMeshSection& Section = LODModel.Sections[SectionIndex];
	FSectionInfo SectionInfo;
	// Determine the material applied to this element of the LOD.
	SectionInfo.Material = InComponent->GetMaterial(Section.MaterialIndex);
	// Author added
    UMaterialInterface* SecondPassMaterial = SectionInfo.SecondPassMaterial;
    //... 
}

Add code in the FStaticMeshSceneProxy::DrawStaticElements function to convert the SecondPass material to FMeshBatch.

The core function of DrawStaticElements is to convert material and vertex data into FMeshBatch for subsequent rendering.

//...  Omit part of the source code
for (int32 BatchIndex = 0; BatchIndex < NumBatches; BatchIndex++)
{
    FMeshBatch BaseMeshBatch;
	
    if (GetMeshElement(LODIndex, BatchIndex, SectionIndex, PrimitiveDPG, bIsMeshElementSelected, true, BaseMeshBatch))
    {
        if (NumRuntimeVirtualTextureTypes > 0)
        {
            // Runtime virtual texture mesh elements.
            FMeshBatch MeshBatch(BaseMeshBatch);
            SetupMeshBatchForRuntimeVirtualTexture(MeshBatch);
            for (ERuntimeVirtualTextureMaterialType MaterialType : RuntimeVirtualTextureMaterialTypes)
            {
                MeshBatch.RuntimeVirtualTextureMaterialType = (uint32)MaterialType;
                PDI->DrawMesh(MeshBatch, FLT_MAX);
            }
        }
        {
            PDI->DrawMesh(BaseMeshBatch, FLT_MAX);
        }
    	
        // The author added that GetSecondMeshElement is used to obtain the information of SecondPass material
        if (GetSecondMeshElement(LODIndex, BatchIndex, SectionIndex, PrimitiveDPG, bIsMeshElementSelected, true, BaseMeshBatch))
        {
            FMeshBatch MeshBatch(BaseMeshBatch);
            PDI->DrawMesh(MeshBatch, FLT_MAX);
        }

        //...  Omit part of the source code
    }
}

FStaticMeshSceneProxy::GetSecondMeshElement function is implemented by the author imitating GetMeshElement.

Almost all codes are consistent, except for the replacement of materials:

bool FStaticMeshSceneProxy::GetSecondMeshElement(
	int32 LODIndex,
	int32 BatchIndex,
	int32 SectionIndex,
	uint8 InDepthPriorityGroup,
	bool bUseSelectionOutline,
	bool bAllowPreCulledIndices,
	FMeshBatch& OutMeshBatch
) const
{
	const ERHIFeatureLevel::Type FeatureLevel = GetScene().GetFeatureLevel();
	const FStaticMeshLODResources& LOD = RenderData->LODResources[LODIndex];
	const FStaticMeshVertexFactories& VFs = RenderData->LODVertexFactories[LODIndex];
	const FStaticMeshSection& Section = LOD.Sections[SectionIndex];
	const FLODInfo& ProxyLODInfo = LODs[LODIndex];

	// Get SecondPass Material core code
	UMaterialInterface* MaterialInterface = nullptr;
	MaterialInterface = ProxyLODInfo.Sections[SectionIndex].SecondPassMaterial;
	if (MaterialInterface == nullptr)
	{
		return false;
	}
	FMaterialRenderProxy* MaterialRenderProxy = MaterialInterface->GetRenderProxy();
	if (MaterialRenderProxy == nullptr)
	{
		return false;
	}
	const FMaterial* Material = MaterialRenderProxy->GetMaterial(FeatureLevel);
	//...  Omit part of the source code
}

So far, the SecondPass setting of static Mesh can be realized through the modification of the above code.

5 SecondPass of skeletalmesh

Consistent with the above, first give the modified files involved in the core functions:

SkeletalMesh.h/cpp
SkeletalMeshTypes.h
SkinnedMeshComponent.h/cpp

SkeletalMesh.h

Add an array to the USkeletalMesh class to set the SecondPass material:

// UE source code location, add after this line of code
TArray<FSkeletalMaterial> Materials;
// Author added
UPROPERTY(EditAnywhere, Category = SkeletalMesh)
TArray<UMaterialInterface*> SecondPassMaterials;

USkinnedMeshComponent.h

In the USkinnedMeshComponent class, add the following code:

UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Mesh", meta = (DisplayName = "SecondPass Material "))
	UMaterialInterface* SecondPassMaterial = nullptr;
UMaterialInterface* GetSecondPassMaterial(int32 MaterialIndex) const;

USkinnedMeshComponent.cpp

Implement GetSecondPassMaterial interface

UMaterialInterface* USkinnedMeshComponent::GetSecondPassMaterial(int32 MaterialIndex) const
{
	if (SkeletalMesh && SkeletalMesh->SecondPassMaterials.IsValidIndex(MaterialIndex) && SkeletalMesh->SecondPassMaterials[MaterialIndex])
	{
		return SkeletalMesh->SecondPassMaterials[MaterialIndex];
	}
	return SecondPassMaterial;
}

In the USkinnedMeshComponent::GetUsedMaterials function, add:

if( SkeletalMesh )
{
    //...  Omit part of the source code
    // The author added SecondPass Materials
    for (int32 MatIdx = 0; MatIdx < SkeletalMesh->SecondPassMaterials.Num(); ++MatIdx)
    {
        if (SkeletalMesh->SecondPassMaterials.IsValidIndex(MatIdx) && SkeletalMesh->SecondPassMaterials[MatIdx])
        {
            OutMaterials.Add(SkeletalMesh->SecondPassMaterials[MatIdx]);
        }
    }
}
// Author added
if (SecondPassMaterial)
{
    OutMaterials.Add(SecondPassMaterial);
}

SkeletalMeshTypes.h

Add a SecondPass material to FSectionElementInfo class and modify the constructor:

/** info for section element in an LOD */
struct FSectionElementInfo
{
    FSectionElementInfo(UMaterialInterface* InMaterial, bool bInEnableShadowCasting, int32 InUseMaterialIndex, UMaterialInterface* InOutlineMaterial = nullptr)
    : Material( InMaterial )
    // Author added
    , SecondPassMaterial(InOutlineMaterial)
    , bEnableShadowCasting( bInEnableShadowCasting )
    , UseMaterialIndex( InUseMaterialIndex )
    #if WITH_EDITOR
    , HitProxy(NULL)
    #endif
    {}

    UMaterialInterface* Material;
    // Author added
    UMaterialInterface* SecondPassMaterial;
	
};

SkeletalMesh.cpp

Modify the FSkeletalMeshSceneProxy constructor:

// Author added
UMaterialInterface* SecondPassMaterial = Component->GetSecondPassMaterial(UseMaterialIndex);
LODSection.SectionElements.Add(
    FSectionElementInfo(
        Material,
        bSectionCastsShadow,
        UseMaterialIndex,
        SecondPassMaterial /* Author added */
        ));
MaterialsInUse_GameThread.Add(Material);

FSkeletalMeshSceneProxy::GetMeshElementsConditionallySelectable add the following code:

// UE source code location, add after this line of code
GetDynamicElementsSection(Views, ViewFamily, VisibilityMap, LODData, LODIndex, SectionIndex, bSectionSelected, SectionElementInfo, bInSelectable, Collector);

// Author added
if (SectionElementInfo.SecondPassMaterial)
{
	FSectionElementInfo SecondPassInfo(SectionElementInfo.SecondPassMaterial, false,SectionElementInfo.UseMaterialIndex);
	GetDynamicElementsSection(Views, ViewFamily, VisibilityMap, LODData, LODIndex, SectionIndex, bSectionSelected, OutlinePassInfo, bInSelectable, Collector);
}

• GetMeshElementsConditionallySelectable is called by GetDynamicMeshElements. Getdynamicalelementssection is called to collect FMeshBatch.

So far, through the modification of the above code, the SecondPass setting of dynamic Mesh can be realized.

6. Application of secondpass

This article has always mentioned the implementation of the stroke function of SecondPass in cartoon rendering. Then, let's implement the simplest material for tracing in UE.

The edges of the model are checked using the Inverted Hull algorithm.

The basic idea is to use front culling to expand the normal, so as to hook the edge around the model.

Frontal culling

The Inverted Hull algorithm requires FrontFaceCulling.

Since the phantom native does not support the setting of culled faces, it can be realized by combining the TwoSidedSign material node and Masked mixed mode.

TwoSidedSign returns 1 when the triangle is positive and - 1 when the triangle is negative. Multiply by - 1 and it's the opposite. Then there are: the positive value of the triangle is - 1 and the negative value is 1. Set to Opacity mask. So as to eliminate the front.

The principle is as follows:

• Because the stroking material is Mask type, when it is positive, the Opacity is - 1, which is less than the default Opacity Clip value (0.333), and will be clipped out; The positive one won't.
  

The vertex expands outward along the normal direction

The World Position Offset of the material is used to receive the offset of the vertex.

stroke color

Set the material shading model to Unlit mode and EmissiveColor to stroke color.

To sum up, the simplest stroking material can be realized. The complete material is as follows:

Effect optimization

In the above method, the thickness of the stroke line will change with the distance.

In order to ensure that the thickness of the stroke line will not change with the distance, it is necessary to expand a fixed distance in the NDC coordinate system;

However, the NDC coordinates cannot be directly affected in the material blueprint. What can be controlled is the World Position Offset, so we need to find other ways:

• Clip Space needs to be divided by w before being converted to NDC. This value is the Z value of View Space. It is necessary to eliminate the influence of this dynamic Z value. You need to multiply the distance of the expansion by Z.

After this operation, accessing the World Position Offset can ensure that the expansion value of NDC is fixed.

7 Summary

The modification described above provides a core function detail of realizing SecondPass material rendering in UE.

Here is an example of the effect:

However, the function of the engine is to be provided to art and programmers, and the corresponding UI panel needs to be customized and optimized to provide a more friendly use environment. The realization of this part is left to readers.

reference

• Static mesh multi Pass drawing scheme based on plug-in
• Multiple Pass sketching scheme of SkeletalMesh based on plug-in
• Using C + + to solve the problem of SkeletalMesh multi Pass rendering of UE4
• UE4 stroke function
• Mesh Drawing Pipeline