Summary

GPUImage is a well-known open source library for image processing, which allows you to use GPU accelerated filters and other special effects on pictures, videos, and cameras. Compared with the Core Image framework, you can use custom filters based on the interface provided by GPUImage. Project address: https://github.com/BradLarson/GPUImage
This article mainly reads the source code of GLProgram and GPUImageContext, two important classes in the GPUImage framework. These two classes are the foundation of the GPUI mage framework, and the knowledge involved in them also includes the OpenGL ES foundation and multithreading foundation. The following is the source code content:
GLProgram
GPUImageContext

Basics

Reading GPUImage source code requires a certain amount of knowledge. Here I list some basic knowledge needed: OpenGL ES 2.0, AV Foundation, Core Graphics. If you are not familiar with the above framework, you can learn something about it. I wrote some before. OpenGL topic http://www.jianshu.com/c/30e2e76bc140 Relevant thematic series, you can also go to learn.

GLProgram

Before learning OpenGL, the first thing we need to do is initialize the OpenGL ES environment, compile, link vertex shaders and fragment shaders. GLProgram is encapsulated in GPUImage to deal with the creation of OpenGL ES programs and other related work.
GLProgram's methods are not very many. Here's a brief introduction:

• The initialization method can initialize the path or string of the vertex shader and the path and string of the slice source shader according to the need.

- (id)initWithVertexShaderString:(NSString *)vShaderString 
            fragmentShaderString:(NSString *)fShaderString;
- (id)initWithVertexShaderString:(NSString *)vShaderString 
          fragmentShaderFilename:(NSString *)fShaderFilename;
- (id)initWithVertexShaderFilename:(NSString *)vShaderFilename 
            fragmentShaderFilename:(NSString *)fShaderFilename;

The initialization process includes the creation and compilation of vertex slice source shader, the creation of shader program, and the attachment of vertex slice source shader to shader program.

- (id)initWithVertexShaderString:(NSString *)vShaderString 
            fragmentShaderString:(NSString *)fShaderString;
{
    if ((self = [super init])) 
    {
        _initialized = NO;

        // Initialize an array of attributes
        attributes = [[NSMutableArray alloc] init];
        // Initialize uniform attribute array
        uniforms = [[NSMutableArray alloc] init];

        // Create a shader program
        program = glCreateProgram();

        // Compiled vertex shader
        if (![self compileShader:&vertShader 
                            type:GL_VERTEX_SHADER 
                          string:vShaderString])
        {
            NSLog(@"Failed to compile vertex shader");
        }

        // Compiler source shader
        // Create and compile fragment shader
        if (![self compileShader:&fragShader 
                            type:GL_FRAGMENT_SHADER 
                          string:fShaderString])
        {
            NSLog(@"Failed to compile fragment shader");
        }

        // Attach vertex source shader to shader program
        glAttachShader(program, vertShader);
        glAttachShader(program, fragShader);
    }

    return self;
}

• Link programs, like compiled languages, require links for OpenGL programs.

- (BOOL)link
{
//    CFAbsoluteTime startTime = CFAbsoluteTimeGetCurrent();

    GLint status;

    // Link Shader Program
    glLinkProgram(program);

    // Get link status
    glGetProgramiv(program, GL_LINK_STATUS, &status);
    // Link failure returns
    if (status == GL_FALSE)
        return NO;

    // If the link succeeds, you can delete the relevant shader and release the resources.
    if (vertShader)
    {
        glDeleteShader(vertShader);
        vertShader = 0;
    }
    if (fragShader)
    {
        glDeleteShader(fragShader);
        fragShader = 0;
    }

    // Setting Initialization Success Identification
    self.initialized = YES;

//    CFAbsoluteTime linkTime = (CFAbsoluteTimeGetCurrent() - startTime);
//    NSLog(@"Linked in %f ms", linkTime * 1000.0);

    return YES;
}

• Use the shader program

- (void)use
{
    glUseProgram(program);
}

• When passing values to the shader, we need to get the location of the relevant attributes. GLProgram also provides an interface for us to get the location of variables.

- (void)addAttribute:(NSString *)attributeName
{
    // First determine whether the current attribute already exists
    if (![attributes containsObject:attributeName])
    {
        // If there is no such thing as adding an array of attributes first, then binding the location of the attribute to the location in the array of attributes
        [attributes addObject:attributeName];
        glBindAttribLocation(program, 
                             (GLuint)[attributes indexOfObject:attributeName],
                             [attributeName UTF8String]);
    }
}
// END:addattribute
// START:indexmethods
- (GLuint)attributeIndex:(NSString *)attributeName
{
    // Gets the position of the shader attribute variable, that is, the position of the array (based on previous binding relationships)
    return (GLuint)[attributes indexOfObject:attributeName];
}
- (GLuint)uniformIndex:(NSString *)uniformName
{
    // Get the location of Uniform variables
    return glGetUniformLocation(program, [uniformName UTF8String]);
}

• Release resources. When destructing, clean up related resources such as shaders.

- (void)dealloc
{
    if (vertShader)
        glDeleteShader(vertShader);

    if (fragShader)
        glDeleteShader(fragShader);

    if (program)
        glDeleteProgram(program);

}

GPUImageContext

The GPUImageContext class provides OpenGL ES basic context, GPUImage related processing threads, GLProgram cache, frame cache. Because it is a context object, the module provides more access and setting related methods.

• Attribute List

// GPUImage handles related and serial queues drawn by OpenGL
@property(readonly, nonatomic) dispatch_queue_t contextQueue;
// Currently used shader program
@property(readwrite, retain, nonatomic) GLProgram *currentShaderProgram;
// OpenGLES context object
@property(readonly, retain, nonatomic) EAGLContext *context;
// Texture Caching in CoreVideo
@property(readonly) CVOpenGLESTextureCacheRef coreVideoTextureCache;
// Frame Cache
@property(readonly) GPUImageFramebufferCache *framebufferCache;

• Initialization process. The queue identifier is set by dispatch_queue_set_specific s during initialization for reasons mentioned in Effective Objective-C 2.0 or in my previous blog Resuming hot issues of Effective Objective-C 2.0

- (id)init;
{
    if (!(self = [super init]))
    {
        return nil;
    }

    // Create OpenGL Rendering Queue
    openGLESContextQueueKey = &openGLESContextQueueKey;
    _contextQueue = dispatch_queue_create("com.sunsetlakesoftware.GPUImage.openGLESContextQueue", GPUImageDefaultQueueAttribute());

#if OS_OBJECT_USE_OBJC
    // Setting Queue Identification
    dispatch_queue_set_specific(_contextQueue, openGLESContextQueueKey, (__bridge void *)self, NULL);
#endif
    // Initialize Shader Cache Related Array
    shaderProgramCache = [[NSMutableDictionary alloc] init];
    shaderProgramUsageHistory = [[NSMutableArray alloc] init];

    return self;
}

• Method List

// Get queue identity
+ (void *)contextKey;
// Singleton object
+ (GPUImageContext *)sharedImageProcessingContext;
// Get the processing queue
+ (dispatch_queue_t)sharedContextQueue;
// Frame Cache
+ (GPUImageFramebufferCache *)sharedFramebufferCache;
// Setting the current context
+ (void)useImageProcessingContext;
- (void)useAsCurrentContext;
// Setting up the current GL program
+ (void)setActiveShaderProgram:(GLProgram *)shaderProgram;
- (void)setContextShaderProgram:(GLProgram *)shaderProgram;
// Support for acquiring device OpenGLES-related features
+ (GLint)maximumTextureSizeForThisDevice;
+ (GLint)maximumTextureUnitsForThisDevice;
+ (GLint)maximumVaryingVectorsForThisDevice;
+ (BOOL)deviceSupportsOpenGLESExtension:(NSString *)extension;
+ (BOOL)deviceSupportsRedTextures;
+ (BOOL)deviceSupportsFramebufferReads;
// Texture size adjustment to ensure that texture does not exceed the maximum size supported by OpenGLES
+ (CGSize)sizeThatFitsWithinATextureForSize:(CGSize)inputSize;
// Rendering Cache on Device
- (void)presentBufferForDisplay;
// To create GLProgram, first look it up in the cache, and if not, create it
- (GLProgram *)programForVertexShaderString:(NSString *)vertexShaderString fragmentShaderString:(NSString *)fragmentShaderString;
// Create Sharegroup
- (void)useSharegroup:(EAGLSharegroup *)sharegroup;
// Manage fast texture upload
+ (BOOL)supportsFastTextureUpload;
@end

Since GPUImageContext is equivalent to a context object, and mainly manages other objects, there is not much complex business in this category. Here we mainly look at several methods:

• Adjust the texture size to ensure that the texture does not exceed the maximum size supported by OpenGLES:

+ (CGSize)sizeThatFitsWithinATextureForSize:(CGSize)inputSize;
{
    GLint maxTextureSize = [self maximumTextureSizeForThisDevice]; 
    if ( (inputSize.width < maxTextureSize) && (inputSize.height < maxTextureSize) )
    {
        return inputSize;
    }

    CGSize adjustedSize;
    if (inputSize.width > inputSize.height)
    {
        adjustedSize.width = (CGFloat)maxTextureSize;
        adjustedSize.height = ((CGFloat)maxTextureSize / inputSize.width) * inputSize.height;
    }
    else
    {
        adjustedSize.height = (CGFloat)maxTextureSize;
        adjustedSize.width = ((CGFloat)maxTextureSize / inputSize.height) * inputSize.width;
    }

    return adjustedSize;
}

• Get the maximum texture size supported by OpenGLES.

+ (GLint)maximumTextureSizeForThisDevice;
{
    static dispatch_once_t pred;
    static GLint maxTextureSize = 0;

    dispatch_once(&pred, ^{
        [self useImageProcessingContext];
        glGetIntegerv(GL_MAX_TEXTURE_SIZE, &maxTextureSize);
    });

    return maxTextureSize;
}

• To create GLProgram, first look it up in the cache, and if not, create it

- (GLProgram *)programForVertexShaderString:(NSString *)vertexShaderString fragmentShaderString:(NSString *)fragmentShaderString;
{
    NSString *lookupKeyForShaderProgram = [NSString stringWithFormat:@"V: %@ - F: %@", vertexShaderString, fragmentShaderString];
    GLProgram *programFromCache = [shaderProgramCache objectForKey:lookupKeyForShaderProgram];

    if (programFromCache == nil)
    {
        programFromCache = [[GLProgram alloc] initWithVertexShaderString:vertexShaderString fragmentShaderString:fragmentShaderString];
        [shaderProgramCache setObject:programFromCache forKey:lookupKeyForShaderProgram];
//        [shaderProgramUsageHistory addObject:lookupKeyForShaderProgram];
//        if ([shaderProgramUsageHistory count] >= MAXSHADERPROGRAMSALLOWEDINCACHE)
//        {
//            for (NSUInteger currentShaderProgramRemovedFromCache = 0; currentShaderProgramRemovedFromCache < 10; currentShaderProgramRemovedFromCache++)
//            {
//                NSString *shaderProgramToRemoveFromCache = [shaderProgramUsageHistory objectAtIndex:0];
//                [shaderProgramUsageHistory removeObjectAtIndex:0];
//                [shaderProgramCache removeObjectForKey:shaderProgramToRemoveFromCache];
//            }
//        }
    }

    return programFromCache;
}

• Create the EAGLContext context object using the API of kEAGL Rendering APIOpenGLES2, OpenGL ES 2.0.

- (EAGLContext *)createContext;
{
    EAGLContext *context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2 sharegroup:_sharegroup];
    NSAssert(context != nil, @"Unable to create an OpenGL ES 2.0 context. The GPUImage framework requires OpenGL ES 2.0 support to work.");
    return context;
}

• Set the current context object and the current shader program.

- (void)setContextShaderProgram:(GLProgram *)shaderProgram;
{
    EAGLContext *imageProcessingContext = [self context];
    if ([EAGLContext currentContext] != imageProcessingContext)
    {
        [EAGLContext setCurrentContext:imageProcessingContext];
    }

    if (self.currentShaderProgram != shaderProgram)
    {
        self.currentShaderProgram = shaderProgram;
        [shaderProgram use];
    }
}

summary

GLProgram is closely related to shader program creation, including creation, compilation, linking, use and so on.
GPUImageContext is the context object of GPUImage. It manages OpenGLES context object, GL program, frame cache and other basic components of GPUImage.