Android continuous integration practice -- ABI subcontracting and special channel compilation requirements

Posted by arunmj82 on Sun, 16 Jan 2022 07:14:38 +0100

preface

After the continuous integration two years ago, it has been used. There are no major changes to be made recently. Until the domestic application market began to promote 32-bit and 64 bit architecture subcontracting.
Just a while ago, because of the name of our app, we needed to change the name in the vivo store before it could be put on the shelf, which was uncomfortable. In addition to automatically packaging other channels for each release, we had to compile and strengthen the vivo channel separately, which was just solved this time.
This article is a supplement to the third article.
Note that this article is based on continuous integration configured by myself. The following is a review of the series.

Series articles

Android continuous integration practice (I) -- building Gitlab server from 0

Android continuous integration practice (II) -- configure docker + gitlab runner to realize online automatic compilation

Android continuous integration practice (III) -- writing gitlab-ci.yml automation

Android continuous integration practice (IV) -- configuring WebHook notification compilation results

Android continuous integration practice (V) -- ABI subcontracting and special channel compilation requirements

Text start

ABI subcontracting

Achieve goals

When the official version is released, the apk of arm64-v8a(64 bit) and arm64-v7a (32 bit) are compiled automatically and automatically;
When the beta is released, compile an apk that contains both armeabi-v7a and x86 so libraries.

practice
  1. abi segmentation

    abi segmentation can be realized by directly configuring the split node of gradle. Subcontracting is required only for the release version, so it is also necessary to judge whether it is release or debug during abi segmentation.
    Enable abi segmentation during release; Disable segmentation during debug, and configure multiple architecture so libraries using the original ndk filtering

    // main module's build.gradle
    android{
        ...
         // Architecture subcontracting configuration, only release version needs architecture subcontracting; The debug version also follows the previous ndk filtering configuration
         splits {
             boolean isReleaseTask = gradle.startParameter.taskNames.any { it.contains("Release") }
             if (!isReleaseTask) {
                 // debug uses the ndk filtering method (the split execution time is too early, and the ndk configuration does not take effect in buildTypes)
                 android.defaultConfig.ndk { abiFilters "armeabi-v7a", "x86" }
             }
             // release will enable abi segmentation
             abi {
                 enable isReleaseTask
                 reset()
                 include "armeabi-v7a", "arm64-v8a"
                 universalApk false
             }
         }
         ...
     }
    
    
  2. apk file name modification

    After the abi is split, the abi configuration needs to be reflected in the apk name. After the continuous integration process is compiled, we need to get the apk to continue to perform the tasks of automatic reinforcement and automatic multi-channel. Therefore, we need a known and determined file name to facilitate the subsequent continuous integration tasks to get the apk.

     // main module's build.gradle
     ...
     import com.android.build.OutputFile
     ...
     android{
        ...
         applicationVariants.all { variant ->
             variant.outputs.all { output ->
                 def abiName = output.getFilter(OutputFile.ABI)
                 if (abiName != null)
                     outputFileName = "${variant.buildType.name}_${defaultConfig.applicationId}_${defaultConfig.versionName}_${defaultConfig.versionCode}_${abiName}.apk"
                 else
                     outputFileName = "${variant.buildType.name}_${defaultConfig.applicationId}_${defaultConfig.versionName}_${defaultConfig.versionCode}.apk"
             }
         }
         ...
     }
    
  3. Modify the configuration of automatic reinforcement

    Now that the release is packaged, there are two apks, so the subsequent reinforcement and multi-channel tasks also need to support two apks

     //Modify file 360jiagu
    
     #!/usr/bin/env bash
     
     ################
     #360 Reinforcement profile #
     ################
     
     ...
     
     # 32-bit apk path
     APK_NAME_32=release_com.wln100.future_${CI_COMMIT_TAG}_${CI_PIPELINE_ID}_armeabi-v7a.apk
     APK_PATH_32=app/build/outputs/apk/release/${APK_NAME_32}   #apk path requiring reinforcement
     # 64 bit apk path
     APK_NAME_64=release_com.wln100.future_${CI_COMMIT_TAG}_${CI_PIPELINE_ID}_arm64-v8a.apk
     APK_PATH_64=app/build/outputs/apk/release/${APK_NAME_64}   #apk path requiring reinforcement
     # Output reinforcement package path
     DEST=app/build/outputs/apk/release/
     
     ...
     
     echo "------ running! ------"
     
     chmod +x ${JAVACMD}
     ${JAVACMD} -jar ${BASE} -version
     ${JAVACMD} -jar ${BASE} -login ${NAME} ${PASSWORD}
     ${JAVACMD} -jar ${BASE} -importsign ${KEYSTORE_PATH} ${KEY_PASSWORD} ${KEY_ALIAS} ${STORE_PASSWORD} # Configure signature information
     ${JAVACMD} -jar ${BASE} -showsign
     ${JAVACMD} -jar ${BASE} -deletemulpkg # Clear configured channel information
     ${JAVACMD} -jar ${BASE} -importmulpkg ./channels.txt # Configure channel information
     ${JAVACMD} -jar ${BASE} -showmulpkg
     ${JAVACMD} -jar ${BASE} -showconfig
     echo "------ Start reinforcing 32 bits apk ------"
     ${JAVACMD} -jar ${BASE} -jiagu ${APK_PATH_32} ${DEST} -autosign -automulpkg
     echo "------ Start hardening 64 bit apk ------"
     ${JAVACMD} -jar ${BASE} -jiagu ${APK_PATH_64} ${DEST} -autosign -automulpkg
     
     echo "------ finished! ------"
     
    

After the above configuration, verify the following results with the label of the packaged official version:

Special channel compilation requirements

In particular, the compilation requirements of special channels are not related to the abi subcontracting above, and they are independent. If your project needs abi subcontracting and automatically outputs channel packages with special needs, you need to integrate yourself

Achieve goals

During continuous integration, the consolidated multi-channel apk is automatically compiled, and the apk of vivo channel needs to change the application name

practice

The automatic multi-channel is realized by relying on the sdk reinforced by 360. Now, if you want to change the app name when the vivo channel is automatically compiled, the vivo channel cannot be processed uniformly by the multi-channel commands reinforced by 360.

How to package the apk of vivo channel without automatic compilation? The modified version number and version name are consistent with the version number and version name generated by automatic compilation; Modify the original channel data in the list file to vivo; Modify app in resource file_ Name is the name of operation requirements; Execute the compilation task gradlew assemblyrelease and wait until the compilation is completed...

This manual process is not complicated but cumbersome. It must be involved in the program integration process to realize automation and free both hands.

At the beginning, the idea of implementation is to directly use gradle's product variant configuration node productFlavors to configure a vivo channel. This is feasible, but not perfect. After configuring the variant, no matter release or debug, the variant package will be packaged after specifying the variant. If not specified, all variant packages will be packaged. However, what we need is to package vivo as a special channel during release, which is not needed in other cases.
The package command structure after defining the variant is [compilation type + product variant], =. Since the debug compilation type will also support variants after defining the variant, why not directly define a new compilation type, which is juxtaposed with the default release and debug compilation types.

  1. Remove the configuration of vivo channel

  2. New compilation type releaseVivo

    //main module's build.gradle  
    ...
    android{
        ...
        buildTypes{
            ...
             create("releaseVivo") {
                 initWith(getByName("release")) //Inherit the configuration of release compilation type  
                 manifestPlaceholders["app_label"] = "@string/app_name_vivo" //Modify the app name of vivo channel
                 manifestPlaceholders["umeng_channel"] = "vivo" //Set vivo channel ID field
                 matchingFallbacks = ["release"]  //Other module s do not define this compilation type. If it does not match, use the release compilation type
             }
             ...
        }
        ...
    }
    ...
    
    

    Enter the gradle command gradlew assemblyreleasevivo. After compiling, it can see the apk of vivo channel

  3. Modification gitlab-ci.yml

    ...
    
    # Compilation phase
    # Make formal package with label in the format of "x.x.x"
     release:
       stage: build
       only:
           - /^[\d]+\.[\d]+\.[\d]+$/
       except:
           - branches
       script:
           - ./update-version-code
           - ./gradlew assembleRelease assembleReleaseVivo
       artifacts:
           paths:
               - app/build/outputs/apk/release/
               - app/build/outputs/apk/releaseVivo/
               - app/build/outputs/mapping/release/
       tags:
           - android
     
    
    #region reinforcement and channel packaging stage
    
     reinforceAndChannel:
       stage: reinforceAndChannel
       only:
         - /^[\d]+\.[\d]+\.[\d]+$/
       script:
         - ./360jiagu
       artifacts:
         paths:
           - app/build/outputs/apk/release/
           - app/build/outputs/apk/releaseVivo/
           - app/build/outputs/mapping/release/
       tags:
         - android
    
     #endregion
     
     ...
    
    
  4. Modify the reinforcement command to support simultaneous reinforcement of apk compiled by releaseVivo

     #!/usr/bin/env bash
    
     ################
     #360 Reinforcement profile #
     ################
     
     ...
     
     APK_NAME=release_com.wln100.future_${CI_COMMIT_TAG}_${CI_PIPELINE_ID}.apk
     APK_PATH=app/build/outputs/apk/release/${APK_NAME}   #apk path requiring reinforcement
     DEST=app/build/outputs/apk/release/  #Output reinforcement package path
     
     # vivo installation package reinforcement path configuration
     VIVO_APK_NAME=releaseVivo_com.wln100.future_${CI_COMMIT_TAG}_${CI_PIPELINE_ID}.apk
     VIVO_APK_PATH=app/build/outputs/apk/releaseVivo/${VIVO_APK_NAME}   #apk path requiring reinforcement
     VIVO_DEST=app/build/outputs/apk/releaseVivo/  #Output reinforcement package path
     
     ...
     
     echo "------ running! ------"
     
     chmod +x ${JAVACMD}
     ${JAVACMD} -jar ${BASE} -version
     ${JAVACMD} -jar ${BASE} -login ${NAME} ${PASSWORD}
     ${JAVACMD} -jar ${BASE} -importsign ${KEYSTORE_PATH} ${KEY_PASSWORD} ${KEY_ALIAS} ${STORE_PASSWORD} # Configure signature information
     ${JAVACMD} -jar ${BASE} -showsign
     ${JAVACMD} -jar ${BASE} -deletemulpkg # Clear configured channel information
     ${JAVACMD} -jar ${BASE} -importmulpkg ./channels.txt # Configure channel information
     ${JAVACMD} -jar ${BASE} -showmulpkg
     ${JAVACMD} -jar ${BASE} -showconfig
     ${JAVACMD} -jar ${BASE} -jiagu ${APK_PATH} ${DEST} -autosign -automulpkg
     
     echo "------ Start reinforcement vivo channel apk ------"
     ${JAVACMD} -jar ${BASE} -deletemulpkg # Clear configured channel information
     ${JAVACMD} -jar ${BASE} -showconfig
     ${JAVACMD} -jar ${BASE} -jiagu ${VIVO_APK_PATH} ${VIVO_DEST} -autosign
     
     echo "------ finished! ------"
     
    

After the above configuration, verify the following results with the label of the packaged official version:

Q&A

Confused? Incomplete code?

Looking back at the series of articles on continuous integration, the code is complete, and the repeated ones are not added in this article

gradle compilation encountered OOM?

Two compilation tasks are performed asynchronously at the same time, which requires more memory. Increase the jvm memory configuration

# In project gradle Modify the configuration in properties  
...
# My setting is, initial 2g, maximum 4g  
org.gradle.jvmargs=-Xms2048M -Xmx4096m -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8
...

ERROR: Uploading artifacts to coordinator... too large archive

After the architecture is divided, the output apk has doubled. Originally, one apk has 6 channel packages, a total of 7 apks. Now 32-bit and 64 bit have multiple channels respectively, with 14 apks. The final output file size can be imagined. The maximum size of gitlab output file I set is 300m, which needs to be increased:

Some ideas

Recently, aab(Android App Bundle) is also gradually heating up, and the google store outside the wall has asked for mandatory support. The stores inside the wall only seem to support uploading apps in aab format. In addition, when it comes to continuous integration, it may be necessary to output aab format files during release compilation and apk files during debug compilation in the future; How the domestic reinforcement tools support the reinforcement of aab files has not been studied (or the aab files actually do not need reinforcement?)