Actual combat wsl2 0 arm cross tool chain - step pit cross compilation incompatibility

Posted by chmpdog on Thu, 10 Feb 2022 11:08:30 +0100

Actual combat wsl2 0 arm cross tool chain - step pit cross compilation incompatibility

environment

Cross compiler: arm linux gnueabihf-4.9.4-2017.01-linaro is installed in VMware UBUNTU 18.04 x64 and WSL 2.0 UBUNTU 20.04 x64 (non docker environment). The installation package gcc-linaro-4.9.4-2017.01-x86 is downloaded from the official website_ 64_ arm-linux-gnueabihf. tar. xz
Target terminal: kernel and file system, which is gcc-linaro-4.9-2016.02-x86 under VMware UBUNTU 18.04 x64 platform_ 64_ arm-linux-gnueabihf. tar. XZ cross compiled

Problem description

In most cases, the target terminal looks normal, however, there are always a small number of machines that behave strangely
In the terminals with occasional problems, no abnormality was found at first. After careful analysis, it was found that there was a low probability of "CPU seemingly stuck"
During the simulation test, it was found that the user name and group of the installation package published in WSL 2.0 (non docker environment) after remote upgrade with root user were not root account
Add process memory and CPU running status tracking and monitoring code. Some terminals with problems will run out of memory every 1 hour

Problem solving

If you can't find the reason from the code analysis, try to start with the root account and see if it's a platform problem. Compiled and packaged from VMware UBUNTU 18.04 x64 platform, it was found that the root account problem disappeared miraculously
From VMware UBUNTU 18.04 x64 cross compilation and packaging release, the suspected "CPU jam" problem disappeared

Problem tracing

Delete the temporary file, clean the daemon in the cross compilation project, and then export the target file information compiled by VMware UBUNTU 18.04 x64 and WSL 2.0 UBUNTU 20.04 x64 from arm linux gnueabihf readelf. Some slight differences are found

VMware UBUNTU 18.04 x64 platform

Unwind table index '.ARM.exidx' at offset 0x12fc contains 8 entries:

0x10e68: 0x1 [cantunwind]

0x10f48: @0x112cc           // VMware UBUNTU 18.04 x64 only
  Compact model index: 1
  0x97      vsp = r7
  0x0d      vsp = vsp + 56
  0x84 0x08 pop {r7, r14}
  0xb0      finish
  0xb0      finish

0x10fb4: 0x80978008          // VMware UBUNTU 18.04 x64 only
  Compact model index: 0
  0x97      vsp = r7
  0x80 0x08 pop {r7}

0x10fcc: @0x112d8
  Compact model index: 1
  0x97      vsp = r7
  0x03      vsp = vsp + 16
  0x84 0x08 pop {r7, r14}
  0xb0      finish
  0xb0      finish

0x10ff8: @0x112e4
  Compact model index: 1
  0x97      vsp = r7
  0x0a      vsp = vsp + 44
  0x84 0x09 pop {r4, r7, r14}
  0xb0      finish
  0xb0      finish

0x1116c: @0x112f0          // VMware UBUNTU 18.04 x64 only
  Compact model index: 1
  0x97      vsp = r7
  0x01      vsp = vsp + 8
  0x84 0x08 pop {r7, r14}
  0xb0      finish
  0xb0      finish

0x111b4: 0x80978408          // VMware UBUNTU 18.04 x64 only
  Compact model index: 0
  0x97      vsp = r7
  0x84 0x08 pop {r7, r14}

0x111c4: 0x1 [cantunwind]


Symbol table '.dynsym' contains 54 entries:
  Num:    Value  Size Type    Bind   Vis      Ndx Name
    0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
    1: 0001133c     0 NOTYPE  LOCAL  DEFAULT   17 __exidx_end
    2: 000112fc     0 NOTYPE  LOCAL  DEFAULT   16 __exidx_start
    3: 00021528     4 OBJECT  GLOBAL DEFAULT   25 g_sys_res_ctor
    4: 00000000     0 FUNC    GLOBAL DEFAULT  UND sync@GLIBC_2.4 (2)
    5: 00000000     0 FUNC    GLOBAL DEFAULT  UND __aeabi_atexit@CXXABI_ARM_1.3.3 (3)
    6: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon25GetAndCaching
    7: 00021524     0 NOTYPE  GLOBAL DEFAULT   24 _edata
    8: 00021518     0 NOTYPE  WEAK   DEFAULT   24 data_start
    9: 00011210     4 OBJECT  GLOBAL DEFAULT   15 _IO_stdin_used
    10: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11GetInstanceEv
    11: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10LogManager9LogPrintf
    12: 00010ff9   370 FUNC    GLOBAL DEFAULT   13 main
    13: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_setschedpoli@GLIBC_2.4 (4)
    14: 00021530     0 NOTYPE  GLOBAL DEFAULT   25 __end__
    15: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon16CreateAllProc
    16: 00011208     0 FUNC    GLOBAL DEFAULT   14 _fini
    17: 00021530     0 NOTYPE  GLOBAL DEFAULT   25 __bss_end__
    18: 00000000     0 FUNC    GLOBAL DEFAULT  UND abort@GLIBC_2.4 (2)
    19: 00021530     0 NOTYPE  GLOBAL DEFAULT   25 _bss_end__
    20: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_getschedpoli@GLIBC_2.4 (4)
    21: 00000000     0 FUNC    GLOBAL DEFAULT  UND system@GLIBC_2.4 (4)
    22: 00010e69     0 FUNC    GLOBAL DEFAULT   13 _start
    23: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10SysResCtor5Sem_PE17T
    24: 00000000     0 FUNC    GLOBAL DEFAULT  UND _Z11DebugPrintfPKcz
    25: 00010d10     0 FUNC    GLOBAL DEFAULT   11 _init
    26: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
    27: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_destroy@GLIBC_2.4 (4)
    28: 00010fcd    42 FUNC    GLOBAL DEFAULT   13 _Z20MonitorProcessThreadP
    29: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11IsNoMemeoryEv
    30: 00021524     0 NOTYPE  GLOBAL DEFAULT   25 __bss_start__
    31: 00021518     0 NOTYPE  GLOBAL DEFAULT   24 __data_start
    32: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_init@GLIBC_2.4 (4)
    33: 00021530     0 NOTYPE  GLOBAL DEFAULT   25 _end
    34: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11InitAppStatEv
    35: 00021524     0 NOTYPE  GLOBAL DEFAULT   25 __bss_start
    36: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZNSt8ios_base4InitC1Ev@GLIBCXX_3.4 (5)
    37: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon8WatchDogEv
    38: 00000000     0 FUNC    GLOBAL DEFAULT  UND usleep@GLIBC_2.4 (2)
    39: 000111c5    64 FUNC    GLOBAL DEFAULT   13 __libc_csu_init
    40: 00000000     0 FUNC    GLOBAL DEFAULT  UND __aeabi_unwind_cpp_pr1@GCC_3.5 (6)
    41: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_create@GLIBC_2.4 (4)
    42: 00010fb5    24 FUNC    GLOBAL DEFAULT   13 _Z18VersionCodeDeclarev
    43: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon14MonitorProces
    44: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTab
    45: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_setschedpara@GLIBC_2.4 (4)
    46: 00010f49   108 FUNC    GLOBAL DEFAULT   13 SetThreadSchedPolicyRR
    47: 00011205     2 FUNC    GLOBAL DEFAULT   13 __libc_csu_fini
    48: 00000000     0 FUNC    GLOBAL DEFAULT  UND __aeabi_unwind_cpp_pr0@GCC_3.5 (6)          // VMware UBUNTU 18.04 x64 only
    49: 00000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.4 (2)
    50: 00000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
    51: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
    52: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10SysResCtor11GetInsta
    53: 00010e5c     0 FUNC    GLOBAL DEFAULT  UND _ZNSt8ios_base4InitD1Ev@GLIBCXX_3.4 (5)

Histogram for bucket list length (total of 37 buckets):
Length  Number     % of total  Coverage
      0  6          ( 16.2%)
      1  15         ( 40.5%)     28.3%
      2  11         ( 29.7%)     69.8%
      3  4          ( 10.8%)     92.5%
      4  1          (  2.7%)    100.0%

WSL 2.0 UBUNTU 20.04 x64 platform

Unwind table index '.ARM.exidx' at offset 0x12d8 contains 4 entries:

0x10e5c: 0x1 [cantunwind]

0x10fc0: @0x112c0
  Compact model index: 1
  0x97      vsp = r7
  0x03      vsp = vsp + 16
  0x84 0x08 pop {r7, r14}
  0xb0      finish
  0xb0      finish

0x10fec: @0x112cc
  Compact model index: 1
  0x97      vsp = r7
  0x0a      vsp = vsp + 44
  0x84 0x09 pop {r4, r7, r14}
  0xb0      finish
  0xb0      finish

0x11160: 0x1 [cantunwind]


Symbol table '.dynsym' contains 53 entries:
  Num:    Value  Size Type    Bind   Vis      Ndx Name
    0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
    1: 000112f8     0 NOTYPE  LOCAL  DEFAULT   17 __exidx_end
    2: 000112d8     0 NOTYPE  LOCAL  DEFAULT   16 __exidx_start
    3: 000214e4     4 OBJECT  GLOBAL DEFAULT   25 g_sys_res_ctor
    4: 00000000     0 FUNC    GLOBAL DEFAULT  UND sync@GLIBC_2.4 (2)
    5: 00000000     0 FUNC    GLOBAL DEFAULT  UND __aeabi_atexit@CXXABI_ARM_1.3.3 (3)
    6: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon25GetAndCaching
    7: 000214e0     0 NOTYPE  GLOBAL DEFAULT   24 _edata
    8: 000214d4     0 NOTYPE  WEAK   DEFAULT   24 data_start
    9: 00011204     4 OBJECT  GLOBAL DEFAULT   15 _IO_stdin_used
    10: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11GetInstanceEv
    11: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10LogManager9LogPrintf
    12: 00010fed   370 FUNC    GLOBAL DEFAULT   13 main
    13: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_setschedpoli@GLIBC_2.4 (4)
    14: 000214ec     0 NOTYPE  GLOBAL DEFAULT   25 __end__
    15: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon16CreateAllProc
    16: 000111fc     0 FUNC    GLOBAL DEFAULT   14 _fini
    17: 000214ec     0 NOTYPE  GLOBAL DEFAULT   25 __bss_end__
    18: 00000000     0 FUNC    GLOBAL DEFAULT  UND abort@GLIBC_2.4 (2)
    19: 000214ec     0 NOTYPE  GLOBAL DEFAULT   25 _bss_end__
    20: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_getschedpoli@GLIBC_2.4 (4)
    21: 00000000     0 FUNC    GLOBAL DEFAULT  UND system@GLIBC_2.4 (4)
    22: 00010e5d     0 FUNC    GLOBAL DEFAULT   13 _start
    23: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10SysResCtor5Sem_PE17T
    24: 00000000     0 FUNC    GLOBAL DEFAULT  UND _Z11DebugPrintfPKcz
    25: 00010d04     0 FUNC    GLOBAL DEFAULT   11 _init
    26: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
    27: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_destroy@GLIBC_2.4 (4)
    28: 00010fc1    42 FUNC    GLOBAL DEFAULT   13 _Z20MonitorProcessThreadP
    29: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11IsNoMemeoryEv
    30: 000214e0     0 NOTYPE  GLOBAL DEFAULT   25 __bss_start__
    31: 000214d4     0 NOTYPE  GLOBAL DEFAULT   24 __data_start
    32: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_init@GLIBC_2.4 (4)
    33: 000214ec     0 NOTYPE  GLOBAL DEFAULT   25 _end
    34: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon11InitAppStatEv
    35: 000214e0     0 NOTYPE  GLOBAL DEFAULT   25 __bss_start
    36: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZNSt8ios_base4InitC1Ev@GLIBCXX_3.4 (5)
    37: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon8WatchDogEv
    38: 00000000     0 FUNC    GLOBAL DEFAULT  UND usleep@GLIBC_2.4 (2)
    39: 000111b9    64 FUNC    GLOBAL DEFAULT   13 __libc_csu_init
    40: 00000000     0 FUNC    GLOBAL DEFAULT  UND __aeabi_unwind_cpp_pr1@GCC_3.5 (6)
    41: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_create@GLIBC_2.4 (4)
    42: 00010fa9    24 FUNC    GLOBAL DEFAULT   13 _Z18VersionCodeDeclarev
    43: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN6Daemon14MonitorProces
    44: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTab
    45: 00000000     0 FUNC    GLOBAL DEFAULT  UND pthread_attr_setschedpara@GLIBC_2.4 (4)
    46: 00010f3d   108 FUNC    GLOBAL DEFAULT   13 SetThreadSchedPolicyRR
    47: 000111f9     2 FUNC    GLOBAL DEFAULT   13 __libc_csu_fini
    48: 00000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.4 (2)
    49: 00000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
    50: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
    51: 00000000     0 FUNC    GLOBAL DEFAULT  UND _ZN10SysResCtor11GetInsta
    52: 00010e50     0 FUNC    GLOBAL DEFAULT  UND _ZNSt8ios_base4InitD1Ev@GLIBCXX_3.4 (5)

Histogram for bucket list length (total of 37 buckets):
Length  Number     % of total  Coverage
      0  6          ( 16.2%)
      1  16         ( 43.2%)     30.8%
      2  10         ( 27.0%)     69.2%
      3  4          ( 10.8%)     92.3%
      4  1          (  2.7%)    100.0%

In disassembly, in addition to the difference of offset address, there is no difference in the target file under the file comparator

Further testing

Compare the target file information of arm linux gnueabihf-4.9.4-2017.01-linaro cross compilation, disassembly and arm linux gnueabihf readelf under VMware UBUNTU 18.04 x64 and WSL 2.0 Docker UBUNTU 18.04 x64 platforms. Except for address offset, the other information is exactly the same

Summary

WSL 2.0 UBUNTU xx.xx environment is fast and easy to use. However, under special circumstances, cross compilation will lead to some problems. It is speculated that it may be related to the compatible transformation of MS to Linux platform.
It is recommended to publish the target files in the production environment and try to use the verified system platforms such as VMware Linux

Topics: Linux Docker IoT

Programmer Think

Actual combat wsl2 0 arm cross tool chain - step pit cross compilation incompatibility

Actual combat wsl2 0 arm cross tool chain - step pit cross compilation incompatibility

environment

Problem description

Problem solving

Problem tracing

Further testing

Summary

Hot Topics