1 Purpose
Use qemu to build arm simulator under CentOS 7.
Reference blog:
Using qemu to build arm simulator under centos - cold water Sitian blog Park (cnblogs.com)
2 Environment
CentOS 7 version: Linux version 3.10 0-693. el7. x86_ 64 ( builder@kbuilder.dev.centos.org ) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Aug 22 21:09:27 UTC 2017
It involves the use of the following tools:
Arm cross compiler: arm none Linux gnueabi GCC version 4.8 three
QEMU: qemu-2.11.0
uboot: u-boot-2018.01
Linux kernel: linux-4.1 thirty-eight
busybox: busybox-1.24.1
3. Arm cross tool installation
The version I downloaded is: arm-2014.05-29-arm-none-linux-gnueabi-i686-pc-linux-gnu. It can be used after direct decompression without installation.
Download address:
Arm none Linux gnueabi GCC download address code world (codetd.com)
Or: Linaro Releases
Installation steps:
1. Decompression:
tar -vxf arm-2014.05-29-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
I put it in the / usr/local/arm path.
2. Edit vim /etc/profile
Add at the end of the document:
export PATH=$PATH:/usr/local/arm/arm-2014.05/bin
Note that the equal sign "=" above is English, not Chinese, otherwise the following errors will occur.
-bash: export: `PATH＝/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/usr/local/arm/arm-2014-05/bin': not a valid identifier
3. Execute
source /etc/profile
4. Testing
Check the version information. Some problems are encountered here. The records are as follows:
$ arm-none-linux-gnueabi-gcc -v -bash: /usr/local/arm/arm-2014-05/bin/arm-none-linux-gnueabi-gcc: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory
The prompt is missing. The so format is wrong. According to netizens, install glibc After i686:
$ sudo yum install glibc.i686
View version information:
$ arm-none-linux-gnueabi-gcc -v ... ... Thread model: posix gcc version 4.8.3 20140320 (prerelease) (Sourcery CodeBench Lite 2014.05-29)
As you can see, the installed version is GCC version 4.8 3 20140320
4. Install QEMU
Download address:
Index of / (qemu.org) , I downloaded 2.11 Version 0.
Installation steps:
1. Decompression:
tar -vxf qemu-2.11.0.tar.xz
2. Configure compilation options:
./configure --enable-kvm --enable-debug --enable-vnc --enable-werror --target-list=arm-softmmu --audio-drv-list=
configure script is used to generate Makefile, and its options can be used/ configure --help view. The meanings of the options used here are as follows:
--Enable KVM: compile the KVM module so that QEMU can use KVM to access the virtualization services provided by the hardware.
--Enable VNC: enables VNC.
--Enalbe werror: at compile time, all warnings are treated as errors.
--Target list: select the architecture of the target machine. By default, all schemas are compiled, but in order to complete the compilation faster, you can specify the required schema.
Some errors occurred in this step:
$ ./configure --enable-kvm --enable-debug --enable-vnc --enable-werror --target-list=arm-softmmu --audio-drv-list=
ERROR: zlib check failed
Make sure to have the zlib libs and headers installed.
The zlib development package is missing. You need to install the corresponding version:
$ yum install ghc-zlib-devel.x86_64
Re execute the configuration, and an error is reported:
$ ./configure --enable-kvm --enable-debug --enable-vnc --enable-werror --target-list=arm-softmmu --audio-drv-list= ERROR: glib-2.22 gthread-2.0 is required to compile QEMU
glib-2.22 library needs to be installed:
$ yum install glib2-devel.x86_64
Re execute the configuration, and an error is reported:
$ ./configure --enable-kvm --enable-debug --enable-vnc --enable-werror --target-list=arm-softmmu --audio-drv-list=
ERROR: pixman >= 0.21.8 not present.
Please install the pixman devel package.
pixman needs to be installed:
$ yum install pixman-devel.x86_64
It was executed again and finally succeeded.
3. Compile
$ make -j8
4. Installation
$ make install ... ... install -d -m 0755 "/usr/local/bin" install -c -m 0755 qemu-system-arm "/usr/local/bin"
5 install uboot
Download address:
Index of /pub/u-boot/ (denx.de)
I downloaded: u-boot-2018.01 tar. bz2
Installation steps:
1. Decompression:
tar jvxf u-boot-2018.01.tar.bz2
2. Enter the uboot source file directory and execute:
export ARCH=arm export CROSS_COMPILE=arm-none-linux-gnueabi- make vexpress_ca9x4_defconfig make
3. You will be prompted that the gcc version is too low:
$ make scripts/kconfig/conf --silentoldconfig Kconfig CHK include/config.h CFG u-boot.cfg GEN include/autoconf.mk GEN include/autoconf.mk.dep *** Your GCC is older than 6.0 and is not supported
You can choose a higher version of GCC to reinstall, or modify the configuration of the uboot source file.
Here, choose to modify the configuration of the uboot source file:
$ vim arch/arm/config.mk
Comment out 65 lines.
4. Compile
Execute make directly:
$ make
After compilation, if the u-boot file is generated in the directory, the compilation is successful.
5. Write the script run. Exe in the U-Boot source directory sh
qemu-system-arm \
-M vexpress-a9 \
-nographic \
-m 512M \
-kernel u-boot
Then Chmod + X run SH add file execution permission.
6. Final execution/ run.sh
You can see that bootloader is started, but you will be prompted that there is no image file.
Wrong Image Format for bootm command ERROR: can't get kernel image!
6 compiling kernel
Download address:
Index of /pub/linux/kernel/v4.x/
I downloaded linux-4.1 38.
Compilation steps:
1. Find the configuration file vexpress that needs to compile the kernel from arch / arm / configurations_ Defconfig, copy to the root directory of the source code.
$ cp arch/arm/configs/vexpress_defconfig ./
2. Execute command
export ARCH=arm export CROSS_COMPILE=arm-none-linux-gnueabi- make vexpress_defconfig make zImage make modules make dtbs
3. If the zImage file is generated in the arch/arm/boot directory after compilation, the compilation is successful.
$ ls arch/arm/boot/ bootp/ compressed/ dts/ Image* install.sh Makefile zImage*
At this time, you can directly use QEMU to test whether the kernel is normal:
$ qemu-system-arm -M vexpress-a9 -m 512M -kernel ./linux-4.1.38/arch/arm/boot/zImage -dtb ./linux-4.1.38/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "console=ttyAMA0"
Here are the parameters of the qemu command:
-M vexpress-a9 simulation vexpress-a9 Veneer, you can use-M ?Parameter to get the qemu All boards supported by version -m 512M Board running physical memory 512 M -kernel ./linux-4.1.38/arch/arm/boot/zImage tell qemu Running kernel image path of board -nographic No graphical interface is used, only serial port is used -append "console=ttyAMA0" Kernel startup parameters, which tell the kernel here vexpress What is the serial port device for single board operation tty.
Seeing the kernel print indicates that the kernel is normal, but there is no file system at this time, and the kernel can't get up.
At this time, you can only exit by killing the QEMU system arm process.
ps -A | grep qemu-system-arm | awk '{print $1}' | xargs sudo kill -9
7 compiling busybox
Download address:
Index of /downloads (busybox.net)
I downloaded: busybox-1.24 1.tar. bz2.
Operation steps:
1. Enter the Busybox source file directory and execute make menuconfig.
$ cd busybox-1.24.1/ $ make menuconfig
Select Build BusyBox as a static binary (no shared libs).
2. Execute make and make install
$ make LINK busybox_unstripped Static linking against glibc, can't use --gc-sections Trying libraries: crypt m Library crypt is not needed, excluding it Library m is needed, can't exclude it (yet) Final link with: m DOC busybox.pod DOC BusyBox.txt DOC busybox.1 DOC BusyBox.html
$ make install -------------------------------------------------- You will probably need to make your busybox binary setuid root to ensure all configured applets will work properly. --------------------------------------------------
_ Generate the required file bin linuxrc sbin usr in the install directory.
$ ls _install/ bin/ linuxrc@ sbin/ usr/
The program in this directory is the command required for the board to run.
8 make root file system
The root file system produced uses ext3 file format.
First, a directory structure is formed in the Centos7 host environment. The files and directories stored in it are exactly the same as the directory structure required for running on the board, and then packaged into an image (SD card in the view of the development board). This temporary directory structure is called the root directory.
Operation steps:
1. Create the rootfs directory (as the root directory) and put all the files in the root file system here.
sudo mkdir -p rootfs/proc/ sudo mkdir -p rootfs/sys/ sudo mkdir -p rootfs/tmp/ sudo mkdir -p rootfs/root/ sudo mkdir -p rootfs/var/ sudo mkdir -p rootfs/mnt/
sudo cp etc rootfs/ -arf
Download address of etc Directory: https://files.cnblogs.com/files/pengdonglin137/etc.tar.gz
2. Copy the busybox command to the root directory
cp ../qemu/busybox/busybox-1.24.1/_install/* -r ./
3. Copy the runtime from the tool chain to the lib directory
cp -arf /usr/local/arm/arm-2014-05/arm-none-linux-gnueabi/lib rootfs/ rm rootfs/lib/*.a arm-none-linux-gnueabi-strip rootfs/lib/*
4. Create 4 tty end devices
sudo mknod rootfs/dev/tty1 c 4 1 sudo mknod rootfs/dev/tty2 c 4 2 sudo mknod rootfs/dev/tty3 c 4 3 sudo mknod rootfs/dev/tty4 c 4 4
5. Generate 32M image
dd if=/dev/zero of=a9rootfs.ext3 bs=1M count=32
6. Format to ext3 file system
mkfs.ext3 a9rootfs.ext3
7. Copy files to image
mkdir tmpfs mount -t ext3 a9rootfs.ext3 tmpfs/ -o loop cp -r rootfs/* tmpfs/ umount tmpfs
8. Run
qemu-system-arm -M vexpress-a9 -m 512M -kernel /home/qemu/kernel/linux-4.1.38/arch/arm/boot/zImage -dtb /home/qemu/kernel/linux-4.1.38/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 console=ttyAMA0" -sd a9rootfs.ext3
Complete startup log:
$ qemu-system-arm -M vexpress-a9 -m 512M -kernel /home/qemu/kernel/linux-4.1.38/arch/arm/boot/zImage -dtb /home/qemu/kernel/linux-4.1.38/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 rw console=ttyAMA0" -sd a9rootfs.ext3
WARNING: Image format was not specified for 'a9rootfs.ext3' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Booting Linux on physical CPU 0x0
Initializing cgroup subsys cpuset
Linux version 4.1.38 (root@localhost.localdomain) (gcc version 4.8.3 20140320 (prerelease) (Sourcery CodeBench Lite 2014.05-29) ) #1 SMP Sun Dec 26 16:34:51 CST 2021
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine model: V2P-CA9
Memory policy: Data cache writeback
CPU: All CPU(s) started in SVC mode.
PERCPU: Embedded 11 pages/cpu @9fbc0000 s16192 r8192 d20672 u45056
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048
Kernel command line: root=/dev/mmcblk0 rw console=ttyAMA0
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 12288 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 32768 bytes
early log buf free: 14916(91%)
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 512652K/524288K available (4849K kernel code, 154K rwdata, 1376K rodata, 260K init, 148K bss, 11636K reserved, 0K cma-reserved)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xa0800000 - 0xff000000 (1512 MB)
lowmem : 0x80000000 - 0xa0000000 ( 512 MB)
modules : 0x7f000000 - 0x80000000 ( 16 MB)
.text : 0x80008000 - 0x8061c6a8 (6226 kB)
.init : 0x8061d000 - 0x8065e000 ( 260 kB)
.data : 0x8065e000 - 0x80684a00 ( 155 kB)
.bss : 0x80687000 - 0x806ac1f8 ( 149 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Hierarchical RCU implementation.
Additional per-CPU info printed with stalls.
RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS:16 nr_irqs:16 16
GIC CPU mask not found - kernel will fail to boot.
GIC CPU mask not found - kernel will fail to boot.
L2C: platform modifies aux control register: 0x02020000 -> 0x02420000
L2C: DT/platform modifies aux control register: 0x02020000 -> 0x02420000
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 dynamic clock gating disabled, standby mode disabled
L2C-310 cache controller enabled, 8 ways, 128 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x46420001
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
clocksource arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
Console: colour dummy device 80x30
Calibrating delay loop... 400.58 BogoMIPS (lpj=2002944)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x60008280 - 0x600082d8
Brought up 1 CPUs
SMP: Total of 1 processors activated (400.58 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 0
clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor ladder
cpuidle: using governor menu
of_amba_device_create(): amba_device_add() failed (-19) for /memory-controller@100e0000
of_amba_device_create(): amba_device_add() failed (-19) for /memory-controller@100e1000
of_amba_device_create(): amba_device_add() failed (-19) for /watchdog@100e5000
of_amba_device_create(): amba_device_add() failed (-19) for /smb/motherboard/iofpga@7,00000000/sysctl@01000
of_amba_device_create(): amba_device_add() failed (-19) for /smb/motherboard/iofpga@7,00000000/wdt@0f000
hw-breakpoint: debug architecture 0x4 unsupported.
Serial: AMBA PL011 UART driver
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 38, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
1000a000.uart: ttyAMA1 at MMIO 0x1000a000 (irq = 39, base_baud = 0) is a PL011 rev1
1000b000.uart: ttyAMA2 at MMIO 0x1000b000 (irq = 40, base_baud = 0) is a PL011 rev1
1000c000.uart: ttyAMA3 at MMIO 0x1000c000 (irq = 41, base_baud = 0) is a PL011 rev1
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
Switched to clocksource arm,sp804
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 1 counters available
futex hash table entries: 1024 (order: 4, 65536 bytes)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
9p: Installing v9fs 9p2000 file system support
io scheduler noop registered (default)
clcd-pl11x 10020000.clcd: PL111 rev2 at 0x10020000
clcd-pl11x 10020000.clcd: /clcd@10020000 hardware, 1024x768@59 display
Console: switching to colour frame buffer device 128x48
clcd-pl11x 1001f000.clcd: PL111 rev2 at 0x1001f000
clcd-pl11x 1001f000.clcd: /smb/motherboard/iofpga@7,00000000/clcd@1f000 hardware, 640x480@59 display
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
libphy: smsc911x-mdio: probed
smsc911x 4e000000.ethernet eth0: attached PHY driver [Generic PHY] (mii_bus:phy_addr=4e000000.etherne:01, irq=-1)
smsc911x 4e000000.ethernet eth0: MAC Address: 52:54:00:12:34:56
isp1760 4f000000.usb: bus width: 32, oc: digital
isp1760 4f000000.usb: NXP ISP1760 USB Host Controller
isp1760 4f000000.usb: new USB bus registered, assigned bus number 1
isp1760 4f000000.usb: Scratch test failed.
isp1760 4f000000.usb: can't setup: -19
isp1760 4f000000.usb: USB bus 1 deregistered
usbcore: registered new interface driver usb-storage
mousedev: PS/2 mouse device common for all mice
rtc-pl031 10017000.rtc: rtc core: registered pl031 as rtc0
mmci-pl18x 10005000.mmci: Got CD GPIO
mmci-pl18x 10005000.mmci: Got WP GPIO
mmci-pl18x 10005000.mmci: No vqmmc regulator found
mmci-pl18x 10005000.mmci: mmc0: PL181 manf 41 rev0 at 0x10005000 irq 34,35 (pio)
input: AT Raw Set 2 keyboard as /devices/platform/smb/smb:motherboard/smb:motherboard:iofpga@7,00000000/10006000.kmi/serio0/input/input0
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
mmc0: new SD card at address 4567
mmcblk0: mmc0:4567 QEMU! 32.0 MiB
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
aaci-pl041 10004000.aaci: FIFO 512 entries
oprofile: using arm/armv7-ca9
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
rtc-pl031 10017000.rtc: setting system clock to 2021-12-26 13:20:20 UTC (1640524820)
ALSA device list:
#0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 33
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb/smb:motherboard/smb:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
kjournald starting. Commit interval 5 seconds
EXT3-fs (mmcblk0): using internal journal
EXT3-fs (mmcblk0): mounted filesystem with writeback data mode
VFS: Mounted root (ext3 filesystem) on device 179:0.
Freeing unused kernel memory: 260K (8061d000 - 8065e000)
random: nonblocking pool is initialized
Please press Enter to activate this console.