I have used maaxboard bsp in amother armv8 cortex-a53 processor (same as maaxboard) & facing below issue specially related to mmu:
In the elfloader crt0.s file if i add the below code to disable mmu
MRS x3, SCTLR_EL1
BIC x3, x3, (1<<0)
BIC x3, x3, (1<<2)
BIC x3, x3, (1<<12)
MSR SCTLR_EL1, x3
ISB
In the maaxboard it works fine, but the other armv8 the execution stops for a while (few sec) and then continues.
The the “kernel/src/arch/arm/64/kernel/vspace.c” activete_kernel_vspace()
I have to comment below function to boot kernel in the other armv8 cortex-a53 h/w as:
BOOT_CODE void activate_kernel_vspace(void)
{
cleanInvalidateL1Caches();
setCurrentKernelVSpaceRoot(ttbr_new(0, addrFromKPPtr(armKSGlobalKernelPGD)));
/* Prevent elf-loader address translation to fill up TLB */
//setCurrentUserVSpaceRoot(ttbr_new(0, addrFromKPPtr(armKSGlobalUserVSpace)));
invalidateLocalTLB();
lockTLBEntry(KERNEL_ELF_BASE);
}
Because InvalidateLocalTLB() gets hanged. After commenting the above function kernel boots completely.
When the user space “setCurrentUserVSpaceRoot()” is called by the root process multiple times to update the pagetable. During the process the system gets hanged abruptly. MSR(“ttbr0_el1”, ttbr.words[0]); is not stable and works just by chance for some time, but mostly hangs.
I am able to boot the system till the user land but mmu is not working properly with two observations
a) disabling mmu using MRS x3, SCTLR_EL1
BIC x3, x3, (1<<0) works but stops for several sec.
b) updating ttbr0_el1 is not stable & the system hangs abruptly.
Did any one faced the similar issue & can you please suggest, because i tried a lot of mmu related config change and nothing helped.
I tried it in the elfloader crt0.s in the _start function file before calling main(). This is just for experimental purpose to disable mmu and reenable it. Because “ttbr0_el1” update is causing problem in aarch64 cortex-a53 board, but the same code works fine in imx8 (ARM cortex-a53) board.
seL4 release for maaxboard works fine for mmu like update to “ttbr0_el1” on maaxboard but not on another cortex-a53 based NXP chipset having same ARM architecture. ?
What SoC are you using? Does your SoC implement FEAT_D128 and is it enabled perhaps? Which values of ttbr0_el1 give problems and which values work?
I highly recommend reverting all MMU disabling code and either debug this problem fully within Elfloader or in the seL4 kernel, but not both. Also try using a non-SMP kernel to simplify debugging, if you haven’t already.
Edit: Also, what exception level are you running, EL1 or EL2? Try enabling or disabling hypervisor support to see if it makes a difference.
Disables the data and instruction caches. I don’t think that’s safe to do without cleaning all caches first.
Anyway, your problem seems to be unrelated to seL4 as such, as any update to ttbr0_el1 gives problems. You might be hitting some errata or there is a mismatch in expectations between what Elfloader/seL4 does and what your hardware expects. Is Linux running on your board?
My hardware is running Linux, issue is with seL4.
My h/w & maaxboard are same interms of ARM architecture (Cortex-a53) on maaxboard mmu works fine hence i cloned the same ARM settings and hence everythings works fine except the mmu.
There are 3 observations with respect to the mmu problem:
disabling mmu in elfloader crt0.s file in mmaxboard has no effect but in my board the program execution stops for few sec before proceeding with the boot.
In kernel after “after ttbr0_el1” is updated InvalidateLocalTLB() hangs
In user space update to “ttbr0_el1” abroptly hangs at any time. (unstable)
There is no Errata with respect to ARM on this issue. Hence the question is what is missed or not considered while writing the ARM specific code in seL4 for Cortex-a53/aarch64.
If you’re using the same SoC as the maaxboard board, your issue is most likely related to bootup. E.g. there is a bug in your board bring-up which corrupts some memory or system registers somewhere that later gives those MMU problems.
If you are using a different SoC that only also uses the Cortex-A53 CPU, you really can’t just use the maaxboard seL4 platform, you have to do a proper port of seL4.
NXP’s BSP and the Linux in there support all NXP hardware, so the maaxboard bsp working on your SoC doesn’t say that much.
To validate the same i want to run seL4 in HYPERVISOR mode.
What setting/sample project is need to build seL4 in ARM_HYPERWISOR mode such that different configuration of mmu shall be used in EL2.
For sel4test, all you need to do is enable ARM_HYP, you don’t need to actually run a VM or anything. It may not officially be supported for the maaxboard, but it should work.
Can you please hint as how the mode shall be switched to EL2 when compiled with -DARM_HYP=1 Because in maaxboard BSP i could not find the code to switch mode from EL1(default mode) to EL2. Because the function “is_hyp_mode()” is returning mode as EL1 in my case below boot log for reference-
ELF-loader started on CPU: ARM Ltd. Cortex-A53 r0p4
paddr=[80a7c000…80fc51bf]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive…found at 80bc3810.
Loaded DTB from 80bc3810.
paddr=[80248000…80254fff]
ELF-loading image ‘kernel’ to 80000000
paddr=[80000000…80247fff]
vaddr=[8080000000…8080247fff]
virt_entry=8080000000
ELF-loading image ‘sel4test-driver’ to 80255000
paddr=[80255000…8068bfff]
vaddr=[400000…836fff]
virt_entry=40f0b0
Enabling MMU and paging
Jumping to kernel-image entry point…
You probably need to modify U-Boot to stay in EL2. But if your BSP doesn’t support EL2 by default, I wouldn’t bother trying to get it to work and focus on fixing your original problem instead.
We have cloned the maaxboard BSP for our hardware, both being Cortex-a53 & made changes related to peripheral mapping. There is no arm specific changes, we expect arm specific changes should work the same way its working in maaxboard:
When i try to build maaxboard image and test it on maaxboard both EL1 & EL2 mode works properly, but in my hardware El1 mode have mmu related issue as explained in the previous thread & EL2 is not getting selected while reading “currentEL” register. The maaxboard build and run log is as below-
u-boot=> fatload mmc 0:1 0x40000000 sel4test-driver-image-arm-maaxboardinhyp
5739160 bytes read in 268 ms (20.4 MiB/s)
u-boot=> bootelf 0x40000000
Starting application at 0x40a81000 …
ELF-loader started on CPU: ARM Ltd. Cortex-A53 r0p4
paddr=[40a81000…40fde137]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive…found at 40be2d88.
Loaded DTB from 40be2d88.
paddr=[40246000…40250fff]
ELF-loading image ‘kernel’ to 40000000
paddr=[40000000…40245fff]
vaddr=[8040000000…8040245fff]
virt_entry=8040000000
ELF-loading image ‘sel4test-driver’ to 40251000
paddr=[40251000…40687fff]
vaddr=[400000…836fff]
virt_entry=40f0b0
Enabling hypervisor MMU and paging
Jumping to kernel-image entry point…
u-boot=> fatload mmc 0:1 0x40000000 sel4test-driver-image-arm-maaxboardinosmode
5620456 bytes read in 250 ms (21.4 MiB/s)
u-boot=> bootelf 0x40000000
Starting application at 0x40a77000 …
ELF-loader started on CPU: ARM Ltd. Cortex-A53 r0p4
paddr=[40a77000…40fb1137]
No DTB passed in from boot loader.
Looking for DTB in CPIO archive…found at 40bbb530.
Loaded DTB from 40bbb530.
paddr=[40241000…4024bfff]
ELF-loading image ‘kernel’ to 40000000
paddr=[40000000…40240fff]
vaddr=[ffffff8040000000…ffffff8040240fff]
virt_entry=ffffff8040000000
ELF-loading image ‘sel4test-driver’ to 4024c000
paddr=[4024c000…40682fff]
vaddr=[400000…836fff]
virt_entry=40f0b0
Enabling MMU and paging
Jumping to kernel-image entry point…
The question is which part of the code selects EL2/EL1 while compiling for maaxboard using the flag “-DARM_HYP=TRUE” because the bootloader for maaxboard is the same.
The question is which part of the code selects EL2/EL1 while compiling for maaxboard using the flag “-DARM_HYP=TRUE” because the bootloader for maaxboard is the same.
The bootloader determines the initial exception level, either EL1 or EL2. Elfloader drops down to EL1 if seL4 is compiled without hypervisor support and Elfloader is booted in EL2, otherwise it stays in EL2. You can’t change from EL1 to EL2, you can only drop exception levels. What exception level the bootloader runs at depends on either the hardware and/or the earlier board bootup code and configuration.
Considering your hardware is different, you want to use either the KernelCustomDTSOverlay or KernelCustomDTSOverlay options to modify the maaxboard’s DTS to match your actual hardware. Also double check whether all the other settings in src/plat/maaxboard/config.cmake are correct for your board (should be fine if you’re using an i.MX 8M).
From what you’ve told me, I think on your hardware Elfloader/seL4 is trying to use some device memory as normal memory, and that won’t work of course. It would explain the one second delay and other weird behaviour.