Root CNode Inaccessible After seL4_TCB_SetSpace (Guard 0, Radix 12)

New to seL4
1

Hello,

I am currently trying to change the cspace of my sel4 root task so that the root cnode only occupies 12 of my available 64 bits.
This works to some degree. I use seL4_TCB_SetSpace to change the guard size to 0. I do this by setting a guard value of 1 and a guard size of 0 with the seL4_CNode_CapData struct. I figured this would be the way, because otherwise the seL4_CNode_CapData would be 0 and the manual states:

If set to zero, this parameter has no effect

After this, I can use the capability to the own TCB of the root task, shift its CPtr bits by 64 - 12 = 52 and do a seL4_TCB_ReadRegisters to obtain an seL4_IllegalOperation and a print on the console:

TCB ReadRegisters: Attempted to read our own registers.

This means that shifting the bits of the CPtr by 52 works as expected after doing our seL4_TCB_SetSpace call.

The problem is now that the shifted CPtr to the root cnode does not seem to work.
When I use seL4_Untyped_Retype with a correctly shifted untyped cap, it produces a kernel panic with the following message on the console:

[handleInvocation/298 T0x807ffc9400 “rootserver” @2055b4]: Lookup of extra caps failed.

The only way this could fail is when the root argument of the seL4_Untyped_Retype call could not be resolved.

Below I added my code. I currently use rust-sel4 for this, which could be the culprit. For easier understanding, I’ve added the parameter names stated in the manual to each call

#[root_task(heap_size = 1024 * 64)]
fn main(bootinfo: &sel4::BootInfoPtr) -> ! {
    sel4::debug_println!("In root task");

    let null = sel4::init_thread::slot::NULL.cap();
    let cnode = sel4::init_thread::slot::CNODE.cap();
    let vspace = sel4::init_thread::slot::VSPACE.cap();
    let tcb = sel4::init_thread::slot::TCB.cap();

    tcb.tcb_set_space(
        null.cptr(),             // fault_ep
        cnode,                   // cspace_root
        CNodeCapData::new(1, 0), // cspace_root_data: guard value = 1, guard size = 0
        vspace,                  // vspace_root
    )
    .unwrap();

    const RADIX_BITS: u8 = 12;
    const SHIFT: u8 = WORD_SIZE as u8 - RADIX_BITS;

    // Sanity check
    let tcb = sel4::cap::Tcb::from_bits(tcb.bits() << SHIFT);
    let registers = tcb.tcb_read_registers(false, 1);
    // Debug prints "Attempted to read our own registers."
    assert_eq!(sel4::Error::IllegalOperation, registers.unwrap_err());

    let largest_ut = find_largest_kernel_untyped(bootinfo);
    let largest_ut = sel4::cap::Untyped::from_bits(largest_ut.bits() << SHIFT);

    // Shift original cnode CPtr by the same amount we shifted for the tcb sanity check
    let cnode = sel4::cap::CNode::from_bits(cnode.bits() << SHIFT);

    let dest = AbsoluteCPtr::new(
        cnode, // root
        CPtrWithDepth::from_bits_with_depth(
            0, // node_index
            0, // node_offset
        ),
    );
    largest_ut.untyped_retype(
        &sel4::cap_type::VSpace::object_blueprint(), // type & size_bits
        &dest,                                       // root & node_index & node_depth
        200,                                         // node_offset
        1,                                           // num_objects
    );

    unreachable!();
}

EDIT:
Here is the qemu output of this program:

In root task
<<seL4(CPU 0) [decodeReadRegisters/984 T0x807ffc9400 "rootserver" @20574c]: TCB ReadRegisters: Attempted to read our own registers.>>
<<seL4(CPU 0) [handleInvocation/298 T0x807ffc9400 "rootserver" @2055b4]: Lookup of extra caps failed.>>
Found thread has no fault handler while trying to handle:
cap fault in send phase at address 0x20000000000000
in thread 0x807ffc9400 "rootserver" at address 0x2055b4
With stack:
0x21a490: 0x0
[...]
2

Yes, the work-around of setting guard to something for zero size is unfortunately needed.

Interesting, it turns out I was wrong in thinking caps need to be fully resolvable for syscalls to work. handleInvocation uses resolveAddressBits, which doesn’t return an error if the bits used is less than the requested number of bits to resolve, but only for caps that are not CNodes. It does note the number of unresolved, remaining bits, so the caller can still check and error out if it wants to. But it are only CNode operations that actually require the full word size bits to be resolved, because if there are remaining bits, lookup will continue into the sub-CNode.

So that answers your question: Using non-full-word addressable caps works only for things that are not CNodes. The lookup for the extra caps fails because one of the extra caps is the destination CNode where the cap to the newly created object will be placed in.

Edit: Section 3.3.1 Capability Address Lookup of the manual only talks about the case where lookup succeed, it doesn’t mention when it fails. With that in mind it’s mostly correct, but it should be improved to make clear when a lookup actually fails.

1 Like
3

Thank you, that helps a whole lot and makes sense.
So CNodes are not “partially” addressable, because then the Kernel wouldn’t know if I meant the CNode itself or the capability at slot position 0 inside the CNode

This means that if I want to address the root node, after I changed the guard to 0, I would need to mutate the CNode reference at slot position 2 to a reference with the guard size 40, so that I can use CPtr (2 << 52) | 2 to address itself. Because if I remember correctly, you once wrote somewhere:

guard bits are not a property of the CNode, but a part of the
reference to a CNode

I think you just made me understand the whole CSpace/CNode concept.
Thanks a lot!

4

You’re welcome! And I learned something new too.

So CNodes are not “partially” addressable, because then the Kernel wouldn’t know if I meant the CNode itself or the capability at slot position 0 inside the CNode

Yes, the code tries to keep going into every CNode.

This means that if I want to address the root node, after I changed the guard to 0, I would need to mutate the CNode reference at slot position 2 to a reference with the guard size 40, so that I can use CPtr (2 << 52) | 2 to address itself.

That would (almost) work, but I think would be unnecessarily complicated, as you would do cnode[cnode[cnode]]], while all you need is cnode[cnode]. If you use a guard of 64 - 12 = 52 bits you could address the cnode more directly with CPtr 2 << 52 (and the guard value will be the lowest bits). But checking the code, this will throw a depth mismatch error, so sadly you have to do it the more complicated way you do. resolveAddressBits could have easily checked whether we ended up at a guard boundary and return the last CNode if we did instead of throwing a stupid depth mismatch fault.

However, you have a chicken and egg problem here, you can’t change the CNode guards of the TCB and the CNode cap to itself in the CNode both at once. So you probably need to mint a copy of the root CNode with the right guard size and use that after the switch, instead of the original cap.