Skip to content

Conversation

@bjia56
Copy link
Contributor

@bjia56 bjia56 commented Mar 13, 2025

NetBSD's mprotect seems to be more restrictive than other OSes, where it does not allow less restrictive mappings than the original mmap, and can be subject to PaX restrictions. According to NetBSD documentation, there is a PROT_MPROTECT macro to define allowed protections for later uses of mprotect, without granting the permissions immediately in mmap. This can be used to ensure that the full range of protections blink could make in the course of execution are permitted.

NetBSD's mprotect seems to be more restrictive than other OSes, where it
does not allow less restrictive mappings than the original mmap, and can
be subject to PaX restrictions. According to NetBSD documentation, there
is a PROT_MPROTECT macro to define allowed protections for later uses of
mprotect, without granting the permissions immediately in mmap. This can
be used to ensure that the full range of protections blink could make in
the course of execution are permitted.
@jart
Copy link
Owner

jart commented Mar 13, 2025

What NetBSD architecture? I don't get this behavior on x86-64.

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

I get this on both x86_64 and aarch64, NetBSD 10.0 running in VMs

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

My setup:

vmactions# uname -a 
NetBSD vmactions.org 10.0 NetBSD 10.0 (GENERIC) #0: Thu Mar 28 08:33:33 UTC 2024  [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC amd64

Building the latest code from master with ./configure --static, then running:

vmactions# o/blink/blink third_party/cosmo/tinyhello.elf 
assertion failed
blink/loader.c:71:3146 assertion failed: !ProtectVirtual(m->system, base, vaddr + amt - base, PROT_READ | PROT_WRITE, false) (ENOBUFS)
         PC 400078 segfault
         AX 0000000000000000  CX 0000000000000000  DX 0000000000000000  BX 0000000000000000
         SP 0000000000000000  BP 0000000000000000  SI 0000000000000000  DI 0000000000000000
         R8 0000000000000000  R9 0000000000000000 R10 0000000000000000 R11 0000000000000000
        R12 0000000000000000 R13 0000000000000000 R14 0000000000000000 R15 0000000000000000
         FS 0000000000000000  GS 0000000000000000 OPS 0                FLG ......
        third_party/cosmo/tinyhello.elf
        000000000000 000000400078 UNKNOWN
        <blink backtrace unavailable>
tlb_misses                       = 1
tlb_resets                       = 1
[1]   Abort trap (core dumped) o/blink/blink third_party/cosmo/tinyhello.elf

@jart
Copy link
Owner

jart commented Mar 13, 2025

Try writing a C program locally on the system to confirm. Something like:

TEST(mprotect, loosenRestriction) {
  char *p;
  ASSERT_NE(MAP_FAILED, (p = mmap(0, 1, PROT_READ | PROT_WRITE,
                                  MAP_ANONYMOUS | MAP_PRIVATE, -1, 0)));
  ASSERT_SYS(0, 0, mprotect(p, 1, PROT_READ | PROT_EXEC));
  ASSERT_SYS(0, 0, munmap(p, 1));
}

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

I converted the test into a standalone program:

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>

int main(void) {
    char *p = mmap(0, 1, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (p == MAP_FAILED) {
        perror("mmap failed");
        return EXIT_FAILURE;
    }

    if (mprotect(p, 1, PROT_READ | PROT_EXEC) != 0) {
        perror("mprotect failed");
        munmap(p, 1);
        return EXIT_FAILURE;
    }

    if (munmap(p, 1) != 0) {
        perror("munmap failed");
        return EXIT_FAILURE;
    }

    return EXIT_SUCCESS;
}

Then:

vmactions# cc test.c
vmactions# ./a.out 
mprotect failed: Permission denied

Changing to the following line allows this to succeed:

char *p = mmap(0, 1, PROT_READ | PROT_WRITE | PROT_MPROTECT(PROT_EXEC), MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

This is probably relevant:

vmactions# sysctl security.pax.mprotect.global
security.pax.mprotect.global = 1

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

Disabling PaX mprotect allows the original test to succeed

@jart
Copy link
Owner

jart commented Mar 13, 2025

Is PaX enabled by default in the stock installation these days?

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

According to NetBSD 8.0's release notes, PaX mprotect is enabled by default on x86_64 and arm64: https://www.netbsd.org/releases/formal-8/NetBSD-8.0.html
I don't see any notes about PaX in subsequent release notes other than bugfixes.

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 13, 2025

It appears that PaX mprotect can be disabled on binaries with paxctl: https://man.netbsd.org/paxctl.8

@bjia56
Copy link
Contributor Author

bjia56 commented Mar 14, 2025

For my use case, running paxctl +m on the compiled blink binary before adding it with apelink is sufficient. I'll leave it up to you if you want this PR in or not.

@jart
Copy link
Owner

jart commented Mar 14, 2025

Thanks for the update. I'd rather not make it default to RWX memory on NetBSD because it has a WIN32-like memory restriction feature. Since there'll probably be another feature lurking in NetBSD somewhere that demands a W^X invariant. Appreciate you taking the time to send the change though! Glad you're unblocked.

@jart jart closed this Mar 14, 2025
@jart
Copy link
Owner

jart commented Mar 14, 2025

Actually on second thought, having just noticed the comment that PAX is enabled by default in the stock install, what would you recommend? Would it be possible to configure the Blink build system to set the flag on the binary automatically for our users?

@jart jart reopened this Mar 14, 2025
@bjia56
Copy link
Contributor Author

bjia56 commented Mar 14, 2025

I think a short-term fix would be to add paxctl +m to the build system so binaries work, however it's a bit sketchy to disable a security feature on Blink entirely (since it could, as a VM, be used to run untrusted x86_64 binaries). A more permanent fix could be to identify the problematic parts and use mremap with MAP_REMAPDUP to duplicate mappings when more permissions are required. Unfortunately, I am not as familiar with Blink internals at this point to make the latter change.

Copy link
Owner

@jart jart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that we should respect the security blankets each platform
puts in place. This change does a better job at that, since it disables
PAX surgically, whereas paxctl -m would disable PAX systemically.

since it could, as a VM, be used to run untrusted x86_64 binaries

Stock open source Blink shouldn't be thought of as a security layer for
untrusted binaries. Blink aims to ensure programs work above all, so
that people have a good experience using the software. That means Blink
lets your programs do what they want to do. However Blink is designed
with the security use case in mind. It should take minimal eng effort to
modify blink/syscall.c to enforce whatever security policy you want. If
you do that, then Blink can potentially offer you a much stronger layer
of hardware/system isolation than alternatives like KVM, gVisor, etc.
Especially if you disable JIT and have a SECCOMP policy too. We also
really should add soft floating point to make it even more extreme.

@jart jart merged commit 4980d2c into jart:master Mar 14, 2025
2 checks passed
@bjia56 bjia56 deleted the netbsd-mprotect branch March 14, 2025 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants