[NTOS] Implement support for global pages #7937

tkreuzer · 2025-04-26T15:05:29Z

Purpose

This PR implements support for global pages (kernel pages that are not flushed out of the TLB on a process switch).

Proposed changes

[NDK][NTOS] Implement inline KeGetCurrentProcess
[NTOS:KE/x64] Make sure that KPROCESS::ActiveProcessors is set correctly
[NTOS:KE] Implement KiIpiSendRequest stub (later used for SMP TLB shootdown)
[NTOS::KE] Rewrite TLB flushing (to support SMP and global pages)
[NTOS:MM] Implement support for global pages

TODO (not addressed in this PR)

Make PDEs global
Optimize global VA state lookup with a bitmap

Testbot runs (Filled in by Devs)

This is important for properly flushing the TLB on SMP.

Copilot

Pull Request Overview

This PR implements support for global pages and updates the TLB flushing mechanism across the kernel. Key changes include:

Removal of the old inline global page setup in favor of a dedicated MiInitializeGlobalPages() call.
Replacement of legacy TLB flush calls (KeFlushCurrentTb, KeFlushProcessTb, etc.) with new Kx-prefixed functions.
Introduction of new IPI routines and header updates to support SMP and architectural differences.

Reviewed Changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
ntoskrnl/mm/ARM3/i386/init.c	Removes inline global page configuration and calls MiInitializeGlobalPages(). Adjusts TLB flush function.
ntoskrnl/mm/ARM3/hypermap.c	Updates TLB flushing calls to use KeFlushSingleTb / KeFlushRangeTb.
ntoskrnl/ke/tlbflush.c	Adds new implementations for TLB flushing routines supporting SMP.
ntoskrnl/ke/ipi.c and ntoskrnl/ke/amd64/ipi.c	Introduces stub and new implementations respectively for IPI requests.
Other files (cpu.c, freeze.c, stubs.c, headers)	Update flush functions calls and inline definitions to align with new APIs.
ntoskrnl/include/config.h	Enables global page support by defining GLOBAL_PAGES_AWESOME.

ntoskrnl/ke/ipi.c

ntoskrnl/ke/amd64/ipi.c

ntoskrnl/include/internal/ke_x.h

HBelusca · 2025-04-29T20:10:10Z

ntoskrnl/ke/ipi.c


+VOID
+NTAPI
+KiIpiSendRequest(


It looks like this new version supersedes the old one at line 75, while the function at line 75 appears to be more or less the implementation for KiIpiSend() (of line 36).

HBelusca · 2025-04-29T20:31:15Z

ntoskrnl/ke/amd64/kiinit.c

    KeActiveProcessors |= 1ULL << Cpu;

+    /* We are running the initial system process now */
+    InterlockedOr64(&KiInitialProcess.Pcb.ActiveProcessors, 1ULL << Cpu);


This looks like this is already done in KiInitializeHandBuiltThread (see line 134) already, no? (it's invoked either for CPU 0 byKiInitializeKernel() -- which in turn is invoked from KiSystemStartupBootStack, or, it's invoked for CPU != 0 directly by KiSystemStartupBootStack()).

HBelusca · 2025-04-29T20:36:42Z

ntoskrnl/ke/tlbflush.c

+
+VOID
+NTAPI
+KeFlushSingleTb(


These main flush Tb functions should raise irql to SYNCH_LEVEL (with KeRaiseIrqlToSynchLevel) and lower it back once finished.

What would be the point of that?

Some synchronization issues perhaps (~~or for using the ipi routines~~)? I'm just stating what the previous functions were doing, and something that has been observed elsewhere:
https://community.osr.com/t/more-questions-about-mapping-user-kernel-mode-memory/13920/9

That is the usual "That's what Windows does" argument. We are not Windows. Our code isn't Windows code. Windows has all kinds of magic sauce, changing from version to version. For example Windows has some timestamp stuff going on (https://azius.com/the-page-fault-in-nonpaged-pool-syndrome/). We don't have that. What we have is either an IPI on SMP, which does the raise already, or simply one or more invlpg instructions. There is nothing to synchronize. In the worst case you'd invalidate TLB entries that have already been invalidated by a cr3 switch.

HBelusca · 2025-04-29T20:39:54Z

ntoskrnl/mm/ARM3/hypermap.c

        //
        Offset = MI_ZERO_PTES;
        PointerPte->u.Hard.PageFrameNumber = Offset;
-        KeFlushProcessTb();


MiMapPageInHyperSpace at line 69 does a similar flush; why isn't it needed here in this case?

The problem here was that a process flush wouln't work, because these are global PTEs (See line 151). I changed it back to where it was before, just turning it into a global range flush instead of the process flush. It's all not optimal, but it should work just fine. I might improve it in a separate PR.

HBelusca · 2025-04-29T21:02:06Z

ntoskrnl/ke/amd64/thrdini.c

-
-        /* Set new TSS fields */
-        //Pcr->TssBase->IoMapBase = NewProcess->IopmOffset;
+        KiSwapProcess(NewProcess, OldProcess);


Note: a similar change could be done in x86 in KiSwapContextExit (compare with the x86 KiSwapProcess implementation) -- this would ensure having the SMP-specific code run there as well.

This is not an SMP PR and won't change anything in x86 that isn't strictly needed. I'll leave that part to @DarkFire01.

HBelusca · 2025-04-29T21:05:53Z

ntoskrnl/include/internal/x86x64/ke.h

+#endif
+
+static const ULONG KxFlushIndividualProcessPagesMaximum = 33; // Based on Linux tlb_single_page_flush_ceiling
+static const ULONG KxFlushIndividualGlobalPagesMaximum = 100; // Just a guess


What are these variables for? Some sort of arbitrary threshold that when reached in KeFlushRangeTb and friends, a full flush is done?

Yes. Use the source, Luke.

HBelusca · 2025-04-29T21:07:06Z

ntoskrnl/mm/ARM3/mminit.c

+    {
+        /* Set it on the template PTE and PDE */
+        ValidKernelPte.u.Hard.Global = TRUE;
+        //ValidKernelPde.u.Hard.Global = TRUE; // FIXME: disabled for now


What's happening if you enable this one?

As usual: crash and burn. PDEs just need more work.

HBelusca · 2025-04-29T21:14:28Z

ntoskrnl/mm/ARM3/wslist.cpp

            /* Make this a transition PTE */
            MI_MAKE_TRANSITION_PTE(PointerPte, Page, Protection);
-            KeInvalidateTlbEntry(MiAddressToPte(PointerPte));
+            KeFlushSingleTb(MiAddressToPte(PointerPte), FALSE);


Noob question (I could have asked it in other places): What dictates the choice of flushing the TLB only for the processors on which the "current" process is running, versus, flushing unconditionally on all processors?

If it's a process local page, then there is no need to flush it on processors that don't run in the context of the current process, because they first need to switch to it, i.e. switch cr3, which flushes the TLB already for non-global pages. And WSLE entries are process local.

HBelusca · 2025-04-29T21:16:27Z

ntoskrnl/ke/amd64/cpu.c

-    KeFlushCurrentTb();
-
-    /* Update the flush stamp and return to original IRQL */
-    InterlockedExchangeAdd(&KiTbFlushTimeStamp, 1);


Ah, by the way, about this timestamp, you may find this reading interesting: https://azius.com/the-page-fault-in-nonpaged-pool-syndrome/
(The MS KB mentioned in the article can be found here.)

Yes, I am aware of it. We don't make use of this feature and I doubt we ever will, because IMO it's pointless. It doesn't work in user-mode, and in kernel mode there are much better ways to avoid flushes.

Assert that only the single zeroing thread calls these functions.

- Add support for global pages - Add support for SMP - Improve naming to be more explicit about what the individual functions do - Unify shared code

tkreuzer added 3 commits April 18, 2025 01:33

[NDK][NTOS] Implement inline KeGetCurrentProcess

08ad46a

[NTOS:KE/x64] Make sure that KPROCESS::ActiveProcessors is set correctly

35edb85

This is important for properly flushing the TLB on SMP.

[NTOS:KE] Implement KiIpiSendRequest stub

c3d0f1b

tkreuzer requested review from DarkFire01, EricKohl, Extravert-ir, TAN-Gaming, ThFabba, binarymaster, Copilot and hpoussin April 26, 2025 15:05

tkreuzer self-assigned this Apr 26, 2025

tkreuzer requested a review from HeisSpiter as a code owner April 26, 2025 15:05

Copilot AI reviewed Apr 26, 2025

View reviewed changes

ntoskrnl/ke/ipi.c Show resolved Hide resolved

ntoskrnl/ke/amd64/ipi.c Show resolved Hide resolved

github-actions bot added the kernel&hal Code changes to the ntoskrnl and HAL label Apr 26, 2025

HBelusca self-requested a review April 26, 2025 15:24

binarymaster added the enhancement For PRs with an enhancement/new feature. label Apr 26, 2025

binarymaster added this to ReactOS PRs Apr 26, 2025

github-project-automation bot moved this to New PRs in ReactOS PRs Apr 26, 2025

binarymaster removed this from ReactOS PRs Apr 26, 2025

binarymaster added this to the x64 bringup milestone Apr 26, 2025

HBelusca reviewed Apr 29, 2025

View reviewed changes

HBelusca self-requested a review April 29, 2025 21:28

tkreuzer added 3 commits May 15, 2025 16:21

[NTOS:MM] Add ASSERTs to Mi(Un)MapPagesInZeroSpace

d02b648

Assert that only the single zeroing thread calls these functions.

[NTOS::KE] Rewrite TLB flushing

d4cfb8c

- Add support for global pages - Add support for SMP - Improve naming to be more explicit about what the individual functions do - Unify shared code

[NTOS:MM] Implement support for global pages

12d75df

tkreuzer force-pushed the ntos/global_pages branch from 8d8c2ff to 12d75df Compare June 12, 2025 11:49

DarkFire01 approved these changes Jun 12, 2025

View reviewed changes

Uh oh!

[NTOS] Implement support for global pages #7937

Are you sure you want to change the base?

[NTOS] Implement support for global pages #7937

Uh oh!

Conversation

tkreuzer commented Apr 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Proposed changes

TODO (not addressed in this PR)

Testbot runs (Filled in by Devs)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HBelusca May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tkreuzer May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tkreuzer commented Apr 26, 2025 •

edited

Loading

HBelusca May 14, 2025 •

edited

Loading

tkreuzer May 15, 2025 •

edited

Loading