Now that we reclaim the memory range that is created by KASLR before
the start of the kernel image, there's no need to be conservative with
the KASLR offset.
This ensures we don't just waste the memory range between the default
base load address and the actual load address that was shifted by the
KASLR offset.
If we crashed in the middle of mapping in Regions, some of the regions
may not have a page directory yet, and will result in a crash when
Region::remap() is called.
If someone specifically wants contiguous memory in the low-physical-
address-for-DMA range ("super pages"), they can use the
allocate_dma_buffer_pages() helper.
Function-local `static constexpr` variables can be `constexpr`. This
can reduce memory consumption, binary size, and offer additional
compiler optimizations.
These changes result in a stripped x86_64 kernel binary size reduction
of 592 bytes.
As make<T> is infallible, it really should not be used anywhere in the
Kernel. Instead replace with fallible `new (nothrow)` calls, that will
eventually be error-propagated.
When a page fault led to the mapping of a new physical page, we were
updating the page tables for *every* region that shared the same
underlying VMObject.
Let's just not do that, avoiding a bunch of unnecessary page table
updates and TLB invalidations.
Ideally the x86 fault handler would only do x86 specific things and
delegate the rest of the work to MemoryManager. This patch moves some of
the address checks to a more generic place.
This avoids taking and releasing the MM lock just to reject an address
that we can tell from just looking at it that it won't ever be in the
kernel regions tree.
When the values we're setting are not actually u32s and the size of the
area we're setting is PAGE_SIZE-aligned and a multiple of PAGE_SIZE in
size, there's no point in using fast_u32_fill, as that forces us to use
STOSDs instead of STOSQs.
This allows us to enable Write-Combine on e.g. framebuffers,
significantly improving performance on bare metal.
To keep things simple we right now only use one of up to three bits
(bit 7 in the PTE), which maps to the PA4 entry in the PAT MSR, which
we set to the Write-Combine mode on each CPU at boot time.
We were already using a non-intrusive RedBlackTree, and since the kernel
regions tree is non-owning, this is a trivial conversion that makes a
bunch of the tree operations infallible (by being allocation-free.) :^)
PageDirectory gets initialized step-by-step in
PageDirectory::try_create_for_userspace(). This initialization may fail
anywhere in this function - for example, we may not be able to
allocate a directory table, in which case
PageDirectory::try_create_for_userspace() will return a null pointer.
We recognize this condition and early-return ENOMEM. However, at this
point, we need to correctly destruct the only partially initialized
PageDirectory. Previously, PageDirectory::~PageDirectory() would assume
that the object it was destructing was always fully initialized. It now
uses the new helper PageDirectory::is_cr3_initialized() to correctly
recognize when the directory table was not yet initialized. This helper
checks if the pointer to the directory table is null. Only if it is not
null does the destructor try to fetch the directory table using
PageDirectory::cr3().
These infallible resource factory functions were only there to ease the
conversion to the new factory functions. Since all child classes of
VMObject now use the fallible resource factory functions, we don't
need the infallible versions anymore.
This commit moves the allocation of the resources required for
SharedInodeVMObject from its constructors to its factory functions.
We're making this change to expose the fallibility of the allocation.
This commit moves the allocation of the resources required for
PrivateInodeVMObject from its constructors to its factory functions.
We're making this change to expose the fallibility of the allocation.
This commit moves the allocation of the resources required for
InodeVMObject from its constructors to the constructors of its child
classes.
We're making this change to give the child classes the chance to expose
the fallibility of the allocation.
This commit moves the allocation of the resources required for
AnonymousVMObject from its constructors to its factory functions.
We're making this change to expose the fallibility of the allocation.
This commit moves the allocation of the resources required for VMObject
from its constructors to the constructors of its child classes.
We're making this change to give the child classes the chance to expose
the fallibility of the allocation.
The only purpose of the remap() in Region::try_clone() is to ensure
non-writable page table entries for CoW regions. If a region is already
non-writable, there's no need to waste time updating the page tables.
When mapping or unmapping completely inaccessible memory regions,
we don't need to update the page tables at all. This saves a bunch of
time in some situations, most notably during dynamic linking, where we
make a large VM reservation and immediately throw it away. :^)
We were already only tracking kernel regions, this patch just makes it
more clear by having it reflected in the name of the registration
helpers.
We also stop calling them for userspace regions, avoiding some spinlock
action in such cases.
This optimization was added when region lookup was O(n), before we had
the O(log n) RedBlackTree. Let's remove it to simplify the code, as we
have no evidence that it remains valuable.
Previously we would only remove them from the map if they were attached
to an AddressSpace, even though we would always add them to the map on
construction. This results in an assertion failure on destruction if
the page directory was never attached to an AddressSpace. (for example,
on an allocation failure of said AddressSpace)
This mostly just moved the problem, as a lot of the callers are not
capable of propagating the errors themselves, but it's a step in the
right direction.
When deleting an entire AddressSpace, we don't need to do TLB flushes
at all (since the entire page directory is going away anyway).
We also don't need to deallocate VM ranges one by one, since the entire
VM range allocator will be deleted anyway.
The purpose of the PageDirectory::m_page_tables map was really just
to act as ref-counting storage for PhysicalPage objects that were
being used for the directory's page tables.
However, this was basically redundant, since we can find the physical
address of each page table from the page directory, and we can find the
PhysicalPage object from MemoryManager::get_physical_page_entry().
So if we just manually ref() and unref() the pages when they go in and
out of the directory, we no longer need PageDirectory::m_page_tables!
Not only does this remove a bunch of kmalloc() traffic, it also solves
a race condition that would occur when lazily adding a new page table
to a directory:
Previously, when MemoryManager::ensure_pte() would call HashMap::set()
to insert the new page table into m_page_tables, if the HashMap had to
grow its internal storage, it would call kmalloc(). If that kmalloc()
would need to perform heap expansion, it would end up calling
ensure_pte() again, which would clobber the page directory mapping used
by the outer invocation of ensure_pte().
The net result of the above bug would be that any invocation of
MemoryManager::ensure_pte() could erroneously return a pointer into
a kernel page table instead of the correct one!
This whole problem goes away when we remove the HashMap, as ensure_pte()
no longer does anything that allocates from the heap.
