KBufferBuilder is always allowed to expand if it wants to. This
restriction was added a long time ago when it was unsafe to allocate
VM while generating ProcFS contents.
Use a Mutex instead of a SpinLock to protect the per-FileDescription
generated data cache. This allows processes to go to sleep while
waiting their turn.
Also don't try to be clever by reusing existing cache buffers.
Just allocate KBuffers as needed (and make sure to surface failures.)
KBufferBuilder exists for code that wants to build a KBuffer instead
of a String. KBuffer is backed by anonymous VM, while String is backed
by a kernel heap allocation.
Advisory locks don't actually prevent other processes from writing to
the file, but they do prevent other processes looking to acquire and
advisory lock on the file.
This implementation currently only adds non-blocking locks, which are
all I need for now.
Before we start disabling acquisition of the big process lock for
specific syscalls, make sure to document and assert that all the
lock is held during all syscalls.
There are cases where we want to conditionally take a lock, but still
would like to use an RAII type to make sure we don't leak the lock.
This was previously impossible to do with `MutexLocker` due to it's
design. This commit tweaks the design to allow the object to be
initialized to an "empty" state without a lock associated, so it does
nothing, and then later a lock can be "attached" to the locker.
I realized that the get_lock() API's where also unused, and would no
longer make sense for empty locks, so they were removed.
Move these to MM to simplify the flow of the syscall handler.
While here, also make sure we hold the process space lock for
the duration of the validation to avoid potential issues where
another thread attempts to modify the process space during the
validation. This will allow us to move the validation out of the
big process lock scope in a future change.
Additionally utilize the new no_lock variants of functions to avoid
unnecessary recursive process space spinlock acquisitions.
Currently all syscalls run under the Process:m_big_lock, which is an
obvious bottleneck. Long term we would like to remove the big lock and
replace it with the required fine grained locking.
To facilitate this goal we need a way of gradually decomposing the big
lock into the all of the required fine grained locks. This commit
introduces instrumentation to the syscall table, allowing the big lock
requirement to be toggled on/off per syscall.
Eventually when we are finished, no syscall will required the big lock,
and we'll be able to remove all of this instrumentation.
The entire process is not needed, just require the user to pass in the
Space. Also provide no_lock variant to use when you already have the
VM/Space lock acquired, to avoid unnecessary recursive spinlock
acquisitions.
Depending on the exact layout of the .ksyms section the kernel would
fail to boot because the kernel_load_end variable didn't account for the
section's size.
The non CPU specific code of the kernel shouldn't need to deal with
architecture specific registers, and should instead deal with an
abstract view of the machine. This allows us to remove a variety of
architecture specific ifdefs and helps keep the code slightly more
portable.
We do this by exposing the abstract representation of instruction
pointer, stack pointer, base pointer, return register, etc on the
RegisterState struct.
Allocate all the RX buffers in one big memory region (and same for TX.)
This removes 38 lines from every crash dump (and just seems like a
reasonable idea in general.)
We need to cast physical addresses to PhysicalPtr instead of FlatPtr,
which is currently always 64 bits. However, if one day we were to
support 32 bit non-pae mode then it would also truncate appropriately.
This switches tracking CPU usage to more accurately measure time in
user and kernel land using either the TSC or another time source.
This will also come in handy when implementing a tickless kernel mode.
As threads come and go, we can't simply account for how many time
slices the threads at any given point may have been using. We need to
also account for threads that have since disappeared. This means we
also need to track how many time slices we have expired globally.
However, because this doesn't account for context switches outside of
the system timer tick values may still be under-reported. To solve this
we will need to track more accurate time information on each context
switch.
This also fixes top's cpu usage calculation which was still based on
the number of context switches.
Fixes#6473
This adds a ".profile" extension to perfcore files written by the
Kernel. Also, the process name is now visible in the perfcore filename.
Furthermore, this patch adds error handling for the case where the
filename generated by the Kernel is already taken. In that case, a digit
will be added to the filename (before the extension).
This also adds some more error logging to dump_perfcore().
This implements a simple bootloader that is capable of loading ELF64
kernel images. It does this by using QEMU/GRUB to load the kernel image
from disk and pass it to our bootloader as a Multiboot module.
The bootloader then parses the ELF image and sets it up appropriately.
The kernel's entry point is a C++ function with architecture-native
code.
Co-authored-by: Liav A <liavalb@gmail.com>
When a Thread is being unblocked and we need to re-lock the process
big_lock and re-locking blocks again, then we may end up in
Thread::block again while still servicing the original lock's
Thread::block. So permit recursion as long as it's only the big_lock
that we block on again.
Fixes#8822
We often get queried for the root inode, and it will always be cached
in memory anyway, so let's make Ext2FS::root_inode() fast by keeping
the root inode in a dedicated member variable.
To count the remaining children, we simply need to traverse the
directory and increment a counter. No need for a custom virtual that
all file systems have to implement. :^)
Unless we're accessing mutex-guarded metadata, there's no need to
acquire the inode lock.
The file system ID or inode index of a constructed inode will never
change, for example.
We should never request a regions removal that we don't currently
own. We currently assert this everywhere else by all callers.
Instead lets just push the assert down into the RedBlackTree removal
and assume that we will always successfully remove the region.
This is a much more ergonomic option than getting a
`VERIFY_NOT_REACHED()` failure at run-time. I encountered this issue
with Clang, where sized deallocation is not the default due to ABI
breakage concerns.
Note that we can't simply just not declare these functions, because the
C++ standard states:
> If this function with size parameter is defined, the program shall
> also define the version without the size parameter.
The compiler will use these to allocate objects that have alignment
requirements greater than that of our normal `operator new` (4/8 byte
aligned).
This means we can now use smart pointers for over-aligned types.
Fixes a FIXME.
By default, the compiler will assume that `operator new` returns
pointers that are aligned correctly for every built-in type. This is not
the case in the kernel on x64, since the assumed alignment is 16
(because of long double), but the kmalloc blocks are only
`alignas(void*)`.
Thread::yield_and_release_relock_big_lock releases the big lock, yields
and then relocks the big lock.
Thread::yield_assuming_not_holding_big_lock yields assuming the big
lock is not being held.
When blocking on a Lock other than the big lock and we're holding the
big lock, we need to release the big lock first. This fixes some
deadlocks where a thread blocks while holding the big lock, preventing
other threads from getting the big lock in order to unblock the waiting
thread.
When a Lock blocks (e.g. due to a mode mismatch or because someone
else holds it) the lock mode will be updated to what was requested.
There were also some cases where restoring locks may have not worked
as intended as it may have been held already by the same thread.
Fixes#8787
The kernel doesn't currently boot when using an address other than
0xc0000000 because the page tables aren't set up properly for that
but this at least lets us build the kernel.
The 32-bit boot code jumps to 0xc0000000 + entry address once page
tables are set up. This is unnecessary for 64-bit mode because we'll
do another far jump just moments later.
I botched this in 859e5741ff, the check
was supposed to be with Process::is_kernel_process().
This fixes an issue with zombie processes hanging around forever.
Thanks tomuta for spotting it! :^)
Reimplement directory traversal in terms of read_bytes() instead of
doing direct block access. This lets us avoid taking the inode lock
while iterating over the directory contents.
Once we've finalized all the file system metadata in flush_writes(),
we no longer need to hold the file system lock during the call to
BlockBasedFileSystem::flush_writes().
Ext2FS::get_inode() will remember unknown inode indices that it has
been asked about and put them into the inode cache as null inodes.
flush_writes() was not null-checking these while iterating, which
was a bug I finally managed to hit.
Flushing also seemed like a good time to drop unknown inodes from
the cache, since there's no good reason to hold to them indefinitely.
The file system lock is meant to protect the file system metadata
(super blocks, bitmaps, etc.) Not protect processes from reading
independent parts of the disk at once.
This patch introduces a new lock to protect the *block cache* instead,
which is the real thing that needs synchronization.
Forcing users of a FileDescription to seek before they can read/write
makes it inherently racy. This patch adds variants of read/write that
simply ignore the "current offset" of the description in favor of a
caller-supplied offset.
This data structure is a much better fit for what is essentially a
sorted list of non-overlapping ranges.
Not using Vector means we no longer have to worry about Vector buffers
getting huge. Only nice & small allocations from now on.
This adds a new section .ksyms at the end of the linker map, reserves
5MiB for it (which are after end_of_kernel_image so they get re-used
once MemoryManager is initialized) and then embeds the symbol map into
the kernel binary with objcopy. This also shrinks the .ksyms section to
the real size of the symbol file (around 900KiB at the moment).
By doing this we can make the symbol map available much earlier in the
boot process, i.e. even before VFS is available.
We leak a ref() onto every user process when constructing them,
either via Process::create_user_process(), or via Process::sys$fork().
This ref() is balanced by a corresponding unref() in
Thread::WaitBlockCondition::finalize().
Since kernel processes don't have a leaked ref() on them, this led to
an extra Process::unref() on kernel processes during finalization.
This happened during every boot, with the `init_stage2` process.
Found by turning off kfree() scrubbing. :^)
In case we are about to delete the PID directory, we clear the Process
pointer. If someone still holds a reference to the PID directory (by
opening it), we still need to delete the process, but we can't delete
the directory, so we will keep it alive, but any operation on it will
fail by propogating the error to userspace about that the Process was
deleted and therefore there's no meaning to trying to do operations on
the directory.
Fixes#8576.
The C++ standard says that it's legal to call the `delete` operator with
a null pointer argument, in which case it should be a no-op. I
encountered this issue when running a kernel that's compiled with Clang.
I assume this fact was used for some kind of optimization.
It's possible that another thread might try to exit the process just
about the same time another thread does the same, or a crash happens.
Also, we may not be able to kill all other threads instantly as they
may be blocked in the kernel (though in this case they would get killed
before ever returning back to user mode. So keep track of whether
Process::die was already called and ignore it on subsequent calls.
Fixes#8485
We can have multiple PhysicalRegions (often the case when there is a
huge amount of RAM) so we really shouldn't print a debug message any
time someone tries to allocate from one. They will move on to another
region anyway.
We were incorrectly using sizeof(PhysicalPageEntry) for some address
calculations instead of sizeof(PageTableEntry).
It still worked correctly because they happen to be the same size.
We now keep all the PhysicalZones on one of two intrusive lists within
the PhysicalRegion.
The "usable" list contains all zones that can be allocated from,
and the "full" list contains all zones with no free pages.
Instead of creating a PhysicalRegion and then expanding it over and
over as we traverse the memory map on boot, we now compute the final
size of the contiguous physical range up front, and *then* create a
PhysicalRegion object.
Nobody was using this API to request anythign about `PAGE_SIZE`
alignment, so let's get rid of it for now. We can reimplement it if
we end up needing it.
Also note that it wasn't actually used anywhere.
The previous allocator was very naive and kept the state of all pages
in one big bitmap. When allocating, we had to scan through the bitmap
until we found an unset bit.
This patch introduces a new binary buddy allocator that manages the
physical memory pages.
Each PhysicalRegion is divided into zones (PhysicalZone) of 16MB each.
Any extra pages at the end of physical RAM that don't fit into a 16MB
zone are turned into 15 or fewer 1MB zones.
Each zone starts out with one full-sized block, which is then
recursively subdivided into halves upon allocation, until a block of
the request size can be returned.
There are more opportunities for improvement here: the way zone objects
are allocated and stored is non-optimal. Same goes for the allocation
of buddy block state bitmaps.
Threads that don't make syscalls still need to be killed, and we can
do that at any time we want so long the thread is in user mode and
not somehow blocked (e.g. page fault).
This reverts commit 3c3a1726df.
We cannot blindly kill threads just because they're not executing in a
system call. Being blocked (including in a page fault) needs proper
unblocking and potentially kernel stack cleanup before we can mark a
thread as Dying.
Fixes#8691
This enables the Lock class to block a thread even while the thread is
working on a BlockCondition. A thread can still only be either blocked
by a Lock or a BlockCondition.
This also establishes a linked list of threads that are blocked by a
Lock and unblocking directly unlocks threads and wakes them directly.
It's possible that a timer may have been queued to be executed by
the timer irq handler, but if we're in a critical section on the
same processor and are trying to cancel that timer, we would spin
forever waiting for it to be executed.
This re-arranges the order of how things are initialized so that we
try to initialize process and thread management earlier. This is
neccessary because a lot of the code uses the Lock class, which really
needs to have a running scheduler in place so that we can properly
preempt.
This also enables us to potentially initialize some things in parallel.
Instead of each PhysicalPage knowing whether it comes from the
supervisor pages or from the user pages, we can just check in both
sets when freeing a page.
It's just a handful of pointer range checks, nothing expensive.
There appears to be no reason why the process registration needs
to happen under the space spin lock. As the first thread is not started
yet it should be completely uncontested, but it's still bad practice.
If no other thread is ready to be run we don't need to switch to the
idle thread and wait for the next timer interrupt. We can just give
the thread another timeslice and keep it running.
We need some overflow checks due to the implementation of TmpFS.
When size_t is 32 bits and off_t is 64 bits, we might overflow our
KBuffer max size and confuse the KBuffer set_size code, causing a VERIFY
failure. Make sure that resulting offset + size will fit in a size_t.
Another constraint, we make sure that the resulting offset + size will
be less than half of the maximum value of a size_t, because we double
the KBuffer size each time we resize it.
We had an inconsistency in valid user addresses. is_user_range() was
checking against the kernel base address, but previous changes caused
the maximum valid user addressable range to be 32 MiB below that.
This patch stops mmap(MAP_FIXED) of a range between these two bounds
from panic-ing the kernel in RangeAllocator::allocate_specific.
Previously we would simply assume that Region allocation always
succeeded. There is still one such assumption when splitting user
regions inside a Space. That will be dealt with in a separate commit.
It is not legal to resize a VMObject after it has been created.
As far as I can tell, this code would never actually run since the
object was already populated with physical pages due to using
AllocationStrategy::AllocateNow.
Previously, VirtualFileSystem::mkdir() would always return ENOENT if
no parent custody was returned by resolve_path(). This is incorrect when
e.g. the user has no search permission in a component of the path
prefix (=> EACCES), or if on component of the path prefix is a file (=>
ENOTDIR). This patch fixes that behavior.
This was only used by a single class (AK::ByteBuffer) in the kernel
and not in an OOM-safe way.
Now that ByteBuffer no longer uses it, there's no need for the kernel
heap to burden itself with supporting this.
C++14 gave us sized operator delete, but we haven't been taking
advantage of it. Let's get to a point where it can help us by
adding kfree_sized(void*, size_t).
This removes some assertions from KLexicalPath::basename() by supporting
paths with trailing slashes, empty paths, paths consisting of only
slashes and paths with ending "." and ".." segments.
The spec requires a flush after setting the new buffer resource id,
which is required by QEMUs SDL backend but not the GTK backend. This
brings us in line with the spec and makes it work for the SDL backend.
The System V ABI for both x86 and x86_64 requires that the stack pointer
is 16-byte aligned on entry. Previously we did not align the stack
pointer properly.
As far as "main" was concerned the stack alignment was correct even
without this patch due to how the C++ _start function and the kernel
interacted, i.e. the kernel misaligned the stack as far as the ABI
was concerned but that misalignment (read: it was properly aligned for
a regular function call - but misaligned in terms of what the ABI
dictates) was actually expected by our _start function.
This involves refactoring VirtIOConsole into VirtIOConsole and
VirtIOConsolePort. VirtIOConsole is the VirtIODevice, it owns multiple
VirtIOConsolePorts as well as two control queues. Each
VirtIOConsolePort is a CharacterDevice.