As a nearby comment says, "This is a terrible approximation".
This doesn't make things less terrible, but it does make things
more correct in the given framework of terribleness.
Fixes#17156.
A lot of places were relying on AK/Traits.h to give it strnlen, memcmp,
memcpy and other related declarations.
In the quest to remove inclusion of LibC headers from Kernel files, deal
with all the fallout of this included-everywhere header including less
things.
In cases where we know a string literal will fit in the short string
storage, we can do so at compile time without needing to handle error
propagation. If the provided string literal is too long, a compilation
error will be emitted due to the failed VERIFY statement being a non-
constant expression.
This does not need to be defined in Format.h. This causes FixedPoint.h
to be included everywhere. This is particularly going to be an issue
when trying to include <CoreServices/CoreServices.h> on macOS. The macOS
SDK defines its own FixedPoint structure which will conflict with ours.
The Unicode spec defines much more complicated caseless matching
algorithms in its Collation spec. This implements the "basic" case
folding comparison.
`get()` is intended as a replacement for `get_deprecated()` and `get_ptr
()`. The former returns the same value for "key not found" and "key
found and contains `null`" which is ambiguous. The latter returns a raw
pointer which is spooky. Returning `Optional<JsonValue const&>` covers
all the previous uses for these.
The `get_foo()` methods are helpers to make user code less verbose. Most
of the time, we only want a specific type of value: if we want a number
and get a string, we respond the same as if the value was not there at
all. These make that easier to express.
This also adjusts the `has_i32()` method and friends to examine the
value instead of just looking at the underlying type.
The existing `is_i32()` and friends only check if `i32` is their
internal type, but a value such as `0` could be literally any integer
type internally. `is_integer<T>()` instead determines whether the
contained value is an integer and can fit inside T.
While at it, rename the `read_trivial_value` and `write_trivial_value`
functions to `read_value` and `write_value` respectively, since we'll
add compatibility for non-trivial types down the line.
This error was introduced by 9a7accdd and had a significant impact on
`BufferedFile` behavior. Hence, we started seeing crash in test262.
By itself, the issue was a wrong calculation of the internal reading
spans when using the `read` and `until` parameters. Which can lead to
at worse crash in VERIFY and at least weird behaviors as missed needles
or detections out of bounds.
It was also accompanied by an erroneous test.
This patch fixes the bug, the test and also provides more tests.
If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might
not be available in the global namespace. Follow the pattern established
in LibTest to fully qualify AK types in macros to avoid this problem.
This parameter allows to start searching after an offset. For example,
to resume a search.
It is unfortunately a breaking change in API so this patch also modifies
one user and one test.
The previously defined operator was swap-based. With the defaulted
implementation, both integers are now copied, but it doesn't matter as
only the `ByteBuffer` allocates memory (i.e. non-null integers values
won't affect the destruction).
These are formatters that can only be used with debug print
functions, such as dbgln(). Currently this is limited to
Formatter<ErrorOr<T>>. With this you can still debug log ErrorOr
values (good for debugging), but trying to use them in any
String::formatted() call will fail (which prevents .to_string()
errors with the new failable strings being ignored).
You make a formatter debug only by adding a constexpr method like:
static constexpr bool is_debug_only() { return true; }
This implements a FlyString that will de-duplicate String instances. The
FlyString will store the raw encoded data of the String instance: If the
String is a short string, FlyString holds the String::ShortString bytes;
otherwise FlyString holds a pointer to the Detail::StringData.
FlyString itself does not know about String's storage or how to refcount
its Detail::StringData. It defers to String to implement these details.
Since AK can't refer to LibUnicode directly, the strategy here is that
if you need case transformations, you can link LibUnicode and receive
them. If you try to use either of these methods without linking it, then
you'll of course get a linker error (note we don't do any fallbacks to
e.g. ASCII case transformations). If you don't need these methods, you
don't have to link LibUnicode.
DeprecatedFlyString relies heavily on DeprecatedString's StringImpl, so
let's rename it to A) match the name of DeprecatedString, B) write a new
FlyString class that is tied to String.
This allows us to make all comparision operators on the class constexpr
without pulling in a bunch of boilerplate. We don't use the `<compare>`
header because it doesn't compile in the main serenity cross-build due
to the include paths to LibC being incompatible with how libc++ expects
them to be for clang builds.
The previous implementation of Statistics::median() was slightly
incorrect with an even number of elements since in those cases it needs
to be the arithmetic mean of the two elements that share the middle
position.
These instances were detected by searching for files that include
AK/Memory.h, but don't match the regex:
\\b(fast_u32_copy|fast_u32_fill|secure_zero|timing_safe_compare)\\b
This regex is pessimistic, so there might be more files that don't
actually use any memory function.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
stdlib.h, but don't match the regex:
\\b(_abort|abort|abs|aligned_alloc|arc4random|arc4random_buf|arc4random_
uniform|atexit|atof|atoi|atol|atoll|bsearch|calloc|clearenv|div|div_t|ex
it|_Exit|EXIT_FAILURE|EXIT_SUCCESS|free|getenv|getprogname|grantpt|labs|
ldiv|ldiv_t|llabs|lldiv|lldiv_t|malloc|malloc_good_size|malloc_size|mble
n|mbstowcs|mbtowc|mkdtemp|mkstemp|mkstemps|mktemp|posix_memalign|posix_o
penpt|ptsname|ptsname_r|putenv|qsort|qsort_r|rand|RAND_MAX|random|reallo
c|realpath|secure_getenv|serenity_dump_malloc_stats|serenity_setenv|sete
nv|setprogname|srand|srandom|strtod|strtof|strtol|strtold|strtoll|strtou
l|strtoull|system|unlockpt|unsetenv|wcstombs|wctomb)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use anything from the stdlib.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/Concepts.h, but don't match the regex:
\\b(AnyString|Arithmetic|ArrayLike|DerivedFrom|Enum|FallibleFunction|Flo
atingPoint|Fundamental|HashCompatible|Indexable|Integral|IterableContain
er|IteratorFunction|IteratorPairWith|OneOf|OneOfIgnoringCV|SameAs|Signed
|SpecializationOf|Unsigned|VoidFunction)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any concepts.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/StdLibExtras.h, but don't match the regex:
\\b(abs|AK_REPLACED_STD_NAMESPACE|array_size|ceil_div|clamp|exchange|for
ward|is_constant_evaluated|is_power_of_two|max|min|mix|move|_RawPtr|RawP
tr|round_up_to_power_of_two|swap|to_underlying)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any "extra stdlib" functions.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/Format.h, but don't match the regex:
\\b(CheckedFormatString|critical_dmesgln|dbgln|dbgln_if|dmesgln|FormatBu
ilder|__FormatIfSupported|FormatIfSupported|FormatParser|FormatString|Fo
rmattable|Formatter|__format_value|HasFormatter|max_format_arguments|out
|outln|set_debug_enabled|StandardFormatter|TypeErasedFormatParams|TypeEr
asedParameter|VariadicFormatParams|v_critical_dmesgln|vdbgln|vdmesgln|vf
ormat|vout|warn|warnln|warnln_if)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any formatting functions.
Observe that this revealed that Userland/Libraries/LibC/signal.cpp is
missing an include.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
This step would ideally not have been necessary (increases amount of
refactoring and templates necessary, which in turn increases build
times), but it gives us a couple of nice properties:
- SpinlockProtected inside Singleton (a very common combination) can now
obtain any lock rank just via the template parameter. It was not
previously possible to do this with SingletonInstanceCreator magic.
- SpinlockProtected's lock rank is now mandatory; this is the majority
of cases and allows us to see where we're still missing proper ranks.
- The type already informs us what lock rank a lock has, which aids code
readability and (possibly, if gdb cooperates) lock mismatch debugging.
- The rank of a lock can no longer be dynamic, which is not something we
wanted in the first place (or made use of). Locks randomly changing
their rank sounds like a disaster waiting to happen.
- In some places, we might be able to statically check that locks are
taken in the right order (with the right lock rank checking
implementation) as rank information is fully statically known.
This refactoring even more exposes the fact that Mutex has no lock rank
capabilites, which is not fixed here.
Using policy based design `SinglyLinkedList` and
`SinglyLinkedListWithCount` can be combined into one class which takes
a policy to determine how to keep track of the size of the list. The
default policy is to use list iteration to count the items in the list
each time. The `WithCount` form is a different policy which tracks the
size, but comes with the overhead of storing the count and
incrementing/decrementing on each modification.
This model is extensible to have other forms of counting by
implementing only a new policy instead of implementing a totally new
type.
A possible integer overflow might have occured inside the function in
case (number % unit) * 10 did not fit into a u64. So it is verified that
this does not happen at the beginning of the function.
These instances were detected by searching for files that include
IterationDecision.h, but don't match the regex:
\\bIterationDecision(?!\.h>)\\b
This is the only symbol defined by IterationDecision.h.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
Array.h, but don't match the regex:
\\b(Array(?!\.h>)|iota_array|integer_sequence_generate_array)\\b
These are the three symbols defined by Array.h.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
In 7c5e30daaa, the focus was "only" on
Userland/Libraries/, whereas this commit cleans up the remaining
headers in the repo, and any new badly-formatted include.
The class is very similar to `CircularDuplexStream` in its behavior.
Main differences are that `CircularBuffer`:
- does not inherit from `AK::Stream`
- uses `ErrorOr` for its API
- is heap allocated (and OOM-Safe)
This patch also add some tests.
This file does not contain any architecture specific implementations,
so we can move it to the Kernel base directory. Also update the relevant
include paths.
Mark other ErrorOr types as friends, and fix a typo in the &&
constructor, so that we can create an ErrorOr<Core::Object> from an
ErrorOr<GUI::Widget>. Also, add some requires() clauses to these
constructors so the error messages are clearer.
Previously any backslash and the character following it were ignored.
This commit adds a fall through to match the character following the
backslash without checking whether it is "special".
This allows callers to use the following semantics:
using MyVariant = Variant<Empty, int>;
template<typename T>
size_t size() { return TypeList<T>::size; }
auto s = size<MyVariant>();
This will be needed for an upcoming IPC change, which will result in us
knowing the Variant type, but not the underlying variadic types that the
Variant holds.
This shrinks sizeof(Error) from 32 bytes to 24 bytes, which in turn will
shrink sizeof(ErrorOr<T>) by the same amount (in cases where sizeof(T)
is less than sizeof(Error)).
Instead of avoiding overflow-checking builtins with AK_COMPILER_CLANG,
we can use the preprocessor's __has_builtin() mechanism to check if
they are available.
`OwnPtrWithCustomDeleter` was a decorator which provided the ability
to add a custom deleter to `OwnPtr` by wrapping and taking the deleter
as a run-time argument to the constructor. This solution means that no
additional space is needed for the `OwnPtr` because it doesn't need to
store a pointer to the deleter, but comes at the cost of having an
extra type that stores a pointer for every instance.
This logic is moved directly into `OwnPtr` by adding a template
argument that is defaulted to the default deleter for the type. This
means that the type itself stores the pointer to the deleter instead
of every instance and adds some type safety by encoding the deleter in
the type itself instead of taking a run-time argument.
This class is a smart pointer that let you provide a custom deleter to
free the pointer.
It is quite primitive compared to other smart pointers but can still be
useful when interacting with C types that provides a custom `free()`
function.
This was removed in a910961f37d1da9dafb6385e348266746354cf98 in favour
of the more general USING_AK_GLOBALLY #define, but Ladybird (and
probably other projects) depend on the smaller hammer to include STL
headers and keep the USING_AK_GLOBALLY behaviour, so put it back and
preserve its behaviour.
Note that this still keeps the old behaviour of putting things in std by
default on serenity so the tools can be happy, but if USING_AK_GLOBALLY
is unset, AK behaves like a good citizen and doesn't try to put things
in the ::std namespace.
std::nothrow_t and its friends get to stay because I'm being told that
compilers assume things about them and I can't yeet them into a
different namespace...for now.
A couple headers expected names to be in the global namespace, qualify
those names to make sure they're resolved even when the names are not
exported.
One header placed its functions in the global namespace, move those to
the AK namespace to make the concepts resolve.
Implement insertion sort in AK. The cutoff value 7 is a magic number
here, values [5, 15] should work well. Main idea of the cutoff is to
reduce recursion performed by quicksort to speed up sorting
of small partitions.
Note that Jakt only allows StringView creation from string literals, so
none of the invariants in the class are broken by this (if used only
from within Jakt).
This allows the user to transform the contents of the optional (if any
exists), without manually unwrapping and then rewrapping it.
This is needed by the Jakt runtime.
This is used in Jakt, and providing that value from Jakt's side is more
trouble than doing this.
Considering this class is bound to go away, a little
backwards-compatible API change is just fine.
The previous moved-from state was the null string. This violates both
our invariant that String is never null, and also the C++ contract that
the moved-from state must be valid but unspecified. The empty short
string state is of course valid, so it satisfies both invariants. It
also allows us to remove any extra checks for the null state.
The reason this change is made is primarily because swap() requires
moved-from objects to be reassignable (C++ allows this). Because the
move assignment of String would not check the null state, it crashed
trying to increment the data reference count (nullptr signals a
non-short string). This meant that e.g. quick_sort'ing String would
crash immediately.
s p a c e s h i p o p e r a t o r
Comparing UTF-8 can be done by simple byte lexicographic comparison per
definition, so we just piggy-back on StringView's high-performance
comparator.
Similar to how LibJS and LibSQL used to behave, the boolean constructor
of JsonValue is currently allowing pointers to be used to construct a
boolean value. Explicitly disallow such construction.
This allows us to pass the new String type to functions that take a
StringView directly, having to call bytes_as_string_view() every time
gets old quickly.
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.
Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.
Problems in DeprecatedString:
- It assumes string allocation never fails. This makes it impossible
to use in allocation-sensitive contexts, and is the reason we had to
ban DeprecatedString from the kernel entirely.
- The awkward null state. DeprecatedString can be null. It's different
from the empty state, although null strings are considered empty.
All code is immediately nicer when using Optional<DeprecatedString>
but DeprecatedString came before Optional, which is how we ended up
like this.
- The encoding of the underlying data is ambiguous. For the most part,
we use it as if it's always UTF-8, but there have been cases where
we pass around strings in other encodings (e.g ISO8859-1)
- operator[] and length() are used to iterate over DeprecatedString one
byte at a time. This is done all over the codebase, and will *not*
give the right results unless the string is all ASCII.
How we solve these issues in the new String:
- Functions that may allocate now return ErrorOr<String> so that ENOMEM
errors can be passed to the caller.
- String has no null state. Use Optional<String> when needed.
- String is always UTF-8. This is validated when constructing a String.
We may need to add a bypass for this in the future, for cases where
you have a known-good string, but for now: validate all the things!
- There is no operator[] or length(). You can get the underlying data
with bytes(), but for iterating over code points, you should be using
an UTF-8 iterator.
Furthermore, it has two nifty new features:
- String implements a small string optimization (SSO) for strings that
can fit entirely within a pointer. This means up to 3 bytes on 32-bit
platforms, and 7 bytes on 64-bit platforms. Such small strings will
not be heap-allocated.
- String can create substrings without making a deep copy of the
substring. Instead, the superstring gets +1 refcount from the
substring, and it acts like a view into the superstring. To make
substrings like this, use the substring_with_shared_superstring() API.
One caveat:
- String does not guarantee that the underlying data is null-terminated
like DeprecatedString does today. While this was nifty in a handful of
places where we were calling C functions, it did stand in the way of
shared-superstring substrings.
Previously we allowed the end_offset to be larger than the chunk itself,
which made it so that certain input sizes would make the logic attempt
to delete a nonexistent object.
Fixes#16308.
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
This patch adds support for 128-bit floating points in FloatExtractor.
This is required to build SerenityOS on MacOS/aarch64. It might break
building for Raspberry Pi.
AK internals like to use concepts and details without a fully qualified
name, which usually works just fine because we make everything
AK-related available to the unqualified namespace.
However, this breaks as soon as we start not using `USING_AK_GLOBALLY`,
due to those identifiers no longer being made available. Instead, we
just export those into the `AK` namespace instead.
This patch adds the `USING_AK_GLOBALLY` macro which is enabled by
default, but can be overridden by build flags.
This is a step towards integrating Jakt and AK types.
Unlike iterator_at_byte_offset(), this function assumes the provided
byte offset is a valid offset into the UTF-8 character stream.
This avoids walking the stream from the start.
There was a subtle mismatch between the obviously expected behavior
of BumpAllocator::for_each_chunk() and its actual implementation.
You'd think it would invoke the callback with the address of each chunk,
but actually it also took the liberty of adding sizeof(ChunkHeader) to
this address. UniformBumpAllocator::destroy_all() relied on this to
get the right address for objects to delete.
The bug happened in BumpAllocator::deallocate_all(), where we use
for_each_chunk() to walk the list of chunks and munmap() them.
To avoid memory mapping churn, we keep a global cache of 1 chunk around.
Since we were being called with the offset chunk address, it meant that
the cached chunk shifted 16 bytes away from its real address every time
we re-added it to the cache.
Eventually the cached chunk address would leave its memory region
entirely, and at that point, any attempt to allocate from it would yield
an address outside the region, causing memory corruption.
Even if the pointer value is const, the value they point to is not
necessarily const, so these functions should not add the qualifier.
This also removes the redundant non-const implementations of these
operators.
