Commit graph

3498 commits

Author SHA1 Message Date
ovf 898b8ffcb6 LibWeb: Avoid assertion failure on parsing numeric character references 2021-07-28 18:32:22 +02:00
Timothy Flynn 39f971e42b LibUnicode: Begin implementing special Unicode case folding
This implements unconditional special case folding, and conditional
folding for non-locale cases. Worth noting that the only conditional,
non-locale special case is for converting an uppercase sigma to
lowercase.
2021-07-27 21:04:36 +01:00
ovf 13c7d55320 LibWeb: Fix parsing of character references in attribute values 2021-07-27 00:03:43 +02:00
Timothy Flynn 4dda3edc9e LibUnicode: Introduce a Unicode library for interacting with UCD files
The Unicode standard publishes the Unicode Character Database (UCD) with
information about every code point, such as each code point's upper case
mapping. LibUnicode exists to download and parse UCD files at build time
and to provide accessors to that data.

As a start, LibUnicode includes upper- and lower-case code point
converters.
2021-07-26 17:03:55 +01:00
brapru 7e40c17460 AK: Create MACAddress from string
Previously there was no way to create a MACAddress by passing a direct
address as a string. This will allow programs like the arp utility to
create a MACAddress instance by user-passed addresses.
2021-07-25 17:57:08 +02:00
Luke a00b5fc7b7 Tests: Add tests for the quoted printable decoder 2021-07-24 20:11:28 +04:30
Timothy Flynn 345ef6abba LibRegex: Support ECMA-262 Unicode escapes of the form "\u{code_point}"
When the Unicode flag is set, regular expressions may escape code points
by surrounding the hexadecimal code point with curly braces, e.g. \u{41}
is the character "A".

When the Unicode flag is not set, this should be considered a repetition
symbol - \u{41} is the character "u" repeated 41 times. This is left as
a TODO for now.
2021-07-23 23:06:57 +01:00
Timothy Flynn 47f6bb38a1 LibRegex: Support UTF-16 RegexStringView and improve Unicode matching
When the Unicode option is not set, regular expressions should match
based on code units; when it is set, they should match based on code
points. To do so, the regex parser must combine surrogate pairs when
the Unicode option is set. Further, RegexStringView needs to know if
the flag is set in order to return code point vs. code unit based
string lengths and substrings.
2021-07-23 23:06:57 +01:00
Brian Gianforcaro c2282ee28d Tests: Add test coverage for sys$pledge(..) argument validation 2021-07-23 19:02:25 +02:00
Brian Gianforcaro fa448456a9 Tests: Add test coverage for sys$unveil(..) argument validation 2021-07-23 19:02:25 +02:00
Ali Mohammad Pur d40d10aae7 AK: Implement {any,all}_of(IterableContainer&&, Predicate)
This is a generally nicer-to-use version of the existing {any,all}_of()
that doesn't require the user to explicitly provide two iterators.
As a bonus, it also allows arbitrary iterators (as opposed to the hard
requirement of providing SimpleIterators in the iterator version).
2021-07-22 22:56:20 +02:00
Ali Mohammad Pur 6c9ef20010 AK: Add a CommonType<Ts...> type trait
Also adds a simple-ish test for CommonType.
2021-07-22 22:56:20 +02:00
Timothy Flynn 9b83cd1abf AK: Add Utf16View for decoding UTF-16 strings
Also includes a way to transcode from and to UTF-8 strings.
2021-07-22 09:10:44 +02:00
Andreas Kling c7d891765c LibGfx: Use "try_" prefix for static factory functions
Also mark them as [[nodiscard]].
2021-07-21 18:02:15 +02:00
Andrew Kaster 64aac345d3 AK: Use new Formatter for each element in Formatter<Vector<T>>
The state of the formatter for the previous element should be thrown
away for each iteration. This showed up when trying to format a
Vector<String>, since Formatter<StringView> was unhappy about some state
that gets set when it's called. Add a test for Formatter<Vector>.
2021-07-19 05:17:05 +04:30
Peter Bindels ef85c4f747 Tests: Make mmap test point to new kernel address too
During a recent commit the 64-bit kernel was moved to a different
address, breaking this test (unnoticed). This fixes it, so we can
turn on breaking x86_64 tests on the CI again.
2021-07-18 22:08:20 +02:00
Ali Mohammad Pur f364fcec5d LibRegex+Everywhere: Make LibRegex more unicode-aware
This commit makes LibRegex (mostly) capable of operating on any of
the three main string views:
- StringView for raw strings
- Utf8View for utf-8 encoded strings
- Utf32View for raw unicode strings

As a result, regexps with unicode strings should be able to properly
handle utf-8 and not stop in the middle of a code point.
A future commit will update LibJS to use the correct type of string
depending on the flags.
2021-07-18 21:10:55 +04:30
Ali Mohammad Pur e5af15a6e9 LibRegex: Don't do out-of-bound match accesses when a test fails 2021-07-18 21:10:55 +04:30
Peter Bindels b748f11f2d Tests: Disable test if platform has no UserspaceEmulator
Disable test for component with same check that component itself has.
2021-07-18 12:49:33 +01:00
Timothy Flynn 06ddfcde89 AK: Allow setting both width and precision when formatting a string 2021-07-18 02:57:49 +02:00
Timothy Flynn 87848cdf7d AK: Track byte length, rather than code point length, in Utf8View::trim
Utf8View::trim uses Utf8View::substring_view to return its result, which
requires the input to be a byte offset/length rather than code point
length.
2021-07-17 16:59:59 +01:00
Peter Bindels 660a8982e7 LibM: Turn off builtins, fix tests & implementation
While trying to port to Clang we found that the functions as
implemented didn't actually work, and replacing them with a blatantly
broken function also did not break the tests on the GCC build. It
turns out we've been testing GCC's builtins by many tests. This
removes the use of builtins for LibM's tests (so we test the whole
function). It turns off the denormal test for scalbn (which was not
implemented) and comments out the tgamma(0.5) test which is too
inaccurate to be usable (and too complicated for me to fix). The gamma
function was made accurate for all other test cases, and asin received
two more layers of Taylor expansion to bring it within error margin
for the tests.
2021-07-17 17:37:20 +02:00
Max Wipfli b6e995ca3c Tests: Use pointers in TestHTMLTokenizer to avoid copying HTMLTokens 2021-07-17 16:24:57 +04:30
Max Wipfli 918bde98b1 LibWeb: Hide implementation details of HTMLToken attribute list
Previously, HTMLToken would expose the Vector<Attribute> directly to
its users. In preparation for a future change, all users now use
implementation-agnostic APIs which do not expose the Vector directly.
2021-07-17 16:24:57 +04:30
Ali Mohammad Pur 65cd5526cb LibWasm+Everywhere: Make the instruction count limit configurable
...and enable it for LibWeb and test-wasm.
Note that `wasm` will not be limited by this.
2021-07-17 01:13:39 +04:30
Ali Mohammad Pur 23b48f8fe1 Revert "LibWasm: Some more performance stuff (#8812)"
This reverts commit 35394dbfaa.
I pushed the wrong button again, hopefully this will be the last of
such incidents.
2021-07-17 01:11:28 +04:30
Ali Mohammad Pur 35394dbfaa
LibWasm: Some more performance stuff (#8812)
* wasm: Don't try to print the function results if it traps

* LibWasm: Inline some very hot functions

These are mostly pretty small functions too, and they were about ~10%
of runtime.

* LibWasm+Everywhere: Make the instruction count limit configurable

...and enable it for LibWeb and test-wasm.
Note that `wasm` will not be limited by this.

* LibWasm: Remove a useless use of ScopeGuard

There are no multiple exit paths in that function, so we can just put
the ending logic right at the end of the function instead.
2021-07-17 01:04:37 +04:30
Timothy 5f3e6085a2 AK/Tests: Add test for EnumBits has_any_flag()
This test will pass when any flag in the mask is present in the value.
2021-07-16 11:49:50 +02:00
Timothy 371911b1b5 AK/Tests: Add test for EnumBits has_flag()
This test requires that all values in the mask are present in the value
as well.
2021-07-16 11:49:50 +02:00
Ali Mohammad Pur 5d170810db AK: Make JsonParser correctly parse unsigned values larger than u32
Prior to this, it'd try to stuff them into an i64, which could fail and
give us nothing.
Even though this is an extension we've made to JSON, the parser should
be able to correctly round-trip from whatever our serialiser has
generated.
2021-07-15 01:47:35 +02:00
Max Wipfli 2404ad6897 LibWeb: Fix assertion failure when tokenizing JS regex literals
This fixes parsing the following regular expression: /</g;

It also adds a simple script element to the HTMLTokenizer regression
test, which also contains that specific regex.
2021-07-15 01:47:22 +02:00
Max Wipfli a9a54914bf Tests: Add comments to the HTMLTokenizer regression test file 2021-07-15 00:48:45 +02:00
Max Wipfli 5a44a0b9f4 Tests: Add a basic test suite for HTMLTokenizer
The test suite includes a few basic tests and a very crude regression
test, which just concatenates the to_string() of all tokens and checks
the String's hash to be equal. This relies on the format of
HTMLToken::to_string() to stay the same, which is not ideal.
2021-07-14 23:03:36 +02:00
Daniel Bertalan 6821cd45ed Tests: Fix compile errors on Clang
Since Clang enables a couple of warnings that we don't have in GCC,
these were not caught before. Included fixes:

- Use correct printf format string for `size_t`
- Don't compare Nonnull(Ref|Own)Ptr` to nullptr
- Fix unsigned int& => unsigned long& conversion
2021-07-14 13:12:25 +02:00
Hendiadyoin1 3cc59f6454 Tests: Rename write-oob
The "ue-" prefix makes it clearer that it should be run with the
UserspaceEmulator
2021-07-14 11:26:34 +04:30
Hendiadyoin1 bdde44a833 Tests: Assure that UE does not regress via ue ls 2021-07-14 11:26:34 +04:30
Andrew Kaster 2ca1193d9a Userland+Tests: Convert test_io to be LibTest based
Add a few extra tests as well, to make sure we don't have any future
TmpFs regressions.
2021-07-11 19:42:00 +02:00
Andrew Kaster 4cc75501d7 Utilities+Tests: Convert test_efault to be LibTest based
This test exposed a kernel panic in is_user_range calculations, so let's
convert it to be a LibTest test so we can prevent regressions in mmap,
the page allocator, and the memory manager.
2021-07-11 19:42:00 +02:00
Andreas Kling 88c8451973 AK: Bring back FixedArray<T>
Let's bring this class back, but without the confusing resize() API.
A FixedArray<T> is simply a fixed-size array of T.

The size is provided at run-time, unlike Array<T> where the size is
provided at compile-time.
2021-07-11 17:42:31 +02:00
Ali Mohammad Pur 1c584e9d80 LibRegex: Correctly parse BRE bracket expressions
Commonly, bracket expressions are in fact, enclosed in brackets.
2021-07-10 22:58:24 +04:30
Ali Mohammad Pur daa6d99e6e LibRegex: Add support for non-extended regular expressions in regcomp()
Fixes part of #8506.
2021-07-10 13:33:08 +02:00
Jan de Visser a034774e3a LibSQL+SQLServer: Build SQLServer system service
This patch introduces the SQLServer system server. This service is
supposed to be the only process/application talking to database storage.
This makes things like locking and caching more reliable, easier to
implement, and more efficient.

In LibSQL we added a client component that does the ugly IPC nitty-
gritty for you. All that's needed is setting a number of event handler
lambdas and you can connect to databases and execute statements on them.

Applications that wish to use this SQLClient class obviously need to
link LibSQL and LibIPC.
2021-07-08 17:55:59 +04:30
Jan de Visser 30691549fd LibSQL: Move Order and Nulls enums from SQL::AST to SQL namespace
The Order enum is used in the Meta component of LibSQL. Using this enum
meant having to include the monster AST/AST.h include file. Furthermore,
they are sort of basic and therefore can live in the general SQL
namespace. Moved to LibSQL/Type.h.

Also introduced a new class, SQLResult, which is needed in future
patches.
2021-07-08 17:55:59 +04:30
Gunnar Beutner 75ba74a216 Tests: Build all tests on x86_64
This builds some previously-disabled tests for x86_64.
2021-07-07 15:29:18 +02:00
Timothy Flynn 65003241e4 LibRegex: Allow dollar signs in ECMA262 named capture groups
Fixes 1 test262 test.
2021-07-06 22:33:17 +01:00
Andrew Kaster d0447f23b8 Tests+Base: Convert stack-smash to be LibTest based and stop skipping it
Now that the test is converted to be LibTest based, we can remove it
from the exclude list in /home/anon/.config/Tests.ini.

Prior to this it would crash and fail because it was signaled instead of
returning normally with exit code 0.
2021-07-06 17:22:45 +02:00
Andrew Kaster e4013f6cc6 Tests: Set a default font in BenchmarkGfxPainter to prevent crashes
After the changes to LibGfx to make default font management handled in
WindowServer instead of each GUI application to allow for global font
broadcasts, the two LibGfx tests broke. The non-benchmark was fixed in
8f96d2, but the benchmark was left in the dust because nobody really
runs it manually :^(
2021-07-06 17:22:45 +02:00
Idan Horowitz 53f70e5208 LibJS: Remove the default length & attributes from define_native_*
These are usually incorrect, and people sometimes forget to add the
correct values as a result of them being optional, so they should just
be specified explicitly.
2021-07-06 14:20:30 +01:00
stelar7 ce314c54bd JsonParser: Bring parser more to spec 2021-07-05 12:36:19 +02:00
Linus Groh 1ac3d253c5 LibWeb/WebAssembly+test-wasm: Use get_without_side_effects() more 2021-07-04 22:07:36 +01:00
Tobias Christiansen f35c25a7eb Tests: Add test for String::roman_number_from() 2021-07-04 22:17:03 +02:00
Gunnar Beutner 3bbe86d8ea Everywhere: Prefer using "..."sv over StringView { "..." } 2021-07-04 14:24:03 +02:00
Daniel Bertalan 2ef28602ba Tests: Add tests for Optional's conditionally trivial functions 2021-07-04 07:24:41 +04:30
Daniel Bertalan 39dd13fd17 AK: Destroy original value when assigning to Variant 2021-07-04 07:24:41 +04:30
Daniel Bertalan 9323d459b4 Tests: Add test for supported operation type traits 2021-07-04 07:24:41 +04:30
Max Wipfli 4b87dd5c5c Tests: Add test for String::find with empty needle
This adds a test case for String::find and String::find_all with empty
needles. The expected behavior is in line with what the C++ standard
library (and other languages standard libraries) expect.
2021-07-02 21:54:21 +02:00
Max Wipfli 9cc35d1ba3 AK: Implement String::find_any_of() and StringView::find_any_of()
This implements StringUtils::find_any_of() and uses it in
String::find_any_of() and StringView::find_any_of(). All uses of
find_{first,last}_of have been replaced with find_any_of(), find() or
find_last(). find_{first,last}_of have subsequently been removed.
2021-07-02 21:54:21 +02:00
Max Wipfli 3bdaed501e AK+Everywhere: Remove StringView::find_{first,last}_of(char) methods
This removes StringView::find_first_of(char) and find_last_of(char) and
replaces all its usages with find and find_last respectively. This is
because those two methods are functionally equivalent.
find_{first,last}_of should only be used if searching for multiple
different characters, which is never the case with the char argument.

This also adds the [[nodiscard]] to the remaining find_{first,last}_of
methods.
2021-07-02 21:54:21 +02:00
Spencer Dixon dd3996e207 Tests+LibThreading: Add new tests for LibThreading for detach() 2021-07-02 17:52:45 +02:00
Ali Mohammad Pur b538e15548 LibWasm: Give traps a reason and display it when needed
This makes debugging wasm code a bit easier, as we now know what fails
instead of just "too bad, something went wrong".
2021-07-02 04:53:01 +04:30
Gal Horowitz 172d81a717 LibCrypto: Add tests for SignedBigInteger bitwise operations 2021-07-01 11:37:16 +02:00
Max Wipfli d8be530397 AK+Everywhere: Remove "null state" of LexicalPath
This removes the default constructor of LexicalPath, and subsequently
modifies all its users to accommodate the change.
2021-06-30 11:13:54 +02:00
Max Wipfli 4c018909f7 AK: Make LexicalPath immutable
This replaces the current LexicalPath::append() API with a new method
that returns a new LexicalPath object and doesn't touch the this-object.
With this, LexicalPath is now immutable. It also adds a
LexicalPath::parent() method and the relevant test cases.
2021-06-30 11:13:54 +02:00
Max Wipfli 1e80022282 Tests: Rewrite tests for LexicalPath 2021-06-30 11:13:54 +02:00
Max Wipfli 9b8f35259c AK: Remove the LexicalPath::is_valid() API
Since this is always set to true on the non-default constructor and
subsequently never modified, it is somewhat pointless. Furthermore,
there are arguably no invalid relative paths.
2021-06-30 11:13:54 +02:00
Andrew Kaster 601c9e89de Userland+Tests: Split out generic test runner from JS TestRunner
Split out the functionality to gather multiple tests from the filesystem
and run them in turn into Test::TestRunner, and leave the JavaScript
specific test harness logic in Test::JS::TestRunner and friends.
2021-06-30 08:18:28 +04:30
Andrew Kaster 44a6715584 Tests: TestProcFs cannot assume stdin/stdout/stderr are the same
If someone runs the test with shell redirection going on, or in a way
that changes any of the standard file descriptors this assumption will
not hold. When running from a terminal normally, it is true however.

Instead, check that /proc/self/fd/[0,1,2] are symlinks, and can be
stat-d by verifying that both stat and lstat succeed, and give different
struct stat contents.
2021-06-30 08:18:28 +04:30
Idan Horowitz 005d75656e LibCrypto: Replace from_base{2,8,10,16}() & to_base10 with from_base(N)
This allows us to support parsing and serializing BigIntegers to and
from any base N (such that 2 <= N <= 36).
2021-06-29 16:55:54 +01:00
Ali Mohammad Pur 745a1dbb5d AK: Add and use the RemoveCVReference<T> type trait 2021-06-28 01:08:41 +04:30
Andreas Kling ba9d5c4d54 LibJS: Rename Function => FunctionObject 2021-06-27 22:36:04 +02:00
Ali Mohammad Pur 37b0f55104 AK: Make the constexpr StringView methods actually constexpr
Also add some tests to ensure that they _remain_ constexpr.
In general, any runtime assertions, weirdo C casts, pointer aliasing,
and such shenanigans should be gated behind the (helpfully newly added)
AK::is_constant_evaluated() function when the intention is to write
constexpr-capable code.
a.k.a. deliver promises of constexpr-ness :P
2021-06-27 20:54:59 +01:00
Andreas Kling beb43f673e AK: Undo bogus Variant::downcast() rename
I accidentally renamed these to verify_cast() when doing the global
AK::downcast() rename.
2021-06-26 21:27:58 +02:00
Andreas Kling ee3a73ddbb AK: Rename downcast<T> => verify_cast<T>
This makes it much clearer what this cast actually does: it will
VERIFY that the thing we're casting is a T (using is<T>()).
2021-06-24 19:57:01 +02:00
Daniel Bertalan f820917a76 Everywhere: Use nothrow new with adopt_{ref,own}_if_nonnull
This commit converts naked `new`s to `AK::try_make` and `AK::try_create`
wherever possible. If the called constructor is private, this can not be
done, so we instead now use the standard-defined and compiler-agnostic
`new (nothrow)`.
2021-06-24 17:35:49 +04:30
Gunnar Beutner 6600835802 Tests: Disable kernel and LibC tests for x86_64
Some of the tests assume 32-bit addresses, so let's disable them for now.
2021-06-24 09:27:13 +02:00
Jan de Visser bd5a04fffe LibSQL: Reduce run time of TestSqlDatabase
Scanning tables is a linear process using pointers in the table's
tuples, and does not involve more 'stochastic' code paths like index
traversals. Therefore the 1000 and 10000 row tests were basically
overkill and added nothing we can't find out with less rows.
2021-06-24 09:23:14 +02:00
Max Wipfli b3c3b78b01 Tests: Reduce runtime of TestCharacterTypes
This declares all test cases which compare function outputs over the
entire Unicode range as `BENCHMARK_CASE`, to avoid them being run by CI.
This reduces runtime of TestCharacterTypes (without benchmarks) by about
one third.
2021-06-24 09:16:28 +02:00
Jan de Visser 5c4890411b LibSQL: Make lexer and parser more standard SQL compliant
SQL was standardized before there was consensus on sane language syntax
constructs had evolved. The language is mostly case-insensitive, with
unquoted text converted to upper case. Identifiers can include lower
case characters and other 'special' characters by enclosing the
identifier with double quotes. A double quote is escaped by doubling it.
Likewise, a single quote in a literal string is escaped by doubling it.

All this means that the strategy used in the lexer, where a token's
value is a StringView 'window' on the source string, does not work,
because the value needs to be massaged before being handed to the
parser. Therefore a token now has a String containing its value. Given
the limited lifetime of a token, this is acceptable overhead.

Not doing this means that for example quote removal and double quote
escaping would need to be done in the parser or in AST node
construction, which would spread lexing basically all over the place.
Which would be suboptimal.

There was some impact on the sql utility and SyntaxHighlighter component
which was addressed by storing the token's end position together with
the start position in order to properly highlight it.

Finally, reviewing the tests for parsing numeric literals revealed an
inconsistency in which tokens we accept or reject: `1a` is accepted but
`1e` is rejected. Related to this is the fate of `0x`. Added a FIXME
reminding us to address this.
2021-06-24 00:36:53 +02:00
Jan de Visser 4198f7e1af LibSQL: Move Lexer and Parser machinery to AST directory
The SQL engine is expected to be a fairly sizeable piece of software.
Therefore we're starting to restructure the codebase for growth.
2021-06-24 00:36:53 +02:00
coderdreams 49340f98f7 LibSQL: Create databases in writable directory 2021-06-22 18:54:40 +04:30
coderdreams 9ffb3e7e30 LibCore: Add unit test for File::read_line 2021-06-22 18:54:40 +04:30
Ali Mohammad Pur 9c5d38b7db Meta+LibWasm: Add support for module linking tests
This commit makes the linking tests in the wasm spec test run.
2021-06-22 00:26:25 +04:30
Jan de Visser 87bd69559f LibSQL: Database layer
This patch implements the beginnings of a database API allowing for the
creation of tables, inserting rows in those tables, and retrieving those
rows.
2021-06-19 22:06:45 +02:00
Jan de Visser 267eb3b329 LibSQL: Hash index implementation for the SQL storage layer
This patch implements a basic hash index. It uses the extendible hashing
algorith. Also includes a test file.
2021-06-19 22:06:45 +02:00
Jan de Visser 224804b424 LibSQL: BTree index, Heap, and Meta objects for SQL Storage layer
Unfortunately this patch is quite large.

The main functionality included are a BTree index implementation and
the Heap class which manages persistent storage.

Also included are a Key subclass of the Tuple class, which is a
specialization for index key tuples. This "dragged in" the Meta layer,
which has classes defining SQL objects like tables and indexes.
2021-06-19 22:06:45 +02:00
Jan de Visser 2a46529170 LibSQL: Basic dynamic value classes for SQL Storage layer
This patch adds the basic dynamic value classes used by the SQL Storage
layer. The most elementary class is Value, which holds a typed Value
which can be converted to standard C++ types. A Tuple is a collection
of Values described by a TupleDescriptor, which specifies the names,
types, and ordering of the elements in the Tuple.

Tuples and Values can be serialized and deserialized to and from
ByteBuffers. This is mechanism which is used to save them to disk.

Tuples are used as keys in SQL indexes and rows in SQL tables.

Also included is a test file.
2021-06-19 22:06:45 +02:00
Idan Horowitz 5e53a690ac AK: Add support for keeping trailing zeros in fixed precision floats
This uses the same syntax as zero padding integers:
String::formatted("{:0.5}", 1.234) => "1.23400"
2021-06-19 16:13:59 +01:00
Peter Bocan 4d5ffd364a LibCrypto+LibTLS: Split and move test suite into Tests directory
This change splits test-crypto.cpp from Userland into separate test
suites located in Tests/ directory.
2021-06-19 19:05:36 +04:30
Kyle Ambroff-Kao d173945dad Tests: Add a test for ProcFS fd interaction
Co-authored-by: Tim Schumacher <timschumi@gmx.de>
2021-06-18 10:15:14 +02:00
Ali Mohammad Pur ac6a3d068d AK: Add some tests for hexdump formatting 2021-06-17 18:44:00 +04:30
Gunnar Beutner 631d36fd98 Everywhere: Add component declarations
This adds component declarations so that users can select to not build
certain parts of the OS.
2021-06-17 11:03:51 +02:00
sin-ack 9a9e7f03f2 Tests: Add test for case-insensitive matching 2021-06-16 16:30:12 +04:30
Brian Gianforcaro b7f8343f87 AK+Tests: Add IntrusiveList<T,...>::insert_before(..) method
The insert_before method on AK::InlineLinkedList is used, so in order to
achieve feature parity, we need to implement it for AK::IntrusiveList as
well.
2021-06-16 10:40:01 +02:00
Idan Horowitz 8c7fe8d6c8 AK: Add support for removing SinglyLinkedList nodes during iteration
This commit also fixes the now-broken usage of SinglyLinkedList::remove
in the Piano application.
2021-06-15 23:59:21 +01:00
Jelle Raaijmakers 10e8b99038 LibC: Make getopt modify argv again
A POSIX-compatibility fix was introduced in 64740a0214 to make the
compilation of the `diffutils` port work, which expected a
`char* const* argv` signature.

And indeed, the POSIX spec does not mention permutation of `argv`:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html

However, most implementations do modify `argv` as evidenced by
documentation such as:
https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic
    /LSB-Core-generic/libutil-getopt-3.html

  "The function prototype was aligned with POSIX 1003.1-2008 (ISO/IEC
   9945-2009) despite the fact that it modifies argv, and the library
   maintainers are unwilling to change this."

Change the behavior back to permutate `argc` to allow for the following
command line argument order to work again:

  unzip ./file.zip -o target-dir

Without this change, `./file.zip` in the example above would have been
ignored completely.
2021-06-13 17:05:03 +01:00
Andreas Kling dc65f54c06 AK: Rename Vector::append(Vector) => Vector::extend(Vector)
Let's make it a bit more clear when we're appending the elements from
one vector to the end of another vector.
2021-06-12 13:24:45 +02:00
Idan Horowitz 77c2db4183 LibJS: Add all of the WeakMap.prototype methods (delete, get, has, set) 2021-06-12 10:44:28 +01:00
Hendiadyoin1 5ffe23e4f3 AK+LibX86: Generalize u128/256 to AK::UFixedBigInt
Doing these as custom classes might be faster, especially when writing
them in SSE, but this would cause a lot of Code duplication and due to
the nature of constexprs and the intelligence of the compiler they might
be using SSE/MMX either way
2021-06-11 18:14:11 +04:30
Idan Horowitz a00d154522 LibJS: Notify WeakSets when heap cells are sweeped
This is an implementation of the following optional optimization:
https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 21:52:25 +01:00
Ali Mohammad Pur 3d94b5051d AK: Make Vector capable of holding reference types
This commit makes it possible to instantiate `Vector<T&>` and use it
to store references to `T` in a vector.
All non-pointer observers are made to return the reference, and the
pointer observers simply yield the underlying pointer.
Note that the 'find_*' methods act on the values and not the pointers
that are stored in the vector.
This commit also makes errors in various vector methods much more
readable by directly using requires-clauses on them.
And finally, it should be noted that Vector cannot hold temporaries :^)
2021-06-08 19:14:24 +02:00
Timothy Flynn c7cd81bce8 LibSQL: Limit the number of nested subqueries
SQLite hasn't documented a limit on https://www.sqlite.org/limits.html
for the maximum number of nested subqueries. However, its parser is
generated with Yacc and has an internal limit of 100 for general nested
statements.

Fixes https://crbug.com/oss-fuzz/35022.
2021-06-08 19:08:13 +02:00
Timothy Flynn 4e974e6d60 LibSQL: Rename expression tree depth limit test case
Meant to rename this before committing the test - 'stack_limit' isn't a
great name when there's multiple test cases for various stack overflows.
2021-06-08 19:08:13 +02:00
Jelle Raaijmakers d7126fbbc2 LibCore/ArgsParser: Learn how to stop on first non-option
We need this for utilities like `env`, that do not gain anything by
parsing the options passed to the command they are supposed to
execute.
2021-06-08 11:30:58 +02:00
Jelle Raaijmakers 0b0bce78f6 LibCore/ArgsParser: Add test suite
This adds a very basic test suite for ArgsParser that we can use to set
a baseline of functionality that we want to make sure keeps working.
2021-06-08 11:30:58 +02:00
Brian Gianforcaro f5e04759cc AK: Add IntrusiveList::size_slow() to match InlineLinkedList
The functionality is needed to replace InlineLinkedList with
IntrusiveList in the Kernel Process class.
2021-06-07 09:42:55 +02:00
Ali Mohammad Pur 51c2c69357 AK+Everywhere: Disallow constructing Functions from incompatible types
Previously, AK::Function would accept _any_ callable type, and try to
call it when called, first with the given set of arguments, then with
zero arguments, and if all of those failed, it would simply not call the
function and **return a value-constructed Out type**.
This lead to many, many, many hard to debug situations when someone
forgot a `const` in their lambda argument types, and many cases of
people taking zero arguments in their lambdas to ignore them.
This commit reworks the Function interface to not include any such
surprising behaviour, if your function instance is not callable with
the declared argument set of the Function, it can simply not be
assigned to that Function instance, end of story.
2021-06-06 00:27:30 +04:30
Timothy Flynn f8f36effc9 LibSQL: Limit the allowed depth of an expression tree
According to the definition at https://sqlite.org/lang_expr.html, SQL
expressions could be infinitely deep. For practicality, SQLite enforces
a maxiumum expression tree depth of 1000. Apply the same limit in
LibSQL to avoid stack overflow in the expression parser.

Fixes https://crbug.com/oss-fuzz/34859.
2021-06-05 23:48:18 +04:30
Sahan Fernando d02e7b3811 LibWasm: Move Wasm::BytecodeInterpreter into its own header 2021-06-05 14:31:54 +04:30
Max Wipfli 99d5555134 AK: Do not trim away non-ASCII bytes when parsing URL
Because non-ASCII code points have negative byte values, trimming away
control characters requires checking for negative bytes values.

This also adds a test case with a URL containing non-ASCII code points.
2021-06-05 10:53:31 +02:00
Ali Mohammad Pur be62e4d1d7 LibWasm: Load and instantiate tables
This commit is a fairly large refactor, mainly because it unified the
two different ways that existed to represent references.
Now Reference values are also a kind of value.
It also implements a printer for values/references instead of copying
the implementation everywhere.
2021-06-04 16:07:42 +04:30
R Smith 5a6f0ef1bc AK: Don’t drop lines between \r and \n in StringView::lines() (#7662)
StringView::lines() supports line-separators “\n”, “\r”, and “\r\n”.
The method will drop an entire line if it is surrounded by “\r”
and “\n” separators on the left and right sides respectively.
2021-06-04 12:06:08 +04:30
DexesTTP e01f1c949f AK: Do not VERIFY on invalid code point bytes in UTF8View
The previous behavior was to always VERIFY that the UTF-8 bytes were
valid when iterating over the code points of an UTF8View. This change
makes it so we instead output the 0xFFFD 'REPLACEMENT CHARACTER'
code point when encountering invalid bytes, and keep iterating the
view after skipping one byte.

Leaving the decision to the consumer would break symmetry with the
UTF32View API, which would in turn require heavy refactoring and/or
code duplication in generic code such as the one found in
Gfx::Painter and the Shell.

To make it easier for the consumers to detect the original bytes, we
provide a new method on the iterator that returns a Span over the
data that has been decoded. This method is immediately used in the
TextNode::compute_text_for_rendering method, which previously did
this in a ad-hoc waay.

This also add tests for the new behavior in TestUtf8.cpp, as well
as reinforcements to the existing tests to check if the underlying
bytes match up with their expected values.
2021-06-03 18:28:27 +04:30
Max Wipfli 1c9d87c455 Tests: Add tests for most functions in AK/CharacterType.h 2021-06-03 13:31:46 +02:00
Brian Gianforcaro ce74fce0df Tests: Add test coverage for AK::IntrusiveList reverse iterator support 2021-06-03 13:27:40 +02:00
Timothy Flynn a870eac0eb LibSQL: Report a syntax error for unsupported LIMIT clause syntax
Rather than aborting when a LIMIT clause of the form 'LIMIT expr, expr'
is encountered, fail the parser with a syntax error. This will be nicer
for the user and fixes the following fuzzer bug:
https://crbug.com/oss-fuzz/34837
2021-06-03 08:30:13 +02:00
Ali Mohammad Pur b15a5d6ada LibWasm: Ensure that value signs are preserved when casting
Also makes normal arithmetic operations more spec-compliant by actually
ignoring overflow on them.
2021-06-02 16:09:16 +04:30
Ali Mohammad Pur 56bf80251c LibWasm: Implement reference instructions (ref.{null,func,is_null}) 2021-06-02 16:09:16 +04:30
Timothy Flynn ab79599a5e LibSQL: Return an error for empty common table expression lists
SQL::CommonTableExpressionList is required to be non-empty. Return an
error if zero common table expressions were parsed.

Fixes #7627
2021-06-01 23:48:21 +04:30
Ali Mohammad Pur 944855ca18 AK+Everywhere: Fix compiletime format parsing of replacement fields 2021-06-01 23:12:17 +04:30
Andreas Kling c0d1a75881 AK: Strip leading/trailing C0-control-or-space in URLs correctly
We have to stop scanning once we hit a non-strippable character.
Add some tests to cover this.
2021-06-01 13:22:04 +02:00
Max Wipfli f3fda59abd AK: Enable direct comparsion of Optional<T> and T
This patch introduces a new operator== to compare an Optional to its
contained type directly. If the Optional does not contain a value, the
comparison will always return false.

This also adds a test case for the new behavior as well as comparison
between Optional objects themselves.
2021-06-01 11:38:17 +02:00
Andreas Kling 407d6cd9e4 AK: Rename Utf8CodepointIterator => Utf8CodePointIterator 2021-06-01 09:45:52 +02:00
Max Wipfli c7857f3572 Tests: Add more tests for AK::URL
This adds more tests for AK::URL. Furthermore, this also changes some
tests to conform to what the reworked URL class does (and the URL
specification mostly expects).
2021-06-01 09:28:05 +02:00
Max Wipfli 14506e8f5e AK: Implement Utf8CodepointIterator::peek(size_t)
This adds a peek method for Utf8CodepointIterator, which enables it to
be used in some parsing cases where peeking is necessary.

peek(0) is equivalent to operator*, expect that peek() does not contain
any assertions and will just return an empty Optional<u32>.

This also implements a test case for iterating UTF-8.
2021-06-01 09:28:05 +02:00
Andrew Kaster 0af192ff8d AK: Handle LEB128 encoded values that are too large for the result type
Previously, we would go crazy and shift things way out of bounds.
Add tests to verify that the decoding algorithm is safe around the
limits of the result type.
2021-05-31 14:25:27 +04:30
Andrew Kaster c59cf0da07 AK: Add tests for LEB128 decoder 2021-05-31 14:25:27 +04:30
Andrew Kaster 7b4dc590e7 AK+Userland: Use akaster@serenityos.org for my copyright headers 2021-05-30 14:35:34 +01:00
Andreas Kling c584421592 AK: Make HashTable::operator=(HashTable&&) clear the moved-from table
This is consistent with how other AK containers behave when moved from.
2021-05-30 14:34:32 +02:00
Ali Mohammad Pur f387da4a90 LibTest+test-js: Add back the lost test262 parser test option
Fixes #7566.
2021-05-30 10:34:44 +01:00
Ali Mohammad Pur 90de1ded55 Kernel: Ensure that an unveil node with no permission is never accepted
Otherwise nodes inheriting from root may still be accessed with
`access(..., F_OK)`.
Also adds a test case to TestKernelUnveil about this behaviour.
2021-05-29 22:05:34 +02:00
Tim Schumacher e87a9a7f9f Tests: Add tests for LexicalPath dirname handling 2021-05-27 18:21:36 +04:30
Andrew Kaster a223ef3c4f Tests: Use ByteBuffer::create_zeroed in TestDeflate instead of memset
The round trip compress test wants the first half of the byte buffer to
be filled with random data, and the second half to be all zeroes. The
strategy of using memset on ByteBuffer::offset_pointer confuses
__builtin_memset_chk when building with -fsanitize=undefined. It thinks
that the buffer is using inline capacity when we can prove to ourselves
pretty easily that it's not. To avoid this, just create the buffer
zeroed to start, and then fill the first half with the random data.
2021-05-27 15:18:03 +02:00
Jesse Buhagiar 2b123cc592 AK: Implement AK::Stack 2021-05-26 16:36:53 +04:30
Gunnar Beutner c81b3e1ee3 LibC: Implement strerror_r()
This implements the XSI-compliant version of strerror_r() - as opposed
to the GNU-specific variant.

The function explicitly saves errno so as to not accidentally change it
with one of the calls to other functions.
2021-05-25 17:36:02 +02:00
Andreas Kling de395a3df2 AK+Everywhere: Consolidate String::index_of() and String::find()
We had two functions for doing mostly the same thing. Combine both
of them into String::find() and use that everywhere.

Also add some tests to cover basic behavior.
2021-05-24 11:59:18 +02:00
Ali Mohammad Pur 6b4d7b6c19 AK: Fix Variant construction from lvalue references
Fixes #7371 and appends its test cases.
2021-05-22 09:34:31 +02:00
Andreas Kling 8f96d20b86 Tests: Remove default font tests from LibGfx/TestFontHandling
The system default font functions now rely on communication with
WindowServer and so we can't really test them here.
2021-05-21 21:02:43 +02:00
Andrew Kaster 7e905ea201 Tests: Install non-LibTest based Kernel tests into Kernel/Legacy subdir
This makes it easier to run tests we know will work in CI, and ignore
ones that need some help to be repeatable.
2021-05-21 12:05:34 +04:30
Ali Mohammad Pur 3283c8a495 LibWasm: Make the instantiation process produce an OwnPtr
Managing the instantiated modules becomes a pain if they're on the
stack, since an instantiated module will eventually reference itself.
To make using this simpler, just avoid copying the instance.
2021-05-21 00:15:23 +01:00
Ali Mohammad Pur efb106069b LibWasm: Decouple ModuleInstance from the AbstractMachine
This fixes a FIXME and will allow linking only select modules together,
instead of linking every instantiated module into a big mess of exported
entities :P
2021-05-21 00:15:23 +01:00
Ali Mohammad Pur 24b2a6c93a LibWasm+Meta: Implement instantiation/execution primitives in test-wasm
This also optionally generates a test suite from the WebAssembly
testsuite, which can be enabled via passing `INCLUDE_WASM_SPEC_TESTS`
to cmake, which will generate test-wasm-compatible tests and the
required fixtures.
The generated directories are excluded from git since there's no point
in committing them.
2021-05-21 00:15:23 +01:00
Ali Mohammad Pur b3c13c3e8a LibWasm+Meta: Add test-wasm and optionally test the conformance tests
This only tests "can it be parsed", but the goal of this commit is to
provide a test framework that can be built upon :)
The conformance tests are downloaded, compiled* and installed only if
the INCLUDE_WASM_SPEC_TESTS cmake option is enabled.
(*) Since we do not yet have a wast parser, the compilation is delegated
to an external tool from binaryen, `wasm-as`, which is required for the
test suite download/install to succeed.
This *does* run the tests in CI, but it currently does not include the
spec conformance tests.
2021-05-21 00:15:23 +01:00
r-paiva edcfbdf4bd AK: Added contains_in_range to Vector
Vector::contains_in_range() allows the search of an element in a given
range on a vector object.
Also added testcases for the new Vector method.
2021-05-20 23:53:06 +02:00
Andreas Kling 8a6c37deef LibGfx: Remove Gfx::FontDatabase::default_bold_fixed_width_font()
Ask for a bold_variant() of the default_fixed_width_font() instead.
2021-05-20 20:55:29 +02:00
Andreas Kling 6a012ad79f LibGfx: Remove Gfx::FontDatabase::default_bold_font()
Instead use default_font().bold_variant() in cases where we want a bold
variant of the default font. :^)
2021-05-20 20:55:29 +02:00
Itamar 832e9c6e02 LibCpp: Add regression tests for the parser
For each .cpp file in the test suite data, there is a .ast file that
represents the "known good" baseline of the parser result.

Each .cpp file goes through the parser, and the result of
invoking `ASTNode::dump()` on the root node is compared to the
baseline to find regressions.

We also check that there were no parser errors when parsing the .cpp
files.
2021-05-19 23:19:07 +02:00
Lenny Maiorani 2b64d163cd Tests: static vs non-static constexpr variables
Problem:
- `static` variables consume memory and sometimes are less
  optimizable.
- `static const` variables can be `constexpr`, usually.
- `static` function-local variables require an initialization check
  every time the function is run.

Solution:
- If a global `static` variable is only used in a single function then
  move it into the function and make it non-`static` and `constexpr`.
- Make all global `static` variables `constexpr` instead of `const`.
- Change function-local `static const[expr]` variables to be just
  `constexpr`.
2021-05-19 21:21:03 +01:00
Maciej Zygmanowski 80077cea86 AK: Add String::find_all() and String::count() 2021-05-19 20:51:51 +01:00
Gunnar Beutner d954c11f66 Everywhere: Add missing includes for <AK/OwnPtr.h>
Previously <AK/Function.h> also included <AK/OwnPtr.h>. That's about to
change though. This patch fixes a few build problems that will occur
when that change happens.
2021-05-19 21:36:57 +02:00
Timothy Flynn 145e246a5e AK: Allow AK::Variant::visit to return a value
This changes Variant::visit() to forward the value returned by the
selected visitor invocation. By perfectly forwarding the returned value,
this allows for the visitor to return by value or reference.

Note that all provided visitors must return the same type - the compiler
will otherwise fail with the message: "inconsistent deduction for auto
return type".
2021-05-19 20:41:09 +02:00
Ali Mohammad Pur d897abf4c2 LibWeb: Implement test-web in terms of LibTest/JavaScriptTestRunner
This deduplicates the test-js copy-ism :^)
2021-05-18 18:48:15 +01:00
Ali Mohammad Pur f137c1bfaa LibJS+LibTest: Move out the test-js test runner into LibTest 2021-05-18 18:48:15 +01:00
Hediadyoin1 5a8c220101 Kernel: Add a test for multi-region mprotect 2021-05-18 16:50:52 +02:00
Gunnar Beutner c99fd217e2 AK: Make LexicalPath handle relative paths correctly
Previously LexicalPath would consider "." and ".." as equivalent to
"/". This is not true though.
2021-05-18 08:11:21 +02:00
Andrew Kaster 73adbb319c AK: Don't read past the end in BitmapView::count_in_range()
The current code is factored such that reads to the entirety of the last
byte should be dropped. This was relying on the fact that last would be
one past the end in that case. Instead of actually reading that byte
when it's completely out of bounds of the bitmask, just skip reads that
would be invalid. Add more tests to make sure that the behavior is
correct for byte aligned reads of byte aligned bitmaps.
2021-05-16 21:58:14 +01:00
Gunnar Beutner fcaf98361f AK: Turn ByteBuffer into a value type
Previously ByteBuffer would internally hold a RefPtr to the byte
buffer and would behave like a reference type, i.e. copying a
ByteBuffer would not create a duplicate byte buffer, but rather
two objects which refer to the same internal buffer.

This also changes ByteBuffer so that it has some internal capacity
much like the Vector<T> type. Unlike Vector<T> however a byte
buffer's data may be uninitialized.

With this commit ByteBuffer makes use of the kmalloc_good_size()
API to pick an optimal allocation size for its internal buffer.
2021-05-16 17:49:42 +02:00
Brendan Coles 0629b4e170 Tests: Add LibELF tests 2021-05-15 11:02:04 +01:00
Jean-Baptiste Boric eecf7a2097 LibC: Move mman.h to sys/mman.h
POSIX mandates that it is placed there.
2021-05-14 22:24:02 +02:00
Jean-Baptiste Boric e16894af5a LibC: Do not include errno.h inside unistd.h
POSIX does not mandate this, therefore let's not do it.
2021-05-14 22:24:02 +02:00
Jean-Baptiste Boric 5a0468c21f Userland: Migrate from arc4random_uniform() to get_random_uniform() 2021-05-14 22:24:02 +02:00
Andrew Kaster e96451edc9 Tests: Don't use TestRunners after their scope ends in test-js
The TestRunner objects at the end of test-js are destroyed after the
if/else that chooses whether to run the 262 parser tests or the standard
tests. Accessing TestRunner::the() after the lifetime of the TestRunners
ends is UB, so return the Test::Counts from run() instead. Also, fix the
destructor of TestRunner to set s_the to nullptr so that if anyone tries
this type of shenanigains again, they'll get a crash :^).
2021-05-14 08:34:00 +01:00
Andrew Kaster e1b8a2e517 Tests: Mark use-after-scope NeverDestroyed test NO_SANITIZE_ADDRESS
The should_not_destroy test case intentionally performs an invalid stack
access on a NeverDestroyed to confirm that the destructor for the held
type was not called.
2021-05-14 08:34:00 +01:00
Andrew Kaster 55d338b66f Tests: Free all memory allocated with regcomp in RegexLibC tests
The C interface (posix interface?) for regexes has no "initialize"
function, only a free function. The comment in regcomp in
LibRegex/C/Regex.cpp notes that calling regcomp without a regfree is an
error, and will leak memory. Every single time regcomp is called on a
regex_t*, it will allocate new memory.

Make sure that all the regcomp calls are paired with a regfree in the
tests program
2021-05-14 08:34:00 +01:00
Andrew Kaster 09fe9f4542 Tests: Fix use-after-free in TestRefPtr.self_observers
We can't unref an object to destruction while there's still a live
RefPtr to the object, otherwise the RefPtr destructor will try to
destroy it again, accessing the refcount of a destroyed object (before
realizing that oops! the object is already dead)
2021-05-14 08:34:00 +01:00
Brian Gianforcaro d07309a180 AK: Introduce adopt_ref_if_nonnull(..) to aid in Kernel OOM hardening
Unfortunately adopt_ref requires a reference, which obviously does not
work well with when attempting to harden against allocation failure.
The adopt_ref_if_nonnull() variant will allow you to avoid using bare
pointers, while still allowing you to handle allocation failure.
2021-05-13 08:29:01 +02:00
sin-ack 60eb4adac2 Tests: Add InodeWatcher and FileWatcher tests
This patch adds some rudimentary tests for InodeWatcher.  It tests the
basic functionality, but maybe there are corner cases I haven't caught.
Additionally, this is our first LibCore test. :^)
2021-05-12 22:38:20 +02:00
Ali Mohammad Pur a91a49337c LibCore+Everywhere: Move OpenMode out of IODevice
...and make it an enum class so people don't omit "OpenMode".
2021-05-12 11:00:45 +01:00
Ali Mohammad Pur 02de813950 AK: Add a Tuple implementation
Please don't use this outside of metaprogramming needs, *please*.
2021-05-11 14:09:17 +01:00
Ali Mohammad Pur 4fdbac236d AK/Variant: Deduplicate the contained types
This allows the construction of `Variant<int, int, int>`.
While this might not seem useful, it is very useful for making variants
that contain a series of member function pointers, which I plan to use
in LibGL for glGenLists() and co.
2021-05-11 14:09:17 +01:00
Brian Gianforcaro 10a594e6fe Tests: Fix install of test-js and test-web
When these were moved, there was a copy paste bug in the install
directives of both of these binaries.
2021-05-09 11:04:38 +02:00
Brian Gianforcaro 9720ad3901 Tests: Move Userland/Utilities/test-js to Tests/LibJS 2021-05-08 00:04:10 +01:00
Brian Gianforcaro b390554ad8 Tests: Move Userland/Utilities/test-web to Tests/LibWeb 2021-05-08 00:04:10 +01:00
Valtteri Koskivuori 1069979ddf AK: Implement Span::starts_with()
Useful for checking for contents at the start of a span.
2021-05-07 11:46:53 +01:00
Ali Mohammad Pur aacbee8ed8 Tests: Add tests for Checked<T>::div() overflow 2021-05-07 09:26:11 +02:00
Brian Gianforcaro 6e918e4e02 Tests: Move LibRegex tests to Tests/LibRegex 2021-05-06 17:54:28 +02:00
Brian Gianforcaro 070cba9b0d Tests: Move LibCompress tests to Tests/LibCompress 2021-05-06 17:54:28 +02:00
Brian Gianforcaro 597de3356f Tests: Move LibSQL tests to Tests/LibSQL 2021-05-06 17:54:28 +02:00
Brian Gianforcaro 67322b0702 Tests: Move AK tests to Tests/AK 2021-05-06 17:54:28 +02:00
Brian Gianforcaro fd0dbd1ebf Tests: Establish root Tests directory, move Userland/Tests there
With the goal of centralizing all tests in the system, this is a
first step to establish a Tests sub-tree. It will contain all of
the unit tests and test harnesses for the various components in the
system.
2021-05-06 17:54:28 +02:00
Ben Wiederhake 29eceebdbf Tests: Build automatically, fix compilation errors 2020-08-02 17:15:36 +02:00
AnotherTest 1ad51325ad Kernel+LibC: Implement 'memmem'
This commit adds an implementation of memmem, using the Bitap text
search algorithm for needles smaller than 32 bytes, and a naive loop
search for longer needles.
2020-08-01 08:39:26 +02:00
Sahan Fernando 0ba9651e6e LibC: Replace Berkley's qsort() with AK::dual_pivot_quick_sort() wrapper 2020-07-03 19:29:36 +02:00
Ben Wiederhake 4bff3defa8 LibC: strtod accuracy tests no longer need to skip 2020-05-11 10:52:24 +02:00
Ben Wiederhake c19d5943f1 LibC: Demonstrate strtod inaccuracy
See also #1979.
2020-05-11 10:52:24 +02:00
Ben Wiederhake b24cfd36ae Kernel: Demonstrate kernel crash on invalid fcntl 2020-05-03 22:46:28 +02:00
Michael Lelli 58a34fbe09
Kernel: Fix pledge syscall applying new pledges when it fails (#2076)
If the exec promises fail to apply, then the normal promises should
not apply either. Add a test for this fixed functionality.
2020-05-03 00:41:18 +02:00
Brian Gianforcaro 60fc939e81 Tests: Add test case for pthread_cond_timedwait with a timeout.
Add a test case that the timeout argument to pthread_cond_timedwait
works in LibPthread. This change also validates the new support for
timeouts to the futex syscall, as that's how condition variables are
implemented.
2020-04-26 21:31:52 +02:00
Ben Wiederhake 28e1da344d Kernel: Demonstrate race condition in clock_nanosleep
This adds a test for the race condition in clock_nanosleep.
The crux is that clock_nanosleep verifies that the output buffer
is writable *before* sleeping, and writes to it *after* sleeping.
In the meantime, a concurrent thread can make the output buffer
unwritable, e.g. by deallocating it.

This testcase is needlessly complex because pthread_kill is
not implemented yet.  I tried to keep it as simple as possible.

Here is the relevant part of dmesg:
[nanosleep-race-outbuf-munmap(22:22)]: Unblock nanosleep-race-outbuf-munmap(20:20) due to signal
nanosleep-race-outbuf-munmap(20:20) Unrecoverable page fault, write to address 0x02130016
CRASH: Page Fault. Process: nanosleep-race-outbuf-munmap(20)
[nanosleep-race-outbuf-munmap(20:20)]: 0xc01160ff  memcpy +44
[nanosleep-race-outbuf-munmap(20:20)]: 0xc014de64  Kernel::Process::crash(int, unsigned int) +782
[nanosleep-race-outbuf-munmap(20:20)]: 0xc01191b5  illegal_instruction_handler +0
[nanosleep-race-outbuf-munmap(20:20)]: 0xc011965b  page_fault_handler +649
[nanosleep-race-outbuf-munmap(20:20)]: 0xc0117233  page_fault_asm_entry +22
[nanosleep-race-outbuf-munmap(20:20)]: 0xc011616b  copy_to_user +102
[nanosleep-race-outbuf-munmap(20:20)]: 0xc015911f  Kernel::Process::sys(Kernel::Syscall::SC_clock_nanosleep_params const*) +457
[nanosleep-race-outbuf-munmap(20:20)]: 0xc015daad  syscall_handler +1130
[nanosleep-race-outbuf-munmap(20:20)]: 0xc015d597  syscall_asm_entry +29
[nanosleep-race-outbuf-munmap(20:20)]: 0x08048437  main +146
[nanosleep-race-outbuf-munmap(20:20)]: 0x08048573  _start +94

Most importantly, note that it crashes *inside*
Kernel::Process::sys.
Instead, the correct behavior is to return -EFAULT.
2020-03-03 20:13:32 +01:00
Andreas Kling 998765a7a6 LibC: The exec() family of functions should not search "." by default
We should only execute the filename verbatim if it contains a slash (/)
character somewhere. Otherwise, we need to look through the entries in
the PATH environment variable.

This fixes an issue where you could easily "override" system programs
by placing them in a directory you control, and then waiting for
someone to come there and run e.g "ls" :^)

Test: LibC/exec-should-not-search-current-directory.cpp
2020-02-01 16:14:09 +01:00
Andreas Kling c44b4d61f3 Kernel: Make Inode::lookup() return a RefPtr<Inode>
Previously this API would return an InodeIdentifier, which meant that
there was a race in path resolution where an inode could be unlinked
in between finding the InodeIdentifier for a path component, and
actually resolving that to an Inode object.

Attaching a test that would quickly trip an assertion before.

Test: Kernel/path-resolution-race.cpp
2020-02-01 10:56:17 +01:00
Andreas Kling c17f80e720 Kernel: AnonymousVMObject::create_for_physical_range() should fail more
Previously it was not possible for this function to fail. You could
exploit this by triggering the creation of a VMObject whose physical
memory range would wrap around the 32-bit limit.

It was quite easy to map kernel memory into userspace and read/write
whatever you wanted in it.

Test: Kernel/bxvga-mmap-kernel-into-userspace.cpp
2020-01-28 20:48:07 +01:00
Sergey Bugaev 6466c3d750 Kernel: Pass correct permission flags when opening files
Right now, permission flags passed to VFS::open() are effectively ignored, but
that is going to change.

* O_RDONLY is 0, but it's still nicer to pass it explicitly
* POSIX says that binding a Unix socket to a symlink shall fail with EADDRINUSE
2020-01-18 23:51:22 +01:00
Andreas Kling 862b3ccb4e Kernel: Enforce W^X between sys$mmap() and sys$execve()
It's now an error to sys$mmap() a file as writable if it's currently
mapped executable by anyone else.

It's also an error to sys$execve() a file that's currently mapped
writable by anyone else.

This fixes a race condition vulnerability where one program could make
modifications to an executable while another process was in the kernel,
in the middle of exec'ing the same executable.

Test: Kernel/elf-execve-mmap-race.cpp
2020-01-18 23:40:12 +01:00
Andreas Kling c6e552ac8f Kernel+LibELF: Don't blindly trust ELF symbol offsets in symbolication
It was possible to craft a custom ELF executable that when symbolicated
would cause the kernel to read from user-controlled addresses anywhere
in memory. You could then fetch this memory via /proc/PID/stack

We fix this by making ELFImage hand out StringView rather than raw
const char* for symbol names. In case a symbol offset is outside the
ELF image, you get a null StringView. :^)

Test: Kernel/elf-symbolication-kernel-read-exploit.cpp
2020-01-16 22:11:31 +01:00
Andreas Kling 8c5cd97b45 Kernel: Fix kernel null deref on process crash during join_thread()
The join_thread() syscall is not supposed to be interruptible by
signals, but it was. And since the process death mechanism piggybacked
on signal interrupts, it was possible to interrupt a pthread_join() by
killing the process that was doing it, leading to confusing due to some
assumptions being made by Thread::finalize() for threads that have a
pending joiner.

This patch fixes the issue by making "interrupted by death" a distinct
block result separate from "interrupted by signal". Then we handle that
state in join_thread() and tidy things up so that thread finalization
doesn't get confused by the pending joiner being gone.

Test: Tests/Kernel/null-deref-crash-during-pthread_join.cpp
2020-01-10 19:23:45 +01:00
Andreas Kling 76c20642f0 Kernel: Ignore closed fd's when considering select() unblock
This fixes a null RefPtr deref (which asserts) in the scheduler if a
file descriptor being select()'ed is closed by a second thread while
blocked in select().

Test: Kernel/null-deref-close-during-select.cpp
2020-01-09 12:36:42 +01:00
Andreas Kling fe9680f0a4 Kernel: Validate PROT_READ and PROT_WRITE against underlying file
This patch fixes some issues with the mmap() and mprotect() syscalls,
neither of whom were checking the permission bits of the underlying
files when mapping an inode MAP_SHARED.

This made it possible to subvert execution of any running program
by simply memory-mapping its executable and replacing some of the code.

Test: Kernel/mmap-write-into-running-programs-executable-file.cpp
2020-01-07 19:32:32 +01:00
Andreas Kling 5387a19268 Kernel: Make Process::file_description() vend a RefPtr<FileDescription>
This encourages callers to strongly reference file descriptions while
working with them.

This fixes a use-after-free issue where one thread would close() an
open fd while another thread was blocked on it becoming readable.

Test: Kernel/uaf-close-while-blocked-in-read.cpp
2020-01-07 15:53:42 +01:00