This changes Web::Bindings::throw_dom_exception_if_needed() to return a
JS::ThrowCompletionOr instead of an Optional. This allows callers to
wrap the invocation with a TRY() macro instead of making a follow-up
call to should_return_empty(). Further, this removes all invocations to
vm.exception() in the generated bindings.
This also required converting URLSearchParams::for_each and the callback
function it invokes to ThrowCompletionOr. With this, the ReturnType enum
used by WrapperGenerator is removed as all callers would be using
ReturnType::Completion.
On my machine, this script took about 3.4 seconds, and was responsible
for essentially all of the time taken by the precommit hook.
The script is a faithful 1:1 reimplementation, even the regexes are
identical. And yet, it takes about 0.02 seconds, making the pre-commit
hook lightning fast again. Apparently python is just faster in this
case.
Fun fact:
- Just reading all ~4000 files took bash about 1.2 seconds
- Checking the license took another 1.8 seconds in total
- Checking for math.h took another 0.4 seconds in total
- Checking for '#pragma once' took another 0.4 seconds in total
The timing is highly load-dependent, so they don't exactly add up to 3.4
seconds. However, it's good enough to determine that bash is no longer
fit for the purpose of this script.
'bootmode' now only controls which set of services are started by
SystemServer, so it is more appropriate to rename it to system_mode, and
no longer validate it in the Kernel.
Bootmode used to control framebuffers, panic behavior, and SystemServer.
This patch factors framebuffer control into a separate flag.
Note that the combination 'bootmode=self-test fbdev=on' leads to
unexpected behavior, which can only be fixed in a later commit.
This makes all pages look and feel the same, because they all use the
default CSS generated by pandoc. Also, it inserts the banner everywhere
at the top, not only into the top-level index.html.
Credit to @xSlendiX for suggesting that `-B` works here.
To ensure everything works as expected, a unit test was added with
multiple scenarios.
This binary has to have the SetUID flag, and we also bind-mount the
/usr/Tests directory to allow running of SetUID binaries.
Both at the same time because many of them call construct() in call()
and I'm not keen on adding a bunch of temporary plumbing to turn
exceptions into throw completions.
Also changes the return value of construct() to Object* instead of Value
as it always needs to return an object; allowing an arbitrary Value is a
massive foot gun.
The old versions were renamed to JS_DECLARE_OLD_NATIVE_FUNCTION and
JS_DEFINE_OLD_NATIVE_FUNCTION, and will be eventually removed once all
native functions were converted to the new format.
Adds support for methods whose last parameter is a variadic DOMString.
We constructor a Vector<String> of the remaining arguments to pass to
the C++ implementation.
As of the Clang 13 upgrade, we only need to build the toolchain once and
can use that toolchain for both x86_64 and i686. To do this, this breaks
the main Azure configuration into 3 "stages" (Lagom, Toolchain, and
Serenity), where the Serenity stage depends on the Toolchain stage.
This has the added benefit of uploading a new prebuilt toolchain cache
sooner than before, which should help alleviate pressure from PRs.
This commit updates the Clang toolchain's version to 13.0.0, which comes
with better C++20 support and improved handling of new features by
clang-format. Due to the newly enabled `-Bsymbolic-functions` flag, our
Clang binaries will only be 2-4% slower than if we dynamically linked
them, but we save hundreds of megabytes of disk space.
The `BuildClang.sh` script has been reworked to build the entire
toolchain in just three steps: one for the compiler, one for GNU
binutils, and one for the runtime libraries. This reduces the complexity
of the build script, and will allow us to modify the CI configuration to
only rebuild the libraries when our libc headers change.
Most of the compile flags have been moved out to a separate CMake cache
file, similarly to how the Android and Fuchsia toolchains are
implemented within the LLVM repo. This provides a nicer interface than
the heaps of command-line arguments.
We no longer build separate toolchains for each architecture, as the
same Clang binary can compile code for multiple targets.
The horrible mess that `SERENITY_CLANG_ARCH` was, has been removed in
this commit. Clang happily accepts an `i686-pc-serenity` target triple,
which matches what our GCC toolchain accepts.
Note our Attribute class is what the spec refers to as just "Attr". The
main differences between the existing implementation and the spec are
just that the spec defines more fields.
Attributes can contain namespace URIs and prefixes. However, note that
these are not parsed in HTML documents unless the document content-type
is XML. So for now, these are initialized to null. Web pages are able to
set the namespace via JavaScript (setAttributeNS), so these fields may
be filled in when the corresponding APIs are implemented.
The main change to be aware of is that an attribute is a node. This has
implications on how attributes are stored in the Element class. Nodes
are non-copyable and non-movable because these constructors are deleted
by the EventTarget base class. This means attributes cannot be stored in
a Vector or HashMap as these containers assume copyability / movability.
So for now, the Vector holding attributes is changed to hold RefPtrs to
attributes instead. This might change when attribute storage is
implemented according to the spec (by way of NamedNodeMap).
This adds the ParamatizedType, as `Vector<String>` doesn't encode the
full type information. It is a separate struct as you can't have
`Vector<Type>` inside of `Type`. This also makes Type RefCounted
because I had to make parse_type return a pointer to make dynamic
casting work correctly.
The reason I made it RefCounted instead of using a NonnullOwnPtr is
because it causes compiler errors that I don't want to figure out right
now.
Apparently it breaks the fuzzer build. There's probably a better fix
for this, but for now just unbreak the fuzzer build.
Keep this for non-fuzzer builds though since it's apparently a 17%
speedup for running test262 tests :^)
Lagom: Build with -fno-no-semantic-interposition
We build with this in non-lagom builds, and serenity's gcc even adds it
to its CC1_SPEC. Let's use it for lagom too.
Reduces the number of dynamic relocations in liblagom-js.so.0.0.0 (per
`objdump -R`) from 15133 to 14534, and increases its size back to 91M
(95156800 bytes), probably due to more inlining being possible.
This might help perf of lagom binaries.
We build with this in non-lagom builds, so there's no reason not
to use it in lagom builds as well.
Reduces the size of liblagom-js.so.0.0.0 from 94M to 90M
(from 98352784 to 93831056 bytes to be exact).
Typically size_t is used for indices, but we can take advantage of the
knowledge that there is approximately only 46K unique strings in the
generated UnicodeLocale.cpp file. Therefore, we can get away with using
u16 to store indices. There is a VERIFY that will fail if we ever exceed
the limits of u16.
On x86_64 builds, this reduces libunicode.so from 9.2 MiB to 7.3 MiB.
On i686 builds, this reduces libunicode.so from 3.9 MiB to 3.3 MiB.
These savings are entirely in the .rodata section of the shared library.
Note there are a couple of type differences between the spec and the IDL
file added in this commit. For example, we will need to support a type
of Variant to handle spec types such as "(double or sequence<double>)".
But for now, this allows web pages to construct an IntersectionObserver
with any valid type.
The *_from_string() and resolve_*_alias() generated methods are the last
remaining users of HashMap in the LibUnicode generated files (read: the
last methods not using compile-time structures). This converts these
methods to use an array containing pairs of hash values to the desired
lookup value.
Because this code generation is the same between GenerateUnicodeData.cpp
and GenerateUnicodeLocale.cpp, this adds a GeneratorUtil.h header to the
LibUnicode generators to contain the method that generates the methods.
This concept is not present in ECMAScript, and it bothers me every time
I see it.
It's only used by WrapperGenerator, and even there only relevant in two
places, so let's fully remove it from LibJS and use a simple ternary
expression instead:
cpp_name = js_name.is_null() && legacy_null_to_empty_string
? String::empty()
: js_name.to_string(global_object);
Previously this would generate the following code:
JS::Value foo_value;
if (!foo.is_undefined())
foo_value = foo;
Which is dangerous as we're passing an empty value around, which could
be exposed to user code again. This is fine with "= null", for which it
also generates:
else
foo_value = JS::js_null();
So, in summary: a value of type `any`, not `required`, with no default
value and no initializer from user code will now default to undefined
instead of an empty value.
Meta/Lagom/ReadMe.md never had any other name; not sure how that typo
happened.
The link to the non-existent directory is especially vexing because the
text goes on to explain that we don't want such a directory to exist.
Found by running markdown-checker, and 'wget'ing all external links.
The list-format strings used for Intl.ListFormat are small, but quite
heavily duplicated. For example, the string "{0}, {1}" appears 6,519
times. Generate unique strings for this data to avoid duplication.
In the generated UnicodeLocale.cpp file, there are 296,408 strings for
localizations of languages, territories, scripts, currencies & keywords.
Of these, only 43,848 (14.8%) are actually unique, so there are quite a
large number of duplicated strings.
This generates a single compile-time array to store these strings. The
arrays for the localizations now store an index into this single array
rather than duplicating any strings.
Some CLDR languages.json / territories.json files contain localizations
for some lanuages/territories that are otherwise not present in the CLDR
database. We already don't generate anything in UnicodeLocale.cpp for
these anomalies, but this will stop us from even storing that data in
the generator's memory.
This doesn't affect the output of the generator, but will have an effect
after an upcoming commit to unique-ify all of the strings in the CLDR.
There are only 112 code points with special casing rules, so this array
is quite small (compared to the size 34,626 UnicodeData hash map that is
also storing this data). Removing all casing rules from UnicodeData will
happen in a subsequent commit.
Currently, all casing information (simple and special) are stored in a
compile-time array of size 34,626, then statically copied to a hash map
at runtime. In an effort to reduce the resulting memory usage, store the
simple casing rules in standalone compile-time arrays. The uppercase map
is size 1,450 and the lowercase map is size 1,433. Any code point not in
a map will implicitly have an identity mapping.
It's "Clang" (capitalized). Silently building gcc if the toolchain
isn't know seems a bit unfriendly. So print a warning for this
(and for other unknown toolchains).
I used "git grep -FIn http://" to find all occurrences, and looked at
each one. If an occurrence was really just a link, and if a https
version exists, and if our Browser can access it at least as well as the
http version, then I changed the occurrence to https.
I'm happy to report that I didn't run into a single site where Browser
can't deal with the https version.
Having IDL constructors call FooWrapper::create(impl) directly was
creating a wrapper directly without telling the impl object about the
wrapper. This meant that we had wrapped C++ objects with a null
wrapper() pointer.
This introduces 3 classes: NodeList, StaticNodeList and LiveNodeList.
NodeList is the base of the static and live versions. Static is a
snapshot whereas live acts on the underlying data and thus inhibits
the same issues we have currently with HTMLCollection.
They were split into separate classes to not have them weirdly
mis-mashed together.
The create functions for static and live both return a NNRP to the base
class. This is to prevent having to do awkward casting at creation
and/or return, as the bindings expect to see the base NodeList only.
Instead of setting it to the default object prototype and then
immediately setting it again via internal_set_prototype_of, we can just
set it directly in the parent constructor call.
Until we're confident that RequestServer doesn't need this runtime debug
dump helper, it's much nicer if everyone has it built in, so they can
simply send a SIGINFO if they see it acting up.
Since we don't support IDL typedefs or unions yet, the responsibility
of verifying the type of the argument is temporarily moved from the
generated Wrapper to the implementation.
This patch makes both of these classes inherit from RefCounted and
Bindings::Wrappable, plus some minimal rejigging to allow us to keep
using them internally while also exposing them to web content.
This adds support for the [Unscopable] extended attribute to attributes
and functions.
I believe it should be applicable to all interface members, but I
haven't done that here.
Otherwise we'd end up trying to delete the wrong connection if a
connection made before us is deleted.
Fixes _some_ RequestServer spins (though not all...).
This commit also adds a small debug mechanism to RequestServer (which
can be enabled by turning REQUEST_SERVER_DEBUG on), that can dump all
the current active connections in the cache, what they're doing, and how
long they've been doing that by sending it a SIGINFO.
This currently only supports pair iterables (i.e. iterable<key, value>)
support for value iterables (i.e. iterable<value>) is left as TODO().
Since currently our cmake setup calls the WrapperGenerator separately
and unconditionally for each (hard-coded) output file iterable wrappers
have to be explicitly marked so in the CMakeLists.txt declaration, we
could likely improve this in the future by querying WrapperGenerator
for the outputs based on the IDL.
This patch essentially just splits the non return-specific logic from
generate_return_statement (i.e. the wrapping of the cpp value into
a javascript one) into a separate function generate_wrap_statement that
can be used to wrap any cpp value during wrapper generation.
This custom attribute will be used for objects that hold onto arbitrary
JS::Value's. This is needed as JS::Handle can only be constructed for
objects that implement JS::Cell, which JS::Value doesn't.
This works by overriding the `visit_edges` function in the wrapper.
This overridden function calls the base `visit_edges` and then forwards
it to the underlying implementation.
This will be used for CustomEvent, which must hold onto an arbitrary
JS::Value for it's entire lifespan.
A legacy platform object is a non-global platform object that
implements a special operation. A special operation is a getter, setter
and/or deleter. This is particularly used for old collection types,
such as HTMLCollection, NodeList, etc.
This will be used to make these spec-compliant and remove their custom
wrappers. Additionally, it will be used to implement collections that
we don't have yet, such as DOMStringMap.
This does a few things, that are hard to separate. For a while now, it's
been confuzing what `StyleValue::is_foo()` actually means. It sometimes
was used to check the type, and sometimes to see if it could return a
certain value type. The new naming scheme is:
- `is_length()` - is it a LengthStyleValue?
- `as_length()` - casts it to LengthStyleValue
- `has_length()` - can it return a Length?
- `to_length()` - gets the internal value out (eg, Length)
This also means, no more `static_cast<LengthStyleValue const&>(*this)`
stuff when dealing with StyleValues. :^)
Hopefully this will be a bit clearer going forward. There are lots of
places using the original methods, so I'll be going through them to
hopefully catch any issues.
These now crash as VM::call() uses ThrowExceptionOr<T>, which refuses to
hold an empty JS::Value as its non-exception result.
We only need to return an empty value when should_return_empty() says
so for the return value of throw_dom_exception_if_needed().
Co-authored-by: Luke Wilde <lukew@serenityos.org>
For `number` and `integer` types, you can add a range afterwards to add
a range check, using similar syntax to that used in the CSS specs. For
example:
```json
"font-weight": {
...
"valid-types": [
"number [1,1000]"
],
...
}
```
This limits any numbers to the range `1 <= n <= 1000`.
Previously, we have not been validating the values for CSS declarations
inside the Parser. This causes issues, since we should be discarding
invalid style declarations, so that previous ones are used instead. For
example, in this code:
```css
.foo {
width: 2em;
width: orange;
}
```
... the `width: orange` declaration overwrites the `width: 2em` one,
even though it is invalid. According to the spec, `width: orange` should
be rejected at parse time, and discarded, leaving `width: 2em` as the
resulting value.
Many properties (mostly shorthands) are parsed specially, and so they
are already rejected if they are invalid. But for simple properties, we
currently accept any value. With `property_accepts_value()`, we can
check if the value is valid in `parse_css_value()`, and reject it if it
is not.
We already expand shorthands in the cascade, so there's no need to
preserve them in the output.
This patch reorganizes the CSS::PropertyID enum values so that we can
easily iterate over all shorthand or longhand properties.
We bust the prebuilt cache when any header in e.g. LibC changes. Doing a
full toolchain rebuild probably isn't necessary, so this adds a separate
ccache to speed up toolchain builds.
Currently, the templated steps in Caches.yml rely on the environment
variable CCACHE_DIR being set to configure the ccache location. To
prepare for multiple ccache paths, do not rely on this environment
variable because only one ccache can use it at a time. Instead, pass
the path into the template as a parameter.
This patch adds a basic initial implementation of these API's.
Since LibWeb currently doesn't support workers, this implementation of
messaging doesn't bother with serializing and deserializing messages.
When parsing shorthand values, we'd like to use
`property_initial_value()` to get their longhand property values,
instead of hard-coding them as we currently do. That involves
recursively calling that function while the `initial_values` map is
being initialized, which causes problems because the shorthands appear
alphabetically before their longhand components, so the longhands aren't
initialized yet!
The solution here is to perform 2 passes when generating the code,
outputting properties without "longhands" first, and the rest after.
This could potentially cause issues when shorthands have multiple
levels, in particular `border` -> `border-color` -> `border-left-color`.
But, we do not currently define a default value for `border`, and
`border-color` takes only a single value, so it's fine for now. :^)
If the superbuild created the lagom binary directory, it won't set
BUILD_LAGOM=ON in the CMake cache for that binary directory. Insert a
check into build_target() to make sure that if the user explicitly asks
for the lagom target, we have all our ducks in a row.
This requires exposing the `configure` step on the `serenity`
ExternalProject in the SuperBuild CMakeLists so that we can continue to
only build the generated sources and not the entire OS.
Multi-lib distros like Gentoo and Fedora install lagom-core.so into
lagom-install/lib64 rather than lib. Set the install RPATH based on
CMAKE_INSTALL_LIBDIR to avoid the wrong path being set in the binaries.
Also apply macOS specific RPATH rules to fix the build on that platform.
Direct build commands to the SuperBuild's binary directory, and
image/run commands to the Serenity binary directory.
As a side benefit, make the lagom target only build Lagom instead of the
entire OS alongside Lagom.
Replace the old logic where we would start with a host build, and swap
all the CMake compiler and target variables underneath it to trick
CMake into building for Serenity after we configured and built the Lagom
code generators.
The SuperBuild creates two ExternalProjects, one for Lagom and one for
Serenity. The Serenity project depends on the install stage for the
Lagom build. The SuperBuild also generates a CMakeToolchain file for the
Serenity build to use that replaces the old toolchain file that was only
used for Ports.
To ensure that code generators are rebuilt when core libraries such as
AK and LibCore are modified, developers will need to direct their manual
`ninja` invocations to the SuperBuild's binary directory instead of the
Serenity binary directory.
This commit includes warning coalescing and option style cleanup for the
affected CMakeLists in the Kernel, top level, and runtime support
libraries. A large part of the cleanup is replacing USE_CLANG_TOOLCHAIN
with the proper CMAKE_CXX_COMPILER_ID variable, which will no longer be
confused by a host clang compiler.
This common strategy of having a serenity_option() macro defined in
either the Lagom or top level CMakeLists.txt allows us to do two things:
First, we can more clearly see which options are Serenity-specific,
Lagom-specific, or common between the target and host builds.
Second, it enables the upcoming SuperBuild changes to set() the options
in the SuperBuild's CMake cache and forward each target's options to the
corresponding ExternalProject.
This makes it so we don't need to specify the full path to all the
helper scripts we include() from different places in the codebase and
feels a lot cleaner.
We'll use this to prevent repeating common tool dependencies. They all
depend on LibCore and AK only. We also want to encapsulate common
install rules for them.
There are a few violations with signal handling that I won't be able to
fix it until later this week. So lets put lock rank enforcement under a
debug option for now so other folks don't hit these crashes until rank
enforcement is more fleshed out.
This namespace will be used for all interfaces defined in the URL
specification, like URL and URLSearchParams.
This has the unfortunate side-effect of requiring us to use the fully
qualified AK::URL name whenever we want to refer to the AK class, so
this commit also fixes all such references.
This lets you query if a given Quirk applies to a given PropertyID.
Currently this applies only to the "Hashless hex color" and "Unitless
length" quirks.
This removes the awkward String::replace API which was the only String
API which mutated the String and replaces it with a new immutable
version that returns a new String with the replacements applied. This
also fixes a couple of UAFs that were caused by the use of this API.
As an optimization an equivalent StringView::replace API was also added
to remove an unnecessary String allocations in the format of:
`String { view }.replace(...);`
There's only a couple of cases like this, but there are some locale
paths in the CLDR that contain variants. For example, there isn't a
en-US path, but there is a en-US-POSIX path. This interferes with the
operation to search for locales by name. The algorithm is such that
searching for en-US will not result in en-US-POSIX being found. To
resolve this, we should remove variants from the locale name.
This data informs consumers how to join lists of values. For example,
in en-US, the list ["a", "b", "c"] formatted to a string should become
"a, b, and c".
This is to simply the Default Case Conversion implementation. Otherwise,
the implementation would need to determine which special casing rule to
apply, instead of just picking the first match.
The amount of aliases in the likely-subtags dataset is quite large, so
this also needed to change the way the data is generated. Otherwise, the
compiler would complain about the size of the generated code.
Previously, a static method was generated that would effectively parse
the dataset into a HashMap of Unicode::LanguageID at runtime. We now
perform that parsing at generation-time, and instead generate an Array
of a structure similar to Unicode::LanguageID (we cannot use the same
structure because it contains String and Optional, which cannot be used
at compile-time).
This option is similar to the qgrub option, but instead of starting a
QEMU PIIX4 machine, it starts a QEMU Q35 machine, booting a grub image
disk within it.
The DOM specification says that the primary use case for these is to
give Promises abort semantics. It is also a prerequisite for Fetch,
as it is used to make Fetch abortable.
a
CLDR contains a set of likely subtag data where, given a locale, you can
resolve what is the most likely language, script, or territory of that
locale. This data is needed for resolving territory aliases. These
aliases might contain multiple territories, and we need to resolve which
of those territories is most likely correct for a locale.
Note that the likely subtag data is quite huge (a few thousand entries).
As an optimization encouraged by the spec, we only generate the smallest
subset of this data that we actually need (about 150 entries).
Most alias substitutions are "simple", meaning that alias matching is
done by examining a single locale subtag. However, there are a handful
of "complex" aliases where matching is done by examining multiple
subtags. For example, the variant subtag "lojban" causes the locale
"art-lojban" to be canonicalized to "jbo", but only when the language
subtag is "art" (i.e. this should not occur for the locale "en-lojban").
This generates a method to perform complex alias matching.
CLDR contains a set of aliases for languages, territories, etc. that no
longer are meant to be used (e.g. due to deprecation). For example, the
language "aam" is deprecated and should be canonicalized as "aas".
