Commit graph

595 commits

Author SHA1 Message Date
Andrew Kaster 891bc81f30 Meta: Remove syslog.h from clang-format exclusion list
Before Libraries was moved to Userland/Libraries syslog.h had a bunch
of manually aligned defines and array initializations.

Andreas seems to have formatted the file with clang-format as part of
that file move. Since syslog.h is now properly formatted, we don't
need to exclude it from the linter list.
2021-08-07 15:36:51 +02:00
sin-ack 0d468f2282 Kernel: Implement a ISO 9660 filesystem reader :^)
This commit implements the ISO 9660 filesystem as specified in ECMA 119.
Currently, it only supports the base specification and Joliet or Rock
Ridge support is not present. The filesystem will normalize all
filenames to be lowercase (same as Linux).

The filesystem can be mounted directly from a file. Loop devices are
currently not supported by SerenityOS.

Special thanks to Lubrsi for testing on real hardware and providing
profiling help.

Co-Authored-By: Luke <luke.wilde@live.co.uk>
2021-08-07 15:21:58 +02:00
Liav A 02c72de090 Meta: Remove legacy hardware components from Q35 machine
As this is a test machine I use personally to test "modern" hardware
setups, it feels quite comfortable to not care too much about VGA with
this type of machine.
Also, we don't actively use the IDE controller on this machine type, so
let's just remove it :^)
2021-08-07 11:49:07 +02:00
sin-ack bb609cee7f Meta: Add run-local.sh
This allows one to set their desired parameters for run.sh without the
need to set them in every terminal session or add it to the user account
shell files. If a run-local.sh file exists at the repository root and is
executable, it will be sourced. The file can contain any variables that
are expected to be set in run.sh.
2021-08-07 11:47:07 +02:00
Mandar Kulkarni 566702ffa2 Meta: Fix wrong path for AK Tests in check-ak-test-files.sh 2021-08-05 19:58:55 +02:00
Gunnar Beutner d9d2287a11 Meta: Detect nested KVM support for WSL2 and use that when available
This allows running QEMU inside WSL2 for hosts which have nested KVM
and WSLg support (e.g. Windows 11).

Running QEMU inside the WSL2 VM is slightly slower than running QEMU
on Windows, probably because of how WSLg handles screen updates.
2021-08-04 21:15:41 +02:00
Gunnar Beutner 723c89af67 Meta: Automatically enable WHPX when possible 2021-08-04 21:15:41 +02:00
Gunnar Beutner 5445155dba Meta: Auto-detect where QEMU is installed on Windows 2021-08-04 21:15:41 +02:00
Liav A cca1498e09 Meta: Make the Q35 machine more realistic
Although it is nice to test the system without too many devices, in
reality bare metal hardware is far more complex than the default skeleon
that QEMU provides. As a preparation of supporting more devices, we
need to ensure we are capable of at least booting on complex hardware
setups without easily-observable problems. Later on, this can be the
foundations of testing new drivers :^)
2021-08-03 21:08:01 +02:00
Gunnar Beutner 31655c9486 Meta: Prefer to use the 64-bit QEMU binary if that's available
Some users might not have qemu-system-i386 installed.
2021-08-03 13:59:57 +01:00
Gunnar Beutner b082488e94 Meta: Prefer to use -machine pcspk-audiodev for QEMU >=5.1
This gets rid of the following warning message from QEMU on startup:

qemu-system-i386: warning: '-soundhw pcspk' is deprecated, please set a
backend using '-machine pcspk-audiodev=<name>' instead

Fixes #4093.
2021-08-03 13:59:57 +01:00
Gunnar Beutner 74af43ed97 Meta: Improve WSL detection for the run.sh script
For users who use a custom kernel with WSL our previous method of
detecting WSL doesn't work. This new check instead detects WSL by
checking if the wslpath utility is available.
2021-08-02 11:47:29 +01:00
Andrew Kaster 40b0767d88 Meta: Add BUILD_SHARED_LIBS option for Lagom builds
This standard CMake option controls whether add_library() calls will
use STATIC or SHARED by default. The flag is set to on by default
since that's what we want for normal CI jobs and local builds and the
test262 runner, but disabled for oss-fuzz builds.

This should finally fix the oss-fuzz build after it was broken in #9017

oss-fuzz un-breakage was verified by running the following commands in
the oss-fuzz repo:

python infra/helper.py build_image serenity
python infra/helper.py build_fuzzers --sanitizer address --engine afl \
    --architecture x86_64 serenity /path/to/local/checkout/Meta/Lagom
python infra/helper.py check_build --sanitizer address --engine afl \
    --architecture x86_64 serenity
2021-08-02 09:05:28 +02:00
x-yl 8e8b1c8333 Meta: Explicitly specify the disk format in the QEMU options
Otherwise we're getting this warning:

WARNING: Image format was not specified for '_disk_image' and probing
         guessed raw. Automatically detecting the format is dangerous
         for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
2021-08-02 00:35:22 +02:00
Gunnar Beutner c73c40a21d Meta: Fix compatibility with QEMU 5.x
QEMU 5 doesn't support -machine pcspk-audiodev so we need to fall back
to using -soundhw for that.
2021-08-02 00:35:22 +02:00
Sam Atkins e54531244f LibWeb: Define proper debug symbols for CSS Parser and Tokenizer
You can now turn debug logging for them on using `CSS_PARSER_DEBUG` and
`CSS_TOKENIZER_DEBUG`.
2021-07-31 00:18:11 +02:00
Gunnar Beutner 8b2e76b838 Meta: Make audiodev detection more robust
This fixes audio on Windows as well as Linux systems which don't have
SDL.

Fixes #8903.
2021-07-30 23:51:26 +02:00
Gunnar Beutner dff8b3d2d9 Meta: Make rsync a hard dependency and remove the fallback code
Previously we'd fall back to using cp if rsync wasn't available. Not
only is this considerably slower it also breaks when some of the files
in the target directory are symlinks because cp tries to dereference
them.

Fixes #8672.
2021-07-30 23:50:29 +02:00
Timothy Flynn d485cf29d7 LibRegex+LibUnicode: Begin implementing Unicode property escapes
This supports some binary property matching. It does not support any
properties not yet parsed by LibUnicode, nor does it support value
matching (such as Script_Extensions=Latin).
2021-07-30 21:26:31 +01:00
Brian Gianforcaro a4dc1eb330 Lagom: Fix oss-fuzz build error due to CMake typo
The OSS-Fuzz build was failing with the following error:

 /usr/bin/ld: cannot find -l{}
2021-07-30 20:28:59 +01:00
Andrew Kaster 32d076ef54 Lagom: Add proper install rules
Create the proper export files to allow Lagom to be a well-behaved
ExternalProject, based on the example project from the cmake-init
project generator here:

https://github.com/friendlyanon/cmake-init-shared-static
2021-07-29 21:46:25 +01:00
Linus Groh 7392a73066 Lagom: Exclude GMLAutocompleteProvider.cpp from GML library sources 2021-07-29 21:46:25 +01:00
Andrew Kaster b6e5117f65 Lagom: Change to shared library build for all Lagom code
Split the Lagom build into shared libraries to match the Serenity build.

This reduces the cognitive load when trying to edit the Lagom CMakeLists
significantly. It also reduces the amount of source files that must be
compiled to run each test or host program significantly.

Also re-organize all the build rules into sections. And reorganize the
CMakeLists file in general.
2021-07-29 21:46:25 +01:00
Andrew Kaster de2bf3f333 Meta/LibTest: Build object libraries for test main files
By using the power of object libraries and $<TARGET_OBJECTS> we can make
sure to only build TestMain.cpp and JavaScriptTestRunnerMain.cpp once.

Previously we built these cpp files into object files once for every
single test executable. This change reduces the number of total compile
jobs in a Serenity target build by around 100.
2021-07-29 21:46:25 +01:00
Andrew Kaster af9be6e093 LibTTF/LibGfx: Remove circular dependency by merging LibTTF into LibGfx
LibTTF has a concrete dependency on LibGfx for things like Gfx::Bitmap,
and LibGfx has a concrete dependency in the TTF::Font class in
Gfx::FontDatabase. This circular dependency works fine for Serenity and
Lagom Linux builds of the two libraries. It also works fine for static
library builds on Lagom macOS builds.

However, future changes will make Lagom use shared libraries, and
circular library dependencies are not tolerated in macOS.
2021-07-29 21:46:25 +01:00
Andreas Kling b3d27c2340 LibGUI: Add GUI_HOVER_DEBUG runtime debugging flag (environment)
You can now see the outline of GUI widgets when hovering them.
For example:

    $ export GUI_HOVER_DEBUG=1
    $ FileManager

Then move the mouse around in the file manager. :^)
2021-07-28 01:01:02 +02:00
Karol Kosek 220dd28b02 Meta: Ignore everything after the git cut line in a lint-commit.sh hook
I have set up a commit.verbose variable in my git config,
which shows the patch diff on bottom of the commit message.

Unfortunately the character limit was also applied to the diff,
which meant that I got a false-positive lint error almost every time.
2021-07-27 22:35:49 +01:00
Idan Horowitz 350fb0e3db Meta: Bring lint-commit.sh up to date with the CI commit linter
This was missing 2 of the recently added checks. Also added a reminder
in the CI linter to update the Meta (commit hook) version.
2021-07-27 20:41:49 +01:00
x-yl 4bdd62939a Meta: Set -audiodev explictly
Unfortunately seems like QEMU tries to use SPICE for audio so we need to
explicitly set -audiodev to use either SDL or coreaudio (depending on
platorm)
2021-07-27 19:15:37 +02:00
Gunnar Beutner 57417a3d6e Kernel: Support loading the kernel at almost arbitrary virtual addresses
This enables further work on implementing KASLR by adding relocation
support to the pre-kernel and updating the kernel to be less dependent
on specific virtual memory layouts.
2021-07-27 13:15:16 +02:00
Timothy Flynn 98d8274040 Meta: Add LibUnicode (and its tests) to Lagom
This is primarily to allow using LibUnicode within LibJS and its REPL.

Note: this seems to be the first time that a Lagom dependency requires
generated source files. For this to work, some of Lagom's CMakeLists.txt
commands needed to be re-organized to include the CMake files that fetch
and parse UnicodeData.txt. The paths required to invoke the generator
also differ depending on what is currently building (SerenityOS vs.
Lagom as part of the Serenity build vs. a standalone Lagom build).
2021-07-26 17:03:55 +01:00
Ali Mohammad Pur 466e2a2fb7 Meta: Don't reboot on triple-fault in CI mode
Instead, just make QEMU quit immediately.
2021-07-26 02:29:25 +04:30
Thomas Wagenveld 2002b7e2e3 Meta/run.sh: Allow for overriding of QEMU ethernet device type
You can set the SERENITY_ETHERNET_DEVICE_TYPE environment variable to
pick another device type (i.e. ne2k_pci). Defaults to e1000 as before.
2021-07-24 21:28:22 +02:00
Luke 7e7c65abb6 Lagom/Fuzzers: Add fuzzer for the quoted printable decoder 2021-07-24 20:11:28 +04:30
Gunnar Beutner cc0914ae58 Meta: Add Homebrew paths for macOS on M1 to the PATH env variable 2021-07-24 14:04:13 +02:00
Andreas Kling 3f9e018d9a CrashDaemon: Remove BACKTRACE_DEBUG debugging code
This thing seems to work fine, no need to hang on to old debug code.
2021-07-22 23:34:33 +02:00
Gunnar Beutner 9031ed0b84 Meta: Add a new qextlinux target for the run.sh script
This allows testing the extlinux image with QEMU.
2021-07-20 15:12:19 +02:00
Gunnar Beutner 696e15fcd8 Meta: Update extlinux config for the recent prekernel changes 2021-07-20 15:12:19 +02:00
Gunnar Beutner 6a45ebe282 Meta: Fix syslinux detection on Arch Linux 2021-07-20 15:12:19 +02:00
Gunnar Beutner e3b7ae0c77 Meta: Remove unused bootloader_test target
Due to other changes this is now just a copy of the q35 target.
2021-07-18 22:08:03 +02:00
Gunnar Beutner 61a8f3e81a Kernel: Fix GRUB config by specifying the correct file name 2021-07-18 22:08:03 +02:00
Gunnar Beutner 7e94b090fe Kernel: Introduce basic pre-kernel environment
This implements a simple bootloader that is capable of loading ELF64
kernel images. It does this by using QEMU/GRUB to load the kernel image
from disk and pass it to our bootloader as a Multiboot module.

The bootloader then parses the ELF image and sets it up appropriately.
The kernel's entry point is a C++ function with architecture-native
code.

Co-authored-by: Liav A <liavalb@gmail.com>
2021-07-18 17:31:13 +02:00
Gunnar Beutner 98f8ecd9d2 Kernel: Split debug symbols into a separate file
This speeds up the boot process considerably when specifying the kernel
image via -initrd.
2021-07-18 17:31:13 +02:00
Riyyi 5d1676b05a Meta: Do not warn user about too modern clang-format
To prevent warnings for users of varying distributions, do not warn the
user if the clang-version is higher than the expected version.
2021-07-15 09:27:44 +02:00
Sam Atkins fa7a6fbedd Meta: Fix SPICE detection in run.sh
The previous fix did not work for me, but this does. :^)

Credit goes to @X-yl for actually figuring it out.
2021-07-14 21:17:56 +02:00
x-yl 410eb9e9ff Meta: Don't use SPICE if QEMU doesn't support it
I do seem to have a tendency for breaking everyone's builds :/
2021-07-14 17:17:06 +02:00
x-yl 6b68f16f2c Meta: Change QEMU options in run.sh to use SPICE
If QEMU has the qemu_vdagent chardev (should be present in QEMU 6.1)
then we use that as the SPICE client.

If qemu_vdagent is not present, no SPICE client will be launched by
default because it makes the display a bit choppy.

Set SERENITY_SPICE to override the default behavior and use your default
SPICE client.
2021-07-14 12:33:07 +02:00
Liav A 11fe38346c Meta: Remove options of explicit command line runners in run script
Let's remove the qcmd and q35_cmd options and instead have a simple
"q35" run option. Specifiying the kernel command line was a neat trick
I personally used for many debug sessions, but it seems better to stick
to setting it internally in the kernel or modifying the shell
SERENITY_KERNEL_CMDLINE environment variable to do this.
2021-07-13 23:24:16 +02:00
Liav A c92c944356 Meta: Tweak qemu run script to use pcie-root-ports
With this change, we use 6 PCIe root ports in the Q35 machine, and plug
the bochs-display device into one of those PCIe ports.
We plug the bochs-display as function 0 of that device, because
otherwise SeaBIOS and also the kernel will not detect its presence.
2021-07-13 23:24:16 +02:00
Idan Horowitz f09e361eef CI: Disallow spaces just before the separating colon in commit titles 2021-07-13 19:56:14 +01:00