Commit graph

95 commits

Author SHA1 Message Date
Daniel Bertalan ad649c48da Toolchain: Backport support for the mold linker
The `-fuse-ld=mold` flag was only added in the not-yet-released GCC 12.

Cherry-picked from the following upstream commit:
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=ad964f7eaef9c03ce68a01cfdd7fde9d56524868
2022-01-24 10:41:47 +00:00
Daniel Bertalan bd3ed700f2 Toolchain+Ports: Use the GNU hash format by default
This hash format offers faster symbol lookup than the System V hash.

We've been using it in all our shared libraries for a long time, but did
not have it enabled by default in our toolchain, so ports couldn't make
use of it.
2022-01-24 10:41:47 +00:00
Daniel Bertalan 5576e9c4c5 Kernel+Toolchain: Use .init_array section for global constructors
Before this change, our dynamic linker's global constructor handler
relied on the GNU linker implicitly including the content of `.ctors`
section inside `.init_array`. The mold linker does not do this, so
global constructors would fail to be called in the mold-built userland.

There is no point in sticking to `.ctors`, as most other systems already
use the superior `.init_array` scheme. This commit changes the kernel
linker script to not discard this new section, and enables it by default
in our toolchain.
2022-01-24 10:41:47 +00:00
Daniel Bertalan b3dbf204fc Toolchain: Don't use GNU objcopy in the LLVM toolchain
Our build of LLVM's objcopy now supports the single missing feature
(--update-section) that previously forced us to use the one from GNU
Binutils. This means that there is no reason anymore to build Binutils
alongside LLVM's tools.
2022-01-10 09:55:45 +03:30
tuftedocelot 5810467c97 Build: Remove gzip -k usage in PCI/USB ID files and crypt for OpenBSD
OpenBSD gzip does not have the -k flag to keep the original after
extraction. Work around this by copying the original gzip to the dest
and then extracting. A bit of a hack, but only needs to be done for the
first-time or rebuilds

OpenBSD provides crypt in libc, not libcrypt. Adjust if/else to check
for either and proceed accordingly

Remove outdated OpenBSD checks when building the toolchain
2021-12-29 03:46:56 -08:00
Ben Wiederhake 52e9f25403 Everywhere: Change from http to https where feasible
I used "git grep -FIn http://" to find all occurrences, and looked at
each one. If an occurrence was really just a link, and if a https
version exists, and if our Browser can access it at least as well as the
http version, then I changed the occurrence to https.

I'm happy to report that I didn't run into a single site where Browser
can't deal with the https version.
2021-10-05 02:08:08 +02:00
Idan Horowitz 783a58dbc7 Toolchain: Build aarch64-gdb for cross-debugging on x86 2021-09-07 12:58:46 +02:00
Andreas Kling 9b79867909 Toolchain: Make Kernel/API/ headers available during toolchain build 2021-08-14 20:25:56 +02:00
Daniel Bertalan 15e217ea68 Toolchain: Add LLVM patch and script for building it
This contains all the bits and pieces necessary to build a Clang binary
that will correctly compile SerenityOS.

I had some trouble with getting LLVM building with a single command, so
for now, I decided to build each LLVM component in a separate command
invocation. In the future, we can also make the main llvm build step
architecture-independent, but that would come with extra work to make
library and include paths work.

The binutils build invocation and related boilerplate is duplicated
because we only use `objdump` from GNU binutils in the Clang toolchain,
so most features can be disabled.
2021-08-08 10:55:36 +02:00
Gunnar Beutner 9c431ae7ef Toolchain: Update GCC to version 11.2.0 2021-07-28 21:28:31 +02:00
Gunnar Beutner cfeffbe524 Toolchain: Update binutils to version 2.37 2021-07-28 21:28:31 +02:00
Gunnar Beutner d86275d271 Kernel+Toolchain: Remove the kernel-specific toolchain
This is no longer necessary now that the kernel doesn't use libsupc++
anymore.
2021-07-06 19:08:22 +02:00
Gunnar Beutner 7dedf09f3d Toolchain: Remove dependency check for texinfo
Turns out we don't need that after all.
2021-05-30 23:36:10 +01:00
Gunnar Beutner 259822493f Toolchain: Check whether required tools and libraries are available
Rather than having the toolchain build fail half-way through we should
check whether the user has installed all the required tools and
libraries early on.
2021-05-30 13:06:28 +01:00
Gunnar Beutner afbab621aa Toolchain: Use set -o pipefail to the toolchain build script
Previously the buildstep function would obscure error codes because
the return value of the function was the exit code for the sed command
which caused us to continue execution even though one of the build
steps had failed.

With set -o pipefail the return value of the buildstep function is
the real command's exit code.
2021-05-27 08:58:37 +01:00
Idan Horowitz bcfde43849 Toolchain: Replace the -march=native flag with -mtune=native
This ensures inter-machine compatibility by not emitting any processor
specific instructions. This fixes the issue raised by the non AVX-512
supporting GitHub actions runners.
2021-05-21 15:23:07 +02:00
Brian Gianforcaro 17ff895e1c Toolchain: Enable native host optimizations when building toolchain
-march=native specializes the binaries for the CPU features available on
the CPU the binary is being compiled on. This matches the needs of the
Toolchain, as it's always built and used on that machine only.

This should be safe for the github actions VMs as well, as they all run
on a standard VM SKU in "the cloud".

I saw small but notable improvements in end-2-end build times in my
local testing. Each compilation unit is on average around a second
faster on my Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz.
2021-05-21 10:04:49 +01:00
Brian Gianforcaro 8b856bd5a1 Toolchain: Fix expansion bugs and make BuildIt.sh shellcheck compliant
BuildIt.sh had a bunch of SC2086 errors, where we were not quoting
variables in variable expansions. The logic being:

    Quoting variables prevents word splitting and glob expansion,
    and prevents the script from breaking when input contains spaces,
    line feeds, glob characters and such.

    Reference: https://github.com/koalaman/shellcheck/wiki/SC2086

As bcoles noticed in #6772, shellcheck actually found a real bug here,
where the user's build directory included spaces.

Close: #6772
2021-05-05 21:26:37 +02:00
Gunnar Beutner 8cd62b5780 Toolchain+Ports: Update GCC to version 11.1.0 2021-04-29 10:33:44 +02:00
Brian Gianforcaro 2ef93a3c07 Build: Use variables when concatenating Toolchain paths.
Make this stuff a bit easier to maintain by using the
root level variables to build up the Toolchain paths.

Also leave a note for future editors of BuildIt.sh to
give them warning about the other changes they'll need
to make.
2021-04-27 13:07:04 +02:00
Gunnar Beutner d71f64b97f Toolchain: Build GCC with optimizations 2021-04-20 23:23:23 +02:00
Gunnar Beutner 75d41657d5 Toolchain+Ports: Enable threads for gcc
This enables POSIX threads for GCC and makes the -pthread
argument available.
2021-04-20 21:08:17 +02:00
xackus 6e2f2cd8b1 Toolchain: Don't produce debug symbols
Previously debug symbols were produced and then stripped.
2021-04-19 09:49:14 +02:00
Emanuele Torre 685556ae84 Toolchain/BuildIt.sh: let's have colours on macOS too :^)
MacOS's sed doesn't support "\x1b" expansion unlike GNU sed, but we can
still use bash's $'\x1b' to expand it.
2021-04-18 19:00:49 +02:00
Gunnar Beutner 05d140e7bd Toolchain: Add missing buildstep calls 2021-04-18 15:51:56 +02:00
Gunnar Beutner 8a50c8431b Toolchain: Cache the output of uname -s 2021-04-18 15:51:56 +02:00
Gunnar Beutner c60f02fbc2 Toolchain: Build fix for macOS
Unfortunately their sed behaves a tiny bit different to how
GNU sed does.
2021-04-18 15:51:56 +02:00
Gunnar Beutner a1e0cf80e8 Toolchain: Colorize the toolchain build script's output 2021-04-18 10:55:25 +02:00
Gunnar Beutner d7978a3317 Toolchain: Enable -fexceptions and build a separate libstdc++ for the kernel
This enables building usermode programs with exception handling. It also
builds a libstdc++ without exception support for the kernel.

This is necessary because the libstdc++ that gets built is different
when exceptions are enabled. Using the same library binary would
require extensive stubs for exception-related functionality in the
kernel.
2021-04-18 10:55:25 +02:00
Brian Gianforcaro 73fab93ef5 Toolchain: Update to the latest gcc release 10.3.0
10.3.0 is a bugfix release, with 178 bugs fixed.
See: https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED&resolution=FIXED&target_milestone=10.3
2021-04-14 21:49:54 +02:00
Brian Gianforcaro 5d59affc18 Toolchain: Update binutils to version to latest 2.36.1 release
This updates binutils to the latest release. It looks like some
lines where shuffled around, so I had to fixup out patch to
apply cleanly again.
2021-04-14 21:49:54 +02:00
Gunnar Beutner e95cb252be Toolchain: Install system headers before building GCC
GCC determines whether the system's <limits.h> header is usable
and installs a different version of its own <limits.h> header
depending on whether the system header file exists.

If the system header is missing GCC's <limits.h> header does not
include the system header via #include_next.

For this to work we need to install LibC's headers before
attempting to build GCC.

Also, re-running BuildIt.sh "hides" this problem because at that
point the sysroot directory also already has a <limits.h> header
file from the previous build.
2021-04-12 22:37:34 +02:00
Ben Wiederhake 100ce8fc1f Toolchain: Always strip compiler, not just on CI 2021-03-13 22:42:20 +01:00
Tom 389dddd4b3 Meta: Add a build helper script
This script automatically manages the toolchain and cmake/ninja folders
making it easier to manage the different target architectures.
2021-03-09 07:28:59 +01:00
AnotherTest bd318dcdcd Toolchain: Always remove the previous artifacts when building 2021-02-21 23:50:34 +01:00
joshua stein 4a2209d0e0 Toolchain: Explicitly enable LTO, fix build on OpenBSD 2021-02-14 09:29:22 +01:00
joshua stein 0d215b5548 Build: Sprinkle some portability, fix on OpenBSD
realpath(1) is specific to coreutils and its behavior can be had
with readlink -f

Create the Toolchain Build directory if it doesn't exist before
calling readlink, since realpath(3) on at least OpenBSD will error
on a non-existent path
2021-02-14 09:29:22 +01:00
Andreas Kling 13d7c09125 Libraries: Move to Userland/Libraries/ 2021-01-12 12:17:46 +01:00
meme 23b23cee5a Build: Support non-i686 toolchains
* Add SERENITY_ARCH option to CMake for selecting the target toolchain
* Port all build scripts but continue to use i686
* Update GitHub Actions cache to include BuildIt.sh
2020-12-29 17:42:04 +01:00
Luke 3bdaba0b28 Meta: Add macOS workflow to CI
A good number of contributors use macOS. However, we have a bit of
a tendency of breaking the macOS build without realising it.

Luckily, GitHub Actions does actually supply macOS environments,
so let's use it.
2020-12-28 19:35:32 +01:00
Itamar 758fc8c063 Toolchain: Fix usage of libgcc_s & build PIE executables by default
We can now build the porst with the shared libraries toolchain.
2020-12-14 23:05:53 +01:00
Ben Wiederhake a82a17eee8 Toolchain: Trust cache blindly
This is necessary because cache reusability will be determined by Github Actions.

Note that we only cache if explicitly asked to do so,
which only happens on Github Actions.
2020-11-08 09:58:55 +01:00
Laurent Cimon b4790010a8 Build: Modify various parts to allow the build to succeed on FreeBSD 2020-10-20 14:40:47 +02:00
Linus Groh 8fe89cf441 Toolchain: Set CACHED_TOOLCHAIN_ARCHIVE after computing hash 2020-10-19 21:11:39 +02:00
Linus Groh 602eb98479 Toolchain: Remove cached archive and rebuild if extracting fails
This is currently the case on Travis CI: the file exists but fails to
extract, breaking all the CI builds.
2020-10-19 20:06:59 +02:00
Andreas Kling 583bfa04e2 Toolchain: Upgrade to Binutils 2.35.1 2020-10-12 19:53:25 +02:00
Andreas Kling 75d5f436bc Toolchain: Upgrade to GCC 10.2.0 2020-10-12 19:53:25 +02:00
Ben Wiederhake 3ab706cac3 Travis: Strip every executable
This cuts down the size of a Toolchain cache entry by another quarter.
(About 250 MiB to about 190 MiB).
2020-09-14 11:40:48 +02:00
Ben Wiederhake 5f724b6ca1 Travis: Toolchain only depends on headers, not impls
When libstdc++ was added in 4977fd22b8, just calling
'make install' was the easiest way to install the headers. And the headers are all
that is needed for libstdc++ to determine the ABI. Since then, BuildIt.sh was
rewritten again and again, and somehow everyone just silently assumed that
libstdc++ also depends on libc.a and libm.a, because surely it does?

Turns out, it doesn't! This massively reduces the dependencies of libstdc++,
hopefully meaning that the Toolchain doesn't need to be rebuilt so often on Travis.

Furthermore, the old method of trying to determine the dependency tree with
bash/grep/etc. has finally broken anyways:

    https://travis-ci.com/github/SerenityOS/serenity/builds/179805569#L567

In summary, this should eliminate most of the Toolchain rebuilds on Travis,
and therefore make Travis build blazingly fast! :^)
2020-08-15 16:49:55 +02:00
Ben Wiederhake 3f2b78a063 Build: Build libstdc++ in parallel 2020-08-02 18:50:28 +02:00