Linking clang-13 with debug info takes ~3.6 seconds on a simulated
10-core/20-threads machine. mold spends most of its time (~2.3 seconds)
merging string literals in .debug_str. Input .debug_str sections contain
70 million string literals in total, which is reduced to 2 million after
de-duplication. The input object files contain a lot of duplicates.
clang-13 with debug info is enormous -- it is ~3.1 GiB after linking.
It looks like TBB's concurrent hashmap doesn't scale well with the
input.
In this patch, I implemented our own concurrent hashmap. The hashmap
is extremely lightweight and support only the key-value insertion
operation. It doesn't even support rehashing. It aborts once the hash
table becomes full.
In order to know the correct size for the hashmap before inserting
strings into it, I also implemented HyperLogLog algorithm in this patch.
HyperLogLog is an algorithm that gives a fairly accurate estimate on
the number of unique elements.
With this patch, mold can link clang-13 in ~2.5 seconds, which is ~30%
faster than before.
https://github.com/rui314/mold/issues/73
mold.h:128:19: error: explicit specialization of
‘template<class Key> class tbb::detail::d1::tbb_hash_compare’
outside its namespace must use a nested-name-specifier [-fpermissive]
128 | template<> struct tbb_hash_compare<std::string_view> {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
oneTBB API is changing rapidly, so even if a oneTBB is installed
into a system already, it is likely that that is not a compatible
version with mold. This makes program building unnecessarily hard.
This change intends to solve the problem once and for all by
including oneTBB as a subdirectory and static-link against it at
build-time.
oneTBB is released under the Apache License 2.0, which is compatible
with AGPL. Therefore, we can still distribute the binary of mold
under AGPL.
If you do not want to static-link oneTBB, pass `SYSTEM_TBB=1`
to `make`.
This change also ports mold from oneTBB v2020.3 to v2021.3.0.
Fixes https://github.com/rui314/mold/issues/66
Previously, if a weak undefined symbol cannot be resolved within
an output of the linker, mold turned it into an absolute symbol
with value 0. However, other linkers export such symbols as a
weak dynamic symbol, so that the symbol gets another chance to be
resolved at runtime.
This patch implements the behavior.
This is needed by Gentoo's dev-libs/nsync-1.20.1 package.
Previously, we turned unresolved undefined symbols into dynamic
symbols. This patch changed the behavior so that such symbols
are turned into non-dynamic absolute symbols with value zero.
This change is needed by Gentoo's dev-libs/wayland-protocols-1.21
package.
If an object file in an static archive contains a common symbol
that can be used to resolve an undefined symbol, mold now pulls
out that object file from the archive. Previously, we ignore such
common symbols.
Object files generated by a Fortran compiler often contain a lot of
common symbols, and this change makes a difference.
Fixes Gentoo's sci-astronomy/wcslib-7.3 package, which contains a
program written in Fortran.
