Linking clang-13 with debug info takes ~3.6 seconds on a simulated
10-core/20-threads machine. mold spends most of its time (~2.3 seconds)
merging string literals in .debug_str. Input .debug_str sections contain
70 million string literals in total, which is reduced to 2 million after
de-duplication. The input object files contain a lot of duplicates.
clang-13 with debug info is enormous -- it is ~3.1 GiB after linking.
It looks like TBB's concurrent hashmap doesn't scale well with the
input.
In this patch, I implemented our own concurrent hashmap. The hashmap
is extremely lightweight and support only the key-value insertion
operation. It doesn't even support rehashing. It aborts once the hash
table becomes full.
In order to know the correct size for the hashmap before inserting
strings into it, I also implemented HyperLogLog algorithm in this patch.
HyperLogLog is an algorithm that gives a fairly accurate estimate on
the number of unique elements.
With this patch, mold can link clang-13 in ~2.5 seconds, which is ~30%
faster than before.
https://github.com/rui314/mold/issues/73
