Summary:
Previously the mutation commit extras were the source of truth for mutation
information, and the mutation store served as a kind of cache. This turned out
to be less useful than expected, as oftentimes commits are missing, and the
store is better indexed, so in practice using the store as the source of truth
is better.
This change makes the mutationstore the (sole) source of truth for mutation
data. The extras are kept, but they are now only useful as human-readable
debug information, and to ensure the commit hash is unique.
Collecting the mutation information during commit creation is now done through
a new `mutinfo` object. This is a dict with the same keys as the mutation
extras, for simplicity, but it is now passed through the `committablectx` and
used to generate the mutation store entry directly.
The `mutation.enabled` config option is now used to control all aspects of
enabling mutation.
The `mutation.record` config option is now only used to indicate whether the
mutation extras should also added to the commit. Generally this should be set
to `true`, however the option is retained so that mutation extras can be
stripped by running `hg amend --config mutation.record=false`, which no longer
has the side-effect of not recording mutation information to the store.
The "remote commit" mutation record origin is now obsolete, and won't be
generated anymore.
Pushrebase now relies on the obsmarker information coming back from the server
in order to correctly generate mutation information. We will need to change
this so that the server returns mutation records before we can fully deprecate
obsmarkers.
Reviewed By: DurhamG
Differential Revision: D19410650
fbshipit-source-id: 8d7094e4bfd8d8e97916898d899a8debd339485f
Summary: Begin logging the common directory path for the commit's updated files.
Reviewed By: DurhamG
Differential Revision: D19399717
fbshipit-source-id: d47994f573eac6e7cd1596b0b102cb9577e1eea1
Summary: When I was first prototyping the converter, I used a prefix to make it clear which commits represented merged commit history. This commit removes that prefix so the commit messages are an exact match.
Reviewed By: tchebb
Differential Revision: D19404075
fbshipit-source-id: 73b4aab39b30ce1dcff917e40d459f9615896fca
Summary:
The linknode is part of the history, not the data, no need to force prefetch
the blobs.
Reviewed By: quark-zju
Differential Revision: D19394351
fbshipit-source-id: 2e63d82928ebd5cf8e3a0d1b87d115b45d1428a7
Summary:
Mercurial assumes some specific ordering, let's comment about this to make sure
we don't re-order and introduce subtle bugs.
Reviewed By: quark-zju
Differential Revision: D19394352
fbshipit-source-id: 0f9e02d2c6addf040311a54b8161b06bbeaa6be9
Summary:
The error message and the exception type are slightly different between the
Rust and Python ContentStore. For now, let's just fix this up manually.
Reviewed By: quark-zju
Differential Revision: D19394350
fbshipit-source-id: e432094a9dfcf605568a1890c0303b733e98d203
Summary:
Broadcom has formally replaced OpenNSL with newer OpenNSA:
https://github.com/Broadcom-Network-Switching-Software/OpenNSA/blob/master/README.md
With D16401189, FBOSS no longer depends on OpenNSL, but uses the new OpenNSA
instead. Remove this manifest.
Reviewed By: wez
Differential Revision: D19413708
fbshipit-source-id: f3662f1101d59a5e0e59cb7238797d652564ff90
Summary: The rust telemetry wrapper and the python code were not in sync in how they presented errors to the user on failure. This makes them identical in message and error code (however the "abort" from python is red in color).
Reviewed By: quark-zju
Differential Revision: D19313848
fbshipit-source-id: d5b95e8d5130f20fc85c6664933ee18702f7cc5e
Summary: This eliminates the broken pipes that have been more prevalent recently.
Reviewed By: DurhamG
Differential Revision: D19412153
fbshipit-source-id: 0062eb57ec07dfcbfee3ed9b77cf82cdcda09fb3
Summary:
Previously, NameDag only supports writing directly to disk. That is not easy
for hg to use, since hg knows commits before bookmark names during pull, while
the API requires both commits and bookmarks (to decide which commits belong to
the "master" group) at the same time.
The old API also does not match other storage layers like indexedlog and
metalog, where writes are buffered until explicitly flushed.
This diff adds the "write-in-memory" (named "add_heads"), and "flush" APIs to
make NameDag easier to use. Under the hood, it just copies the "added" portion
of the DAG, and re-use the old API ("build", renamed to "add_heads_and_flush")
to do the job.
Note: The buffered changes pattern cannot be used in IdDag, since Ids might
have to be re-assigned on flush. But NameDag does not expose the raw Ids in its
"normal" interface, so it's fine to reassign Ids on flush.
Reviewed By: markbt
Differential Revision: D19405474
fbshipit-source-id: 75e5e5815c78c3577a0138f48185f6c4b5a80891
Summary:
This is more consistent with the name "IdDag", because both "Name" and "Id" are
noun.
Reviewed By: singhsrb
Differential Revision: D19405473
fbshipit-source-id: a3b7546dd0ddcae5d9ed562838bd6be10a47063c
Summary: This is more consistent with the name "NamedDag".
Reviewed By: singhsrb
Differential Revision: D19405472
fbshipit-source-id: f7023307acaf96bf77c9fa9704dcaf6fc59b56f2
Summary:
This is part of proof-of-concept to test Watchman over TCP, a means
to accelerate hg workflows for the ASIC teams within Facebook.
For more context, refer to master task T55191832
Reviewed By: xavierd
Differential Revision: D18482107
fbshipit-source-id: 08ba3641854d81fe2a1cd8739bb4e9abcbc700ed
Summary: Copied changes from watchman dev area to hg
Reviewed By: wez
Differential Revision: D18574426
fbshipit-source-id: 6c360e42621cef373c5c1d4af34011d80eafc7bf
Summary: `hg gc` can now clean things up there, so suggest that.
Reviewed By: quark-zju, fugalh
Differential Revision: D19413816
fbshipit-source-id: 1c6c08ed9fb4757390883d908531d9b3a7da302d
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:
git clone https://github.com/git/git --bare
hg --config extensions.hggit= clone git.git git-hg
hg --cwd git-hg debugbindag -r 'all()' -o git.bindag
Reviewed By: DurhamG
Differential Revision: D19411825
fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
Summary:
This changes the pattern commonly seen in smartlog:
╷ o e7f5f529 ...
╭─╯
│ o a1b2773d ...
╭─╯
o ecbb4eaa
╷
to:
╷ o e7f5f529 ...
├─╯
│ o a1b2773da ...
├─╯
o ecbb4eaa
╷
The change only applies to commits with a single parent. Vertical lines from
merge commits stay disconnected intentionally. For example:
│ │ o I
│ │ │
│ o │ H
╭─┼─┬─┬─╮
│ │ │ │ o G
│ │ │ │ │
This makes it more obvious that `H` has 5 parents while `I` only has 1 parent -
`I` does not "borrow" `H`'s parents. This problem does not exist if `H` only
has 1 parent.
Reviewed By: quark-zju
Differential Revision: D19407687
fbshipit-source-id: 1046c8e2309f50e3f1620ed21f1b10573759a5f8
Summary:
With the release of OpenNSA (https://github.com/Broadcom-Network-Switching-Software/OpenNSA), we can stop using opennsal altogether!
This diff was created using following steps:
cd fboss/agent
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl_spl_//g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl/bcm/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OPENNSL/BCM/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OpenNSL/Bcm/g' {} \; # for symbols like snmpBcmTransmittedPkts2048to4095Octets
Furthermore:
- In all TARGETS under fboss/agent replace wrapped_opennsl_symbols with wrapped_bcm_symbols.
- hg mv agent/facebook/test/OpenNSLInterface.h agent/facebook/test/BcmInterface.h
- BcmEgress.cpp remove operator== overload for opennsl API altogether, and remove operator== BCM API version from inside unnamed namespace, and put it in fboss namespace.
- BcmCinter.h and BcmCinter.cpp now have some duplicate definitions as opennsl_ got renamed to bcm_. Delete one of the definitions "as appropriate".
- Similarly, remove duplicate definitions in FakeSdk.h and FakeSdk.cpp
- Similarly, remove duplicate definitions in SdkTracer.cpp (remove definitions calling CALL_WRAPPERS_RV_NO_CINTER). And lastly, remove CALL_WRAPPERS_RV_NO_CINTER itself.
- arc lint
- Remove OpennslCompatTests.cpp
- Fix ./agent/hw/bcm/tests/facebook/BcmEgressTest.cpp to have #include
"fboss/agent/facebook/test/MockOpenNSL.h" instead of MockBcm.h
TODO:
- This patch retains the directory structure (facebook/ subdirectory), which will be fixed in subsequent patches.
- Remove opennsl references from TARGETS file, additional cleanup etc.
- Remove all oss directories...
Differential Revision: D16401189
fbshipit-source-id: 607b6c0f97ffcacf3707f6b7a4c9454cc0b24476
Summary:
Broadcom provides this library and they decided to rename it from OpenBCM to
OpenNSA. Thus, rename corresponding fbcode_builder code.
Reviewed By: wez
Differential Revision: D19396687
fbshipit-source-id: 8233dbf4de9342b5a0e54ae275d6c73d43abe6d0
Summary:
Without this change, the next diff will fail tests with metalog programming
errors (write outside transaction).
Some test changes are caused by the being deprecated `rollback` command.
Reviewed By: DurhamG
Differential Revision: D19380939
fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
Summary:
Reverts https://www.mercurial-scm.org/repo/hg/rev/eb586ed5d8ce.
The colon syntax (x:y) is deprecated and is unsupported by segmented changelog.
Reviewed By: DurhamG
Differential Revision: D19394101
fbshipit-source-id: 8c66756f1035ab7660180716a2afa052879f384e
Summary:
As part of requiring transactions for the dirstate, let's wrap its use
in gpg.
Reviewed By: quark-zju
Differential Revision: D18213029
fbshipit-source-id: b05afa6d465e8d2a7b1637fa1c1903e1625a7104
Summary:
As part of making the dirstate require transactions, let's wrap
histedits dirstate changes in transactions.
Reviewed By: quark-zju
Differential Revision: D18213045
fbshipit-source-id: 49baf87e8f8abfe500409f0f79a52eeae09f8ada
Summary:
As part of requiring transactions for the dirstate, let's wrap eol's
manipulation of it.
Reviewed By: quark-zju
Differential Revision: D18213039
fbshipit-source-id: 28b52057e61fe9d88ce9f26be2cbf7482837ba84
Summary:
As part of moving the dirstate to require a transaction, let's wrap the
update in commitcloud.
Reviewed By: quark-zju
Differential Revision: D18213046
fbshipit-source-id: e59c880ea5e8d8808499f94c983c26e7466204e2
Summary:
As part of requiring the dirstate to be part of a transaction, let's
wrap its usage in automv.
Reviewed By: quark-zju
Differential Revision: D18213034
fbshipit-source-id: f8557453351e665ac59da9614c84ab6312cff4ef
Summary:
As part of moving the dirstate to always be part of a transaction,
let's wrap its usage in our amend extension.
Reviewed By: quark-zju
Differential Revision: D18213028
fbshipit-source-id: 3562bb589265dceb3d89c997710738f44eb49f33
Summary:
As part of making the dirstate always participate in the transaction,
let's wrap it's use in perf.py.
Reviewed By: quark-zju
Differential Revision: D18213036
fbshipit-source-id: cd4a7d942556f032d32cbe0630d16414b819ea34
Summary:
A future diff will move all dirstate writes to be in a transaction.
Let's start by moving the transaction write logic to it's own function. A future
diff will delete the non-transaction "write" function.
Reviewed By: markbt
Differential Revision: D18213554
fbshipit-source-id: 7fa414314583a64ccada0f8be287d6064024e020
Summary:
Now that the post dirstate handlers are largely unused, let's move the
fsmonitor state warning into the fsmonitor filesystem. This also allows us to
remove the temporary _newid variable that was used to allow post-transaction
detection of if the transaction happened.
Reviewed By: quark-zju
Differential Revision: D18259479
fbshipit-source-id: da88638d34a80480bbeba8b1531d523785d72c30
Summary:
Previously the localrepo object contained a list of handlers to invoke
after status finished executing. This was used for 1) updating the fsmonitor
state with the new clock, ignore hash, and list of notable files, and 2)
updating the list of ignored files in treestate.
As part of moving all fsmonitor subtlities behind the filesystem abstraction,
this diff moves the fsmonitorstate file updating to be in the filesystem layer.
This also moves it inside the same transaction as the lookup file cleanup, which
removes the need for the hacky self._newid logic to detect if the dirstate was
changed by an external process.
(self._newid was previously added so that our second dirstate transaction, which
updates the fsmonitorstate, could determine if the dirstate had either been
changed by this process earlier in the lookup cleanup transaction, or if it had
been changed by an external process).
Reviewed By: quark-zju
Differential Revision: D18076655
fbshipit-source-id: c5175f61432f8e121b6baacd9587fcd288f67098
Summary:
A future diff is going to move the fsmonitor post-status fixups into
the same transaction as the lookup-marking logic. To prepare for this, let's
refactor the lookup marking logic to separate out the transaciton management
from the actual marking logic.
Reviewed By: quark-zju
Differential Revision: D18076656
fbshipit-source-id: 7711d5b086be7baaff0d9736cf51c6b0a46f237e
Summary:
In an earlier diff we moved the lookup-state resolution from the repo
object to the dirstate object. Now that a filesystem layer exists, let's move
lookup resolution into the filesystem layer so that filesystem.pendingchanges
can now return perfect information.
Reviewed By: quark-zju
Differential Revision: D17749373
fbshipit-source-id: d03177e5e5374bd38c6c9adaa0fe8959052de19e
Summary:
The Python one only does it when needed, let's fix the 2 tests that were
relying on this.
Reviewed By: quark-zju
Differential Revision: D19393501
fbshipit-source-id: 94e004999045264adce2ffa17b4697583ee0fcbd
Summary:
The phases of a commit is a more precise metric for whether a change is public
or not, let's use this instead of testing for data to be present in the local
contentstore.
Reviewed By: quark-zju
Differential Revision: D19393502
fbshipit-source-id: b1c735c8400e8d5a1a7f7013d9a4a1ad423c5023
Summary:
We started to see a lot of tests failing with mononoke timing out at startup.
Increasing the timeout to mitigate the issue.
Reviewed By: krallin
Differential Revision: D19397440
fbshipit-source-id: a8037102b3c8389b98a7c49824fa013fd3e6a080
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.
Reviewed By: markbt
Differential Revision: D19350343
fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.
Reviewed By: markbt
Differential Revision: D19350351
fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
Summary:
Types that are defined by the manifest core crate should have test utilities
defined in the same crate.
This is motivated by various warning in the buck build.
Reviewed By: markbt
Differential Revision: D19350350
fbshipit-source-id: a6a7c3fb54b465aa09a28ff8b70b49a355b328fc
Summary: Without this, `hg status --print0` can output nothing and break tests.
Reviewed By: xavierd
Differential Revision: D19387317
fbshipit-source-id: 4feb82889df9bd1705526027f491db90a2a5f946
