Commit Graph

51642 Commits

Author SHA1 Message Date
Joseph Friesen
1b504b8c93 add TCP support to fsmonitor
Summary:
This is part of proof-of-concept to test Watchman over TCP, a means
to accelerate hg workflows for the ASIC teams within Facebook.

For more context, refer to master task T55191832

Reviewed By: xavierd

Differential Revision: D18482107

fbshipit-source-id: 08ba3641854d81fe2a1cd8739bb4e9abcbc700ed
2020-01-15 13:43:36 -08:00
Joseph Friesen
aebbe4937f Propagate SockPath + TcpSocketTransport changes to hg
Summary: Copied changes from watchman dev area to hg

Reviewed By: wez

Differential Revision: D18574426

fbshipit-source-id: 6c360e42621cef373c5c1d4af34011d80eafc7bf
2020-01-15 13:43:36 -08:00
Wez Furlong
f6a9a6f7e8 eden: du: revise instructions for cleaning up LFS
Summary: `hg gc` can now clean things up there, so suggest that.

Reviewed By: quark-zju, fugalh

Differential Revision: D19413816

fbshipit-source-id: 1c6c08ed9fb4757390883d908531d9b3a7da302d
2020-01-15 12:31:56 -08:00
Jun Wu
85a89c6f96 dag: add git commit graph as test fixture
Summary:
The git repo has a lot of merges. So it's interesting as a testing graph.
The file was generated by:

    git clone https://github.com/git/git --bare
    hg --config extensions.hggit= clone git.git git-hg
    hg --cwd git-hg debugbindag -r 'all()' -o git.bindag

Reviewed By: DurhamG

Differential Revision: D19411825

fbshipit-source-id: 0fec8a16786dc8e6790986bca5c62a76276aa942
2020-01-15 11:03:52 -08:00
Mark Thomas
56cd3eadb5 renderdag: connect vertical lines for non-merge commits
Summary:
This changes the pattern commonly seen in smartlog:

    ╷ o  e7f5f529  ...
    ╭─╯
    │ o  a1b2773d ...
    ╭─╯
    o  ecbb4eaa
    ╷

to:

    ╷ o  e7f5f529  ...
    ├─╯
    │ o  a1b2773da ...
    ├─╯
    o  ecbb4eaa
    ╷

The change only applies to commits with a single parent. Vertical lines from
merge commits stay disconnected intentionally. For example:

    │ │ o  I
    │ │ │
    │ o │      H
    ╭─┼─┬─┬─╮
    │ │ │ │ o  G
    │ │ │ │ │

This makes it more obvious that `H` has 5 parents while `I` only has 1 parent -
`I` does not "borrow" `H`'s parents.  This problem does not exist if `H` only
has 1 parent.

Reviewed By: quark-zju

Differential Revision: D19407687

fbshipit-source-id: 1046c8e2309f50e3f1620ed21f1b10573759a5f8
2020-01-15 06:57:51 -08:00
svcscm
16ed126a04 Updating submodules
Summary:
GitHub commits:

b1dfc1a301
246ea2eeca
688a97477f
2ea21b99a4
8447b7d1b3
028aeb2e38
74a72cd024
120ab7ff18

Reviewed By: yns88

fbshipit-source-id: 27bd8b040ca15dc928504e3281868e5b15094bec
2020-01-15 06:57:50 -08:00
svcscm
3d3e1c0910 Updating submodules
Summary:
GitHub commits:

7d0b279daf
5ae1f5c5ac
871ab4e57b
8dc11efbec
9b0cb594da
30e7bb8331
d5c62240c8
2e8a642a51
427b633fa8

Reviewed By: yns88

fbshipit-source-id: c236d2cd5362d2599665cdadfda526a94195bf3e
2020-01-14 23:23:20 -08:00
Shrikrishna Khare
cbe74a776b OpenNSA: Don't use opennsl, switch to bcm-only API
Summary:
With the release of OpenNSA (https://github.com/Broadcom-Network-Switching-Software/OpenNSA), we can stop using opennsal altogether!

This diff was created using following steps:

cd fboss/agent
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl_spl_//g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl/bcm/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OPENNSL/BCM/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OpenNSL/Bcm/g' {} \; # for symbols like snmpBcmTransmittedPkts2048to4095Octets

Furthermore:

- In all TARGETS under fboss/agent replace wrapped_opennsl_symbols with wrapped_bcm_symbols.
- hg mv agent/facebook/test/OpenNSLInterface.h agent/facebook/test/BcmInterface.h
- BcmEgress.cpp remove operator== overload for opennsl API altogether, and remove operator== BCM API version from inside unnamed namespace, and put it in fboss namespace.
- BcmCinter.h and BcmCinter.cpp now have some duplicate definitions as opennsl_ got renamed to bcm_. Delete one of the definitions "as appropriate".
- Similarly, remove duplicate definitions in FakeSdk.h and FakeSdk.cpp
- Similarly, remove duplicate definitions in SdkTracer.cpp (remove definitions calling CALL_WRAPPERS_RV_NO_CINTER). And lastly, remove CALL_WRAPPERS_RV_NO_CINTER itself.
- arc lint
- Remove OpennslCompatTests.cpp
- Fix ./agent/hw/bcm/tests/facebook/BcmEgressTest.cpp to have #include
  "fboss/agent/facebook/test/MockOpenNSL.h" instead of MockBcm.h

TODO:
- This patch retains the directory structure (facebook/ subdirectory), which will be fixed in subsequent patches.
- Remove opennsl references from TARGETS file, additional cleanup etc.
- Remove all oss directories...

Differential Revision: D16401189

fbshipit-source-id: 607b6c0f97ffcacf3707f6b7a4c9454cc0b24476
2020-01-14 22:11:49 -08:00
Shrikrishna Khare
dbb417440b fbcode_builder: getdeps: fboss: rename OpenBCM to OpenNSA
Summary:
Broadcom provides this library and they decided to rename it from OpenBCM to
OpenNSA. Thus, rename corresponding fbcode_builder code.

Reviewed By: wez

Differential Revision: D19396687

fbshipit-source-id: 8233dbf4de9342b5a0e54ae275d6c73d43abe6d0
2020-01-14 22:11:48 -08:00
Jun Wu
dd1b9cc98d pull: use transaction
Summary:
Without this change, the next diff will fail tests with metalog programming
errors (write outside transaction).

Some test changes are caused by the being deprecated `rollback` command.

Reviewed By: DurhamG

Differential Revision: D19380939

fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
2020-01-14 21:02:27 -08:00
Jun Wu
ba2d83b3f8 pull: remove "new changesets" message
Summary:
Reverts https://www.mercurial-scm.org/repo/hg/rev/eb586ed5d8ce.

The colon syntax (x:y) is deprecated and is unsupported by segmented changelog.

Reviewed By: DurhamG

Differential Revision: D19394101

fbshipit-source-id: 8c66756f1035ab7660180716a2afa052879f384e
2020-01-14 21:02:27 -08:00
svcscm
0c7cafe5db Updating submodules
Summary:
GitHub commits:

05c7e13ac4
832dfe9bcc
74a3fc9102
777263f8ba
c9200f995d
851f7dad3a
77b45ee9a2
1dd7873e08
e9c8ec9809
ccab56ba49
a4ecc79f03
e40ecdf350

Reviewed By: yns88

fbshipit-source-id: 0c9521068551a080baab02ace7defc0ce0363809
2020-01-14 20:19:42 -08:00
Durham Goode
4c5aa36827 gpg: wrap dirstate in transaction
Summary:
As part of requiring transactions for the dirstate, let's wrap its use
in gpg.

Reviewed By: quark-zju

Differential Revision: D18213029

fbshipit-source-id: b05afa6d465e8d2a7b1637fa1c1903e1625a7104
2020-01-14 17:49:05 -08:00
Durham Goode
b52d325ebc histedit: add dirstate transactions
Summary:
As part of making the dirstate require transactions, let's wrap
histedits dirstate changes in transactions.

Reviewed By: quark-zju

Differential Revision: D18213045

fbshipit-source-id: 49baf87e8f8abfe500409f0f79a52eeae09f8ada
2020-01-14 17:49:05 -08:00
Durham Goode
419368da12 eol: wrap dirstate in transaction
Summary:
As part of requiring transactions for the dirstate, let's wrap eol's
manipulation of it.

Reviewed By: quark-zju

Differential Revision: D18213039

fbshipit-source-id: 28b52057e61fe9d88ce9f26be2cbf7482837ba84
2020-01-14 17:49:04 -08:00
Durham Goode
cd3d20e4e3 commitcloud: wrap update in transaction
Summary:
As part of moving the dirstate to require a transaction, let's wrap the
update in commitcloud.

Reviewed By: quark-zju

Differential Revision: D18213046

fbshipit-source-id: e59c880ea5e8d8808499f94c983c26e7466204e2
2020-01-14 17:49:04 -08:00
Durham Goode
b3d7defa23 automv: wrap dirstate in transaction
Summary:
As part of requiring the dirstate to be part of a transaction, let's
wrap its usage in automv.

Reviewed By: quark-zju

Differential Revision: D18213034

fbshipit-source-id: f8557453351e665ac59da9614c84ab6312cff4ef
2020-01-14 17:49:04 -08:00
Durham Goode
61b0ca78e8 fbamend: wrap dirstate in transaction
Summary:
As part of moving the dirstate to always be part of a transaction,
let's wrap its usage in our amend extension.

Reviewed By: quark-zju

Differential Revision: D18213028

fbshipit-source-id: 3562bb589265dceb3d89c997710738f44eb49f33
2020-01-14 17:49:03 -08:00
Durham Goode
b2cc1a40f3 contrib: update perf.py to wrap dirstate in a transaction
Summary:
As part of making the dirstate always participate in the transaction,
let's wrap it's use in perf.py.

Reviewed By: quark-zju

Differential Revision: D18213036

fbshipit-source-id: cd4a7d942556f032d32cbe0630d16414b819ea34
2020-01-14 17:49:03 -08:00
Durham Goode
b2d16b60c1 dirstate: move transaction write code to its own function
Summary:
A future diff will move all dirstate writes to be in a transaction.
Let's start by moving the transaction write logic to it's own function. A future
diff will delete the non-transaction "write" function.

Reviewed By: markbt

Differential Revision: D18213554

fbshipit-source-id: 7fa414314583a64ccada0f8be287d6064024e020
2020-01-14 17:49:03 -08:00
Durham Goode
d663a9dbdf fsmonitor: move warnings into filesystem layer
Summary:
Now that the post dirstate handlers are largely unused, let's move the
fsmonitor state warning into the fsmonitor filesystem. This also allows us to
remove the temporary _newid variable that was used to allow post-transaction
detection of if the transaction happened.

Reviewed By: quark-zju

Differential Revision: D18259479

fbshipit-source-id: da88638d34a80480bbeba8b1531d523785d72c30
2020-01-14 17:49:02 -08:00
Durham Goode
bca361cd3c dirstate: remove most of post dirstate handlers
Summary:
Previously the localrepo object contained a list of handlers to invoke
after status finished executing. This was used for 1) updating the fsmonitor
state with the new clock, ignore hash, and list of notable files, and 2)
updating the list of ignored files in treestate.

As part of moving all fsmonitor subtlities behind the filesystem abstraction,
this diff moves the fsmonitorstate file updating to be in the filesystem layer.
This also moves it inside the same transaction as the lookup file cleanup, which
removes the need for the hacky self._newid logic to detect if the dirstate was
changed by an external process.

(self._newid was previously added so that our second dirstate transaction, which
updates the fsmonitorstate, could determine if the dirstate had either been
changed by this process earlier in the lookup cleanup transaction, or if it had
been changed by an external process).

Reviewed By: quark-zju

Differential Revision: D18076655

fbshipit-source-id: c5175f61432f8e121b6baacd9587fcd288f67098
2020-01-14 17:49:02 -08:00
Durham Goode
af3292ea85 filesystem: refactor lookup marking to be more extensible
Summary:
A future diff is going to move the fsmonitor post-status fixups into
the same transaction as the lookup-marking logic. To prepare for this, let's
refactor the lookup marking logic to separate out the transaciton management
from the actual marking logic.

Reviewed By: quark-zju

Differential Revision: D18076656

fbshipit-source-id: 7711d5b086be7baaff0d9736cf51c6b0a46f237e
2020-01-14 17:49:01 -08:00
Durham Goode
a064e12d93 dirstate: move lookup resolution into the filesystem
Summary:
In an earlier diff we moved the lookup-state resolution from the repo
object to the dirstate object. Now that a filesystem layer exists, let's move
lookup resolution into the filesystem layer so that filesystem.pendingchanges
can now return perfect information.

Reviewed By: quark-zju

Differential Revision: D17749373

fbshipit-source-id: d03177e5e5374bd38c6c9adaa0fe8959052de19e
2020-01-14 17:49:01 -08:00
svcscm
b27e11762f Updating submodules
Summary:
GitHub commits:

44c1810b51
d667a9e28e
3f2526d66b
7588a543d7
ba8fee8527
4e328d236c
76c117b24b
d079b69230
b1d9cd65fa
88c05ec0cf

Reviewed By: yns88

fbshipit-source-id: 6e25292a7d12fe99e1a89aab0b9f19311ea059f1
2020-01-14 15:39:03 -08:00
Xavier Deguillard
b1f3e37e43 tests: the Rust ContentStore always creates .hg/store/packs
Summary:
The Python one only does it when needed, let's fix the 2 tests that were
relying on this.

Reviewed By: quark-zju

Differential Revision: D19393501

fbshipit-source-id: 94e004999045264adce2ffa17b4697583ee0fcbd
2020-01-14 13:46:26 -08:00
Xavier Deguillard
f1c8af3297 remotefilelog: test for phase instead of localcontentstore
Summary:
The phases of a commit is a more precise metric for whether a change is public
or not, let's use this instead of testing for data to be present in the local
contentstore.

Reviewed By: quark-zju

Differential Revision: D19393502

fbshipit-source-id: b1c735c8400e8d5a1a7f7013d9a4a1ad423c5023
2020-01-14 13:46:26 -08:00
Stefan Filip
eca4fbc75d tests: increase timeout for mononoke starting
Summary:
We started to see a lot of tests failing with mononoke timing out at startup.
Increasing the timeout to mitigate the issue.

Reviewed By: krallin

Differential Revision: D19397440

fbshipit-source-id: a8037102b3c8389b98a7c49824fa013fd3e6a080
2020-01-14 12:52:05 -08:00
Stefan Filip
a2ca6bb7dd manifest-tree: move private test function to lib.rs
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.

Reviewed By: markbt

Differential Revision: D19350343

fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
2020-01-14 11:49:53 -08:00
Stefan Filip
2c9fa6054d manifest: update make_tree to be public
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.

Reviewed By: markbt

Differential Revision: D19350351

fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
2020-01-14 11:49:53 -08:00
Stefan Filip
887b776fe1 manifest: add testutil in core package
Summary:
Types that are defined by the manifest core crate should have test utilities
defined in the same crate.

This is motivated by various warning in the buck build.

Reviewed By: markbt

Differential Revision: D19350350

fbshipit-source-id: a6a7c3fb54b465aa09a28ff8b70b49a355b328fc
2020-01-14 11:49:52 -08:00
Jun Wu
1be7e2ea4c hgmain: explicitly flush IO on exit
Summary: Without this, `hg status --print0` can output nothing and break tests.

Reviewed By: xavierd

Differential Revision: D19387317

fbshipit-source-id: 4feb82889df9bd1705526027f491db90a2a5f946
2020-01-14 09:42:48 -08:00
svcscm
c262d1d647 Updating submodules
Summary:
GitHub commits:

cce1e2b815
80362c9c63

Reviewed By: yns88

fbshipit-source-id: 0159101758d94c9c7d179279ab587ae30123c6a0
2020-01-14 09:42:48 -08:00
svcscm
24cab7c3ce Updating submodules
Summary:
GitHub commits:

2156e48924
8c5b4af317
be69716784
4f76ad1fab
0b12b2f13c
0449b53cb1
1481689822
43ffa9bbf0
787d6b6c93

Reviewed By: yns88

fbshipit-source-id: b0080fd1a4c26efbe8f26245fbba7740fbac08f3
2020-01-13 20:16:00 -08:00
Jun Wu
684351a6a9 edenfs-client: fix warnings
Summary:
Fix warnings about unused imports, and fix Cargo.toml to use well-defined
versions.

Reviewed By: singhsrb

Differential Revision: D19386232

fbshipit-source-id: 4eb7478dfb4f34e8338d5090b3ba7225bfcb008f
2020-01-13 18:14:46 -08:00
svcscm
fe9b066361 Updating submodules
Summary:
GitHub commits:

1f8321394d
024c1d0b43
1d57089fc3
3c6f1f782c
21a27b0f8e
23bb716b62
894c6d21af
e3e241d700
ac4e11d84a
c35803ad68
647388f265
50a3288630
b197f0c95a

Reviewed By: yns88

fbshipit-source-id: 1807ac876a126d221c257edbd4732f9a1240e869
2020-01-13 18:07:08 -08:00
Jun Wu
2ade5663b1 mq: remove references about mq
Summary: `mq` was removed. Remove remaining references to it.

Reviewed By: singhsrb

Differential Revision: D19382027

fbshipit-source-id: d194556b63dd6b9a44b949bd24b7e1a7356d5c04
2020-01-13 17:39:43 -08:00
Puneet Kaushik
b01d4a721d Implemented check file name functionality to workaround a Projected FS issue.
Summary:
There is a bug in Projected FS on Windows 1903 and 1909, which BSODs the system on SetInformation call. Projected FS seems to be dereferencing a nullptr while attempting to check if the file is in the backing store. Providing a separate callback to check if the file exists in the backing store will avoid the buggy code (suggested by Microsoft).

This will enable Eden FS on Windows 1903 and 1909.

Reviewed By: simpkins

Differential Revision: D19377522

fbshipit-source-id: ab5e9bc8e2472f3e689d71d05a560bac86211295
2020-01-13 16:15:09 -08:00
Puneet Kaushik
8899c53892 Update Windows EdenMount::getRootTree return a future
Summary: This is a followup on D18666585.

Reviewed By: chadaustin

Differential Revision: D19377523

fbshipit-source-id: f9ab4011d0978a6934de77d1837a37082603cf52
2020-01-13 16:15:09 -08:00
Puneet Kaushik
1350e4f8ce Fixing ObjectStore::create signature in Windows TestMount
Summary: We missed updating this in D18669664.

Reviewed By: simpkins

Differential Revision: D19377524

fbshipit-source-id: e23fb039047855454b4f12353bc375b1a2c8d962
2020-01-13 16:15:08 -08:00
Xavier Deguillard
0be67e6b08 osutil: export O_CLOEXEC
Summary: This is needed in Python3, export it.

Reviewed By: quark-zju

Differential Revision: D19379402

fbshipit-source-id: e5d095005f20c8a940b31e0296293aa8a85c4e98
2020-01-13 16:15:08 -08:00
Adam Simpkins
db881eb225 getdeps: enable the INSTALL_PYTHON_LIB option for Eden builds on Windows
Summary: Enable `INSTALL_PYTHON_LIB=ON` for internal builds of Eden on Windows.

Reviewed By: pkaush

Differential Revision: D19060630

fbshipit-source-id: 40d4f8784958fbeaad4f0d332c55fa0cb810cb5e
2020-01-13 16:10:58 -08:00
Adam Simpkins
6666995b37 add an option to also install the Python DLL on Windows
Summary:
Add an option to CMakeLists.txt so that when building on Windows the Python
DLL will also be installed into Eden's `bin/` directory.  On Windows our
Python-based tools are compiled into `.exe` files that require this DLL.
Installing the Python DLL along side our binaries is an easy way to ensure
that these executables can always find the correct library.

It would perhaps be nicer if we could simply link against Python statically.
Unfortunately the `FindPython3.cmake` file that ships with CMake silently
ignores requests to link statically on Windows.  (They provide a
`Python3_USE_STATIC_LIBS` option, but it is explicitly ignored on Windows.)
In the long run it might be nicer to attempt to fix that.

Reviewed By: pkaush

Differential Revision: D19060632

fbshipit-source-id: 6995fe6b81b820cf8e0713687751c1a811e97f88
2020-01-13 16:10:57 -08:00
Durham Goode
1c573e7569 workingcopy: add error handling and reporting to rust walker
Summary:
Previously, if the rust walker encountered any issues with individual
files it would stop and throw an exception or in some cases panic. To replace
the python walker we need to handle errors more gracefully, so let's add logic
for reporting which files had issues and what those issues were.

This also contains a minor one-line fix to prevent the walker from traversing
directories that contain a '.hg' directory.

Reviewed By: quark-zju

Differential Revision: D19376540

fbshipit-source-id: ede40f2b31aa1e856c564dd9956b9f593f21cf27
2020-01-13 15:26:30 -08:00
Jun Wu
aa64745a35 debugmetalog: add a command to show repo history
Summary:
The command shows how visibility, bookmarks, remotenames change in the repo
over time, using metalog data.

Also add this command to hg rage report.

Reviewed By: markbt

Differential Revision: D19358668

fbshipit-source-id: 9bddc039143b45c29c0c937ee60906ab93a2defe
2020-01-13 14:29:07 -08:00
Jun Wu
cfcf538578 color: define changeset.{public,draft} colors
Summary: The colors are from the our smartlog template. They're used in the next change.

Reviewed By: markbt

Differential Revision: D19358670

fbshipit-source-id: 7476f95767c5fcf1cccdce2119a2ab7ce45d270e
2020-01-13 14:29:06 -08:00
Jun Wu
3895af590a edenfs-client: use clidispatch IO and Error types
Summary:
This simplifies the error handling and makes it more compatible with things that
might capture the output, including the Python testing framework.

Reviewed By: DurhamG

Differential Revision: D19325642

fbshipit-source-id: 53de8b9a8294219e2b8e62831dce236841bd4cbb
2020-01-13 14:23:34 -08:00
Jun Wu
12eaf82291 hgcommands: add edenfs status fast path
Summary: Make `hg status` use the EdenFS Thrift path, similar to the telemetry wrapper.

Reviewed By: DurhamG

Differential Revision: D19325641

fbshipit-source-id: 14777a252d7cb433316511a2a1f1a6649e9cb020
2020-01-13 14:23:34 -08:00
Jun Wu
b9e7f6a995 cliparser: support shared flag definition
Summary:
Make it possible to define reusable flags like walkopts, templateopts that can
be shared among commands.

Reviewed By: DurhamG

Differential Revision: D19325640

fbshipit-source-id: 89a9ad0e59672b1f8cfa22daaacd3c75d8cff61a
2020-01-13 14:23:33 -08:00
Jun Wu
7e9aaf9da5 lib: add edenfs-client
Summary:
Add a crate aiming for talking to EdenFS via Thrift.

This is mostly ported from telemetry. There are lots of things that can be
cleaned up like moving the command portion to hgcommands, or using a better
abstracted rendering layer for outputting, etc. But for now, I just ported
the code as-is.

Reviewed By: DurhamG

Differential Revision: D19325639

fbshipit-source-id: ac70315ed166024857a90aab070e4d521f3e3b58
2020-01-13 14:23:33 -08:00