Commit Graph

2232 Commits

Author SHA1 Message Date
Adam Simpkins
b13f498ab2 re-number the KeySpace IDs to be 0-indexed
Summary:
Update the KeySpace enum values to start at 0 instead of 1.  This simplifies
the code to avoid having to skip over 0 in a few places.

This also makes the `kKeySpaceRecords` array slightly less confusing.  Unlike
the `columns` array used by `RocksDbLocalStore` and the `tableNames` array
used by `SqliteLocalStore`, the `kKeySpaceRecords` array was not previously
indexed by the `KeySpace` enum values.

Reviewed By: wez, strager

Differential Revision: D15307393

fbshipit-source-id: ae8392d02396b4dc3c18e9ee94b198fcbb9b1a34
2019-05-15 12:19:19 -07:00
Adam Simpkins
875a9fa7ff move KeySpace definitions into a header file
Summary:
Move the KeySpace record definitions out of LocalStore.cpp and into a header
file.  This will make it possible to access the key space information from
other files in the future.

I would like to have access to this data directly in the various LocalStore
implementations, as well as in eden_store_util.cpp.  Currently both
RocksDbLocalStore.cpp and SqliteLocalStore.cpp have their own copies of key
space information that should really be shared.

Reviewed By: chadaustin, strager

Differential Revision: D15307392

fbshipit-source-id: ad7cdc705d3dc9834592fe0f14b38d82f91a3b77
2019-05-15 12:19:19 -07:00
Andres Suarez
410b7cd112 Text lint all .gitignore files
Reviewed By: scottrice, pallotron

Differential Revision: D15353820

fbshipit-source-id: 74f9eaadc90363a958692259f5cb66cef91ac8ef
2019-05-15 11:35:00 -07:00
svcscm
5594dddb51 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 7fb8cfc6810e8b4f3a40a9ae8c20b21f43cad1a2
2019-05-15 11:35:00 -07:00
svcscm
846fcfc637 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 63e9b4a8cf5b15a6ba20d1946aac36c1604d8079
2019-05-14 18:57:17 -07:00
Puneet Kaushik
94c336d7bb CMake build scripts for Eden Windows
Summary:
Now Eden on Windows could be build using
python.exe .\opensource\fbcode_builder\getdeps.py build eden

Reviewed By: strager

Differential Revision: D15251103

fbshipit-source-id: 8ac1097b234bb27ce478101bfbc9591251e2e359
2019-05-10 18:33:00 -07:00
svcscm
47150015d8 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: f514c0e7f58af85e42d6ef6e22a05373c0b82735
2019-05-10 17:14:29 -07:00
Udip Pant
e4db25f8b7 update docker os_image to ubuntu18 and gcc7
Summary:
This diff updates docker os_image to ubuntu18 and gcc7 in travis settings.
The dependencies that we work with internally (e.g. gcc7, boost and so on) are not present in the ubuntu-16.04. Further, the support for gcc5 is going away. So maintaining this compatibility for these platforms is painful, especially for new projects, such as mvfst.

Reviewed By: simpkins

Differential Revision: D15286181

fbshipit-source-id: eb0da9556cdb17d0aa132ac2aa4c35f9c6eccd97
2019-05-10 16:39:16 -07:00
John Strizich
b22731e3b5 add option to pass options to git clone
Summary: We need this for dependencies that require submodules

Reviewed By: saifhhasan, GirasoleY

Differential Revision: D15282792

fbshipit-source-id: b0cc8d645e73668252409934fd6741fb211e30ae
2019-05-10 13:55:46 -07:00
Zeyi (Rice) Fan
92a977f6eb fix encoding issue when building on Windows
Summary:
On Windows, the writing operation would fail with:

```
Traceback (most recent call last):
  File ".\opensource\fbcode_builder\getdeps.py", line 400, in <module>
    sys.exit(main())
  File ".\opensource\fbcode_builder\getdeps.py", line 393, in main
    return args.func(args)
  File ".\opensource\fbcode_builder\getdeps.py", line 236, in run
    change_status = fetcher.update()
  File "C:\open\fbsource\fbcode\opensource\fbcode_builder\getdeps\fetcher.py", line 451, in update
    return mapping.mirror(self.build_options.fbsource_dir, self.repo_dir)
  File "C:\open\fbsource\fbcode\opensource\fbcode_builder\getdeps\fetcher.py", line 400, in mirror
    f.write(name + "\n")
  File "C:\Python36\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 104-105: character maps to <undefined>
```

and this is caused by a file in libgit2: https://github.com/libgit2/libgit2/blob/master/tests/resources/status/%E8%BF%99, which is intended to test handling non-ASCII path.

Python on Windows will write file in cp1252 encoding by default, which does not contain that Chines character. (Caveat: that file on my system doesn't have the correct file name as well, it is being encoded in IBM861 for some reason. However the characters in IBM861 does not exist in CP1252 either)

Reviewed By: wez

Differential Revision: D15281521

fbshipit-source-id: 8a75e32bc1042167c945d67e26b549fda83b6b41
2019-05-10 13:17:32 -07:00
Wez Furlong
002101dec8 getdeps: introduce TransientFailure exception type
Summary:
The goal is to return an error code > 127 in the case of a
transient, retryable, infrastructure error.  This diff generates
those in the case of failure in downloading a URL or from interacting
with LFS.

Reviewed By: strager

Differential Revision: D15266838

fbshipit-source-id: 4f52a791320123968869032c37912dded464a86e
2019-05-10 12:52:45 -07:00
Wez Furlong
942723f3bf getdeps: respect cmake WORKING_DIRECTORY for tests
Summary:
the cmake `add_test` and related functions allow specifying
the WORKING_DIRECTORY to use for tests.  We weren't respecting this
value, so this diff looks up the WORKING_DIRECTORY from the ctest
json info and adjusts the buck test info json blob that we pass
on the testpilot.

Since that interface only allows passing an argv array, we use
the `cmake -E chdir` command to run a command in a specified
directory in a portable manner.

Reviewed By: strager

Differential Revision: D15274012

fbshipit-source-id: 1f02d461d73178745794703d455494e31c2e09ed
2019-05-10 12:47:21 -07:00
svcscm
c1315f01c9 Updating submodules
Reviewed By: zpao

fbshipit-source-id: 5a216fbe29ff5a849e75a7ed5a592efa63619268
2019-05-10 10:27:16 -07:00
Puneet Kaushik
94d9bda044 Fix the forward declaration of ImporterOptions
Summary: MSVC doesn't like the ImporterOptions declared as class and defined as struct.

Reviewed By: wez

Differential Revision: D15251102

fbshipit-source-id: e7725df5db8ee39a20a1940a8d045369800aa4c3
2019-05-10 09:25:13 -07:00
Puneet Kaushik
4dd602b299 include correct header for CoCreateGuid
Reviewed By: wez

Differential Revision: D15251104

fbshipit-source-id: 207247a950645aa33e52ecf3570bd063926deea3
2019-05-10 09:25:12 -07:00
svcscm
51dd8f9d94 Updating submodules
Reviewed By: zpao

fbshipit-source-id: 91dc5e42744a02690b9d41cd403d9a21f14d3f9b
2019-05-10 09:25:12 -07:00
svcscm
8e152931b1 Updating submodules
Reviewed By: zpao

fbshipit-source-id: 98bb7ad690a6b1e0a58d7bff4b275b5db359023f
2019-05-09 19:14:30 -07:00
Sinan Cepel
4547deafca Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Reviewed By: dkgi

Differential Revision: D15283973

fbshipit-source-id: b1451da50dbbee0f71260d50caece21799f008f2
2019-05-09 18:17:36 -07:00
Puneet Kaushik
23415bcea5 Remove class name from the member function declaration
Reviewed By: wez

Differential Revision: D15251105

fbshipit-source-id: 068574ba9d5d3c36639ed059f7de4c0f950a9284
2019-05-09 16:33:09 -07:00
Puneet Kaushik
33352a66fe Define EDEN_WIN for Mercurial cdatapack
Summary: scm/hg/lib/clib/portability/mman.h needs EDEN_WIN macro defined to include the correct definition for Eden Windows.

Reviewed By: wez

Differential Revision: D15251101

fbshipit-source-id: 9ecf55f03aac2618398ed4c962d2e210e488b724
2019-05-09 16:33:08 -07:00
Puneet Kaushik
9e24d970a8 Replace EDEN_WIN with _WIN32 for Windows only code
Reviewed By: chadaustin

Differential Revision: D15228464

fbshipit-source-id: 08b30f535fa17a5bcda906e640cc036cd0d59d02
2019-05-09 16:33:08 -07:00
Puneet Kaushik
d991894164 Moving Edenfs's Windows code to eden/fs
Reviewed By: wez

Differential Revision: D15224021

fbshipit-source-id: d046da829a6bceb73c7f05c62a77a0ef9a39e4bc
2019-05-09 16:33:08 -07:00
Yuhan Hao
4ad67b815c remove mstch from fbthrift's fbcode_builder spec
Summary: mstch is no longer required for fbthrift, we can remove it from oss build spec

Reviewed By: vitaut

Differential Revision: D15280267

fbshipit-source-id: 5008e54af9c927f23b0d6acbf0d9beb6e004eee1
2019-05-09 12:39:24 -07:00
Yuhan Hao
d763f711db mstch is no longer required for fbthrift
Summary: mstch is no longer required for fbthrift, we can remove it from manifests

Reviewed By: vitaut

Differential Revision: D15279652

fbshipit-source-id: 1772de7ab51fbfe048808f66290c4ca79de60608
2019-05-09 12:39:24 -07:00
svcscm
361ae853d1 Updating submodules
Reviewed By: zpao

fbshipit-source-id: 31f0b02454d176526e3e0a7c0b44b6266c6b739a
2019-05-09 12:39:24 -07:00
svcscm
415e8ca18d Updating submodules
Reviewed By: zpao

fbshipit-source-id: 98327ac3301e41f5639132e1d801a7bdd67f1956
2019-05-08 19:30:45 -07:00
Chad Austin
a44397fbd6 fix OverlayTest on mac
Summary: tar is in /usr/bin on macOS.

Reviewed By: simpkins

Differential Revision: D15052024

fbshipit-source-id: 5388b2d53f876717310d0f9726b74e8a283d1a89
2019-05-08 17:36:01 -07:00
Zeyi (Rice) Fan
b48f1db479 copy userName_ and userID_ during EdenConfig copy assignment
Summary:
This solves the client certificate mystery. We creates a new `EdenConfig` object
whenever there is a change detected and these two fields are lost during the
copying. Causing the variable substitution not working as expected.

Reviewed By: wez

Differential Revision: D15221180

fbshipit-source-id: 3fba36c8f261cb3dbe25e2e363d65661240be8cf
2019-05-08 16:18:32 -07:00
Adam Simpkins
1ce08e9727 turn ConfigSource into an enum class
Summary: Change `ConfigSource` from a plain `enum` to an `enum class`.

Reviewed By: chadaustin

Differential Revision: D15162811

fbshipit-source-id: 680ec1e785de1a54b623d99477914afa5d0931ca
2019-05-08 12:03:21 -07:00
Wez Furlong
8c4f92f243 getdeps: introduce build --enable-tests option
Summary:
This controls whether tests are built or not.
They are not built by default.  When `--enable-tests` is turned on,
tests are enabled for the named project only, not all of the deps.
This results in a faster build, because eg: the number of tests in
folly is very large and consumers of folly don't want to spend so
much time waiting to build tests when really all they want to do
is build their own project.

Reviewed By: strager

Differential Revision: D15246336

fbshipit-source-id: 2e22fd60c983b059f71639602712087f2a188ec6
2019-05-08 08:21:16 -07:00
Wez Furlong
5c6d7a8904 getdeps: if the hash has changed, force a cmake reconfigure
Summary:
This is needed to correctly pick up changes made to
eg: cmake.defines sections in the manifest for first-party
projects.

Reviewed By: strager

Differential Revision: D15246337

fbshipit-source-id: 35e525e885f87d6136d5ff3b94ebf34516ab947c
2019-05-08 08:21:15 -07:00
Udip Pant
a7a9264951 add fbcode_builder_config for mvfst build in oss
Summary: support for fbcode_build and travis build

Reviewed By: siyengar

Differential Revision: D15184766

fbshipit-source-id: 5117164c324d3d3c867592a88670a53b6f862477
2019-05-07 19:40:54 -07:00
Adam Simpkins
d824184669 move the ConfigSetting code to its own module
Summary:
Move the `ConfigSetting` template and related classes from `EdenConfig.h` to a
new `ConfigSetting.h` header file.

Reviewed By: chadaustin

Differential Revision: D15162815

fbshipit-source-id: 34fba0b8b0e8e50714f622bae642d20d77050673
2019-05-07 17:41:37 -07:00
Adam Simpkins
e0f2d6671f minor clean up to isValidAbsolutePath() in FieldConverter.cpp
Summary:
Update the `isValidAbsolutePath()` implementation to use `kDirSeparator`, and
update the TODO comment.

Reviewed By: chadaustin

Differential Revision: D15162812

fbshipit-source-id: f52f0e85c5eef342723947c6efaa52992da1660b
2019-05-07 17:41:37 -07:00
Adam Simpkins
b4ca002c0c move the FieldConverter code to its own file
Summary:
Move the `FieldConverter` logic out of EdenConfig.h/cpp and into its own
module.

Reviewed By: chadaustin

Differential Revision: D15162816

fbshipit-source-id: f30d81329139e5827b81a609e83baddf71b6af5e
2019-05-07 17:41:36 -07:00
Adam Simpkins
53496dcdd3 rename ClientConfig to CheckoutConfig
Summary:
Rename the `ClientConfig` class to `CheckoutConfig`, to help further progress
on updating our terminology from "client" to "checkout".

Reviewed By: chadaustin

Differential Revision: D15162814

fbshipit-source-id: 3575958f1161d5842c6f0ee9e2d2d20ab20b7372
2019-05-07 17:41:36 -07:00
Adam Simpkins
51641fffc5 turn ReloadableConfig into an implementation class
Summary:
Move the `ReloadableConfig` implementation out of `ServerState` and into the
`ReloadableConfig` class itself.  This also updates `ServerState` to simply
contain a `ReloadableConfig` member variable rather than inheriting from
`ReloadableConfig`.

This makes it easier to build other utilities that need a `ReloadableConfig`
object but do not have a full `ServerState` object, such as the
`eden_store_util` class and the `zero_blob` helper tool used by the
integration tests.

Reviewed By: chadaustin

Differential Revision: D15162813

fbshipit-source-id: 1ebc0ac6c475f4aa1090dd933ebce16bc222674c
2019-05-07 17:41:36 -07:00
svcscm
b70bb662d4 Updating submodules
Reviewed By: zpao

fbshipit-source-id: be10e59bd10367e147cd7ad0711c518d9c314efa
2019-05-07 17:41:36 -07:00
svcscm
45eec107a4 Updating submodules
Reviewed By: zpao

fbshipit-source-id: b3267d5d3b59fb5332d414acae0b74d3edc882e0
2019-05-06 20:27:30 -07:00
Adam Simpkins
b0decd09f8 update the integration tests to stop using hg_import_helper.py
Summary:
Update the integration tests to avoid explicitly using the old
`hg_import_helper.py` script.

Reviewed By: pkaush

Differential Revision: D15223982

fbshipit-source-id: 6e2310d5a9e6e0b95690a07d61d295cc3b1bca92
2019-05-06 16:47:35 -07:00
Adam Simpkins
f4e7e65f3e bump up the timeout in one of the UnixSocket tests
Summary:
This timeout appears to be too short for the last step of the test that sends
32MB of data in multiple send calls of no more than 1000 bytes each.  On
stress test runs this timeout has occasionally been firing and causing the
test to fail.

Reviewed By: chadaustin

Differential Revision: D15221306

fbshipit-source-id: d1f7ed2d58b47512a8f31f8324c61372394a6709
2019-05-06 16:10:41 -07:00
svcscm
6dd34a5b26 Updating submodules
Reviewed By: yns88

fbshipit-source-id: 0cdde8d4f33e5e943498932887949010bb5b116d
2019-05-06 16:10:41 -07:00
svcscm
a5ddb1ca01 Updating submodules
Reviewed By: yns88

fbshipit-source-id: 3e253cbb94aea2cae546d91ec69a16ee18684a40
2019-05-05 20:26:23 -07:00
svcscm
cfd3d04418 Updating submodules
Reviewed By: yns88

fbshipit-source-id: 0972ca4ab41cb6c83da5acb7630c0af010e666fb
2019-05-04 20:23:42 -07:00
svcscm
183f165629 Updating submodules
Reviewed By: yns88

fbshipit-source-id: 49abcc2e79050e1c113ebc9ed25a84299e26eeb6
2019-05-04 17:25:40 -07:00
svcscm
d43ecab69c Updating submodules
Reviewed By: yns88

fbshipit-source-id: 4197239d4f138b1a5171717fea1fe210efcc3121
2019-05-03 16:58:19 -07:00
Wez Furlong
5a1d9629dd fbcode_builder: getdeps: add build --no-deps flag
Summary:
This is useful especially on Windows where the up-to-date
checks for the dependencies take a long time.

The idea is that you might run this to start:

```
$ getdeps.py build eden
```

and then while in the edit/debug/build iteration cycle:

```
$ getdeps.py build --no-deps eden
```

Reviewed By: pkaush

Differential Revision: D15200352

fbshipit-source-id: 086f2f49db967ef4d1914a69fa80067104d79136
2019-05-03 15:59:45 -07:00
Wez Furlong
0ce8361867 fbcode_builder: getdeps: add sqlite3 builder
Summary:
Previously, we were using autoconf to build sqlite,
but that isn't available on Windows.  Instead, here's a builder
that generates a little cmake configuration for building and
installing sqlite.

Using cmake for this means that we can test the same builder
on all platforms that need to pull in sqlite.

Reviewed By: simpkins

Differential Revision: D15179387

fbshipit-source-id: fccf969182565982bd5be55545a2d2625aa99124
2019-05-03 15:59:45 -07:00
Wez Furlong
d0c7e1ccb9 eden: cmake: initial support for running unit tests
Summary:
beholdunittests

This enables some plumbing for running some of the
tests using the gtest/gmock machinery in cmake.

Part of this diff is removing the FindGMock.cmake file from the
eden repo; we now pull this in from the shared cmake library
that is populated by shipit.

Reviewed By: simpkins

Differential Revision: D14993344

fbshipit-source-id: 51caf9518c7f3a083a3b90cda10324c3a8170359
2019-05-03 15:59:45 -07:00
Wez Furlong
ce422057b1 fbcode_builder: getdeps: add LFS caching for ArchiveFetcher
Summary:
When running in FB infra, prefer to download from our local LFS
server rather than going out to the internet.

Fall back to a normal internet download if the LFS get fails for some reason.

Upload to LFS after successfully verifying the hash for the downloaded archive.

Add a subcommand that performs a fetch for all possible platforms so that it
is easier to ensure that the lfs-pointers file is up to date.

Reviewed By: simpkins

Differential Revision: D14978660

fbshipit-source-id: 240fc32fc7003d1e06c88b80d85054dae36e2f31
2019-05-03 15:59:45 -07:00