Commit Graph

3256 Commits

Author SHA1 Message Date
Wez Furlong
1c3ba75f7a eden: pass statfs free space data through from overlay storage
Summary:
This diff passes the free space information through from
the overlay lock file descriptor back up to the kernel when the
filesystem stats are requested.

This makes the user experience with eg: Finder on macOS nicer.

Reviewed By: chadaustin

Differential Revision: D17255859

fbshipit-source-id: ed9f3b9fd386c5706539879513854a9cd1550d8a
2019-09-10 11:31:56 -07:00
svcscm
cb33eb1821 Updating submodules
Summary:
GitHub commits:

cb2d61c32e

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: db2c7b2c66a0e7e94ba04b12a4ff1f87ee30ae5f
2019-09-10 11:31:56 -07:00
svcscm
04589ce586 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 953b96eaffbde22d0596b5db0e0c862f70b9f992
2019-09-10 01:15:43 -07:00
Wez Furlong
5a304aa346 eden: add eden du command for inspecting disk usage
Summary:
Understanding disk usage and cleaning it up is a relatively frequently
asked question, and users are often shocked that "eden is using so much space"
when the reality is that they're just seeing the aggregate from a `du` command
that includes their build artifacts from their bind mounts.

This diff adds `eden du` as a friendly utility that helps to break down the
disk usage into meaningful categories and gives some hints on how to reclaim
space.

Reviewed By: chadaustin

Differential Revision: D17252322

fbshipit-source-id: 7348692c95cd319ca315c5be466b397b4969971d
2019-09-09 20:43:26 -07:00
svcscm
8b7fc1f5ac Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: df4c88ce4bd9e3099c81de7a68aa308fea80d5f7
2019-09-09 20:43:26 -07:00
svcscm
5705924bb0 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 256606da203d497ecbaddfab0b508fd2c6b764c8
2019-09-09 18:57:47 -07:00
Wez Furlong
28a4abaee0 eden: fix unmount on macos
Summary:
The macos fuse kext will block umount(2) until it gets
a reply to the FUSE_DESTROY opcode, causing `eden stop` to always
end up SIGKILL'ing the process, and thus causing an fsck on the next
startup.

So: let's respond to it after dispatching to the destroy method.

Reviewed By: chadaustin

Differential Revision: D17251386

fbshipit-source-id: 947d2aeb65edadbfbbac8b2673933ade283215b0
2019-09-09 17:06:22 -07:00
svcscm
a4b76d73b2 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 525e9bb1f905b471caa1e53eaf661699477a7ffe
2019-09-09 17:06:22 -07:00
svcscm
43680109ba Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 47b5c7d2b5d101901afa122b7050017e694e4c7b
2019-09-09 09:26:40 -07:00
generatedunixname89002005289445
9b9015c137 Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Reviewed By: grievejia

Differential Revision: D17255059

fbshipit-source-id: 4eb207bcefb699c1886d1f0c1511b6d26100ff0e
2019-09-09 07:41:21 -07:00
svcscm
32558b0ae1 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 7701bbfe62955ca44416e4e8658f2cb8b36e918a
2019-09-09 03:41:54 -07:00
svcscm
c8c936c09b Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 231e0062eb7748498cdce912b7d719a29395ea1b
2019-09-09 03:41:54 -07:00
svcscm
c6b4a02491 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: ff5b6537e82a543db8c79a25d11f1db0350b1570
2019-09-08 16:22:35 -07:00
svcscm
3cf13a8bf5 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: cd8005d67c1f00a3c6498766cee0c86bb7e91086
2019-09-07 16:13:03 -07:00
Yedidya Feldblum
3347cc797f Bake templates into the compiler binary
Summary: [Thrift] Bake templates into the compiler binary rather than having to package the templates with the compiler.

Reviewed By: stevegury, vitaut

Differential Revision: D16356056

fbshipit-source-id: b971dce7aada8dc2a26587fbf818e661acd0c555
2019-09-07 16:04:50 -07:00
svcscm
f1680e74de Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 7ace9adb613ed43ef74bbc4ba1b554f7b21137cf
2019-09-07 16:04:50 -07:00
svcscm
86db89902f Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: fbff091b6d8b680a5bf85ffbc5013ab7a89c98f4
2019-09-07 15:22:59 -07:00
svcscm
ad96d9826d Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 53864d806519150a5e3e07324373f11b308aac10
2019-09-06 15:09:38 -07:00
Wez Furlong
6fb1bb0764 getdeps: build all available boost libraries
Summary:
This should make it easier to eg: enable fbzmq on macos at a later
time, and also makes things more explicit about what is being built, and
importantly, by listing these in the manifest ensures that we have a
hash change if we change this list; we wouldn't trigger such a change
if the list were encoded solely in builder.py

Reviewed By: chadaustin

Differential Revision: D17133149

fbshipit-source-id: caf0dd45e262188eeefafe0868ef95ad257a4950
2019-09-06 14:34:19 -07:00
svcscm
310da85dd8 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 9d7dc9cb5a0d78605b28abba6e68db6e3b7e2ccf
2019-09-06 11:59:29 -07:00
Chad Austin
d57fd64f59 make dumping tracepoints its own binary
Summary: artillery doesn't work in mode/mac, so break the CLI dependency and move it into its own binary.

Reviewed By: simpkins

Differential Revision: D17135496

fbshipit-source-id: 2739560ec6930bfd277fb7cd80bef1588640ac67
2019-09-06 10:08:28 -07:00
svcscm
e3591589c1 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: b01fedbe2f90c36caef842d5b2ea9d23afe42dd9
2019-09-06 10:08:27 -07:00
svcscm
61ff5f6741 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 490a59acf2201a80a6497f6cfdd55751928e54d1
2019-09-06 08:19:29 -07:00
Wez Furlong
3091c60805 getdeps: allow aliasing sources for add_fb_python_executable
Summary:
* Fix a minor issue with `add_fb_python_executable` where it would
  emit `--main` even if `MAIN_MODULE` had not been specified
* Introduce the ability to rename python sources in the manifest so
  that they have a different install path.
* Note that this changes the DEPENDS parameters to reference abs_sources;
  previously these were using relative paths to the sources pulled directly
  from the SOURCES arguments, but since those can now contain aliasing
  operations we need to pre-process the list.  I switched to abs_sources
  because that was already being maintained.  If for some reason the
  DEPENDS list needs to be relative paths, this will need to be adjusted.

Reviewed By: simpkins

Differential Revision: D17147417

fbshipit-source-id: d0825bfbdad6be658c078aaac6645e6926a1530f
2019-09-06 07:44:35 -07:00
svcscm
5fd5ed861e Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: c88573b8b090630ff2e03392b8ba0a48703ed184
2019-09-06 07:44:35 -07:00
svcscm
9fa1d2e5f6 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 23e6e7ff4088730619793146590ca31e4cd3c64c
2019-09-06 00:46:11 -07:00
Wez Furlong
9ab72deacc getdeps: add build cache abstraction
Summary:
This diff adds a small abstraction that allows for uploading
and downloading from an artifact cache.

We try to download from this cache at build time, but will only
try to populate it for continuous builds--those are built from
code that has been reviewed and landed on master.  This restriction
helps to avoid thrashing the cache with works in progress and
results in a slightly more trustworthy state of the cache contents.

In addition to this, we choose only to cache third party projects.
The rationale is that our first party projects move too quickly to
be worth caching, especially since the cache granularity is for
the whole project rather than just changed elements of a given
project.

In a later diff I will introduce some implementations of the
cache class that work with eg: Travis or Circle CI caching.

Reviewed By: simpkins

Differential Revision: D16873307

fbshipit-source-id: 2bfb69e36615791747b499073586562f2ca48be9
2019-09-05 23:33:27 -07:00
svcscm
2415287337 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 9f234275c25be22647408f5a5cc06f7292f8e593
2019-09-05 23:33:27 -07:00
svcscm
0401d78964 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: e4cb9d7caf3c4018b8b99aa5bdfe23fbde450ed9
2019-09-05 20:51:48 -07:00
Chad Austin
e824e76a2e enable edenfs-oss build on macOS
Summary: Build the edenfs-oss daemon on macOS with Buck. :)

Reviewed By: fanzeyi

Differential Revision: D17107628

fbshipit-source-id: fed95a8622fb767246044343dc18491430e9500b
2019-09-05 16:57:17 -07:00
Chad Austin
a2e7b0afa9 build hg store on macOS with Buck
Summary:
Build the hg store on macOS with Buck. This primarily involves putting
ServiceRouter and Rust datapack support behind build switches.

Reviewed By: fanzeyi

Differential Revision: D17106241

fbshipit-source-id: da4643a11d6e5d0a6df23bff1d5e1454ab351abe
2019-09-05 16:57:16 -07:00
Chad Austin
03e07bf74f fix eden-buck-mac build
Summary:
The Eden macOS build has regressed in a few ways since it's been
broken due to RocksDB changes. Fix those small issues.

Reviewed By: fanzeyi

Differential Revision: D17107209

fbshipit-source-id: ef2e97a84e2403e7b7f36a4a01e2ceecab69106e
2019-09-05 16:57:16 -07:00
svcscm
28e94ef1db Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: c521cc23f2f96a695beaee343e553353363ad641
2019-09-05 16:57:16 -07:00
svcscm
476dc568da Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 226aa9c419a0330b54897c338b040199134c67d7
2019-09-04 23:16:15 -07:00
svcscm
2a792fda6d Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 5fb5077cb778a65d4a8b06c84b3a28938dbb88b6
2019-09-04 23:16:15 -07:00
svcscm
2bb4a2df46 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 8ea419d0fd1925a895f9993535be27660a583548
2019-09-04 13:00:23 -07:00
Adam Simpkins
92b770a40e fsck: extract orphaned symlink inodes as symlinks
Summary:
Update the fsck code to save any orphaned symlink inodes that it finds as
symlinks in the repair archive directory, rather than saving the contents as a
regular file.

Reviewed By: wez

Differential Revision: D17170346

fbshipit-source-id: 4cba8b27233b728114a80a327ab519b039297aea
2019-09-04 11:08:10 -07:00
Adam Simpkins
3f7e72dc3e automatically perform an fsck scan when needed
Summary:
Use the new `OverlayChecker` class to automatically scan for errors and
attempt to repair them if the overlay was not shutdown cleanly the previous
time it was used.

Reviewed By: wez

Differential Revision: D16596601

fbshipit-source-id: 9923565b101ba953e92909e502be6ef5895c5cbd
2019-09-04 11:08:10 -07:00
Adam Simpkins
0a1802ac0b update OverlayChecker to avoid an extra allocation for each inode
Summary:
Update the OverlayChecker's `inodes_` map to store `InodeInfo` objects
directly, rather than pointers to `InodeInfo` objects allocated on the heap.

This tweaks some of the error handling code to look up `InodeInfo` objects in
the map in a few places, to avoid storing raw pointers to `InodeInfo` objects.
The `InodeInfo` objects generally shouldn't move once we start performing
error checking, but it seems better to be conservative here and avoid using
raw pointers that are only loosely coupled with the original object's
lifetime.

This probably shouldn't really make much of a performance difference in
practice, since this code is likely to be disk I/O bound anyway.  (It does
appear to make a small difference in performance for ASAN-enabled builds when
the inode files are warmed up in the kernel cache.)

Reviewed By: wez

Differential Revision: D16750736

fbshipit-source-id: b089a259cc83ffd2fda6f83b617d95ce4ac467f9
2019-09-04 11:08:10 -07:00
svcscm
84a738ce6c Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 0cef39945756e217dfe2d8793b2918d530462bb8
2019-09-04 11:08:10 -07:00
svcscm
539a85ee92 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 71dafed269808193a769093f68dbb04b1100f6cb
2019-09-03 22:03:26 -07:00
Yedidya Feldblum
d87295fc9d Cut mstch manifest
Summary: [OpenSource] Cut `mstch` manifest, which was but is no longer used by `fbthrift`.

Reviewed By: avalonalex, davidtgoldblatt

Differential Revision: D17160816

fbshipit-source-id: 3b96142455d6990870e8ed3e5ce330d339d5a657
2019-09-03 21:39:29 -07:00
svcscm
b1925c000e Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 0682608d96cd4ef316e0be812d22d9627d2e440d
2019-09-03 21:39:29 -07:00
svcscm
0b35ce3e3c Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 4c6f0b6e6900b3da62cbd7163f469283c867f079
2019-09-03 17:24:55 -07:00
Wez Furlong
ced2517121 getdeps: assign test owner in continuous runs
Summary:
Feed the first-party oncall data through to the testpilot
invocation.  This will set the owner of the test in continuous runs.

The oncall is passed through via the sandcastle module, which means
that you will have to manually pass it through for local testing
(see test plan), but it should automatically get picked up when
sandcastle schedules the `test` step of the job.

Reviewed By: chadaustin

Differential Revision: D17146802

fbshipit-source-id: a4f0e65853a46ed8709594c96db859ede2530b00
2019-09-03 17:09:35 -07:00
svcscm
fc263c7372 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 2df5031d0d5c87b21803d02f165b4607b62b85b4
2019-09-03 17:07:01 -07:00
John Strizich
19784e4aa7 add generated constant definitions to library
Summary: when trying to build openr with `getDeps.py` constant definitions where not available since they were not part of the generated thrift lib. this fixes it.

Reviewed By: chadaustin

Differential Revision: D16064417

fbshipit-source-id: ee2b3e061caf67235f4a873405cf6cd6fb663ef4
2019-09-03 16:39:48 -07:00
svcscm
dd831aa429 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: f8fa065aa374df9a7e66877f1a934fd88702a762
2019-09-03 16:39:48 -07:00
svcscm
1b1e0312e7 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 170bf83df0abbd794b0bff6727b7245ec122ae8b
2019-09-03 13:20:29 -07:00
svcscm
7147d1d758 Updating submodules
Reviewed By: cdelahousse

fbshipit-source-id: 51a81fde82232d0151565babb27b19c573733ec7
2019-09-03 13:20:29 -07:00