Summary: We used to inline smaller blobs into the data table, and only chunk bigger blobs. Performance tests show that we don't have significant issues with round trips to our MySQL infrastructure, so let's simplify the problems of deduplication and hard linking by always storing the data in chunks, and having the keys directly return significant data.
Reviewed By: ahornby
Differential Revision: D21939500
fbshipit-source-id: 6ad73b25ac859fb1b4f067a3016516b713e8b2f5
Summary: [Folly] Cut the FOR_EACH_KV macro, which may be replaced by a combination of range-for and structured bindings.
Differential Revision: D21826182
fbshipit-source-id: ce4712afd3d0d7806eb1fca8c97009da117f982e
Summary:
Add a disabled test that illustrates that binary data fails to
serialize between edenfs and the CLI with the current Thrift
implementation.
Reviewed By: fanzeyi
Differential Revision: D21890379
fbshipit-source-id: 725e99751e1d62c39f4059ec0f1197857c47e3b4
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary: Now that the blobstore_sync_queue has no unique key we do not expect duplicate key errors, and would like to know of any other problems occuring on insert.
Reviewed By: StanislavGlebik
Differential Revision: D21618062
fbshipit-source-id: 85991184234a807553388b456ad76971104000a3
Summary: I made a change before landing my last diff and didn't retest.
Reviewed By: xavierd
Differential Revision: D21944268
fbshipit-source-id: a5c820cef5b0fc019c7be5d096db9d0e7b3e5a44
Summary:
Introduced a new local commit metadata storage system, smallcommitmetadata, which stores a simple local JSON file for a mapping of (node, category) -> value.
This data can be manipulated with the debugsmallcommitmetadata command, and can be consumed in the smartlog using the smallcommitmeta template func.
Total number of entries can be limited with smallcommitmetadata.entrylimit configuration option, which defaults to 100.
Reviewed By: DurhamG
Differential Revision: D21673611
fbshipit-source-id: 2239a47867118dd86b15944058505ddf67548549
Summary:
Let's record the caller of the hg command that's using deprecated
logic. In most cases the caller will be the unix username, but we'd like to get
services to set an environment variable with their oncall so we could associate
hg usage with specific oncalls and eventually automatically file tasks for using
deprecated functionality.
Reviewed By: markbt
Differential Revision: D21891753
fbshipit-source-id: a4de0d348c583777cc3eeb9be26510e6af8b3a27
Summary:
Marks a number of commands as deprecated. The default ui.deprecate()
behavior is to just log the usage to scuba, so this diff doesn't actually change
any behavior.
Reviewed By: markbt
Differential Revision: D21891192
fbshipit-source-id: 4611e31cf630eee4f07443fe242c759c3cb3fe31
Summary:
Adds a simple ui.deprecate() function for deprecating code paths. It
has several levels of deprecation and can optionally automatically escalate
through those levels across a specified time frame.
The levels are:
- Log - just logs the usage to scuba
- Warn - same as Log but also prints a warning to the user
- Slow - same as Warn but also sleeps for 2 seconds
- OptIn - throws an exception, but hints that the exception can be bypassed by
setting 'deprecated.bypass-XXXX=True'
- Block - throws an exception that cannot be bypassed
I also call it from a few locations that are expected to be deprecated already.
Reviewed By: markbt
Differential Revision: D21890496
fbshipit-source-id: faddb301888ef75cc71d46ffb7374f3fe3d044bd
Summary:
`chg` is now integrated as `#chg-compatible` by the test framework. The helper
script no longer works.
Reviewed By: markbt
Differential Revision: D21895009
fbshipit-source-id: 257ed081fbd1a6acc97afbfa2934134e389ab400
Summary:
This requires `--hidden` for `hg tip` to show the pending commits in hooks.
The real fix would be handling pending visibility changes in metalog layer.
Reviewed By: markbt
Differential Revision: D21894321
fbshipit-source-id: 3b2296c9c38b37aee2c57eea0ad03c3acaab7507
Summary:
The test depends on lots of implementation details of obsmakers (ex. prune
markers, etc). In production we use mutation templates tested by
test-mutation.t. Therefore remove the test.
Reviewed By: markbt
Differential Revision: D21894319
fbshipit-source-id: 549423b1f5f45cf567b3c95e5fe3ccfa30b6695a
Summary:
The mount path is required in 2 places: at mount time, and when deleting files.
It turns out that the second one doesn't require it as PrjDeleteFile specify
the filename to be: "A null-terminated Unicode string specifying the path,
relative to the virtualization root, to the file or directory to be deleted."
Thus, we only need it during mount time, so let's pass it to the start method.
Reviewed By: fanzeyi
Differential Revision: D21797080
fbshipit-source-id: 27f5d18841ac077c81ad3f6b3686e20917505c41
Summary:
This will simplify some later diffs where the channel construction doesn't
change any state.
Reviewed By: fanzeyi
Differential Revision: D21797077
fbshipit-source-id: 67d9ee693b067a08986e9bd33f9d91f9a1dadac1
Summary: This is already created, no need for re-recreating it.
Reviewed By: fanzeyi
Differential Revision: D21676314
fbshipit-source-id: 8e3eecfb6069f9bed5bfec7ad6631ed8830d796d
Summary:
These are pure implementation details of ProjectedFs, and don't need to be in
the class, let's move them onto an anonymous namespace in the cpp file.
Reviewed By: chadaustin
Differential Revision: D21676315
fbshipit-source-id: 4fa7088a10256581d7eed2c7cbd90c30a7fef3d2
Summary:
I find this approach to be slighly less error prone due to not having
to manually specify the array size in advance. We lose the field names,
but they should be pretty self explanatory.
Reviewed By: chadaustin
Differential Revision: D21676319
fbshipit-source-id: db4dc993d3dd2a502ae188b08b8d4230939c0415
Summary:
This moves the edenToWinPath inside Windows specific files and makes
the interface clear about the kind of path that is expected.
Reviewed By: fanzeyi
Differential Revision: D21676318
fbshipit-source-id: d153bb543da5f481e94f83d52e45c6cc871d1f54
Summary:
The only reason tests need to set an FsChannel is for calling
cleanupPrjfsCache. For consistency sake, let's do the same as what we do on
unices: verify if we have a valid channel first.
Reviewed By: wez
Differential Revision: D21676316
fbshipit-source-id: 022ca91d32a7ab047a79acccd7d17dfd7906d527
Summary:
The only missing piece on Windows is the bind mount/redirection due to
folly::Subprocess not being present on Windows.
Reviewed By: wez
Differential Revision: D21676313
fbshipit-source-id: a5ba09be04c94b66edf9d40884753afa3865def2
Summary:
We want to use these on Windows too, so let's rename them to not be FUSE
specific.
Reviewed By: wez
Differential Revision: D21676317
fbshipit-source-id: 9f1bde9e0467d8828d4168a69581c8fea54669cd
Summary:
This moves it out of EdenServer and into EdenMount directly. This is closer
to what is happening in the Linux code and thus better.
Reviewed By: fanzeyi
Differential Revision: D21663501
fbshipit-source-id: 84420fcb37586cc2695568f0167b39f91c9bfb63
Summary:
Instead of having 2 different functions for Windows and other unices, let's
only have one. This allows more of the mount code to be shared between both.
Reviewed By: wez
Differential Revision: D21517477
fbshipit-source-id: 185ce864aecdf9a7880a9c705df444ec1e39319a
Summary:
Add a permission checker that verifies that the user is permitted to make
requests on behalf of a service. This uses a different ACL than the usual repo
ACL.
Reviewed By: mitrandir77
Differential Revision: D21886531
fbshipit-source-id: bceb63aed12c11ebe2a3d65a2560c014267b0e20
Summary:
Regexes stored in config need to be comparable so that config is comparable.
Normally regexes are not comparable, so wrap them in a newtype wrapper that
implements comparison, rather than manually implemented PartialEq and Eq for
anything that contains them.
Reviewed By: mitrandir77
Differential Revision: D21886533
fbshipit-source-id: 1fb39c0874daed383624eeda61c903a4731b9cb8
Summary:
The repository write methods are likely to get fairly large. Move
`create_changeset` to its own module so that we can add future methods as
separate modules.
Reviewed By: mitrandir77
Differential Revision: D21886532
fbshipit-source-id: 08de34a400d056bc97cf6e54856f2a2fd185814c
Summary:
batch_derive() is a dangerous function to use. I'd love to delete it but this
function is very useful for backfilling, so unfortunately I can't.
The problem arises when one tries to backfill blame and unodes simultaneously
(or just derive blame which in turn derives unodes). While batch_derive()
tries to be careful with inserting "outer" derived data's mappings (i.e. blame
mapping), it doesn't do it for inner derived data mappings (i.e. unodes). So we
might end up in the situation where we insert unodes mapping before we inserted
all the manifests for it. If this thing fails in the middle of derivation then
we have a corruption.
Let's do not use it in blobimport. It will make derivation slower, but I'd
rather make it slower than incorrect.
Reviewed By: farnz
Differential Revision: D21905619
fbshipit-source-id: c0227df195a8cf4482b2452ca928acbc5750b3e5
Summary: This will be very useful when we migrate to Mononoke.
Reviewed By: StanislavGlebik
Differential Revision: D21912115
fbshipit-source-id: 1ed2204f17f90152a67fd4e10b0ae61aacfc41d7
Summary:
There are people that are hurt by usage of these terms, this should be more
then enough reason to replace these. Newly chosen terms are more
self-explanatory as well.
This doesn't yet touch the actualy config files, as that requires a bit more
effort than 1 diff and will require more coordination.
Reviewed By: krallin
Differential Revision: D21924440
fbshipit-source-id: e24fc638dc8c9d6d20b6f3fa5f0d0bbc91bbf77b
Summary:
This test checks that we can start Mononoke and serve pull/push/update with
filenodes
Reviewed By: ahornby
Differential Revision: D21904753
fbshipit-source-id: 86690c5ed5ce7d022844809b09beb25c7961cac8
Summary: Let's log to ods so that we can check what's hapenning with filenodes.
Reviewed By: ahornby
Differential Revision: D21904400
fbshipit-source-id: e602dfc338c02252cad286176a1965bdc7043d7f
Summary:
This diff migrates add_filenodes method to return FilenodeResult.
That means that all filenodes methods now return FilenodeResult and it's time
now to remove TODOs from derived_data filenodes.
Note that I had to change the test "derive_disabled_filenodes" a bit.
Previously FilenodesOnlyPublic::mapping::get() method immediately returned
FilenodesOnlyPublic::Disabled, while now it returns None if hg changeset is not
derived. This is an expected change in behaviour, so I just updated the test to
try to derive FilenodesOnlyPublic first, which in turns triggers generation of hg changeset.
Reviewed By: ahornby
Differential Revision: D21904401
fbshipit-source-id: f6f4cd14e6cdce5a4b95d8f3f9acff305ae6fa88
Summary:
Similar to get_all_filenodes_maybe_stale() make this method return
FilenodeResult if filenodes are disabled.
Note: this diff adds one TODO in fetch_root_filenode, which will be removed
together with other TODOs in the next diff.
Reviewed By: ahornby
Differential Revision: D21904399
fbshipit-source-id: 1569579699c02eb07021f8143aa652aa192d23bc
Summary:
With the internal streampager, progress bars must be sent on a separate stream so that
streampager can render them correctly.
Reviewed By: quark-zju
Differential Revision: D21906173
fbshipit-source-id: eb41b0bf22807d9cae518b3f676996ab1c642c6e
Summary: Upgrade the internal streampager to the latest version, which includes line wrapping.
Reviewed By: quark-zju
Differential Revision: D21906172
fbshipit-source-id: 1bc63723f55ac115090e7ae0a2541158863056b9
Summary: The test is changed because an extra file is copied during local-repo clone.
Reviewed By: markbt
Differential Revision: D21894318
fbshipit-source-id: 4417fb5473dc2cb98eda4dcc4d0484cc3b31fae2
Summary:
`index2` is the Rust revlog index implementation that has extra requirements,
namely, the revlog cannot be inlined. We have code to make sure changelog is
not inlined, but manifest or filelog can still be inlined.
`index2` is only needed for changelog for phase and visibility calculations.
Therefore, disable it for manifest and filelog.
Reviewed By: singhsrb
Differential Revision: D21917447
fbshipit-source-id: c45f955175c623092e6f728042dbfd3b942fe911
Summary: This diff updated `eden du` to display a summary and display warnings and cleaning information with colors.
Reviewed By: kmancini
Differential Revision: D21885051
fbshipit-source-id: be127b81c92bea1051a80715682cdbccf22f22e3
