Commit Graph

9 Commits

Author SHA1 Message Date
Zeyi (Rice) Fan
61f67c044a batch fetch blobs
Summary: Instead of having `HgBackingStore` fetch blobs separately, we now try to read from hgcache and fetch from `HgImporter` as batches.

Reviewed By: chadaustin

Differential Revision: D20903245

fbshipit-source-id: d8e404d6765f1bcbacbf2a39f83eab0a351a3fe0
2020-05-05 20:59:04 -07:00
Zeyi (Rice) Fan
5bf6b58928 clean up HgQueuedBackingStore
Summary: split functions up.

Reviewed By: chadaustin

Differential Revision: D20808045

fbshipit-source-id: 3160566deb763c888a0bf34557d934feccc5ae3b
2020-05-05 20:59:04 -07:00
Zeyi (Rice) Fan
cc880f9622 populate multiple requests at once from the queue
Summary: This diff makes `HgImportRequestQueue` to be able to return multiple requests in the queue at once.

Reviewed By: chadaustin

Differential Revision: D20197070

fbshipit-source-id: 8cff1780d6e56321a756d30ac0e9b9d5d319c049
2020-05-05 20:59:03 -07:00
Zeyi (Rice) Fan
beb58087c9 track request by attaching the tracker to the Future
Summary: The life of a request is only able to be finished with one of the method. So we can instead having the tracker destroyed when the future is resolved.

Reviewed By: chadaustin

Differential Revision: D20995819

fbshipit-source-id: 5dac2f762513b5d0bcacaab7d0669fc8fdb61e80
2020-05-05 20:59:03 -07:00
Katie Mancini
7e58206c30 Explicitly track current/pending Imports -- fix pending imports
Summary:
Since imports are now queued in `HgQueuedBackingStore`
the metrics for queued imports were broken

the queueing use to happen in `HgBackingStore`, but
 now the queue happens before the imports get here,
 in `HgQueuedBackingStore`, thus these metrics were
only measuring live imports
=> this moves the metrics for queued imports into
`HgQueuedBackingStore` so that they capture both
live and queued imports

**note**: since prefetching in `HgQueuedBackingStore`
is in progress, these metrics are only set-up for when
prefetching this is finished

Reviewed By: chadaustin

Differential Revision: D20714494

fbshipit-source-id: 392337aaa60b44b75184889e6c92cb5d2c2577cc
2020-04-09 12:35:22 -07:00
Zeyi (Rice) Fan
bcc69fc668 implement prefetch for HgQueuedBackingStore
Summary: This is actually missing from `HgQueuedBackingStore`. This diff fixes this and assign low priority to these prefetch requests.

Reviewed By: chadaustin

Differential Revision: D20655681

fbshipit-source-id: f3c92b358e16e980390ac7adcae27d41ae5a7277
2020-04-06 19:12:42 -07:00
Zeyi (Rice) Fan
e32ec4398c make HgImportRequest more extensible and type safer
Summary: Ergonomics improvements. This allows us to have type safety instead of dynamically check for types at runtime.

Reviewed By: chadaustin

Differential Revision: D20740318

fbshipit-source-id: d0a3255105bef22dd236695be9fc9c4220547b83
2020-04-06 19:12:42 -07:00
Zeyi (Rice) Fan
0521a41147 add HgImportRequestQueue
Summary: This diff adds `HgImportRequestQueue` that is responsible for managing incoming requests by their priorities. This queue is later used in the `HgQueuedBackingStore` to prioritize works to the workers of the backing store.

Reviewed By: chadaustin

Differential Revision: D20197069

fbshipit-source-id: 246bbc086054a8021226e9ba6ab26d3bf0cfb7a3
2020-03-17 02:31:24 -07:00
Zeyi (Rice) Fan
7f0b3a77d4 eden: add HgImportRequest to represent an Hg import request
Summary:
This class is used to represent an import request that will be used later in the queue implementation.

When the EdenFS needs to import a blob, it creates an instance of this request and send it to the worker. Then it waits for the promise associated with the request.

In the future, we should be able to change the owned `Promise` into a non-owned `SemiFuture` to a `Promise` somewhere else for merging repetitive import requests.

Reviewed By: chadaustin

Differential Revision: D19184824

fbshipit-source-id: 823aabbed1156acf6306b7aefc76580a540d310d
2020-03-17 02:31:23 -07:00