add an extension that allows server-side hg->git and git->hg lookups
Summary:
This is a massive hack until Dewey becomes capable of answering these questions.
The plan is to use these commands in `arc diff` and `arc patch` to send the Git versions of the base commit over, so that the diff and land infra reaches parity with Git.
This should also unblock configerator-hg and allow the landing strip to work.
Test Plan:
In my `fbandroid-from-git` repo, enabled this extension, and then added the config:
```
[gitlookup]
mapfile = git-mapfile
```
Then, ran
```
hg id -r _gitlookup_hg_1fe73e73af4acadf1865a41e3aaa518a70cc2302 ssh://localhost//data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `76327beadc98c7c0559cd8aa36f0a25d93d781a5`, the Git equivalent of that commit.
After this, ran
```
hg id -r _gitlookup_git_76327beadc98c7c0559cd8aa36f0a25d93d781a5 ssh://localhost/data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `1fe73e73af4acadf1865a41e3aaa518a70cc2302`, as expected.
Reviewers: dschleimer, akushner, davidsp, pyd, mpm, durham
Reviewed By: durham
Subscribers: pradvenkat, akshay, thawan
Differential Revision: https://phabricator.fb.com/D1330813
Tasks: 3751836
2014-05-15 03:17:25 +04:00
|
|
|
# gitlookup.py - server-side support for hg->git and git->hg lookups
|
|
|
|
#
|
|
|
|
# Copyright 2014 Facebook, Inc.
|
|
|
|
#
|
|
|
|
# This software may be used and distributed according to the terms of the
|
|
|
|
# GNU General Public License version 2 or any later version.
|
|
|
|
#
|
|
|
|
# This will look up hashes from an hg-git map file over the wire. Define the
|
|
|
|
# location of the map file with the gitlookup.mapfile config option, then use
|
|
|
|
# thus:
|
|
|
|
#
|
|
|
|
# - get the git equivalent of hg 47d743e068523a9346a5ea4e429eeab185c886c6
|
|
|
|
# hg identify --id -r _gitlookup_hg_47d743e068523a9346a5ea4e429eeab185c886c6 ssh://server/repo
|
|
|
|
# - get the hg equivalent of git 6916a3c30f53878032dea8d01074d8c2a03927bd
|
|
|
|
# hg identify --id -r _gitlookup_git_6916a3c30f53878032dea8d01074d8c2a03927bd ssh://server/repo
|
|
|
|
#
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
# This also provides client and server commands to download all the Git metadata
|
|
|
|
# via bundle2.
|
add an extension that allows server-side hg->git and git->hg lookups
Summary:
This is a massive hack until Dewey becomes capable of answering these questions.
The plan is to use these commands in `arc diff` and `arc patch` to send the Git versions of the base commit over, so that the diff and land infra reaches parity with Git.
This should also unblock configerator-hg and allow the landing strip to work.
Test Plan:
In my `fbandroid-from-git` repo, enabled this extension, and then added the config:
```
[gitlookup]
mapfile = git-mapfile
```
Then, ran
```
hg id -r _gitlookup_hg_1fe73e73af4acadf1865a41e3aaa518a70cc2302 ssh://localhost//data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `76327beadc98c7c0559cd8aa36f0a25d93d781a5`, the Git equivalent of that commit.
After this, ran
```
hg id -r _gitlookup_git_76327beadc98c7c0559cd8aa36f0a25d93d781a5 ssh://localhost/data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `1fe73e73af4acadf1865a41e3aaa518a70cc2302`, as expected.
Reviewers: dschleimer, akushner, davidsp, pyd, mpm, durham
Reviewed By: durham
Subscribers: pradvenkat, akshay, thawan
Differential Revision: https://phabricator.fb.com/D1330813
Tasks: 3751836
2014-05-15 03:17:25 +04:00
|
|
|
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
from mercurial import bundle2, cmdutil, exchange, extensions, encoding, hg
|
2014-06-05 04:17:15 +04:00
|
|
|
from mercurial import util, wireproto, error
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
from mercurial.node import nullid
|
|
|
|
from mercurial.i18n import _
|
2014-11-03 23:39:16 +03:00
|
|
|
import errno, urllib
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
|
|
|
|
cmdtable = {}
|
|
|
|
command = cmdutil.command(cmdtable)
|
add an extension that allows server-side hg->git and git->hg lookups
Summary:
This is a massive hack until Dewey becomes capable of answering these questions.
The plan is to use these commands in `arc diff` and `arc patch` to send the Git versions of the base commit over, so that the diff and land infra reaches parity with Git.
This should also unblock configerator-hg and allow the landing strip to work.
Test Plan:
In my `fbandroid-from-git` repo, enabled this extension, and then added the config:
```
[gitlookup]
mapfile = git-mapfile
```
Then, ran
```
hg id -r _gitlookup_hg_1fe73e73af4acadf1865a41e3aaa518a70cc2302 ssh://localhost//data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `76327beadc98c7c0559cd8aa36f0a25d93d781a5`, the Git equivalent of that commit.
After this, ran
```
hg id -r _gitlookup_git_76327beadc98c7c0559cd8aa36f0a25d93d781a5 ssh://localhost/data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `1fe73e73af4acadf1865a41e3aaa518a70cc2302`, as expected.
Reviewers: dschleimer, akushner, davidsp, pyd, mpm, durham
Reviewed By: durham
Subscribers: pradvenkat, akshay, thawan
Differential Revision: https://phabricator.fb.com/D1330813
Tasks: 3751836
2014-05-15 03:17:25 +04:00
|
|
|
|
|
|
|
def wrapwireprotocommand(command, wrapper):
|
|
|
|
'''Wrap the wire proto command named `command' in table
|
|
|
|
|
|
|
|
Just like extensions.wrapcommand, except for wire protocol commands.
|
|
|
|
'''
|
|
|
|
assert util.safehasattr(wrapper, '__call__')
|
|
|
|
origfn, args = wireproto.commands[command]
|
|
|
|
def wrap(*args, **kwargs):
|
|
|
|
return util.checksignature(wrapper)(
|
|
|
|
util.checksignature(origfn), *args, **kwargs)
|
|
|
|
wireproto.commands[command] = wrap, args
|
|
|
|
return wrapper
|
|
|
|
|
|
|
|
def remotelookup(orig, repo, proto, key):
|
|
|
|
k = encoding.tolocal(key)
|
|
|
|
if k.startswith('_gitlookup_'):
|
|
|
|
ret = _dolookup(repo, k)
|
|
|
|
if ret is not None:
|
|
|
|
success = 1
|
|
|
|
return '%s %s\n' % (success, ret)
|
|
|
|
return orig(repo, proto, key)
|
|
|
|
|
|
|
|
def _dolookup(repo, key):
|
|
|
|
mapfile = repo.ui.configpath('gitlookup', 'mapfile')
|
|
|
|
if mapfile is None:
|
|
|
|
return None
|
|
|
|
# direction: git to hg = g, hg to git = h
|
|
|
|
if key.startswith('_gitlookup_git_'):
|
|
|
|
direction = 'tohg'
|
|
|
|
sha = key[15:]
|
|
|
|
elif key.startswith('_gitlookup_hg_'):
|
|
|
|
direction = 'togit'
|
|
|
|
sha = key[14:]
|
|
|
|
else:
|
|
|
|
return None
|
|
|
|
hggitmap = open(mapfile, 'rb')
|
|
|
|
for line in hggitmap:
|
|
|
|
gitsha, hgsha = line.strip().split(' ', 1)
|
|
|
|
if direction == 'tohg' and sha == gitsha:
|
|
|
|
return hgsha
|
|
|
|
if direction == 'togit' and sha == hgsha:
|
|
|
|
return gitsha
|
|
|
|
return None
|
|
|
|
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
@command('gitgetmeta', [], '[SOURCE]')
|
|
|
|
def gitgetmeta(ui, repo, source='default'):
|
|
|
|
'''get git metadata from a server that supports fb_gitmeta'''
|
|
|
|
source, branch = hg.parseurl(ui.expandpath(source))
|
|
|
|
other = hg.peer(repo, {}, source)
|
|
|
|
ui.status(_('getting git metadata from %s\n') %
|
|
|
|
util.hidepassword(source))
|
2015-04-18 00:20:20 +03:00
|
|
|
kwargs = {'bundlecaps': set(['HG20'])}
|
2014-09-18 03:40:37 +04:00
|
|
|
capsblob = bundle2.encodecaps(bundle2.getrepocaps(repo))
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
kwargs['bundlecaps'].add('bundle2=' + urllib.quote(capsblob))
|
|
|
|
# this would ideally not be in the bundlecaps at all, but adding new kwargs
|
|
|
|
# for wire transmissions is not possible as of Mercurial d19164a018a1
|
|
|
|
kwargs['bundlecaps'].add('fb_gitmeta')
|
|
|
|
kwargs['heads'] = [nullid]
|
2014-09-18 03:40:37 +04:00
|
|
|
kwargs['cg'] = False
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
bundle = other.getbundle('pull', **kwargs)
|
|
|
|
try:
|
|
|
|
op = bundle2.processbundle(repo, bundle)
|
2014-06-05 04:17:15 +04:00
|
|
|
except error.BundleValueError, exc:
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
raise util.Abort('missing support for %s' % exc)
|
|
|
|
writebytes = op.records['fb:gitmeta:writebytes']
|
|
|
|
ui.status(_('wrote %d files (%d bytes)\n') %
|
|
|
|
(len(writebytes), sum(writebytes)))
|
|
|
|
|
2014-11-03 23:39:16 +03:00
|
|
|
gitmetafiles = set(['git-mapfile', 'git-named-branches', 'git-tags',
|
|
|
|
'git-remote-refs'])
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
|
2014-10-08 05:19:17 +04:00
|
|
|
@exchange.getbundle2partsgenerator('gitmeta')
|
|
|
|
def _getbundlegitmetapart(bundler, repo, source, bundlecaps=None, **kwargs):
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
'''send git metadata via bundle2'''
|
2014-10-08 05:19:17 +04:00
|
|
|
if 'fb_gitmeta' in bundlecaps:
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
for fname in sorted(gitmetafiles):
|
|
|
|
try:
|
|
|
|
f = repo.opener(fname)
|
|
|
|
except (IOError, OSError), e:
|
2014-11-03 23:39:16 +03:00
|
|
|
if e.errno != errno.ENOENT:
|
|
|
|
repo.ui.warn(_("warning: unable to read %s: %s\n") %
|
|
|
|
(fname, e))
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
continue
|
2015-04-18 00:20:20 +03:00
|
|
|
part = bundle2.bundlepart('fb:gitmeta',
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
[('filename', fname)],
|
|
|
|
data=f.read())
|
|
|
|
bundler.addpart(part)
|
|
|
|
|
2015-04-18 00:20:20 +03:00
|
|
|
@bundle2.parthandler('fb:gitmeta', ('filename',))
|
add a command and server-side support to fetch git metadata
Summary:
When a machine is newly set up, it needs to have the git mapfile available. Generating it from scratch is prohibitively slow for large repos like `configerator`, so instead add a command that can download the entire git metadata from a server that has it.
This is a temporary hack while I work on a real fix to upstream to hg-git. A real fix would be
- part of hg-git
- not send the entire git metadata over, just the bits that are needed based on a common/heads computation
- be dependent on bundle2
- (probably) be part of the pull operation, not a separate command
- be configurable with an option
This isn't part of hg-git, even in a private branch, to emphasize its temporary nature. `gitlookup` might also not be the best fit, but I want to avoid adding yet another extension for a temporary hack.
Test Plan:
Ran the following commands. Note that the current version of `hg` deployed to devservers is broken, so we need to deploy an updated hg before enabling this. These commands were tested against 1ced7c762592
(in `fbandroid-hg`, with `gitlookup` enabled on both ends and the remotefilelog fix in D1341059)
hg gitgetmeta ../fbandroid-from-git
Saw that 3 files were written out.
hg gitgetmeta ssh://localhost//data/users/sid0/fbandroid-from-git
Saw that 3 files were written out.
Reviewers: dschleimer, davidsp, akushner, durham, pyd
Reviewed By: pyd
Differential Revision: https://phabricator.fb.com/D1341166
Tasks: 3751836
2014-05-21 09:32:54 +04:00
|
|
|
def bundle2getgitmeta(op, part):
|
|
|
|
'''unbundle a bundle2 containing git metadata on the client'''
|
|
|
|
params = dict(part.mandatoryparams)
|
|
|
|
if 'filename' not in params:
|
|
|
|
raise util.Abort(_("gitmeta: 'filename' missing"))
|
|
|
|
fname = params['filename']
|
|
|
|
if fname not in gitmetafiles:
|
|
|
|
ui.warn(_("warning: gitmeta: unknown file '%s' skipped\n") % fname)
|
|
|
|
return
|
|
|
|
f = op.repo.opener(fname, 'w+', atomictemp=True)
|
|
|
|
try:
|
|
|
|
data = part.read()
|
|
|
|
op.repo.ui.note(_('writing .hg/%s\n') % fname)
|
|
|
|
f.write(data)
|
|
|
|
op.records.add('fb:gitmeta:writebytes', len(data))
|
|
|
|
finally:
|
|
|
|
f.close()
|
|
|
|
|
add an extension that allows server-side hg->git and git->hg lookups
Summary:
This is a massive hack until Dewey becomes capable of answering these questions.
The plan is to use these commands in `arc diff` and `arc patch` to send the Git versions of the base commit over, so that the diff and land infra reaches parity with Git.
This should also unblock configerator-hg and allow the landing strip to work.
Test Plan:
In my `fbandroid-from-git` repo, enabled this extension, and then added the config:
```
[gitlookup]
mapfile = git-mapfile
```
Then, ran
```
hg id -r _gitlookup_hg_1fe73e73af4acadf1865a41e3aaa518a70cc2302 ssh://localhost//data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `76327beadc98c7c0559cd8aa36f0a25d93d781a5`, the Git equivalent of that commit.
After this, ran
```
hg id -r _gitlookup_git_76327beadc98c7c0559cd8aa36f0a25d93d781a5 ssh://localhost/data/users/sid0/fbandroid-from-git -i --debug
```
Saw that the last line printed out was `1fe73e73af4acadf1865a41e3aaa518a70cc2302`, as expected.
Reviewers: dschleimer, akushner, davidsp, pyd, mpm, durham
Reviewed By: durham
Subscribers: pradvenkat, akshay, thawan
Differential Revision: https://phabricator.fb.com/D1330813
Tasks: 3751836
2014-05-15 03:17:25 +04:00
|
|
|
def extsetup(ui):
|
|
|
|
wrapwireprotocommand('lookup', remotelookup)
|
|
|
|
|