Summary:
This could help people understand what's different essentially, and make better
decision on what to use.
Test Plan: Seems the vim spell checker is happy.
Reviewers: #sourcecontrol, rmcelroy, stash
Reviewed By: rmcelroy, stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4203039
Signature: t1:4203039:1479461745:9cce0239088707bd8b66055ba66c420e97a3bd03
Summary:
Add `getoffset`, `getallines` to make the new C API `linelog_getalllines`
usable.
Test Plan: This helps the future smartfixup extension to pass its tests
Reviewers: #mercurial, ttung, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716447
Signature: t1:3716447:1471262262:2ad8a547bf956c01e39be3383dc1a50751729c85
Summary:
`copyfrom` does not work because `rhs.buf` is invisible to `self`. Adding
a `readonly` keyword fixes it. Also add some sanity checks and remove an
unnecessary `;`.
Test Plan: `make local` and confirm `copyfrom` works.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716444
Signature: t1:3716444:1471264420:1e58798222e1515ae6d5328ad6f2727cfe41cdb5
Summary:
A newly created empty `linelog` object is not usable (i.e. calling
`annotate` or `replacelines` will raise an error) until `clear` is called.
And `replacelines` won't work until `annotate` is called. Therefore if we
know it's an empty linelog, just call `clear` and `annotate(0)` in
`__init__` to make it more friendly to use.
Test Plan:
`make local`, and confirm
`__import__('linelog').linelog().replacelines(1, 0, 0, 0, 1)` works.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716442
Signature: t1:3716442:1471262694:96370f5d3074bc0941e8ee780181c6b688cff79c
Summary:
The current linelog APIs are designed purely for the "annotate" operation.
It only supports getting lines from one single revisions, which is what
`annotate` does.
However the data structure is also useful to provide lines from different
revisions. Say if you want to know all lines from all revisions for a given
function. Or if you want to confirm if a chunk is "continuous" - nobody
inserts new lines among them. This diff adds the missing API, `getalllines`.
It returns information about all lines the linelog tracks. The caller could
provide an optional interval to limit the lines returned.
Its parameters use low-level raw offsets instead of line numbers, because it
gives the caller more control. The API don't need to handle open or close
interval issues that line numbers may have (see the previous diff about
deletion blocks).
This makes `lineinfo.offset` no longer internal-only - we have exposed the
offset concept to the API parameters. Therefore the comments are updated.
This also requires a clear way to distinguish unconditional jumps from
conditional one. We use `JGE 0` as unconditional jumps. To ensure this,
make `replacelines` reject 0 revision.
Test Plan: `cd linelog && make`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716439
Signature: t1:3716439:1471265245:5a4b2f585284cba71ca3907c8e8826b2b00a4b49
Summary:
The `appendline` logic is used twice in `annotate`: inside and outside the
`for` loop. It will be used again in the following changes. Therefore make
it an inline function.
Test Plan: `cd linelog && make`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716438
Signature: t1:3716438:1471263197:6add9cfc0a9d712258dd2da332aaf306cb1af71a
Summary:
This is a subtle change. Consider the following case, where revision 1 has 3
lines and revision 2 has 1 line inserted between the second and the third
lines of revision 1:
0: +---- INSERTED BY REV 1
| --+ --+
1: | LINE REV=1 LINENUM=0 | DEL BLOCK A | DEL BLOCK B
2: | LINE REV=1 LINENUM=1 | |
| --+ |
3: | +-- INSERTED BY REV 2 |
4: | | LINE REV=2 LINENUM=2 |
| +-- --+
5: | LINE REV=1 LINENUM=2
+----
Now what if the user has run `annotate(rev=1)` and wants to delete the first 2
lines, by calling `replacelines(a1=0, a2=2, ...)`?
The current code would use "DEL BLOCK B", which is more efficient because
the block contains more instructions, and more simple because it is just a
JGE to the a2 address.
While that works fine for the "appending" case (always edit new revisions),
it deletes future lines between the second and third line (in this case,
"LINE REV=2 LINENUM=2"). This is undesired by some users, namely smartfixup.
Surely we can add a boolean flag to the `replacelines` API to decide whether
A or B will be used. But that is too low-level and `replacelines` already
has too many parameters. Today we don't see other requirements to choose
between them precisely. Therefore, just change the default behavior, when
deleting chunks from old revisions to "DEL BLOCK A".
Test Plan: This change is needed to help pass the future smartfixup tests
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716435
Signature: t1:3716435:1471262664:0ae40d95d6389db733a88d9fafe19962d9166e35
Summary: This diff adds an `.hgignore` file for the `linelog` directory.
Test Plan: `make local`, and run `hg status`, make sure no unwanted files are printed.
Reviewers: #mercurial, ttung, akushner
Reviewed By: akushner
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3699575
Signature: t1:3699575:1471151389:7e4de7b75d671d3f0db18337e57c65e9972de059
Summary:
The test will call "replacelines" with random arguments, keeps track of the
content of "lines" on its own. Then it compares its own "lines" with linelog's
annotateresult - should be the same. After that, it verifies the content of
old revisions can be retrieved by using "annotate" correctly.
Test Plan: Run this test
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3709431
Signature: t1:3709431:1471012434:68ca06c0b3b2705740375c32acac8012ded404a5
Summary:
This diff resolves some warnings when building on CentOS 6:
```
linelog/pyext/../linelog.c: In function 'replacelines':
linelog/pyext/../linelog.c:290: warning: unused variable 'result'
....
linelog/pyext/linelog.c: In function '__pyx_f_7linelog_11_filebuffer_resize':
linelog/pyext/linelog.c:3253: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result
```
Test Plan:
Run `make local` with the linelog `setup.py` patch applied and make sure
there are no warnings.
Reviewers: #mercurial, rmcelroy, ttung
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3699378
Signature: t1:3699378:1470863807:10dc21033ab7301cb3d56e04de1c96f4d0287b6e
Summary:
It's "Unnamed unions", a C11 feature.
Although it improves the code readability a bit, gcc 4.4 does not have a
complete for it. Since we have to support gcc 4.4 now, use plain `struct`
instead.
Test Plan: Run `cd linelog && make local` using gcc 4.4
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3699216
Signature: t1:3699216:1470862720:85ee1c4f8c63805aeffca1048ff330b91a096222
Summary:
Finally! This is the linelog class for CPython users. It has all linelog APIs
exposed but hides low-level details about `linelog_buf` and `annotateresult`.
It could use either `_memorybuffer` or `_filebuffer` as the underlying
`linelog_buf`. With `copyfrom`, it has the flexibility to exchange data
between disk and memory.
Test Plan: `cython linelog.pyx`
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3662882
Signature: t1:3662882:1470403425:c17bf2595edc88bc24e328524607f4ede6bbd2a5
Summary:
The memory buffer implements `_buffer.resize` using `realloc`,
making linelog useful for in-memory modifications. For example,
the `smartfixup` extension (D3264203) would use an in-memory
linelog to avoid expensive merge operations and possible merge
conflicts.
Test Plan: `cython2 linelog.pyx`
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3655491
Tasks: 11291522
Signature: t1:3655491:1470402602:891fade362c8e0f6a3af64f1fbdfa0fbf556e68b
Summary:
This diff adds a check: after `linelog_updatelines`, `annotateresult *ar`
should be updated as if we have called a separate `linelog_annotate`.
Test Plan:
Run `afl` for a while and make sure it does not find any crashes:
```
mkdir ./testdir
CC=afl-gcc make
./linelogcli ./testdir/readme init annotate 0 replacelines 10 0:0 0:3 replacelines 20 2:2 1:3 replacelines 30 1:3 1000:1000
afl-fuzz -i testdir -o findingdir ./linelogcli @@ dump annotate 10 replacelines 10 0:0 0:3 replacelines 20 2:2 1:3 replacelines 30 1:3 1000:1000
```
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3650197
Signature: t1:3650197:1470336333:2f5e6d2e914917b33bddc595344d2a0b98067606
Summary:
Previously, `replacelines` assume the `annotateresult` user passed has the
property: `linecount < maxlinecount`. But that's not true after
`linelog_annotateresult_clear`, where `linecount = maxlinecount = 0` and
that leads to `SIGSEGV` if both `a1` and `a2` passed to `replacelines` are `0`.
This issue was found by the following `afl` testcase:
./linelogcli ./testdir/readme init annotate 0 replacelines 10 0:0 0:3 replacelines 20 2:2 1:3 replacelines 30 1:3 1000:1000
afl-fuzz -i testdir -o findingdir ./linelogcli @@ dump annotate 10 replacelines 10 0:0 0:3 replacelines 20 2:2 1:3 replacelines 30 1:3 1000:1000
Test Plan:
Run the above `afl` test. Make sure it won't report a crash in a couple
of minutes.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3676049
Signature: t1:3676049:1470661720:e5c4d6f45ac51d58fae856b08f79982bec44b535
Summary:
The thin wrapper defines common APIs like `resize` `flush`, `close`, and
`copyfrom`. Some of them are used in sub-classes. It also handles
`LINELOG_RESULT_ENEEDRESIZE` and raise `LinelogError` for other errors.
Test Plan: `cython2 linelog.pyx`
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3655463
Tasks: 11291522
Signature: t1:3655463:1470402560:75e552ed3d6205ae45812d1cdfe7bc14cfb1eec3
Summary:
`linelog.pyx` is meant to be the CPython wrapper for linelog.
This diff declares types and functions in `linelog.h`.
According to http://docs.cython.org/en/latest/src/userguide/external_C_code.html,
we have to duplicate them as Cython cannot parse `.h` files.
Test Plan: `cython2 linelog.pyx`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3655437
Tasks: 11291522
Signature: t1:3655437:1470336663:802ede2b82790a6c901ba0cc76c4f39fcac1e6ac
Summary: This is the core API to do writes.
Test Plan: `gcc -Wall -Wextra -Wconversion -c linelog.c`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3641637
Tasks: 12416202
Signature: t1:3641637:1470335236:758df55835ac6c42212783d4cc12302588ce2216
Summary:
Implement `linelog_annotate`, which is the core algorithm for generating
`linelog_annotateresult`. It is basically walking though and executing
the instructions.
Test Plan: `gcc -Wall -Wextra -Wconversion -c linelog.c`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3629591
Tasks: 12416202
Signature: t1:3629591:1470157936:764bb7c5d1208e73d93d26063a8ad380dbceb495
Summary:
As mentioned in D3628571, the `.c` code manages the memory of
`linelog_annotateresult.lines`. We need a resize function for it.
Test Plan:
`gcc -Wall -Wextra -Wconversion -c linelog.c` and it only reports
"defined but not used" warnings.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3629350
Tasks: 12416202
Signature: t1:3629350:1470055356:01993a6df2ccd849ed01961eb996c86bb0a17c78
Summary:
As described in the README (D3628440), the linelog buffer is an array of
instructions. The `.h` file only declares the plain buffer but not the
internal representation of the instruction. They are intended to be
invisible to users.
This diff declares the instruction structure, related types and helpers in
`linelog.c`. `decode` and `encode` translate between the internal (`.c`,
structured `linelog_inst`) and the external (`.h`, plain `linelog_buf`)
representation. Other macros are to help making code shorter and easier to
understand.
Test Plan: `gcc -Wall -Wextra -Wconversion -c linelog.c`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3629132
Signature: t1:3629132:1470157631:4523c53a3af28061828980845d3acfbd82c58361
Summary:
Declare core APIs for linelog:
- `annotate`, the "read" part
- `replacelines`, the "write" part
As mentioned in D3334518, this is part of a rewrite of D3334518.
These 2 APIs are the only ones involving core logic of the linelog structure.
Test Plan: `gcc -Wall -Wextra -Wconversion linelog.h`
Reviewers: #mercurial, zamsden, ttung, simonfar
Reviewed By: simonfar
Subscribers: akushner, zamsden, simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3628848
Tasks: 12416202
Signature: t1:3628848:1470080022:1f434d998c0bf63a494f70d47bc6a6fc05954446
Summary:
Declare some trivial APIs, which either clear the memory or return a value
without much calculation.
Test Plan: `gcc -Wall -Wextra linelog.h`
Reviewers: #mercurial, simonfar, ttung
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3628681
Tasks: 12416202
Signature: t1:3628681:1469647679:437db1610dec8a21a71b2d94c64d6d085321e1cd
Summary:
Add essential data structures for upcoming APIs.
This is part of a rewrite of D3334518. The new API removes everything about
I/O (aka. `stdio.h`) and makes it the caller's responsibility to prepare
(`malloc` or `mmap`) and resize the memory buffer.
This makes it possible to have a much shorter (thus easier to review) code
more dedicated to core algorithm, as the following are no longer necessary:
- I/O related APIs: `open`, `close`, `flush`, `saveas`
- I/O error handling
The error handling part is also changed that we now can use integer error
numbers instead of a boolean value + an error string.
Test Plan: `gcc -Wall -Wextra linelog.h`
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: durham, simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3628571
Tasks: 12416202
Signature: t1:3628571:1469643667:4fbf4fd7e624820ee6d8409e2067c8bd0e99a829
