hledger import -s now runs strict checks on an in-memory copy of the
updated journal, before updating the journal file; if strict checks
fail, nothing is written to disk.
And hledger import now does not update any .latest files until it has
run without error (no failing strict checks, no failure while writing
the journal file). This makes it more idempotent, so you can run it
again after fixing problems.
Strict checks now run only once, at end of the high level read operation,
and not for each individual file; this fixes some spurious --strict failures,
like account declarations not affecting a sibling file as they should.
And .latest file writing now happens as the last step, after passing
strict checks. This is mainly for the import command, but it also
means that hledger print --new now does not update .latest files
if strict checks are failing.
The file reading API has been improved and documented in more detail.
With valuation now preserving more decimal digits, roi could show
excessively precise decimals if there was no known display precision
for the valuation commodity. Now in that situation it limits the
precision to a maximum of 8 digits.
This and the preceding commits were "work in progress" that got out of control.
There's more to do, but this one brings these precision-related improvements
(at least):
When "infinite decimals" arise, they are now generally shown with
8 decimal digits rather than 255.
print and prices no longer add trailing decimal zeros unnecessarily.
Some code has been refactored or given more debug output.
All tests have been updated to match the recent changes.
- The prices comand now more accurately lists the prices that hledger
uses when calculating value reports (similar to what you'd see with
eg `hledger bal -V --debug=2`).
- The prices command's --infer-reverse-prices flag was confusing since
we always infer and use reverse prices; it has been renamed to --show-reverse.
- --infer-market-prices and --show-reverse combine properly.
- --show-reverse now ignores all zero prices rather than giving an error.
- Reverse prices (which can be infinite decimals) are now displayed
with at most 8 decimal digits (rather than the internal precision of
255 digits).
- Filtering prices by cur: or amt: now works properly.
- Price amounts are styled, but all decimal digits are shown.
Cost/value conversion now applies the standard display style, and
sets the display precision equal to the internal decimal precision
(or 8 if the decimal appears to be infinite).
This means value reports and especially `print -V` now show amounts
with more accurate and standard style and precision.
New tests have been added describing and explaining various
style/precision behaviours in print cost/value reports.
Add a description of Match Groups to the manual; Section "Matchers".
Include two examples.
Clarify a description of regular expression features with respect
to match groups.
Expand the description of field assignments to cover match group
interpolation, cross-referencing to Section "Matchers" for the full
description.
Signed-off-by: Jonathan Dowland <jon@dow.land>
Add functional tests for matching substrings in field matchers and
interpolating them into the corresponding field assignments. Check
the following properties and use-cases:
* Use-case 1: matching a portion of a date in a known format
(YYY-MM-DD) and writing a comment-command to warp a posting date.
Useful for credit cards.
* Use-case 2: match a portion of a CSV field and use it as an
account assignment. Useful for my byzantine setup with two
separate ledgers cross-importing to each other.
* Ensure bracketed portions of field matchers are captured.
* Ensure bracketed portions of record matchers are captured.
* Check match token numerical offset is relative to match group,
not the whole rules file.
* Check nested matches work.
* Ensure match group token expansion works with or without
surrounding text.
Signed-off-by: Jonathan Dowland <jon@dow.land>
Replace occurrences of '\N' (where N is a positive number) in field
templates with the corresponding regular expression match group, if it
exists.
E.g. Warp the date to the first of the month for the second posting
if %date (....-..)-..
comment2 date:\1-01
E.g. Strip a prefix from an imported account name
if %account1 liabilities:jon:(.*)
account1 \1
Fixes#2009.
Signed-off-by: Jonathan Dowland <jon@dow.land>
All commands that suport csv output now also support tsv output. The
data is identical, but the fields are separated by tab characters and
there is no quoting or escaping. Tab, carriage return, and newline
characters in data are converted to spaces (this should rarely if ever
happen in practice).
Changes:
1. rename the sandstorm "manage" permission to "edit"
(old permission names: view, add, manage;
new permission names: view, add, edit).
Rationale: "edit" best describes this permission's current powers, to users and to operators.
If we ever added more manager-type features we'd want that to be a new permission,
not a rename of the existing one (which would change the powers of existing users).
2. rename the sandstorm roles for consistency with permissions
(old role names: viewer, editor, manager;
new role names: viewer, adder, editor)
Rationale: it's needed to avoid confusion.
3. add a new option: --allow=view|add|edit|sandstorm (default: add).
'sandstorm' sets permissions according to the X-Sandstorm-Permissions header.
Drop the --capabilities and --capabilities-header options.
Rationale: it's simpler and more intuitive.
4. replace "capability" with "permission" in ui/docs/code.
Rationale: consistent with the above, more familiar.