That 1.31 change was advertised as being for the print command only,
but it affected all commands. Now it affects only print and other
"print-like" commands (ie all commands that show whole journal entries
that we might want to re-parse).
Also three classes of hledger output, and how they modify the
commodity display styles' digit group marks and decimal marks
to suit different consumers, have been identified and documented
(under REPORTING CONCEPTS).
hledger import -s now runs strict checks on an in-memory copy of the
updated journal, before updating the journal file; if strict checks
fail, nothing is written to disk.
And hledger import now does not update any .latest files until it has
run without error (no failing strict checks, no failure while writing
the journal file). This makes it more idempotent, so you can run it
again after fixing problems.
Strict checks now run only once, at end of the high level read operation,
and not for each individual file; this fixes some spurious --strict failures,
like account declarations not affecting a sibling file as they should.
And .latest file writing now happens as the last step, after passing
strict checks. This is mainly for the import command, but it also
means that hledger print --new now does not update .latest files
if strict checks are failing.
The file reading API has been improved and documented in more detail.
With valuation now preserving more decimal digits, roi could show
excessively precise decimals if there was no known display precision
for the valuation commodity. Now in that situation it limits the
precision to a maximum of 8 digits.
This and the preceding commits were "work in progress" that got out of control.
There's more to do, but this one brings these precision-related improvements
(at least):
When "infinite decimals" arise, they are now generally shown with
8 decimal digits rather than 255.
print and prices no longer add trailing decimal zeros unnecessarily.
Some code has been refactored or given more debug output.
All tests have been updated to match the recent changes.
- The prices comand now more accurately lists the prices that hledger
uses when calculating value reports (similar to what you'd see with
eg `hledger bal -V --debug=2`).
- The prices command's --infer-reverse-prices flag was confusing since
we always infer and use reverse prices; it has been renamed to --show-reverse.
- --infer-market-prices and --show-reverse combine properly.
- --show-reverse now ignores all zero prices rather than giving an error.
- Reverse prices (which can be infinite decimals) are now displayed
with at most 8 decimal digits (rather than the internal precision of
255 digits).
- Filtering prices by cur: or amt: now works properly.
- Price amounts are styled, but all decimal digits are shown.
Cost/value conversion now applies the standard display style, and
sets the display precision equal to the internal decimal precision
(or 8 if the decimal appears to be infinite).
This means value reports and especially `print -V` now show amounts
with more accurate and standard style and precision.
New tests have been added describing and explaining various
style/precision behaviours in print cost/value reports.
Add a description of Match Groups to the manual; Section "Matchers".
Include two examples.
Clarify a description of regular expression features with respect
to match groups.
Expand the description of field assignments to cover match group
interpolation, cross-referencing to Section "Matchers" for the full
description.
Signed-off-by: Jonathan Dowland <jon@dow.land>
Add functional tests for matching substrings in field matchers and
interpolating them into the corresponding field assignments. Check
the following properties and use-cases:
* Use-case 1: matching a portion of a date in a known format
(YYY-MM-DD) and writing a comment-command to warp a posting date.
Useful for credit cards.
* Use-case 2: match a portion of a CSV field and use it as an
account assignment. Useful for my byzantine setup with two
separate ledgers cross-importing to each other.
* Ensure bracketed portions of field matchers are captured.
* Ensure bracketed portions of record matchers are captured.
* Check match token numerical offset is relative to match group,
not the whole rules file.
* Check nested matches work.
* Ensure match group token expansion works with or without
surrounding text.
Signed-off-by: Jonathan Dowland <jon@dow.land>
Replace occurrences of '\N' (where N is a positive number) in field
templates with the corresponding regular expression match group, if it
exists.
E.g. Warp the date to the first of the month for the second posting
if %date (....-..)-..
comment2 date:\1-01
E.g. Strip a prefix from an imported account name
if %account1 liabilities:jon:(.*)
account1 \1
Fixes#2009.
Signed-off-by: Jonathan Dowland <jon@dow.land>
All commands that suport csv output now also support tsv output. The
data is identical, but the fields are separated by tab characters and
there is no quoting or escaping. Tab, carriage return, and newline
characters in data are converted to spaces (this should rarely if ever
happen in practice).
