Commit Graph

1804 Commits

Author SHA1 Message Date
Stephen Morgan
b7a2479186 lib: Use uniform naming for stripping prices.
Creates a new function amountStripPrices, and renames removePrices to
postingStripPrices.
2021-04-11 18:31:30 -10:00
Stephen Morgan
278153effa lib: Add more efficient toEncoding for custom ToJSON declarations. 2021-04-10 17:52:33 -10:00
Stephen Morgan
a529207ae7 lib: Do not round Decimal before applying JSON representation if
existing representation is small enough.

Previously the JSON representation of Decimal was rounded to 10 points
of precision before serialising. This sometimes results in an
unnecessary increase of precision.
2021-04-10 17:52:33 -10:00
Stephen Morgan
edbb713cf8 lib,cli: Fix ledgerDateSpan, so that in takes transaction dates into account. 2021-04-03 13:54:23 -10:00
Stephen Morgan
5fbb67b893 lib,cli: Move reportPeriodName to Hledger.Reports.ReportOptions, use it
for HTML and CSV output for compound balance reports.
2021-04-03 13:43:37 -10:00
Eric Mertens
48d558fc7a Tolerate spaces in amount fields in CSV files 2021-03-26 16:39:24 -07:00
Stephen Morgan
b6e20dea13 lib,test: Simplify the JSON representation of AmountPrecision.
It now uses the same JSON representation as Maybe Word8. This means that
the JSON serialisation is now broadly compatible with that used before the
commit f6fa76bba7, differing only in
how it handles numbers outside Word8 and that it can now produce null
for NaturalPrecision.
2021-03-25 15:47:34 -07:00
Stephen Morgan
4cb9dfb5b8 lib: Properly escape quotes in csv output. 2021-03-25 09:41:42 -07:00
Stephen Morgan
4609e79f2c lib,cli,ui,web: A number of AccountName and Journal functions which are
supposed to produce unique sorted use Sets internally to be slightly
more efficient. There is also a new function journalCommodities.
2021-03-23 11:26:30 -07:00
Stephen Morgan
7fe58f1346 lib: More efficiently check whether Amounts are or look zero.
Comparing two Quantity (either with (==) or compare) does a lot of
normalisation (calling roundMax) which is unnecessary if we're comparing
to zero. Do things more directly to save work.

For reg -f examples/10000x10000x10.journal, this results in
- A 12% reduction in heap allocations, from 70GB to 62GB
- A 14% reduction in (profiled) time, from 79s to 70s
Results for bal -f examples/10000x10000x10.journal are of the same order
of magnitude.
2021-03-23 11:26:30 -07:00
Stephen Morgan
522c8a6ad3 lib: In sorting account names, perform lookups on HashSets and HashMaps,
rather than lists. This is probably not an enormous performance sink in real
situations, but it takes a huge amount of time and memory in our
benchmarks (specifically 10000x10000x10.journal).

For bal -f examples/10000x10000x10.journal, this results in
- A 23% reduction in heap allocation, from 27GiB to 21GiB
- A 33% reduction in (profiled) time running, from 26.5s to 17.9s
2021-03-23 11:26:30 -07:00
Stephen Morgan
13589aca2e doc: Remove incorrect haddock comment. 2021-03-23 11:26:30 -07:00
Stephen Morgan
7488140608 lib: Do not call showAmount twice for every posting.
For print -f examples/10000x10000x10.journal, this results in
- A 7.7% reduction in heap allocations, from 7.6GB to 7.1GB.
2021-03-21 07:10:27 -07:00
Stephen Morgan
7aa3d3e760 lib,cli: Some efficiency improvements in register reports.
Strip prices after valuing postings in PostingsReport.
Use renderRow interface for Register report.

For reg -f examples/10000x10000x10.journal, this results in:
- Heap allocations decreasing by 55%, from 68.6GB to 31.2GB
- Resident memory decreasing by 75%, from 254GB to 65GB
- Total (profiled) time decreasing by 55%, from 37s to 20s
2021-03-21 07:10:27 -07:00
Stephen Morgan
d54e276658 lib: Split showMixedAmountB into showMixedAmountB and showAmountsB, the
former being a simple wrapper around the latter.

This removes the need for the showNormalised option, as showMixedAmountB
will always showNormalised and showAmountsB will never do so.

We also strip prices from MixedAmount before displaying if not displaying prices.
2021-03-21 07:10:27 -07:00
Stephen Morgan
d6a4310d8f lib,cli,ui,bin: Eliminate all uses of Mixed outside of Hledger.Data.Amount.
Exceptions are for dealing with the pamount field, which is really just
dealing with an unnormalised list of amounts.

This creates an API for dealing with MixedAmount, so we never have to
access the internals outside of Hledger.Data.Amount.

Also remove a comment, since it looks like #1207 has been resolved.
2021-03-18 09:47:59 +11:00
Stephen Morgan
dabb3ef82e lib,cli,ui,bin: Create a new API for MixedAmount arithmetic. This should
supplant the old interface, which relied on the Num typeclass.

MixedAmount did not have a very good Num instance. The only functions
which were defined were fromInteger, (+), and negate. Furthermore, it
was not law-abiding, as 0 + a /= a in general. Replacements for used
functions are:
0 -> nullmixedamt / mempty
(+) -> maPlus / (<>)
(-) -> maMinus
negate -> maNegate
sum -> maSum
sumStrict -> maSum

Also creates some new constructors for MixedAmount:
mixedAmount :: Amount -> MixedAmount
maAddAmount :: MixedAmount -> Amount -> MixedAmount
maAddAmounts :: MixedAmount -> [Amount] -> MixedAmount

Add Semigroup and Monoid instances for MixedAmount.
Ideally we would remove the Num instance entirely.

The only change needed have nullmixedamt/mempty substitute for
0 without problems was to not squash prices in
mixedAmount(Looks|Is)Zero. This is correct behaviour in any case.
2021-03-18 09:47:21 +11:00
Stephen Morgan
fb02d86884 lib: Do not do any valuation with --value=cost, perform AtEnd valuation with --value=cost,COMM. 2021-03-10 12:42:01 +11:00
Simon Michael
51d582e845 also keep supporting --value=cost,COMM for now (#1499) 2021-03-09 07:37:52 -08:00
Stephen Morgan
efb37c3343 lib: Default to PeriodChange report when using ValueChangeReport. 2021-03-05 10:09:14 -08:00
Stephen Morgan
3f408c2117 lib,cli: Restore old --change option for per-period summation, use --sum for basic ValueChange balance report. 2021-03-05 10:09:14 -08:00
Simon Michael
2505c69f80 lib: transactionFile 2021-03-03 06:44:25 -08:00
Stephen Morgan
edaaef897b lib: Do not include price directives in journalDateSpan. Only include
price directives after the last transaction/posting date if using
--value=end.

Also enlarges the reportspan to encompass full intervals for budget
goals.
2021-03-03 05:26:55 -08:00
Stephen Morgan
7f2536a2a7 lib,cli: Add --valuechange report type for calculating change of value
of accounts, restore --value=end behaviour to that of hledger-1.19.
2021-02-27 18:42:02 -08:00
Stephen Morgan
351648e4fa lib,cli: Add --periodic option to indicate PeriodChange accumulation (renamed from --change). 2021-02-27 18:42:02 -08:00
Stephen Morgan
f0655d1c7f lib: (amount|mixedAmount)(Looks|Is)Zero functions now check whether
both the quantity and the cost are zero. This is usually what you want,
but if you do only want to check whether the quantity is zero, you
can run mixedAmountStripPrices (or similar) before this.

(multiply|divide)(Mixed)?Amount now also multiply or divide the
TotalPrice if it is present, and the old
(multiply|divide)(Mixed)?AmountAndPrice functions are removed.
2021-02-23 14:54:39 -08:00
Stephen Morgan
9d527a9926 lib: normaliseHelper now uses a strict Map for combining amounts
internally, closing a big space leak.

This also now combines Amounts with TotalPrices in the same commodity
when normalising; amounts with TotalPrices were previously never
combined.
2021-02-23 14:54:39 -08:00
Stephen Morgan
ecca7f4e0c lib: Distinguish between an Amount having quantity (or rounded quantity 0), and having both quantity and totalprice 0 (or rounded to 0). 2021-02-23 14:54:39 -08:00
Stephen Morgan
81b778a389 lib: Make fields of Amount, AmountPrice, AmountStyle, and DigitGroupStyle strict. 2021-02-23 14:54:39 -08:00
Stephen Morgan
2ada289e28 lib: Include sign in TotalPrice in Amount, rather than relying on the sign of
aquantity.

Journal entries still require a positive @@ price, but now the sign is
set after parsing, rather than when converting in amountToCost.

The reason for this change is that, if we're going to perform arithmetic
on Amount with TotalCost, then the presence of aquantity=0 means that
amountToCost would render the total cost as 0, because signum 0 == 0.
This makes journal entries like the following impossible to balance:
2000-01-01
    a  0 @@ 10 A
    b  -10 A
2021-02-23 14:54:39 -08:00
Stephen Morgan
a65ef7cd19 lib: Consume list immediately in commodityStylesFromAmounts.
This reduced the maximum heap size per thread from ~850K to ~430K in a
real-world register test.
2021-02-23 14:54:39 -08:00
Simon Michael
5e7c4fc7bc add, print --match: prioritise infix matches
If the search description occurs in full within the other description,
that match gets a +0.5 score boost.
2021-02-20 14:08:47 -08:00
Simon Michael
44508f58c8 add, lib: debug output, refactor similar transactions lookup
add --debug=1 shows the top hits for similar past transactions.

added:
Hledger.Cli.Utils.journalSimilarTransaction
 provides --debug=1 output

changed:
Hledger.Cli.Commands.Add.transactionsSimilarTo -> Hledger.Data.Journal.journalTransactionsSimilarTo
 now takes an extra number-of-results argument
2021-02-20 13:41:37 -08:00
Simon Michael
b31e40ceca lib: avoid shady Not (Or []) in account type queries 2021-02-13 14:58:06 -08:00
Simon Michael
6c392e9f43 bs: don't let just a Cash declaration hide Asset accounts
Clarify the account type queries a bit, and don't let a
declaration of account as Cash cause the fallback regex
for Asset accounts to be ignored.
2021-02-13 14:54:04 -08:00
Simon Michael
b9d33a4294 bs: just declaring a Cash account no longer hides Asset accounts
Since Cash is also an Asset, declaring an account as Cash also disabled the fallback regexp for Asset accounts.
2021-02-13 08:09:45 -08:00
Simon Michael
50bfc26ca7 ;lib: commentAddTagNextLine was broken
It would not add the tag when a comment already existed.
This affected hledger-print-location.hs and probably
the generated-transaction: tag in periodic transactions.
2021-02-12 13:56:02 -08:00
Simon Michael
d865ec5d65 lib: refactor: more consistent amount precision helpers
Hledger.Data.Amount:
renamed:
setAmountPrecision -> amountSetPrecision
setFullPrecision -> amountSetFullPrecision
setMixedAmountPrecision -> mixedAmountSetPrecision
added:
mixedAmountSetFullPrecision
2021-02-05 16:09:49 -08:00
Simon Michael
d5ed8090fa ;lib: document AmountPrecision 2021-02-05 16:09:49 -08:00
Simon Michael
37e49ada70 cli: rename --infer-value to --infer-market-price
For clarity; infer-value was too vague. The old spelling remains
supported for compatibility, but is now deprecated.
When typing, --infer-market or even --infer (for now) is sufficient.
2021-02-05 15:16:39 -08:00
Simon Michael
2a4170dd3b ui: transaction: show all decimal places (like print, cf #931)
On the accounts screen and register screen we round amounts according
to commodity styles, but when you drill down to a transaction you
probably want to see the unrounded amounts.
2021-02-04 08:13:55 -08:00
Simon Michael
f52117120b lib: refactor: rename journal/transaction mapping helpers
Hledger.Data.Journal:
changed:
mapJournalTransactions -> journalMapTransactions
mapJournalPostings -> journalMapPostings
removed:
mapTransactionPostings

Hledger.Data.Transaction:
added:
transactionMapPostings
2021-02-04 08:13:55 -08:00
Simon Michael
6101368954 print: always show all decimal places (#931)
Ensures parseable and more sensible-looking output in more cases, and behaves more like Ledger's print.

There is still an issue with adding trailing zeroes, which would be nice to prevent.
2021-02-04 08:13:55 -08:00
Simon Michael
0274b30742 print: don't add decimal places in assertion/assignment amounts either (#1465) 2021-02-04 06:46:09 -08:00
Simon Michael
201f967411 print: show all decimal places in assertion/assignment amounts (#1465)
Overriding the commodity style, per https://hledger.org/hledger.html#commodity-display-style.
2021-02-03 18:11:44 -08:00
Stephen Morgan
b619b68f71 doc: Remove references to --value=cost, replacing them with --cost.
Since this is option is now just an alias for -B/--cost, and since it
may be removed soon, we make it undocumented, though it will still
behave as before. --value=cost,COMM is unsupported as well.
2021-01-28 13:57:48 -08:00
Stephen Morgan
c9eb7d1bcf lib,cli,ui: Separate costing from valuation; each can now be specified
independently.

You can now combine costing and valuation, for example "--cost
--value=then" will first convert to costs, and then value according to
the "--value=then" strategy. Any valuation strategy can be used with or
without costing.

If multiple valuation and costing strategies are specified on the
command line, then if any of them include costing
(-B/--cost/--value=cost) then amounts will be converted to cost, and for
valuation strategy the rightmost will be used.

--value=cost is deprecated, but still supported and is equivalent to
--cost/-B. --value=cost,COMM is no longer supported, but this behaviour can be
achieved with "--cost --value=then,COMM".
2021-01-28 13:57:48 -08:00
Stephen Morgan
130739e3ef lib,cli,ui: Introduce *ApplyCostValuation functions, which perform both
costing and valuation.

This currently is given a dummy NoCost argument and is equivalent to
"maybe id (*ApplyValuation ...)", but provides a constant interface so
that internal behaviour can be changed freely.
2021-01-28 13:57:48 -08:00
Stephen Morgan
9881ec9652 lib: Use mapMixedAmount more, decreasing reliance on internal representation of Mixed. 2021-01-28 13:57:48 -08:00
Stephen Morgan
9cf5015958 lib: bal --value=end without --historical should not report on fluctuation in value of historical postings. 2021-01-28 13:57:48 -08:00