Commit Graph

891 Commits

Author SHA1 Message Date
Stephen Morgan
06312c353a fix!: register: Tighten up spacing around the date in register reports. (#1655)
As a side effect, this changes the Json representation of the
PostingsReport. The maybe report end date is now replaced with a maybe
period.
2021-08-22 08:15:22 -10:00
Stephen Morgan
34ab164b32 cln: Remove redundant bang patterns. 2021-08-21 17:50:21 -10:00
Stephen Morgan
912b5e6c23 cln: Derive more instances in Hledger.Data.Types to reduce orphans.
Also clean up some outdated or inaccurate comments.
2021-08-21 17:50:21 -10:00
Lawrence
8a5addfb02 lib: implement periodic expressions for weekday et al
Implementation lifts DayOfWeek (renamed to DaysOfWeek) to a list of
days. This should match the previous behavior for single-element lists
2021-08-20 20:30:18 -05:00
Stephen Morgan
4576f8effd pkg!: Migrate the last of old-time over to time, changing ClockTime to
POSIXTime.

This eliminates old-time, which has been deprecated for a while, from
our dependencies.

This introduces a slight incompatibility, as a small number of functions
now take/return POSIXTime instead of ClockTime. Generally you will be
using the current time, in which case you should use getPOSIXTime from
Data.Time.Clock.POSIX instead of getClockTime.

utcTimeToClockTime has been removed, as it is now equivalent to
utcTimeToPOSIXSeconds from Data.Time.Clock.POSIX.
2021-08-19 17:47:19 -10:00
Simon Michael
c135c1a3fc ;cln: slight typos cleanup (#1654) 2021-08-19 17:43:45 -10:00
Lawrence
44e1ea10fa fix: bal: correctly handle empty commodity in commodity-column (#1654)
We can't filter out empty commodity strings since that is a legitimate
group. Simultaneously, we should only include the empty commodity if it
is explicitly used (part of a posting) and not generated as part of
`Amounts.amounts`
2021-08-19 17:37:19 -10:00
Stephen Morgan
ddba9f6ce4 imp: lib,cli: Implement gain report for balance reports.
A gain report will report on unrealised gains by looking at the
difference between the valuation of an amount (by default, --value=end),
and the valuation of the cost of the amount.
2021-08-18 21:44:30 -10:00
Arjen Langebaerd
3426030a91 feat: added commodity style commandline option 2021-08-17 22:05:29 -10:00
Lawrence Wu
f3c07144a8 bal: option to view one commodity per row
This adds the `--commodity-column` option that displays each commodity
on a separate line and the commodities themselves as a separate column.
The initial design considerations are at
simonmichael.hledger.issues.1559

The single-period balance report with `--commodity-column` does not
interoperate with custom formats.
2021-08-08 09:19:25 -10:00
Stephen Morgan
fc8aa602cf lib!: modifyTransactions now takes a Map of commodity styles, and will
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.

This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.

To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
2021-07-30 13:52:35 -10:00
Stephen Morgan
9df574b3c0 fix: reg: register --related should not create duplicate postings when
more than one posting in a transaction matches. (#1629)
2021-07-30 09:06:27 -10:00
Stephen Morgan
7ed2a0aa9b lib!: lib: Remove aismultiplier from Amount.
In Amount, aismultiplier is a boolean flag that will always be False,
except for in TMPostingRules, where it indicates whether the posting
rule is a multiplier. It is therefore unnecessary in the vast majority
of cases. This posting pulls this flag out of Amount and puts it into
TMPostingRule, so it is only kept around when necessary.

This changes the parsing of journals somewhat. Previously you could
include an * before an amount anywhere in a Journal, and it would
happily parse and set the aismultiplier flag true. This will now fail
with a parse error: * is now only acceptable before an amount within an
auto posting rule.

Any usage of the library in which the aismultiplier field is read or set
should be removed. If you truly need its functionality, you should
switch to using TMPostingRule.

This changes the JSON output of Amount, as it will no longer include
aismultiplier.
2021-07-22 19:06:33 -10:00
Stephen Morgan
58516eea77 imp: json: Decrease indentation in JSON output. 2021-07-20 23:28:18 -10:00
Stephen Morgan
c606f874fb imp: json: The keys of JSON objects are now displayed in alphabetical order, making it stable across different systems and compilers. 2021-07-20 20:37:33 -10:00
Stephen Morgan
cf25d7d56d
imp: bal: more predictable sort order with multiple commodities (#1563, #1564)
This change provides more predictable and intuitive behaviour when 
using -S/--sort-amount with multiple commodities. 
It implements a custom Ord (and Eq) instance for MixedAmount 
which substitutes zero for any missing commodities.

As a consequence, all the ways of representing zero with a MixedAmount ([],
[A 0], [A 0, B 0, ...]) are now Eq-ual (==), whereas before they were
not. We have not been able to find anything broken by this change.

* imp: lib: Compare MixedAmounts by substituting zero for any missing commodities. (#1563)
* ;doc: Update docs for new multicommodity sort by amount rules.
2021-07-12 21:11:50 -10:00
Simon Michael
5d2caf3c7f lib: clarify journalEndDate, add journalLastDay 2021-07-12 18:50:06 -10:00
Simon Michael
3cd80e1f57 lib: export transactionCheckBalanced (#1596) 2021-07-07 13:20:13 -10:00
Stephen Morgan
e2198ff1ee lib: Auto-postings with Amount queries should filter only those commodities which match the query.
Also corrects a regression introduced in
8ab29f84b3 where transaction modifier
postings without multipliers would incorrectly be filtered by commodity.
2021-07-01 09:35:56 -10:00
Stephen Morgan
2a9d358627 lib: Make sure automatic postings generated from postings with more than one commodity match on commodity symbol (#1582). 2021-06-30 17:35:43 -10:00
Simon Michael
665fec83cd
Merge pull request #1560 from Xitian9/rationalisevaluation
Clean up valuation functions, and make clear which to use where.
2021-06-07 19:02:44 -10:00
Stephen Morgan
0f1837816d lib,cli,ui,web: Add check balancednoautoconversion command, which checks that
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.

Renames check autobalanced to check balancedwithautoconversion.
2021-06-07 18:58:58 -10:00
Stephen Morgan
68e975adf1 lib,cli,ui,web: Remove unused LANGUAGE pragmas. 2021-06-07 17:33:54 -10:00
Stephen Morgan
55308e1ca8 lib,cli,ui,web: Remove unnecessary CPP when dropping support for GHC 8.2. 2021-06-07 17:33:54 -10:00
Stephen Morgan
c827cbcc3e lib: Refactor to eliminate use of printf. 2021-06-03 23:23:54 -10:00
Stephen Morgan
bf22c3efdd lib,cli: Replace concat(Top|Bottom)Padded with textConcat(Top|Bottom)Padded. 2021-06-03 23:23:54 -10:00
Stephen Morgan
0e59fee251 lib,cli: Export Text.Tabular from Text.Tabular.AsciiWide, clean up import lists. 2021-06-03 23:23:54 -10:00
Stephen Morgan
dc16451de0 lib: Remove unused (amount|mixedAmount|posting|transaction)ApplyCostValuation functions. 2021-05-07 22:25:19 +10:00
Stephen Morgan
d1467d4766 lib: When inferring total prices, use the precision of toamount, rather than NaturalPrecision. 2021-05-02 21:14:36 +10:00
Stephen Morgan
686a0871a9 lib: Make sure fromamount and toamount have opposite signs when inferring prices.
Also change priceInferrerFor so that it would give the correct
(negative) prices when fromamount and toamount have the same sign.
2021-05-02 21:14:36 +10:00
Stephen Morgan
0078f1a520 lib: Infer prices correctly even when there are only balance assignments. 2021-05-01 09:45:29 -10:00
Stephen Morgan
7cb621b82f lib,cli,ui,bin: Remove unnecessary normalisedMixedAmount, replace
normaliseMixedAmountSquashPricesForDisplay with mixedAmountStripPrices.
2021-05-01 09:45:29 -10:00
Stephen Morgan
4780a02e5a lib,cli: Remove showAmounts*B functions, replacing them entirely with showMixedAmount*B functions. 2021-05-01 09:45:29 -10:00
Stephen Morgan
5e7b69356f lib: Change internal representation of MixedAmount to use a strict Map
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.

This commit improves performance. Here are some indicative results.

hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)

hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)

hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)

hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)

hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)

hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)

If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:

- If you really care about the individual Amounts and never normalise
  your MixedAmount (for example, just storing `Mixed amts` and then
  extracting `amts` as a pattern match, then use should switch to using
  [Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
  amount arithmetic (+), (-), then you should replace the constructor
  `Mixed` with the function `mixed`. To extract the list of Amounts, use
  the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
  replace that with `mixedAmountStripPrices`. (N.B. this does something
  slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
  but I don't think there's any use case for squashing prices and then
  keeping the first of the squashed prices around. If you disagree let
  me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
  is now the identity function.
2021-05-01 09:45:29 -10:00
Simon Michael
0f4e462ae7 add a Show instance for AmountDisplayOpts, WideBuilder
For debug logging.
2021-04-13 17:19:22 -10:00
Stephen Morgan
28fd7c65f2 lib: When displaying amounts on a single line, always display at least one amount, even if that would exceed the requested maximum width. Addresses #1526. 2021-04-13 15:29:58 -10:00
Simon Michael
f5121129ec timeclock: print shows timeclock amounts with just 2 decimals, like pre-1.21 (#1527) 2021-04-12 18:40:33 -10:00
Stephen Morgan
b7a2479186 lib: Use uniform naming for stripping prices.
Creates a new function amountStripPrices, and renames removePrices to
postingStripPrices.
2021-04-11 18:31:30 -10:00
Stephen Morgan
278153effa lib: Add more efficient toEncoding for custom ToJSON declarations. 2021-04-10 17:52:33 -10:00
Stephen Morgan
a529207ae7 lib: Do not round Decimal before applying JSON representation if
existing representation is small enough.

Previously the JSON representation of Decimal was rounded to 10 points
of precision before serialising. This sometimes results in an
unnecessary increase of precision.
2021-04-10 17:52:33 -10:00
Stephen Morgan
edbb713cf8 lib,cli: Fix ledgerDateSpan, so that in takes transaction dates into account. 2021-04-03 13:54:23 -10:00
Stephen Morgan
b6e20dea13 lib,test: Simplify the JSON representation of AmountPrecision.
It now uses the same JSON representation as Maybe Word8. This means that
the JSON serialisation is now broadly compatible with that used before the
commit f6fa76bba7, differing only in
how it handles numbers outside Word8 and that it can now produce null
for NaturalPrecision.
2021-03-25 15:47:34 -07:00
Stephen Morgan
4609e79f2c lib,cli,ui,web: A number of AccountName and Journal functions which are
supposed to produce unique sorted use Sets internally to be slightly
more efficient. There is also a new function journalCommodities.
2021-03-23 11:26:30 -07:00
Stephen Morgan
7fe58f1346 lib: More efficiently check whether Amounts are or look zero.
Comparing two Quantity (either with (==) or compare) does a lot of
normalisation (calling roundMax) which is unnecessary if we're comparing
to zero. Do things more directly to save work.

For reg -f examples/10000x10000x10.journal, this results in
- A 12% reduction in heap allocations, from 70GB to 62GB
- A 14% reduction in (profiled) time, from 79s to 70s
Results for bal -f examples/10000x10000x10.journal are of the same order
of magnitude.
2021-03-23 11:26:30 -07:00
Stephen Morgan
522c8a6ad3 lib: In sorting account names, perform lookups on HashSets and HashMaps,
rather than lists. This is probably not an enormous performance sink in real
situations, but it takes a huge amount of time and memory in our
benchmarks (specifically 10000x10000x10.journal).

For bal -f examples/10000x10000x10.journal, this results in
- A 23% reduction in heap allocation, from 27GiB to 21GiB
- A 33% reduction in (profiled) time running, from 26.5s to 17.9s
2021-03-23 11:26:30 -07:00
Stephen Morgan
7488140608 lib: Do not call showAmount twice for every posting.
For print -f examples/10000x10000x10.journal, this results in
- A 7.7% reduction in heap allocations, from 7.6GB to 7.1GB.
2021-03-21 07:10:27 -07:00
Stephen Morgan
d54e276658 lib: Split showMixedAmountB into showMixedAmountB and showAmountsB, the
former being a simple wrapper around the latter.

This removes the need for the showNormalised option, as showMixedAmountB
will always showNormalised and showAmountsB will never do so.

We also strip prices from MixedAmount before displaying if not displaying prices.
2021-03-21 07:10:27 -07:00
Stephen Morgan
d6a4310d8f lib,cli,ui,bin: Eliminate all uses of Mixed outside of Hledger.Data.Amount.
Exceptions are for dealing with the pamount field, which is really just
dealing with an unnormalised list of amounts.

This creates an API for dealing with MixedAmount, so we never have to
access the internals outside of Hledger.Data.Amount.

Also remove a comment, since it looks like #1207 has been resolved.
2021-03-18 09:47:59 +11:00
Stephen Morgan
dabb3ef82e lib,cli,ui,bin: Create a new API for MixedAmount arithmetic. This should
supplant the old interface, which relied on the Num typeclass.

MixedAmount did not have a very good Num instance. The only functions
which were defined were fromInteger, (+), and negate. Furthermore, it
was not law-abiding, as 0 + a /= a in general. Replacements for used
functions are:
0 -> nullmixedamt / mempty
(+) -> maPlus / (<>)
(-) -> maMinus
negate -> maNegate
sum -> maSum
sumStrict -> maSum

Also creates some new constructors for MixedAmount:
mixedAmount :: Amount -> MixedAmount
maAddAmount :: MixedAmount -> Amount -> MixedAmount
maAddAmounts :: MixedAmount -> [Amount] -> MixedAmount

Add Semigroup and Monoid instances for MixedAmount.
Ideally we would remove the Num instance entirely.

The only change needed have nullmixedamt/mempty substitute for
0 without problems was to not squash prices in
mixedAmount(Looks|Is)Zero. This is correct behaviour in any case.
2021-03-18 09:47:21 +11:00
Simon Michael
2505c69f80 lib: transactionFile 2021-03-03 06:44:25 -08:00