Commit Graph

692 Commits

Author SHA1 Message Date
Stephen Morgan
57d055b643 fix: cost: Add inferred equity postings during journal finalisation, so
they can be matched by queries.
2022-01-16 18:37:18 -10:00
Stephen Morgan
5006e26995 imp: check: Do not check commodities for zero amounts with empty
commodity. (#1767)
2022-01-16 08:11:02 -10:00
Stephen Morgan
8eedbbbe87 imp: cost: Generate totally balanced conversion postings for amounts with costs.
Introduce --infer-equity option which will generate conversion postings.
--cost will override --infer-equity.

This means there will no longer be unbalanced transactions, but will be
offsetting conversion postings to balance things out. For example.

2000-01-01
  a   1 AAA @@ 2 BBB
  b  -2 BBB

When converting to cost, this is treated the same as before.
When used with --infer-equity, this is now treated as:

2000-01-01
  a                               1 AAA
  equity:conversion:AAA-BBB:AAA  -1 AAA
  equity:conversion:AAA-BBB:BBB   2 BBB
  b                              -2 BBB

There is a new account type, Conversion/V, which is a subtype of Equity/E.
The first account declared with this type, if any, is used as the base account
for inferred equity postings in conversion transactions, overriding the default
"equity:conversion".

API changes:

Costing has been changed to ConversionOp with three options:
NoConversionOp, ToCost, and InferEquity.
The first correspond to the previous NoCost and Cost options, while the
third corresponds to the --infer-equity flag.  This converts transactions with costs
(one or more transaction prices) to transactions with equity:conversion postings.
It is in ConversionOp because converting to cost with -B/--cost and inferring conversion
equity postings with --infer-equity are mutually exclusive.

Correspondingly, the cost_ record of ReportOpts has been changed to
conversionop_.

This also removes show_costs_ option in ReportOpts, as its functionality
has been replaced by the richer cost_ option.
2021-12-21 10:50:13 -10:00
Simon Michael
0d83bdf6d7 cln: csv: small rename 2021-12-08 16:57:53 -10:00
Simon Michael
1fee70b51f cln: hlint cleanups 2021-11-21 19:33:21 -10:00
Lawrence
fb817ffa55 imp: journal: support decimal-mark directive in journal files
To match csv-rule and override amount-style as described in
https://github.com/simonmichael/hledger/issues/1670
2021-11-18 22:17:07 -10:00
Stephen Morgan
e35d0b7865 fix: csv: Successfully parse empty csv file. (#1183) 2021-11-18 20:50:02 -10:00
Stephen Morgan
87a7a586d4 fix: csv: Handle multiple zero amounts in postings in csv files. (#1733) 2021-11-18 20:48:55 -10:00
Simon Michael
41bde20095 timedot: a D default commodity (and style) is applied to timedot aounts
This means they can be priced and converted.
2021-11-05 23:34:46 -10:00
Stephen Morgan
1bc04685b7 pkg: Drop base-compat-batteries dependency.
Our supported stackage versions are now new enough that we don't need
any of the compatibility features anymore.
2021-10-31 07:56:07 -10:00
Stephen Morgan
4cfd3cb590 lib!: Remove GenericSourcePos, and replace it with either SourcePos or
(SourcePos, SourcePos).

This has been marked for possible removal for a while. We are keeping
strictly more information. Possible edge cases arise with Timeclock and
CsvReader, but I think these are covered.

The particular motivation for getting rid of this is that
GenericSourcePos is creating some awkward import considerations for
little gain. Removing this enables some flattening of the module
dependency tree.
2021-09-20 08:38:33 -10:00
Stephen Morgan
df71d2ddd5 dev: Move journal and text parsing utilities into more appropriate places, to reduce reliance on Hledger.Read.Common. 2021-09-19 17:10:38 -10:00
Stephen Morgan
315e92c08b pkg!: Break functions related to balancing out into a separate module,
Hledger.Data.Balancing.

Both Hledger.Data.Transaction and Hledger.Data.Journal are massive
module with many things in them. Placing the balancing functions, which
are conceptually related, into a separate module helps keep things more
modular.

It also reduces the risk of import cycles, as right now balancing
functions cannot depend on any functions defined outside of
Hledger.Data.Transaction or Hledger.Data.Journal, respectively, if those
modules require basic transaction or journal functions.
2021-09-19 17:10:38 -10:00
Malte Brandy
e31eb58ada lib: Allow multiline comments in csv rules 2021-09-18 12:43:49 -10:00
Simon Michael
326d6e8dae ref: ReportOptions{infer_value_} renamed to infer_prices_
For more consistency with the flag name (--infer-market-prices).
And BalancingOpts{infer_prices_} is now infer_transaction_prices_.
2021-09-18 12:12:31 -10:00
Stephen Morgan
020dd15be4 ref: Simplify parsing of commodity-style option. 2021-09-18 11:41:53 -10:00
Stephen Morgan
0f205295e8 pkg!: test: Use --today in Journal parsing functions. 2021-09-09 14:40:39 -10:00
Stephen Morgan
6905e40c4d pkg!: cli: Add --today option to explicitly set the current date. (#1674)
rawOptsTo* in hledger-lib now takes a day as an argument, and does not
live in the IO monad, since it's now pure.

This is so that we can run tests containing future transactions that
won't fail as soon as ‘the future’ actually arrives.
2021-09-09 14:40:39 -10:00
Simon Michael
5485990cac fix: csv: report correct CSV line number in errors
Some errors in CSV conversion, such as a failing balance assertion,
were always being reported as line 2.
Reported by Lawrence Wu.
2021-09-01 06:58:15 -10:00
Stephen Morgan
8274da81fc cln: tests: Remove test and tests, which are just aliases for testCase
and testGroup.

Replacing these removes a layer of indirection, and reduces the need to
depend on Hledger.Utils.Test.
2021-08-30 16:32:19 -10:00
Stephen Morgan
435ec992f9 dev: lens: Introduce lenses for InputOpts and BalancingOpts. 2021-08-28 19:12:04 -10:00
Stephen Morgan
13206d0b18 cln: Rename balancingOpts to defbalancingopts, to be consistent with definputopts, defreportopts, etc. 2021-08-28 19:12:04 -10:00
Stephen Morgan
d248aec313 cln: hlint: Remove eta reduce warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
32dad455fd cln: hlint: Clean up section related warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
761e00caa4 cln: hlint: Clean up lambda related warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
8bf7c95697 cln: hlint: Clean up Functor related hlint warnings, and NOINLINE warning. 2021-08-27 06:13:56 -10:00
Stephen Morgan
330c21659f cln: hlint: Clean up Maybe related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
435fbf001e cln: hlint: Remove <|> and bimap warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
69502c44fe cln: hlint: Fix record patterns warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
1c211f8ab8 cln: hlint: Fix redundant return warning. 2021-08-26 21:00:35 -10:00
Stephen Morgan
c07ad29a87 imp!: forecast: Implements more intuitive logic for the forecast interval. (#1648)
The forecast period begins on:
- the start date supplied to the `--forecast` argument, if present
- otherwise, the later of
  - the report start date if specified with -b/-p/date:
  - the day after the latest normal (non-periodic) transaction in the journal, if any
- otherwise today.
It ends on:
- the end date supplied to the `--forecast` argument, if present
- otherwise the report end date if specified with -e/-p/date:
- otherwise 180 days (6 months) from today.

Note that the previous behaviour did not quite match the documentation,
so this also acts as a bug fix for #1665.
2021-08-26 20:32:30 -10:00
Simon Michael
e24b7ef90f fix: warning introduced by 993d0b5c 2021-08-22 06:58:08 -10:00
Simon Michael
993d0b5c54 cli: --forecast now rejects instead of silently ignoring an interval
Something that confused me when testing #1632.
2021-08-20 17:55:59 -10:00
Stephen Morgan
4576f8effd pkg!: Migrate the last of old-time over to time, changing ClockTime to
POSIXTime.

This eliminates old-time, which has been deprecated for a while, from
our dependencies.

This introduces a slight incompatibility, as a small number of functions
now take/return POSIXTime instead of ClockTime. Generally you will be
using the current time, in which case you should use getPOSIXTime from
Data.Time.Clock.POSIX instead of getClockTime.

utcTimeToClockTime has been removed, as it is now equivalent to
utcTimeToPOSIXSeconds from Data.Time.Clock.POSIX.
2021-08-19 17:47:19 -10:00
Arjen Langebaerd
3426030a91 feat: added commodity style commandline option 2021-08-17 22:05:29 -10:00
Simon Michael
5c18fb289f ;dev: configure hlint, silence all current warnings 2021-08-11 14:51:46 -10:00
Stephen Morgan
918c243fa9 fix: forecast: Generate forecast transactions at journal finalisation,
rather than as a postprocessing step. (#1638)

This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
2021-08-05 21:10:47 -10:00
Stephen Morgan
c404800fbf ref!: forecast: Move forecast_ from ReportOpts to InputOpts. 2021-08-05 21:10:47 -10:00
Stephen Morgan
fc8aa602cf lib!: modifyTransactions now takes a Map of commodity styles, and will
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.

This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.

To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
2021-07-30 13:52:35 -10:00
Stephen Morgan
04a36d4942 ref: Refactor journalFinalise to clarify flow.
The only semantic difference is that we now apply
journalApplyCommodityStyles before running journalCheckAccountsDeclared
and journalCheckCommoditiesDeclared.
2021-07-30 13:52:35 -10:00
Stephen Morgan
7ed2a0aa9b lib!: lib: Remove aismultiplier from Amount.
In Amount, aismultiplier is a boolean flag that will always be False,
except for in TMPostingRules, where it indicates whether the posting
rule is a multiplier. It is therefore unnecessary in the vast majority
of cases. This posting pulls this flag out of Amount and puts it into
TMPostingRule, so it is only kept around when necessary.

This changes the parsing of journals somewhat. Previously you could
include an * before an amount anywhere in a Journal, and it would
happily parse and set the aismultiplier flag true. This will now fail
with a parse error: * is now only acceptable before an amount within an
auto posting rule.

Any usage of the library in which the aismultiplier field is read or set
should be removed. If you truly need its functionality, you should
switch to using TMPostingRule.

This changes the JSON output of Amount, as it will no longer include
aismultiplier.
2021-07-22 19:06:33 -10:00
Simon Michael
59b1e46b88 ;check, payees, journal: payee directive no longer consumes trailing whitespace (#1580) 2021-06-23 14:13:24 -10:00
Simon Michael
46d3eaf920 ;lib: clarify/extend/refactor some line parsing helpers (#1580) 2021-06-23 14:12:17 -10:00
Simon Michael
b81f8f768d ;csv: amount-setting notes, doc improvements from reddit discussion
https://www.reddit.com/r/plaintextaccounting/comments/nxu1ss/hledger_parsing_csv_with_negative_amount_in_debit/
2021-06-11 16:30:43 -10:00
Stephen Morgan
0f1837816d lib,cli,ui,web: Add check balancednoautoconversion command, which checks that
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.

Renames check autobalanced to check balancedwithautoconversion.
2021-06-07 18:58:58 -10:00
Stephen Morgan
68e975adf1 lib,cli,ui,web: Remove unused LANGUAGE pragmas. 2021-06-07 17:33:54 -10:00
Stephen Morgan
55308e1ca8 lib,cli,ui,web: Remove unnecessary CPP when dropping support for GHC 8.2. 2021-06-07 17:33:54 -10:00
Stephen Morgan
5e7b69356f lib: Change internal representation of MixedAmount to use a strict Map
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.

This commit improves performance. Here are some indicative results.

hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)

hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)

hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)

hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)

hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)

hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)

If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:

- If you really care about the individual Amounts and never normalise
  your MixedAmount (for example, just storing `Mixed amts` and then
  extracting `amts` as a pattern match, then use should switch to using
  [Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
  amount arithmetic (+), (-), then you should replace the constructor
  `Mixed` with the function `mixed`. To extract the list of Amounts, use
  the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
  replace that with `mixedAmountStripPrices`. (N.B. this does something
  slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
  but I don't think there's any use case for squashing prices and then
  keeping the first of the squashed prices around. If you disagree let
  me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
  is now the identity function.
2021-05-01 09:45:29 -10:00
Eric Mertens
48d558fc7a Tolerate spaces in amount fields in CSV files 2021-03-26 16:39:24 -07:00
Stephen Morgan
4cb9dfb5b8 lib: Properly escape quotes in csv output. 2021-03-25 09:41:42 -07:00