Commit Graph

1991 Commits

Author SHA1 Message Date
Stephen Morgan
4cfd3cb590 lib!: Remove GenericSourcePos, and replace it with either SourcePos or
(SourcePos, SourcePos).

This has been marked for possible removal for a while. We are keeping
strictly more information. Possible edge cases arise with Timeclock and
CsvReader, but I think these are covered.

The particular motivation for getting rid of this is that
GenericSourcePos is creating some awkward import considerations for
little gain. Removing this enables some flattening of the module
dependency tree.
2021-09-20 08:38:33 -10:00
Stephen Morgan
df71d2ddd5 dev: Move journal and text parsing utilities into more appropriate places, to reduce reliance on Hledger.Read.Common. 2021-09-19 17:10:38 -10:00
Stephen Morgan
1e766e481e cln: Flatten dependency tree by removing dependencies of Hledger.Reports
on Hledger.Read.

This is done by using equivalent functions/types defined elsewhere
instead.
2021-09-19 17:10:38 -10:00
Stephen Morgan
315e92c08b pkg!: Break functions related to balancing out into a separate module,
Hledger.Data.Balancing.

Both Hledger.Data.Transaction and Hledger.Data.Journal are massive
module with many things in them. Placing the balancing functions, which
are conceptually related, into a separate module helps keep things more
modular.

It also reduces the risk of import cycles, as right now balancing
functions cannot depend on any functions defined outside of
Hledger.Data.Transaction or Hledger.Data.Journal, respectively, if those
modules require basic transaction or journal functions.
2021-09-19 17:10:38 -10:00
Stephen Morgan
d55ce4d759 ref: performance: Use a Set in accountNamesFromPostings, instead of
nubSort.

This has better performance when there is repetition in the account
names.
2021-09-18 16:00:55 -10:00
Stephen Morgan
ab5350e02c ref: performance: Improve performance for some high-use functions.
mixedAmount(Looks|Is)Zero now operate directly on the MixedAmount,
rather than converting them to a list of amounts first.

mixedAmountCost no longer reconstructs the entire MixedAmount when there
are amounts with no cost.

transactionCheckBalanced only checks if signs are okay if sums are not
okay. It also only traverses the list of postings once when picking real
and balanced virtual postings.
2021-09-18 16:00:55 -10:00
Stephen Morgan
a2d7ac5318 ref: balance: Refactor getPostingsByColumn to use groupByDateSpan.
This reduces code duplication, makes the code more idiomatic, and hides
optimisation magic within groupByDateSpan.
2021-09-18 14:40:28 -10:00
Stephen Morgan
35c33f342b cln: Move column grouping functions form Report.PostingsReport to Data.Dates. 2021-09-18 14:40:28 -10:00
Malte Brandy
e31eb58ada lib: Allow multiline comments in csv rules 2021-09-18 12:43:49 -10:00
Simon Michael
326d6e8dae ref: ReportOptions{infer_value_} renamed to infer_prices_
For more consistency with the flag name (--infer-market-prices).
And BalancingOpts{infer_prices_} is now infer_transaction_prices_.
2021-09-18 12:12:31 -10:00
Simon Michael
daa9fddec9 imp: cli: --infer-market-price renamed to --infer-market-prices
It was too easy to get this wrong, even some of the docs did.
Now either plural or singular spelling works
(since we accept flag prefixes).
2021-09-18 12:12:31 -10:00
Stephen Morgan
9f5b652b4e cln: Clean up miscellaneous functions in Hledger.Data.Amount.
This makes imports more explicit, removes printf, and adds stack traces
to unimplemented calls to Num instances for mixed amounts.
2021-09-18 11:41:53 -10:00
Stephen Morgan
020dd15be4 ref: Simplify parsing of commodity-style option. 2021-09-18 11:41:53 -10:00
Stephen Morgan
6b400194e3 cln: Add explicit export list for Hledger.Data.Account.
All modules in hledger-lib now have explicit export lists, with the
exception of Hledger.Data.Types.
2021-09-18 11:41:53 -10:00
Stephen Morgan
c1d85ba17c ref: Move JournalParser and ErroringJournalParser to Hledger.Data.Journal.
This is so that Hledger.Utils does not depend on Hledger.Data in any
way, and allows us more flexibility in refactoring Hledger.Data.
2021-09-18 11:41:53 -10:00
Stephen Morgan
62351dd329 fix: bal: Make sure boring parent accounts in compound balance reports
include historical postings when account declarations have undeclared
parents. (#1698)
2021-09-17 15:24:19 -10:00
Stephen Morgan
3b5262e54a fix: performance: Speed up PostingsReport when called with a large number of intervals. (#1683)
It is still very slow for large journals, but at least it doesn't run
any slower for interval reports.
2021-09-16 17:40:24 -10:00
Stephen Morgan
0f205295e8 pkg!: test: Use --today in Journal parsing functions. 2021-09-09 14:40:39 -10:00
Stephen Morgan
6905e40c4d pkg!: cli: Add --today option to explicitly set the current date. (#1674)
rawOptsTo* in hledger-lib now takes a day as an argument, and does not
live in the IO monad, since it's now pure.

This is so that we can run tests containing future transactions that
won't fail as soon as ‘the future’ actually arrives.
2021-09-09 14:40:39 -10:00
Stephen Morgan
50f73b7434 fix: balance: Show all empty accounts in flat balance reports. (#1688)
Previously we only showed empty leaves.
2021-09-08 18:17:27 -10:00
Simon Michael
5485990cac fix: csv: report correct CSV line number in errors
Some errors in CSV conversion, such as a failing balance assertion,
were always being reported as line 2.
Reported by Lawrence Wu.
2021-09-01 06:58:15 -10:00
Stephen Morgan
8274da81fc cln: tests: Remove test and tests, which are just aliases for testCase
and testGroup.

Replacing these removes a layer of indirection, and reduces the need to
depend on Hledger.Utils.Test.
2021-08-30 16:32:19 -10:00
Stephen Morgan
1c402edb06 pkg!: Remove Hledger.Data.Commodity module.
There are no modules which depend on Hledger.Data.Commodity which don't
also depend on Hledger.Data.Amount. Though Hledger.Data.Amount is a very
large module and might be broken up, Hledger.Data.Commodity only defines
three very small functions which are used, and so can be combined with
little cost.
2021-08-30 15:57:33 -10:00
Stephen Morgan
a0f9d7560f pkg!: Remove Hledger.Utils.Color module.
This module was only used in one place in Hledger.Data.Amount. The code
has been moved directly into Hledger.Utils.
2021-08-30 15:57:33 -10:00
Stephen Morgan
f1994d5aa8 pkg!: Remove Hledger.Utils.Tree module.
This very small module was only used in Hledger.Data.Account, so the
code was moved into that module instead.
2021-08-30 15:57:33 -10:00
Stephen Morgan
1ed06f3bc8 pkg!: Remove Hledger.Utils.UTF8IOCompat module.
This module does nothing beyond define error' and usageError, which have
been moved to Hledger.Utils.
2021-08-30 15:57:33 -10:00
Stephen Morgan
f3eacebc1d dev: lens: Use TemplateHaskell for ReportOpts and ReportSpec.
Also rename overWithReport/setWithReport to overEither/setEither.
2021-08-28 19:12:04 -10:00
Stephen Morgan
435ec992f9 dev: lens: Introduce lenses for InputOpts and BalancingOpts. 2021-08-28 19:12:04 -10:00
Stephen Morgan
c784da3d0c dev: lens: Create a Template Haskell helper for generating classy lenses
for hledger options.

This works for BalancingOpts, InputOpts, ReportOpts, ReportSpec, and CliOpts.
2021-08-28 19:12:04 -10:00
Stephen Morgan
13206d0b18 cln: Rename balancingOpts to defbalancingopts, to be consistent with definputopts, defreportopts, etc. 2021-08-28 19:12:04 -10:00
Stephen Morgan
d248aec313 cln: hlint: Remove eta reduce warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
32dad455fd cln: hlint: Clean up section related warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
761e00caa4 cln: hlint: Clean up lambda related warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
8bf7c95697 cln: hlint: Clean up Functor related hlint warnings, and NOINLINE warning. 2021-08-27 06:13:56 -10:00
Stephen Morgan
21e62ffcbd cln: hlint: Remove unless and $> warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
beecb3c9ac cln: hlint: Clean up list related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
22db5c4a3f cln: hlint: Remove warnings to use library list functions. 2021-08-27 06:13:56 -10:00
Stephen Morgan
330c21659f cln: hlint: Clean up Maybe related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
e13239386f cln: hlint: Clean up == and elem related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
3431b1b0d9 cln: hlint: Clean up map-fusion related hlint warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
eb6047e81b cln: hlint: Remove redundant where and lambda warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
71032c637e cln: hlint: Remove cons warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
435fbf001e cln: hlint: Remove <|> and bimap warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
46b0745412 cln: hlint: Remove Either and Bifunctor related warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
69502c44fe cln: hlint: Fix record patterns warnings. 2021-08-26 21:00:35 -10:00
Stephen Morgan
1c211f8ab8 cln: hlint: Fix redundant return warning. 2021-08-26 21:00:35 -10:00
Stephen Morgan
c07ad29a87 imp!: forecast: Implements more intuitive logic for the forecast interval. (#1648)
The forecast period begins on:
- the start date supplied to the `--forecast` argument, if present
- otherwise, the later of
  - the report start date if specified with -b/-p/date:
  - the day after the latest normal (non-periodic) transaction in the journal, if any
- otherwise today.
It ends on:
- the end date supplied to the `--forecast` argument, if present
- otherwise the report end date if specified with -e/-p/date:
- otherwise 180 days (6 months) from today.

Note that the previous behaviour did not quite match the documentation,
so this also acts as a bug fix for #1665.
2021-08-26 20:32:30 -10:00
Stephen Morgan
aa7a99a437 cln: hlint: Fix hlint warnings in Query.hs. 2021-08-26 07:23:11 -10:00
Stephen Morgan
d13ce0e134 cln: hlint: Remove Unused LANGUAGE pragma ignore. 2021-08-26 07:23:11 -10:00
Stephen Morgan
1e69fd81ea cln: hlint: Remove if warnings. 2021-08-26 07:23:11 -10:00
Stephen Morgan
fed75c58e9 cln: hlint: Clean up hlint warnings not already ignored in hlint.yaml. 2021-08-25 20:44:36 -10:00
Stephen Morgan
063aaf35b5 cln: hlint: Rename pattern variables to avoid hlint parsing errors. 2021-08-25 20:44:36 -10:00
Stephen Morgan
06312c353a fix!: register: Tighten up spacing around the date in register reports. (#1655)
As a side effect, this changes the Json representation of the
PostingsReport. The maybe report end date is now replaced with a maybe
period.
2021-08-22 08:15:22 -10:00
Simon Michael
e24b7ef90f fix: warning introduced by 993d0b5c 2021-08-22 06:58:08 -10:00
Stephen Morgan
34ab164b32 cln: Remove redundant bang patterns. 2021-08-21 17:50:21 -10:00
Stephen Morgan
912b5e6c23 cln: Derive more instances in Hledger.Data.Types to reduce orphans.
Also clean up some outdated or inaccurate comments.
2021-08-21 17:50:21 -10:00
Simon Michael
993d0b5c54 cli: --forecast now rejects instead of silently ignoring an interval
Something that confused me when testing #1632.
2021-08-20 17:55:59 -10:00
Lawrence
8a5addfb02 lib: implement periodic expressions for weekday et al
Implementation lifts DayOfWeek (renamed to DaysOfWeek) to a list of
days. This should match the previous behavior for single-element lists
2021-08-20 20:30:18 -05:00
Stephen Morgan
4576f8effd pkg!: Migrate the last of old-time over to time, changing ClockTime to
POSIXTime.

This eliminates old-time, which has been deprecated for a while, from
our dependencies.

This introduces a slight incompatibility, as a small number of functions
now take/return POSIXTime instead of ClockTime. Generally you will be
using the current time, in which case you should use getPOSIXTime from
Data.Time.Clock.POSIX instead of getClockTime.

utcTimeToClockTime has been removed, as it is now equivalent to
utcTimeToPOSIXSeconds from Data.Time.Clock.POSIX.
2021-08-19 17:47:19 -10:00
Simon Michael
c135c1a3fc ;cln: slight typos cleanup (#1654) 2021-08-19 17:43:45 -10:00
Lawrence
710823e5d7 imp: bal: handle commodity-column flag in compound balance reports (#1654) 2021-08-19 17:37:30 -10:00
Lawrence
277227acf8 fix: budget: handle transpose flag with commodity-columns (#1654)
Budget formatting is quite complicated since we must determine widths
for each of the transposed columns
2021-08-19 17:37:29 -10:00
Lawrence
44e1ea10fa fix: bal: correctly handle empty commodity in commodity-column (#1654)
We can't filter out empty commodity strings since that is a legitimate
group. Simultaneously, we should only include the empty commodity if it
is explicitly used (part of a posting) and not generated as part of
`Amounts.amounts`
2021-08-19 17:37:19 -10:00
Stephen Morgan
ddba9f6ce4 imp: lib,cli: Implement gain report for balance reports.
A gain report will report on unrealised gains by looking at the
difference between the valuation of an amount (by default, --value=end),
and the valuation of the cost of the amount.
2021-08-18 21:44:30 -10:00
Arjen Langebaerd
3426030a91 feat: added commodity style commandline option 2021-08-17 22:05:29 -10:00
Simon Michael
5c18fb289f ;dev: configure hlint, silence all current warnings 2021-08-11 14:51:46 -10:00
Lawrence
5989ca0381 fix: skip budget grand total and average if they look zero
Makes them consistent with the remaining cells and fixes awkward
alignment issue in commodity-column mode where we don't display
anything
2021-08-08 09:19:25 -10:00
Lawrence
198d2211fc budget: option to view one commodity per row
Extension of commodity-column to budget reporting.
2021-08-08 09:19:25 -10:00
Lawrence Wu
f3c07144a8 bal: option to view one commodity per row
This adds the `--commodity-column` option that displays each commodity
on a separate line and the commodities themselves as a separate column.
The initial design considerations are at
simonmichael.hledger.issues.1559

The single-period balance report with `--commodity-column` does not
interoperate with custom formats.
2021-08-08 09:19:25 -10:00
Stephen Morgan
918c243fa9 fix: forecast: Generate forecast transactions at journal finalisation,
rather than as a postprocessing step. (#1638)

This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
2021-08-05 21:10:47 -10:00
Stephen Morgan
c404800fbf ref!: forecast: Move forecast_ from ReportOpts to InputOpts. 2021-08-05 21:10:47 -10:00
Stephen Morgan
5cca04cdc9 imp: areg: aregister no longer excludes future/forecast transactions by default.
To get the previous behaviour, you can call with the additional query "not📅tomorrow-".

This is because we are preparing to move forecast_ to InputOpts, where
it is not accessible to AccountTransactionsReport.
2021-08-05 21:10:47 -10:00
Stephen Morgan
23b0cad6ef fix: areg: Make sure aregister sorts first by transaction register date,
then by parse order to break any ties. (#1642)
2021-08-05 20:49:25 -10:00
Stephen Morgan
ae3c7e8756 fix: areg: Make sure the true original transaction is made available in
AccountTransactionsReport.

Only a limited number of journal transformations are allowed in
accountTransactionReports due to the need to include the original
transaction. Document these.

Make sure to remove currency and amount queries from the reportq, so
they do not cause problems after valuation.

Avoid confusion by calculating transaction date at one point, and
passing that down with the transaction.
2021-08-02 22:47:50 -10:00
Stephen Morgan
b7e40a9e63 ref: areg: Avoid reversing transactions list by making sure we sort them
in the correct order to start with.
2021-08-02 22:47:50 -10:00
Stephen Morgan
4e0dcec096 fix: areg: Explicitly avoid filtering by depth queries, perform
valuation after all reportq filtering is done (in both the historical
and recent postings), and make sure filtering is done at the correct
point based on whether --txn-dates is set. (#1634)
2021-08-02 22:47:50 -10:00
Stephen Morgan
fc8aa602cf lib!: modifyTransactions now takes a Map of commodity styles, and will
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.

This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.

To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
2021-07-30 13:52:35 -10:00
Stephen Morgan
04a36d4942 ref: Refactor journalFinalise to clarify flow.
The only semantic difference is that we now apply
journalApplyCommodityStyles before running journalCheckAccountsDeclared
and journalCheckCommoditiesDeclared.
2021-07-30 13:52:35 -10:00
Stephen Morgan
c537f9426b dev: lens: Use lenses for updateReportSpec(With). 2021-07-30 09:27:39 -10:00
Stephen Morgan
4e9db4e377 dev: lens: Introduce lenses for ReportOpts and ReportSpec. 2021-07-30 09:27:39 -10:00
Simon Michael
2f819725e5 ;lib: accountTransactionsReport: clarify 2021-07-30 09:09:49 -10:00
Stephen Morgan
9df574b3c0 fix: reg: register --related should not create duplicate postings when
more than one posting in a transaction matches. (#1629)
2021-07-30 09:06:27 -10:00
Stephen Morgan
6528f25593 fix: lib: Filter postings before performing valuation and costing in
MultiBalanceReport, PostingsReport, EntriesReport, and
AccountTransactionsReport. (#1625, #1630)
2021-07-30 09:06:27 -10:00
Simon Michael
d2b799a717 feat: bal: select budget( transaction)s by name (#1612)
This makes it possible to keep multiple named budgets in one journal,
and select the one you want with --budget's argument.

More precisely, you can select the subset of periodic transactions
rules which contain a certain fixed, case-insensitive substring.

Only one such --budget argument is supported, the last one on the
command line takes precedence.
2021-07-23 10:38:20 -10:00
Stephen Morgan
b0aa70b27a lib!: Rename the fields of ReportSpec.
This is done to be more consistent with future field naming conventions,
and to make automatic generation of lenses simpler. See discussion in
\#1545.

rsOpts -> _rsReportOpts
rsToday -> _rsDay
rsQuery -> _rsQuery
rsQueryOpts -> _rsQueryOpts
2021-07-23 10:37:08 -10:00
Stephen Morgan
7ed2a0aa9b lib!: lib: Remove aismultiplier from Amount.
In Amount, aismultiplier is a boolean flag that will always be False,
except for in TMPostingRules, where it indicates whether the posting
rule is a multiplier. It is therefore unnecessary in the vast majority
of cases. This posting pulls this flag out of Amount and puts it into
TMPostingRule, so it is only kept around when necessary.

This changes the parsing of journals somewhat. Previously you could
include an * before an amount anywhere in a Journal, and it would
happily parse and set the aismultiplier flag true. This will now fail
with a parse error: * is now only acceptable before an amount within an
auto posting rule.

Any usage of the library in which the aismultiplier field is read or set
should be removed. If you truly need its functionality, you should
switch to using TMPostingRule.

This changes the JSON output of Amount, as it will no longer include
aismultiplier.
2021-07-22 19:06:33 -10:00
Stephen Morgan
3969eb2679 lib!: lib,cli,ui,web: For accountTransactionsReport, generate the overall
reportq from the ReportSpec, rather than being supplied as a separate
option.

This is the same approach used by the other reports, e.g. EntryReport,
PostingReport, MultiBalanceReport. This reduces code duplication, as
previously the reportq had to be separately tweaked in each of 5
different places.

If you call accountTransactionreport, there is no need to separately
derive the report query.
2021-07-22 19:00:37 -10:00
Stephen Morgan
f673e7c2eb lib!: lib,web: Remove unused TransactionReport. Move the useful utility
functions to AccountTransactionsReport.

If you use transactionsReport, you should either use entryReport if you
don't require a running total, or using accountTransactionsReport with
thisacctq as Any or None (depending on what you want included in the
running total).
2021-07-22 19:00:37 -10:00
Stephen Morgan
58516eea77 imp: json: Decrease indentation in JSON output. 2021-07-20 23:28:18 -10:00
Stephen Morgan
c606f874fb imp: json: The keys of JSON objects are now displayed in alphabetical order, making it stable across different systems and compilers. 2021-07-20 20:37:33 -10:00
Simon Michael
87f575e643 ref: bal: rename some report types to clarify/sync with docs
ReportType -> BalanceCalculation
 ChangeReport -> CalcChange
 BudgetReport -> CalcBudget
 ValueChangeReport -> CalcValueChange

BalanceType -> BalanceAccumulation
 PeriodChange -> PerPeriod
 CumulativeChange -> Cumulative
 HistoricalBalance -> Historical

ReportOpts:
 reporttype_ -> balancecalc_
 balancetype_ -> balanceaccum_

CompoundBalanceCommandSpec:
 cbctype -> cbcaccum

Hledger.Reports.ReportOptions:
 balanceTypeOverride -> balanceAccumulationOverride
2021-07-15 14:22:20 -10:00
Stephen Morgan
2df8ad781e
imp: cli: Don't show extra double quotes in missing journal file error. (#1601) 2021-07-13 10:41:51 -10:00
Stephen Morgan
cf25d7d56d
imp: bal: more predictable sort order with multiple commodities (#1563, #1564)
This change provides more predictable and intuitive behaviour when 
using -S/--sort-amount with multiple commodities. 
It implements a custom Ord (and Eq) instance for MixedAmount 
which substitutes zero for any missing commodities.

As a consequence, all the ways of representing zero with a MixedAmount ([],
[A 0], [A 0, B 0, ...]) are now Eq-ual (==), whereas before they were
not. We have not been able to find anything broken by this change.

* imp: lib: Compare MixedAmounts by substituting zero for any missing commodities. (#1563)
* ;doc: Update docs for new multicommodity sort by amount rules.
2021-07-12 21:11:50 -10:00
Simon Michael
a681e5329d fix: ui: transaction: possibly off-by-one valuation date
I'm not sure if it arose in practice, but: ensure TransactionScreen
in V mode would correctly use the journal's last day as valuation date,
not the day after that.
2021-07-12 18:50:06 -10:00
Simon Michael
5d2caf3c7f lib: clarify journalEndDate, add journalLastDay 2021-07-12 18:50:06 -10:00
Simon Michael
3cd80e1f57 lib: export transactionCheckBalanced (#1596) 2021-07-07 13:20:13 -10:00
Stephen Morgan
e2198ff1ee lib: Auto-postings with Amount queries should filter only those commodities which match the query.
Also corrects a regression introduced in
8ab29f84b3 where transaction modifier
postings without multipliers would incorrectly be filtered by commodity.
2021-07-01 09:35:56 -10:00
Stephen Morgan
2a9d358627 lib: Make sure automatic postings generated from postings with more than one commodity match on commodity symbol (#1582). 2021-06-30 17:35:43 -10:00
Stephen Morgan
bb7d04c031 lib,cli: No longer strip prices in journalApplyValuationFromOptsWith and mixedAmountApplyValuationAfterSumFromOptsWith (#1577).
These were theoretically an efficiency improvement, but have been
error-prone. We instead handle stripping prices at the point of
consumption.
2021-06-30 16:45:05 -10:00
Stephen Morgan
613efba1bc ui: Do not log to debug.log when regenerating journal in Transaction
screen (#1556). Also really clear cost setting when doing so.

Since plog is no longer used anywhere, and tends to create bugs when it
is, we remove it.
2021-06-30 16:07:32 -10:00