Commit Graph

273 Commits

Author SHA1 Message Date
Stephen Morgan
a24f9a61df fix: web: Toggle showing zero elements properly even when called with
--empty. (#1237)

Calling with --empty now overrides any cookies determining whether
zero accounts are hidden.
2021-09-18 11:50:36 -10:00
Stephen Morgan
df424a7a96 fix: web: Do not hide empty accounts if they have non-empty subaccounts. (#1237) 2021-09-10 16:44:57 -10:00
Stephen Morgan
3456fcb862 test: ui: web: Use report date from _rsDay for hledger-ui and
hledger-web, rather than getCurrentDay.

File watching in hledger-ui continues to use getCurrentDay.
2021-09-09 14:40:39 -10:00
Stephen Morgan
4cb631c0f2 fix: web: Handle non-regular postings in the add transaction form. (#1058) 2021-09-08 13:21:50 -10:00
Stephen Morgan
8274da81fc cln: tests: Remove test and tests, which are just aliases for testCase
and testGroup.

Replacing these removes a layer of indirection, and reduces the need to
depend on Hledger.Utils.Test.
2021-08-30 16:32:19 -10:00
Stephen Morgan
83aa7324eb dev: cli: Move Template Haskell to get git hash from Hledger.Cli.Version
to Hledger.Cli.

Since the git hash changes whenever any file in the repository changes,
this means Hledger.Cli.Version needs to be recompiled all the time.
Since it is at the bottom of the module hierarchy, this means that the
whole hledger package needs to be recompiled. We instead move the
TemplateHaskell splice to one of the top modules, so much less needs to
be recompiled.

Note: Ghc seems to be able to get out of most of the recompiling a lot
of the time (due to caching?), but this makes things more reliable.
2021-08-30 15:57:33 -10:00
Stephen Morgan
1ed06f3bc8 pkg!: Remove Hledger.Utils.UTF8IOCompat module.
This module does nothing beyond define error' and usageError, which have
been moved to Hledger.Utils.
2021-08-30 15:57:33 -10:00
Stephen Morgan
13206d0b18 cln: Rename balancingOpts to defbalancingopts, to be consistent with definputopts, defreportopts, etc. 2021-08-28 19:12:04 -10:00
Stephen Morgan
32dad455fd cln: hlint: Clean up section related warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
8bf7c95697 cln: hlint: Clean up Functor related hlint warnings, and NOINLINE warning. 2021-08-27 06:13:56 -10:00
Stephen Morgan
330c21659f cln: hlint: Clean up Maybe related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
e13239386f cln: hlint: Clean up == and elem related hlint warnings. 2021-08-27 06:13:56 -10:00
Stephen Morgan
eb6047e81b cln: hlint: Remove redundant where and lambda warnings. 2021-08-26 21:00:35 -10:00
Arsen Arsenović
3504a91b42 ref: web: refactor toBloodhoundJson for Data.Text
This has the advantage of there being no extra unpacking/packing of
Data.Text to/from strings where it isn't necessary.
2021-08-24 05:04:12 -10:00
Arsen Arsenović
9ce55146c8 fix: web: b64 encode user controlled input (#1525)
This fixes a reported Stored XSS vulnerability in toBloodhoundJson by
encoding the user-controlled values in this payload into base64 and
parsing them with atob.

In my exploration of the vulnerability with various payloads I and
others crafted, it would appear that this is the only available XSS in
hledger-web in relation to stored accounts and transaction details. If
there is other parts of the UI which may contain user-controlled data,
they should be examined for similar things. In this instance,
protections provided by yesod and other libraries worked fine, but in a
bit of code that hledger-web was generating, the user could insert a
</Script> tag (which is valid HTML and equivalent to </script> but not
caught by the T.Replace that existed in toBloodhoundJson) in order to
switch out of a script context, allowing the parser to be reset, and for
arbitrary JavaScript to run.

The real fix is a bit more involved, but produces much better results:
Content-Security-Policy headers should be introduced, and using
sha256-<hash of script> or a different algorithm, they should be marked
as trusted in the header. This way, if the (in-browser) parser and
hledger-web generator disagree on the source code of the script, the
script won't run. Note that this would still be susceptible to attacks
that involve changing the script by escaping from the string inside it
or something similar to that, which can be avoided additionally by using
either the method used in this commit, or a proper JSON encoder.

The second approach has the advantage of preventing further XSS, to the
extent specified above, in practice, a combination of both should be
used, b64 for embedded data and the CSP sha256-hash script-src over
everything else, which will eliminate all injected or malformed script
blocks (via CSP), in combination with eliminating any HTML closing tags
which might occur in stored data (via b64).

This vulnerability appears to have been first introduced when
autocompletion was added in hledger-web, git tag hledger-0.24, commit
hash: ec51d28839

Test payload: </Script><svg onload=alert(1)//>

Closes #1525
2021-08-24 05:04:12 -10:00
Arjen Langebaerd
3426030a91 feat: added commodity style commandline option 2021-08-17 22:05:29 -10:00
Simon Michael
c8e5673287 ;web: fix ci, haddock failure 2021-08-07 11:35:19 -10:00
Simon Michael
49ff039312 lib: cli/ui/web: clarify version string code
version -> packageversion
versiondescription -> versionStringFor
2021-08-05 22:54:25 -10:00
Simon Michael
606d992fc8 cli: disable no-longer-used --binary-filename flag
It was still supported (as a hidden flag) by hledger-web,
and now is not.
2021-08-05 21:51:10 -10:00
Stephen Morgan
918c243fa9 fix: forecast: Generate forecast transactions at journal finalisation,
rather than as a postprocessing step. (#1638)

This allows us to have a uniform procedure for balancing transactions,
whether they are normal transactions or forecast transactions, including
dealing with balance assignments, balance assertions, and auto postings.
2021-08-05 21:10:47 -10:00
Stephen Morgan
c404800fbf ref!: forecast: Move forecast_ from ReportOpts to InputOpts. 2021-08-05 21:10:47 -10:00
Stephen Morgan
fc8aa602cf lib!: modifyTransactions now takes a Map of commodity styles, and will
style amounts according to that argument. journalAddForecast and
journalTransform now return an Either String Journal.

This improves efficiency, as we no longer have to restyle all amounts in
the journal after generating auto postings or periodic transactions.
Changing the return type of journalAddForecast and journalTransform
reduces partiality.

To get the previous behaviour for modifyTransaction, use modifyTransaction mempty.
2021-07-30 13:52:35 -10:00
Stephen Morgan
b0aa70b27a lib!: Rename the fields of ReportSpec.
This is done to be more consistent with future field naming conventions,
and to make automatic generation of lenses simpler. See discussion in
\#1545.

rsOpts -> _rsReportOpts
rsToday -> _rsDay
rsQuery -> _rsQuery
rsQueryOpts -> _rsQueryOpts
2021-07-23 10:37:08 -10:00
Stephen Morgan
3969eb2679 lib!: lib,cli,ui,web: For accountTransactionsReport, generate the overall
reportq from the ReportSpec, rather than being supplied as a separate
option.

This is the same approach used by the other reports, e.g. EntryReport,
PostingReport, MultiBalanceReport. This reduces code duplication, as
previously the reportq had to be separately tweaked in each of 5
different places.

If you call accountTransactionreport, there is no need to separately
derive the report query.
2021-07-22 19:00:37 -10:00
Stephen Morgan
f673e7c2eb lib!: lib,web: Remove unused TransactionReport. Move the useful utility
functions to AccountTransactionsReport.

If you use transactionsReport, you should either use entryReport if you
don't require a running total, or using accountTransactionsReport with
thisacctq as Any or None (depending on what you want included in the
running total).
2021-07-22 19:00:37 -10:00
Stephen Morgan
acfbd36fb8 dev: web: Use entriesReport instead of transactionReport.
This produces identical results. The only fields used in the TransactionReport were torig, split, and amt.
- torig is the same as in entriesReport
- since transactionsReport calls accountTransactionsReportItems with
  None as thisacctq, all accounts are considered ‘other accounts’, so
  amt is always zero, and ($if not split && not (mixedAmountLooksZero amt)) never fires.
- So the only thing used is torig, and we can just get that from the
  entriesReport.
2021-07-22 19:00:37 -10:00
Simon Michael
87f575e643 ref: bal: rename some report types to clarify/sync with docs
ReportType -> BalanceCalculation
 ChangeReport -> CalcChange
 BudgetReport -> CalcBudget
 ValueChangeReport -> CalcValueChange

BalanceType -> BalanceAccumulation
 PeriodChange -> PerPeriod
 CumulativeChange -> Cumulative
 HistoricalBalance -> Historical

ReportOpts:
 reporttype_ -> balancecalc_
 balancetype_ -> balanceaccum_

CompoundBalanceCommandSpec:
 cbctype -> cbcaccum

Hledger.Reports.ReportOptions:
 balanceTypeOverride -> balanceAccumulationOverride
2021-07-15 14:22:20 -10:00
Stephen Morgan
3380190d9a fix: web: register chart with multiple commodities/prices, broken in 1.22 (#1597)
Make sure to strip prices from amounts before generating JSON data for
the register chart.
2021-07-12 19:02:38 -10:00
Simon Michael
665fec83cd
Merge pull request #1560 from Xitian9/rationalisevaluation
Clean up valuation functions, and make clear which to use where.
2021-06-07 19:02:44 -10:00
Stephen Morgan
0f1837816d lib,cli,ui,web: Add check balancednoautoconversion command, which checks that
transactions are balanced possibly using explicit prices, but without
inferring any prices. This is included in --strict mode.

Renames check autobalanced to check balancedwithautoconversion.
2021-06-07 18:58:58 -10:00
Stephen Morgan
0a019e2167 lib,cli,web,bin: Replace journalSelectingAmountFromOpts with journalApplyValuationFromOpts.
This also has the effect of allowing valuation in more reports, for
example the transactionReport.
2021-06-08 14:55:05 +10:00
Stephen Morgan
68e975adf1 lib,cli,ui,web: Remove unused LANGUAGE pragmas. 2021-06-07 17:33:54 -10:00
Stephen Morgan
55308e1ca8 lib,cli,ui,web: Remove unnecessary CPP when dropping support for GHC 8.2. 2021-06-07 17:33:54 -10:00
Stephen Morgan
bf063e4538 cli,ui,web: Display full git description in --version report if able.
Remove unused prognameanddetailedversion code branches.
2021-06-02 19:46:18 -10:00
Stephen Morgan
5e7b69356f lib: Change internal representation of MixedAmount to use a strict Map
instead of a list of Amounts. No longer export Mixed constructor, to
keep API clean (if you really need it, you can import it directly from
Hledger.Data.Types). We also ensure the JSON representation of
MixedAmount doesn't change: it is stored as a normalised list of
Amounts.

This commit improves performance. Here are some indicative results.

hledger reg -f examples/10000x1000x10.journal
- Maximum residency decreases from 65MB to 60MB (8% decrease)
- Total memory in use decreases from 178MiB to 157MiB (12% decrease)

hledger reg -f examples/10000x10000x10.journal
- Maximum residency decreases from 69MB to 60MB (13% decrease)
- Total memory in use decreases from 198MiB to 153MiB (23% decrease)

hledger bal -f examples/10000x1000x10.journal
- Total heap usage decreases from 6.4GB to 6.0GB (6% decrease)
- Total memory in use decreases from 178MiB to 153MiB (14% decrease)

hledger bal -f examples/10000x10000x10.journal
- Total heap usage decreases from 7.3GB to 6.9GB (5% decrease)
- Total memory in use decreases from 196MiB to 185MiB (5% decrease)

hledger bal -M -f examples/10000x1000x10.journal
- Total heap usage decreases from 16.8GB to 10.6GB (47% decrease)
- Total time decreases from 14.3s to 12.0s (16% decrease)

hledger bal -M -f examples/10000x10000x10.journal
- Total heap usage decreases from 108GB to 48GB (56% decrease)
- Total time decreases from 62s to 41s (33% decrease)

If you never directly use the constructor Mixed or pattern match against
it then you don't need to make any changes. If you do, then do the
following:

- If you really care about the individual Amounts and never normalise
  your MixedAmount (for example, just storing `Mixed amts` and then
  extracting `amts` as a pattern match, then use should switch to using
  [Amount]. This should just involve removing the `Mixed` constructor.
- If you ever call `mixed`, `normaliseMixedAmount`, or do any sort of
  amount arithmetic (+), (-), then you should replace the constructor
  `Mixed` with the function `mixed`. To extract the list of Amounts, use
  the function `amounts`.
- If you ever call `normaliseMixedAmountSquashPricesForDisplay`, you can
  replace that with `mixedAmountStripPrices`. (N.B. this does something
  slightly different from `normaliseMixedAmountSquashPricesForDisplay`,
  but I don't think there's any use case for squashing prices and then
  keeping the first of the squashed prices around. If you disagree let
  me know.)
- Any remaining calls to `normaliseMixedAmount` can be removed, as that
  is now the identity function.
2021-05-01 09:45:29 -10:00
Stephen Morgan
4609e79f2c lib,cli,ui,web: A number of AccountName and Journal functions which are
supposed to produce unique sorted use Sets internally to be slightly
more efficient. There is also a new function journalCommodities.
2021-03-23 11:26:30 -07:00
Simon Michael
ffb985f1a2 ui, web: make --man and --info work, similar to hledger
(cf #1410)
2021-03-03 05:57:31 -08:00
Arnout Engelen
fe6e9a79fe Allow selecting the date range from the chart
By dragging a region with the mouse
2021-02-17 09:05:13 -08:00
Simon Michael
3f095fb472 web: add form: in description field also offer declared/used payees 2021-01-17 16:37:38 -08:00
Simon Michael
90e82b6624 ;web: sketch of a test for #1409 2021-01-13 10:38:13 -08:00
Stephen Morgan
7e44b89bb4 lib: Remove unused label on TranspactionReport and AccountTransactionsReport. 2021-01-02 15:08:09 +11:00
Simon Michael
69edf9003e web: don't hang when reloading the file, eg after adding a transaction (#1409)
This was caused by an circular reference in an untested code path
in getViewData, added some months ago (lines 211-212 in 260283e2:
"lib,cli,ui,web: Introduce ReportSpec, which holds ReportOpts, the day of the report, and the parsed Query.").

This kind of bug unfortunately seems to be easy to create in Haskell,
and sometimes quite tedious to track down.

For this particular case, we should add a suitable test to the hledger-web
test suite, and/or set up a pre-release human test script, like hledger-ui's.
The bug was very obvious to the first downloader of hledger-web 1.20,
but it seems over the last three months nobody used a hledger-web built
from master. Providing and encouraging use of nightly builds could also help.
2020-12-06 17:49:19 -08:00
Simon Michael
a3dc78f3ff web: --test: don't require ./config/settings.yml 2020-11-16 15:36:17 -08:00
Simon Michael
59f1817bbf ;web: --test: show correct program name 2020-11-16 15:13:43 -08:00
Simon Michael
ee73a6aabf web: --test [-- HSPECARGS] runs the test suite 2020-11-16 14:02:16 -08:00
Simon Michael
3651a5f5f4 ;web: tests: refactor, add a test for --forecast (#1390) 2020-11-13 16:40:33 -08:00
Simon Michael
b76a88aa81 web: fix --forecast breakage since 1.18 (fix #1390)
Replaces the partial HashMap.! with List.elemIndex.
(Although the HashMap is more algorithmically efficient,
its API changed recently, possibly complicating building,
and efficiency is just not going to be an issue here.)
2020-11-12 17:03:39 -08:00
Stephen Morgan
260283e2f1 lib,cli,ui,web: Introduce ReportSpec, which holds ReportOpts, the day of
the report, and the parsed Query.
2020-10-23 14:14:11 -07:00
Stephen Morgan
19ab222599 lib,cli: Remove old impure ReportOpts date functions. 2020-10-23 14:14:11 -07:00
Stephen Morgan
fcbe511d42 ui,web: Adopt new ReportOptions interface, store literal query string in UIOpts. 2020-10-23 14:14:11 -07:00