Commit Graph

885 Commits

Author SHA1 Message Date
Simon Michael
fedf36e823 lib: when the text encoding is UTF-8, ignore any UTF-8 BOM prefix
Paypal's new CSV has this BOM prefix (at the time of writing),
which was causing a confusing parse error.
2018-01-04 16:29:23 -08:00
Simon Michael
81e964502b lib: refactor low-level text file reading 2018-01-04 16:17:25 -08:00
Simon Michael
ecdbc05703 Merge branch 'feature/space-as-digits-sep' of https://github.com/ony/hledger into ony-feature/space-as-digits-sep 2017-12-30 09:29:22 -08:00
Simon Michael
8f55b6137d uses today's prices, not those of last txn date, for -V (fix #683,#648) 2017-12-29 21:42:59 -08:00
Simon Michael
69e255cead lib: deduplicate, refactor valuation code 2017-12-29 16:52:08 -08:00
Simon Michael
d15a2dc4c0 lib: fix journalAccountNames doc
[ci skip]
2017-12-29 12:34:51 -08:00
Simon Michael
ddef7d722d lib: update journal account names api
Now we have:

-- | Sorted unique account names posted to by this journal's transactions.
journalAccountNamesUsed :: Journal -> [AccountName]

-- | Sorted unique account names implied by this journal's transactions -
-- accounts posted to and all their implied parent accounts.
journalAccountNamesImplied :: Journal -> [AccountName]

-- | Sorted unique account names declared by account directives in this journal.
journalAccountNamesDeclared :: Journal -> [AccountName]

-- | Sorted unique account names declared by account directives or posted to
-- by transactions in this journal.
journalAccountNamesDeclaredOrUsed :: Journal -> [AccountName]

-- | Sorted unique account names declared by account directives, or posted to
-- or implied as parents by transactions in this journal.
journalAccountNamesDeclaredOrImplied :: Journal -> [AccountName]

-- | Convenience/compatibility alias for journalAccountNamesImpliedOrUsed.
journalAccountNames :: Journal -> [AccountName]
2017-12-29 12:18:04 -08:00
Simon Michael
d7d75c50fe lib: expandAccountNames, accountNamesFromPostings should sort here 2017-12-29 12:16:46 -08:00
Nadrieril
9e6e06033f lib: Allow balance-only entries in csv reader 2017-12-14 18:26:29 -08:00
Mykola Orliuk
95db5062d6 journal: inherit commodities in included files
Followup on some comments for simonmichael/hledger#487
2017-12-04 12:26:26 -08:00
Mykola Orliuk
9cdd02e0b0 journal: pretty errors for included files
Fixes simonmichael/hledger#660
2017-12-04 12:26:26 -08:00
Mykola Orliuk
fa2e680b2e lib: fix styles after cost application
Fixes simonmichael/hledger#509
2017-12-04 12:26:26 -08:00
Mykola Orliuk
47766ea9d4 journal: support space as digits sep
Resolves simonmichael/hledger#330
2017-11-29 17:20:18 +01:00
Dmitry Astapov
6f92e70575 lib: remove needless numeric comparisons in bal/bs
accountsFromPostings is currently doing excessive work when adding up
postings in each account. It sorts (accountName, amount) tuples which
cause amounts in them to be compared. There is no need to look at amount
here at all since subsequent summing up and counting does not depend on
order. It is enough to sort by accountname only.

Went through similar pieces of code, made them all look uniform.
2017-11-29 07:08:18 -08:00
Mykola Orliuk
e58272f28f journal: use groups sep and prec for decimal hint
Use whole AmountStyle in process of resolving decimal/groups separator
ambiguity.

Resolve simonmichael/hledger#399
2017-11-27 15:47:56 -08:00
Mykola Orliuk
7cbdeb40a4 journal: get decimal hint from default commodity
If appropriate commodity directive is missing fallback to default
commodity directive to get number representation style.
2017-11-27 15:47:56 -08:00
Mykola Orliuk
b7dbe044b0 journal: use decimal sep hint for amount parser
Make use of commodity format directive as a hint for parsing amount.

Kinda resolves simonmichael/hledger#487
2017-11-27 15:47:56 -08:00
Mykola Orliuk
dafdaec1ca journal: factor out rawnumberp 2017-11-27 15:47:56 -08:00
Mykola Orliuk
bcf7a1add5 journal: make numbers parsing more strict
- Hunt down adjacent punctuations with altering char.
- Add some tests dedicated to parsing amounts.
2017-11-27 15:47:56 -08:00
Mykola Orliuk
53ad035b24 journal: limit scope of directives backtracking
Effectively improves error reporting for directives.

Resolves simonmichael/hledger#402
2017-11-27 14:32:48 -08:00
Mykola Orliuk
48623b4ceb lib: fix backtracking in periodexpr parser
- Simplify doctests for periodexpr.
- Besides consuming leading space consume ending space for periodexpr also.
- Drop implicit option (def, def) behaviour of periodexpr. I.e. disallow
  hledger reg -p '' and auto-transaction with heading just '~'.
- Slightly re-factor periodexpr.
- Ensure that reportinginterval doesn't consume trailing space.
  Useful if  we'll start disallowing periods like "every1stjan2009-".
2017-11-27 06:38:47 -08:00
Dmitry Astapov
23f3da4e92 cli: --auto adds automated postings to reports
Ledger-style automated postings, previously supported only by
hledger-budget, have landed as a first-class feature.  The --auto
flag activates them, so that any postings they generate are
included in reports.
2017-11-26 14:57:41 +00:00
Dmitry Astapov
f101d5b515 cli: --forecast adds periodic transactions to reports
Ledger-style periodic transactions, previously supported only by
hledger-budget, have landed as a first-class feature.  The --forecast
flag activates them, so that any transactions they generate are
included in reports.
2017-11-26 14:57:41 +00:00
Dmitry Astapov
50b4d76ce9 lib: runPeriodicTransaction's start date must line up with interval
This is very helpful for periodic transactions, because in budget mode
you need to ensure that no periodic transactions extend past the end
of the journal, and in forecast mode you need to make sure that all
periodic transactions are strictly after the end of the journal.
2017-11-26 14:57:41 +00:00
Dmitry Astapov
597e9c47c9 lib: more periodic transaction tests
Some of these demonstrate that runPeriodicTransaction could generate
transactions ouside of requested DateSpan. This happens because
runPeriodicTransaction uses splitSpan internally, and splitSpan always
generates dateSpans that fully cover original DateSpan, extending
beyound left/right boundary if necessary. This is ok if transactions
are generated for budgeting purpose, but during forecasting care should
be taken to check that all generated transactions are happening past
the end of the real journal.
2017-11-26 14:57:41 +00:00
Dmitry Astapov
950891b55b lib: support "every <weekday>"
A shorter spelling for "every <n>th day of week".
2017-11-26 14:57:41 +00:00
Dmitry Astapov
993e3f2b67 lib: support "every 2nd Thursday of month" in period expressions
Useful for periodic transactions.
2017-11-26 14:57:41 +00:00
Dmitry Astapov
f1b4618f2d lib: support "every 11th Nov" in period expressions
Useful for periodic transactions
Without it, once-per-year periodic transactions always occur on 1st Jan.
2017-11-26 14:57:37 +00:00
Dmitry Astapov
7acb5d45aa lib: make month names in period expressions case-insensitive
Currently only lower-case account names are supported, both on the
command line, and in the journal (in periodic transactions):

This works:
$ hledger balance -p nov

This does not:
$ hledger balance -p Nov

First transaction will parse, second will not:
```
cat every-month.journal                                                                                                                                ~/devel/haskell/darcs-get/hledger/examples
~ aug to sep
    assets
    expenses    $1

~ Aug to Sep
    assets
    expenses    $2
```

$../bin/hledger-budget bal -f every-month.journal
hledger-budget: Failed to parse "Aug to Sep": date parse error ()

This commit fixes both cases.
2017-11-26 00:25:08 +00:00
Dmitry Astapov
4049455f26 lib: Fix splitSpan for nthdayof{week,month} - start of DateSpan was not covered
Demonstration:

Consider year-test.journal:
```
2015/02/01 first half
    expenses  $1
    assets

2015/07/01 second half
    expenses  $2
    assets

2016/02/01 first half
    expenses  $4
    assets

2016/07/01 second half
    expenses  $8
    assets

2017/02/01 first half
    expenses  $16
    assets

2017/07/01 second half
    expenses  $32
    assets
```

Year balances are good:
```
$ hledger balance -f year-test.journal -p yearly
Balance changes in 2015/01/01-2017/12/31:

          || 2015  2016  2017
==========++==================
 assets   ||  $-3  $-12  $-48
 expenses ||   $3   $12   $48
----------++------------------
          ||    0     0     0
```

Note how first transaction in 2015 is not included. Note that this is old period expression, so this bug exsits in master:
```$ hledger balance -f year-test.journal -p 'every 2nd day of month'
Balance changes in 2015/07/02-2017/07/01:

          || 2015/07/02-2015/08/01  2015/08/02-2015/09/01  2015/09/02-2015/10/01  2015/10/02-2015/11/01  2015/11/02-2015/12/01  2015/12/02-2016/01/01  2016/01/02-2016/02/01  2016/02/02-2016/03/01  2016/03/02-2016/04/01  2016/04/02-2016/05/01  2016/05/02-2016/06/01  2016/06/02-2016/07/01  2016/07/02-2016/08/01  2016/08/02-2016/09/01  2016/09/02-2016/10/01  2016/10/02-2016/11/01  2016/11/02-2016/12/01  2016/12/02-2017/01/01  2017/01/02-2017/02/01  2017/02/02-2017/03/01  2017/03/02-2017/04/01  2017/04/02-2017/05/01  2017/05/02-2017/06/01  2017/06/02-2017/07/01
==========++========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
 assets   ||                     0                      0                      0                      0                      0                      0                    $-4                      0                      0                      0                      0                    $-8                      0                      0                      0                      0                      0                      0                   $-16                      0                      0                      0                      0                   $-32
 expenses ||                     0                      0                      0                      0                      0                      0                     $4                      0                      0                      0                      0                     $8                      0                      0                      0                      0                      0                      0                    $16                      0                      0                      0                      0                    $32
----------++------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
          ||                     0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0                      0
```

Note how 2015 is absent entirely. This is new expression, but i think that general nature of bug is the same...
```
$ hledger balance -f year-test.journal -p 'every 4th Apr'
Balance changes in 2016/04/04-2018/04/03:

          || 2016/04/04-2017/04/03  2017/04/04-2018/04/03
==========++==============================================
 assets   ||                  $-24                   $-32
 expenses ||                   $24                    $32
----------++----------------------------------------------
          ||                     0                      0
```
2017-11-25 21:43:13 +00:00
Simon Michael
4ab71f0d0a journal: clarify that txn/posting comments must start with semicolon 2017-11-01 08:04:25 -07:00
jeevcat
3a9ea65b99 Rewrite of BalanceAssertion type to track its source position.
Fixes #481.
2017-10-16 13:25:03 -07:00
Sam Jeeves
11684843a0 Balance assertion errors now show line of failed assertion posting. Fixes #481. 2017-10-14 15:11:19 -07:00
Simon Michael
5de51c9bad fix warnings/errors found by make stacktest 2017-09-30 13:00:22 -10:00
Simon Michael
62c822c1e6 lib: more compact debug output for Accounts 2017-09-29 18:50:18 -10:00
Simon Michael
b5602fd771 bal: --sort-amount sorts tree-mode multi-column balance reports 2017-09-29 18:49:47 -10:00
Simon Michael
d527384261 bal: --sort-amount sorts tree-mode single-column balance reports too 2017-09-28 09:49:03 -10:00
Simon Michael
580ad88dca timedot: fix parsing of month quantities (Nmo)
[ci skip]
2017-09-26 15:11:37 -10:00
Simon Michael
cd998e7f2f slight fix for --sort-amount, don't flip when normal balance unspecified 2017-09-25 19:10:21 -10:00
Simon Michael
d9d92b3bf1 bs/bse/is: --sort-amount puts large liabilities/equities/incomes at top
Compound balance commands like these can now be aware of normal account
balance sign, and sort negative balances accordingly.

This also adds utility-ht as a dependency, only for the uncurry function
right now but it looks potentially useful to have.
2017-09-25 19:06:38 -10:00
Simon Michael
35e2e94228 bal/bs/cf/is: --sort-amount sorts by amount instead of account name 2017-09-25 06:17:46 -10:00
Simon Michael
5c85a1dd1c lib: allow trailing whitespace in regex account aliases
Trailing whitespace in the replacement part of a regular expression
account alias is now significant. Eg, slightly flattening some bank
accounts: --alias '/:somebank:/=somebank '
2017-09-22 16:43:03 -10:00
Simon Michael
7edacdf443 cli: import command! hledger import *.csv adds new txns to main file 2017-09-17 18:57:51 -07:00
Simon Michael
1edb828a75 lib, print: rename --new's save files to .latest.*, easier to manage 2017-09-17 13:26:42 -07:00
Simon Michael
669fa706c0 print: --new shows only transactions added since last time
First cut, error messages could be refined etc.
2017-09-15 09:55:17 -07:00
Simon Michael
e3c4a76119 add & use simpler readJournalFilesWithOpts/InputOpts api 2017-09-14 17:43:32 -07:00
Simon Michael
55cebad0d5 cli: refactor: move commands to a subdirectory, reinstate test command 2017-09-12 10:04:36 -07:00
Simon Michael
9501b43471 lib: pivot -> journalPivot, drop implicit tags notion (#598, #608) 2017-09-05 10:48:35 -07:00
Jakub Zárybnický
72cf6a8219 lib: Fix filtering by payee and note (#598) 2017-09-05 08:37:19 -07:00
Simon Michael
b246d0e990 cf: more restrictive regexp (#584) 2017-08-27 08:26:47 -07:00
Christian G. Warden
deaf9cfbef Exclude Fixed Assets from Cashflow Report
Add "fixed" to the regex used to exclude asset accounts from cashflow
reports.  Assumes accounts are set up something like:
assets:fixed assets:equipment
assets:fixed assets:vehicles
assets:fixed assets:buildings
2017-08-27 04:41:05 -07:00
Simon Michael
f02d35ff39 lib, web: remove obsolete conditionals for old GHC 2017-08-25 17:15:26 -07:00
Simon Michael
1ebf1fec28 timedot: also provide syntax for seconds, days, weeks, months & years 2017-08-21 17:28:57 -07:00
Simon Michael
5cdb60b69b timedot: allow minutes to be logged as Nm 2017-08-20 13:00:29 -07:00
Simon Michael
16c07c537b csv: also switch which posting is converted to cost
After switching the postings, make sure it's the second which is converted
to cost.
2017-08-15 08:40:52 -07:00
Simon Michael
ce5724fee1 csv: switch the order of generated postings, so account1 is first
This is subjective, but simplifies things as we'll be adding more than
two postings. Free-form entry templates will allow custom orderings
in future.
2017-08-15 08:16:49 -07:00
Simon Michael
d7d5f8a064 add support for megaparsec 6 (fixes #594)
Older megaparsec is still supported.
Also cleans up our custom parser types,
and some text (un)packing is done in different places
(possible performance impact).
2017-07-27 19:20:46 -07:00
Simon Michael
8851ebc29f bs/cf/is: support --output-file and --output-format=txt|csv
The CSV output should be reasonably ok for dragging into
a spreadsheet and reformatting.
2017-07-25 19:43:45 -07:00
Nicholas Niro
93fbc0e989 lib: Fix of a bug with the -H option.
This patch fixes a bug that happened when using the -H option on
a period without any transaction. Previously, the behavior was no
output at all even though it should have shown the previous ending balances
of past transactions. (This is similar to previously using -H with -E,
but with the extra advantage of not showing empty accounts)
2017-07-15 08:02:03 +01:00
Nicholas Niro
f0914ff08e lib: Added a new much more thorough test to MultiBalanceReports.
This test makes sure that -H works correctly and it does not show
empty accounts.
2017-07-15 08:02:03 +01:00
Nicholas Niro
9873bcfcba lib: Added a new transaction to samplejournal.
The new entry effectively adds a loan which is placed in the checking account.
This loan is then closed by the "pay off" transaction (which was already
present).

This is mainly to be used as a test point for the -H option; to make
sure -H does not show empty accounts.

All previous tests were changed to reflect the new change.
The documentation of the journal module was updated too.
2017-07-15 08:02:03 +01:00
Nicholas Niro
a7f6b551c5 lib: Implemented a testing context for the module MultiBalanceReports.
Of the 2 tests, the first is a simple test on a specific period.
The second is expected to fail at this point until the new upcoming
code to fix the issue with the history option is implemented.

For the record : this issue happens when we use the -H flag for a period
that does not contain any transactions. Currently, the ending balance
values are only taken into account if the current period contains
a Transaction containing one of the previous populated accounts.

For example, if we have a statement on the 2008/01/01 for $1
and we do a command (with -H) to check the value on the
(without transactions) 2008/01/02, we will not get the $1 from
2008/01/01. In that same example, if we had a transaction for the same
account as 2008/01/01 in say 2008/01/03 then the -H command would
successfully show the statement from 2008/01/03 with the initial amount
that we set in 2008/01/01.
2017-07-15 08:02:03 +01:00
Nicholas Niro
fe2a1b35da lib: Added 2 new tests to BalanceReport.
These tests verify the behavior when we input a very specific period.
The second test is meant to make sure that the new upcoming code in
MultiBalanceReports will not change anything in the behavior of
the module BalanceReport.
2017-07-15 08:02:03 +01:00
Nicholas Niro
a204908be6 lib: Added a comment to explain the type MultiBalanceReportTotals 2017-07-15 08:02:03 +01:00
Simon Michael
b3b33336b4 csv: show the "creating/using rules file" message only with --debug 2017-07-06 14:07:46 +01:00
Simon Michael
e94f04311d csv: fix multiple includes in one rules file 2017-07-05 16:08:13 +01:00
Simon Michael
6614aab5d7 csv: add newest-first directive for more robust same-day ordering 2017-07-05 15:24:17 +01:00
Simon Michael
1d5f3a44d5 lib: debug: add a log0 that writes to debug.log in the current directory
Seems to work somewhat, might have some laziness issues.
2017-06-30 19:14:57 +01:00
Simon Michael
59af88b796 ui: add temporary --status-toggles for testing toggle styles (#564) 2017-06-18 16:20:30 -07:00
Simon Michael
22e751983b ui: allow multiple status filters to be active at once (#564) 2017-06-15 19:16:39 -07:00
Simon Michael
dcc58d4a2b lib: rename ClearedStatus type to Status
(and fix hard-coded enum count)
2017-06-15 19:16:39 -07:00
Simon Michael
430c49154a lib: rename Query's Status constructor to StatusQ 2017-06-15 19:16:39 -07:00
Simon Michael
e119941648 rename "uncleared" status to "unmarked" and --uncleared to --unmarked (#564)
See the issue and linked mail list discussion. Ambiguity between the
uncleared state, and the "not cleared" --uncleared flag causes confusion
and friction. At this point it seems best to break with Ledger and
past hledger, pick a new name and drop --uncleared to put an end to it.
2017-06-15 19:16:39 -07:00
Simon Michael
5d28120f6d lib: multiple status: query terms are OR'd (#564)
Like desc: and acct:. I think this is more intuitive and useful,
so now eg "status: status:!" works (equivalent to -UP or "not:status:*").
2017-06-15 19:16:39 -07:00
Simon Michael
7fc921db86 lib, cli: -U/-P/-C flags can be combined (#564) 2017-06-15 19:16:39 -07:00
Simon Michael
8b073014e4 lib, cli: -U/--uncleared no longer matches pending things (#564)
Also begin using "marked"/"unmarked" terminology where it's helpful.
2017-06-15 19:16:39 -07:00
Simon Michael
c5d63f03ce lib: clean up status functional tests, document ClearedStatus (#564) 2017-06-15 19:16:39 -07:00
Carel Fellinger
da9aa81ec7 Update AutoTransaction.hs
lib: fix unit test some more after 5c095534
2017-06-07 12:56:43 +02:00
Simon Michael
7cc2d86ac0 lib: fix unit tests after 5c095534 2017-06-06 09:43:05 -07:00
Simon Michael
5c09553481 lib: keep amounts aligned when rendering a transaction with posting flags
Related to #563, when rendering a transaction, we reserve two more chars
of width so that amounts remain aligned when there are posting flags.
Affects hledger-ui's transaction screen, print, hledger-rewrite etc.
2017-06-04 21:26:18 -07:00
Simon Michael
1adc636554 lib: show pending posting flags when rendering a transaction (fix #563)
This affects hledger-ui's transaction screen, print, hledger-rewrite etc.
A pending flag on postings is now displayed, just like a cleared flag.
Also there will now be a space between flag and account name.
And it's slightly better at aligning amounts, due to now considering
virtual posting parentheses/brackets or something.
2017-06-04 17:54:57 -07:00
Simon Michael
c9e7bbdc7c clarify recent print changes a bit (#551) 2017-05-30 13:57:52 -07:00
Simon Michael
aca7c0d93b lib: clarify journalBalanceTransactions 2017-05-30 13:57:52 -07:00
Christian G. Warden
d39040c634 Add Support for Rewriting Multipler Postings Into Different Commodities (#557)
When generating a new posting as a multiple of an existing posting,
support conversion to a different commodity.  For example, postings in
hours can be used to generate postings in USD.

Automatic transactions generated from rewrite rules use the commodity,
amount style, and transaction price if the rewrite defines a commodity.
2017-05-30 07:30:15 -07:00
Simon Michael
5e00d2f31c csv: refactor (#548) 2017-05-14 18:17:56 +09:00
Nicolas Wavrant
309d1ccd59 CsvReader: simplify '-' signs if there exist more than one (#548)
It prevents hledger to crash later, as it fails to read
amount strings containing more than one '-'

Fix #524
2017-05-14 09:34:33 +09:00
Simon Michael
10d85bedec bal: show negative amounts in red
The balance command now shows negative amounts in red, when it thinks
ANSI codes are supported, ie when TERM is not "dumb" and stdout is not
being redirected or piped somewhere.
2017-04-25 18:34:09 -07:00
Simon Michael
9a86c9ee52 lib: begin supporting colour
Add some basic helpers for working with ANSI colour codes,
and make strWidth and the various string layout functions aware of them.
2017-04-25 18:27:25 -07:00
Simon Michael
48a84cdd58 lib: note an issue with journalfieldnames ordering 2017-04-19 09:05:21 -07:00
Dmitry Astapov
c4c4be5fad Fix parsing of blank/empty balances + testcases (#539) 2017-04-18 09:07:30 -07:00
Dmitry Astapov
451f9d7307 "balance" parser for csv reader (#537)
* "balance" parser for csv reader

* Some docs for
2017-04-14 14:52:03 -07:00
Simon Michael
646cdf34e2 lib: mark todo for multicolumn balance/bs (#518) 2017-03-31 16:55:04 -07:00
Simon Michael
e8dcef5b5f lib: disable WIP ledger4 parser, lighten dependencies 2017-03-29 21:06:26 -07:00
Moritz Kiefer
f4b3f1c094 Add an option to use unicode in balance tables (#528)
* Add an option to use unicode in balance tables

fixes #522

* Add a test for unicode tables

* Document --pretty-tables

* Support --pretty-tables in BalanceView
2017-03-29 11:12:01 -07:00
Simon Michael
b402fc7e7d lib: optserror -> usageError, consolidate with other error functions (#529) 2017-03-29 08:00:51 -07:00
Simon Michael
ea1f19c71e lib: csv: refactor, allow writing different rules text 2017-02-06 02:34:18 -08:00
Simon Michael
9cfb7bf6af lib: update readJournalFromCsv doc 2017-02-06 01:45:32 -08:00
Simon Michael
7dd34d71b5 ui: transaction: fix pattern match failure when pressing E (fixes #508) 2017-02-05 12:00:45 -08:00
Justin Le
f47df67167 mapping function over journal amounts 2017-02-04 16:38:18 -08:00
Justin Le
6bccb847d5 Added a simple 'showMarketPrice' function to show market price directives in a journal-compatible way (#505)
* added showMarketPrice and Hledger.Data.MarketPrice module

* showMarketPrice implemented using showDate

* attempted to add tests to Hledger.Data.MarketPrice

* moved MarketPrice test to Hledger.Read.JournalReader; fixed documentation on MarketPrice; added MarketPrice module to package.yaml
2017-02-03 18:20:00 -08:00