hledger project notes ===================== * todo/backlog ** documentation, marketing *** developer notes & log *** website hakyll conversion **** add tables of contents **** integrate binaries page ? *** manual *** add missing **** faq **** examples/how-tos **** hledger/ledger comparison/feature matrix **** blog posts *** reduce *** aesthetics **** better screenshots/images ***** use highslide *** automation **** keep blurbs in sync ***** README file ***** hledger.hs module description ***** hledger.cabal description field (exclude home page link) ***** home page description (http://joyful.com/Hledger/editform) ***** mail list description (http://groups.google.com/groups/hledger -> edit welcome msg) ***** gmane description ***** darcsweb description **** keep Options.hs and README option list in sync **** discovering commands for --help **** announcements *** liveness **** show feeds on site ? ***** commits (darcsweb) **** weekly repo activity summary on list **** available feeds *** screencasts **** intro ***** intro to hledger ****** place in the world ****** basic installation ****** quick demo ****** where to go from here ***** installing hledger on windows ***** installing hledger on mac ***** installing hledger on unix ***** accessing hledger's support forums ****** website ****** mail list ****** irc channel ***** reporting a hledger bug **** using ***** income/expense tracking ***** time tracking ***** downloading bank data ***** reconciling with bank statement ***** see time reports by day/week/month/project ***** get accurate numbers for client billing and tax returns ***** find unpaid invoices **** developing ***** intro to hledger development ***** testing hleder ***** documenting hledger ***** a hledger coding example ***** a tour of hledger's code **** ledger cooperation *** developer docs **** developer notes & log **** roadmap ***** 1.0 culmination of 0.x releases - stable/usable/documented followup releases are 1.01, 1.02.. GHC 6.12/HP 2010 primary platform GHC 6.10/HP 2009 also supported if possible GHC 6.8 might work for core features, but not officially supported separate ledger package ? license ? separate vty, web packages ? support plugins ? web: loli+hsp+hack+simpleserver/happstack, or yesod+hstringtemplate+wai+simpleserver/happstack ? add: completion ? chart: register charts ? histogram: cleaned up/removed complete user manual binaries for all platforms ? ***** 2.0 development releases are.. 1.60, 1.61.. or 1.98.01, 1.98.02.. separate ledger lib plugins Decimal binaries for all platforms **** internal api docs **** external api docs **** DEVGUIDE ***** How to do anything that needs doing in the hledger project. ****** website & documentation ******* overview of hledger docs ******* how the site is built ******* convenience urls list.hledger.org - mail list bugs.hledger.org - issue tracker bugs.hledger.org/1 - go to specific issue bugs.hledger.org/new - create a new issue hledger.org/{list,bugs}/* also works ****** issue tracking ****** testing ****** coding ****** funding process ***** reference ****** unsafe things which may fail at runtime include.. ******* incomplete pattern matching ******* error ******* printf ******* read **** functional programming hledger is written in the Haskell programming language; it demonstrates a pure functional implementation of ledger. *** presenting/live demos *** develop funding process **** license change ? **** donate button, see chimoo guy **** funding document 2009/01 ***** text ======= funding ======= vision ====== How to grow the hledger project ? I'm looking for ways to fund active and sustainable hledger development by me and others. A secondary goal is to develop new sustainable models and processes for funding free software developers and other community projects. This is sometimes the point in a free sw project's development where the project leader seemingly loses the plot, alienates contributors and destroys the community's good-will dynamic. I've seen it many times, but a few have succeeded and I want to be one of them - so that I can eat, have a modicum of stability and do my best work in service to the community. At worst, I'll look bad but the project will still be out there. At best I'll live more easily and joyfully while serving the cause of Financial Solvency! So I'm beginning by posting these notes and inviting your thoughts - as much or as little as folks want to give. How could we do this so that all benefit ? funding models ============== Brainstorming some possible funding models & processes. * grants How to find possible grant sources ? * con * getting grant funding is a whole new field to study * slow and time intensive, I imagine * donations Solicit donations. * pro * simple * con * often difficult * donators do not feel a direct benefit * shareware Release the project under a non-free license, requiring commercial users to pay the fee on an honour basis (eg). * pro * flexible, low administration, encourages trust * con * effectively closed-source ? would inhibit collaboration * benefit is still indirect, only a proportion will pay * enforcement/guilt may come into play * limited-time premium branch The funded version of hledger gets some desirable premium features before the free version and is closed-source. Funders/customers pay a fixed price for immediate access to the funded version. Yearly, a new funded version is released and the old funded version is merged into the free version. (To gain experience it could be done on a smaller scale, eg monthly/quarterly.) * pro * all features reach community, predictably * customers are also community funders * customers receive direct benefit from paying * con * free sw developers compete/outshine the premium branch * bounties Some (or all) feature, bugfix, project management or other tasks are published with a bounty attached. When the bounty is paid by one or more funders, the task is performed and delivered. Or, bounty is paid on completion of task (honour system). * pro * funders receive direct benefit * bounties using fundable.org (eg) A more organised form of the above, perhaps facilitating trust, co-funding and larger bounties. * pro * proven process developed by others * con * fundable takes a cut * hosted service Offer hosted and managed ledgers, perhaps with premium features, for a monthly fee * pro * proven model * clear benefit to customers, especially non-technies * con * success of free/self-installed version competes with hosting service * some will avoid web-hosting their financial data * customisation Offer per-user customisations, possibly to be merged in the trunk, for a fee * support Offer user/developer support for a fee * training Offer application and/or financial training for a fee * profit sharing/tithing Each period (quarter, half-year, year), donate 10% (eg) to project contributors and/or supporting projects * transparent funding Funding and usage of funds is published on the web as a ledger * opaque funding All funding and spending need not be made public strengths ========= hledger has some aptitudes in this area: * hledger deals with money => hledger users will tend to have some money * hledger's purpose is to increase financial success => users will feel its value to their bottom line * hledger is a tool that can support project funding, eg by publishing community funding data weaknesses ========== * hledger doesn't have a nice ui yet * hledger has a limited featureset * hledger requires work, eg data entry and chart of accounts maintenance * hledger is geeky * there is competition * hledger has no compelling market niche (aside from payment-averse free software users) competitors/fellow niche inhabitants ==================================== * web apps * netsuite * sql-ledger, ledgersmb * wesabe * ... * desktop apps * quickbooks * quicken * ms money * grisbi * gnucash * excel * ledger! * ... ***** responses ****** albino have you considered talking to business who hate their financial sw and going from there ****** gwern most haskellers have never heard of hledger, sounds arrogant or hubristic to talk of charging for it **** home edition **** real-time project ledger ** packaging, installability *** easier installation **** all platforms ***** binary build/publish process ***** reduce dependencies ***** split packages ? **** linux ***** keep up with debian/ubuntu haskell packagers **** mac ***** easy data entry, then ***** easy installer ***** easy startup **** windows ***** easy data entry, then ***** easy installer ***** easy startup ** testing *** documentation **** site up, current **** demo up, current **** haddock building, current **** doctests ? *** unit **** hunit **** quickcheck **** easier unit test development *** functional **** ledger file parsing tests ***** test all ledger file format features ***** clarify hledgerisms in file format - that hledger can read but ledger can't **** ledger 3 baseline tests **** auto-compare xml output with ledger's ? **** allow multiple tests in .test ? *** performance **** speed, benchmark tests **** memory usage *** build & packaging **** use -Wall and anything else useful **** build with multiple ghc versions **** cabal test **** hackage upload **** cabal install with: ***** ghc 6.8 ***** ghc 6.10.x ***** windows ***** linux ***** macos ***** no flags ***** happs flag ***** vty flag *** field **** talkback, auto bug reports **** usability **** download & usage stats ** errors *** balance: stack space overflow on timelog entries with colons <<< i 2010/1/1 09:15:00 a:b o 2010/1/1 09:45:00 >>>2 Stack space overflow: current size 8388608 bytes. Use `+RTS -Ksize' to increase it. *** register: eats memory $ hours stats Ledger statistics as of 2010-04-01 ---------------------------------- File : current.timelog Period : 2007-02-05 to 2010-01-31 (1091 days) Last transaction : 2010-01-30 (61 days ago) Transactions : 4333 (4.0 per day) Transactions last 30 days: 0 (0.0 per day) Transactions last 7 days : 0 (0.0 per day) Accounts : 947 Account tree depth : 8 Commodities : 2 (h, p) $ hours reg ...Nx100Mb... *** register: with reporting interval and depth, should aggregate deeper txns to specified depth *** parsing: timelog conversion gives unbalanced txns hledger -f- print <<< i 2009/01/02 10:18:26 work:business:systems:website:outage o 2009/01/02 10:35:03 >>> 2009/01/02 * 10:18-10:35 work:business:systems:website:outage 0.3h *** print: virtual parentheses/brackets throw off alignment <<< 2009/01/27 (1/2009) Foobar 2 VLL:VLL_1_Year -7140.00 USD ; Some comment (Company:Baz) -7140.00 USD FLL:Foreign 6000.0 USD VLL_VAT 1140.0 USD >>> 2009/03/27 (000007) Foobar 1 FLL 6783.00 USD (Company:Baz) 6783.00 USD VAT -5700.00 USD VLL_VAT -1083.00 USD *** parsing: misleading error when final newline missing hledger -f- balance <<< 2010/1/1 a 1 b >>> >>>2 "-" (line 1, column 9): unexpected "\r" expecting digit, effective date, cleared flag, transaction code or description and/or comment >>>= 1 *** parsing: wrong line number in error hledger -f- print <<< 2010-03-18 Postage Foo 11.04 GBP @@ 12.46 EUR ; @ 1.129 EUR Bar >>> hledger: parse error at (line 1, column 24): unexpected "@" expecting comment or new-line *** parsing: errors should include the file name This one does: $ hledger -b today "/Users/simon/personal/current.journal" (line 4, column 1) in included file "2010.journal": "2010.journal" (line 1013, column 1): unexpected "\n" could not balance this transaction (real postings are off by $-3156.72) This one doesn't: $ hledger -f 2008.journal stat hledger: parse error at (line 1, column 59): unexpected "[" expecting comment or new-line *** parsing: better leap year checking ledger: While parsing file "/Users/simon/personal/2010.ledger", line 442: While parsing transaction: > 2/27=2/29 (20100201ucla) ucla payment Error: Day of month is not valid for year *** convert: rules file can't have blank lines after last data ** refactoring, cleanup *** more modularity **** packages/namespace ***** make cli reusable ****** balance sheet add-on script ***** more package splits ****** hledger-web ****** hledger-vty ****** makefile review/update **** plugin strategy **** export lists **** graph and reduce dependencies *** clarify levels of abstraction **** web ui balance sheet view - data model, view layout **** hledger web framework - define routes, handlers/views/actions/controllers/presenters, skins/styles.. **** happstack - ? happstack api.. **** hledger app platform - hledger.hs, Options, Utils, withLedgerDo.. **** hledger lib - Ledger, TimeLog, Account, Transaction, Commodity.. **** hledger dev platform - make build, ci, test, bench, prof, check, release.. **** general libs - directory, parsec, regex-*, HUnit, time.. **** cabal - hledger.cabal, hackage.. **** ghc - ghc 6.8, 6.10.. **** haskell 98 **** unix/windows/mac platform *** inspiration http://community.haskell.org/~ndm/downloads/paper-hoogle_overview-19_nov_2008.pdf -> Design Guidelines ** features *** register --wide and/or --format ... *** --related *** Double -> Decimal *** parsing: safety check that effective date > actual (to catch eg 2009/12/30=1/4) *** parsing: support @@ *** parsing: support D *** parsing: accept all real-world ledger files As far as I know it currently accepts all ledger 2.6-era files. Add support for ledger 3 file format as/when that stabilises. It would be nice to optionally report/upload parse errors when they happen. *** talkback feature gather data on real-world installation & usage issues simplify bug reporting/handling improve reliability *** upload feature *** payee & account anonymisation *** easier timelog formats *** implicit timelog account *** add: completion etc. *** convert: generalise/reuse add's history awareness *** web: charts (Chart or google) *** wide/customisable/consistent layout *** --flat balance report format #not: #hledger bal clearview # 43.4h TIME:work:jobs:clearview # 1.7h bad retention collection # 0.1h cash receipts sales co heading # 0.4h commissions dashboard breakage # 4.1h commissions:total detail starting balance # 0.5h cvmanage issues # 1.3h dashboard error # 0.5h update error logging system # 1.0h filter views by sales co:commissions # 1.0h html2ps error # 6.1h payments have wrong sales co # 0.4h site hang # 0.2h sysadmin # 26.6h title 24 #-------------------- # 43.5h #but: #hledger bal clearview --flat --depth 5 # 1.7h bad retention collection # 0.1h cash receipts sales co heading # 0.4h commissions dashboard breakage # 4.1h commissions:total detail starting balance # 0.5h cvmanage issues # 1.3h dashboard error # 1.0h filter views by sales co:commissions # 1.0h html2ps error # 6.1h payments have wrong sales co # 0.4h site hang # 0.2h sysadmin # 26.6h title 24 #-------------------- # 43.4h # #- only depth 5 accounts are shown #- extra amount in parent account, if any, is added as misc #- visible numbers add up to the visible total *** effective/actual dates for postings *** better web ui/gui *** nice reports with charts *** ledger-compatible xml output ? *** allow no command or unordered command with pattern args *** more automated bank data conversion *** ofx download *** watching a changing ledger *** more date syntax ? last nov, next friday, optional this, week of *** more period syntax ? every N days, biweekly *** accept multiple -f files *** more reliable tidy layout from print *** parse more file formats - gnucash, qif, ofx, csv.. *** i18n import Codec.Binary.UTF8.String (encodeString, decodeString) import Data.ByteString.UTF8 (fromString, toString) import Data.ByteString.Char8 (pack, unpack) import Data.Text.Encoding (decodeUtf8) *** speed, memory usage *** clear, documented interfaces/surfaces *** plugin architecture/modular packaging **** goals ***** allow separately-packaged functionality to be discovered at run-time and integrated within the hledger ui. Example: user installs hledger-ofx package from hackage, or adds Ofx.hs to their ~/.hledger/plugins/; then "ofx" is among the commands listed by hledger --help, and/or is a new command available in the web and vty interfaces, and/or is a new file format understood by the convert command. **** issues to consider ***** what is the api for plugins ? they'll want to import Ledger lib, to work with ledger data structures ***** there are different kinds of "plugin". What could plugins provide ? ****** commands - for all uis, or for one or more of them (cli, web, vty..). A command may itself be a new ui. ****** import/export formats ****** skins/styles/templates for uis, eg the web ui ? **** techniques to consider ***** running executables provided by plugins a cli command plugin: cli execs the executable with same arguments a web command plugin: web ui runs the executable as a subprocess and captures the output ***** linking plugins into main app with direct-plugins simplification of plugins lib main app needs to know the types used in plugin's interface weakens type safety, avoiding runtime errors requires extra care requires whole-program linking at plugin load time plugins can be discovered by querying ghc for installed packages or modules in a known part of the hierarchy maintained and keen to help ***** linking plugins into main app with plugins (original) more complex than above more type-safe/featureful ? ***** doing whatever xmonad does with dyre ***** interpreting plugins under control of main app with hint ghci in an IO-like monad types need converting, etc. plugins may run more slowly plugins can be discovered/loaded by module path or by loading files directly * misc ** principles *** we aim to make reliable, maintainable, usable, useful software, sustainably. *** "bugs" are errors, as in the programmers messed up *** docs before packaging before tests before fixes before refactoring before features *** automate *** measure *** test continuously, test everything *** less is more *** code review/pair programming *** inspiration "...simplicity of design was the most essential, guiding principle. Clarity of concepts, economy of features, efficiency and reliability of implementations were its consequences." --Niklaus Wirth "The competent programmer is fully aware of the limited size of his own skull. He therefore approaches his task with full humility, and avoids clever tricks like the plague." --Edsger Dijkstra ** things I want to know *** time where have I been spending my time in recent weeks ? where have I spent my time today ? what is my status wrt spending plan for this week/month/year ? what is my current status wrt time spending goals ? *** money where have I been spending my money ? what is my status wrt spending plan for this week/month/year ? what is my current status wrt spending/savings goals ? what are all my current balances ? what does my balance history look like ? what does my balance future look like ? are there any cashflow, tax, budgetary problems looming ? *** charts [1:27pm] I have decided I am not getting enough visible day-to-day value out of my ledger, I need more of that to stay motivated [1:27pm] What do you think will help in that? [1:27pm] I think some simple self-updating charts, or even good reports in a visible place [1:28pm] something I don't have to spend an hour fiddling with to get answers [1:38pm] Nafai: identifying/designing some useful reports/charts seems to be blocking me [1:39pm] there are probably some standard ones I should use [1:40pm] a graph of daily net worth is probably one of the simplest [1:58pm] what else.. a chart of weekly expenses in key categories [1:58pm] ditto, monthly [1:58pm] a chart of monthly income [1:59pm] those three should help me be more clear about cashflow status [2:00pm] also I'd like something that shows me how much I am on top of financial tracking - how current my numbers are, when last reconciled etc - at a glance [2:01pm] another simple one: current balances in all accounts [2:01pm] those would be a great start [2:04pm] daily net worth, weekly expense, monthly expense, monthly income, confidence/currentness report, and balance report [2:05pm] let's see, which of those 6 would give most payoff right now [2:05pm] probably 5 [2:06pm] how could I measure that ? [2:06pm] number of days since last ledger entry.. [2:06pm] number of ledger entries in last 30 days (compared to average) [2:07pm] number of days since last cleared checking entry (indicating an online reconcile) [2:08pm] those would be a good start. How do I make those visual [2:09pm] well I guess the first step is a script to print them ** data representation *** http://www.python.org/dev/peps/pep-0327/ *** http://www.n-heptane.com/nhlab/repos/Decimal/ *** http://www.n-heptane.com/nhlab/repos/Decimal/Money.hs *** http://www2.hursley.ibm.com/decimal/ ** docs *** http://en.wikibooks.org/wiki/Accounting *** http://books.google.com/books?id=4V8pZmpwmBYC&lpg=PP1&dq=analysis%20patterns&pg=PA95#v=onepage&q&f=false *** lwn grumpy editor articles **** http://lwn.net/Articles/149383/ **** http://lwn.net/Articles/153043/ **** http://lwn.net/Articles/233627/ **** http://lwn.net/Articles/314577/ **** http://lwn.net/Articles/387967/ (free after 5/27) ** software *** http://gnucash.org *** http://www.xtuple.com/postbooks *** http://weberp.org **** http://www.weberp.org/weberp/doc/Manual/ManualContents.php ** code snippets -- trace a MixedAmount matrace :: MixedAmount -> MixedAmount matrace a@(Mixed as) = trace (show as) a -- normalise and trace a MixedAmount nmatrace :: MixedAmount -> MixedAmount nmatrace a = trace (show as) a where (Mixed as) = normaliseMixedAmount a -- cabal test import System.FilePath main = defaultMainWithHooks $ simpleUserHooks { runTests = runTests' } runTests' :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO () runTests' _ _ _ lbi = system testprog >> return () where testprog = (buildDir lbi) "hledger" "hledger test" ** issues with pandoc's rst support *** treats HOME's h2s as h1 *** quotes only first line of a :: literal block *** doesn't support http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#indirect-hyperlink-targets ** must-read article on currency & capital gains accounting http://www.mscs.dal.ca/~selinger/accounting/tutorial.html#1.2 ** feedback *** fabrice niessen +For me, what would be very useful for a 1.0 version would be: + +- @check directive (see beancount), but implemented as a comment for ledger, + so that ledger does not get confused by this, and that you can implement + more features without breaking backward compatibility; + +- account declaration (see beancount), in ledger comments. Giving an account + number would (or could) help for the reporting stuff, for knowing which + value to get to read, for inserting in a given report; + +- some built-in ratios for being able to see the health of the finances (see + my Excel file, if you're interested); + +- easier standard outputs, such as the one above (with expenses and income in + 2 columns). + +- real report generation (I thought at LaTeX as in SQL Ledger, but I am now + heading and producing reports through Org, which is 1000x better). Results + soon. + * journal ** 2010 *** 5/4 **** balance sheet pomodoro 1 started balance sheet script began refactoring for importable Hledger.Cli.* set up missing tools on netbook: haskell-mode adapt to distro & ghc 6.12 upgrade install missing cabal packages tighten dependency to avoid testpack 2.0 api change ghc-pkg dump error **** balance sheet pomodoro 2 set up work log adapt to distro & ghc 6.12 upgrade: ghc-pkg dump error (cabal clean) tools setup: hasktags move Options to Hledger.Cli got trivial balancesheet script working deal with darcs mv screwup *** 5/6 **** review/cleanup pomodoro review/record pending changes develop work log/backlog website hakyll conversion *** 5/19 researched current web libs finished move to Hledger module space cleaned up notes *** 5/20 converted manual to markdown more detailed installation docs