git.iko.soy/typeable/pandoc
on rc/2.11.3.1 help sign in
typeable/pandoc/CONTRIBUTING.md
467 lines 19 kB history raw
1Contributing to pandoc
2======================
4Welcome to pandoc! Very soon after its beginnings in 2006, pandoc
5has been influenced, improved, and modified, by users, devs, and
6newcomers alike. The project thrives on its active community. It is
7great to have you here.
9How can I help?
10---------------
12There are many ways in which you can support pandoc. Here are a few
13ideas:
15 * Participate in online discussions. The [pandoc-discuss] mailing
16 list is a good place for this. This gives valuable input and
17 makes it easier to improve the program.
19 * Help with questions. Every request that is answered by the wider
20 community frees time for programming contributors. This will
21 speed up development of new features and issue fixes. Don't
22 underestimate your knowledge, please share it!
24 Good places to help are the [pandoc-discuss] mailing list, Q/A
25 sites like StackOverflow, community forums (e.g.
26 [RStudio][RStudio Community], [Zettlr][Zettlr Forum]), and, for
27 technical questions, the GitHub [issue tracker].
29 * Write or improve documentation. If you ran into a problem which
30 took more time to figure out than expected, please consider to
31 save other users from the same experience. People writing the
32 documentation tend to lack an outside view, so please help
33 provide one. Good documentation is both difficult and extremely
34 important.
36 The official docs are not the only place for documentation.
37 Pandoc also has a [Wiki][pandoc wiki]. Private blogs can serve
38 as documentation just as the official manual can.
40 * Contribute code. No matter whether it's a small fix in a format
41 template or a huge lump of Haskell code: help is welcome. It's
42 usually a good idea to talk about the plans early, as this can
43 prevent unnecessary work. See below for more information.
45 * Last but not least: consider funding the development and
46 maintenance of pandoc financially. You can find sponsor buttons
47 on the [pandoc website] and the [GitHub repository][GitHub
48 repo].
50A rich ecosystem of libraries, editors, filters, and templates has
51developed around pandoc; conversely, pandoc builds and depends on a
52large number of libraries. Contributing to any of these projects is
53another way that can help to ensure stability, and to keep pushing
54the boundaries of what is possible with pandoc.
56[RStudio Community]: https://community.rstudio.com/
57[Zettlr Forum]: https://forum.zettlr.com/
58[pandoc wiki]: https://github.com/jgm/pandoc/wiki
59[pandoc website]: https://pandoc.org
60[GitHub repo]: https://github.com/jgm/pandoc
62Have a question?
63----------------
65Ask on [pandoc-discuss].
68Found a bug?
69------------
71Bug reports are welcome! Please report all bugs on pandoc's GitHub
72[issue tracker].
74Before you submit a bug report, search the [open issues] *and* [closed issues]
75to make sure the issue hasn't come up before. Also, check the [User's Guide] and
76[FAQs] for anything relevant.
78Make sure you can reproduce the bug with the [latest released
79version] of pandoc---or, even better, the development version,
80since the bug may have been fixed since the last release.
81[Nightly builds] are available, so you don't need to compile
82from source to test against the development version.
83(To fetch a nightly, visit the link, click the topmost "Nightly"
84in the table, then choose your platform under "Artifacts." Note
85that you must be logged in with a GitHub account.)
87Your report should give detailed, *reproducible* instructions, including
89 * the pandoc version (check using `pandoc -v`)
90 * the exact command line used
91 * the exact input used
92 * the output received
93 * the output you expected instead
95A small test case (just a few lines) is ideal. If your input is large,
96try to whittle it down to a *minimum working example*.
98Out of scope?
99-------------
101A less than perfect conversion does not necessarily mean there's
102a bug in pandoc. Quoting from the MANUAL:
104> Because pandoc's intermediate representation of a document is less
105> expressive than many of the formats it converts between, one should
106> not expect perfect conversions between every format and every other.
107> Pandoc attempts to preserve the structural elements of a document, but
108> not formatting details such as margin size. And some document elements,
109> such as complex tables, may not fit into pandoc's simple document
110> model. While conversions from pandoc's Markdown to all formats aspire
111> to be perfect, conversions from formats more expressive than pandoc's
112> Markdown can be expected to be lossy.
114For example, both `docx` and `odt` formats can represent margin size, but
115because pandoc's internal document model does not contain a representation of
116margin size, this information will be lost on converting from docx
117to `odt`. (You can, however, customize margin size using `--reference-doc`.)
119So before submitting a bug report, consider whether it might be
120"out of scope." If it concerns a feature of documents that isn't
121representable in pandoc's Markdown, then it very likely is.
122(If in doubt, you can always ask on [pandoc-discuss].)
124Fixing bugs from the issue tracker
125----------------------------------
127Almost all the bugs on the issue tracker have one or more associated
128tags. These are used to indicate the *complexity* and *nature* of a
129bug. There is not yet a way to indicate priority. An up to date
130summary of issues can be found on [GitHub labels].
132* [good first issue] — The perfect starting point for new contributors. The
133 issue is generic and can be resolved without deep knowledge of the code
134 base.
135* [enhancement] — A feature which would be desirable. We recommend
136 you discuss any proposed enhancement on pandoc-discuss before
137 writing code.
138* [bug] — A problem which needs to be fixed.
139* [complexity:low] — The fix should only be a couple of lines.
140* [complexity:high] — The fix might require structural changes or in depth
141 knowledge of the code base.
142* [new:reader] — A request to add a new input format.
143* [new:writer] — A request to add a new output format.
144* [docs] — A discrepancy, or ambiguity in the documentation.
145* [status:in-progress] — Someone is actively working on or planning to work on the
146 ticket.
147* [status:more-discussion-needed] — It is unclear what the correct approach
148 to solving the ticket is. Before starting on tickets such as this it
149 would be advisable to post on the ticket.
150* [status:more-info-needed] — We require more information from a user before
151 we can classify a report properly.
153Issues related to a specific format are tagged accordingly, e.g. feature request
154or bug reports related to Markdown are labelled with [format:markdown].
156Have an idea for a new feature?
157-------------------------------
159First, search [pandoc-discuss] and the issue tracker (both [open issues] *and*
160[closed issues]) to make sure that the idea has not been discussed before.
162Explain the rationale for the feature you're requesting. Why would this
163feature be useful? Consider also any possible drawbacks, including backwards
164compatibility, new library dependencies, and performance issues.
166Features are very rarely "implement and forget", as all code must be
167maintained. This is especially relevant for large or complex
168contributions. It is helpful to be sympathetic to that fact, and to
169communicate future plans and availability clearly.
171Any potential new feature is best discussed on [pandoc-discuss]
172before opening an issue.
174Patches and pull requests
175-------------------------
177Patches and pull requests are welcome. Before you put time into a nontrivial
178patch, it is a good idea to discuss it on [pandoc-discuss], especially if it is
179for a new feature (rather than fixing a bug).
181Please follow these guidelines:
1831. Each patch (commit) should make a single logical change (fix a bug, add
184 a feature, clean up some code, add documentation). Everything
185 related to that change should be included (including tests and
186 documentation), and nothing unrelated should be included.
1882. The first line of the commit message should be a short description
189 of the whole commit (ideally <= 50 characters). Then there should
190 be a blank line, followed by a more detailed description of the
191 change.
1933. Follow the stylistic conventions you find in the existing
194 pandoc code. Use spaces, not tabs, and wrap code to 80 columns.
195 Always include type signatures for top-level functions.
196 Consider installing [EditorConfig], this will help you to follow the
197 coding style prevalent in pandoc.
1994. Your code should compile without warnings (`-Wall` clean).
2015. Run the tests to make sure your code does not introduce new bugs.
202 (See below under [Tests](#tests).) All tests should pass.
2046. It is a good idea to add test cases for the bug you are fixing. (See
205 below under [Tests](#tests).) If you are adding a new writer or reader,
206 you must include tests.
2087. If you are adding a new feature, include updates to `MANUAL.txt`.
2108. All code must be released under the general license governing pandoc
211 (GPL v2).
2139. It is better not to introduce new dependencies. Dependencies on
214 external C libraries should especially be avoided.
21610. We aim for compatibility with ghc versions from 8.0 to the
217 latest release. All pull requests and commits are tested
218 automatically on GitHub Actions.
220Tests
221-----
223Tests can be run as follows:
225 cabal install --only-dependencies --enable-tests
226 cabal configure --enable-tests
227 cabal build
228 cabal test
230or, if you're using [stack],
232 stack setup
233 stack test
235The test program is `test/test-pandoc.hs`.
237To run particular tests (pattern-matching on their names), use
238the `-p` option:
240 cabal install pandoc --enable-tests
241 cabal test --test-options='-p markdown'
243Or with stack:
245 stack test --test-arguments='-p markdown'
247It is often helpful to add `-j4` (run tests in parallel)
248and `--hide-successes` (don't clutter output with successes)
249to the test arguments as well.
251If you add a new feature to pandoc, please add tests as well, following
252the pattern of the existing tests. The test suite code is in
253`test/test-pandoc.hs`. If you are adding a new reader or writer, it is
254probably easiest to add some data files to the `test` directory, and
255modify `test/Tests/Old.hs`. Otherwise, it is better to modify the module
256under the `test/Tests` hierarchy corresponding to the pandoc module you
257are changing. Alternatively, you may add a "command test" to
258the `/test/command/` hierarchy, following the pattern of the tests there.
259These test files should have a meaningful name, which can include the issue
260number and/or the feature that's being tested. For example, `5474-tables.md`
261refers to both issue and feature.
263You can rebuild the golden tests in `tests/` by passing
264`--accept` to the test script. (If you're using stack, `stack
265test --test-arguments "--accept"`; or `make TESTARGS=--accept`).
266Then check the changed golden files for accuracy, and
267commit the changes. For docx or pptx tests, open the files in Word
268or Powerpoint to ensure that they weren't corrupted and that
269they had the expected result, and mention the Word/Powerpoint
270version and OS in your commit comment.
272Code style
273----------
275Pandoc uses [hlint] to identify opportunities for code improvements
276like redundant brackets or unnecessary `Language` extensions.
277However, sometimes there are cases where there are good reasons to
278use code different from what hlint proposes. In these cases, the
279respective warning should be disabled in the file `.hlint.yaml`.
281There should be no errors when running `hlint .`; this is checked by
282the continuous integration (CI) setup. It is recommended that
283contributors check their code with a local hlint installation, but
284relying on the CI is fine, too.
286A good way to ensure no new warnings are introduced is to use a Git
287[pre-commit hook] which runs hlint on all updated Haskell files
288before creating a commit:
290 #!/bin/sh
291 git diff --diff-filter=MA --cached --name-only | grep '\.hs$' | \
292 xargs hlint --hint .hlint.yaml
294Saving this to `.git/hooks/pre-commit`, and making the script
295executable, will prevent accidental introduction of potentially
296problematic code.
298Benchmarks
299----------
301To run benchmarks with cabal:
303 cabal configure --enable-benchmarks
304 cabal build
305 cabal bench
307With stack:
309 stack bench
311You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
312to get some statistics on memory usage. (Eventually this should
313be incorporated into the benchmark suite.)
315Using the REPL
316--------------
318With a recent version of cabal, you can do `cabal repl` and get
319a ghci REPL for working with pandoc. With [stack], use
320`stack ghci`.
322We recommend using the following `.ghci` file (which can be
323placed in the source directory):
325 :set -fobject-code
326 :set -XTypeSynonymInstances
327 :set -XScopedTypeVariables
328 :set -XOverloadedStrings
330Profiling
331---------
333To diagnose a performance issue with parsing, first try using
334the `--trace` option. This will give you a record of when block
335parsers succeed, so you can spot backtracking issues.
337To use the GHC profiler with cabal:
339 cabal clean
340 cabal install --enable-library-profiling --enable-executable-profiling
341 pandoc +RTS -p -RTS [file]...
342 less pandoc.prof
344With stack:
346 stack clean
347 stack install --profile
348 pandoc +RTS -p -RTS [file]...
349 less pandoc.prof
351Templates
352---------
354The default templates live in `data/templates`, which is a git
355subtree linked to <https://github.com/jgm/pandoc-templates.git>.
356The purpose of maintaining a separate repository is to allow
357people to maintain variant templates as a fork.
359You can modify the templates and submit patches without worrying
360much about this: when these patches are merged, we will
361push them to the main templates repository by doing
363 git subtree push --prefix=data/templates templates master
365where `templates` is a remote pointing to the templates
366repository.
368The code
369--------
371Pandoc has a publicly accessible git repository on
372GitHub: <http://github.com/jgm/pandoc>. To get a local copy of the source:
374 git clone https://github.com/jgm/pandoc.git
376The source for the main pandoc program is `pandoc.hs`. The source for
377the pandoc library is in `src/`, the source for the tests is in
378`test/`, and the source for the benchmarks is in `benchmark/`.
380The modules `Text.Pandoc.Definition`, `Text.Pandoc.Builder`, and
381`Text.Pandoc.Generic` are in a separate library `pandoc-types`. The code can
382be found in <http://github.com/jgm/pandoc-types>.
384To build pandoc, you will need a working installation of the
385[Haskell platform].
387The library is structured as follows:
389 - `Text.Pandoc` is a top-level module that exports what is needed
390 by most users of the library. Any patches that add new readers
391 or writers will need to make changes here, too.
392 - `Text.Pandoc.Definition` (in `pandoc-types`) defines the types
393 used for representing a pandoc document.
394 - `Text.Pandoc.Builder` (in `pandoc-types`) provides functions for
395 building pandoc documents programmatically.
396 - `Text.Pandoc.Generics` (in `pandoc-types`) provides functions allowing
397 you to promote functions that operate on parts of pandoc documents
398 to functions that operate on whole pandoc documents, walking the
399 tree automatically.
400 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
401 are the writers.
402 - `Text.Pandoc.Biblio` is a utility module for formatting citations
403 using citeproc-hs.
404 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
405 cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
406 the preprocessor [hsb2hs].
407 - `Text.Pandoc.Highlighting` contains the interface to the
408 skylighting library, which is used for code syntax highlighting.
409 - `Text.Pandoc.ImageSize` is a utility module containing functions for
410 calculating image sizes from the contents of image files.
411 - `Text.Pandoc.MIME` contains functions for associating MIME types
412 with extensions.
413 - `Text.Pandoc.Options` defines reader and writer options.
414 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
415 LaTeX source.
416 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
417 - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
418 the needs of pandoc.
419 - `Text.Pandoc.SelfContained` contains functions for making an HTML
420 file "self-contained," by importing remotely linked images, CSS,
421 and JavaScript and turning them into `data:` URLs.
422 - `Text.Pandoc.Shared` is a grab-bag of shared utility functions.
423 - `Text.Pandoc.Writers.Shared` contains utilities used in writers only.
424 - `Text.Pandoc.Slides` contains functions for splitting a markdown document
425 into slides, using the conventions described in the MANUAL.
426 - `Text.Pandoc.Templates` defines pandoc's templating system.
427 - `Text.Pandoc.UTF8` contains functions for converting text to and from
428 UTF8 bytestrings (strict and lazy).
429 - `Text.Pandoc.Asciify` contains functions to derive ascii versions of
430 identifiers that use accented characters.
431 - `Text.Pandoc.UUID` contains functions for generating UUIDs.
432 - `Text.Pandoc.XML` contains functions for formatting XML.
434Lua filters
435-----------
437If you've written a useful pandoc [lua filter](lua-filters.html),
438you may want to consider submitting a pull request to the
439[lua-filters repository](https://github.com/pandoc/lua-filters).
441[open issues]: https://github.com/jgm/pandoc/issues
442[closed issues]: https://github.com/jgm/pandoc/issues?q=is%3Aissue+is%3Aclosed
443[latest released version]: https://github.com/jgm/pandoc/releases/latest
444[Nightly builds]: https://github.com/jgm/pandoc/actions?query=workflow%3ANightly
445[pandoc-discuss]: http://groups.google.com/group/pandoc-discuss
446[issue tracker]: https://github.com/jgm/pandoc/issues
447[User's Guide]: http://pandoc.org/MANUAL.html
448[FAQs]: http://pandoc.org/faqs.html
449[EditorConfig]: http://editorconfig.org/
450[Haskell platform]: http://www.haskell.org/platform/
451[hlint]: https://hackage.haskell.org/package/hlint
452[hsb2hs]: http://hackage.haskell.org/package/hsb2hs
453[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
454[GitHub labels]: https://github.com/jgm/pandoc/labels
455[good first issue]:https://github.com/jgm/pandoc/labels/good%20first%20issue
456[enhancement]: https://github.com/jgm/pandoc/labels/enhancement
457[bug]: https://github.com/jgm/pandoc/labels/bug
458[complexity:low]: https://github.com/jgm/pandoc/labels/complexity:low
459[complexity:high]: https://github.com/jgm/pandoc/labels/complexity:high
460[docs]: https://github.com/jgm/pandoc/labels/docs
461[format:markdown]: https://github.com/jgm/pandoc/labels/format:markdown
462[new:reader]: https://github.com/jgm/pandoc/labels/new:reader
463[new:writer]: https://github.com/jgm/pandoc/labels/new:writer
464[status:in-progress]: https://github.com/jgm/pandoc/labels/status:in-progress
465[status:more-discussion-needed]: https://github.com/jgm/pandoc/labels/status:more-discussion-needed
466[status:more-info-needed]: https://github.com/jgm/pandoc/labels/status:more-info-needed
467[stack]: https://github.com/commercialhaskell/stack