git.iko.soy/typeable/pandoc
on 2.13 help sign in
typeable/pandoc/CONTRIBUTING.md
474 lines 19 kB history raw
1Contributing to pandoc
2======================
4Welcome to pandoc! Very soon after its beginnings in 2006, pandoc
5has been influenced, improved, and modified, by users, devs, and
6newcomers alike. The project thrives on its active community. It is
7great to have you here.
9How can I help?
10---------------
12There are many ways in which you can support pandoc. Here are a few
13ideas:
15 * Participate in online discussions. The [pandoc-discuss] mailing
16 list is a good place for this. This gives valuable input and
17 makes it easier to improve the program.
19 * Help with questions. Every request that is answered by the wider
20 community frees time for programming contributors. This will
21 speed up development of new features and issue fixes. Don't
22 underestimate your knowledge, please share it!
24 Good places to help are the [pandoc-discuss] mailing list, Q/A
25 sites like StackOverflow, community forums (e.g.
26 [RStudio][RStudio Community], [Zettlr][Zettlr Forum]), and, for
27 technical questions, the GitHub [issue tracker].
29 * Write or improve documentation. If you ran into a problem which
30 took more time to figure out than expected, please consider to
31 save other users from the same experience. People writing the
32 documentation tend to lack an outside view, so please help
33 provide one. Good documentation is both difficult and extremely
34 important.
36 The official docs are not the only place for documentation.
37 Pandoc also has a [Wiki][pandoc wiki]. Private blogs can serve
38 as documentation just as the official manual can.
40 * Contribute code. No matter whether it's a small fix in a format
41 template or a huge lump of Haskell code: help is welcome. It's
42 usually a good idea to talk about the plans early, as this can
43 prevent unnecessary work. See below for more information.
45 * Last but not least: consider funding the development and
46 maintenance of pandoc financially. You can find sponsor buttons
47 on the [pandoc website] and the [GitHub repository][GitHub
48 repo].
50A rich ecosystem of libraries, editors, filters, and templates has
51developed around pandoc; conversely, pandoc builds and depends on a
52large number of libraries. Contributing to any of these projects is
53another way that can help to ensure stability, and to keep pushing
54the boundaries of what is possible with pandoc.
56[RStudio Community]: https://community.rstudio.com/
57[Zettlr Forum]: https://forum.zettlr.com/
58[pandoc wiki]: https://github.com/jgm/pandoc/wiki
59[pandoc website]: https://pandoc.org
60[GitHub repo]: https://github.com/jgm/pandoc
62Have a question?
63----------------
65Ask on [pandoc-discuss].
68Found a bug?
69------------
71Bug reports are welcome! Please report all bugs on pandoc's GitHub
72[issue tracker].
74Before you submit a bug report, search the [open issues] *and* [closed issues]
75to make sure the issue hasn't come up before. Also, check the [User's Guide] and
76[FAQs] for anything relevant.
78Make sure you can reproduce the bug with the [latest released
79version] of pandoc---or, even better, the development version,
80since the bug may have been fixed since the last release.
81[Nightly builds] are available, so you don't need to compile
82from source to test against the development version.
83(To fetch a nightly, visit the link, click the topmost "Nightly"
84in the table, then choose your platform under "Artifacts." Note
85that you must be logged in with a GitHub account.)
87Your report should give detailed, *reproducible* instructions, including
89 * the pandoc version (check using `pandoc -v`)
90 * the exact command line used
91 * the exact input used
92 * the output received
93 * the output you expected instead
95A small test case (just a few lines) is ideal. If your input is large,
96try to whittle it down to a *minimum working example*.
98Out of scope?
99-------------
101A less than perfect conversion does not necessarily mean there's
102a bug in pandoc. Quoting from the MANUAL:
104> Because pandoc's intermediate representation of a document is less
105> expressive than many of the formats it converts between, one should
106> not expect perfect conversions between every format and every other.
107> Pandoc attempts to preserve the structural elements of a document, but
108> not formatting details such as margin size. And some document elements,
109> such as complex tables, may not fit into pandoc's simple document
110> model. While conversions from pandoc's Markdown to all formats aspire
111> to be perfect, conversions from formats more expressive than pandoc's
112> Markdown can be expected to be lossy.
114For example, both `docx` and `odt` formats can represent margin size, but
115because pandoc's internal document model does not contain a representation of
116margin size, this information will be lost on converting from docx
117to `odt`. (You can, however, customize margin size using `--reference-doc`.)
119So before submitting a bug report, consider whether it might be
120"out of scope." If it concerns a feature of documents that isn't
121representable in pandoc's Markdown, then it very likely is.
122(If in doubt, you can always ask on [pandoc-discuss].)
124Fixing bugs from the issue tracker
125----------------------------------
127Almost all the bugs on the issue tracker have one or more associated
128tags. These are used to indicate the *complexity* and *nature* of a
129bug. There is not yet a way to indicate priority. An up to date
130summary of issues can be found on [GitHub labels].
132* [good first issue] — The perfect starting point for new contributors. The
133 issue is generic and can be resolved without deep knowledge of the code
134 base.
135* [enhancement] — A feature which would be desirable. We recommend
136 you discuss any proposed enhancement on pandoc-discuss before
137 writing code.
138* [bug] — A problem which needs to be fixed.
139* [complexity:low] — The fix should only be a couple of lines.
140* [complexity:high] — The fix might require structural changes or in depth
141 knowledge of the code base.
142* [new:reader] — A request to add a new input format.
143* [new:writer] — A request to add a new output format.
144* [docs] — A discrepancy, or ambiguity in the documentation.
145* [status:in-progress] — Someone is actively working on or planning to work on the
146 ticket.
147* [status:more-discussion-needed] — It is unclear what the correct approach
148 to solving the ticket is. Before starting on tickets such as this it
149 would be advisable to post on the ticket.
150* [status:more-info-needed] — We require more information from a user before
151 we can classify a report properly.
153Issues related to a specific format are tagged accordingly, e.g. feature request
154or bug reports related to Markdown are labelled with [format:markdown].
156Have an idea for a new feature?
157-------------------------------
159First, search [pandoc-discuss] and the issue tracker (both [open issues] *and*
160[closed issues]) to make sure that the idea has not been discussed before.
162Explain the rationale for the feature you're requesting. Why would this
163feature be useful? Consider also any possible drawbacks, including backwards
164compatibility, new library dependencies, and performance issues.
166Features are very rarely "implement and forget", as all code must be
167maintained. This is especially relevant for large or complex
168contributions. It is helpful to be sympathetic to that fact, and to
169communicate future plans and availability clearly.
171Any potential new feature is best discussed on [pandoc-discuss]
172before opening an issue.
174Patches and pull requests
175-------------------------
177Patches and pull requests are welcome. Before you put time into a nontrivial
178patch, it is a good idea to discuss it on [pandoc-discuss], especially if it is
179for a new feature (rather than fixing a bug).
181Please follow these guidelines:
1831. Each patch (commit) should make a single logical change (fix a bug, add
184 a feature, clean up some code, add documentation). Everything
185 related to that change should be included (including tests and
186 documentation), and nothing unrelated should be included.
1882. The first line of the commit message should be a short description
189 of the whole commit (ideally <= 50 characters). Then there should
190 be a blank line, followed by a more detailed description of the
191 change.
1933. Follow the stylistic conventions you find in the existing
194 pandoc code. Use spaces, not tabs, and wrap code to 80 columns.
195 Always include type signatures for top-level functions.
196 Consider installing [EditorConfig], this will help you to follow the
197 coding style prevalent in pandoc.
1994. Your code should compile without warnings (`-Wall` clean).
2015. Run the tests to make sure your code does not introduce new bugs.
202 (See below under [Tests](#tests).) All tests should pass.
2046. It is a good idea to add test cases for the bug you are fixing. (See
205 below under [Tests](#tests).) If you are adding a new writer or reader,
206 you must include tests.
2087. If you are adding a new feature, include updates to `MANUAL.txt`.
2108. All code must be released under the general license governing pandoc
211 (GPL v2).
2139. It is better not to introduce new dependencies. Dependencies on
214 external C libraries should especially be avoided.
21610. We aim for compatibility with ghc versions from 8.0 to the
217 latest release. All pull requests and commits are tested
218 automatically on GitHub Actions.
220Tests
221-----
223Tests can be run as follows:
225 cabal install --only-dependencies --enable-tests
226 cabal configure --enable-tests
227 cabal build
228 cabal test
230or, if you're using [stack],
232 stack setup
233 stack test
235The test program is `test/test-pandoc.hs`.
237To run particular tests (pattern-matching on their names), use
238the `-p` option:
240 cabal install pandoc --enable-tests
241 cabal test --test-options='-p markdown'
243Or with stack:
245 stack test --test-arguments='-p markdown'
247It is often helpful to add `-j4` (run tests in parallel) and
248`--hide-successes` (don't clutter output with successes) to the test
249arguments as well. Collecting all options in a `cabal.project.local`
250file in the project's root directory can help to keep `cabal`
251commands short. E.g.:
253 flags: +embed_data_files
254 tests: True
255 test-show-details: direct
256 test-options: -j4 --hide-successes
258If you add a new feature to pandoc, please add tests as well, following
259the pattern of the existing tests. The test suite code is in
260`test/test-pandoc.hs`. If you are adding a new reader or writer, it is
261probably easiest to add some data files to the `test` directory, and
262modify `test/Tests/Old.hs`. Otherwise, it is better to modify the module
263under the `test/Tests` hierarchy corresponding to the pandoc module you
264are changing. Alternatively, you may add a "command test" to
265the `/test/command/` hierarchy, following the pattern of the tests there.
266These test files should have a meaningful name, which can include the issue
267number and/or the feature that's being tested. For example, `5474-tables.md`
268refers to both issue and feature.
270You can rebuild the golden tests in `tests/` by passing
271`--accept` to the test script. (If you're using stack, `stack
272test --test-arguments "--accept"`; or `make TESTARGS=--accept`).
273Then check the changed golden files for accuracy, and
274commit the changes. For docx or pptx tests, open the files in Word
275or Powerpoint to ensure that they weren't corrupted and that
276they had the expected result, and mention the Word/Powerpoint
277version and OS in your commit comment.
279Code style
280----------
282Pandoc uses [hlint] to identify opportunities for code improvements
283like redundant brackets or unnecessary `Language` extensions.
284However, sometimes there are cases where there are good reasons to
285use code different from what hlint proposes. In these cases, the
286respective warning should be disabled in the file `.hlint.yaml`.
288There should be no errors when running `hlint .`; this is checked by
289the continuous integration (CI) setup. It is recommended that
290contributors check their code with a local hlint installation, but
291relying on the CI is fine, too.
293A good way to ensure no new warnings are introduced is to use a Git
294[pre-commit hook] which runs hlint on all updated Haskell files
295before creating a commit:
297 #!/bin/sh
298 git diff --diff-filter=MA --cached --name-only | grep '\.hs$' | \
299 xargs hlint --hint .hlint.yaml
301(If you are using GNU `xargs`, add the `-r` option immediately
302after `xargs`.)
304Saving this to `.git/hooks/pre-commit`, and making the script
305executable, will prevent accidental introduction of potentially
306problematic code.
308Benchmarks
309----------
311To run benchmarks with cabal:
313 cabal configure --enable-benchmarks
314 cabal build
315 cabal bench
317With stack:
319 stack bench
322Using the REPL
323--------------
325With a recent version of cabal, you can do `cabal repl` and get
326a ghci REPL for working with pandoc. With [stack], use
327`stack ghci`.
329We recommend using the following `.ghci` file (which can be
330placed in the source directory):
332 :set -fobject-code
333 :set -XTypeSynonymInstances
334 :set -XScopedTypeVariables
335 :set -XOverloadedStrings
337Profiling
338---------
340To diagnose a performance issue with parsing, first try using
341the `--trace` option. This will give you a record of when block
342parsers succeed, so you can spot backtracking issues.
344To use the GHC profiler with cabal:
346 cabal clean
347 cabal install --enable-library-profiling --enable-executable-profiling
348 pandoc +RTS -p -RTS [file]...
349 less pandoc.prof
351With stack:
353 stack clean
354 stack install --profile
355 pandoc +RTS -p -RTS [file]...
356 less pandoc.prof
358Templates
359---------
361The default templates live in `data/templates`, which is a git
362subtree linked to <https://github.com/jgm/pandoc-templates.git>.
363The purpose of maintaining a separate repository is to allow
364people to maintain variant templates as a fork.
366You can modify the templates and submit patches without worrying
367much about this: when these patches are merged, we will
368push them to the main templates repository by doing
370 git subtree push --prefix=data/templates templates master
372where `templates` is a remote pointing to the templates
373repository.
375The code
376--------
378Pandoc has a publicly accessible git repository on
379GitHub: <https://github.com/jgm/pandoc>. To get a local copy of the source:
381 git clone https://github.com/jgm/pandoc.git
383The source for the main pandoc program is `pandoc.hs`. The source for
384the pandoc library is in `src/`, the source for the tests is in
385`test/`, and the source for the benchmarks is in `benchmark/`.
387The modules `Text.Pandoc.Definition`, `Text.Pandoc.Builder`, and
388`Text.Pandoc.Generic` are in a separate library `pandoc-types`. The code can
389be found in <https://github.com/jgm/pandoc-types>.
391To build pandoc, you will need a working installation of the
392[Haskell platform].
394The library is structured as follows:
396 - `Text.Pandoc` is a top-level module that exports what is needed
397 by most users of the library. Any patches that add new readers
398 or writers will need to make changes here, too.
399 - `Text.Pandoc.Definition` (in `pandoc-types`) defines the types
400 used for representing a pandoc document.
401 - `Text.Pandoc.Builder` (in `pandoc-types`) provides functions for
402 building pandoc documents programmatically.
403 - `Text.Pandoc.Generics` (in `pandoc-types`) provides functions allowing
404 you to promote functions that operate on parts of pandoc documents
405 to functions that operate on whole pandoc documents, walking the
406 tree automatically.
407 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
408 are the writers.
409 - `Text.Pandoc.Biblio` is a utility module for formatting citations
410 using citeproc-hs.
411 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
412 cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
413 the preprocessor [hsb2hs].
414 - `Text.Pandoc.Highlighting` contains the interface to the
415 skylighting library, which is used for code syntax highlighting.
416 - `Text.Pandoc.ImageSize` is a utility module containing functions for
417 calculating image sizes from the contents of image files.
418 - `Text.Pandoc.MIME` contains functions for associating MIME types
419 with extensions.
420 - `Text.Pandoc.Options` defines reader and writer options.
421 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
422 LaTeX source.
423 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
424 - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
425 the needs of pandoc.
426 - `Text.Pandoc.SelfContained` contains functions for making an HTML
427 file "self-contained," by importing remotely linked images, CSS,
428 and JavaScript and turning them into `data:` URLs.
429 - `Text.Pandoc.Shared` is a grab-bag of shared utility functions.
430 - `Text.Pandoc.Writers.Shared` contains utilities used in writers only.
431 - `Text.Pandoc.Slides` contains functions for splitting a markdown document
432 into slides, using the conventions described in the MANUAL.
433 - `Text.Pandoc.Templates` defines pandoc's templating system.
434 - `Text.Pandoc.UTF8` contains functions for converting text to and from
435 UTF8 bytestrings (strict and lazy).
436 - `Text.Pandoc.Asciify` contains functions to derive ascii versions of
437 identifiers that use accented characters.
438 - `Text.Pandoc.UUID` contains functions for generating UUIDs.
439 - `Text.Pandoc.XML` contains functions for formatting XML.
441Lua filters
442-----------
444If you've written a useful pandoc [lua filter](lua-filters.html),
445you may want to consider submitting a pull request to the
446[lua-filters repository](https://github.com/pandoc/lua-filters).
448[open issues]: https://github.com/jgm/pandoc/issues
449[closed issues]: https://github.com/jgm/pandoc/issues?q=is%3Aissue+is%3Aclosed
450[latest released version]: https://github.com/jgm/pandoc/releases/latest
451[Nightly builds]: https://github.com/jgm/pandoc/actions?query=workflow%3ANightly
452[pandoc-discuss]: https://groups.google.com/group/pandoc-discuss
453[issue tracker]: https://github.com/jgm/pandoc/issues
454[User's Guide]: https://pandoc.org/MANUAL.html
455[FAQs]: https://pandoc.org/faqs.html
456[EditorConfig]: https://editorconfig.org/
457[Haskell platform]: https://www.haskell.org/platform/
458[hlint]: https://hackage.haskell.org/package/hlint
459[hsb2hs]: https://hackage.haskell.org/package/hsb2hs
460[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
461[GitHub labels]: https://github.com/jgm/pandoc/labels
462[good first issue]:https://github.com/jgm/pandoc/labels/good%20first%20issue
463[enhancement]: https://github.com/jgm/pandoc/labels/enhancement
464[bug]: https://github.com/jgm/pandoc/labels/bug
465[complexity:low]: https://github.com/jgm/pandoc/labels/complexity:low
466[complexity:high]: https://github.com/jgm/pandoc/labels/complexity:high
467[docs]: https://github.com/jgm/pandoc/labels/docs
468[format:markdown]: https://github.com/jgm/pandoc/labels/format:markdown
469[new:reader]: https://github.com/jgm/pandoc/labels/new:reader
470[new:writer]: https://github.com/jgm/pandoc/labels/new:writer
471[status:in-progress]: https://github.com/jgm/pandoc/labels/status:in-progress
472[status:more-discussion-needed]: https://github.com/jgm/pandoc/labels/status:more-discussion-needed
473[status:more-info-needed]: https://github.com/jgm/pandoc/labels/status:more-info-needed
474[stack]: https://github.com/commercialhaskell/stack