git.iko.soy/typeable/pandoc
on rc/2.11.2 help sign in
typeable/pandoc/CONTRIBUTING.md
403 lines 16 kB history raw
1Contributing to pandoc
2======================
4Have a question?
5----------------
7Ask on [pandoc-discuss].
10Found a bug?
11------------
13Bug reports are welcome! Please report all bugs on pandoc's GitHub
14[issue tracker].
16Before you submit a bug report, search the [open issues] *and* [closed issues]
17to make sure the issue hasn't come up before. Also, check the [User's Guide] and
18[FAQs] for anything relevant.
20Make sure you can reproduce the bug with the [latest released
21version] of pandoc---or, even better, the development version,
22since the bug may have been fixed since the last release.
23[Nightly builds] are available, so you don't need to compile
24from source to test against the development version.
25(To fetch a nightly, visit the link, click the topmost "Nightly"
26in the table, then choose your platform under "Artifacts." Note
27that you must be logged in with a GitHub account.)
29Your report should give detailed, *reproducible* instructions, including
31 * the pandoc version (check using `pandoc -v`)
32 * the exact command line used
33 * the exact input used
34 * the output received
35 * the output you expected instead
37A small test case (just a few lines) is ideal. If your input is large,
38try to whittle it down to a *minimum working example*.
40Out of scope?
41-------------
43A less than perfect conversion does not necessarily mean there's
44a bug in pandoc. Quoting from the MANUAL:
46> Because pandoc's intermediate representation of a document is less
47> expressive than many of the formats it converts between, one should
48> not expect perfect conversions between every format and every other.
49> Pandoc attempts to preserve the structural elements of a document, but
50> not formatting details such as margin size. And some document elements,
51> such as complex tables, may not fit into pandoc's simple document
52> model. While conversions from pandoc's Markdown to all formats aspire
53> to be perfect, conversions from formats more expressive than pandoc's
54> Markdown can be expected to be lossy.
56For example, both `docx` and `odt` formats can represent margin size, but
57because pandoc's internal document model does not contain a representation of
58margin size, this information will be lost on converting from docx
59to `odt`. (You can, however, customize margin size using `--reference-doc`.)
61So before submitting a bug report, consider whether it might be
62"out of scope." If it concerns a feature of documents that isn't
63representable in pandoc's Markdown, then it very likely is.
64(If in doubt, you can always ask on [pandoc-discuss].)
66Fixing bugs from the issue tracker
67----------------------------------
69Almost all the bugs on the issue tracker have one or more associated
70tags. These are used to indicate the *complexity* and *nature* of a
71bug. There is not yet a way to indicate priority. An up to date
72summary of issues can be found on [GitHub labels].
74* [good first issue] — The perfect starting point for new contributors. The
75 issue is generic and can be resolved without deep knowledge of the code
76 base.
77* [enhancement] — A feature which would be desirable. We recommend
78 you discuss any proposed enhancement on pandoc-discuss before
79 writing code.
80* [bug] — A problem which needs to be fixed.
81* [complexity:low] — The fix should only be a couple of lines.
82* [complexity:high] — The fix might require structural changes or in depth
83 knowledge of the code base.
84* [new:reader] — A request to add a new input format.
85* [new:writer] — A request to add a new output format.
86* [docs] — A discrepancy, or ambiguity in the documentation.
87* [status:in-progress] — Someone is actively working on or planning to work on the
88 ticket.
89* [status:more-discussion-needed] — It is unclear what the correct approach
90 to solving the ticket is. Before starting on tickets such as this it
91 would be advisable to post on the ticket.
92* [status:more-info-needed] — We require more information from a user before
93 we can classify a report properly.
95Issues related to a specific format are tagged accordingly, e.g. feature request
96or bug reports related to Markdown are labelled with [format:markdown].
98Have an idea for a new feature?
99-------------------------------
101First, search [pandoc-discuss] and the issue tracker (both [open issues] *and*
102[closed issues]) to make sure that the idea has not been discussed before.
104Explain the rationale for the feature you're requesting. Why would this
105feature be useful? Consider also any possible drawbacks, including backwards
106compatibility, new library dependencies, and performance issues.
108It is best to discuss a potential new feature on [pandoc-discuss]
109before opening an issue.
111Patches and pull requests
112-------------------------
114Patches and pull requests are welcome. Before you put time into a nontrivial
115patch, it is a good idea to discuss it on [pandoc-discuss], especially if it is
116for a new feature (rather than fixing a bug).
118Please follow these guidelines:
1201. Each patch (commit) should make a single logical change (fix a bug, add
121 a feature, clean up some code, add documentation). Everything
122 related to that change should be included (including tests and
123 documentation), and nothing unrelated should be included.
1252. The first line of the commit message should be a short description
126 of the whole commit (ideally <= 50 characters). Then there should
127 be a blank line, followed by a more detailed description of the
128 change.
1303. Follow the stylistic conventions you find in the existing
131 pandoc code. Use spaces, not tabs, and wrap code to 80 columns.
132 Always include type signatures for top-level functions.
133 Consider installing [EditorConfig], this will help you to follow the
134 coding style prevalent in pandoc.
1364. Your code should compile without warnings (`-Wall` clean).
1385. Run the tests to make sure your code does not introduce new bugs.
139 (See below under [Tests](#tests).) All tests should pass.
1416. It is a good idea to add test cases for the bug you are fixing. (See
142 below under [Tests](#tests).) If you are adding a new writer or reader,
143 you must include tests.
1457. If you are adding a new feature, include updates to `MANUAL.txt`.
1478. All code must be released under the general license governing pandoc
148 (GPL v2).
1509. It is better not to introduce new dependencies. Dependencies on
151 external C libraries should especially be avoided.
15310. We aim for compatibility with ghc versions from 8.0 to the
154 latest release. All pull requests and commits are tested
155 automatically on GitHub Actions.
157Tests
158-----
160Tests can be run as follows:
162 cabal install --only-dependencies --enable-tests
163 cabal configure --enable-tests
164 cabal build
165 cabal test
167or, if you're using [stack],
169 stack setup
170 stack test
172The test program is `test/test-pandoc.hs`.
174To run particular tests (pattern-matching on their names), use
175the `-p` option:
177 cabal install pandoc --enable-tests
178 cabal test --test-options='-p markdown'
180Or with stack:
182 stack test --test-arguments='-p markdown'
184It is often helpful to add `-j4` (run tests in parallel)
185and `--hide-successes` (don't clutter output with successes)
186to the test arguments as well.
188If you add a new feature to pandoc, please add tests as well, following
189the pattern of the existing tests. The test suite code is in
190`test/test-pandoc.hs`. If you are adding a new reader or writer, it is
191probably easiest to add some data files to the `test` directory, and
192modify `test/Tests/Old.hs`. Otherwise, it is better to modify the module
193under the `test/Tests` hierarchy corresponding to the pandoc module you
194are changing. Alternatively, you may add a "command test" to
195the `/test/command/` hierarchy, following the pattern of the tests there.
196These test files should have a meaningful name, which can include the issue
197number and/or the feature that's being tested. For example, `5474-tables.md`
198refers to both issue and feature.
200You can rebuild the golden tests in `tests/` by passing
201`--accept` to the test script. (If you're using stack, `stack
202test --test-arguments "--accept"`; or `make TESTARGS=--accept`).
203Then check the changed golden files for accuracy, and
204commit the changes. For docx or pptx tests, open the files in Word
205or Powerpoint to ensure that they weren't corrupted and that
206they had the expected result, and mention the Word/Powerpoint
207version and OS in your commit comment.
209Code style
210----------
212Pandoc uses [hlint] to identify opportunities for code improvements
213like redundant brackets or unnecessary `Language` extensions.
214However, sometimes there are cases where there are good reasons to
215use code different from what hlint proposes. In these cases, the
216respective warning should be disabled in the file `.hlint.yaml`.
218There should be no errors when running `hlint .`; this is checked by
219the continuous integration (CI) setup. It is recommended that
220contributors check their code with a local hlint installation, but
221relying on the CI is fine, too.
223A good way to ensure no new warnings are introduced is to use a Git
224[pre-commit hook] which runs hlint on all updated Haskell files
225before creating a commit:
227 #!/bin/sh
228 git diff --cached --name-only | grep '.hs$' | xargs hlint
230Saving this to `.git/hooks/pre-commit`, and making the script
231executable, will prevent accidental introduction of potentially
232problematic code.
234Benchmarks
235----------
237To run benchmarks with cabal:
239 cabal configure --enable-benchmarks
240 cabal build
241 cabal bench
243With stack:
245 stack bench
247You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
248to get some statistics on memory usage. (Eventually this should
249be incorporated into the benchmark suite.)
251Using the REPL
252--------------
254With a recent version of cabal, you can do `cabal repl` and get
255a ghci REPL for working with pandoc. With [stack], use
256`stack ghci`.
258We recommend using the following `.ghci` file (which can be
259placed in the source directory):
261 :set -fobject-code
262 :set -XTypeSynonymInstances
263 :set -XScopedTypeVariables
264 :set -XOverloadedStrings
266Profiling
267---------
269To diagnose a performance issue with parsing, first try using
270the `--trace` option. This will give you a record of when block
271parsers succeed, so you can spot backtracking issues.
273To use the GHC profiler with cabal:
275 cabal clean
276 cabal install --enable-library-profiling --enable-executable-profiling
277 pandoc +RTS -p -RTS [file]...
278 less pandoc.prof
280With stack:
282 stack clean
283 stack install --profile
284 pandoc +RTS -p -RTS [file]...
285 less pandoc.prof
287Templates
288---------
290The default templates live in `data/templates`, which is a git
291subtree linked to <https://github.com/jgm/pandoc-templates.git>.
292The purpose of maintaining a separate repository is to allow
293people to maintain variant templates as a fork.
295You can modify the templates and submit patches without worrying
296much about this: when these patches are merged, we will
297push them to the main templates repository by doing
299 git subtree push --prefix=data/templates templates master
301where `templates` is a remote pointing to the templates
302repository.
304The code
305--------
307Pandoc has a publicly accessible git repository on
308GitHub: <http://github.com/jgm/pandoc>. To get a local copy of the source:
310 git clone https://github.com/jgm/pandoc.git
312The source for the main pandoc program is `pandoc.hs`. The source for
313the pandoc library is in `src/`, the source for the tests is in
314`test/`, and the source for the benchmarks is in `benchmark/`.
316The modules `Text.Pandoc.Definition`, `Text.Pandoc.Builder`, and
317`Text.Pandoc.Generic` are in a separate library `pandoc-types`. The code can
318be found in <http://github.com/jgm/pandoc-types>.
320To build pandoc, you will need a working installation of the
321[Haskell platform].
323The library is structured as follows:
325 - `Text.Pandoc` is a top-level module that exports what is needed
326 by most users of the library. Any patches that add new readers
327 or writers will need to make changes here, too.
328 - `Text.Pandoc.Definition` (in `pandoc-types`) defines the types
329 used for representing a pandoc document.
330 - `Text.Pandoc.Builder` (in `pandoc-types`) provides functions for
331 building pandoc documents programmatically.
332 - `Text.Pandoc.Generics` (in `pandoc-types`) provides functions allowing
333 you to promote functions that operate on parts of pandoc documents
334 to functions that operate on whole pandoc documents, walking the
335 tree automatically.
336 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
337 are the writers.
338 - `Text.Pandoc.Biblio` is a utility module for formatting citations
339 using citeproc-hs.
340 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
341 cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
342 the preprocessor [hsb2hs].
343 - `Text.Pandoc.Highlighting` contains the interface to the
344 skylighting library, which is used for code syntax highlighting.
345 - `Text.Pandoc.ImageSize` is a utility module containing functions for
346 calculating image sizes from the contents of image files.
347 - `Text.Pandoc.MIME` contains functions for associating MIME types
348 with extensions.
349 - `Text.Pandoc.Options` defines reader and writer options.
350 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
351 LaTeX source.
352 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
353 - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
354 the needs of pandoc.
355 - `Text.Pandoc.SelfContained` contains functions for making an HTML
356 file "self-contained," by importing remotely linked images, CSS,
357 and JavaScript and turning them into `data:` URLs.
358 - `Text.Pandoc.Shared` is a grab-bag of shared utility functions.
359 - `Text.Pandoc.Writers.Shared` contains utilities used in writers only.
360 - `Text.Pandoc.Slides` contains functions for splitting a markdown document
361 into slides, using the conventions described in the MANUAL.
362 - `Text.Pandoc.Templates` defines pandoc's templating system.
363 - `Text.Pandoc.UTF8` contains functions for converting text to and from
364 UTF8 bytestrings (strict and lazy).
365 - `Text.Pandoc.Asciify` contains functions to derive ascii versions of
366 identifiers that use accented characters.
367 - `Text.Pandoc.UUID` contains functions for generating UUIDs.
368 - `Text.Pandoc.XML` contains functions for formatting XML.
370Lua filters
371-----------
373If you've written a useful pandoc [lua filter](lua-filters.html),
374you may want to consider submitting a pull request to the
375[lua-filters repository](https://github.com/pandoc/lua-filters).
377[open issues]: https://github.com/jgm/pandoc/issues
378[closed issues]: https://github.com/jgm/pandoc/issues?q=is%3Aissue+is%3Aclosed
379[latest released version]: https://github.com/jgm/pandoc/releases/latest
380[Nightly builds]: https://github.com/jgm/pandoc/actions?query=workflow%3ANightly
381[pandoc-discuss]: http://groups.google.com/group/pandoc-discuss
382[issue tracker]: https://github.com/jgm/pandoc/issues
383[User's Guide]: http://pandoc.org/MANUAL.html
384[FAQs]: http://pandoc.org/faqs.html
385[EditorConfig]: http://editorconfig.org/
386[Haskell platform]: http://www.haskell.org/platform/
387[hlint]: https://hackage.haskell.org/package/hlint
388[hsb2hs]: http://hackage.haskell.org/package/hsb2hs
389[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
390[GitHub labels]: https://github.com/jgm/pandoc/labels
391[good first issue]:https://github.com/jgm/pandoc/labels/good%20first%20issue
392[enhancement]: https://github.com/jgm/pandoc/labels/enhancement
393[bug]: https://github.com/jgm/pandoc/labels/bug
394[complexity:low]: https://github.com/jgm/pandoc/labels/complexity:low
395[complexity:high]: https://github.com/jgm/pandoc/labels/complexity:high
396[docs]: https://github.com/jgm/pandoc/labels/docs
397[format:markdown]: https://github.com/jgm/pandoc/labels/format:markdown
398[new:reader]: https://github.com/jgm/pandoc/labels/new:reader
399[new:writer]: https://github.com/jgm/pandoc/labels/new:writer
400[status:in-progress]: https://github.com/jgm/pandoc/labels/status:in-progress
401[status:more-discussion-needed]: https://github.com/jgm/pandoc/labels/status:more-discussion-needed
402[status:more-info-needed]: https://github.com/jgm/pandoc/labels/status:more-info-needed
403[stack]: https://github.com/commercialhaskell/stack