git.iko.soy/typeable/pandoc
on 2.9 help sign in
typeable/pandoc/CONTRIBUTING.md
371 lines 15 kB history raw
1Contributing to pandoc
2======================
4Have a question?
5----------------
7Ask on [pandoc-discuss].
10Found a bug?
11------------
13Bug reports are welcome! Please report all bugs on pandoc's github
14[issue tracker].
16Before you submit a bug report, search the [open issues] *and* [closed issues]
17to make sure the issue hasn't come up before. Also, check the [User's Guide] and
18[FAQs] for anything relevant.
20Make sure you can reproduce the bug with the [latest released version] of pandoc
21(or, even better, the [development version]).
23Your report should give detailed, *reproducible* instructions, including
25 * the pandoc version (check using `pandoc -v`)
26 * the exact command line used
27 * the exact input used
28 * the output received
29 * the output you expected instead
31A small test case (just a few lines) is ideal. If your input is large,
32try to whittle it down to a *minimum working example*.
34Out of scope?
35-------------
37A less than perfect conversion does not necessarily mean there's
38a bug in pandoc. Quoting from the MANUAL:
40> Because pandoc's intermediate representation of a document is less
41> expressive than many of the formats it converts between, one should
42> not expect perfect conversions between every format and every other.
43> Pandoc attempts to preserve the structural elements of a document, but
44> not formatting details such as margin size. And some document elements,
45> such as complex tables, may not fit into pandoc's simple document
46> model. While conversions from pandoc's Markdown to all formats aspire
47> to be perfect, conversions from formats more expressive than pandoc's
48> Markdown can be expected to be lossy.
50For example, both `docx` and `odt` formats can represent margin size, but
51because pandoc's internal document model does not contain a representation of
52margin size, this information will be lost on converting from docx
53to `odt`. (You can, however, customize margin size using `--reference-doc`.)
55So before submitting a bug report, consider whether it might be
56"out of scope." If it concerns a feature of documents that isn't
57representable in pandoc's Markdown, then it very likely is.
58(If in doubt, you can always ask on [pandoc-discuss].)
60Fixing bugs from the issue tracker
61----------------------------------
63Almost all the bugs on the issue tracker have one or more associated
64tags. These are used to indicate the *complexity* and *nature* of a
65bug. There is not yet a way to indicate priority. An up to date
66summary of issues can be found on [GitHub labels].
68* [good first issue] — The perfect starting point for new contributors. The
69 issue is generic and can be resolved without deep knowledge of the code
70 base.
71* [enhancement] — A feature which would be desirable. We recommend
72 you discuss any proposed enhancement on pandoc-discuss before
73 writing code.
74* [bug] — A problem which needs to be fixed.
75* [complexity:low] — The fix should only be a couple of lines.
76* [complexity:high] — The fix might require structural changes or in depth
77 knowledge of the code base.
78* [new:reader] — A request to add a new input format.
79* [new:writer] — A request to add a new output format.
80* [docs] — A discrepancy, or ambiguity in the documentation.
81* [status:in-progress] — Someone is actively working on or planning to work on the
82 ticket.
83* [status:more-discussion-needed] — It is unclear what the correct approach
84 to solving the ticket is. Before starting on tickets such as this it
85 would be advisable to post on the ticket.
86* [status:more-info-needed] — We require more information from a user before
87 we can classify a report properly.
89Issues related to a specific format are tagged accordingly, e.g. feature request
90or bug reports related to Markdown are labelled with [format:markdown].
92Have an idea for a new feature?
93-------------------------------
95First, search [pandoc-discuss] and the issue tracker (both [open issues] *and*
96[closed issues]) to make sure that the idea has not been discussed before.
98Explain the rationale for the feature you're requesting. Why would this
99feature be useful? Consider also any possible drawbacks, including backwards
100compatibility, new library dependencies, and performance issues.
102It is best to discuss a potential new feature on [pandoc-discuss]
103before opening an issue.
105Patches and pull requests
106-------------------------
108Patches and pull requests are welcome. Before you put time into a nontrivial
109patch, it is a good idea to discuss it on [pandoc-discuss], especially if it is
110for a new feature (rather than fixing a bug).
112Please follow these guidelines:
1141. Each patch (commit) should make a single logical change (fix a bug, add
115 a feature, clean up some code, add documentation). Everything
116 related to that change should be included (including tests and
117 documentation), and nothing unrelated should be included.
1192. The first line of the commit message should be a short description
120 of the whole commit (ideally <= 50 characters). Then there should
121 be a blank line, followed by a more detailed description of the
122 change.
1243. Follow the stylistic conventions you find in the existing
125 pandoc code. Use spaces, not tabs, and wrap code to 80 columns.
126 Always include type signatures for top-level functions.
127 Consider installing [EditorConfig], this will help you to follow the
128 coding style prevalent in pandoc.
1304. Your code should compile without warnings (`-Wall` clean).
1325. Run the tests to make sure your code does not introduce new bugs.
133 (See below under [Tests](#tests).) All tests should pass.
1356. It is a good idea to add test cases for the bug you are fixing. (See
136 below under [Tests](#tests).) If you are adding a new writer or reader,
137 you must include tests.
1397. If you are adding a new feature, include updates to `MANUAL.txt`.
1418. All code must be released under the general license governing pandoc
142 (GPL v2).
1449. It is better not to introduce new dependencies. Dependencies on
145 external C libraries should especially be avoided.
14710. We aim for compatibility with ghc versions from 8.0 to the
148 latest release. All pull requests and commits are tested
149 automatically on CircleCI.
151Tests
152-----
154Tests can be run as follows:
156 cabal install --only-dependencies --enable-tests
157 cabal configure --enable-tests
158 cabal build
159 cabal test
161or, if you're using [stack],
163 stack setup
164 stack test
166The test program is `test/test-pandoc.hs`.
168To run particular tests (pattern-matching on their names), use
169the `-p` option:
171 cabal install pandoc --enable-tests
172 cabal test --test-options='-p markdown'
174Or with stack:
176 stack test --test-arguments='-p markdown'
178It is often helpful to add `-j4` (run tests in parallel)
179and `--hide-successes` (don't clutter output with successes)
180to the test arguments as well.
182If you add a new feature to pandoc, please add tests as well, following
183the pattern of the existing tests. The test suite code is in
184`test/test-pandoc.hs`. If you are adding a new reader or writer, it is
185probably easiest to add some data files to the `test` directory, and
186modify `test/Tests/Old.hs`. Otherwise, it is better to modify the module
187under the `test/Tests` hierarchy corresponding to the pandoc module you
188are changing. Alternatively, you may add a "command test" to
189the `/test/command/` hierarchy, following the pattern of the tests there.
190These test files should have a meaningful name, which can include the issue
191number and/or the feature that's being tested. For example, `5474-tables.md`
192refers to both issue and feature.
194You can rebuild the golden tests in `tests/` by passing
195`--accept` to the test script. (If you're using stack, `stack
196test --test-arguments "--accept"`; or `make TESTARGS=--accept`).
197Then check the changed golden files for accuracy, and
198commit the changes. For docx or pptx tests, open the files in Word
199or Powerpoint to ensure that they weren't corrupted and that
200they had the expected result, and mention the Word/Powerpoint
201version and OS in your commit comment.
203Benchmarks
204----------
206To run benchmarks with cabal:
208 cabal configure --enable-benchmarks
209 cabal build
210 cabal bench
212With stack:
214 stack bench
216You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
217to get some statistics on memory usage. (Eventually this should
218be incorporated into the benchmark suite.)
220Using the REPL
221--------------
223With a recent version of cabal, you can do `cabal repl` and get
224a ghci REPL for working with pandoc. With [stack], use
225`stack ghci`.
227We recommend using the following `.ghci` file (which can be
228placed in the source directory):
230 :set -fobject-code
231 :set -XTypeSynonymInstances
232 :set -XScopedTypeVariables
233 :set -XOverloadedStrings
235Profiling
236---------
238To diagnose a performance issue with parsing, first try using
239the `--trace` option. This will give you a record of when block
240parsers succeed, so you can spot backtracking issues.
242To use the GHC profiler with cabal:
244 cabal clean
245 cabal install --enable-library-profiling --enable-executable-profiling
246 pandoc +RTS -p -RTS [file]...
247 less pandoc.prof
249With stack:
251 stack clean
252 stack install --profile
253 pandoc +RTS -p -RTS [file]...
254 less pandoc.prof
256Templates
257---------
259The default templates live in `data/templates`, which is a git
260subtree linked to <https://github.com/jgm/pandoc-templates.git>.
261The purpose of maintaining a separate repository is to allow
262people to maintain variant templates as a fork.
264You can modify the templates and submit patches without worrying
265much about this: when these patches are merged, we will
266push them to the main templates repository by doing
268 git subtree push --prefix=data/templates templates master
270where `templates` is a remote pointing to the templates
271repository.
273The code
274--------
276Pandoc has a publicly accessible git repository on
277github: <http://github.com/jgm/pandoc>. To get a local copy of the source:
279 git clone https://github.com/jgm/pandoc.git
281The source for the main pandoc program is `pandoc.hs`. The source for
282the pandoc library is in `src/`, the source for the tests is in
283`test/`, and the source for the benchmarks is in `benchmark/`.
285The modules `Text.Pandoc.Definition`, `Text.Pandoc.Builder`, and
286`Text.Pandoc.Generic` are in a separate library `pandoc-types`. The code can
287be found in <http://github.com/jgm/pandoc-types>.
289To build pandoc, you will need a working installation of the
290[Haskell platform].
292The library is structured as follows:
294 - `Text.Pandoc` is a top-level module that exports what is needed
295 by most users of the library. Any patches that add new readers
296 or writers will need to make changes here, too.
297 - `Text.Pandoc.Definition` (in `pandoc-types`) defines the types
298 used for representing a pandoc document.
299 - `Text.Pandoc.Builder` (in `pandoc-types`) provides functions for
300 building pandoc documents programmatically.
301 - `Text.Pandoc.Generics` (in `pandoc-types`) provides functions allowing
302 you to promote functions that operate on parts of pandoc documents
303 to functions that operate on whole pandoc documents, walking the
304 tree automatically.
305 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
306 are the writers.
307 - `Text.Pandoc.Biblio` is a utility module for formatting citations
308 using citeproc-hs.
309 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
310 cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
311 the preprocessor [hsb2hs].
312 - `Text.Pandoc.Highlighting` contains the interface to the
313 skylighting library, which is used for code syntax highlighting.
314 - `Text.Pandoc.ImageSize` is a utility module containing functions for
315 calculating image sizes from the contents of image files.
316 - `Text.Pandoc.MIME` contains functions for associating MIME types
317 with extensions.
318 - `Text.Pandoc.Options` defines reader and writer options.
319 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
320 LaTeX source.
321 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
322 - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
323 the needs of pandoc.
324 - `Text.Pandoc.SelfContained` contains functions for making an HTML
325 file "self-contained," by importing remotely linked images, CSS,
326 and JavaScript and turning them into `data:` URLs.
327 - `Text.Pandoc.Shared` is a grab-bag of shared utility functions.
328 - `Text.Pandoc.Writers.Shared` contains utilities used in writers only.
329 - `Text.Pandoc.Slides` contains functions for splitting a markdown document
330 into slides, using the conventions described in the MANUAL.
331 - `Text.Pandoc.Templates` defines pandoc's templating system.
332 - `Text.Pandoc.UTF8` contains functions for converting text to and from
333 UTF8 bytestrings (strict and lazy).
334 - `Text.Pandoc.Asciify` contains functions to derive ascii versions of
335 identifiers that use accented characters.
336 - `Text.Pandoc.UUID` contains functions for generating UUIDs.
337 - `Text.Pandoc.XML` contains functions for formatting XML.
339Lua filters
340-----------
342If you've written a useful pandoc [lua filter](lua-filters.html),
343you may want to consider submitting a pull request to the
344[lua-filters repository](https://github.com/pandoc/lua-filters).
346[open issues]: https://github.com/jgm/pandoc/issues
347[closed issues]: https://github.com/jgm/pandoc/issues?q=is%3Aissue+is%3Aclosed
348[latest released version]: https://github.com/jgm/pandoc/releases/latest
349[development version]: https://github.com/pandoc-extras/pandoc-nightly/releases/latest
350[pandoc-discuss]: http://groups.google.com/group/pandoc-discuss
351[issue tracker]: https://github.com/jgm/pandoc/issues
352[User's Guide]: http://pandoc.org/MANUAL.html
353[FAQs]: http://pandoc.org/faqs.html
354[EditorConfig]: http://editorconfig.org/
355[Haskell platform]: http://www.haskell.org/platform/
356[hsb2hs]: http://hackage.haskell.org/package/hsb2hs
357[GitHub labels]: https://github.com/jgm/pandoc/labels
358[good first issue]:https://github.com/jgm/pandoc/labels/good%20first%20issue
359[enhancement]: https://github.com/jgm/pandoc/labels/enhancement
360[bug]: https://github.com/jgm/pandoc/labels/bug
361[complexity:low]: https://github.com/jgm/pandoc/labels/complexity:low
362[complexity:high]: https://github.com/jgm/pandoc/labels/complexity:high
363[docs]: https://github.com/jgm/pandoc/labels/docs
364[format:markdown]: https://github.com/jgm/pandoc/labels/format:markdown
365[new:reader]: https://github.com/jgm/pandoc/labels/new:reader
366[new:writer]: https://github.com/jgm/pandoc/labels/new:writer
367[status:in-progress]: https://github.com/jgm/pandoc/labels/status:in-progress
368[status:more-discussion-needed]: https://github.com/jgm/pandoc/labels/status:more-discussion-needed
369[status:more-info-needed]: https://github.com/jgm/pandoc/labels/status:more-info-needed
370[stack]: https://github.com/commercialhaskell/stack