git.iko.soy/typeable/pandoc
on master help sign in
366 files changed+8653−5638
since splitsnapshots
unifiedsplit
.circleci/config.yml+1−1 .editorconfig+8 .github/ISSUE_TEMPLATE−11 .github/ISSUE_TEMPLATE/bug_report.md+25 .github/ISSUE_TEMPLATE/config.yml+9 .github/ISSUE_TEMPLATE/feature_request.md+21 .github/workflows/ci.yml+21−6 .github/workflows/nightly.yml+9−2 .hlint.yaml+3−5 AUTHORS.md+14 CONTRIBUTING.md+16−11 INSTALL.md+4 MANUAL.txt+218−71 Makefile+28−15 RELEASE-CHECKLIST+1 SECURITY.md+12 benchmark/benchmark-pandoc.hs+13−12 cabal.project+9−8 changelog.md+795−8 data/docx/word/settings.xml+6−16 data/templates/article.jats_publishing+1−1 data/templates/default.context+7−3 data/templates/default.dzslides+1−1 data/templates/default.jats_articleauthoring+1−1 data/templates/default.latex+50−25 data/templates/default.revealjs+79−129 data/templates/default.s5+3 data/templates/default.slideous+7 data/templates/default.slidy+5 data/templates/styles.html+2−1 doc/epub.md+9−2 doc/faqs.md+128 doc/filters.md+2 doc/jats.md+9 doc/libraries.md+4 doc/lua-filters.md+2−2 doc/org.md+16−2 doc/using-the-pandoc-api.md+25−12 linux/make_artifacts.sh+19−7 man/pandoc.1+233−49 pandoc.cabal+40−30 src/Text/Pandoc/App.hs+40−21 src/Text/Pandoc/App/CommandLineOptions.hs+30−8 src/Text/Pandoc/App/FormatHeuristics.hs+13−7 src/Text/Pandoc/App/Opt.hs+145−57 src/Text/Pandoc/Asciify.hs+13−390 src/Text/Pandoc/BCP47.hs−112 src/Text/Pandoc/Citeproc.hs+75−56 src/Text/Pandoc/Citeproc/BibTeX.hs+25−15 src/Text/Pandoc/Citeproc/Data.hs+7−6 src/Text/Pandoc/Citeproc/Locator.hs+20−4 src/Text/Pandoc/Class/CommonState.hs+1−1 src/Text/Pandoc/Class/IO.hs+21−22 src/Text/Pandoc/Class/PandocMonad.hs+24−34 src/Text/Pandoc/Error.hs+23−16 src/Text/Pandoc/Extensions.hs+9−1 src/Text/Pandoc/Highlighting.hs+5−5 src/Text/Pandoc/ImageSize.hs+5−1 src/Text/Pandoc/Logging.hs+32−9 src/Text/Pandoc/Lua/Filter.hs+4−1 src/Text/Pandoc/Lua/Module/MediaBag.hs+3−3 src/Text/Pandoc/Lua/Module/Pandoc.hs+8−3 src/Text/Pandoc/Lua/Module/Utils.hs+1−1 src/Text/Pandoc/MIME.hs+10 src/Text/Pandoc/MediaBag.hs+45−14 src/Text/Pandoc/Options.hs+17−17 src/Text/Pandoc/PDF.hs+16−3 src/Text/Pandoc/Parsing.hs+360−242 src/Text/Pandoc/Readers.hs+55−48 src/Text/Pandoc/Readers/BibTeX.hs+14−8 src/Text/Pandoc/Readers/CSV.hs+7−7 src/Text/Pandoc/Readers/CommonMark.hs+46−4 src/Text/Pandoc/Readers/Creole.hs+5−6 src/Text/Pandoc/Readers/CslJson.hs+5−4 src/Text/Pandoc/Readers/DocBook.hs+30−15 src/Text/Pandoc/Readers/Docx.hs+59−42 src/Text/Pandoc/Readers/Docx/Combine.hs+5−5 src/Text/Pandoc/Readers/Docx/Parse.hs+121−21 src/Text/Pandoc/Readers/Docx/Util.hs+7 src/Text/Pandoc/Readers/DokuWiki.hs+9−8 src/Text/Pandoc/Readers/FB2.hs+6−3 src/Text/Pandoc/Readers/HTML.hs+48−28 src/Text/Pandoc/Readers/HTML/Table.hs+35−15 src/Text/Pandoc/Readers/Haddock.hs+11−7 src/Text/Pandoc/Readers/Ipynb.hs+5−3 src/Text/Pandoc/Readers/JATS.hs+13−6 src/Text/Pandoc/Readers/Jira.hs+13−9 src/Text/Pandoc/Readers/LaTeX.hs+35−38 src/Text/Pandoc/Readers/LaTeX/Citation.hs+1−1 src/Text/Pandoc/Readers/LaTeX/Lang.hs+122−119 src/Text/Pandoc/Readers/LaTeX/Macro.hs+45−14 src/Text/Pandoc/Readers/LaTeX/Parsing.hs+48−7 src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+140−66 src/Text/Pandoc/Readers/LaTeX/Table.hs+6−1 src/Text/Pandoc/Readers/LaTeX/Types.hs+14−1 src/Text/Pandoc/Readers/Man.hs+18−11 src/Text/Pandoc/Readers/Markdown.hs+93−73 src/Text/Pandoc/Readers/MediaWiki.hs+16−7 src/Text/Pandoc/Readers/Metadata.hs+44−17 src/Text/Pandoc/Readers/Muse.hs+10−9 src/Text/Pandoc/Readers/Native.hs+7−5 src/Text/Pandoc/Readers/OPML.hs+9−4 src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2 src/Text/Pandoc/Readers/Odt/ContentReader.hs+12−7 src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+2−2 src/Text/Pandoc/Readers/Odt/StyleReader.hs+8−7 src/Text/Pandoc/Readers/Org.hs+4−7 src/Text/Pandoc/Readers/Org/Blocks.hs+7−4 src/Text/Pandoc/Readers/Org/Inlines.hs+6−5 src/Text/Pandoc/Readers/Org/Parsing.hs+5−5 src/Text/Pandoc/Readers/RST.hs+78−110 src/Text/Pandoc/Readers/Roff.hs+16−18 src/Text/Pandoc/Readers/TWiki.hs+7−27 src/Text/Pandoc/Readers/Textile.hs+94−88 src/Text/Pandoc/Readers/TikiWiki.hs+6−6 src/Text/Pandoc/Readers/Txt2Tags.hs+47−21 src/Text/Pandoc/Readers/Vimwiki.hs+12−7 src/Text/Pandoc/SelfContained.hs+4−7 src/Text/Pandoc/Shared.hs+41−64 src/Text/Pandoc/Sources.hs+195 src/Text/Pandoc/Templates.hs+4 src/Text/Pandoc/Writers.hs+3−3 src/Text/Pandoc/Writers/AsciiDoc.hs+10−4 src/Text/Pandoc/Writers/BibTeX.hs+1−1 src/Text/Pandoc/Writers/ConTeXt.hs+44−38 src/Text/Pandoc/Writers/CslJson.hs+4−3 src/Text/Pandoc/Writers/Docbook.hs+50−7 src/Text/Pandoc/Writers/Docx.hs+60−244 src/Text/Pandoc/Writers/Docx/Table.hs+227 src/Text/Pandoc/Writers/Docx/Types.hs+185 src/Text/Pandoc/Writers/DokuWiki.hs+3−1 src/Text/Pandoc/Writers/EPUB.hs+3−3 src/Text/Pandoc/Writers/GridTable.hs+157 src/Text/Pandoc/Writers/HTML.hs+107−34 src/Text/Pandoc/Writers/Haddock.hs+13−2 src/Text/Pandoc/Writers/JATS.hs+67−44 src/Text/Pandoc/Writers/JATS/References.hs+3−2 src/Text/Pandoc/Writers/JATS/Table.hs+25−5 src/Text/Pandoc/Writers/JATS/Types.hs+12−3 src/Text/Pandoc/Writers/Jira.hs+41−18 src/Text/Pandoc/Writers/LaTeX.hs+37−12 src/Text/Pandoc/Writers/LaTeX/Caption.hs−1 src/Text/Pandoc/Writers/LaTeX/Lang.hs+61−60 src/Text/Pandoc/Writers/LaTeX/Table.hs+21−8 src/Text/Pandoc/Writers/LaTeX/Types.hs+2−1 src/Text/Pandoc/Writers/LaTeX/Util.hs+4−3 src/Text/Pandoc/Writers/Man.hs+2−2 src/Text/Pandoc/Writers/Markdown.hs+54−56 src/Text/Pandoc/Writers/Markdown/Inline.hs+36−21 src/Text/Pandoc/Writers/Ms.hs+24−8 src/Text/Pandoc/Writers/Muse.hs+6−3 src/Text/Pandoc/Writers/ODT.hs+4−3 src/Text/Pandoc/Writers/OpenDocument.hs+50−54 src/Text/Pandoc/Writers/Org.hs+24−14 src/Text/Pandoc/Writers/Powerpoint.hs+1−1 src/Text/Pandoc/Writers/Powerpoint/Output.hs+43−31 src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+2−3 src/Text/Pandoc/Writers/RST.hs+14−10 src/Text/Pandoc/Writers/RTF.hs+16−10 src/Text/Pandoc/Writers/Shared.hs+18−3 src/Text/Pandoc/Writers/TEI.hs+4−3 src/Text/Pandoc/Writers/Texinfo.hs+22−15 src/Text/Pandoc/Writers/ZimWiki.hs+4−2 src/Text/Pandoc/XML.hs+28−2 src/Text/Pandoc/XML/Light/Proc.hs−1 src/Text/Pandoc/XML/Light/Types.hs+4−4 stack.yaml+11−6 test/Tests/Command.hs+7−14 test/Tests/Helpers.hs+23 test/Tests/Old.hs+3−12 test/Tests/Readers/Docx.hs+25−5 test/Tests/Readers/EPUB.hs+1−1 test/Tests/Readers/HTML.hs+20−5 test/Tests/Readers/Jira.hs+14 test/Tests/Readers/Markdown.hs+3−3 test/Tests/Readers/Odt.hs+1 test/Tests/Writers/ConTeXt.hs+122−113 test/Tests/Writers/Docbook.hs+43−6 test/Tests/Writers/Docx.hs+5 test/Tests/Writers/HTML.hs+57−52 test/Tests/Writers/JATS.hs+135−91 test/Tests/Writers/Jira.hs+52 test/Tests/Writers/OOXML.hs+3−1 test/command/1279.md+1−1 test/command/2103.md+8 test/command/2874.md+2−2 test/command/3752.md+60 test/command/3792.md+1−1 test/command/4063.md+29 test/command/4164.md+1−1 test/command/4442.md+1−1 test/command/4465.md+15 test/command/5304.md+1−1 test/command/5367.md+6−2 test/command/5519.md+1−1 test/command/6026.md+19 test/command/6541.md+27 test/command/6620.md+3 test/command/6658.md+75 test/command/6709.md+11 test/command/6890.md+2−2 test/command/6925.md+2−2 test/command/6958.md+10 test/command/7041.md+23 test/command/7099.md+1−1 test/command/7129.md+27 test/command/7132.md+10 test/command/7134.md+16 test/command/7145.md+12 test/command/7155.md+15 test/command/7173.md+137 test/command/7208.md+6 test/command/7214.md+28 test/command/7216.md+19 test/command/7266.md+10 test/command/7272.md+27 test/command/7278.md+38 test/command/7282.md+7 test/command/7288.md+40 test/command/7299.md+23 test/command/7321.md+24 test/command/7323.md+29 test/command/7324.md+25 test/command/7339.md+11 test/command/7340.md+6 test/command/7376.md+16 test/command/7394.md+85 test/command/7397.md+14 test/command/7400.md+9 test/command/7416.md+19 test/command/averroes.bib+95 test/command/biblatex-basic.md+5−3 test/command/biblatex-examples.bib−1674 test/command/biblatex-inproceedings.md+1−1 test/command/biblatex-jaffe.md+2−2 test/command/biblatex-moraux.md+1−1 test/command/biblatex-nussbaum.md−63 test/command/biblatex-quotes.md+1−1 test/command/biblatex-sigfridsson.md+1−1 test/command/biblatex-spiegelberg.md+2−2 test/command/biblatex-test-case-conversion.md+2−2 test/command/biblatex-textnormal.md+1−1 test/command/chap1/spider.png test/command/chap1/text.md+17 test/command/chap2/spider.png test/command/chap2/text.md+5 test/command/defaults-inheritance-2.md+1−1 test/command/duplicate_attributes.md+7 test/command/mmd-metadata.md+20 test/command/newif.md+55 test/command/pandoc-citeproc-119.md+1−1 test/command/pandoc-citeproc-320a.md+3−3 test/command/pandoc-citeproc-322.md+1−1 test/command/pandoc-citeproc-356.md+2−7 test/command/pandoc-citeproc-65.md+1−1 test/command/pandoc-citeproc-70.md+2−2 test/command/pandoc-citeproc-chicago-author-date.md+7−7 test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+6−7 test/command/svg.md+1−1 test/command/toc.md+2−2 test/command/unicode-collation.md+130 test/docbook-reader.docbook+3−1 test/docx/0_level_headers.native+2−2 test/docx/golden/block_quotes.docx test/docx/golden/codeblock.docx test/docx/golden/comments.docx test/docx/golden/custom_style_no_reference.docx test/docx/golden/custom_style_preserve.docx test/docx/golden/custom_style_reference.docx test/docx/golden/definition_list.docx test/docx/golden/document-properties-short-desc.docx test/docx/golden/document-properties.docx test/docx/golden/headers.docx test/docx/golden/image.docx test/docx/golden/inline_code.docx test/docx/golden/inline_formatting.docx test/docx/golden/inline_images.docx test/docx/golden/link_in_notes.docx test/docx/golden/links.docx test/docx/golden/lists.docx test/docx/golden/lists_continuing.docx test/docx/golden/lists_multiple_initial.docx test/docx/golden/lists_restarting.docx test/docx/golden/nested_anchors_in_header.docx test/docx/golden/notes.docx test/docx/golden/raw-blocks.docx test/docx/golden/raw-bookmarks.docx test/docx/golden/table_one_row.docx test/docx/golden/table_with_list_cell.docx test/docx/golden/tables-default-widths.docx test/docx/golden/tables.docx test/docx/golden/track_changes_deletion.docx test/docx/golden/track_changes_insertion.docx test/docx/golden/track_changes_move.docx test/docx/golden/track_changes_scrubbed_metadata.docx test/docx/golden/unicode.docx test/docx/golden/verbatim_subsuper.docx test/docx/image_vml_as_object.docx test/docx/image_vml_as_object.native+2 test/docx/sdt_elements.native+9−10 test/docx/table_captions_no_field.docx test/docx/table_captions_no_field.native+34 test/docx/table_captions_with_field.docx test/docx/table_captions_with_field.native+54 test/docx/table_header_rowspan.docx test/docx/table_header_rowspan.native+189 test/docx/table_one_header_row.docx test/docx/table_one_header_row.native+18 test/docx/table_one_row.docx test/docx/table_one_row.native+4−4 test/docx/table_variable_width.native+12−17 test/docx/table_with_list_cell.native+3−3 test/docx/tables-default-widths.native+92 test/docx/tables.native+9−9 test/epub/features.native+2−2 test/epub/wasteland.epub test/fb2/images.fb2+1−1 test/fb2/test.jpg test/html-reader.native+2−1 test/jira-reader.native+1−1 test/lhs-test.html+2−1 test/lhs-test.html+lhs+2−1 test/odt/native/tab.native+1 test/odt/odt/tab.odt test/pipe-tables.native+2−10 test/pptx/code-custom_templated.pptx test/pptx/code_templated.pptx test/pptx/document-properties-short-desc_templated.pptx test/pptx/document-properties_templated.pptx test/pptx/endnotes_templated.pptx test/pptx/endnotes_toc_templated.pptx test/pptx/images_templated.pptx test/pptx/inline_formatting_templated.pptx test/pptx/lists_templated.pptx test/pptx/raw_ooxml_templated.pptx test/pptx/reference_depth.pptx test/pptx/remove_empty_slides_templated.pptx test/pptx/slide_breaks_slide_level_1_templated.pptx test/pptx/slide_breaks_templated.pptx test/pptx/slide_breaks_toc_templated.pptx test/pptx/speaker_notes_after_metadata_templated.pptx test/pptx/speaker_notes_afterheader_templated.pptx test/pptx/speaker_notes_afterseps_templated.pptx test/pptx/speaker_notes_templated.pptx test/pptx/start_numbering_at_templated.pptx test/pptx/tables_templated.pptx test/pptx/two_column_templated.pptx test/tables.latex+27−3 test/tables.ms+30 test/tables/nordics.latex+22−4 test/tables/planets.latex+4−4 test/tables/students.latex+10−2 test/test-pandoc.hs+1−2 test/writer.context+42−38 test/writer.html4+2−1 test/writer.html5+2−1 test/writer.jats_archiving+37−47 test/writer.jats_articleauthoring+41−49 test/writer.jats_publishing+37−47 test/writer.jira+23−35 test/writer.ms+1−1 test/writer.opml+1−1 test/writer.org+7−7 test/writer.zimwiki+3−3 test/writers-lang-and-dir.context+4 tools/build-arm.sh+110
.circleci/config.yml+1−1
@@ -2,7 +2,7 @@ version: "2.1"
22jobs:
33 build:
44 macos:
5− xcode: 9.4.1
5+ xcode: 11.1.0
66 environment:
77 HOMEBREW_NO_AUTO_UPDATE: 1
88 steps:
.editorconfig+8
@@ -8,6 +8,14 @@ indent_size = 2
88insert_final_newline = true
99trim_trailing_whitespace = true
1010
11+[*.{zip,docx}]
12+charset =
13+end_of_line =
14+indent_style =
15+indent_size =
16+insert_final_newline =
17+trim_trailing_whitespace =
18+
1119[*.{markdown,md}]
1220trim_trailing_whitespace = false
1321
.github/ISSUE_TEMPLATEdeleted−11
@@ -1,11 +0,0 @@
1−Asking a question?
2−Please use the pandoc-discuss mailing list instead:
3−https://groups.google.com/forum/#!forum/pandoc-discuss
4−This tracker is for bug reports and enhancement requests.
5−
6−Reporting a bug?
7−Please include your pandoc version and an example that allows us
8−to reproduce the problem (complete input, exact command line
9−used, expected and actual output). The smaller the example, the
10−better.
11−
.github/ISSUE_TEMPLATE/bug_report.mdnew file+25
@@ -0,0 +1,25 @@
1+---
2+name: Bug report
3+about: Create a report to help us improve
4+title: ''
5+labels: 'bug'
6+assignees: ''
7+
8+---
9+
10+<!--
11+Thank you for reporting an issue! Before you continue, please make sure that you have
12+
13+- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
14+- searched the pandoc-discuss mailing list for solutions: https://groups.google.com/forum/#!forum/pandoc-discuss
15+- reproduced your issue on the latest release of pandoc, or on https://pandoc.org/try
16+
17+Note that this bug tracker is for reporting bugs, not asking questions. For questions, use the pandoc-discuss mailing list.
18+-->
19+
20+**Explain the problem.**
21+Include the exact command line you used and all inputs necessary to reproduce the issue. Please create as minimal an example as possible, to help the maintainers isolate the problem. Explain the output you received and how it differs from what you expected.
22+
23+**Pandoc version?**
24+What version of pandoc are you using, on what OS?
25+
.github/ISSUE_TEMPLATE/config.ymlnew file+9
@@ -0,0 +1,9 @@
1+blank_issues_enabled: false
2+contact_links:
3+ - name: Ask a question
4+ url: https://groups.google.com/forum/#!forum/pandoc-discuss
5+ about: "Discussion forum for pandoc (may be used as an email list)"
6+ - name: Contributing Guidelines
7+ url: https://pandoc.org/CONTRIBUTING.html
8+ about: "Guidelines for submitting bug reports and code improvements"
9+
.github/ISSUE_TEMPLATE/feature_request.mdnew file+21
@@ -0,0 +1,21 @@
1+---
2+name: Feature request
3+about: Suggest an idea for this project
4+title: ''
5+labels: 'enhancement'
6+assignees: ''
7+
8+---
9+
10+<!--
11+Thank you for suggesting a feature! Before you continue, please make sure that you have
12+
13+- read the contributing guidelines: https://pandoc.org/CONTRIBUTING.html
14+- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
15+- searched the pandoc-discuss mailing list for relevant discussions: https://groups.google.com/forum/#!forum/pandoc-discuss
16+-->
17+
18+**Describe your proposed improvement and the problem it solves.**
19+
20+**Describe alternatives you've considered.**
21+
.github/workflows/ci.yml+21−6
@@ -44,32 +44,47 @@ jobs:
4444 matrix:
4545 versions:
4646 - ghc: '8.0.2'
47− cabal: '2.2'
48− prefix: 'new-'
47+ cabal: '2.4'
48+ prefix: 'v2-'
49+ cabalopts: ''
4950 testopts: ''
5051 - ghc: '8.2.2'
5152 cabal: '2.4'
5253 prefix: 'v2-'
54+ cabalopts: ''
5355 testopts: ''
5456 - ghc: '8.4.4'
5557 cabal: '2.4'
5658 prefix: 'v2-'
59+ cabalopts: '-f-embed_data_files'
5760 testopts: ''
5861 - ghc: '8.6.5'
5962 cabal: '3.2'
6063 prefix: ''
64+ cabalopts: ''
6165 testopts: '--test-option=--hide-successes'
6266 - ghc: '8.8.4'
6367 cabal: '3.2'
6468 prefix: ''
69+ cabalopts: ''
6570 testopts: '--test-option=--hide-successes'
66− - ghc: '8.10.2'
71+ - ghc: '8.10.4'
6772 cabal: '3.2'
6873 prefix: ''
74+ cabalopts: '-ftrypandoc'
75+ testopts: '--test-option=--hide-successes'
76+ - ghc: '9.0.1'
77+ cabal: '3.4'
78+ prefix: ''
79+ cabalopts: ''
6980 testopts: '--test-option=--hide-successes'
7081 steps:
7182 - uses: actions/checkout@v2
7283
84+ # needed by memory
85+ - name: Install numa
86+ run: sudo apt-get install libnuma-dev
87+
7388 # need to install older cabal/ghc versions from ppa repository
7489
7590 - name: Install recent cabal/ghc
@@ -101,14 +116,14 @@ jobs:
101116 - name: Install dependencies
102117 run: |
103118 cabal ${{ matrix.versions.prefix }}update
104− cabal ${{ matrix.versions.prefix }}build --dependencies-only --enable-tests --disable-optimization
119+ cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --dependencies-only --enable-tests --disable-optimization
105120
106121 - name: Build and test
107122 run: |
108− cabal ${{ matrix.versions.prefix }}build --enable-tests --disable-optimization 2>&1 | tee build.log
123+ cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --enable-tests --disable-optimization 2>&1 | tee build.log
109124 # fail if warnings in local build
110125 ! grep -q ": *[Ww]arning:" build.log || exit 1
111− cabal ${{ matrix.versions.prefix }}test --disable-optimization ${{ matrix.versions.testopts }}
126+ cabal ${{ matrix.versions.prefix }}test ${{ matrix.versions.cabalopts }} --disable-optimization ${{ matrix.versions.testopts }}
112127
113128 windows:
114129
.github/workflows/nightly.yml+9−2
@@ -10,15 +10,22 @@ jobs:
1010 runs-on: ubuntu-18.04
1111 steps:
1212 - uses: actions/checkout@v2
13+
14+ - name: Install recent cabal/ghc
15+ uses: haskell/actions/setup@v1
16+ with:
17+ ghc-version: '8.10.3'
18+ cabal-version: '3.2'
19+
1320 - name: Install dependencies
1421 run: |
1522 ghc --version
1623 cabal --version
1724 cabal v2-update
18− cabal v2-build --dependencies-only all
25+ cabal v2-build --dependencies-only
1926 - name: Build
2027 run: |
21− cabal v2-install all
28+ cabal v2-install
2229 strip $HOME/.cabal/bin/pandoc
2330 - name: Install artifact
2431 run: |
.hlint.yaml+3−5
@@ -9,6 +9,7 @@
99# Ignore some builtin hints
1010#
1111- ignore: {name: "Avoid lambda"}
12+- ignore: {name: "Use bimap"}
1213- ignore: {name: "Eta reduce"}
1314- ignore: {name: "Evaluate"}
1415- ignore: {name: "Reduce duplication"} # TODO: could be more fine-grained
@@ -39,13 +40,10 @@
3940 name: "Redundant return"
4041 within: Text.Pandoc.Citeproc.BibTeX
4142
42−# TODO: check
43+# With recent hlint this rule has given false positives on TH
4344- ignore:
4445 name: "Redundant bracket"
45− within:
46− - Text.Pandoc.Citeproc
47− - Text.Pandoc.Citeproc.BibTeX
48− - Text.Pandoc.Citeproc.MetaValue
46+
4947
5048- ignore:
5149 name: "Use <$>"
AUTHORS.md+14
@@ -20,6 +20,7 @@
2020- Andreas Lööw
2121- Andrew Dunning
2222- Andy Morris
23+- Anti-Distinctlyminty
2324- Antoine Latter
2425- Antonio Terceiro
2526- Arata Mizuki
@@ -35,6 +36,7 @@
3536- Benoit Schweblin
3637- Bjorn Buckwalter
3738- Blake Eryx
39+- Bodigrim
3840- Bradley Kuhn
3941- Brent Yorgey
4042- Brian Leung
@@ -45,6 +47,7 @@
4547- Carlos Sosa
4648- Cécile Chemin
4749- Cédric Couralet
50+- Charanjit Singh
4851- Chris Black
4952- Christian Conkle
5053- Christian Despres
@@ -75,8 +78,10 @@
7578- Eigil Rischel
7679- Emanuel Evans
7780- Emerson Harkin
81+- Emily Bourke
7882- Emily Eisenberg
7983- Eric Kow
84+- Erik Rask
8085- Eric Seidel
8186- Eric Schrijver
8287- Ethan Riley
@@ -86,6 +91,7 @@
8691- Florian Beeres
8792- Florian Eitel
8893- Florian Klink
94+- Florian Kohrt
8995- Francesco Occhipinti
9096- François Gannaz
9197- Frederik Elwert
@@ -146,6 +152,7 @@
146152- Josef Svenningsson
147153- Joseph C. Sible
148154- Julien Cretel
155+- Julien Dutant
149156- Juliette Fourcot
150157- Juliusz Gonera
151158- Justin Bogner
@@ -228,6 +235,7 @@
228235- Raymond Ehlers
229236- Recai Oktaş
230237- Roland Hieber
238+- Roman Beránek
231239- RyanGlScott
232240- Salim B
233241- Sascha Wilde
@@ -245,6 +253,7 @@
245253- Stefan Dresselhaus
246254- Sumit Sahrawat
247255- Tarik Graba
256+- Tatiana Porras
248257- Thenaesh Elango
249258- Thomas Hodgson
250259- Thomas Weißschuh
@@ -278,6 +287,7 @@
278287- a-vrma
279288- andrebauer
280289- argent0
290+- badumont
281291- blmage
282292- bucklereed
283293- bumper314
@@ -293,8 +303,11 @@
293303- kaizshang91
294304- lux-lth
295305- lwolfsonkin
306+- mbrackeantidot
296307- nkalvi
297308- niszet
309+- nuew
310+- obcat
298311- oltolm
299312- oquechy
300313- quasicomputational
@@ -306,6 +319,7 @@
306319- shreevatsa.public
307320- takahashim
308321- taotieren
322+- tecosaur
309323- TEC
310324- tgkokk
311325- the-solipsist
CONTRIBUTING.md+16−11
@@ -244,9 +244,16 @@ Or with stack:
244244
245245 stack test --test-arguments='-p markdown'
246246
247−It is often helpful to add `-j4` (run tests in parallel)
248−and `--hide-successes` (don't clutter output with successes)
249−to the test arguments as well.
247+It is often helpful to add `-j4` (run tests in parallel) and
248+`--hide-successes` (don't clutter output with successes) to the test
249+arguments as well. Collecting all options in a `cabal.project.local`
250+file in the project's root directory can help to keep `cabal`
251+commands short. E.g.:
252+
253+ flags: +embed_data_files
254+ tests: True
255+ test-show-details: direct
256+ test-options: -j4 --hide-successes
250257
251258If you add a new feature to pandoc, please add tests as well, following
252259the pattern of the existing tests. The test suite code is in
@@ -311,9 +318,6 @@ With stack:
311318
312319 stack bench
313320
314−You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
315−to get some statistics on memory usage. (Eventually this should
316−be incorporated into the benchmark suite.)
317321
318322Using the REPL
319323--------------
@@ -402,11 +406,11 @@ The library is structured as follows:
402406 tree automatically.
403407 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
404408 are the writers.
405− - `Text.Pandoc.Biblio` is a utility module for formatting citations
406− using citeproc-hs.
409+ - `Text.Pandoc.Citeproc.*` contain the code for citation handling,
410+ including an interface to the [citeproc] library.
407411 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
408− cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
409− the preprocessor [hsb2hs].
412+ cabal flag is used.
413+ - `Text.Pandoc.Emoji` is a thin wrapper around [emojis].
410414 - `Text.Pandoc.Highlighting` contains the interface to the
411415 skylighting library, which is used for code syntax highlighting.
412416 - `Text.Pandoc.ImageSize` is a utility module containing functions for
@@ -417,7 +421,6 @@ The library is structured as follows:
417421 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
418422 LaTeX source.
419423 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
420− - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
421424 the needs of pandoc.
422425 - `Text.Pandoc.SelfContained` contains functions for making an HTML
423426 file "self-contained," by importing remotely linked images, CSS,
@@ -452,6 +455,8 @@ you may want to consider submitting a pull request to the
452455[EditorConfig]: https://editorconfig.org/
453456[Haskell platform]: https://www.haskell.org/platform/
454457[hlint]: https://hackage.haskell.org/package/hlint
458+[citeproc]: https://hackage.haskell.org/package/citeproc
459+[emojis]: https://hackage.haskell.org/package/emojis
455460[hsb2hs]: https://hackage.haskell.org/package/hsb2hs
456461[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
457462[GitHub labels]: https://github.com/jgm/pandoc/labels
INSTALL.md+4
@@ -34,6 +34,10 @@ without SVG support), [Python] (to use Pandoc filters), and
3434
3535 choco install rsvg-convert python miktex
3636
37+Using multiple installation methods can result in two separate
38+installations of pandoc; it is recommended to properly uninstall
39+pandoc before switching to an alternative installation method.
40+
3741By default, Pandoc creates PDFs using LaTeX.
3842We recommend installing it via [MiKTeX].
3943
MANUAL.txt+218−71
@@ -1,7 +1,7 @@
11---
22title: Pandoc User's Guide
33author: John MacFarlane
4−date: March 4, 2021
4+date: June 20, 2021
55---
66
77# Synopsis
@@ -361,15 +361,15 @@ header when requesting a document from a URL:
361361 will be used. On \*nix and macOS systems this will be the `pandoc`
362362 subdirectory of the XDG data directory (by default,
363363 `$HOME/.local/share`, overridable by setting the `XDG_DATA_HOME`
364− environment variable). If that directory does not exist,
365− `$HOME/.pandoc` will be used (for backwards compatibility).
366− In Windows the default user data directory is
364+ environment variable). If that directory does not exist and
365+ `$HOME/.pandoc` exists, it will be used (for backwards compatibility).
366+ On Windows the default user data directory is
367367 `C:\Users\USERNAME\AppData\Roaming\pandoc`.
368368 You can find the default user data directory on your system by
369369 looking at the output of `pandoc --version`.
370− A `reference.odt`, `reference.docx`, `epub.css`, `templates`,
371− `slidy`, `slideous`, or `s5` directory
372− placed in this directory will override pandoc's normal defaults.
370+ Data files placed in this directory (for example, `reference.odt`,
371+ `reference.docx`, `epub.css`, `templates`) will override
372+ pandoc's normal defaults.
373373
374374`-d` *FILE*, `--defaults=`*FILE*
375375
@@ -675,12 +675,12 @@ header when requesting a document from a URL:
675675: Extract images and other media contained in or linked from
676676 the source document to the path *DIR*, creating it if
677677 necessary, and adjust the images references in the document
678− so they point to the extracted files. If the source format is
679− a binary container (docx, epub, or odt), the media is
680− extracted from the container and the original
681− filenames are used. Otherwise the media is read from the
682− file system or downloaded, and new filenames are constructed
683− based on SHA1 hashes of the contents.
678+ so they point to the extracted files. Media are downloaded,
679+ read from the file system, or extracted from a binary
680+ container (e.g. docx), as needed. The original file paths
681+ are used if they are relative paths not containing `..`.
682+ Otherwise filenames are constructed from the SHA1 hash of
683+ the contents.
684684
685685`--abbreviations=`*FILE*
686686
@@ -691,9 +691,9 @@ header when requesting a document from a URL:
691691 system default, use
692692 `pandoc --print-default-data-file=abbreviations`. The only
693693 use pandoc makes of this list is in the Markdown reader.
694− Strings ending in a period that are found in this list will
695− be followed by a nonbreaking space, so that the period will
696− not produce sentence-ending space in formats like LaTeX.
694+ Strings found in this list will be followed by a nonbreaking
695+ space, and the period will not produce sentence-ending space
696+ in formats like LaTeX. The strings may not contain spaces.
697697
698698[`pandocfilters`]: https://github.com/jgm/pandocfilters
699699[PHP]: https://github.com/vinai/pandocfilters-php
@@ -891,6 +891,11 @@ header when requesting a document from a URL:
891891 will not be searched. For example:
892892 `--resource-path=.:test` will search the working directory
893893 and the `test` subdirectory, in that order.
894+ This option can be used repeatedly. Search path components
895+ that come later on the command line will be searched before
896+ those that come earlier, so
897+ `--resource-path foo:bar --resource-path baz:bim` is
898+ equivalent to `--resource-path baz:bim:foo:bar`.
894899
895900`--request-header=`*NAME*`:`*VAL*
896901
@@ -1410,13 +1415,12 @@ results only for basic math, usually you will want to use
14101415`--gladtex`
14111416
14121417: Enclose TeX math in `<eq>` tags in HTML output. The resulting HTML
1413− can then be processed by [GladTeX] to produce images of the typeset
1414− formulas and an HTML file with links to these images.
1415− So, the procedure is:
1418+ can then be processed by [GladTeX] to produce SVG images of the typeset
1419+ formulas and an HTML file with these images embedded.
14161420
14171421 pandoc -s --gladtex input.md -o myfile.htex
1418− gladtex -d myfile-images myfile.htex
1419− # produces myfile.html and images in myfile-images
1422+ gladtex -d image_dir myfile.htex
1423+ # produces myfile.html and images in image_dir
14201424
14211425[MathML]: https://www.w3.org/Math/
14221426[MathJax]: https://www.mathjax.org
@@ -1672,6 +1676,34 @@ one line:
16721676verbosity: INFO
16731677```
16741678
1679+In fields that expect a file path (or list of file paths), the
1680+following syntax may be used to interpolate environment variables:
1681+
1682+``` yaml
1683+csl: ${HOME}/mycsldir/special.csl
1684+```
1685+
1686+`${USERDATA}` may also be used; this will always resolve to the
1687+user data directory that is current when the defaults file is
1688+parsed, regardless of the setting of the environment
1689+variable `USERDATA`.
1690+
1691+`${.}` will resolve to the directory containing the default
1692+file itself. This allows you to refer to resources contained
1693+in that directory:
1694+
1695+``` yaml
1696+epub-cover-image: ${.}/cover.jpg
1697+epub-metadata: ${.}/meta.xml
1698+resource-path:
1699+- . # the working directory from which pandoc is run
1700+- ${.}/images # the images subdirectory of the directory
1701+ # containing this defaults file
1702+```
1703+
1704+This environment variable interpolation syntax *only* works in
1705+fields that expect file paths.
1706+
16751707Default files can be placed in the `defaults` subdirectory of
16761708the user data directory and used from any directory. For
16771709example, one could create a file specifying defaults for writing
@@ -2282,8 +2314,8 @@ math equations flush left using [YAML metadata](#layout) or with
22822314
22832315These affect HTML output when [producing slide shows with pandoc].
22842316
2285−All [reveal.js configuration options] are available as variables.
2286−To turn off boolean flags that default to true in reveal.js, use `0`.
2317+`institute`
2318+: author affiliations: can be a list when there are multiple authors
22872319
22882320`revealjs-url`
22892321: base URL for reveal.js documents (defaults to
@@ -2303,6 +2335,9 @@ To turn off boolean flags that default to true in reveal.js, use `0`.
23032335: additional attributes for the title slide of reveal.js slide shows.
23042336 See [background in reveal.js and beamer] for an example.
23052337
2338+All [reveal.js configuration options] are available as variables.
2339+To turn off boolean flags that default to true in reveal.js, use `0`.
2340+
23062341[reveal.js configuration options]: https://revealjs.com/config/
23072342
23082343### Variables for Beamer slides
@@ -3719,7 +3754,7 @@ definition:
37193754Note that space between items in a definition list is required.
37203755(A variant that loosens this requirement, but disallows "lazy"
37213756hard wrapping, can be activated with `compact_definition_lists`: see
3722−[Non-pandoc extensions], below.)
3757+[Non-default extensions], below.)
37233758
37243759[^3]: I have been influenced by the suggestions of [David
37253760 Wheeler](https://justatheory.com/2009/02/modest-markdown-proposal/).
@@ -4144,10 +4179,12 @@ option is used. All of the metadata will appear in a single block
41444179at the beginning of the document.
41454180
41464181Note that [YAML] escaping rules must be followed. Thus, for example,
4147−if a title contains a colon, it must be quoted. The pipe character
4148−(`|`) can be used to begin an indented block that will be interpreted
4149−literally, without need for escaping. This form is necessary
4150−when the field contains blank lines or block-level formatting:
4182+if a title contains a colon, it must be quoted, and if it contains a
4183+backslash escape, then it must be ensured that it is not treated as a
4184+[YAML escape sequence]. The pipe character (`|`) can be used to begin
4185+an indented block that will be interpreted literally, without need for
4186+escaping. This form is necessary when the field contains blank lines
4187+or block-level formatting:
41514188
41524189 ---
41534190 title: 'This is the title: it contains a colon'
@@ -4161,6 +4198,12 @@ when the field contains blank lines or block-level formatting:
41614198 It consists of two paragraphs.
41624199 ...
41634200
4201+The literal block after the `|` must be indented relative to the
4202+line containing the `|`. If it is not, the YAML will be invalid
4203+and pandoc will not interpret it as metadata. For an overview
4204+of the complex rules governing YAML, see the [Wikipedia entry on
4205+YAML syntax].
4206+
41644207Template variables will be set automatically from the metadata. Thus, for
41654208example, in writing HTML, the variable `abstract` will be set to the HTML
41664209equivalent of the Markdown in the `abstract` field:
@@ -4207,6 +4250,24 @@ will be interpreted as markdown. For example:
42074250 \renewcommand{\section}[1]{\clearpage\oldsection{#1}}
42084251 ```
42094252
4253+Note: the `yaml_metadata_block` extension works with
4254+`commonmark` as well as `markdown` (and it is enabled by default
4255+in `gfm` and `commonmark_x`). However, in these formats the
4256+following restrictions apply:
4257+
4258+- The YAML metadata block must occur at the beginning of the
4259+ document (and there can be only one). If multiple files are
4260+ given as arguments to pandoc, only the first can be a YAML
4261+ metadata block.
4262+
4263+- The leaf nodes of the YAML structure are parsed in isolation from
4264+ each other and from the rest of the document. So, for
4265+ example, you can't use a reference link in these contexts
4266+ if the link definition is somewhere else in the document.
4267+
4268+[YAML escape sequence]: https://yaml.org/spec/1.2/spec.html#id2776092
4269+[Wikipedia entry on YAML syntax]: https://en.m.wikipedia.org/wiki/YAML#Syntax
4270+
42104271## Backslash escapes
42114272
42124273#### Extension: `all_symbols_escapable` ####
@@ -4899,37 +4960,68 @@ Inline and regular footnotes may be mixed freely.
48994960
49004961#### Extension: `citations` ####
49014962
4902−Markdown citations go inside square brackets and are separated
4903−by semicolons. Each citation must have a key, composed of '@' +
4904−the citation identifier from the database, and may optionally
4905−have a prefix, a locator, and a suffix. The citation key must
4906−begin with a letter, digit, or `_`, and may contain
4907−alphanumerics, `_`, and internal punctuation characters
4908−(`:.#$%&-+?<>~/`). Here are some examples:
4909−
4910− Blah blah [see @doe99, pp. 33-35; also @smith04, chap. 1].
4911−
4912− Blah blah [@doe99, pp. 33-35, 38-39 and *passim*].
4913−
4914− Blah blah [@smith04; @doe99].
4915−
4916−`pandoc` detects locator terms in the [CSL locale files].
4917−Either abbreviated or unabbreviated forms are accepted. In the `en-US`
4918−locale, locator terms can be written in either singular or plural forms,
4919−as `book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
4920−`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`, `fol.`/`fols.`;
4921−`number`, `no.`/`nos.`; `line`, `l.`/`ll.`; `note`, `n.`/`nn.`; `opus`,
4922−`op.`/`opp.`; `page`, `p.`/`pp.`; `paragraph`, `para.`/`paras.`; `part`,
4923−`pt.`/`pts.`; `section`, `sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`;
4924−`verse`, `v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
4963+To cite a bibliographic item with an identifier foo, use the
4964+syntax `@foo`. Normal citations should be included in square
4965+brackets, with semicolons separating distinct items:
4966+
4967+ Blah blah [@doe99; @smith2000; @smith2004].
4968+
4969+How this is rendered depends on the citation style. In an
4970+author-date style, it might render as
4971+
4972+ Blah blah (Doe 1999, Smith 2000, 2004).
4973+
4974+In a footnote style, it might render as
4975+
4976+ Blah blah.[^1]
4977+
4978+ [^1]: John Doe, "Frogs," *Journal of Amphibians* 44 (1999);
4979+ Susan Smith, "Flies," *Journal of Insects* (2000);
4980+ Susan Smith, "Bees," *Journal of Insects* (2004).
4981+
4982+See the [CSL user documentation] for more information about CSL
4983+styles and how they affect rendering.
4984+
4985+Unless a citation key start with a letter, digit, or `_`,
4986+and contains only alphanumerics and internal punctuation
4987+characters (`:.#$%&-+?<>~/`), it must be surrounded
4988+by curly braces, which are not considered part of the key.
4989+In `@Foo_bar.baz.`, the key is `Foo_bar.baz`. The final
4990+period is not *internal* punctuation, so it is not included in
4991+the key. In `@{Foo_bar.baz.}`, the key is `Foo_bar.baz.`, including
4992+the final period. The curly braces are recommended if you use URLs as
4993+keys: `[@{https://example.com/bib?name=foobar&date=2000}, p. 33]`.
4994+
4995+Citation items may optionally include a prefix, a locator, and
4996+a suffix. In
4997+
4998+ Blah blah [see @doe99, pp. 33-35 and *passim*; @smith04, chap. 1].
4999+
5000+The first item (`doe99`) has prefix `see `, locator `pp. 33-35`,
5001+and suffix `and *passim*`. The second item (`smith04`) has
5002+locator `chap. 1` and no prefix or suffix.
5003+
5004+Pandoc uses some heuristics to separate the locator from the
5005+rest of the subject. It is sensitive to the locator terms
5006+defined in the [CSL locale files]. Either abbreviated or
5007+unabbreviated forms are accepted. In the `en-US` locale, locator
5008+terms can be written in either singular or plural forms, as
5009+`book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
5010+`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`,
5011+`fol.`/`fols.`; `number`, `no.`/`nos.`; `line`, `l.`/`ll.`;
5012+`note`, `n.`/`nn.`; `opus`, `op.`/`opp.`; `page`, `p.`/`pp.`;
5013+`paragraph`, `para.`/`paras.`; `part`, `pt.`/`pts.`; `section`,
5014+`sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`; `verse`,
5015+`v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
49255016locator term is used, "page" is assumed.
49265017
4927−`pandoc` will use heuristics to distinguish the locator
4928−from the suffix. In complex cases, the locator can be enclosed
4929−in curly braces:
5018+In complex cases, you can force something to be treated as
5019+a locator by enclosing it in curly braces or prevent parsing
5020+the suffix as locator by prepending curly braces:
49305021
49315022 [@smith{ii, A, D-Z}, with a suffix]
49325023 [@smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5024+ [@smith{}, 99 years later]
49335025
49345026A minus sign (`-`) before the `@` will suppress mention of
49355027the author in the citation. This can be useful when the
@@ -4937,13 +5029,26 @@ author is already mentioned in the text:
49375029
49385030 Smith says blah [-@smith04].
49395031
4940−You can also write an in-text citation, as follows:
5032+You can also write an author-in-text citation, by omitting the
5033+square brackets:
49415034
49425035 @smith04 says blah.
49435036
49445037 @smith04 [p. 33] says blah.
49455038
5039+This will cause the author's name to be rendered, followed by
5040+the bibliographical details. Use this form when you want to
5041+make the citation the subject of a sentence.
5042+
5043+When you are using a note style, it is usually better to let
5044+citeproc create the footnotes from citations rather than writing
5045+an explicit note. If you do write an explicit note that
5046+contains a citation, note that normal citations will be put in
5047+parentheses, while author-in-text citations will not. For
5048+this reason, it is sometimes preferable to use the
5049+author-in-text style inside notes when using a note style.
49465050
5051+[CSL user documentation]: https://citationstyles.org/authors/
49475052[CSL]: https://docs.citationstyles.org/en/stable/specification.html
49485053[CSL markup specs]: https://docs.citationstyles.org/en/1.0/release-notes.html#rich-text-markup-within-fields
49495054[Chicago Manual of Style]: https://chicagomanualofstyle.org
@@ -4952,13 +5057,44 @@ You can also write an in-text citation, as follows:
49525057[finding and editing styles]: https://citationstyles.org/authors/
49535058[CSL locale files]: https://github.com/citation-style-language/locales
49545059
4955−## Non-pandoc extensions
5060+## Non-default extensions
49565061
49575062The following Markdown syntax extensions are not enabled by default
49585063in pandoc, but may be enabled by adding `+EXTENSION` to the format
49595064name, where `EXTENSION` is the name of the extension. Thus, for
49605065example, `markdown+hard_line_breaks` is Markdown with hard line breaks.
49615066
5067+#### Extension: `rebase_relative_paths` ####
5068+
5069+Rewrite relative paths for Markdown links and images, depending
5070+on the path of the file containing the link or image link. For
5071+each link or image, pandoc will compute the directory of the
5072+containing file, relative to the working directory, and prepend
5073+the resulting path to the link or image path.
5074+
5075+The use of this extension is best understood by example.
5076+Suppose you have a a subdirectory for each chapter of a book,
5077+`chap1`, `chap2`, `chap3`. Each contains a file `text.md` and a
5078+number of images used in the chapter. You would like to have
5079+`![image](spider.jpg)` in `chap1/text.md` refer to
5080+`chap1/spider.jpg` and `![image](spider.jpg)` in `chap2/text.md`
5081+refer to `chap2/spider.jpg`. To do this, use
5082+
5083+ pandoc chap*/*.md -f markdown+rebase_relative_paths
5084+
5085+Without this extension, you would have to use
5086+`![image](chap1/spider.jpg)` in `chap1/text.md` and
5087+`![image](chap2/spider.jpg)` in `chap2/text.md`. Links with
5088+relative paths will be rewritten in the same way as images.
5089+
5090+Absolute paths and URLs are not changed. Neither are empty
5091+paths or paths consisting entirely of a fragment, e.g., `#foo`.
5092+
5093+Note that relative paths in reference links and images will
5094+be rewritten relative to the file containing the link
5095+reference definition, not the file containing the reference link
5096+or image itself, if these differ.
5097+
49625098#### Extension: `attributes` ####
49635099
49645100Allows attributes to be attached to any inline or block-level
@@ -5492,10 +5628,23 @@ A few other metadata fields affect bibliography formatting:
54925628
54935629`lang`
54945630: The `lang` field will affect how the style is localized,
5495− for example in the translation of labels and the use
5496− of quotation marks. (For backwards compatibility,
5497− `locale` may be used instead of `lang`, but this use
5498− is deprecated.)
5631+ for example in the translation of labels, the use
5632+ of quotation marks, and the way items are sorted.
5633+ (For backwards compatibility, `locale` may be used instead
5634+ of `lang`, but this use is deprecated.)
5635+
5636+ A BCP 47 language tag is expected: for example, `en`,
5637+ `de`, `en-US`, `fr-CA`, `ug-Cyrl`. The unicode extension
5638+ syntax (after `-u-`) may be used to specify options for
5639+ collation (sorting) more precisely. Here are some examples:
5640+
5641+ - `zh-u-co-pinyin` -- Chinese with the Pinyin collation.
5642+ - `es-u-co-trad` -- Spanish with the traditional collation
5643+ (with `Ch` sorting after `C`).
5644+ - `fr-u-kb` -- French with "backwards" accent sorting
5645+ (with `coté` sorting after `côte`).
5646+ - `en-US-u-kf-upper` -- English with uppercase letters sorting
5647+ before lower (default is lower before upper).
54995648
55005649`notes-after-punctuation`
55015650: If true (the default), pandoc will put footnote citations
@@ -5507,12 +5656,6 @@ A few other metadata fields affect bibliography formatting:
55075656 punctuation.
55085657
55095658
5510−
5511−
5512−
5513−
5514−
5515−
55165659# Slide shows
55175660
55185661You can use pandoc to produce an HTML + JavaScript slide presentation
@@ -5605,7 +5748,11 @@ rules:
56055748 * A heading at the slide level always starts a new slide.
56065749
56075750 * Headings *below* the slide level in the hierarchy create
5608− headings *within* a slide.
5751+ headings *within* a slide. (In beamer, a "block" will be
5752+ created. If the heading has the class `example`, an
5753+ `exampleblock` environment will be used; if it has the class
5754+ `alert`, an `alertblock` will be used; otherwise a regular
5755+ `block` will be used.)
56095756
56105757 * Headings *above* the slide level in the hierarchy create
56115758 "title slides," which just contain the section title
@@ -6183,7 +6330,7 @@ Then edit `my.theme` and use it like this:
61836330If you are not satisfied with the built-in highlighting, or you
61846331want highlight a language that isn't supported, you can use the
61856332`--syntax-definition` option to load a [KDE-style XML syntax definition
6186−file](https://docs.kde.org/stable5/en/applications/katepart/highlight.html).
6333+file](https://docs.kde.org/stable5/en/kate/katepart/highlight.html).
61876334Before writing your own, have a look at KDE's [repository of syntax
61886335definitions](https://github.com/KDE/syntax-highlighting/tree/master/data/syntax).
61896336
@@ -6357,9 +6504,9 @@ application, here are some things to keep in mind:
635765044. The HTML generated by pandoc is not guaranteed to be safe.
63586505 If `raw_html` is enabled for the Markdown input, users can
63596506 inject arbitrary HTML. Even if `raw_html` is disabled,
6360− users can include dangerous content in attributes for
6361− headings, spans, and code blocks. To be safe, you should
6362− run all the generated HTML through an HTML sanitizer.
6507+ users can include dangerous content in URLs and attributes.
6508+ To be safe, you should run all the generated HTML through
6509+ an HTML sanitizer.
63636510
63646511# Authors
63656512
Makefile+28−15
@@ -2,25 +2,31 @@ version?=$(shell grep '^[Vv]ersion:' pandoc.cabal | awk '{print $$2;}')
22pandoc=$(shell find dist -name pandoc -type f -exec ls -t {} \; | head -1)
33SOURCEFILES?=$(shell git ls-tree -r master --name-only | grep "\.hs$$")
44BRANCH?=master
5+ARCH=$(shell uname -m)
6+DOCKERIMAGE=registry.gitlab.b-data.ch/ghc/ghc4pandoc:8.10.4
57COMMIT=$(shell git rev-parse --short HEAD)
68TIMESTAMP=$(shell date "+%Y%m%d_%H%M")
7−LATESTBENCH=$(word 1,$(shell ls -t bench_*.csv))
8−ifeq ($(LATESTBENCH),)
9−BASELINE=
9+LATESTBENCH=$(word 1,$(shell ls -t bench_*.csv 2>/dev/null))
10+BASELINE?=$(LATESTBENCH)
11+ifeq ($(BASELINE),)
12+BASELINECMD=
1013else
11−BASELINE=--baseline $(LATESTBENCH)
14+BASELINECMD=--baseline $(BASELINE)
1215endif
13−GHCOPTS=-fdiagnostics-color=always
16+GHCOPTS=-fdiagnostics-color=always -j4 +RTS -A8m -RTS
1417WEBSITE=../../web/pandoc.org
1518REVISION?=1
16−BENCHARGS?=--timeout=6 +RTS -T -RTS $(if $(PATTERN),--pattern "$(PATTERN)",)
19+# For gauge:
20+# BENCHARGS?=--small --ci=0.90 --match=pattern $(PATTERN)
21+# For tasty-bench:
22+BENCHARGS?=--csv bench_$(TIMESTAMP).csv $(BASELINECMD) --timeout=6 +RTS -T -RTS $(if $(PATTERN),--pattern "$(PATTERN)",)
1723
1824quick:
19− stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='-j +RTS -A64m -RTS' --test-arguments='-j4 --hide-successes $(TESTARGS)'
25+ stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='$(GHCOPTS)' --test-arguments='-j4 --hide-successes $(TESTARGS)'
2026
2127quick-cabal:
2228 cabal new-configure . --ghc-options '$(GHCOPTS)' --disable-optimization --enable-tests
23− cabal new-build . --disable-optimization
29+ cabal new-build -j4 . --disable-optimization
2430 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
2531
2632full-cabal:
@@ -29,7 +35,7 @@ full-cabal:
2935 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
3036
3137full:
32− stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 -j4 $(GHCOPTS)'
38+ stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 $(GHCOPTS)'
3339
3440ghci:
3541 stack ghci --flag 'pandoc:embed_data_files'
@@ -45,11 +51,14 @@ test:
4551ghcid:
4652 ghcid -c "stack repl --flag 'pandoc:embed_data_files'"
4753
48−bench:
49− stack bench --benchmark-arguments='$(BENCHARGS) $(BASELINE) --csv bench_$(TIMESTAMP).csv' --ghc-options '$(GHCOPTS)'
54+ghcid-test:
55+ ghcid -c "stack repl --ghc-options=-XNoImplicitPrelude --flag 'pandoc:embed_data_files' --ghci-options=-fobject-code pandoc:lib pandoc:test-pandoc"
5056
51−weigh:
52− stack build --ghc-options '$(GHCOPTS)' pandoc:weigh-pandoc && stack exec weigh-pandoc
57+bench:
58+ stack bench \
59+ --ghc-options '$(GHCOPTS)' \
60+ --benchmark-arguments='$(BENCHARGS)' 2>&1 | \
61+ tee "bench_latest.txt"
5362
5463reformat:
5564 for f in $(SOURCEFILES); do echo $$f; stylish-haskell -i $$f ; done
@@ -81,10 +90,14 @@ checkdocs:
8190debpkg: man/pandoc.1
8291 docker run -v `pwd`:/mnt \
8392 -v `pwd`/linux/artifacts:/artifacts \
93+ --user $(id -u):$(id -g) \
8494 -e REVISION=$(REVISION) \
8595 -w /mnt \
86− utdemir/ghc-musl:v12-libgmp-ghc8101 bash \
87− /mnt/linux/make_artifacts.sh
96+ --memory=0 \
97+ --rm \
98+ $(DOCKERIMAGE) \
99+ bash \
100+ /mnt/linux/make_artifacts.sh 2>&1 > docker.log
88101
89102man/pandoc.1: MANUAL.txt man/pandoc.1.before man/pandoc.1.after
90103 pandoc $< -f markdown -t man -s \
RELEASE-CHECKLIST+1
@@ -4,6 +4,7 @@
44[ ] Finalize changelog
55[ ] push release candidate branch rc/VERSION and (if it builds
66 successfully) download artifacts
7+[ ] run tools/build-arm.sh to create and download arm64 linux package
78[ ] download macos artifacts from CircleCI
89[ ] Use 'make' in macos and windows artifacts to sign code
910[ ] make update-website
SECURITY.mdnew file+12
@@ -0,0 +1,12 @@
1+# Security Policy
2+
3+## Supported Versions
4+
5+Only the most recent version of pandoc is supported with security updates.
6+
7+## Reporting a Vulnerability
8+
9+To report a vulnerability, email the maintainer, jgm@berkeley.edu.
10+But first please read the section of the manual entitled
11+[A note on security](https://pandoc.org/MANUAL.html#a-note-on-security),
12+which describes some security guarantees pandoc does NOT make.
benchmark/benchmark-pandoc.hs+13−12
@@ -18,12 +18,13 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
1818-}
1919import Text.Pandoc
2020import Text.Pandoc.MIME
21−import Control.Monad (when)
21+import Control.DeepSeq (force)
2222import Control.Monad.Except (throwError)
2323import qualified Text.Pandoc.UTF8 as UTF8
2424import qualified Data.ByteString as B
2525import qualified Data.Text as T
2626import Test.Tasty.Bench
27+-- import Gauge
2728import qualified Data.ByteString.Lazy as BL
2829import Data.Maybe (mapMaybe)
2930import Data.List (sortOn)
@@ -31,10 +32,10 @@ import Data.List (sortOn)
3132readerBench :: Pandoc
3233 -> T.Text
3334 -> Maybe Benchmark
35+readerBench _ name
36+ | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
3437readerBench doc name = either (const Nothing) Just $
3538 runPure $ do
36− when (name `elem` ["bibtex", "biblatex", "csljson"]) $
37− throwError $ PandocSomeError $ name <> " not supported for benchmark"
3839 (rdr, rexts) <- getReader name
3940 (wtr, wexts) <- getWriter name
4041 case (rdr, wtr) of
@@ -56,19 +57,19 @@ readerBench doc name = either (const Nothing) Just $
5657
5758getImages :: IO [(FilePath, MimeType, BL.ByteString)]
5859getImages = do
59− ll <- BL.readFile "test/lalune.jpg"
60− mv <- BL.readFile "test/movie.jpg"
61− return [("lalune.jpg", "image/jpg", ll)
62− ,("movie.jpg", "image/jpg", mv)]
60+ ll <- B.readFile "test/lalune.jpg"
61+ mv <- B.readFile "test/movie.jpg"
62+ return [("lalune.jpg", "image/jpg", BL.fromStrict ll)
63+ ,("movie.jpg", "image/jpg", BL.fromStrict mv)]
6364
6465writerBench :: [(FilePath, MimeType, BL.ByteString)]
6566 -> Pandoc
6667 -> T.Text
6768 -> Maybe Benchmark
69+writerBench _ _ name
70+ | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
6871writerBench imgs doc name = either (const Nothing) Just $
6972 runPure $ do
70− when (name `elem` ["bibtex", "biblatex", "csljson"]) $
71− throwError $ PandocSomeError $ name <> " not supported for benchmark"
7273 (wtr, wexts) <- getWriter name
7374 case wtr of
7475 TextWriter writerFun ->
@@ -90,10 +91,10 @@ main :: IO ()
9091main = do
9192 inp <- UTF8.toText <$> B.readFile "test/testsuite.txt"
9293 let opts = def
93− let doc = either (error . show) id $ runPure $ readMarkdown opts inp
94− imgs <- getImages
94+ let doc = either (error . show) force $ runPure $ readMarkdown opts inp
9595 defaultMain
96− [ bgroup "writers" $ mapMaybe (writerBench imgs doc . fst)
96+ [ env getImages $ \imgs ->
97+ bgroup "writers" $ mapMaybe (writerBench imgs doc . fst)
9798 (sortOn fst
9899 writers :: [(T.Text, Writer PandocPure)])
99100 , bgroup "readers" $ mapMaybe (readerBench doc . fst)
cabal.project+9−8
@@ -1,12 +1,13 @@
11packages: pandoc.cabal
2−
32tests: True
3+flags: +embed_data_files
44
5−package pandoc
6− flags: +embed_data_files
7− ghc-options: -j +RTS -A256m -RTS
5+source-repository-package
6+ type: git
7+ location: https://github.com/jgm/doctemplates
8+ tag: 428c26d5303cf7a2b1051fe1ffd9aafe9ba71c81
89
9−-- source-repository-package
10−-- type: git
11−-- location: https://github.com/jgm/citeproc
12−-- tag: d44e24696ab444090d0e63e321c3a573f68b2e74
10+source-repository-package
11+ type: git
12+ location: https://github.com/jgm/citeproc
13+ tag: 4ce0501cd6f9c86eee087afcf878c256e49b9615
changelog.md+795−8
@@ -1,12 +1,783 @@
11# Revision history for pandoc
22
3−## pandoc 2.12 (UNRELEASED -- PROVISIONAL)
3+## pandoc 2.14.0.3 (2021-06-22)
4+
5+ * Text.Pandoc.MediaBag `insertMediaBag`: ensure we get a sane mediaPath
6+ for URLs (#7391). In earlier 2.14.x versions, we'd get
7+ incorrect paths for resources downloaded from URLs when the
8+ media are extracted (including in PDF production).
9+ * Text.Pandoc.Parsing: improve `emailAddress` (#7398).
10+ Previously the parser would accept characters in domains
11+ that are illegal in domains, and this sometimes caused it
12+ to gobble bits of the following text.
13+ * txt2tags reader: modify the email address parser so
14+ it still includes form parameters, even after the change to
15+ `emailAddress` in Text.Pandoc.Parsing.
16+ * Text.Pandoc.Readers.Metadata: Fix regression with comment-only YAML
17+ metadata blocks (#7400).
18+ * reveal.js writer and template: better handling of options. Previously
19+ it was impossible to specify false values for options that default to
20+ true (e.g. `center`); setting the option to false just caused the portion
21+ of the template setting the option to be omitted. Now we prepopulate
22+ all the variables with their default values, including them all
23+ unconditionally and allowing them to be overridden.
24+ * Markdown writer: Fix regression in code blocks with attributes (#7397).
25+ Code blocks with a single class but nonempty attributes
26+ were having attributes drop as a result of #7242.
27+ * LaTeX writer:
28+ + Add strut at end of minipage if it contains line breaks.
29+ Without them, the last line is not as tall as it should be in
30+ some cases.
31+ + Always use a minipage for cells with line breaks, when
32+ width information is available (#7393). Otherwise the way we treat them
33+ can lead to content that overflows a cell.
34+ + Use `\strut` instead of `~` before `\\` in empty line.
35+ * Use lts-18.0 stack resolver.
36+ * Require skylighting 0.10.5.2 (adding support for Swift).
37+ * Require commonmark 0.2.1.
38+ * Rephrase section on unsafe HTML in manual.
39+ * Create SECURITY.md
40+
41+## pandoc 2.14.0.2 (2021-06-13)
42+
43+ * Fix MediaBag regressions (#7345). iIn the 2.14 release `--extract-media`
44+ stopped working as before; there could be mismatches between the
45+ paths in the rendered document and the extracted media.
46+ This patch makes several changes that restore the earlier behavior
47+ (while keeping the same API). The `mediaPath` in 2.14 was always
48+ constructed from the SHA1 hash of the media contents. Now, we
49+ preserve the original path unless it's an absolute path or contains
50+ `..` segments (in that case we use a path based on the SHA1 hash of
51+ the contents).
52+
53+ In Text.Pandoc.MediaBag, `mediaDirectory` and `mediaItems` now use the
54+ `mediaPath`, rather than the mediabag key, for the first component of the
55+ tuple. This makes more sense, I think, and fits with the documentation of
56+ these functions; eventually, though, we should rework the API so that
57+ `mediaItems` returns both the keys and the MediaItems.
58+
59+ In Text.Pandoc.Class.IO, rewriting of source paths in `extractMedia` has
60+ been fixed.
61+
62+ In Text.Pandoc.Class.PandocMonad, `fillMediaBag` has been modified so that
63+ it doesn't modify image paths (that was part of the problem in #7345).
64+
65+ We now do path normalization (e.g. `\` separators on Windows) in
66+ writing the media.
67+
68+ * Text.Pandoc.PDF:
69+
70+ + Text.Pandoc.PDF: Fix regression in 2.14 for generation of PDFs with
71+ SVGs (#7344).
72+ + Only print relevant part of environment on `--verbose`. Since
73+ `--verbose` output might be put in an issue, we want to avoid
74+ spilling out secrets in environment variables.
75+
76+ * Markdown reader: fix pipe table regression in 2.11.4 (#7343).
77+ Previously pipe tables with empty headers (that is, a header
78+ line with all empty cells) would be rendered as headerless
79+ tables. This broke in 2.11.4. The fix here is to produce an
80+ AST with an empty table head when a pipe table has all empty
81+ header cells.
82+
83+ * LaTeX reader: don't allow optional `*` on symbol control sequences
84+ (#7340). Generally we allow optional starred variants of LaTeX commands
85+ (since many allow them, and if we don't accept these explicitly,
86+ ignoring the star usually gives acceptable results). But we
87+ don't want to do this for `\(*\)` and similar cases.
88+
89+ * Docx reader: handle absolute URIs in Relationship Target (#7374).
90+
91+ * Docx writer: fix handling of empty table headers (Albert Krewinkel,
92+ #7369). A table header which does not contain any cells is now treated as
93+ an empty header.
94+
95+ * LaTeX writer: Fix regression in table header position (#7347).
96+ In recent versions the table headers were no longer bottom-aligned
97+ (if more than one line). This patch fixes that by using minipages
98+ for table headers in non-simple tables.
99+
100+ * CommonMark writer:
101+
102+ + Do not use simple class for fenced-divs (Jan Tojnar, amends #7242.)
103+ + Do not throw away attributes when `Ext_attributes` is enabled.
104+ `Ext_attributes` covers at least the following:
105+ `Ext_fenced_code_attributes`, `Ext_header_attributes`,
106+ `Ext_inline_code_attributes`, `Ext_link_attributes`.
107+
108+ * Markdown writer:
109+
110+ + Allow `pipe_tables` to be disabled for commonmark formats
111+ (`commonmark_x`, `gfm`) (#7375).
112+ + Re-use functions from Text.Pandoc.Markdown.Inline (Jan Tojnar).
113+
114+ * DocBook writer: Remove non-existent admonitions (Jan Tojnar).
115+ `attention`, `error` and `hint` are reStructuredText specific.
116+
117+ * HTML writer: Don't omit width attribute on div (#7342).
118+
119+ * Text.Pandoc.MIME, `extensionFromMimeType`: add a few special cases.
120+ When we do a reverse lookup in the MIME table, we just get the
121+ last match, so when the same mime type is associated with several
122+ different extensions, we sometimes got weird results, e.g. `.vs`
123+ for `text/plain`. These special cases help us get the most standard
124+ extensions for mime types like `text/plain`.
125+
126+ * Lua utils: fix handling of table headers in `from_simple_table` (Albert
127+ Krewinkel, #7369). Passing an empty list of header cells now results
128+ in an empty table header.
129+
130+ * Text.Pandoc.Citeproc:
131+
132+ + Avoid duplicate classes and attributes on references div.
133+ + Fix regression in citeproc processing (#7376). If inline
134+ references are used (in the metadata `references` field), we
135+ should still only include in the bibliography items that are
136+ actually cited (unless `nocite` is used).
137+
138+ * Require citeproc 0.4.0.1. This fixes a bug which led to doubled
139+ "et al." in some (rare) circumstances.
140+
141+ * MANUAL.txt:
142+
143+ + Mention GladTeX for EPUB export (Sebastian Humenda).
144+ This updates the manual and the web site about the GladTeX usage.
145+ + More details and a useful link for YAML syntax.
146+
147+ * CONTRIBUTING.md: update modules overview (Albert Krewinkel).
148+
149+ * using-the-pandoc-api.md: switch from String to Text (Albert Krewinkel).
150+
151+
152+## pandoc 2.14.0.1 (2021-06-01)
153+
154+ * Commonmark reader: Fix regression in 2.14 with YAML metdata block parsing,
155+ which could cause the document body to be omitted after metadata (#7339).
156+
157+ * HTML reader: fix column width regression in 2.14 (#7334).
158+ Column widths specified with a style attribute were off by a factor of 100.
159+
160+ * Markdown reader: in `rebasePaths`, check for both Windows and Posix
161+ absolute paths. Previously Windows pandoc was treating
162+ `/foo/bar.jpg` as non-absolute.
163+
164+ * Text.Pandoc.Logging: In rendering `LoadedResource`, use relative paths.
165+
166+ * Docx writer: fix regression on captions (#7328). The "Table Caption"
167+ style was no longer getting applied. (It was overwritten by "Compact.")
168+
169+ * Use commonmark-extensions 0.2.1.2
170+
171+## pandoc 2.14 (2021-05-28)
172+
173+ * Change reader types, allowing better tracking of source positions
174+ [API change]. Previously, when multiple file arguments were provided,
175+ pandoc simply concatenated them and passed the contents to the readers,
176+ which took a Text argument. As a result, the readers had no way of knowing
177+ which file was the source of any particular bit of text. This meant that
178+ we couldn't report accurate source positions on errors or include accurate
179+ source positions as attributes in the AST. More seriously, it meant that
180+ we couldn't resolve resource paths relative to the files containing them
181+ (see e.g. #5501, #6632, #6384, #3752).
182+
183+ * Add `rebase_relative_paths` extension (#3752). When enabled, this
184+ extension rewrites relative image and link paths by prepending
185+ the (relative) directory of the containing file. This
186+ behavior is useful when your input sources are split
187+ into multiple files, across several directories, with files
188+ referring to images stored in the same directory. The
189+ extension can be enabled for all markdown and commonmark-based formats.
190+
191+ * Add Text.Pandoc.Sources (exported module), with a `Sources` type and a
192+ `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs
193+ [API change]. A parsec `Stream` instance is provided for `Sources`. The
194+ module also exports versions of parsec's `satisfy` and other Char parsers
195+ that track source positions accurately from a `Sources` stream (or any
196+ instance of the new `UpdateSourcePos` class).
197+
198+ * Text.Pandoc.Parsing
199+
200+ + Export the modified Char parsers defined in Text.Pandoc.Sources
201+ instead of the ones parsec provides. Modified parsers to use a
202+ `Sources` as stream [API change].
203+ + Improve include file functions [API change]. Remove old
204+ `insertIncludedFileF`. Give `insertIncludedFile` a more general type,
205+ allowing it to be used where `insertIncludedFileF` was.
206+ + Add parameter to the `citeKey` parser from
207+ Text.Pandoc.Parsing, which controls whether the `@{..}`
208+ syntax is allowed [API change].
209+
210+ * Text.Pandoc.Error: Modified the constructor `PandocParsecError` to take a
211+ `Sources` rather than a `Text` as first argument, so parse error locations
212+ can be accurately reported.
213+
214+ * Fix source position reporting for YAML bibliographies (#7273).
215+
216+ * Issue error message when reader or writer format is malformed
217+ (#7231). Previously we exited with an error status but (due to a bug)
218+ no message.
219+
220+ * Smarter smart quotes (#7216, #2103). Treat a leading `"` with no
221+ closing `"` as a left curly quote. This supports the practice, in
222+ fiction, of continuing paragraphs quoting the same speaker without an
223+ end quote. It also helps with quotes that break over lines in line blocks.
224+
225+ * Markdown reader:
226+
227+ + Use MetaInlines not MetaBlocks for multimarkdown metadata fields.
228+ This gives better results in converting to e.g. pandoc markdown.
229+ + Implement curly-brace syntax for Markdown citation keys (#6026).
230+ The change provides a way to use citation keys that contain
231+ special characters not usable with the standard citation key syntax.
232+ Example: `@{foo_bar{x}'}` for the key `foo_bar{x}`. It also allows
233+ separating citation keys from immediately following text, e.g. `@{foo}A`.
234+
235+ * RST reader:
236+
237+ + Seek include files in the directory of the file
238+ containing the include directive, as RST requires (#6632).
239+ + Use `insertIncludedFile` from Text.Pandoc.Parsing
240+ instead of reproducing much of its code.
241+
242+ * Org reader: Resolve org includes relative to the directory containing the
243+ file containing the INCLUDE directive (#5501).
244+
245+ * ODT reader: Treat tabs as spaces (#7185, niszet).
246+
247+ * Docx reader:
248+
249+ + Add handling of vml image objects (#7257, mbrackeantidot).
250+ + Support new table features (Emily Bourke, #6316): column
251+ spans, row spans, multiple header rows, table description
252+ (parsed as a simple caption), captions, column widths.
253+
254+ * LaTeX reader:
255+
256+ + Improved siunitx support (#6658, #6620).
257+ + Better support for `\xspace` (#7299).
258+ + Improve parsing of `\def` macros. We previously set "verbatim mode"
259+ even for parsing the initial `\def`; this caused problems
260+ for `\def` nested inside another `\def`.
261+ + Implement `\newif`.
262+
263+ * ConTeXt writer: improve ordered lists (#5016, Denis Maier).
264+ Change ordered list from itemize to enumerate. Add new
265+ itemgroup for ordered lists. Remove manual insertion of
266+ width attributes. Use tabular figures in ordered list
267+ enumerators.
268+
269+ * HTML reader:
270+
271+ + Don't fail on unmatched closing "script" tag (Albert Krenkel, #7282).
272+ + Keep h1 tags as normal headers (#2293, Albert Krewinkel).
273+ The tags `<title>` and `<h1 class="title">` often contain the same
274+ information, so the latter was dropped from the document. However, as
275+ this can lead to loss of information, the heading is now always
276+ retained. Use `--shift-heading-level-by=-1` to turn the `<h1>`
277+ into the document title, or a filter to restore the previous behavior.
278+ + Handle relative lengths (e.g. `2*`) in HTML column widths (#4063).
279+ See <https://www.w3.org/TR/html4/types.html#h-6.6>.
280+
281+ * DocBook/JATS readers:
282+
283+ + Fix mathml regression caused by the switch in XML libraries (#7173).
284+ + Fix "phrase" in DocBook: take classes from "role" not "class" (#7195).
285+
286+ * DocBook reader: ensure that first and last names are separated (#6541).
287+
288+ * Jira reader (Albert Krewinkel, #7218):
289+
290+ + Support "smart" links: `[alias|https://example.com|smart-card]` syntax.
291+ + Allow spaces and most unicode characters in attachment links.
292+ + No longer require a newline character after `{noformat}`.
293+ + Only allow URI path segment characters in bare links.
294+ + The `file:` schema is no longer allowed in bare links; these
295+ rarely make sense.
296+
297+ * Plain writer: handle superscript unicode minus (#7276).
298+
299+ * LaTeX writer:
300+
301+ + Better handling of line breaks in simple tables (#7272).
302+ Now we also handle the case where they're embedded in other elements,
303+ e.g. spans.
304+ + For beamer output, support `exampleblock` and `alertblock` (#7278).
305+ A block will be rendered as an `exampleblock` if the heading
306+ has class `example` and an `alertblock` if it has class `alert`.
307+ + Separate successive quote chars with thin space (#6958,
308+ Albert Krewinkel). Successive quote characters are separated with
309+ a thin space to improve readability and to prevent unwanted ligatures.
310+ Detection of these quotes sometimes had failed if the second quote
311+ was nested in a span element.
312+ + Separate successive quote chars with thin space (#6958, Albert
313+ Krewinkel).
314+
315+ * EPUB Writer: Fix belongs-to-collection XML id choice (#7267, nuew).
316+ The epub writer previously used the same XML id for both the book
317+ identifier and the epub collection. This causes an error on epubcheck.
318+
319+ * BibTeX/BibLaTeX writer: Handle `annote` field (#7266).
320+
321+ * ZimWiki writer: allow links and emphasis in headers (#6605,
322+ Albert Krewinkel).
323+
324+ * ConTeXt writer:
325+
326+ + Support blank lines in line blocks (#6564, Albert Krewinkel,
327+ thanks to @denismaier).
328+ + Use span identifiers as reference anchors (#7246, Albert Krewinkel).
329+
330+ * HTML writer:
331+
332+ + Keep attributes from code nested below `pre` tag (#7221,
333+ Albert Krewinkel). If a code block is defined with `<pre><code
334+ class="language-x">…</code></pre>`, where the `<pre>` element has no
335+ attributes, then the attributes from the `<code>` element are used
336+ instead. Any leading `language-` prefix is dropped in the code's
337+ `class` attribute are dropped to improve syntax highlighting.
338+ + Ensure headings only have valid attribs in HTML4 (#5944, Albert
339+ Krewinkel).
340+ + Parse `<header>` as a Div (Albert Krewinkel).
341+
342+ * Org writer:
343+
344+ + Inline latex envs need newlines (#7252, tecosaur).
345+ As specified in https://orgmode.org/manual/LaTeX-fragments.html, an
346+ inline \begin{}...\end{} LaTeX block must start on a new line.
347+ + Use LaTeX style maths deliminators (#7196, tecosaur).
348+
349+ * JATS writer (Albert Krewinkel):
350+
351+ + Use either styled-content or named-content for spans (#7211).
352+ If the element has a content-type attribute, or at least one class,
353+ then that value is used as `content-type` and the span is put inside
354+ a `<named-content>` element. Otherwise a `<styled-content>` element
355+ is used instead.
356+ + Reduce unnecessary use of `<p>` elements for wrapping (#7227).
357+ The `<p>` element is used for wrapping in cases were the contents
358+ would otherwise not be allowed in a certain context. Unnecessary
359+ wrapping is avoided, especially around quotes (`<disp-quote>` elements).
360+ + Convert spans to `<named-content>` elements (#7211). Spans with
361+ attributes are converted to `<named-content>` elements instead of
362+ being wrapped with `<milestone-start/>` and `<milestone-end>`
363+ elements. Milestone elements are not allowed in documents using the
364+ articleauthoring tag set, so this change ensures the creation of valid
365+ documents.
366+ + Add footnote number as label in backmatter (#7210). Footnotes in the
367+ backmatter are given the footnote's number as a label. The
368+ articleauthoring output is unaffected from this change, as footnotes
369+ are placed inline there.
370+ + Escape disallows chars in identifiers. XML identifiers must start
371+ with an underscore or letter, and can contain only a limited set
372+ of punctuation characters. Any IDs not adhering to these rules are
373+ rewritten by writing the offending characters as `Uxxxx`,
374+ where `xxxx` is the character's hex code.
375+
376+ * Jira writer: use `{color}` when span has a color attribute
377+ (Albert Krewinkel, tarleb/jira-wiki-markup#10).
378+
379+ * Docx writer:
380+
381+ + Autoset table width if no column has an explicit width (Albert
382+ Krewinkel).
383+ + Extract Table handling into separate module (Albert Krewinkel).
384+ + Support colspans and rowspans in tables (Albert Krewinkel, #6315).
385+ + Support multirow table headers (Albert Krewinkel).
386+ + Improve integration of settings from reference.docx (#1209).
387+ This change allows users to create a reference.docx that
388+ sets `w:proofState` for spelling or grammar to `dirty`,
389+ so that spell/grammar checking will be triggered on the
390+ generated docx.
391+ + Copy over more settings from reference.docx (#7240). From settings.xml
392+ in the reference-doc, we now include: `zoom`, `embedSystemFonts`,
393+ `doNotTrackMoves`, `defaultTabStop`, `drawingGridHorizontalSpacing`,
394+ `drawingGridVerticalSpacing`, `displayHorizontalDrawingGridEvery`,
395+ `displayVerticalDrawingGridEvery`, `characterSpacingControl`,
396+ `savePreviewPicture`, `mathPr`, `themeFontLang`, `decimalSymbol`,
397+ `listSeparator`, `autoHyphenation`, `compat`.
398+ + Set zoom to 100% by default in settings.xml.
399+ + Align math options more with current Word defaults (e.g. Cambria Math
400+ font).
401+ + Remove `rsid`s from default settings.xml. Word will add these
402+ when revisions are made.
403+
404+ * Ms writer: Handle tables with multiple paragraphs (#7288).
405+ Previously they overflowed the table cell width. We now set line lengths
406+ per-cell and restore them after the table has been written.
407+
408+ * Markdown writer:
409+
410+ + Use cleaner braceless syntax for code blocks with a
411+ single class (#7242, Jan Tojnar).
412+ + Add quotes properly in markdown YAML metadata fields (#7245).
413+ This fixes a bug, which caused the writer to look at the *last*
414+ rather than the *first* character in determining whether quotes
415+ were needed. So we got spurious quotes in some cases and
416+ didn't get necessary quotes in others.
417+ + Use `@{..}` syntax for citations when needed.
418+ + Use fewer unneeded escapes for `#` (see #6259).
419+ + Improve escaping of `@`. We need to escape literal `@` before
420+ `{` because of the new citation syntax.
421+
422+ * Commonmark writer: Use backslash escapes for `<` and `|`...
423+ instead of entities (#7208).
424+
425+ * Powerpoint writer: allow `monofont` to be specified in metadata
426+ (#7187).
427+
428+ * LaTeX template:
429+
430+ + Use non-starred names for xcolor color names (#6109).
431+ This should make svgnames and x11names work properly.
432+ + Fix bad vertical spacing after bibliography (#7234, badumont).
433+ + List of figures before list of tables (#7235, Julien Dutant).
434+ + Move CSL macro definitions before header-includes so they can be
435+ overridden (#7286).
436+ + Improve treatment of CSL `entry-spacing` (#7296).
437+ Previously with the default template settings (`indent` variable
438+ not set), we would get interparagraph spaces separating bib
439+ entries even with `entry-spacing="0"`. On the other hand,
440+ setting `entry-spacing="2"` gave ridiculously large spacing.
441+ This change makes the spacing caused by `entry-spacing` a multiple
442+ of `\parskip` by default, which gives aesthetically reasonable
443+ output. Those who want a larger or smaller unit (e.g. because
444+ they use `indent` which sets `\parskip` to 0) may
445+ `\setlength{\cslentryspacingunit}{10pt}` in header-includes
446+ to override the defaults.
447+ + Move title, author, date up to top of preamble (#7295).
448+ This allows header-includes to use them, and puts them
449+ in a position where you can see them immediately.
450+ + Define commands for zero width non-joiner character
451+ (#6639, Albert Krewinkel). The zero-width non-joiner character
452+ is used to avoid ligatures (e.g. in German).
453+
454+ * ConTeXt template:
455+
456+ + Define `enumerate` itemgroup (#5016, Denis Maier).
457+ + List of figures before list of tables (#7235, Julien Dutant).
458+
459+ * reveal.js template:
460+
461+ + Support `toc-title` (#7171, Florian Kohrt).
462+ + Use `hash: true` by default rather than `history: true` (#6968).
463+
464+ * HTML-based slide shows: add support for `institute` (#7289, Thomas
465+ Hodgson).
466+
467+ * Text.Pandoc.Extensions: Add constructor `Ext_rebase_relative_paths` to
468+ `Extensions` [API change].
469+
470+ * Text.Pandoc.XML.Light: add Eq, Ord instances for Content,
471+ Element, Attr, CDataKind [API change].
472+
473+ * Text.Pandoc.MediaBag:
474+
475+ + Change type to use a `Text` key instead of `[FilePath]`.
476+ We normalize the path and use `/` separators for consistency.
477+ + Export `MediaItem` type [API change].
478+ + Change `MediaBag` type to a map from Text to MediaItem [API change].
479+ + `lookupMedia` now returns a `MediaItem` [API change].
480+ + Change `insertMedia` so it sets the `mediaPath` to a filename based on
481+ the SHA1 hash of the contents. This will be used when contents
482+ are extracted.
483+
484+ * Text.Pandoc.Class.PandocMonad:
485+
486+ + Remove `fetchMediaResource` [API change]. Use `fetchItem` to get
487+ resources in `fillMediaBag`.
488+ + Add informational message in `downloadOrRead` indicating what path
489+ local resources have been loaded from.
490+
491+ * Text.Pandoc.Logging:
492+
493+ + Remove single quotes around paths in messages.
494+ + Add LoadedResource constructor to LogMessage [API change].
495+ This is for INFO-level messages telling where image data has been
496+ loaded from. (This can vary because of the resource path.)
497+
498+ * Text.Pandoc.Asciify: simplify code and export `toAsciiText` [API change].
499+ Instead of encoding a giant (and incomplete) map, we now
500+ just use unicode-transforms to normalize the text to
501+ a canonical decomposition, and manipulate the result.
502+
503+ * App: allow tabs expansion even if file-scope is used (Albert Krewinkel,
504+ #6709). Tabs in plain-text inputs are now handled correctly, even if
505+ the `--file-scope` flag is used.
506+
507+ * Add new internal module Text.Pandoc.Writers.GridTable (Albert Krewinkel).
508+
509+ * Text.Pandoc.Highlighting: Change type of `languagesByExtension`, adding
510+ a parameter for a `SyntaxMap` [API change] (Jan Tojnar, #7241).
511+ Languages defined using `--syntax-definition` were not recognized by
512+ `languagesByExtension`. This patch corrects that, allowing the writers
513+ to see all custom definitions. The LaTeX writer still uses the default
514+ syntax map, but that's okay in that context, since
515+ `--syntax-definition` won't create new listings styles.
516+
517+ * Text.Pandoc.Citeproc:
518+
519+ + Ensure that CSL-related attributes are passed on to a Div with id
520+ 'refs'. Otherwise things like `entry-spacing` won't work when
521+ such Divs are used.
522+ + Use metadata's `lang` for the lang parameter of citeproc, overriding
523+ `localeLanguage`.
524+ + Recognize locators spelled with a capital letter (#7323).
525+ + Add a comma and a space in front of the suffix if it doesn't start
526+ with space or punctuation (#7324).
527+ + Don't detect math elements as locators (#7321).
528+
529+ * Remove Text.Pandoc.BCP47 module [API change]. Use types and functions
530+ from UnicodeCollation.Lang instead. This is a richer implementation
531+ of BCP 47.
532+
533+ * Text.Pandoc.Shared:
534+
535+ + Fix regression in grid tables for wide characters (#7214).
536+ In the translation from String to Text, a char-width-sensitive
537+ `splitAt'` was dropped. This commit reinstates it and uses it to make
538+ `splitTextByInstances` char-width sensitive.
539+ + Add `getLang` (formerly in the now-removed BCP47) [API change].
540+
541+ * Text.Pandoc.SelfContained: use `application/octet-stream`
542+ for unknown mime types instead of halting with an error (#7202).
543+
544+ * Lua filters: respect Inlines/Blocks filter functions in `pandoc.walk_*`
545+ (Albert Krewinkel).
546+
547+ * Add text as build-depend for trypandoc (#7193, Roman Beránek).
548+
549+ * Bump upper-bounds for network-uri, time, attoparsec.
550+
551+ * Use citeproc 0.4.
552+
553+ * Use texmath 0.12.3.
554+
555+ * Use jira-wiki-markup 1.3.5 (Albert Krewinkel).
556+
557+ * Require latest skylighting (fixes a bug in XML syntax highlighting).
558+
559+ * Use latest xml-conduit.
560+
561+ * Use latest commonmark, commonmark-extensions, commonmark-pandoc.
562+
563+ * Use haddock-library-1.10.0 (Albert Krewinkel).
564+
565+ * Allow compilation with base 4.15 (Albert Krewinkel).
566+
567+ * MANUAL:
568+
569+ + Add information about `lang` and bibliography sorting.
570+ + Add info about YAML escape sequences, link to spec (#7152,
571+ Albert Krewinkel).
572+ + Note that `institute` variable works for HTML-based slides.
573+ + Update documentation on citation syntax.
574+ + Add citation example for locators and suffixes (Tristan Stenner)
575+
576+ * Updated and fixed typos in documentation (Charanjit Singh,
577+ Anti-Distinctlyminty, Tatiana Porras, obcat).
578+
579+ * Add instructions for installing pandoc-types before compiling filter.
580+
581+ * INSTALL: add note that parallel installations should be avoided
582+ (#6865).
583+
584+ * Remove `biblatex-nussbaum.md` test. It is basically the same
585+ as `biblaetx-quotes.md`.
586+
587+ * Command tests: fail if a file contains no tests---and fix a
588+ test that failed in that way!
589+
590+ * Use smaller images in tests, reducing the size of the source tarball by 8 MB.
591+
592+
593+## pandoc 2.13 (2021-03-21)
594+
595+ * Support `yaml_metadata_block` extension for `commonmark`, `gfm` (#6537).
596+ This supported is a bit more limited than with pandoc's
597+ `markdown`. The YAML block must be the first thing in the input,
598+ and the leaf notes are parsed in isolation from the rest of
599+ the document. So, for example, you can't use reference
600+ links if the references are defined later in the document.
601+
602+ * Fix fallback to default partials when custom templates are used.
603+ If the directory containing a template does not contain the partial,
604+ it should be sought in the default templates, but this was not
605+ working properly (#7164).
606+
607+ * Handle `nocite` better with `--biblatex` and `--natbib` (#4585).
608+ Previously the nocite metadata field was ignored with these formats.
609+ Now it populates a `nocite-ids` template variable and causes a
610+ `\nocite` command to be issued.
611+
612+ * Text.Pandoc.Citeproc: apply `fixLinks` correctly (#7130). This is code
613+ that incorporates a prefix like `https://doi.org/` into a following link
614+ when appropriate.
615+
616+ * Text.Pandoc.Shared:
617+
618+ + Remove `backslashEscapes`, `escapeStringUsing` [API change]. Replace
619+ these inefficient association list lookups with more efficient escaping
620+ functions in the writers that used them (for a 10-25% performance boost
621+ in org, haddock, rtf, texinfo writers).
622+ + Remove `ToString`, `ToText` typeclasses [API change]. These were needed
623+ for the transition from String to Text, but they are no longer used and
624+ may clash with other things.
625+ + Simplify `compactDL`.
626+
627+ * Text.Pandoc.Parsing:
628+
629+ + Change type of `readWithM` so that it is no longer polymorphic
630+ [API change]. The `ToText` class has been removed, and now that we've
631+ completed the transition to Text we no longer need this to operate
632+ on Strings.
633+ + Remove `F` type synonym [API change]. Muse and Org were defining their
634+ own `F` anyway.
635+
636+ * Text.Pandoc.Readers.Metadata:
637+
638+ + Export `yamlMetaBlock` [API change].
639+ + Make `yamlBsToMeta`, `yamlBsToRefs` polymorphic on the parser state
640+ [API change].
641+
642+ * Markdown reader: Fix regression with `tex_math_backslash` (#7155).
643+
644+ * MediaWiki reader: Allow block-level content in notes (ref) (#7145).
645+
646+ * Jira reader (Albert Krewinkel):
647+
648+ + Fixed parsing of autolinks (i.e., of bare URLs in the text).
649+ Previously an autolink would take up the rest of a line, as spaces
650+ were allowed characters in these items.
651+ + Emoji character sequences no longer cause parsing failures. This was
652+ due to missing backtracking when emoji parsing fails.
653+ + Mark divs created from panels with class "panel".
654+
655+ * RST reader: fix logic for ending comments (#7134). Previously comments
656+ sometimes got extended too far.
657+
658+ * DocBook writer: include Header attributes as XML attributes on
659+ section (Erik Rask). Attributes with key names that are not allowed
660+ as XML attributes are dropped, as are attributes with invalid values
661+ and `xml:id` (DocBook 5) and `id` (DocBook 4).
662+
663+ * Docx writer:
664+
665+ + Make `nsid` in `abstractNum` deterministic. Previously we assigned
666+ a random number, but we don't need random values, so now we just
667+ assign a value based on the list marker.
668+ + Use integral values for `w:tblW` (#7141).
669+
670+ * Jira writer (Albert Krewinkel):
671+
672+ + Block quotes are only rendered as `bq.` if they do not contain a
673+ linebreak.
674+ + Jira writer: improve div/panel handling. Include div attributes in
675+ panels, always render divs with class `panel` as panels, and
676+ avoid nesting of panels.
677+
678+ * HTML writer: Add warnings on duplicate attribute values.
679+ This prevents emitting invalid HTML. Ultimately it would be good to
680+ prevent this in the types themselves, but this is better for now.
681+
682+ * Org writer: Prevent unintended creation of ordered list items (#7132,
683+ Albert Krewinkel). Adjust line wrapping if default wrapping would cause
684+ a line to be read as an ordered list item.
685+
686+ * JATS templates: support 'equal-contrib' attrib for authors (Albert
687+ Krewinkel). Authors who contributed equally to a paper may be marked
688+ with `equal-contrib`.
689+
690+ * reveal.js template: replace JS comment with HTML (#7154, Florian Kohrt).
691+
692+ * Text.Pandoc.Logging: Add `DuplicateAttribute` constructor to `LogMessage`.
693+ [API change]
694+
695+ * Use `-j4` for linux release build. This speeds up the build dramatically
696+ on arm.
697+
698+ * cabal.project: remove ghcoptions. Move flags to top level, so they can
699+ be set differently on the command line.
700+
701+ * Require latest texmath, skylighting, citeproc, jira-wiki-markup.
702+ (The latest skylighting fixes a bad bug with Haskell syntax highlighting.)
703+ Narrow version bounds for texmath, skylighting, and citeproc, since
704+ the test output depend on them.
705+
706+ * Use doclayout 0.3.0.2. This significantly reduces the time and memory
707+ needed to compile pandoc.
708+
709+ * Use `foldl'` instead of `foldl` everywhere.
710+
711+ * Update bounds for random (#7156, Alexey Kuleshevich).
712+
713+ * Remove uses of some partial functions.
714+
715+ * Don't bake in a larger stack size for the executable.
716+
717+ * Test improvements:
718+
719+ + Use `getExecutablePath` from base, avoiding the dependency on
720+ `executable-path`.
721+ + Factor out `setupEnvironment` in Helpers, to avoid code duplication.
722+ + Fix finding of data files by setting teh `pandoc_datadir` environment
723+ variable when we shell out to pandoc. This avoids the need to use
724+ `--data-dir` for the tests, which caused problems finding `pandoc.lua`
725+ when compiling without the `embed_data_files` flag (#7163).
726+
727+ * Benchmark improvements:
728+
729+ + Build `+RTS -A8m -RTS` into default ghc-options for benchmark.
730+ This is necessary to get accurate benchmark results; otherwise we
731+ are largely measuring garbage collecting, some not related to the
732+ current benchmark.
733+ + Allow specifying BASELINE file in 'make bench' for comparison
734+ (otherwise the latest benchmark is chosen by default).
735+ + Force `readFile` in benchmarks early (Bodigrim).
736+
737+ * CONTRIBUTING: suggest using a `cabal.project.local` file (#7153,
738+ Albert Krewinkel).
739+
740+ * Add ghcid-test to Makefile. This loads the test suite in ghcid.
741+
742+
743+## pandoc 2.12 (2021-03-08)
744+
745+ * `--resource-path` now accumulates if specified multiple
746+ times (#6152). Resource paths specified later on the command line are
747+ prepended to those specified earlier. Thus,
748+ `--resource-path foo --resource-path bar:baz` is equivalent to
749+ `--resource-path bar:bas:foo`. (The previous behavior was
750+ for the last `--resource-path` to replace all the rest.)
751+ `resource-path` in defaults files behaves the same way: it
752+ will be prepended to the resource path set by earlier
753+ command line options or defaults files. This change
754+ facilitates the use of multiple defaults files: each can
755+ specify a directory containing resources it refers to
756+ without clobbering the resource paths set by the others.
757+
758+ * Allow defaults files to refer to the home directory, the
759+ user data directory, and the directory containing the defaults file
760+ itself (#5871, #5982, #5977). In fields that expect file paths
761+ (and only in these fields),
762+
763+ + `${VARIABLE}` will expand to the value of the environment variable
764+ `VARIABLE` (and in particular `${HOME}` will expand to the path
765+ of the home directory). A warning will be raised for undefined
766+ variables.
767+ + `${USERDATA}` will expand to the path of the user data
768+ directory in force when the defaults file is being processed.
769+ + `${.}` will expand to the directory containing the defaults file.
770+ (This allows default files to be placed in a directory containing
771+ resources they make use of.)
4772
5773 * When downloading content from URL arguments, be sensitive to
6774 the character encoding (#5600). We can properly handle UTF-8 and latin1
7775 (ISO-8859-1); for others we raise an error. Fall back to latin1 if
8776 no charset is given in the mime type and UTF-8 decoding fails.
9777
778+ * Allow abbreviations that don't end in a period to be
779+ specified using `--abbreviations` (#7124).
780+
10781 * Add new unexported module Text.Pandoc.XML.Light, as well
11782 as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc,
12783 Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).
@@ -102,6 +873,7 @@
102873 Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Inline}.
103874 Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map
104875 instead of individual commands.
876+ + Handle table cells containing `&` in `\verb` (#7129).
105877
106878 * Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change].
107879
@@ -139,9 +911,10 @@
139911 * Jira reader:
140912
141913 + Modified the Doc parser to skip leading blank lines. This fixes
142− parsing of documents which start with multiple blank lines (#7095).
143− + Prevent URLs within link aliases to be treated as autolinks
144− (#6944).
914+ parsing of documents which start with multiple blank lines (Albert
915+ Krewinkel, #7095).
916+ + Prevent URLs within link aliases to be treated as autolinks (Albert
917+ Krewinkel, #6944).
145918
146919 * Text.Pandoc.Shared
147920
@@ -150,6 +923,10 @@
150923 and `underlineSpan` (which had been deprecated in April 2020)
151924 [API change].
152925 + Export `handleTaskListItem` (Albert Krewinkel) [API change].
926+ + Change `defaultUserDataDirs` to `defaultUserDataDir` [API
927+ change]. We determine what is the default user data directory
928+ by seeing whether the XDG directory and/or legacy
929+ directory exist.
153930
154931 * BibTeX writer:
155932
@@ -184,7 +961,10 @@
184961
185962 * Jira writer:
186963
187− + Use Span identifiers as anchors (Albert Krewinkel,
964+ + Use Span identifiers as anchors (Albert Krewinkel).
965+ + Use `{noformat}` instead of `{code}` for unknown languages (Albert
966+ Krewinkel). Code blocks which are not marked as a language supported
967+ by Jira are rendered as preformatted text via `{noformat}` blocks.
188968
189969 * LaTeX writer:
190970
@@ -262,6 +1042,8 @@
2621042 * Text.Pandoc.App
2631043
2641044 + Add `parseOptionsFromArgs` [API change, new exported function].
1045+ + Add fields for CSL options to `Opt` [API change]:
1046+ `optCSL`, `optbibliography`, `optCitationAbbreviations`.
2651047
2661048 * Text.Pandoc.Citeproc.BibTeX
2671049
@@ -305,7 +1087,6 @@
3051087 + Export `renderError` [API change].
3061088 + Refactor `handleError` to use `renderError`. This allows us render
3071089 error messages without exiting.
308− + Remove unused variables (Albert Krewinkel)
3091090
3101091 * Text.Pandoc.Extensions
3111092
@@ -444,6 +1225,9 @@
4441225 * Remove `weigh-pandoc`. It's not really useful any more, now that our
4451226 regular benchmarks include data on allocation.
4461227
1228+ * Improve linux package build process and add script to
1229+ automate building an arm64 binary package.
1230+
4471231
4481232## pandoc 2.11.4 (2021-01-22)
4491233
@@ -1036,8 +1820,11 @@
10361820 translated to `C:\/foo/bar`, which caused problems.
10371821 With this fix, the backslashes are removed.
10381822
1039− * Text.Pandoc.Logging: Add constructor `ATXHeadingInLHS` constructor
1040− to `LogMessage` [API change].
1823+ * Text.Pandoc.Logging:
1824+
1825+ + Add constructor `ATXHeadingInLHS` to `LogMessage` [API change].
1826+ + Add constructor `EnvironmentVariableUndefined` to
1827+ `LogMessage` [API change].
10411828
10421829 * Fix error that is given when people specify `doc` output (#6834,
10431830 gison93).
data/docx/word/settings.xml+6−16
@@ -1,6 +1,6 @@
11<?xml version="1.0" encoding="UTF-8"?>
22<w:settings xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:w10="urn:schemas-microsoft-com:office:word" xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:sl="http://schemas.openxmlformats.org/schemaLibrary/2006/main">
3− <w:zoom w:percent="90" />
3+ <w:zoom w:percent="100" />
44 <w:embedSystemFonts />
55 <w:proofState w:spelling="clean" w:grammar="clean" />
66 <w:stylePaneFormatFilter w:val="0004" />
@@ -17,31 +17,21 @@
1717 <w:characterSpacingControl w:val="doNotCompress" />
1818 <w:savePreviewPicture />
1919 <w:rsids>
20− <w:rsidRoot w:val="00590D07" />
21− <w:rsid w:val="00011C8B" />
22− <w:rsid w:val="004E29B3" />
23− <w:rsid w:val="00590D07" />
24− <w:rsid w:val="00784D58" />
25− <w:rsid w:val="008D6863" />
26− <w:rsid w:val="00B86B75" />
27− <w:rsid w:val="00BC48D5" />
28− <w:rsid w:val="00C36279" />
29− <w:rsid w:val="00E315A3" />
3020 </w:rsids>
3121 <m:mathPr>
32− <m:mathFont m:val="Lucida Grande" />
22+ <m:mathFont m:val="Cambria Math" />
3323 <m:brkBin m:val="before" />
3424 <m:brkBinSub m:val="--" />
35− <m:smallFrac m:val="false" />
36− <m:dispDef m:val="false" />
25+ <m:smallFrac m:val="0" />
26+ <m:dispDef />
3727 <m:lMargin m:val="0" />
3828 <m:rMargin m:val="0" />
3929 <m:wrapRight />
4030 <m:intLim m:val="subSup" />
41− <m:naryLim m:val="subSup" />
31+ <m:naryLim m:val="undOvr" />
4232 </m:mathPr>
4333 <w:themeFontLang w:val="en-US" />
4434 <w:clrSchemeMapping w:bg1="light1" w:t1="dark1" w:bg2="light2" w:t2="dark2" w:accent1="accent1" w:accent2="accent2" w:accent3="accent3" w:accent4="accent4" w:accent5="accent5" w:accent6="accent6" w:hyperlink="hyperlink" w:followedHyperlink="followedHyperlink" />
4535 <w:decimalSymbol w:val="." />
4636 <w:listSeparator w:val="," />
47−</w:settings>
37+</w:settings>
data/templates/article.jats_publishing+1−1
@@ -86,7 +86,7 @@ $endif$
8686$if(author)$
8787<contrib-group>
8888$for(author)$
89−<contrib contrib-type="author">
89+<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
9090$if(author.orcid)$
9191<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
9292$endif$
data/templates/default.context+7−3
@@ -51,6 +51,7 @@ $endif$
5151
5252% use microtypography
5353\definefontfeature[default][default][script=latn, protrusion=quality, expansion=quality, itlc=yes, textitalics=yes, onum=yes, pnum=yes]
54+\definefontfeature[default:tnum][default][tnum=yes, pnum=no]
5455\definefontfeature[smallcaps][script=latn, protrusion=quality, expansion=quality, smcp=yes, onum=yes, pnum=yes]
5556\setupalign[hz,hanging]
5657\setupitaliccorrection[global, always]
@@ -97,6 +98,9 @@ $endif$
9798\setupitemize[autointro] % prevent orphan list intro
9899\setupitemize[indentnext=no]
99100
101+\defineitemgroup[enumerate]
102+\setupenumerate[each][fit][itemalign=left,distance=.5em,style={\feature[+][default:tnum]}]
103+
100104\setupfloat[figure][default={here,nonumber}]
101105\setupfloat[table][default={here,nonumber}]
102106
@@ -160,12 +164,12 @@ $endfor$
160164$if(toc)$
161165\completecontent
162166$endif$
163−$if(lot)$
164−\completelistoftables
165−$endif$
166167$if(lof)$
167168\completelistoffigures
168169$endif$
170+$if(lot)$
171+\completelistoftables
172+$endif$
169173
170174$body$
171175
data/templates/default.dzslides+1−1
@@ -171,7 +171,7 @@ $if(subtitle)$
171171 <h1 class="subtitle">$subtitle$</h1>
172172$endif$
173173 <footer>
174− $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span>$endif$ · $if(date)$<span class="date">$date$</span>$endif$
174+ $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span> · $endif$$if(institute)$<span class="institute">$for(institute)$$institute$$sep$, $endfor$</span> · $endif$$if(date)$<span class="date">$date$</span>$endif$
175175 </footer>
176176</section>
177177$endif$
data/templates/default.jats_articleauthoring+1−1
@@ -19,7 +19,7 @@ $endif$
1919$if(author)$
2020<contrib-group>
2121$for(author)$
22−<contrib contrib-type="author">
22+<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
2323$if(author.orcid)$
2424<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
2525$endif$
data/templates/default.latex+50−25
@@ -2,7 +2,7 @@
22\PassOptionsToPackage{unicode$for(hyperrefoptions)$,$hyperrefoptions$$endfor$}{hyperref}
33\PassOptionsToPackage{hyphens}{url}
44$if(colorlinks)$
5−\PassOptionsToPackage{dvipsnames,svgnames*,x11names*}{xcolor}
5+\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}
66$endif$
77$if(dir)$
88$if(latex-dir-rtl)$
@@ -156,6 +156,25 @@ $if(CJKmainfont)$
156156 \fi
157157$endif$
158158\fi
159+$if(zero-width-non-joiner)$
160+%% Support for zero-width non-joiner characters.
161+\makeatletter
162+\def\zerowidthnonjoiner{%
163+ % Prevent ligatures and adjust kerning, but still support hyphenating.
164+ \texorpdfstring{%
165+ \textormath{\nobreak\discretionary{-}{}{\kern.03em}%
166+ \ifvmode\else\nobreak\hskip\z@skip\fi}{}%
167+ }{}%
168+}
169+\makeatother
170+\ifPDFTeX
171+ \DeclareUnicodeCharacter{200C}{\zerowidthnonjoiner}
172+\else
173+ \catcode`^^^^200c=\active
174+ \protected\def ^^^^200c{\zerowidthnonjoiner}
175+\fi
176+%% End of ZWNJ support
177+$endif$
159178$if(beamer)$
160179$if(theme)$
161180\usetheme[$for(themeoptions)$$themeoptions$$sep$,$endfor$]{$theme$}
@@ -325,6 +344,31 @@ $endif$
325344$if(pagestyle)$
326345\pagestyle{$pagestyle$}
327346$endif$
347+$if(csl-refs)$
348+\newlength{\cslhangindent}
349+\setlength{\cslhangindent}{1.5em}
350+\newlength{\csllabelwidth}
351+\setlength{\csllabelwidth}{3em}
352+\newlength{\cslentryspacingunit} % times entry-spacing
353+\setlength{\cslentryspacingunit}{\parskip}
354+\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
355+ {% don't indent paragraphs
356+ \setlength{\parindent}{0pt}
357+ % turn on hanging indent if param 1 is 1
358+ \ifodd #1
359+ \let\oldpar\par
360+ \def\par{\hangindent=\cslhangindent\oldpar}
361+ \fi
362+ % set entry spacing
363+ \setlength{\parskip}{#2\cslentryspacingunit}
364+ }%
365+ {}
366+\usepackage{calc}
367+\newcommand{\CSLBlock}[1]{#1\hfill\break}
368+\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
369+\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
370+\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
371+$endif$
328372$for(header-includes)$
329373$header-includes$
330374$endfor$
@@ -372,27 +416,8 @@ $for(bibliography)$
372416\addbibresource{$bibliography$}
373417$endfor$
374418$endif$
375−$if(csl-refs)$
376−\newlength{\cslhangindent}
377−\setlength{\cslhangindent}{1.5em}
378−\newlength{\csllabelwidth}
379−\setlength{\csllabelwidth}{3em}
380−\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
381− {% don't indent paragraphs
382− \setlength{\parindent}{0pt}
383− % turn on hanging indent if param 1 is 1
384− \ifodd #1 \everypar{\setlength{\hangindent}{\cslhangindent}}\ignorespaces\fi
385− % set entry spacing
386− \ifnum #2 > 0
387− \setlength{\parskip}{#2\baselineskip}
388− \fi
389− }%
390− {}
391−\usepackage{calc}
392−\newcommand{\CSLBlock}[1]{#1\hfill\break}
393−\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
394−\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
395−\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
419+$if(nocite-ids)$
420+\nocite{$for(nocite-ids)$$it$$sep$, $endfor$}
396421$endif$
397422$if(csquotes)$
398423\usepackage{csquotes}
@@ -469,12 +494,12 @@ $endif$
469494}
470495$endif$
471496$endif$
472−$if(lot)$
473−\listoftables
474−$endif$
475497$if(lof)$
476498\listoffigures
477499$endif$
500+$if(lot)$
501+\listoftables
502+$endif$
478503$if(linestretch)$
479504\setstretch{$linestretch$}
480505$endif$
data/templates/default.revealjs+79−129
@@ -52,6 +52,9 @@ $endif$
5252$for(author)$
5353 <p class="author">$author$</p>
5454$endfor$
55+$for(institute)$
56+ <p class="institute">$institute$</p>
57+$endfor$
5558$if(date)$
5659 <p class="date">$date$</p>
5760$endif$
@@ -59,7 +62,12 @@ $endif$
5962$endif$
6063$if(toc)$
6164<section id="$idprefix$TOC">
65+<nav role="doc-toc">
66+$if(toc-title)$
67+<h2 id="$idprefix$toc-title">$toc-title$</h2>
68+$endif$
6269$table-of-contents$
70+</nav>
6371</section>
6472$endif$
6573
@@ -69,7 +77,7 @@ $body$
6977
7078 <script src="$revealjs-url$/dist/reveal.js"></script>
7179
72− // reveal.js plugins
80+ <!-- reveal.js plugins -->
7381 <script src="$revealjs-url$/plugin/notes/notes.js"></script>
7482 <script src="$revealjs-url$/plugin/search/search.js"></script>
7583 <script src="$revealjs-url$/plugin/zoom/zoom.js"></script>
@@ -82,256 +90,198 @@ $endif$
8290 // Full list of configuration options available at:
8391 // https://revealjs.com/config/
8492 Reveal.initialize({
85−$if(center)$
86− // Determines whether slide content should be vertically centered
87− center: $center$,
88−$endif$
89−$if(controls)$
9093 // Display controls in the bottom right corner
9194 controls: $controls$,
92−$endif$
93−$if(controlsTutorial)$
95+
9496 // Help the user learn the controls by providing hints, for example by
9597 // bouncing the down arrow when they first encounter a vertical slide
9698 controlsTutorial: $controlsTutorial$,
97−$endif$
98−$if(controlsLayout)$
99+
99100 // Determines where controls appear, "edges" or "bottom-right"
100101 controlsLayout: '$controlsLayout$',
101−$endif$
102−$if(controlsBackArrows)$
102+
103103 // Visibility rule for backwards navigation arrows; "faded", "hidden"
104104 // or "visible"
105105 controlsBackArrows: '$controlsBackArrows$',
106−$endif$
107−$if(progress)$
106+
108107 // Display a presentation progress bar
109108 progress: $progress$,
110−$endif$
111−$if(slideNumber)$
109+
112110 // Display the page number of the current slide
113111 slideNumber: $slideNumber$,
114−$endif$
115−$if(showSlideNumber)$
112+
116113 // 'all', 'print', or 'speaker'
117114 showSlideNumber: '$showSlideNumber$',
118−$endif$
119−$if(hash)$
115+
120116 // Add the current slide number to the URL hash so that reloading the
121117 // page/copying the URL will return you to the same slide
122118 hash: $hash$,
123−$endif$
124−$if(hashOneBasedIndex)$
119+
125120 // Start with 1 for the hash rather than 0
126121 hashOneBasedIndex: $hashOneBasedIndex$,
127−$endif$
122+
123+ // Flags if we should monitor the hash and change slides accordingly
124+ respondToHashChanges: $respondToHashChanges$,
125+
128126 // Push each slide change to the browser history
129− history: $if(history)$$history$$else$true$endif$,
130−$if(keyboard)$
127+ history: $history$,
128+
131129 // Enable keyboard shortcuts for navigation
132130 keyboard: $keyboard$,
133−$endif$
134−$if(overview)$
131+
135132 // Enable the slide overview mode
136133 overview: $overview$,
137−$endif$
138−$if(center)$
134+
135+ // Disables the default reveal.js slide layout (scaling and centering)
136+ // so that you can use custom CSS layout
137+ disableLayout: false,
138+
139139 // Vertical centering of slides
140140 center: $center$,
141−$endif$
142−$if(touch)$
141+
143142 // Enables touch navigation on devices with touch input
144143 touch: $touch$,
145−$endif$
146−$if(loop)$
144+
147145 // Loop the presentation
148146 loop: $loop$,
149−$endif$
150−$if(rtl)$
147+
151148 // Change the presentation direction to be RTL
152149 rtl: $rtl$,
153−$endif$
154−$if(navigationMode)$
150+
155151 // see https://revealjs.com/vertical-slides/#navigation-mode
156152 navigationMode: '$navigationMode$',
157−$endif$
158−$if(shuffle)$
153+
159154 // Randomizes the order of slides each time the presentation loads
160155 shuffle: $shuffle$,
161−$endif$
162−$if(fragments)$
156+
163157 // Turns fragments on and off globally
164158 fragments: $fragments$,
165−$endif$
166−$if(fragmentInURL)$
159+
167160 // Flags whether to include the current fragment in the URL,
168161 // so that reloading brings you to the same fragment position
169162 fragmentInURL: $fragmentInURL$,
170−$endif$
171−$if(embedded)$
163+
172164 // Flags if the presentation is running in an embedded mode,
173165 // i.e. contained within a limited portion of the screen
174166 embedded: $embedded$,
175−$endif$
176−$if(help)$
167+
177168 // Flags if we should show a help overlay when the questionmark
178169 // key is pressed
179170 help: $help$,
180−$endif$
181−$if(pause)$
182− // Flags if it should be possible to pause the presentation (blackout)
183− pause: $pause$,
184−$endif$
185−$if(showNotes)$
171+
172+ // Flags if it should be possible to pause the presentation (blackout)
173+ pause: $pause$,
174+
186175 // Flags if speaker notes should be visible to all viewers
187176 showNotes: $showNotes$,
188−$endif$
189−$if(autoPlayMedia)$
190− // Global override for autoplaying embedded media (video/audio/iframe)
191− // - null: Media will only autoplay if data-autoplay is present
192− // - true: All media will autoplay, regardless of individual setting
193− // - false: No media will autoplay, regardless of individual setting
177+
178+ // Global override for autoplaying embedded media (null/true/false)
194179 autoPlayMedia: $autoPlayMedia$,
195−$endif$
196−$if(preloadIframes)$
197− // Global override for preloading lazy-loaded iframes
198− // - null: Iframes with data-src AND data-preload will be loaded when within
199− // the viewDistance, iframes with only data-src will be loaded when visible
200− // - true: All iframes with data-src will be loaded when within the viewDistance
201− // - false: All iframes with data-src will be loaded only when visible
180+
181+ // Global override for preloading lazy-loaded iframes (null/true/false)
202182 preloadIframes: $preloadIframes$,
203−$endif$
204−$if(autoSlide)$
183+
205184 // Number of milliseconds between automatically proceeding to the
206185 // next slide, disabled when set to 0, this value can be overwritten
207186 // by using a data-autoslide attribute on your slides
208187 autoSlide: $autoSlide$,
209−$endif$
210−$if(autoSlideStoppable)$
188+
211189 // Stop auto-sliding after user input
212190 autoSlideStoppable: $autoSlideStoppable$,
213−$endif$
214−$if(autoSlideMethod)$
191+
215192 // Use this method for navigation when auto-sliding
216193 autoSlideMethod: $autoSlideMethod$,
217−$endif$
218−$if(defaultTiming)$
194+
219195 // Specify the average time in seconds that you think you will spend
220196 // presenting each slide. This is used to show a pacing timer in the
221197 // speaker view
222198 defaultTiming: $defaultTiming$,
223−$endif$
224−$if(totalTime)$
225− // Specify the total time in seconds that is available to
226− // present. If this is set to a nonzero value, the pacing
227− // timer will work out the time available for each slide,
228− // instead of using the defaultTiming value
229− totalTime: $totalTime$,
230−$endif$
231−$if(minimumTimePerSlide)$
232− // Specify the minimum amount of time you want to allot to
233− // each slide, if using the totalTime calculation method. If
234− // the automated time allocation causes slide pacing to fall
235− // below this threshold, then you will see an alert in the
236− // speaker notes window
237− minimumTimePerSlide: $minimumTimePerSlide$,
238−$endif$
239−$if(mouseWheel)$
199+
240200 // Enable slide navigation via mouse wheel
241201 mouseWheel: $mouseWheel$,
242−$endif$
243−$if(rollingLinks)$
244− // Apply a 3D roll to links on hover
245− rollingLinks: $rollingLinks$,
246−$endif$
247−$if(hideInactiveCursor)$
202+
203+ // The display mode that will be used to show slides
204+ display: '$display$',
205+
248206 // Hide cursor if inactive
249207 hideInactiveCursor: $hideInactiveCursor$,
250−$endif$
251−$if(hideCursorTime)$
208+
252209 // Time before the cursor is hidden (in ms)
253210 hideCursorTime: $hideCursorTime$,
254−$endif$
255−$if(hideAddressBar)$
256− // Hides the address bar on mobile devices
257− hideAddressBar: $hideAddressBar$,
258−$endif$
259−$if(previewLinks)$
211+
260212 // Opens links in an iframe preview overlay
261213 previewLinks: $previewLinks$,
262−$endif$
263−$if(transition)$
264− // Transition style
265− transition: '$transition$', // none/fade/slide/convex/concave/zoom
266−$endif$
267−$if(transitionSpeed)$
268− // Transition speed
269− transitionSpeed: '$transitionSpeed$', // default/fast/slow
270−$endif$
271−$if(backgroundTransition)$
214+
215+ // Transition style (none/fade/slide/convex/concave/zoom)
216+ transition: '$transition$',
217+
218+ // Transition speed (default/fast/slow)
219+ transitionSpeed: '$transitionSpeed$',
220+
272221 // Transition style for full page slide backgrounds
273− backgroundTransition: '$backgroundTransition$', // none/fade/slide/convex/concave/zoom
274−$endif$
275−$if(viewDistance)$
222+ // (none/fade/slide/convex/concave/zoom)
223+ backgroundTransition: '$backgroundTransition$',
224+
276225 // Number of slides away from the current that are visible
277226 viewDistance: $viewDistance$,
278−$endif$
279−$if(mobileViewDistance)$
227+
280228 // Number of slides away from the current that are visible on mobile
281229 // devices. It is advisable to set this to a lower number than
282230 // viewDistance in order to save resources.
283231 mobileViewDistance: $mobileViewDistance$,
284−$endif$
285232$if(parallaxBackgroundImage)$
233+
286234 // Parallax background image
287235 parallaxBackgroundImage: '$parallaxBackgroundImage$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
288236$else$
289237$if(background-image)$
238+
290239 // Parallax background image
291240 parallaxBackgroundImage: '$background-image$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
292241$endif$
293242$endif$
294243$if(parallaxBackgroundSize)$
244+
295245 // Parallax background size
296246 parallaxBackgroundSize: '$parallaxBackgroundSize$', // CSS syntax, e.g. "2100px 900px"
297247$endif$
298248$if(parallaxBackgroundHorizontal)$
249+
299250 // Amount to move parallax background (horizontal and vertical) on slide change
300251 // Number, e.g. 100
301252 parallaxBackgroundHorizontal: $parallaxBackgroundHorizontal$,
302253$endif$
303254$if(parallaxBackgroundVertical)$
255+
304256 parallaxBackgroundVertical: $parallaxBackgroundVertical$,
305257$endif$
306258$if(width)$
259+
307260 // The "normal" size of the presentation, aspect ratio will be preserved
308261 // when the presentation is scaled to fit different resolutions. Can be
309262 // specified using percentage units.
310263 width: $width$,
311264$endif$
312265$if(height)$
266+
313267 height: $height$,
314268$endif$
315269$if(margin)$
270+
316271 // Factor of the display size that should remain empty around the content
317272 margin: $margin$,
318273$endif$
319274$if(minScale)$
275+
320276 // Bounds for smallest/largest possible scale to apply to content
321277 minScale: $minScale$,
322278$endif$
323279$if(maxScale)$
280+
324281 maxScale: $maxScale$,
325282$endif$
326−$if(zoomKey)$
327− // Modifier key used to click-zoom to part of the slide
328− zoomKey: '$zoomKey$',
329−$endif$
330−$if(display)$
331− // The display mode that will be used to show slides
332− display: '$display$',
333−$endif$
334283$if(mathjax)$
284+
335285 math: {
336286 mathjax: '$mathjaxurl$',
337287 config: 'TeX-AMS_HTML-full',
data/templates/default.s5+3
@@ -61,6 +61,9 @@ $endif$
6161$if(author)$
6262 <h3 class="author">$for(author)$$author$$sep$<br/>$endfor$</h3>
6363$endif$
64+$if(institute)$
65+ <h3 class="institute">$for(institute)$$institute$$sep$<br/>$endfor$</h3>
66+$endif$
6467$if(date)$
6568 <h4 class="date">$date$</h4>
6669$endif$
data/templates/default.slideous+7
@@ -63,9 +63,16 @@ $if(title)$
6363$if(subtitle)$
6464 <h1 class="subtitle">$subtitle$</h1>
6565$endif$
66+$if(author)$
6667 <p class="author">
6768$for(author)$$author$$sep$<br/>$endfor$
6869 </p>
70+$endif$
71+$if(institute)$
72+ <p class="institute">
73+$for(institute)$$institute$$sep$<br/>$endfor$
74+ </p>
75+$endif$
6976$if(date)$
7077 <p class="date">$date$</p>
7178$endif$
data/templates/default.slidy+5
@@ -52,6 +52,11 @@ $if(author)$
5252$for(author)$$author$$sep$<br/>$endfor$
5353 </p>
5454$endif$
55+$if(institute)$
56+ <p class="institute">
57+$for(institute)$$institute$$sep$<br/>$endfor$
58+ </p>
59+$endif$
5560$if(date)$
5661 <p class="date">$date$</p>
5762$endif$
data/templates/styles.html+2−1
@@ -14,7 +14,7 @@ body {
1414 padding-top: $if(margin-top)$$margin-top$$else$50px$endif$;
1515 padding-bottom: $if(margin-bottom)$$margin-bottom$$else$50px$endif$;
1616 hyphens: auto;
17− word-wrap: break-word;
17+ overflow-wrap: break-word;
1818 text-rendering: optimizeLegibility;
1919 font-kerning: normal;
2020}
@@ -93,6 +93,7 @@ $endif$
9393pre code {
9494 padding: 0;
9595 overflow: visible;
96+ overflow-wrap: normal;
9697}
9798.sourceCode {
9899 background-color: transparent;
doc/epub.md+9−2
@@ -141,8 +141,15 @@ Pandoc has an EPUB3 writer. It renders LaTeX math into MathML, which
141141EPUB3 readers are supposed to support (but unfortunately few do).
142142
143143Of course, this isn't much help if you want EPUB2 output (`pandoc -t epub2`)
144−or target readers that don't support MathML. Then you should try using the
145−`--webtex` option, which will use a web service to convert the TeX to an image.
144+or target readers that don't support MathML. Then you have two options:
145+
146+1. Use the [`--webtex`](https://pandoc.org/MANUAL.html#option--webtex) option,
147+ which will use a web service to convert the TeX to an image.
148+2. Use the [`--gladtex`](https://pandoc.org/MANUAL.html#option--gladtex) option
149+ to convert maths into SVG images on your local machine.
150+
151+Both GladTeX and WebTeX add the LaTeX source of the formula as alternative text
152+of the image, increasing accessibility for blind users.
146153
147154[KindleGen]: https://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000765211
148155[EPUB]: https://en.wikipedia.org/wiki/EPUB
doc/faqs.mdnew file+128
@@ -0,0 +1,128 @@
1+% FAQs
2+
3+::: faqs
4+
5+## How can I convert a whole directory of files from Markdown to RTF?
6+
7+On linux or OSX:
8+
9+ for f in *.txt; do pandoc "$f" -s -o "${f%.txt}.rtf"; done
10+
11+
12+In Windows Powershell:
13+
14+ gci -r -i *.txt |foreach{$rtf=$_.directoryname+"\"+$_.basename+".rtf";pandoc -f markdown -s $_.name -o $rtf}
15+
16+## I used pandoc to convert a document to ICML (or OPML or RTF), and when I try to open it I'm told it's invalid. What have I done wrong?
17+
18+Be sure to use the `-s` or `--standalone` flag, or you just get a
19+fragment, not a full document with the required header:
20+
21+ pandoc -s -f markdown -t icml -o my.icml my.md
22+
23+## I get a blank document when I try to convert a markdown document in Chinese to PDF.
24+
25+By default, pandoc uses pdflatex to generate the PDF, and pdflatex
26+doesn't handle Chinese characters. But you can change the default to
27+use xelatex instead. You should also make sure you're using a font
28+with Chinese glyphs. For example:
29+
30+ pandoc -o c.pdf --pdf-engine=xelatex -V mainfont='Adobe Ming Std'
31+
32+## The Windows installer does a single user install, rather than installing pandoc for all users. How can I install pandoc for all users?
33+
34+Run the following command as admin:
35+
36+ msiexec /i pandoc-VERSION.msi ALLUSERS=1
37+
38+This will put pandoc in `C:\Program Files\Pandoc`.
39+You can install Pandoc to a different directory by setting APPLICATIONFOLDER parameter,
40+for example:
41+
42+ msiexec /i pandoc-1.11.1.msi ALLUSERS=1 APPLICATIONFOLDER="C:\Pandoc"
43+
44+## How do I change the margins in PDF output?
45+
46+The option
47+
48+ -V geometry:margin=1in
49+
50+will set the margins to one inch on each side. If you don't want uniform
51+margins, you can do something like
52+
53+ -V geometry:"top=2cm, bottom=1.5cm, left=1cm, right=1cm"
54+
55+Or
56+
57+ -V geometry:"left=3cm, width=10cm"
58+
59+For more options, see the documentation for the LaTeX [geometry
60+package](https://www.ctan.org/pkg/geometry).
61+
62+## How does pandoc compare to multimarkdown?
63+
64+Here is a [wiki
65+page](https://github.com/jgm/pandoc/wiki/Pandoc-vs-Multimarkdown)
66+comparing the two.
67+
68+## When I specify an image width of 50% and convert to LaTeX, pandoc sets the height to textheight and the aspect ratio isn't preserved. How can I prevent this?
69+
70+For example, if you convert an image with `{width="50%"}`, the LaTeX produced
71+will be `\includegraphics[width=0.5\textwidth,height=\textheight]`.
72+
73+This output presupposes the following code in pandoc's default latex
74+template:
75+
76+```
77+% Scale images if necessary, so that they will not overflow the page
78+% margins by default, and it is still possible to overwrite the defaults
79+% using explicit options in \includegraphics[width, height, ...]{}
80+\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
81+```
82+
83+If you don't have this in your custom template, you should
84+add it. If we didn't set the `height` explicitly in this way,
85+the image would not be resized correctly unless it was
86+being resized to smaller than its original size.
87+
88+## Pandoc sometimes uses too much memory. How can I limit the memory used by pandoc?
89+
90+`pandoc +RTS -M30m -RTS` will limit heap memory to 30MB.
91+When converting a document requires more than this, an out of
92+memory error will be issued.
93+
94+## When using `--include-in-header` with PDF or LaTeX output, how do I reference tex declarations coming after `$header-includes$` in the default template?
95+
96+For various reasons, the `$header-includes$` are not at the very
97+end of the LaTeX preamble. This poses a problem when the code
98+you are inserting depends on declarations in the preamble coming
99+after the `$header-includes$` location. For example, you might
100+want to reference the `\author` and `\title` metadata values
101+(set at the very bottom of the preamble) and present them in
102+margins. In that case you can wrap your code in `etoolbox`'s
103+`\AtEndPreamble`. The technique is demonstrated in [this
104+gist](https://gist.github.com/JohnLukeBentley/9dda6166b9ee5c4127afd2b8cd16b70a).
105+When using `\AtEndPreamble`, keep any `makeatletter` or
106+`makeatother` outside of the `\AtEndPreamble`, as shown in the
107+example.
108+
109+## How can I convert PDFs to other formats using pandoc?
110+
111+You can't. You can try opening the PDF in Word or Google Docs
112+and saving in a format from which pandoc can convert directly.
113+
114+## Do I really need to install a 1 GB TeX installation to produce a PDF using pandoc?
115+
116+No. You can get by with a relatively small TeX installation,
117+for example, by starting with MacTeX's Basic TeX distribution
118+and using the `tlmgr` tool to install a few packages required by pandoc
119+(see https://pandoc.org/MANUAL.html#creating-a-pdf).
120+
121+Or, you can produce PDFs via HTML and `wkhtmltopdf`,
122+or via groff ms and `pdfroff`. (These don't produce as nice
123+topography as TeX, particularly when it comes to math, but they
124+may be fine for many purposes.)
125+
126+
127+:::
128+
doc/filters.md+2
@@ -394,6 +394,8 @@ automatically assign it `Just` the target format or `Nothing`.
394394
395395We compile our script:
396396
397+ # first, make sure pandoc-types is installed:
398+ cabal install --lib pandoc-types --package-env .
397399 ghc --make handleRuby
398400
399401Then run it:
doc/jats.md+9
@@ -50,6 +50,13 @@ Metadata Values
5050 list must also occur as the `id` of an affiliation listed in
5151 the top-level `affiliation` list.
5252
53+ `equal-contrib`
54+ : boolean attribute used to mark authors who contributed
55+ equally to the work. The
56+ [`equal-contrib`][attr:equal-contrib] attribute is added
57+ to the author's [`<contrib>`] element if this is set to a
58+ truthy value.
59+
5360 `cor-id`
5461 : identifier linking to the contributor's correspondence
5562 information. The info itself must be stored in as an item in
@@ -355,6 +362,7 @@ Required metadata values:
355362[Ringgold]: https://ringgold.com/
356363[attr:content-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/content-type.html
357364[attr:date-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/date-type.html
365+[attr:equal-contrib]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/equal-contrib.html
358366[attr:fn-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/fn-type.html
359367[attr:institution-id-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/institution-id-type.html
360368[attr:iso-8601-date]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/iso-8601-date.html
@@ -398,6 +406,7 @@ Required metadata values:
398406[`<addr-line>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/addr-line.html
399407[`<aff>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/aff.html
400408[`<city>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/city.html
409+[`<contrib>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/contrib.html
401410[`<country>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/country.html
402411[`<institution-id>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-id.html
403412[`<institution-wrap>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-wrap.html
doc/libraries.md+4
@@ -15,6 +15,9 @@ pandoc:
1515[texmath]
1616: Conversion of math between tex, Word equation, MathML, and GNU eqn.
1717
18+[unicode-collation]
19+: Proper Unicode collation (sorting).
20+
1821[doclayout]
1922: Combinators for laying out a textual document, with support
2023 for line wrapping, tabular layout, and more.
@@ -56,4 +59,5 @@ pandoc:
5659[rfc5051]: https://hackage.haskell.org/package/rfc5051
5760[emojis]: https://hackage.haskell.org/package/emojis
5861[jira-wiki-markup]: https://hackage.haskell.org/package/jira-wiki-markup
62+[unicode-collation]: https://hackage.haskell.org/package/unicode-collation
5963
doc/lua-filters.md+2−2
@@ -109,7 +109,7 @@ element filtering function. In other words, filter entries will
109109be called for each corresponding element in the document,
110110getting the respective element as input.
111111
112−The return of a filter function must one of the following:
112+The return of a filter function must be one of the following:
113113
114114- nil: this means that the object should remain unchanged.
115115- a pandoc object: this must be of the same type as the input
@@ -3205,7 +3205,7 @@ Usage:
32053205
32063206# Module pandoc.List
32073207
3208−The this module defines pandoc's list type. It comes with useful
3208+This module defines pandoc's list type. It comes with useful
32093209methods and convenience functions.
32103210
32113211## Constructor
doc/org.md+16−2
@@ -25,8 +25,7 @@ The following export keywords are supported:
2525- LANGUAGE: document language; included as plain-text metadata
2626 field `lang`. The value should be a [BCP47 language tag].
2727
28−- SELECT_TAGS: tags which select a tree for export. Currently
29− *unsupported*.
28+- SELECT_TAGS: tags which select a tree for export.
3029
3130- EXCLUDE\_TAGS: tags which prevent a subtree from being
3231 exported. Fully supported.
@@ -37,6 +36,21 @@ The following export keywords are supported:
3736 defaults to stdout unless a target has to be given as a command
3837 line option.
3938
39+::: {.alert .alert-info}
40+Pandoc tries to be compatible with org-mode when exporting an org document. If
41+you find some behavior confusing, please do refer to org-mode
42+[Export-Settings](https://orgmode.org/manual/Export-Settings.html)
43+documentation. For example, a common confusion
44+([#3214](https://github.com/jgm/pandoc/issues/3214 "Problem with headers lower
45+then 3 in org-mode reader"), [#5169](https://github.com/jgm/pandoc/issues/5169
46+"org mode headings past level three converted to numbered outline list"),
47+[#6145](https://github.com/jgm/pandoc/issues/6145 "Headers 4 levels deep render
48+differently"), [#7236](https://github.com/jgm/pandoc/issues/7236 "In Org mode,
49+Header with level > 3 are not recognized as headers")) is treatment of headers
50+with level > 3 differently because org-mode sets `org-export-headline-levels`
51+(configurable with `#+OPTIONS: H:3`) to 3 by default.
52+:::
53+
4054[BCP47 language tag]: https://tools.ietf.org/html/bcp47
4155
4256Format-specific options
doc/using-the-pandoc-api.md+25−12
@@ -14,7 +14,7 @@ and types is available at
1414Pandoc is structured as a set of *readers*, which translate
1515various input formats into an abstract syntax tree (the
1616Pandoc AST) representing a structured document, and a set of
17−*writers*, which render this AST into various input formats.
17+*writers*, which render this AST into various output formats.
1818Pictorially:
1919
2020```
@@ -90,8 +90,14 @@ Some notes:
9090Let's look at the types of `readMarkdown` and `writeRST`:
9191
9292```haskell
93−readMarkdown :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
94−writeRST :: PandocMonad m => WriterOptions -> Pandoc -> m Text
93+readMarkdown :: (PandocMonad m, ToSources a)
94+ => ReaderOptions
95+ -> a
96+ -> m Pandoc
97+writeRST :: PandocMonad m
98+ => WriterOptions
99+ -> Pandoc
100+ -> m Text
95101```
96102
97103The `PandocMonad m =>` part is a typeclass constraint.
@@ -133,7 +139,7 @@ report :: PandocMonad m => LogMessage -> m ()
133139-- | Fetch an image or other item from the local filesystem or the net.
134140-- Returns raw content and maybe mime type.
135141fetchItem :: PandocMonad m
136− => String
142+ => Text
137143 -> m (B.ByteString, Maybe MimeType)
138144
139145-- Set the resource path searched by 'fetchItem'.
@@ -155,6 +161,13 @@ Note that `PandocIO` is an instance of `MonadIO`, so you can
155161use `liftIO` to perform arbitrary IO operations inside a pandoc
156162conversion chain.
157163
164+`readMarkdown` is polymorphic in its second argument, which
165+can be any type that is an instance of the `ToSources`
166+typeclass. You can use `Text`, as in the example above.
167+But you can also use `[(FilePath, Text)]`, if the input comes
168+from multiple files and you want to track source positions
169+accurately.
170+
158171# Options
159172
160173The first argument of each reader or writer is for
@@ -200,8 +213,8 @@ of the Monoid typeclass and can easily be concatenated:
200213import Text.Pandoc.Builder
201214
202215mydoc :: Pandoc
203−mydoc = doc $ header 1 (text "Hello!")
204− <> para (emph (text "hello world") <> text ".")
216+mydoc = doc $ header 1 (text (T.pack "Hello!"))
217+ <> para (emph (text (T.pack "hello world")) <> text (T.pack "."))
205218
206219main :: IO ()
207220main = print mydoc
@@ -248,16 +261,16 @@ import qualified Data.Text as T
248261import Data.List (intersperse)
249262
250263data Station = Station{
251− address :: String
252− , name :: String
253− , cardsAccepted :: [String]
264+ address :: T.Text
265+ , name :: T.Text
266+ , cardsAccepted :: [T.Text]
254267 } deriving Show
255268
256269instance FromJSON Station where
257270 parseJSON (Object v) = Station <$>
258271 v .: "street_address" <*>
259272 v .: "station_name" <*>
260− (words <$> (v .:? "cards_accepted" .!= ""))
273+ (T.words <$> (v .:? "cards_accepted" .!= ""))
261274 parseJSON _ = mzero
262275
263276createLetter :: [Station] -> Pandoc
@@ -315,7 +328,7 @@ users to override the system defaults. If you want to disable
315328this behavior, use `setUserDataDir Nothing`.
316329
317330To render a template, use `renderTemplate'`, which takes two
318−arguments, a template (String) and a context (any instance
331+arguments, a template (Text) and a context (any instance
319332of ToJSON). If you want to create a context from the metadata
320333part of a Pandoc document, use `metaToJSON'` from
321334[Text.Pandoc.Writers.Shared]. If you also want to incorporate
@@ -413,7 +426,7 @@ concatenated together. Here's an example that returns a
413426list of the URLs linked to in a document:
414427
415428```haskell
416−listURLs :: Pandoc -> [String]
429+listURLs :: Pandoc -> [Text]
417430listURLs = query urls
418431 where urls (Link _ _ (src, _)) = [src]
419432 urls _ = []
linux/make_artifacts.sh+19−7
@@ -2,13 +2,23 @@ set -e
22
33MACHINE=$(uname -m)
44case "$MACHINE" in
5− x86_64) ARCHITECTURE=amd64;;
6− i686) ARCHITECTURE=i386;;
7− i386) ARCHITECTURE=i386;;
5+ x86_64) ARCHITECTURE=amd64;;
6+ i686) ARCHITECTURE=i386;;
7+ i386) ARCHITECTURE=i386;;
8+ aarch64) ARCHITECTURE=arm64;;
9+ *) ARCHITECTURE=unknown;;
810esac
911
1012ARTIFACTS="${ARTIFACTS:-/artifacts}"
1113
14+# This is our sentinel that tells us when we're done.
15+rm -f $ARTIFACTS/DONE
16+
17+clean_up() {
18+ echo "All done!" > "$ARTIFACTS/DONE"
19+}
20+trap clean_up EXIT
21+
1222# build binaries
1323
1424cabal --version
@@ -16,9 +26,9 @@ ghc --version
1626
1727cabal v2-update
1828cabal v2-clean
19−cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-split-sections -optc-Os -optl=-pthread' pandoc
20−cabal v2-build
21−cabal v2-test -j1
29+cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-j4 +RTS -A256m -RTS -split-sections -optc-Os -optl=-pthread' pandoc
30+cabal v2-build -j4
31+cabal v2-test -j4
2232for f in $(find dist-newstyle -name 'pandoc' -type f -perm /400); do cp $f /artifacts/; done
2333
2434# make deb
@@ -68,5 +78,7 @@ mv pandoc $TARGET/bin
6878strip $TARGET/bin/pandoc
6979gzip -9 $TARGET/share/man/man1/pandoc.1
7080
71−tar cvzf $TARGET-linux-amd64.tar.gz $TARGET
81+tar cvzf $TARGET-linux-$ARCHITECTURE.tar.gz $TARGET
7282rm -r $TARGET
83+
84+exit 0
man/pandoc.1+233−49
@@ -1,7 +1,7 @@
11'\" t
2−.\" Automatically generated by Pandoc 2.11.4
2+.\" Automatically generated by Pandoc 2.14.0.1
33.\"
4−.TH "Pandoc User\[cq]s Guide" "" "March 4, 2021" "pandoc 2.12" ""
4+.TH "Pandoc User\[cq]s Guide" "" "June 20, 2021" "pandoc 2.14.0.3" ""
55.hy
66.SH NAME
77pandoc - general markup converter
@@ -457,16 +457,15 @@ On *nix and macOS systems this will be the \f[C]pandoc\f[R] subdirectory
457457of the XDG data directory (by default, \f[C]$HOME/.local/share\f[R],
458458overridable by setting the \f[C]XDG_DATA_HOME\f[R] environment
459459variable).
460−If that directory does not exist, \f[C]$HOME/.pandoc\f[R] will be used
461−(for backwards compatibility).
462−In Windows the default user data directory is
460+If that directory does not exist and \f[C]$HOME/.pandoc\f[R] exists, it
461+will be used (for backwards compatibility).
462+On Windows the default user data directory is
463463\f[C]C:\[rs]Users\[rs]USERNAME\[rs]AppData\[rs]Roaming\[rs]pandoc\f[R].
464464You can find the default user data directory on your system by looking
465465at the output of \f[C]pandoc --version\f[R].
466−A \f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
467−\f[C]templates\f[R], \f[C]slidy\f[R], \f[C]slideous\f[R], or
468−\f[C]s5\f[R] directory placed in this directory will override
469−pandoc\[cq]s normal defaults.
466+Data files placed in this directory (for example,
467+\f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
468+\f[C]templates\f[R]) will override pandoc\[cq]s normal defaults.
470469.TP
471470\f[B]\f[CB]-d\f[B]\f[R] \f[I]FILE\f[R], \f[B]\f[CB]--defaults=\f[B]\f[R]\f[I]FILE\f[R]
472471Specify a set of default option settings.
@@ -717,11 +716,11 @@ Extract images and other media contained in or linked from the source
717716document to the path \f[I]DIR\f[R], creating it if necessary, and adjust
718717the images references in the document so they point to the extracted
719718files.
720−If the source format is a binary container (docx, epub, or odt), the
721−media is extracted from the container and the original filenames are
722−used.
723−Otherwise the media is read from the file system or downloaded, and new
724−filenames are constructed based on SHA1 hashes of the contents.
719+Media are downloaded, read from the file system, or extracted from a
720+binary container (e.g.\ docx), as needed.
721+The original file paths are used if they are relative paths not
722+containing \f[C]..\f[R].
723+Otherwise filenames are constructed from the SHA1 hash of the contents.
725724.TP
726725\f[B]\f[CB]--abbreviations=\f[B]\f[R]\f[I]FILE\f[R]
727726Specifies a custom abbreviations file, with abbreviations one to a line.
@@ -731,9 +730,9 @@ system default.
731730To see the system default, use
732731\f[C]pandoc --print-default-data-file=abbreviations\f[R].
733732The only use pandoc makes of this list is in the Markdown reader.
734−Strings ending in a period that are found in this list will be followed
735−by a nonbreaking space, so that the period will not produce
736−sentence-ending space in formats like LaTeX.
733+Strings found in this list will be followed by a nonbreaking space, and
734+the period will not produce sentence-ending space in formats like LaTeX.
735+The strings may not contain spaces.
737736.SS General writer options
738737.TP
739738\f[B]\f[CB]-s\f[B]\f[R], \f[B]\f[CB]--standalone\f[B]\f[R]
@@ -924,6 +923,11 @@ Note that, if \f[C]--resource-path\f[R] is specified, the working
924923directory must be explicitly listed or it will not be searched.
925924For example: \f[C]--resource-path=.:test\f[R] will search the working
926925directory and the \f[C]test\f[R] subdirectory, in that order.
926+This option can be used repeatedly.
927+Search path components that come later on the command line will be
928+searched before those that come earlier, so
929+\f[C]--resource-path foo:bar --resource-path baz:bim\f[R] is equivalent
930+to \f[C]--resource-path baz:bim:foo:bar\f[R].
927931.TP
928932\f[B]\f[CB]--request-header=\f[B]\f[R]\f[I]NAME\f[R]\f[B]\f[CB]:\f[B]\f[R]\f[I]VAL\f[R]
929933Set the request header \f[I]NAME\f[R] to the value \f[I]VAL\f[R] when
@@ -1500,16 +1504,16 @@ inserted.
15001504.TP
15011505\f[B]\f[CB]--gladtex\f[B]\f[R]
15021506Enclose TeX math in \f[C]<eq>\f[R] tags in HTML output.
1503−The resulting HTML can then be processed by GladTeX to produce images of
1504−the typeset formulas and an HTML file with links to these images.
1505−So, the procedure is:
1507+The resulting HTML can then be processed by GladTeX to produce SVG
1508+images of the typeset formulas and an HTML file with these images
1509+embedded.
15061510.RS
15071511.IP
15081512.nf
15091513\f[C]
15101514pandoc -s --gladtex input.md -o myfile.htex
1511−gladtex -d myfile-images myfile.htex
1512−# produces myfile.html and images in myfile-images
1515+gladtex -d image_dir myfile.htex
1516+# produces myfile.html and images in image_dir
15131517\f[R]
15141518.fi
15151519.RE
@@ -1887,6 +1891,38 @@ verbosity: INFO
18871891\f[R]
18881892.fi
18891893.PP
1894+In fields that expect a file path (or list of file paths), the following
1895+syntax may be used to interpolate environment variables:
1896+.IP
1897+.nf
1898+\f[C]
1899+csl: ${HOME}/mycsldir/special.csl
1900+\f[R]
1901+.fi
1902+.PP
1903+\f[C]${USERDATA}\f[R] may also be used; this will always resolve to the
1904+user data directory that is current when the defaults file is parsed,
1905+regardless of the setting of the environment variable
1906+\f[C]USERDATA\f[R].
1907+.PP
1908+\f[C]${.}\f[R] will resolve to the directory containing the default file
1909+itself.
1910+This allows you to refer to resources contained in that directory:
1911+.IP
1912+.nf
1913+\f[C]
1914+epub-cover-image: ${.}/cover.jpg
1915+epub-metadata: ${.}/meta.xml
1916+resource-path:
1917+- . # the working directory from which pandoc is run
1918+- ${.}/images # the images subdirectory of the directory
1919+ # containing this defaults file
1920+\f[R]
1921+.fi
1922+.PP
1923+This environment variable interpolation syntax \f[I]only\f[R] works in
1924+fields that expect file paths.
1925+.PP
18901926Default files can be placed in the \f[C]defaults\f[R] subdirectory of
18911927the user data directory and used from any directory.
18921928For example, one could create a file specifying defaults for writing
@@ -2555,10 +2591,9 @@ YAML metadata or with \f[C]-M classoption=fleqn\f[R].
25552591.SS Variables for HTML slides
25562592.PP
25572593These affect HTML output when [producing slide shows with pandoc].
2558−.PP
2559−All reveal.js configuration options are available as variables.
2560−To turn off boolean flags that default to true in reveal.js, use
2561−\f[C]0\f[R].
2594+.TP
2595+\f[B]\f[CB]institute\f[B]\f[R]
2596+author affiliations: can be a list when there are multiple authors
25622597.TP
25632598\f[B]\f[CB]revealjs-url\f[B]\f[R]
25642599base URL for reveal.js documents (defaults to
@@ -2577,6 +2612,10 @@ base URL for Slideous documents (defaults to \f[C]slideous\f[R])
25772612\f[B]\f[CB]title-slide-attributes\f[B]\f[R]
25782613additional attributes for the title slide of reveal.js slide shows.
25792614See background in reveal.js and beamer for an example.
2615+.PP
2616+All reveal.js configuration options are available as variables.
2617+To turn off boolean flags that default to true in reveal.js, use
2618+\f[C]0\f[R].
25802619.SS Variables for Beamer slides
25812620.PP
25822621These variables change the appearance of PDF slides using
@@ -4236,7 +4275,7 @@ Term 2
42364275Note that space between items in a definition list is required.
42374276(A variant that loosens this requirement, but disallows \[lq]lazy\[rq]
42384277hard wrapping, can be activated with \f[C]compact_definition_lists\f[R]:
4239−see Non-pandoc extensions, below.)
4278+see Non-default extensions, below.)
42404279.SS Numbered example lists
42414280.SS Extension: \f[C]example_lists\f[R]
42424281.PP
@@ -4757,7 +4796,9 @@ All of the metadata will appear in a single block at the beginning of
47574796the document.
47584797.PP
47594798Note that YAML escaping rules must be followed.
4760−Thus, for example, if a title contains a colon, it must be quoted.
4799+Thus, for example, if a title contains a colon, it must be quoted, and
4800+if it contains a backslash escape, then it must be ensured that it is
4801+not treated as a YAML escape sequence.
47614802The pipe character (\f[C]|\f[R]) can be used to begin an indented block
47624803that will be interpreted literally, without need for escaping.
47634804This form is necessary when the field contains blank lines or
@@ -4779,6 +4820,13 @@ abstract: |
47794820\f[R]
47804821.fi
47814822.PP
4823+The literal block after the \f[C]|\f[R] must be indented relative to the
4824+line containing the \f[C]|\f[R].
4825+If it is not, the YAML will be invalid and pandoc will not interpret it
4826+as metadata.
4827+For an overview of the complex rules governing YAML, see the Wikipedia
4828+entry on YAML syntax.
4829+.PP
47824830Template variables will be set automatically from the metadata.
47834831Thus, for example, in writing HTML, the variable \f[C]abstract\f[R] will
47844832be set to the HTML equivalent of the Markdown in the \f[C]abstract\f[R]
@@ -4844,6 +4892,21 @@ header-includes:
48444892 \[ga]\[ga]\[ga]
48454893\f[R]
48464894.fi
4895+.PP
4896+Note: the \f[C]yaml_metadata_block\f[R] extension works with
4897+\f[C]commonmark\f[R] as well as \f[C]markdown\f[R] (and it is enabled by
4898+default in \f[C]gfm\f[R] and \f[C]commonmark_x\f[R]).
4899+However, in these formats the following restrictions apply:
4900+.IP \[bu] 2
4901+The YAML metadata block must occur at the beginning of the document (and
4902+there can be only one).
4903+If multiple files are given as arguments to pandoc, only the first can
4904+be a YAML metadata block.
4905+.IP \[bu] 2
4906+The leaf nodes of the YAML structure are parsed in isolation from each
4907+other and from the rest of the document.
4908+So, for example, you can\[cq]t use a reference link in these contexts if
4909+the link definition is somewhere else in the document.
48474910.SS Backslash escapes
48484911.SS Extension: \f[C]all_symbols_escapable\f[R]
48494912.PP
@@ -5724,27 +5787,70 @@ Inline and regular footnotes may be mixed freely.
57245787.SS Citation syntax
57255788.SS Extension: \f[C]citations\f[R]
57265789.PP
5727−Markdown citations go inside square brackets and are separated by
5728−semicolons.
5729−Each citation must have a key, composed of `\[at]' + the citation
5730−identifier from the database, and may optionally have a prefix, a
5731−locator, and a suffix.
5732−The citation key must begin with a letter, digit, or \f[C]_\f[R], and
5733−may contain alphanumerics, \f[C]_\f[R], and internal punctuation
5734−characters (\f[C]:.#$%&-+?<>\[ti]/\f[R]).
5735−Here are some examples:
5790+To cite a bibliographic item with an identifier foo, use the syntax
5791+\f[C]\[at]foo\f[R].
5792+Normal citations should be included in square brackets, with semicolons
5793+separating distinct items:
57365794.IP
57375795.nf
57385796\f[C]
5739−Blah blah [see \[at]doe99, pp. 33-35; also \[at]smith04, chap. 1].
5740−
5741−Blah blah [\[at]doe99, pp. 33-35, 38-39 and *passim*].
5797+Blah blah [\[at]doe99; \[at]smith2000; \[at]smith2004].
5798+\f[R]
5799+.fi
5800+.PP
5801+How this is rendered depends on the citation style.
5802+In an author-date style, it might render as
5803+.IP
5804+.nf
5805+\f[C]
5806+Blah blah (Doe 1999, Smith 2000, 2004).
5807+\f[R]
5808+.fi
5809+.PP
5810+In a footnote style, it might render as
5811+.IP
5812+.nf
5813+\f[C]
5814+Blah blah.[\[ha]1]
57425815
5743−Blah blah [\[at]smith04; \[at]doe99].
5816+[\[ha]1]: John Doe, \[dq]Frogs,\[dq] *Journal of Amphibians* 44 (1999);
5817+Susan Smith, \[dq]Flies,\[dq] *Journal of Insects* (2000);
5818+Susan Smith, \[dq]Bees,\[dq] *Journal of Insects* (2004).
5819+\f[R]
5820+.fi
5821+.PP
5822+See the CSL user documentation for more information about CSL styles and
5823+how they affect rendering.
5824+.PP
5825+Unless a citation key start with a letter, digit, or \f[C]_\f[R], and
5826+contains only alphanumerics and internal punctuation characters
5827+(\f[C]:.#$%&-+?<>\[ti]/\f[R]), it must be surrounded by curly braces,
5828+which are not considered part of the key.
5829+In \f[C]\[at]Foo_bar.baz.\f[R], the key is \f[C]Foo_bar.baz\f[R].
5830+The final period is not \f[I]internal\f[R] punctuation, so it is not
5831+included in the key.
5832+In \f[C]\[at]{Foo_bar.baz.}\f[R], the key is \f[C]Foo_bar.baz.\f[R],
5833+including the final period.
5834+The curly braces are recommended if you use URLs as keys:
5835+\f[C][\[at]{https://example.com/bib?name=foobar&date=2000}, p. 33]\f[R].
5836+.PP
5837+Citation items may optionally include a prefix, a locator, and a suffix.
5838+In
5839+.IP
5840+.nf
5841+\f[C]
5842+Blah blah [see \[at]doe99, pp. 33-35 and *passim*; \[at]smith04, chap. 1].
57445843\f[R]
57455844.fi
57465845.PP
5747−\f[C]pandoc\f[R] detects locator terms in the CSL locale files.
5846+The first item (\f[C]doe99\f[R]) has prefix \f[C]see\f[R], locator
5847+\f[C]pp. 33-35\f[R], and suffix \f[C]and *passim*\f[R].
5848+The second item (\f[C]smith04\f[R]) has locator \f[C]chap. 1\f[R] and no
5849+prefix or suffix.
5850+.PP
5851+Pandoc uses some heuristics to separate the locator from the rest of the
5852+subject.
5853+It is sensitive to the locator terms defined in the CSL locale files.
57485854Either abbreviated or unabbreviated forms are accepted.
57495855In the \f[C]en-US\f[R] locale, locator terms can be written in either
57505856singular or plural forms, as \f[C]book\f[R],
@@ -5766,14 +5872,15 @@ singular or plural forms, as \f[C]book\f[R],
57665872\f[C]\[sc]\f[R]/\f[C]\[sc]\[sc]\f[R].
57675873If no locator term is used, \[lq]page\[rq] is assumed.
57685874.PP
5769−\f[C]pandoc\f[R] will use heuristics to distinguish the locator from the
5770−suffix.
5771−In complex cases, the locator can be enclosed in curly braces:
5875+In complex cases, you can force something to be treated as a locator by
5876+enclosing it in curly braces or prevent parsing the suffix as locator by
5877+prepending curly braces:
57725878.IP
57735879.nf
57745880\f[C]
57755881[\[at]smith{ii, A, D-Z}, with a suffix]
57765882[\[at]smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5883+[\[at]smith{}, 99 years later]
57775884\f[R]
57785885.fi
57795886.PP
@@ -5787,7 +5894,8 @@ Smith says blah [-\[at]smith04].
57875894\f[R]
57885895.fi
57895896.PP
5790−You can also write an in-text citation, as follows:
5897+You can also write an author-in-text citation, by omitting the square
5898+brackets:
57915899.IP
57925900.nf
57935901\f[C]
@@ -5796,13 +5904,64 @@ You can also write an in-text citation, as follows:
57965904\[at]smith04 [p. 33] says blah.
57975905\f[R]
57985906.fi
5799−.SS Non-pandoc extensions
5907+.PP
5908+This will cause the author\[cq]s name to be rendered, followed by the
5909+bibliographical details.
5910+Use this form when you want to make the citation the subject of a
5911+sentence.
5912+.PP
5913+When you are using a note style, it is usually better to let citeproc
5914+create the footnotes from citations rather than writing an explicit
5915+note.
5916+If you do write an explicit note that contains a citation, note that
5917+normal citations will be put in parentheses, while author-in-text
5918+citations will not.
5919+For this reason, it is sometimes preferable to use the author-in-text
5920+style inside notes when using a note style.
5921+.SS Non-default extensions
58005922.PP
58015923The following Markdown syntax extensions are not enabled by default in
58025924pandoc, but may be enabled by adding \f[C]+EXTENSION\f[R] to the format
58035925name, where \f[C]EXTENSION\f[R] is the name of the extension.
58045926Thus, for example, \f[C]markdown+hard_line_breaks\f[R] is Markdown with
58055927hard line breaks.
5928+.SS Extension: \f[C]rebase_relative_paths\f[R]
5929+.PP
5930+Rewrite relative paths for Markdown links and images, depending on the
5931+path of the file containing the link or image link.
5932+For each link or image, pandoc will compute the directory of the
5933+containing file, relative to the working directory, and prepend the
5934+resulting path to the link or image path.
5935+.PP
5936+The use of this extension is best understood by example.
5937+Suppose you have a a subdirectory for each chapter of a book,
5938+\f[C]chap1\f[R], \f[C]chap2\f[R], \f[C]chap3\f[R].
5939+Each contains a file \f[C]text.md\f[R] and a number of images used in
5940+the chapter.
5941+You would like to have \f[C]![image](spider.jpg)\f[R] in
5942+\f[C]chap1/text.md\f[R] refer to \f[C]chap1/spider.jpg\f[R] and
5943+\f[C]![image](spider.jpg)\f[R] in \f[C]chap2/text.md\f[R] refer to
5944+\f[C]chap2/spider.jpg\f[R].
5945+To do this, use
5946+.IP
5947+.nf
5948+\f[C]
5949+pandoc chap*/*.md -f markdown+rebase_relative_paths
5950+\f[R]
5951+.fi
5952+.PP
5953+Without this extension, you would have to use
5954+\f[C]![image](chap1/spider.jpg)\f[R] in \f[C]chap1/text.md\f[R] and
5955+\f[C]![image](chap2/spider.jpg)\f[R] in \f[C]chap2/text.md\f[R].
5956+Links with relative paths will be rewritten in the same way as images.
5957+.PP
5958+Absolute paths and URLs are not changed.
5959+Neither are empty paths or paths consisting entirely of a fragment,
5960+e.g., \f[C]#foo\f[R].
5961+.PP
5962+Note that relative paths in reference links and images will be rewritten
5963+relative to the file containing the link reference definition, not the
5964+file containing the reference link or image itself, if these differ.
58065965.SS Extension: \f[C]attributes\f[R]
58075966.PP
58085967Allows attributes to be attached to any inline or block-level element.
@@ -6432,9 +6591,29 @@ entries (for author-date and numerical styles only).
64326591.TP
64336592\f[B]\f[CB]lang\f[B]\f[R]
64346593The \f[C]lang\f[R] field will affect how the style is localized, for
6435−example in the translation of labels and the use of quotation marks.
6594+example in the translation of labels, the use of quotation marks, and
6595+the way items are sorted.
64366596(For backwards compatibility, \f[C]locale\f[R] may be used instead of
64376597\f[C]lang\f[R], but this use is deprecated.)
6598+.RS
6599+.PP
6600+A BCP 47 language tag is expected: for example, \f[C]en\f[R],
6601+\f[C]de\f[R], \f[C]en-US\f[R], \f[C]fr-CA\f[R], \f[C]ug-Cyrl\f[R].
6602+The unicode extension syntax (after \f[C]-u-\f[R]) may be used to
6603+specify options for collation (sorting) more precisely.
6604+Here are some examples:
6605+.IP \[bu] 2
6606+\f[C]zh-u-co-pinyin\f[R] \[en] Chinese with the Pinyin collation.
6607+.IP \[bu] 2
6608+\f[C]es-u-co-trad\f[R] \[en] Spanish with the traditional collation
6609+(with \f[C]Ch\f[R] sorting after \f[C]C\f[R]).
6610+.IP \[bu] 2
6611+\f[C]fr-u-kb\f[R] \[en] French with \[lq]backwards\[rq] accent sorting
6612+(with \f[C]cot\['e]\f[R] sorting after \f[C]c\[^o]te\f[R]).
6613+.IP \[bu] 2
6614+\f[C]en-US-u-kf-upper\f[R] \[en] English with uppercase letters sorting
6615+before lower (default is lower before upper).
6616+.RE
64386617.TP
64396618\f[B]\f[CB]notes-after-punctuation\f[B]\f[R]
64406619If true (the default), pandoc will put footnote citations after
@@ -6555,6 +6734,11 @@ A heading at the slide level always starts a new slide.
65556734.IP \[bu] 2
65566735Headings \f[I]below\f[R] the slide level in the hierarchy create
65576736headings \f[I]within\f[R] a slide.
6737+(In beamer, a \[lq]block\[rq] will be created.
6738+If the heading has the class \f[C]example\f[R], an
6739+\f[C]exampleblock\f[R] environment will be used; if it has the class
6740+\f[C]alert\f[R], an \f[C]alertblock\f[R] will be used; otherwise a
6741+regular \f[C]block\f[R] will be used.)
65586742.IP \[bu] 2
65596743Headings \f[I]above\f[R] the slide level in the hierarchy create
65606744\[lq]title slides,\[rq] which just contain the section title and help to
@@ -7524,7 +7708,7 @@ The HTML generated by pandoc is not guaranteed to be safe.
75247708If \f[C]raw_html\f[R] is enabled for the Markdown input, users can
75257709inject arbitrary HTML.
75267710Even if \f[C]raw_html\f[R] is disabled, users can include dangerous
7527−content in attributes for headings, spans, and code blocks.
7711+content in URLs and attributes.
75287712To be safe, you should run all the generated HTML through an HTML
75297713sanitizer.
75307714.SH AUTHORS
pandoc.cabal+40−30
@@ -1,6 +1,6 @@
11cabal-version: 2.2
22name: pandoc
3−version: 2.12
3+version: 2.14.0.3
44build-type: Simple
55license: GPL-2.0-or-later
66license-file: COPYING.md
@@ -12,7 +12,7 @@ stability: alpha
1212homepage: https://pandoc.org
1313category: Text
1414tested-with: GHC == 8.2.2, GHC == 8.4.4, GHC == 8.6.5,
15− GHC == 8.8.4, GHC == 8.10.2
15+ GHC == 8.8.4, GHC == 8.10.2, GHC == 9.0.1
1616synopsis: Conversion between markup formats
1717description: Pandoc is a Haskell library for converting from one markup
1818 format to another, and a command-line tool that uses
@@ -208,12 +208,16 @@ extra-source-files:
208208 test/command/*.md
209209 test/command/*.csl
210210 test/command/biblio.bib
211− test/command/biblatex-examples.bib
211+ test/command/averroes.bib
212212 test/command/A.txt
213213 test/command/B.txt
214214 test/command/C.txt
215215 test/command/D.txt
216216 test/command/01.csv
217+ test/command/chap1/spider.png
218+ test/command/chap2/spider.png
219+ test/command/chap1/text.md
220+ test/command/chap2/text.md
217221 test/command/defaults1.yaml
218222 test/command/defaults2.yaml
219223 test/command/defaults3.yaml
@@ -398,8 +402,7 @@ flag trypandoc
398402
399403common common-options
400404 default-language: Haskell2010
401− build-depends: base >= 4.9 && < 5,
402− text >= 1.1.1.0 && < 1.3
405+ build-depends: base >= 4.9 && < 5
403406 ghc-options: -Wall -fno-warn-unused-do-bind
404407 -Wincomplete-record-updates
405408 -Wnoncanonical-monad-instances
@@ -427,7 +430,7 @@ common common-options
427430common common-executable
428431 import: common-options
429432 build-depends: pandoc
430− ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
433+ ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
431434
432435
433436library
@@ -439,29 +442,30 @@ library
439442 SHA >= 1.6 && < 1.7,
440443 aeson >= 0.7 && < 1.6,
441444 aeson-pretty >= 0.8.5 && < 0.9,
442− attoparsec >= 0.12 && < 0.14,
445+ array >= 0.5 && < 0.6,
446+ attoparsec >= 0.12 && < 0.15,
443447 base64-bytestring >= 0.1 && < 1.3,
444448 binary >= 0.7 && < 0.11,
445449 blaze-html >= 0.9 && < 0.10,
446450 blaze-markup >= 0.8 && < 0.9,
447451 bytestring >= 0.9 && < 0.12,
448452 case-insensitive >= 1.2 && < 1.3,
449− citeproc >= 0.3.0.7 && < 0.4,
450− commonmark >= 0.1.1.3 && < 0.2,
451− commonmark-extensions >= 0.2.0.4 && < 0.3,
452− commonmark-pandoc >= 0.2 && < 0.3,
453+ citeproc >= 0.4.0.1 && < 0.4.1,
454+ commonmark >= 0.2.1 && < 0.3,
455+ commonmark-extensions >= 0.2.1.2 && < 0.3,
456+ commonmark-pandoc >= 0.2.1.1 && < 0.3,
453457 connection >= 0.3.1,
454458 containers >= 0.4.2.1 && < 0.7,
455459 data-default >= 0.4 && < 0.8,
456460 deepseq >= 1.3 && < 1.5,
457461 directory >= 1.2.3 && < 1.4,
458− doclayout >= 0.3 && < 0.4,
462+ doclayout >= 0.3.0.1 && < 0.4,
459463 doctemplates >= 0.9 && < 0.10,
460464 emojis >= 0.1 && < 0.2,
461465 exceptions >= 0.8 && < 0.11,
462466 file-embed >= 0.0 && < 0.1,
463467 filepath >= 1.1 && < 1.5,
464− haddock-library >= 1.8 && < 1.10,
468+ haddock-library >= 1.10 && < 1.11,
465469 hslua >= 1.1 && < 1.4,
466470 hslua-module-path >= 0.1.0 && < 0.2.0,
467471 hslua-module-system >= 0.2 && < 0.3,
@@ -470,30 +474,31 @@ library
470474 http-client-tls >= 0.2.4 && < 0.4,
471475 http-types >= 0.8 && < 0.13,
472476 ipynb >= 0.1 && < 0.2,
473− jira-wiki-markup >= 1.3.3 && < 1.4,
477+ jira-wiki-markup >= 1.4 && < 1.5,
474478 mtl >= 2.2 && < 2.3,
475479 network >= 2.6,
476− network-uri >= 2.6 && < 2.7,
480+ network-uri >= 2.6 && < 2.8,
477481 pandoc-types >= 1.22 && < 1.23,
478482 parsec >= 3.1 && < 3.2,
479483 process >= 1.2.3 && < 1.7,
480− random >= 1 && < 1.2,
481− safe >= 0.3 && < 0.4,
484+ random >= 1 && < 1.3,
485+ safe >= 0.3.18 && < 0.4,
482486 scientific >= 0.3 && < 0.4,
483− skylighting >= 0.10.4 && < 0.11,
484− skylighting-core >= 0.10.4 && < 0.11,
487+ skylighting >= 0.10.5.2 && < 0.10.6,
488+ skylighting-core >= 0.10.5.2 && < 0.10.6,
485489 split >= 0.2 && < 0.3,
486490 syb >= 0.1 && < 0.8,
487491 tagsoup >= 0.14.6 && < 0.15,
488492 temporary >= 1.1 && < 1.4,
489− texmath >= 0.12.1 && < 0.13,
493+ texmath >= 0.12.3 && < 0.12.4,
490494 text >= 1.1.1.0 && < 1.3,
491495 text-conversions >= 0.3 && < 0.4,
492− time >= 1.5 && < 1.10,
496+ time >= 1.5 && < 1.12,
493497 unicode-transforms >= 0.3 && < 0.4,
494498 unordered-containers >= 0.2 && < 0.3,
495499 xml >= 1.3.12 && < 1.4,
496− xml-conduit >= 1.7 && < 1.10,
500+ xml-conduit >= 1.9.1.1 && < 1.10,
501+ unicode-collation >= 0.1.1 && < 0.2,
497502 zip-archive >= 0.2.3.4 && < 0.5,
498503 zlib >= 0.5 && < 0.7
499504 if os(windows) && arch(i386)
@@ -512,6 +517,7 @@ library
512517 Text.Pandoc.Options,
513518 Text.Pandoc.Extensions,
514519 Text.Pandoc.Shared,
520+ Text.Pandoc.Sources,
515521 Text.Pandoc.MediaBag,
516522 Text.Pandoc.Error,
517523 Text.Pandoc.Filter,
@@ -601,7 +607,6 @@ library
601607 Text.Pandoc.Asciify,
602608 Text.Pandoc.Emoji,
603609 Text.Pandoc.ImageSize,
604− Text.Pandoc.BCP47,
605610 Text.Pandoc.Class,
606611 Text.Pandoc.Citeproc
607612 other-modules: Text.Pandoc.App.CommandLineOptions,
@@ -658,6 +663,9 @@ library
658663 Text.Pandoc.Readers.Metadata,
659664 Text.Pandoc.Readers.Roff,
660665 Text.Pandoc.Writers.Docx.StyleMap,
666+ Text.Pandoc.Writers.Docx.Table,
667+ Text.Pandoc.Writers.Docx.Types,
668+ Text.Pandoc.Writers.GridTable
661669 Text.Pandoc.Writers.JATS.References,
662670 Text.Pandoc.Writers.JATS.Table,
663671 Text.Pandoc.Writers.JATS.Types,
@@ -730,7 +738,7 @@ executable trypandoc
730738 main-is: trypandoc.hs
731739 hs-source-dirs: trypandoc
732740 if flag(trypandoc)
733− build-depends: aeson, http-types, wai >= 0.3, wai-extra >= 3.0.24
741+ build-depends: aeson, http-types, text, wai >= 0.3, wai-extra >= 3.0.24
734742 buildable: True
735743 else
736744 buildable: False
@@ -744,13 +752,11 @@ test-suite test-pandoc
744752 Diff >= 0.2 && < 0.5,
745753 Glob >= 0.7 && < 0.11,
746754 QuickCheck >= 2.4 && < 2.15,
747− base64-bytestring >= 0.1 && < 1.3,
748755 bytestring >= 0.9 && < 0.12,
749756 containers >= 0.4.2.1 && < 0.7,
750757 directory >= 1.2.3 && < 1.4,
751758 doctemplates >= 0.9 && < 0.10,
752759 exceptions >= 0.8 && < 0.11,
753− executable-path >= 0.0 && < 0.1,
754760 filepath >= 1.1 && < 1.5,
755761 hslua >= 1.1 && < 1.4,
756762 mtl >= 2.2 && < 2.3,
@@ -761,8 +767,8 @@ test-suite test-pandoc
761767 tasty-hunit >= 0.9 && < 0.11,
762768 tasty-lua >= 0.2 && < 0.3,
763769 tasty-quickcheck >= 0.8 && < 0.11,
764− temporary >= 1.1 && < 1.4,
765− time >= 1.5 && < 1.10,
770+ text >= 1.1.1.0 && < 1.3,
771+ time >= 1.5 && < 1.12,
766772 xml >= 1.3.12 && < 1.4,
767773 zip-archive >= 0.2.3.4 && < 0.5
768774 other-modules: Tests.Old
@@ -828,7 +834,11 @@ benchmark benchmark-pandoc
828834 hs-source-dirs: benchmark
829835 build-depends: bytestring,
830836 containers,
831− tasty,
837+ -- gauge >= 0.2 && < 0.3,
832838 tasty-bench >= 0.2 && <= 0.3,
833839 mtl >= 2.2 && < 2.3,
834− time
840+ text >= 1.1.1.0 && < 1.3,
841+ time,
842+ deepseq
843+ -- we increase heap size to avoid benchmarking garbage collection:
844+ ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
src/Text/Pandoc/App.hs+40−21
@@ -47,20 +47,21 @@ import System.FilePath ( takeBaseName, takeExtension )
4747import System.IO (nativeNewline, stdout)
4848import qualified System.IO as IO (Newline (..))
4949import Text.Pandoc
50+import Text.Pandoc.Builder (setMeta)
5051import Text.Pandoc.MIME (getCharset)
5152import Text.Pandoc.App.FormatHeuristics (formatFromFilePaths)
5253import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), defaultOpts,
53− IpynbOutput (..) )
54+ IpynbOutput (..))
5455import Text.Pandoc.App.CommandLineOptions (parseOptions, parseOptionsFromArgs,
5556 options)
5657import Text.Pandoc.App.OutputSettings (OutputSettings (..), optToOutputSettings)
57−import Text.Pandoc.BCP47 (Lang (..), parseBCP47)
58+import Text.Collate.Lang (Lang (..), parseLang)
5859import Text.Pandoc.Filter (Filter (JSONFilter, LuaFilter), applyFilters)
5960import Text.Pandoc.PDF (makePDF)
6061import Text.Pandoc.SelfContained (makeSelfContained)
6162import Text.Pandoc.Shared (eastAsianLineBreakFilter, stripEmptyParagraphs,
6263 headerShift, isURI, tabFilter, uriPathToPath, filterIpynbOutput,
63− defaultUserDataDirs, tshow, findM)
64+ defaultUserDataDir, tshow)
6465import Text.Pandoc.Writers.Shared (lookupMetaString)
6566import Text.Pandoc.Readers.Markdown (yamlToMeta)
6667import qualified Text.Pandoc.UTF8 as UTF8
@@ -71,6 +72,15 @@ import System.Posix.Terminal (queryTerminal)
7172
7273convertWithOpts :: Opt -> IO ()
7374convertWithOpts opts = do
75+ datadir <- case optDataDir opts of
76+ Nothing -> do
77+ d <- defaultUserDataDir
78+ exists <- doesDirectoryExist d
79+ return $ if exists
80+ then Just d
81+ else Nothing
82+ Just _ -> return $ optDataDir opts
83+
7484 let outputFile = fromMaybe "-" (optOutputFile opts)
7585 let filters = optFilters opts
7686 let verbosity = optVerbosity opts
@@ -85,12 +95,6 @@ convertWithOpts opts = do
8595 Just xs | not (optIgnoreArgs opts) -> xs
8696 _ -> ["-"]
8797
88− datadir <- case optDataDir opts of
89− Nothing -> do
90− ds <- defaultUserDataDirs
91− findM doesDirectoryExist ds
92− Just _ -> return $ optDataDir opts
93−
9498 let runIO' :: PandocIO a -> IO a
9599 runIO' f = do
96100 (res, reports) <- runIOorExplode $ do
@@ -156,9 +160,11 @@ convertWithOpts opts = do
156160 else optTabStop opts)
157161
158162
159− let readSources :: [FilePath] -> PandocIO Text
160− readSources srcs = convertTabs . T.intercalate (T.pack "\n") <$>
161− mapM readSource srcs
163+ let readSources :: [FilePath] -> PandocIO [(FilePath, Text)]
164+ readSources srcs =
165+ mapM (\fp -> do
166+ t <- readSource fp
167+ return (if fp == "-" then "" else fp, convertTabs t)) srcs
162168
163169
164170 outputSettings <- optToOutputSettings opts
@@ -196,8 +202,8 @@ convertWithOpts opts = do
196202 Just f -> readFileStrict f
197203
198204 case lookupMetaString "lang" (optMetadata opts) of
199− "" -> setTranslations $ Lang "en" "" "US" []
200− l -> case parseBCP47 l of
205+ "" -> setTranslations $ Lang "en" Nothing (Just "US") [] [] []
206+ l -> case parseLang l of
201207 Left _ -> report $ InvalidLang l
202208 Right l' -> setTranslations l'
203209
@@ -251,13 +257,17 @@ convertWithOpts opts = do
251257 let sourceToDoc :: [FilePath] -> PandocIO Pandoc
252258 sourceToDoc sources' =
253259 case reader of
254− TextReader r
255− | optFileScope opts || readerNameBase == "json" ->
256− mconcat <$> mapM (readSource >=> r readerOpts) sources'
257− | otherwise ->
258− readSources sources' >>= r readerOpts
259− ByteStringReader r ->
260− mconcat <$> mapM (readFile' >=> r readerOpts) sources'
260+ TextReader r
261+ | readerNameBase == "json" ->
262+ mconcat <$> mapM (readSource >=> r readerOpts) sources'
263+ | optFileScope opts ->
264+ -- Read source and convert tabs (see #6709)
265+ let readSource' = fmap convertTabs . readSource
266+ in mconcat <$> mapM (readSource' >=> r readerOpts) sources'
267+ | otherwise ->
268+ readSources sources' >>= r readerOpts
269+ ByteStringReader r ->
270+ mconcat <$> mapM (readFile' >=> r readerOpts) sources'
261271
262272
263273 when (readerNameBase == "markdown_github" ||
@@ -275,12 +285,21 @@ convertWithOpts opts = do
275285 report $ Deprecated "pandoc-citeproc filter"
276286 "Use --citeproc instead."
277287
288+ let cslMetadata =
289+ maybe id (setMeta "csl") (optCSL opts) .
290+ (case optBibliography opts of
291+ [] -> id
292+ xs -> setMeta "bibliography" xs) .
293+ maybe id (setMeta "citation-abbreviations")
294+ (optCitationAbbreviations opts) $ mempty
295+
278296 doc <- sourceToDoc sources >>=
279297 ( (if isJust (optExtractMedia opts)
280298 then fillMediaBag
281299 else return)
282300 >=> return . adjustMetadata (metadataFromFile <>)
283301 >=> return . adjustMetadata (<> optMetadata opts)
302+ >=> return . adjustMetadata (<> cslMetadata)
284303 >=> applyTransforms transforms
285304 >=> applyFilters readerOpts filters [T.unpack format]
286305 >=> maybe return extractMedia (optExtractMedia opts)
src/Text/Pandoc/App/CommandLineOptions.hs+30−8
@@ -31,7 +31,7 @@ import Data.Aeson.Encode.Pretty (encodePretty', Config(..), keyOrder,
3131 defConfig, Indent(..), NumberFormat(..))
3232import Data.Bifunctor (second)
3333import Data.Char (toLower)
34−import Data.List (intercalate, sort)
34+import Data.List (intercalate, sort, foldl')
3535#ifdef _WINDOWS
3636#if MIN_VERSION_base(4,12,0)
3737import Data.List (isPrefixOf)
@@ -48,12 +48,13 @@ import System.FilePath
4848import System.IO (stdout)
4949import Text.DocTemplates (Context (..), ToContext (toVal), Val (..))
5050import Text.Pandoc
51+import Text.Pandoc.Builder (setMeta)
5152import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), IpynbOutput (..),
52− DefaultsState (..), addMeta, applyDefaults,
53+ DefaultsState (..), applyDefaults,
5354 fullDefaultsPath)
5455import Text.Pandoc.Filter (Filter (..))
5556import Text.Pandoc.Highlighting (highlightingStyles)
56−import Text.Pandoc.Shared (ordNub, elemText, safeStrRead, defaultUserDataDirs)
57+import Text.Pandoc.Shared (ordNub, elemText, safeStrRead, defaultUserDataDir)
5758import Text.Printf
5859
5960#ifdef EMBED_DATA_FILES
@@ -92,7 +93,7 @@ parseOptionsFromArgs options' defaults prg rawArgs = do
9293 ("Try " ++ prg ++ " --help for more information.")
9394
9495 -- thread option data structure through all supplied option actions
95− opts <- foldl (>>=) (return defaults) actions
96+ opts <- foldl' (>>=) (return defaults) actions
9697 let mbArgs = case args of
9798 [] -> Nothing
9899 xs -> Just xs
@@ -287,7 +288,8 @@ options =
287288 , Option "" ["resource-path"]
288289 (ReqArg
289290 (\arg opt -> return opt { optResourcePath =
290− splitSearchPath arg })
291+ splitSearchPath arg ++
292+ optResourcePath opt })
291293 "SEARCHPATH")
292294 "" -- "Paths to search for images and other resources"
293295
@@ -939,13 +941,12 @@ options =
939941 (NoArg
940942 (\_ -> do
941943 prg <- getProgName
942− defaultDatadirs <- defaultUserDataDirs
944+ defaultDatadir <- defaultUserDataDir
943945 UTF8.hPutStrLn stdout
944946 $ T.pack
945947 $ prg ++ " " ++ T.unpack pandocVersion ++
946948 compileInfo ++
947− "\nUser data directory: " ++
948− intercalate " or " defaultDatadirs ++
949+ "\nUser data directory: " ++ defaultDatadir ++
949950 ('\n':copyrightMessage)
950951 exitSuccess ))
951952 "" -- "Print version"
@@ -1053,6 +1054,27 @@ setVariable key val (Context ctx) = Context $ M.alter go key ctx
10531054 go (Just (ListVal xs)) = Just $ ListVal $ xs ++ [toVal val]
10541055 go (Just x) = Just $ ListVal [x, toVal val]
10551056
1057+addMeta :: String -> String -> Meta -> Meta
1058+addMeta k v meta =
1059+ case lookupMeta k' meta of
1060+ Nothing -> setMeta k' v' meta
1061+ Just (MetaList xs) ->
1062+ setMeta k' (MetaList (xs ++ [v'])) meta
1063+ Just x -> setMeta k' (MetaList [x, v']) meta
1064+ where
1065+ v' = readMetaValue v
1066+ k' = T.pack k
1067+
1068+readMetaValue :: String -> MetaValue
1069+readMetaValue s
1070+ | s == "true" = MetaBool True
1071+ | s == "True" = MetaBool True
1072+ | s == "TRUE" = MetaBool True
1073+ | s == "false" = MetaBool False
1074+ | s == "False" = MetaBool False
1075+ | s == "FALSE" = MetaBool False
1076+ | otherwise = MetaString $ T.pack s
1077+
10561078-- On Windows with ghc 8.6+, we need to rewrite paths
10571079-- beginning with \\ to \\?\UNC\. -- See #5127.
10581080normalizePath :: FilePath -> FilePath
src/Text/Pandoc/App/FormatHeuristics.hs+13−7
@@ -15,18 +15,24 @@ module Text.Pandoc.App.FormatHeuristics
1515 ) where
1616
1717import Data.Char (toLower)
18+import Data.Foldable (asum)
1819import Data.Text (Text)
1920import System.FilePath (takeExtension)
2021
21−-- Determine default format based on file extensions.
22+-- | Determines default format based on file extensions; uses the format
23+-- of the first extension that's associated with a format.
24+--
25+-- Examples:
26+--
27+-- > formatFromFilePaths ["text.unknown", "no-extension"]
28+-- Nothing
29+--
30+-- > formatFromFilePaths ["my.md", "other.rst"]
31+-- Just "markdown"
2232formatFromFilePaths :: [FilePath] -> Maybe Text
23−formatFromFilePaths [] = Nothing
24−formatFromFilePaths (x:xs) =
25− case formatFromFilePath x of
26− Just f -> Just f
27− Nothing -> formatFromFilePaths xs
33+formatFromFilePaths = asum . map formatFromFilePath
2834
29−-- Determine format based on file extension
35+-- | Determines format based on file extension.
3036formatFromFilePath :: FilePath -> Maybe Text
3137formatFromFilePath x =
3238 case takeExtension (map toLower x) of
src/Text/Pandoc/App/Opt.hs+145−57
@@ -22,28 +22,29 @@ module Text.Pandoc.App.Opt (
2222 , IpynbOutput (..)
2323 , DefaultsState (..)
2424 , defaultOpts
25− , addMeta
2625 , applyDefaults
2726 , fullDefaultsPath
2827 ) where
2928import Control.Monad.Except (MonadIO, liftIO, throwError, (>=>), foldM)
3029import Control.Monad.State.Strict (StateT, modify, gets)
31−import System.FilePath ( addExtension, (</>), takeExtension )
30+import System.FilePath ( addExtension, (</>), takeExtension, takeDirectory )
31+import System.Directory ( canonicalizePath )
3232import Data.Char (isLower, toLower)
3333import Data.Maybe (fromMaybe)
3434import GHC.Generics hiding (Meta)
35−import Text.Pandoc.Builder (setMeta)
3635import Text.Pandoc.Filter (Filter (..))
37−import Text.Pandoc.Logging (Verbosity (WARNING))
36+import Text.Pandoc.Logging (Verbosity (WARNING), LogMessage(..))
3837import Text.Pandoc.Options (TopLevelDivision (TopLevelDefault),
3938 TrackChanges (AcceptChanges),
4039 WrapOption (WrapAuto), HTMLMathMethod (PlainMath),
4140 ReferenceLocation (EndOfDocument),
4241 ObfuscationMethod (NoObfuscation),
4342 CiteMethod (Citeproc))
44−import Text.Pandoc.Class (readFileLazy, fileExists, setVerbosity, PandocMonad)
43+import Text.Pandoc.Class (readFileLazy, fileExists, setVerbosity, report,
44+ PandocMonad(lookupEnv), getUserDataDir)
4545import Text.Pandoc.Error (PandocError (PandocParseError, PandocSomeError))
46−import Text.Pandoc.Shared (camelCaseStrToHyphenated, defaultUserDataDirs, findM, ordNub)
46+import Text.Pandoc.Shared (camelCaseStrToHyphenated, defaultUserDataDir,
47+ findM, ordNub)
4748import qualified Text.Pandoc.Parsing as P
4849import Text.Pandoc.Readers.Metadata (yamlMap)
4950import Text.Pandoc.Class.PandocPure
@@ -52,7 +53,7 @@ import Data.Text (Text, unpack)
5253import Data.Default (def)
5354import qualified Data.Text as T
5455import qualified Data.Map as M
55−import Text.Pandoc.Definition (Meta(..), MetaValue(..), lookupMeta)
56+import Text.Pandoc.Definition (Meta(..), MetaValue(..))
5657import Data.Aeson (defaultOptions, Options(..))
5758import Data.Aeson.TH (deriveJSON)
5859import Control.Applicative ((<|>))
@@ -156,6 +157,9 @@ data Opt = Opt
156157 , optNoCheckCertificate :: Bool -- ^ Disable certificate validation
157158 , optEol :: LineEnding -- ^ Style of line-endings to use
158159 , optStripComments :: Bool -- ^ Skip HTML comments
160+ , optCSL :: Maybe FilePath -- ^ CSL stylesheet
161+ , optBibliography :: [FilePath] -- ^ Bibliography files
162+ , optCitationAbbreviations :: Maybe FilePath -- ^ Citation abbreviations
159163 } deriving (Generic, Show)
160164
161165instance FromYAML (Opt -> Opt) where
@@ -175,17 +179,128 @@ instance (PandocMonad m, MonadIO m)
175179 dataDir <- case M.lookup "data-dir" opts of
176180 Nothing -> return Nothing
177181 Just v -> Just . unpack <$> parseYAML v
178− f <- parseOptions $ M.toList m
182+ f <- parseOptions (M.toList m)
179183 case M.lookup "defaults" opts of
180184 Just v -> do
181185 g <- parseDefaults v dataDir
182− return $ g >=> f
183− Nothing -> return f
186+ return $ g >=> f >=> resolveVarsInOpt
187+ Nothing -> return $ f >=> resolveVarsInOpt
184188 where
185189 toText (Scalar _ (SStr s)) = s
186190 toText _ = ""
187191 parseYAML n = failAtNode n "Expected a mapping"
188192
193+resolveVarsInOpt :: forall m. (PandocMonad m, MonadIO m)
194+ => Opt -> StateT DefaultsState m Opt
195+resolveVarsInOpt
196+ opt@Opt
197+ { optTemplate = oTemplate
198+ , optMetadataFiles = oMetadataFiles
199+ , optOutputFile = oOutputFile
200+ , optInputFiles = oInputFiles
201+ , optSyntaxDefinitions = oSyntaxDefinitions
202+ , optAbbreviations = oAbbreviations
203+ , optReferenceDoc = oReferenceDoc
204+ , optEpubMetadata = oEpubMetadata
205+ , optEpubFonts = oEpubFonts
206+ , optEpubCoverImage = oEpubCoverImage
207+ , optLogFile = oLogFile
208+ , optFilters = oFilters
209+ , optDataDir = oDataDir
210+ , optExtractMedia = oExtractMedia
211+ , optCss = oCss
212+ , optIncludeBeforeBody = oIncludeBeforeBody
213+ , optIncludeAfterBody = oIncludeAfterBody
214+ , optIncludeInHeader = oIncludeInHeader
215+ , optResourcePath = oResourcePath
216+ , optCSL = oCSL
217+ , optBibliography = oBibliography
218+ , optCitationAbbreviations = oCitationAbbreviations
219+ }
220+ = do
221+ oTemplate' <- mapM resolveVars oTemplate
222+ oMetadataFiles' <- mapM resolveVars oMetadataFiles
223+ oOutputFile' <- mapM resolveVars oOutputFile
224+ oInputFiles' <- mapM (mapM resolveVars) oInputFiles
225+ oSyntaxDefinitions' <- mapM resolveVars oSyntaxDefinitions
226+ oAbbreviations' <- mapM resolveVars oAbbreviations
227+ oReferenceDoc' <- mapM resolveVars oReferenceDoc
228+ oEpubMetadata' <- mapM resolveVars oEpubMetadata
229+ oEpubFonts' <- mapM resolveVars oEpubFonts
230+ oEpubCoverImage' <- mapM resolveVars oEpubCoverImage
231+ oLogFile' <- mapM resolveVars oLogFile
232+ oFilters' <- mapM resolveVarsInFilter oFilters
233+ oDataDir' <- mapM resolveVars oDataDir
234+ oExtractMedia' <- mapM resolveVars oExtractMedia
235+ oCss' <- mapM resolveVars oCss
236+ oIncludeBeforeBody' <- mapM resolveVars oIncludeBeforeBody
237+ oIncludeAfterBody' <- mapM resolveVars oIncludeAfterBody
238+ oIncludeInHeader' <- mapM resolveVars oIncludeInHeader
239+ oResourcePath' <- mapM resolveVars oResourcePath
240+ oCSL' <- mapM resolveVars oCSL
241+ oBibliography' <- mapM resolveVars oBibliography
242+ oCitationAbbreviations' <- mapM resolveVars oCitationAbbreviations
243+ return opt{ optTemplate = oTemplate'
244+ , optMetadataFiles = oMetadataFiles'
245+ , optOutputFile = oOutputFile'
246+ , optInputFiles = oInputFiles'
247+ , optSyntaxDefinitions = oSyntaxDefinitions'
248+ , optAbbreviations = oAbbreviations'
249+ , optReferenceDoc = oReferenceDoc'
250+ , optEpubMetadata = oEpubMetadata'
251+ , optEpubFonts = oEpubFonts'
252+ , optEpubCoverImage = oEpubCoverImage'
253+ , optLogFile = oLogFile'
254+ , optFilters = oFilters'
255+ , optDataDir = oDataDir'
256+ , optExtractMedia = oExtractMedia'
257+ , optCss = oCss'
258+ , optIncludeBeforeBody = oIncludeBeforeBody'
259+ , optIncludeAfterBody = oIncludeAfterBody'
260+ , optIncludeInHeader = oIncludeInHeader'
261+ , optResourcePath = oResourcePath'
262+ , optCSL = oCSL'
263+ , optBibliography = oBibliography'
264+ , optCitationAbbreviations = oCitationAbbreviations'
265+ }
266+
267+ where
268+ resolveVars :: FilePath -> StateT DefaultsState m FilePath
269+ resolveVars [] = return []
270+ resolveVars ('$':'{':xs) =
271+ let (ys, zs) = break (=='}') xs
272+ in if null zs
273+ then return $ '$':'{':xs
274+ else do
275+ val <- lookupEnv' ys
276+ (val ++) <$> resolveVars (drop 1 zs)
277+ resolveVars (c:cs) = (c:) <$> resolveVars cs
278+ lookupEnv' :: String -> StateT DefaultsState m String
279+ lookupEnv' "." = do
280+ mbCurDefaults <- gets curDefaults
281+ maybe (return "")
282+ (fmap takeDirectory . liftIO . canonicalizePath)
283+ mbCurDefaults
284+ lookupEnv' "USERDATA" = do
285+ mbodatadir <- mapM resolveVars oDataDir
286+ mbdatadir <- getUserDataDir
287+ defdatadir <- liftIO defaultUserDataDir
288+ return $ fromMaybe defdatadir (mbodatadir <|> mbdatadir)
289+ lookupEnv' v = do
290+ mbval <- fmap T.unpack <$> lookupEnv (T.pack v)
291+ case mbval of
292+ Nothing -> do
293+ report $ EnvironmentVariableUndefined (T.pack v)
294+ return mempty
295+ Just x -> return x
296+ resolveVarsInFilter (JSONFilter fp) =
297+ JSONFilter <$> resolveVars fp
298+ resolveVarsInFilter (LuaFilter fp) =
299+ LuaFilter <$> resolveVars fp
300+ resolveVarsInFilter CiteprocFilter = return CiteprocFilter
301+
302+
303+
189304parseDefaults :: (PandocMonad m, MonadIO m)
190305 => Node Pos
191306 -> Maybe FilePath
@@ -428,26 +543,18 @@ doOpt (k',v) = do
428543 (parseYAML v >>= \x -> return (\o -> o{ optCss = optCss o <>
429544 [unpack x] }))
430545 "bibliography" ->
431− do let addItem x o = o{ optMetadata =
432− addMeta "bibliography" (T.unpack x)
433− (optMetadata o) }
434− (parseYAML v >>= \(xs :: [Text]) -> return $ \o ->
435− foldr addItem o xs)
436− <|>
437− (parseYAML v >>= \(x :: Text) -> return $ \o -> addItem x o)
546+ (parseYAML v >>= \x -> return (\o ->
547+ o{ optBibliography = optBibliography o <>
548+ map unpack x }))
549+ <|>
550+ (parseYAML v >>= \x -> return (\o ->
551+ o{ optBibliography = optBibliography o <>
552+ [unpack x] }))
438553 "csl" ->
439− do let addItem x o = o{ optMetadata =
440− addMeta "csl" (T.unpack x)
441− (optMetadata o) }
442− (parseYAML v >>= \(xs :: [Text]) -> return $ \o ->
443− foldr addItem o xs)
444− <|>
445− (parseYAML v >>= \(x :: Text) -> return $ \o -> addItem x o)
554+ parseYAML v >>= \x -> return (\o -> o{ optCSL = unpack <$> x })
446555 "citation-abbreviations" ->
447− parseYAML v >>= \x ->
448− return (\o -> o{ optMetadata =
449− addMeta "citation-abbreviations" (T.unpack x)
450− (optMetadata o) })
556+ parseYAML v >>= \x -> return (\o -> o{ optCitationAbbreviations =
557+ unpack <$> x })
451558 "ipynb-output" ->
452559 parseYAML v >>= \x -> return (\o -> o{ optIpynbOutput = x })
453560 "include-before-body" ->
@@ -476,7 +583,8 @@ doOpt (k',v) = do
476583 optIncludeInHeader o <> [unpack x] }))
477584 "resource-path" ->
478585 parseYAML v >>= \x ->
479− return (\o -> o{ optResourcePath = map unpack x })
586+ return (\o -> o{ optResourcePath = map unpack x <>
587+ optResourcePath o })
480588 "request-headers" ->
481589 parseYAML v >>= \x ->
482590 return (\o -> o{ optRequestHeaders = x })
@@ -562,6 +670,9 @@ defaultOpts = Opt
562670 , optNoCheckCertificate = False
563671 , optEol = Native
564672 , optStripComments = False
673+ , optCSL = Nothing
674+ , optBibliography = []
675+ , optCitationAbbreviations = Nothing
565676 }
566677
567678parseStringKey :: Node Pos -> Parser Text
@@ -575,31 +686,11 @@ yamlToMeta (Mapping _ _ m) =
575686 either (fail . show) return $ runEverything (yamlMap pMetaString m)
576687 where
577688 pMetaString = pure . MetaString <$> P.manyChar P.anyChar
578− runEverything p = runPure (P.readWithM p def "")
689+ runEverything p =
690+ runPure (P.readWithM p (def :: P.ParserState) ("" :: Text))
579691 >>= fmap (Meta . flip P.runF def)
580692yamlToMeta _ = return mempty
581693
582−addMeta :: String -> String -> Meta -> Meta
583−addMeta k v meta =
584− case lookupMeta k' meta of
585− Nothing -> setMeta k' v' meta
586− Just (MetaList xs) ->
587− setMeta k' (MetaList (xs ++ [v'])) meta
588− Just x -> setMeta k' (MetaList [x, v']) meta
589− where
590− v' = readMetaValue v
591− k' = T.pack k
592−
593−readMetaValue :: String -> MetaValue
594−readMetaValue s
595− | s == "true" = MetaBool True
596− | s == "True" = MetaBool True
597− | s == "TRUE" = MetaBool True
598− | s == "false" = MetaBool False
599− | s == "False" = MetaBool False
600− | s == "FALSE" = MetaBool False
601− | otherwise = MetaString $ T.pack s
602−
603694-- | Apply defaults from --defaults file.
604695applyDefaults :: (PandocMonad m, MonadIO m)
605696 => Opt
@@ -625,12 +716,9 @@ fullDefaultsPath dataDir file = do
625716 let fp = if null (takeExtension file)
626717 then addExtension file "yaml"
627718 else file
628− dataDirs <- liftIO defaultUserDataDirs
629− let fps = fp : case dataDir of
630− Nothing -> map (</> ("defaults" </> fp))
631− dataDirs
632− Just dd -> [dd </> "defaults" </> fp]
633− fromMaybe fp <$> findM fileExists fps
719+ defaultDataDir <- liftIO defaultUserDataDir
720+ let defaultFp = fromMaybe defaultDataDir dataDir </> "defaults" </> fp
721+ fromMaybe fp <$> findM fileExists [fp, defaultFp]
634722
635723-- | In a list of lists, append another list in front of every list which
636724-- starts with specific element.
src/Text/Pandoc/Asciify.hs+13−390
@@ -10,396 +10,19 @@
1010Function to convert accented latin letters to their unaccented
1111ascii equivalents (used in constructing HTML identifiers).
1212-}
13−module Text.Pandoc.Asciify (toAsciiChar)
13+module Text.Pandoc.Asciify (toAsciiChar, toAsciiText)
1414where
15−import Data.Char (isAscii)
16−import qualified Data.Map as M
15+import Data.Char (isAscii, isMark)
16+import qualified Data.Text.Normalize as TN
17+import Data.Text (Text)
18+import qualified Data.Text as T
1719
18−toAsciiChar :: Char -> Maybe Char
19−toAsciiChar c | isAscii c = Just c
20− | otherwise = M.lookup c asciiMap
20+toAsciiText :: Text -> Text
21+toAsciiText = T.filter isAscii . TN.normalize (TN.NFD)
2122
22−asciiMap :: M.Map Char Char
23−asciiMap = M.fromList
24− [('\192','A')
25− ,('\193','A')
26− ,('\194','A')
27− ,('\195','A')
28− ,('\196','A')
29− ,('\197','A')
30− ,('\199','C')
31− ,('\200','E')
32− ,('\201','E')
33− ,('\202','E')
34− ,('\203','E')
35− ,('\204','I')
36− ,('\205','I')
37− ,('\206','I')
38− ,('\207','I')
39− ,('\209','N')
40− ,('\210','O')
41− ,('\211','O')
42− ,('\212','O')
43− ,('\213','O')
44− ,('\214','O')
45− ,('\217','U')
46− ,('\218','U')
47− ,('\219','U')
48− ,('\220','U')
49− ,('\221','Y')
50− ,('\224','a')
51− ,('\225','a')
52− ,('\226','a')
53− ,('\227','a')
54− ,('\228','a')
55− ,('\229','a')
56− ,('\231','c')
57− ,('\232','e')
58− ,('\233','e')
59− ,('\234','e')
60− ,('\235','e')
61− ,('\236','i')
62− ,('\237','i')
63− ,('\238','i')
64− ,('\239','i')
65− ,('\241','n')
66− ,('\242','o')
67− ,('\243','o')
68− ,('\244','o')
69− ,('\245','o')
70− ,('\246','o')
71− ,('\249','u')
72− ,('\250','u')
73− ,('\251','u')
74− ,('\252','u')
75− ,('\253','y')
76− ,('\255','y')
77− ,('\256','A')
78− ,('\257','a')
79− ,('\258','A')
80− ,('\259','a')
81− ,('\260','A')
82− ,('\261','a')
83− ,('\262','C')
84− ,('\263','c')
85− ,('\264','C')
86− ,('\265','c')
87− ,('\266','C')
88− ,('\267','c')
89− ,('\268','C')
90− ,('\269','c')
91− ,('\270','D')
92− ,('\271','d')
93− ,('\274','E')
94− ,('\275','e')
95− ,('\276','E')
96− ,('\277','e')
97− ,('\278','E')
98− ,('\279','e')
99− ,('\280','E')
100− ,('\281','e')
101− ,('\282','E')
102− ,('\283','e')
103− ,('\284','G')
104− ,('\285','g')
105− ,('\286','G')
106− ,('\287','g')
107− ,('\288','G')
108− ,('\289','g')
109− ,('\290','G')
110− ,('\291','g')
111− ,('\292','H')
112− ,('\293','h')
113− ,('\296','I')
114− ,('\297','i')
115− ,('\298','I')
116− ,('\299','i')
117− ,('\300','I')
118− ,('\301','i')
119− ,('\302','I')
120− ,('\303','i')
121− ,('\304','I')
122− ,('\305','i')
123− ,('\308','J')
124− ,('\309','j')
125− ,('\310','K')
126− ,('\311','k')
127− ,('\313','L')
128− ,('\314','l')
129− ,('\315','L')
130− ,('\316','l')
131− ,('\317','L')
132− ,('\318','l')
133− ,('\323','N')
134− ,('\324','n')
135− ,('\325','N')
136− ,('\326','n')
137− ,('\327','N')
138− ,('\328','n')
139− ,('\332','O')
140− ,('\333','o')
141− ,('\334','O')
142− ,('\335','o')
143− ,('\336','O')
144− ,('\337','o')
145− ,('\340','R')
146− ,('\341','r')
147− ,('\342','R')
148− ,('\343','r')
149− ,('\344','R')
150− ,('\345','r')
151− ,('\346','S')
152− ,('\347','s')
153− ,('\348','S')
154− ,('\349','s')
155− ,('\350','S')
156− ,('\351','s')
157− ,('\352','S')
158− ,('\353','s')
159− ,('\354','T')
160− ,('\355','t')
161− ,('\356','T')
162− ,('\357','t')
163− ,('\360','U')
164− ,('\361','u')
165− ,('\362','U')
166− ,('\363','u')
167− ,('\364','U')
168− ,('\365','u')
169− ,('\366','U')
170− ,('\367','u')
171− ,('\368','U')
172− ,('\369','u')
173− ,('\370','U')
174− ,('\371','u')
175− ,('\372','W')
176− ,('\373','w')
177− ,('\374','Y')
178− ,('\375','y')
179− ,('\376','Y')
180− ,('\377','Z')
181− ,('\378','z')
182− ,('\379','Z')
183− ,('\380','z')
184− ,('\381','Z')
185− ,('\382','z')
186− ,('\416','O')
187− ,('\417','o')
188− ,('\431','U')
189− ,('\432','u')
190− ,('\461','A')
191− ,('\462','a')
192− ,('\463','I')
193− ,('\464','i')
194− ,('\465','O')
195− ,('\466','o')
196− ,('\467','U')
197− ,('\468','u')
198− ,('\486','G')
199− ,('\487','g')
200− ,('\488','K')
201− ,('\489','k')
202− ,('\490','O')
203− ,('\491','o')
204− ,('\496','j')
205− ,('\500','G')
206− ,('\501','g')
207− ,('\504','N')
208− ,('\505','n')
209− ,('\512','A')
210− ,('\513','a')
211− ,('\514','A')
212− ,('\515','a')
213− ,('\516','E')
214− ,('\517','e')
215− ,('\518','E')
216− ,('\519','e')
217− ,('\520','I')
218− ,('\521','i')
219− ,('\522','I')
220− ,('\523','i')
221− ,('\524','O')
222− ,('\525','o')
223− ,('\526','O')
224− ,('\527','o')
225− ,('\528','R')
226− ,('\529','r')
227− ,('\530','R')
228− ,('\531','r')
229− ,('\532','U')
230− ,('\533','u')
231− ,('\534','U')
232− ,('\535','u')
233− ,('\536','S')
234− ,('\537','s')
235− ,('\538','T')
236− ,('\539','t')
237− ,('\542','H')
238− ,('\543','h')
239− ,('\550','A')
240− ,('\551','a')
241− ,('\552','E')
242− ,('\553','e')
243− ,('\558','O')
244− ,('\559','o')
245− ,('\562','Y')
246− ,('\563','y')
247− ,('\894',';')
248− ,('\7680','A')
249− ,('\7681','a')
250− ,('\7682','B')
251− ,('\7683','b')
252− ,('\7684','B')
253− ,('\7685','b')
254− ,('\7686','B')
255− ,('\7687','b')
256− ,('\7690','D')
257− ,('\7691','d')
258− ,('\7692','D')
259− ,('\7693','d')
260− ,('\7694','D')
261− ,('\7695','d')
262− ,('\7696','D')
263− ,('\7697','d')
264− ,('\7698','D')
265− ,('\7699','d')
266− ,('\7704','E')
267− ,('\7705','e')
268− ,('\7706','E')
269− ,('\7707','e')
270− ,('\7710','F')
271− ,('\7711','f')
272− ,('\7712','G')
273− ,('\7713','g')
274− ,('\7714','H')
275− ,('\7715','h')
276− ,('\7716','H')
277− ,('\7717','h')
278− ,('\7718','H')
279− ,('\7719','h')
280− ,('\7720','H')
281− ,('\7721','h')
282− ,('\7722','H')
283− ,('\7723','h')
284− ,('\7724','I')
285− ,('\7725','i')
286− ,('\7728','K')
287− ,('\7729','k')
288− ,('\7730','K')
289− ,('\7731','k')
290− ,('\7732','K')
291− ,('\7733','k')
292− ,('\7734','L')
293− ,('\7735','l')
294− ,('\7738','L')
295− ,('\7739','l')
296− ,('\7740','L')
297− ,('\7741','l')
298− ,('\7742','M')
299− ,('\7743','m')
300− ,('\7744','M')
301− ,('\7745','m')
302− ,('\7746','M')
303− ,('\7747','m')
304− ,('\7748','N')
305− ,('\7749','n')
306− ,('\7750','N')
307− ,('\7751','n')
308− ,('\7752','N')
309− ,('\7753','n')
310− ,('\7754','N')
311− ,('\7755','n')
312− ,('\7764','P')
313− ,('\7765','p')
314− ,('\7766','P')
315− ,('\7767','p')
316− ,('\7768','R')
317− ,('\7769','r')
318− ,('\7770','R')
319− ,('\7771','r')
320− ,('\7774','R')
321− ,('\7775','r')
322− ,('\7776','S')
323− ,('\7777','s')
324− ,('\7778','S')
325− ,('\7779','s')
326− ,('\7786','T')
327− ,('\7787','t')
328− ,('\7788','T')
329− ,('\7789','t')
330− ,('\7790','T')
331− ,('\7791','t')
332− ,('\7792','T')
333− ,('\7793','t')
334− ,('\7794','U')
335− ,('\7795','u')
336− ,('\7796','U')
337− ,('\7797','u')
338− ,('\7798','U')
339− ,('\7799','u')
340− ,('\7804','V')
341− ,('\7805','v')
342− ,('\7806','V')
343− ,('\7807','v')
344− ,('\7808','W')
345− ,('\7809','w')
346− ,('\7810','W')
347− ,('\7811','w')
348− ,('\7812','W')
349− ,('\7813','w')
350− ,('\7814','W')
351− ,('\7815','w')
352− ,('\7816','W')
353− ,('\7817','w')
354− ,('\7818','X')
355− ,('\7819','x')
356− ,('\7820','X')
357− ,('\7821','x')
358− ,('\7822','Y')
359− ,('\7823','y')
360− ,('\7824','Z')
361− ,('\7825','z')
362− ,('\7826','Z')
363− ,('\7827','z')
364− ,('\7828','Z')
365− ,('\7829','z')
366− ,('\7830','h')
367− ,('\7831','t')
368− ,('\7832','w')
369− ,('\7833','y')
370− ,('\7840','A')
371− ,('\7841','a')
372− ,('\7842','A')
373− ,('\7843','a')
374− ,('\7864','E')
375− ,('\7865','e')
376− ,('\7866','E')
377− ,('\7867','e')
378− ,('\7868','E')
379− ,('\7869','e')
380− ,('\7880','I')
381− ,('\7881','i')
382− ,('\7882','I')
383− ,('\7883','i')
384− ,('\7884','O')
385− ,('\7885','o')
386− ,('\7886','O')
387− ,('\7887','o')
388− ,('\7908','U')
389− ,('\7909','u')
390− ,('\7910','U')
391− ,('\7911','u')
392− ,('\7922','Y')
393− ,('\7923','y')
394− ,('\7924','Y')
395− ,('\7925','y')
396− ,('\7926','Y')
397− ,('\7927','y')
398− ,('\7928','Y')
399− ,('\7929','y')
400− ,('\8175','`')
401− ,('\8490','K')
402− ,('\8800','=')
403− ,('\8814','<')
404− ,('\8815','>')
405− ]
23+toAsciiChar :: Char -> Maybe Char
24+toAsciiChar c = case T.unpack (TN.normalize TN.NFD (T.singleton c)) of
25+ (x:xs) | isAscii x
26+ , all isMark xs
27+ -> Just x
28+ _ -> Nothing
src/Text/Pandoc/BCP47.hsdeleted−112
@@ -1,112 +0,0 @@
1−{-# LANGUAGE OverloadedStrings #-}
2−{- |
3− Module : Text.Pandoc.BCP47
4− Copyright : Copyright (C) 2017-2021 John MacFarlane
5− License : GNU GPL, version 2 or above
6−
7− Maintainer : John MacFarlane <jgm@berkeley.edu>
8− Stability : alpha
9− Portability : portable
10−
11−Functions for parsing and rendering BCP47 language identifiers.
12−-}
13−module Text.Pandoc.BCP47 (
14− getLang
15− , parseBCP47
16− , Lang(..)
17− , renderLang
18− )
19−where
20−import Control.Monad (guard)
21−import Data.Char (isAlphaNum, isAscii, isLetter, isLower, isUpper)
22−import Text.Pandoc.Definition
23−import Text.Pandoc.Options
24−import Text.DocTemplates (FromContext(..))
25−import qualified Data.Text as T
26−import qualified Text.Parsec as P
27−
28−-- | Represents BCP 47 language/country code.
29−data Lang = Lang{ langLanguage :: T.Text
30− , langScript :: T.Text
31− , langRegion :: T.Text
32− , langVariants :: [T.Text] }
33− deriving (Eq, Ord, Show)
34−
35−-- | Render a Lang as BCP 47.
36−renderLang :: Lang -> T.Text
37−renderLang lang = T.intercalate "-" (langLanguage lang : filter (not . T.null)
38− ([langScript lang, langRegion lang] ++ langVariants lang))
39−
40−-- | Get the contents of the `lang` metadata field or variable.
41−getLang :: WriterOptions -> Meta -> Maybe T.Text
42−getLang opts meta =
43− case lookupContext "lang" (writerVariables opts) of
44− Just s -> Just s
45− _ ->
46− case lookupMeta "lang" meta of
47− Just (MetaBlocks [Para [Str s]]) -> Just s
48− Just (MetaBlocks [Plain [Str s]]) -> Just s
49− Just (MetaInlines [Str s]) -> Just s
50− Just (MetaString s) -> Just s
51− _ -> Nothing
52−
53−-- | Parse a BCP 47 string as a Lang. Currently we parse
54−-- extensions and private-use fields as "variants," even
55−-- though officially they aren't.
56−parseBCP47 :: T.Text -> Either T.Text Lang
57−parseBCP47 lang =
58− case P.parse bcp47 "lang" lang of
59− Right r -> Right r
60− Left e -> Left $ T.pack $ show e
61− where bcp47 = do
62− language <- pLanguage
63− script <- P.option "" pScript
64− region <- P.option "" pRegion
65− variants <- P.many (pVariant P.<|> pExtension P.<|> pPrivateUse)
66− P.eof
67− return Lang{ langLanguage = language
68− , langScript = script
69− , langRegion = region
70− , langVariants = variants }
71− asciiLetter = P.satisfy (\c -> isAscii c && isLetter c)
72− pLanguage = do
73− cs <- P.many1 asciiLetter
74− let lcs = length cs
75− guard $ lcs == 2 || lcs == 3
76− return $ T.toLower $ T.pack cs
77− pScript = P.try $ do
78− P.char '-'
79− x <- P.satisfy (\c -> isAscii c && isLetter c && isUpper c)
80− xs <- P.count 3
81− (P.satisfy (\c -> isAscii c && isLetter c && isLower c))
82− return $ T.toLower $ T.pack (x:xs)
83− pRegion = P.try $ do
84− P.char '-'
85− cs <- P.many1 asciiLetter
86− let lcs = length cs
87− guard $ lcs == 2 || lcs == 3
88− return $ T.toUpper $ T.pack cs
89− pVariant = P.try $ do
90− P.char '-'
91− ds <- P.option "" (P.count 1 P.digit)
92− cs <- P.many1 asciiLetter
93− let var = ds ++ cs
94− lv = length var
95− guard $ if null ds
96− then lv >= 5 && lv <= 8
97− else lv == 4
98− return $ T.toLower $ T.pack var
99− pExtension = P.try $ do
100− P.char '-'
101− cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
102− let lcs = length cs
103− guard $ lcs >= 2 && lcs <= 8
104− return $ T.toLower $ T.pack cs
105− pPrivateUse = P.try $ do
106− P.char '-'
107− P.char 'x'
108− P.char '-'
109− cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
110− guard $ not (null cs) && length cs <= 8
111− let var = "x-" ++ cs
112− return $ T.toLower $ T.pack var
src/Text/Pandoc/Citeproc.hs+75−56
@@ -18,7 +18,6 @@ import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
1818import Text.Pandoc.Citeproc.BibTeX (readBibtexString, Variant(..))
1919import Text.Pandoc.Citeproc.MetaValue (metaValueToReference, metaValueToText)
2020import Text.Pandoc.Readers.Markdown (yamlToRefs)
21−import qualified Text.Pandoc.BCP47 as BCP47
2221import Text.Pandoc.Builder (Inlines, Many(..), deleteMeta, setMeta)
2322import qualified Text.Pandoc.Builder as B
2423import Text.Pandoc.Definition as Pandoc
@@ -28,7 +27,7 @@ import Text.Pandoc.Error (PandocError(..))
2827import Text.Pandoc.Extensions (pandocExtensions)
2928import Text.Pandoc.Logging (LogMessage(..))
3029import Text.Pandoc.Options (ReaderOptions(..))
31−import Text.Pandoc.Shared (stringify, ordNub, blocksToInlines, tshow)
30+import Text.Pandoc.Shared (stringify, ordNub, tshow)
3231import qualified Text.Pandoc.UTF8 as UTF8
3332import Text.Pandoc.Walk (query, walk, walkM)
3433import Control.Applicative ((<|>))
@@ -49,7 +48,6 @@ import Data.Text (Text)
4948import qualified Data.Text as T
5049import System.FilePath (takeExtension)
5150import Safe (lastMay, initSafe)
52−-- import Debug.Trace as Trace (trace, traceShowId)
5351
5452
5553processCitations :: PandocMonad m => Pandoc -> m Pandoc
@@ -76,8 +74,7 @@ processCitations (Pandoc meta bs) = do
7674
7775 let linkCites = maybe False truish $ lookupMeta "link-citations" meta
7876 let opts = defaultCiteprocOptions{ linkCitations = linkCites }
79− let result = Citeproc.citeproc opts style (localeLanguage locale)
80− refs citations
77+ let result = Citeproc.citeproc opts style mblang refs citations
8178 mapM_ (report . CiteprocWarning) (resultWarnings result)
8279 let sopts = styleOptions style
8380 let classes = "references" : -- TODO remove this or keep for compatibility?
@@ -91,24 +88,21 @@ processCitations (Pandoc meta bs) = do
9188 _ -> id) $ []
9289 let bibs = mconcat $ map (\(ident, out) ->
9390 B.divWith ("ref-" <> ident,["csl-entry"],[]) . B.para .
94− walk (convertQuotes locale) . insertSpace $ out)
91+ walk (convertQuotes locale) .
92+ insertSpace $ out)
9593 (resultBibliography result)
96− let moveNotes = maybe True truish $
97− lookupMeta "notes-after-punctuation" meta
98− let cits = map (walk fixLinks . walk (convertQuotes locale)) $
94+ let moveNotes = styleIsNoteStyle sopts &&
95+ maybe True truish (lookupMeta "notes-after-punctuation" meta)
96+ let cits = map (walk (convertQuotes locale)) $
9997 resultCitations result
10098
101− let fixQuotes = case localePunctuationInQuote locale of
102− Just True ->
103− B.toList . movePunctuationInsideQuotes . B.fromList
104− _ -> id
105−
10699 let metanocites = lookupMeta "nocite" meta
107100 let Pandoc meta'' bs' =
108101 maybe id (setMeta "nocite") metanocites .
109− walk (map capitalizeNoteCitation .
110− fixQuotes . mvPunct moveNotes locale) .
111− walk deNote .
102+ walk (mvPunct moveNotes locale) .
103+ (if styleIsNoteStyle sopts
104+ then walk addNote . walk deNote
105+ else id) .
112106 evalState (walkM insertResolvedCitations $ Pandoc meta' bs)
113107 $ cits
114108 return $ Pandoc meta''
@@ -197,7 +191,9 @@ getReferences mblocale (Pandoc meta bs) = do
197191 then const True
198192 else (`Set.member` citeIds)
199193 let inlineRefs = case lookupMeta "references" meta of
200− Just (MetaList rs) -> mapMaybe metaValueToReference rs
194+ Just (MetaList rs) ->
195+ filter (idpred . unItemId . referenceId)
196+ $ mapMaybe metaValueToReference rs
201197 _ -> []
202198 externalRefs <- case lookupMeta "bibliography" meta of
203199 Just (MetaList xs) ->
@@ -294,7 +290,7 @@ insertResolvedCitations (Cite cs ils) = do
294290 [] -> return (Cite cs ils)
295291 (x:xs) -> do
296292 put xs
297− return $ Cite cs (B.toList x)
293+ return $ Cite cs (walk fixLinks $ B.toList x)
298294insertResolvedCitations x = return x
299295
300296getCitations :: Locale
@@ -376,7 +372,6 @@ formatFromExtension fp = case dropWhile (== '.') $ takeExtension fp of
376372
377373
378374isNote :: Inline -> Bool
379−isNote (Note _) = True
380375isNote (Cite _ [Note _]) = True
381376 -- the following allows citation styles that are "in-text" but use superscript
382377 -- references to be treated as if they are "notes" for the purposes of moving
@@ -389,6 +384,12 @@ isSpacy Space = True
389384isSpacy SoftBreak = True
390385isSpacy _ = False
391386
387+movePunctInsideQuotes :: Locale -> [Inline] -> [Inline]
388+movePunctInsideQuotes locale
389+ | localePunctuationInQuote locale == Just True
390+ = B.toList . movePunctuationInsideQuotes . B.fromList
391+ | otherwise
392+ = id
392393
393394mvPunct :: Bool -> Locale -> [Inline] -> [Inline]
394395mvPunct moveNotes locale (x : xs)
@@ -401,7 +402,8 @@ mvPunct moveNotes locale (q : s : x : ys)
401402 in if moveNotes
402403 then if T.null spunct
403404 then q : x : mvPunct moveNotes locale ys
404− else q : Str spunct : x : mvPunct moveNotes locale
405+ else movePunctInsideQuotes locale
406+ [q , Str spunct , x] ++ mvPunct moveNotes locale
405407 (B.toList
406408 (dropTextWhile isPunctuation (B.fromList ys)))
407409 else q : x : mvPunct moveNotes locale ys
@@ -413,9 +415,10 @@ mvPunct moveNotes locale (Cite cs ils : ys)
413415 , moveNotes
414416 = let s = stringify ys
415417 spunct = T.takeWhile isPunctuation s
416− in Cite cs (init ils
417− ++ [Str spunct | not (endWithPunct False (init ils))]
418− ++ [last ils]) :
418+ in Cite cs (movePunctInsideQuotes locale $
419+ init ils
420+ ++ [Str spunct | not (endWithPunct False (init ils))]
421+ ++ [last ils]) :
419422 mvPunct moveNotes locale
420423 (B.toList (dropTextWhile isPunctuation (B.fromList ys)))
421424mvPunct moveNotes locale (s : x : ys) | isSpacy s, isNote x =
@@ -431,7 +434,7 @@ mvPunct _ _ [] = []
431434-- move https://doi.org etc. prefix inside link text (#6723):
432435fixLinks :: [Inline] -> [Inline]
433436fixLinks (Str t : Link attr [Str u1] (u2,tit) : xs)
434− | t <> u1 == u2
437+ | u2 == t <> u1
435438 = Link attr [Str (t <> u1)] (u2,tit) : fixLinks xs
436439fixLinks (x:xs) = x : fixLinks xs
437440fixLinks [] = []
@@ -501,7 +504,8 @@ insertRefs refkvs refclasses meta refs bs =
501504 put True
502505 -- refHeader isn't used if you have an explicit references div
503506 let cs' = ordNub $ cs ++ refclasses
504− return $ Div ("refs",cs',kvs) (xs ++ refs)
507+ let kvs' = ordNub $ kvs ++ refkvs
508+ return $ Div ("refs",cs',kvs') (xs ++ refs)
505509 go x = return x
506510
507511refTitle :: Meta -> Maybe [Inline]
@@ -560,42 +564,62 @@ extractText (FancyVal x) = toText x
560564extractText (NumVal n) = T.pack (show n)
561565extractText _ = mempty
562566
563−capitalizeNoteCitation :: Inline -> Inline
564−capitalizeNoteCitation (Cite cs [Note [Para ils]]) =
565− Cite cs
566− [Note [Para $ B.toList $ addTextCase Nothing CapitalizeFirst
567− $ B.fromList ils]]
568−capitalizeNoteCitation x = x
569−
570−deNote :: [Inline] -> [Inline]
571−deNote [] = []
572−deNote (Note bs:rest) =
573− Note (walk go bs) : deNote rest
567+-- Here we take the Spans with class csl-note that are left
568+-- after deNote has removed nested ones, and convert them
569+-- into real notes.
570+addNote :: Inline -> Inline
571+addNote (Span ("",["csl-note"],[]) ils) =
572+ Note [Para $
573+ B.toList . addTextCase Nothing CapitalizeFirst . B.fromList $ ils]
574+addNote x = x
575+
576+-- Here we handle citation notes that occur inside footnotes
577+-- or other citation notes, in a note style. We don't want
578+-- notes inside notes, so we convert these to parenthesized
579+-- or comma-separated citations.
580+deNote :: Inline -> Inline
581+deNote (Note bs) =
582+ case bs of
583+ [Para (cit@(Cite (c:_) _) : ils)]
584+ | citationMode c /= AuthorInText ->
585+ -- if citation is first in note, no need to parenthesize.
586+ Note [Para (walk removeNotes $ cit : walk addParens ils)]
587+ _ -> Note (walk removeNotes . walk addParens $ bs)
588+
574589 where
575− go [] = []
576− go (Cite (c:cs) ils : zs)
590+ addParens [] = []
591+ addParens (Cite (c:cs) ils : zs)
577592 | citationMode c == AuthorInText
578− = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) : go zs
593+ = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) :
594+ addParens zs
579595 | otherwise
580− = Cite (c:cs) (concatMap noteInParens ils) : go zs
581− go (x:xs) = x : go xs
596+ = Cite (c:cs) (concatMap noteInParens ils) : addParens zs
597+ addParens (x:xs) = x : addParens xs
598+
599+ removeNotes (Span ("",["csl-note"],[]) ils) = Span ("",[],[]) ils
600+ removeNotes x = x
601+
582602 needsPeriod [] = True
583603 needsPeriod (Str t:_) = case T.uncons t of
584604 Nothing -> False
585605 Just (c,_) -> isUpper c
586606 needsPeriod (Space:zs) = needsPeriod zs
587607 needsPeriod _ = False
588− noteInParens (Note bs')
608+
609+ noteInParens (Span ("",["csl-note"],[]) ils)
589610 = Space : Str "(" :
590− removeFinalPeriod (blocksToInlines bs') ++ [Str ")"]
611+ removeFinalPeriod ils ++ [Str ")"]
591612 noteInParens x = [x]
592− noteAfterComma needsPer (Note bs')
613+
614+ noteAfterComma needsPer (Span ("",["csl-note"],[]) ils)
615+ | not (null ils)
593616 = Str "," : Space :
594− (if needsPer
595− then id
596− else removeFinalPeriod) (blocksToInlines bs')
617+ if needsPer
618+ then ils
619+ else removeFinalPeriod ils
597620 noteAfterComma _ x = [x]
598−deNote (x:xs) = x : deNote xs
621+
622+deNote x = x
599623
600624-- Note: we can't use dropTextWhileEnd indiscriminately,
601625-- because this would remove the final period on abbreviations like Ibid.
@@ -630,13 +654,8 @@ removeFinalPeriod ils =
630654
631655bcp47LangToIETF :: PandocMonad m => Text -> m (Maybe Lang)
632656bcp47LangToIETF bcplang =
633− case BCP47.parseBCP47 bcplang of
657+ case parseLang bcplang of
634658 Left _ -> do
635659 report $ InvalidLang bcplang
636660 return Nothing
637− Right lang ->
638− return $ Just
639− $ Lang (BCP47.langLanguage lang)
640− (if T.null (BCP47.langRegion lang)
641− then Nothing
642− else Just (BCP47.langRegion lang))
661+ Right lang -> return $ Just lang
src/Text/Pandoc/Citeproc/BibTeX.hs+25−15
@@ -59,10 +59,11 @@ data Variant = Bibtex | Biblatex
5959 deriving (Show, Eq, Ord)
6060
6161-- | Parse BibTeX or BibLaTeX into a list of 'Reference's.
62−readBibtexString :: Variant -- ^ bibtex or biblatex
62+readBibtexString :: ToSources a
63+ => Variant -- ^ bibtex or biblatex
6364 -> Locale -- ^ Locale
6465 -> (Text -> Bool) -- ^ Filter on citation ids
65− -> Text -- ^ bibtex/biblatex text
66+ -> a -- ^ bibtex/biblatex text
6667 -> Either ParseError [Reference Inlines]
6768readBibtexString variant locale idpred contents = do
6869 case runParser (((resolveCrossRefs variant <$> bibEntries) <* eof) >>=
@@ -70,9 +71,11 @@ readBibtexString variant locale idpred contents = do
7071 filter (\item -> idpred (identifier item) &&
7172 entryType item /= "xdata"))
7273 (fromMaybe defaultLang $ localeLanguage locale, Map.empty)
73− "" contents of
74+ (initialSourceName sources) sources of
7475 Left err -> Left err
7576 Right xs -> return xs
77+ where
78+ sources = toSources contents
7679
7780-- | Write BibTeX or BibLaTeX given given a 'Reference'.
7881writeBibtexString :: WriterOptions -- ^ options (for writing LaTex)
@@ -155,6 +158,7 @@ writeBibtexString opts variant mblang ref =
155158 , "langid"
156159 , "abstract"
157160 , "keywords"
161+ , "annote"
158162 ]
159163 Bibtex ->
160164 [ "author"
@@ -174,6 +178,7 @@ writeBibtexString opts variant mblang ref =
174178 , "address"
175179 , "type"
176180 , "note"
181+ , "annote"
177182 ]
178183
179184 valToInlines (TextVal t) = B.text t
@@ -205,10 +210,13 @@ writeBibtexString opts variant mblang ref =
205210 [ (", " <>) <$> nameGiven name,
206211 nameDroppingParticle name ]
207212
208− mblang' = (parseLang <$> getVariableAsText "language") <|> mblang
213+ mblang' = case getVariableAsText "language" of
214+ Just l -> either (const Nothing) Just $ parseLang l
215+ Nothing -> mblang
209216
210217 titlecase = case mblang' of
211− Just (Lang "en" _) -> titlecase'
218+ Just lang | langLanguage lang == "en"
219+ -> titlecase'
212220 Nothing -> titlecase'
213221 _ ->
214222 case variant of
@@ -331,12 +339,12 @@ writeBibtexString opts variant mblang ref =
331339 renderFields = mconcat . intersperse ("," <> cr) . mapMaybe renderField
332340
333341defaultLang :: Lang
334−defaultLang = Lang "en" (Just "US")
342+defaultLang = Lang "en" Nothing (Just "US") [] [] []
335343
336344-- a map of bibtex "string" macros
337345type StringMap = Map.Map Text Text
338346
339−type BibParser = Parser Text (Lang, StringMap)
347+type BibParser = Parser Sources (Lang, StringMap)
340348
341349data Item = Item{ identifier :: Text
342350 , sourcePos :: SourcePos
@@ -351,9 +359,7 @@ itemToReference locale variant item = do
351359 bib item $ do
352360 let lang = fromMaybe defaultLang $ localeLanguage locale
353361 modify $ \st -> st{ localeLang = lang,
354− untitlecase = case lang of
355− (Lang "en" _) -> True
356− _ -> False }
362+ untitlecase = langLanguage lang == "en" }
357363
358364 id' <- asks identifier
359365 otherIds <- (Just <$> getRawField "ids")
@@ -711,7 +717,7 @@ itemToReference locale variant item = do
711717
712718
713719bib :: Item -> Bib a -> BibParser a
714−bib entry m = fst <$> evalRWST m entry (BibState True (Lang "en" (Just "US")))
720+bib entry m = fst <$> evalRWST m entry (BibState True defaultLang)
715721
716722resolveCrossRefs :: Variant -> [Item] -> [Item]
717723resolveCrossRefs variant entries =
@@ -803,7 +809,7 @@ bibEntries = do
803809 (bibComment <|> bibPreamble <|> bibString))
804810
805811bibSkip :: BibParser ()
806−bibSkip = () <$ take1WhileP (/='@')
812+bibSkip = skipMany1 (satisfy (/='@'))
807813
808814bibComment :: BibParser ()
809815bibComment = do
@@ -828,6 +834,9 @@ bibString = do
828834 updateState (\(l,m) -> (l, Map.insert k v m))
829835 return ()
830836
837+take1WhileP :: Monad m => (Char -> Bool) -> ParserT Sources u m Text
838+take1WhileP f = T.pack <$> many1 (satisfy f)
839+
831840inBraces :: BibParser Text
832841inBraces = do
833842 char '{'
@@ -859,7 +868,7 @@ fieldName = resolveAlias . T.toLower
859868
860869isBibtexKeyChar :: Char -> Bool
861870isBibtexKeyChar c =
862− isAlphaNum c || c `elem` (".:;?!`'()/*@_+=-[]*&" :: [Char])
871+ isAlphaNum c || c `elem` (".:;?!`'()$/*@_+=-[]*&" :: [Char])
863872
864873bibItem :: BibParser Item
865874bibItem = do
@@ -1456,8 +1465,9 @@ resolveKey lang ils = Walk.walk go ils
14561465 go x = x
14571466
14581467resolveKey' :: Lang -> Text -> Text
1459−resolveKey' lang@(Lang l _) k =
1460− case Map.lookup l biblatexStringMap >>= Map.lookup (T.toLower k) of
1468+resolveKey' lang k =
1469+ case Map.lookup (langLanguage lang) biblatexStringMap >>=
1470+ Map.lookup (T.toLower k) of
14611471 Nothing -> k
14621472 Just (x, _) -> either (const k) stringify $ parseLaTeX lang x
14631473
src/Text/Pandoc/Citeproc/Data.hs+7−6
@@ -10,7 +10,7 @@ import qualified Data.Text.Encoding as TE
1010import qualified Data.Text as T
1111import Data.Text (Text)
1212import Text.Pandoc.Citeproc.Util (toIETF)
13−import Citeproc (Lang(..), parseLang)
13+import Text.Collate.Lang (Lang(..), parseLang)
1414
1515biblatexLocalizations :: [(FilePath, ByteString)]
1616biblatexLocalizations = $(embedDir "citeproc/biblatex-localization")
@@ -21,11 +21,12 @@ biblatexStringMap :: M.Map Text (M.Map Text (Text, Text))
2121biblatexStringMap = foldr go mempty biblatexLocalizations
2222 where
2323 go (fp, bs) =
24− let Lang lang _ = parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp)
25− ls = T.lines $ TE.decodeUtf8 bs
26− in if length ls > 4
27− then M.insert lang (toStringMap $ map (T.splitOn "|") ls)
28− else id
24+ let ls = T.lines $ TE.decodeUtf8 bs
25+ in case parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp) of
26+ Right lang | length ls > 4
27+ -> M.insert (langLanguage lang)
28+ (toStringMap $ map (T.splitOn "|") ls)
29+ _ -> id
2930 toStringMap = foldr go' mempty
3031 go' [term, x, y] = M.insert term (x, y)
3132 go' _ = id
src/Text/Pandoc/Citeproc/Locator.hs+20−4
@@ -7,6 +7,7 @@ where
77import Citeproc.Types
88import Data.Text (Text)
99import qualified Data.Text as T
10+import Data.List (foldl')
1011import Text.Parsec
1112import Text.Pandoc.Definition
1213import Text.Pandoc.Parsing (romanNumeral)
@@ -19,7 +20,7 @@ parseLocator :: Locale -> [Inline] -> (Maybe (Text, Text), [Inline])
1920parseLocator locale inp =
2021 case parse (pLocatorWords (toLocatorMap locale)) "suffix" $ splitInp inp of
2122 Right r -> r
22− Left _ -> (Nothing, inp)
23+ Left _ -> (Nothing, maybeAddComma inp)
2324
2425splitInp :: [Inline] -> [Inline]
2526splitInp = splitStrWhen (\c -> isSpace c || (isPunctuation c && c /= ':'))
@@ -41,9 +42,17 @@ pLocatorWords locMap = do
4142 -- i.e. the first one will be " 9"
4243 return $
4344 if T.null la && T.null lo
44− then (Nothing, s)
45+ then (Nothing, maybeAddComma s)
4546 else (Just (la, T.strip lo), s)
4647
48+maybeAddComma :: [Inline] -> [Inline]
49+maybeAddComma [] = []
50+maybeAddComma ils@(Space : _) = ils
51+maybeAddComma ils@(Str t : _)
52+ | Just (c, _) <- T.uncons t
53+ , isPunctuation c = ils
54+maybeAddComma ils = Str "," : Space : ils
55+
4756pLocatorDelimited :: LocatorMap -> LocatorParser (Text, Text)
4857pLocatorDelimited locMap = try $ do
4958 _ <- pMatchChar "{" (== '{')
@@ -96,7 +105,7 @@ pLocatorLabel' locMap lim = go ""
96105 t <- anyToken
97106 ts <- manyTill anyToken (try $ lookAhead lim)
98107 let s = acc <> stringify (t:ts)
99− case M.lookup (T.strip s) locMap of
108+ case M.lookup (T.toCaseFold $ T.strip s) locMap of
100109 -- try to find a longer one, or return this one
101110 Just l -> go s <|> return (l, False)
102111 Nothing -> go s
@@ -139,7 +148,7 @@ pBalancedBraces braces p = try $ do
139148 where
140149 except = notFollowedBy pBraces >> p
141150 -- outer and inner
142− surround = foldl (\a (open, close) -> sur open close except <|> a)
151+ surround = foldl' (\a (open, close) -> sur open close except <|> a)
143152 except
144153 braces
145154
@@ -180,6 +189,7 @@ pPageUnit = roman <|> plainUnit
180189 plainUnit = do
181190 ts <- many1 (notFollowedBy pSpace >>
182191 notFollowedBy pLocatorPunct >>
192+ notFollowedBy pMath >>
183193 anyToken)
184194 let s = stringify ts
185195 -- otherwise look for actual digits or -s
@@ -210,6 +220,12 @@ pMatchChar msg f = satisfyTok f' <?> msg
210220pSpace :: LocatorParser Inline
211221pSpace = satisfyTok (\t -> isSpacey t || t == Str "\160") <?> "space"
212222
223+pMath :: LocatorParser Inline
224+pMath = satisfyTok isMath
225+ where
226+ isMath (Math{}) = True
227+ isMath _ = False
228+
213229satisfyTok :: (Inline -> Bool) -> LocatorParser Inline
214230satisfyTok f = tokenPrim show (\sp _ _ -> sp) (\tok -> if f tok
215231 then Just tok
src/Text/Pandoc/Class/CommonState.hs+1−1
@@ -19,7 +19,7 @@ where
1919
2020import Data.Default (Default (def))
2121import Data.Text (Text)
22−import Text.Pandoc.BCP47 (Lang)
22+import Text.Collate.Lang (Lang)
2323import Text.Pandoc.MediaBag (MediaBag)
2424import Text.Pandoc.Logging (LogMessage, Verbosity (WARNING))
2525import Text.Pandoc.Translations (Translations)
src/Text/Pandoc/Class/IO.hs+21−22
@@ -62,7 +62,7 @@ import Text.Pandoc.Definition (Pandoc, Inline (Image))
6262import Text.Pandoc.Error (PandocError (..))
6363import Text.Pandoc.Logging (LogMessage (..), messageVerbosity, showLogMessage)
6464import Text.Pandoc.MIME (MimeType)
65−import Text.Pandoc.MediaBag (MediaBag, lookupMedia, mediaDirectory)
65+import Text.Pandoc.MediaBag (MediaBag, MediaItem(..), lookupMedia, mediaItems)
6666import Text.Pandoc.Walk (walk)
6767import qualified Control.Exception as E
6868import qualified Data.ByteString as B
@@ -200,33 +200,32 @@ alertIndent (l:ls) = do
200200extractMedia :: (PandocMonad m, MonadIO m) => FilePath -> Pandoc -> m Pandoc
201201extractMedia dir d = do
202202 media <- getMediaBag
203− case [fp | (fp, _, _) <- mediaDirectory media] of
204− [] -> return d
205− fps -> do
206− mapM_ (writeMedia dir media) fps
207− return $ walk (adjustImagePath dir fps) d
203+ let items = mediaItems media
204+ if null items
205+ then return d
206+ else do
207+ mapM_ (writeMedia dir) items
208+ return $ walk (adjustImagePath dir media) d
208209
209210-- | Write the contents of a media bag to a path.
210211writeMedia :: (PandocMonad m, MonadIO m)
211− => FilePath -> MediaBag -> FilePath
212+ => FilePath
213+ -> (FilePath, MimeType, BL.ByteString)
212214 -> m ()
213−writeMedia dir mediabag subpath = do
214− -- we join and split to convert a/b/c to a\b\c on Windows;
215− -- in zip containers all paths use /
216− let fullpath = dir </> unEscapeString (normalise subpath)
217− let mbcontents = lookupMedia subpath mediabag
218− case mbcontents of
219− Nothing -> throwError $ PandocResourceNotFound $ pack subpath
220− Just (_, bs) -> do
221− report $ Extracting $ pack fullpath
222− liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
223− logIOError $ BL.writeFile fullpath bs
215+writeMedia dir (fp, _mt, bs) = do
216+ -- we normalize to get proper path separators for the platform
217+ let fullpath = normalise $ dir </> fp
218+ liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
219+ logIOError $ BL.writeFile fullpath bs
224220
225221-- | If the given Inline element is an image with a @src@ path equal to
226222-- one in the list of @paths@, then prepends @dir@ to the image source;
227223-- returns the element unchanged otherwise.
228−adjustImagePath :: FilePath -> [FilePath] -> Inline -> Inline
229−adjustImagePath dir paths (Image attr lab (src, tit))
230− | unpack src `elem` paths
231− = Image attr lab (pack (normalise $ dir </> unpack src), tit)
224+adjustImagePath :: FilePath -> MediaBag -> Inline -> Inline
225+adjustImagePath dir mediabag (Image attr lab (src, tit)) =
226+ case lookupMedia (T.unpack src) mediabag of
227+ Nothing -> Image attr lab (src, tit)
228+ Just item ->
229+ let fullpath = dir </> mediaPath item
230+ in Image attr lab (T.pack fullpath, tit)
232231adjustImagePath _ _ x = x
src/Text/Pandoc/Class/PandocMonad.hs+24−34
@@ -1,4 +1,5 @@
11{-# LANGUAGE CPP #-}
2+{-# LANGUAGE TupleSections #-}
23{-# LANGUAGE FlexibleContexts #-}
34{-# LANGUAGE FlexibleInstances #-}
45{-# LANGUAGE MultiParamTypeClasses #-}
@@ -37,7 +38,6 @@ module Text.Pandoc.Class.PandocMonad
3738 , setUserDataDir
3839 , getUserDataDir
3940 , fetchItem
40− , fetchMediaResource
4141 , getInputFiles
4242 , setInputFiles
4343 , getOutputFile
@@ -57,8 +57,7 @@ module Text.Pandoc.Class.PandocMonad
5757import Codec.Archive.Zip
5858import Control.Monad.Except (MonadError (catchError, throwError),
5959 MonadTrans, lift, when)
60−import Data.Digest.Pure.SHA (sha1, showDigest)
61−import Data.Maybe (fromMaybe)
60+import Data.List (foldl')
6261import Data.Time (UTCTime)
6362import Data.Time.Clock.POSIX (POSIXTime, utcTimeToPOSIXSeconds,
6463 posixSecondsToUTCTime)
@@ -66,16 +65,16 @@ import Data.Time.LocalTime (TimeZone, ZonedTime, utcToZonedTime)
6665import Network.URI ( escapeURIString, nonStrictRelativeTo,
6766 unEscapeString, parseURIReference, isAllowedInURI,
6867 parseURI, URI(..) )
69−import System.FilePath ((</>), (<.>), takeExtension, dropExtension,
70− isRelative, splitDirectories)
68+import System.FilePath ((</>), takeExtension, dropExtension,
69+ isRelative, splitDirectories, makeRelative)
7170import System.Random (StdGen)
72−import Text.Pandoc.BCP47 (Lang(..), parseBCP47, renderLang)
71+import Text.Collate.Lang (Lang(..), parseLang, renderLang)
7372import Text.Pandoc.Class.CommonState (CommonState (..))
7473import Text.Pandoc.Definition
7574import Text.Pandoc.Error
7675import Text.Pandoc.Logging
77−import Text.Pandoc.MIME (MimeType, getMimeType, extensionFromMimeType)
78−import Text.Pandoc.MediaBag (MediaBag, lookupMedia)
76+import Text.Pandoc.MIME (MimeType, getMimeType)
77+import Text.Pandoc.MediaBag (MediaBag, lookupMedia, MediaItem(..))
7978import Text.Pandoc.Shared (uriPathToPath, safeRead)
8079import Text.Pandoc.Translations (Term(..), Translations, lookupTerm,
8180 readTranslations)
@@ -284,7 +283,7 @@ readFileFromDirs (d:ds) f = catchError
284283toLang :: PandocMonad m => Maybe T.Text -> m (Maybe Lang)
285284toLang Nothing = return Nothing
286285toLang (Just s) =
287− case parseBCP47 s of
286+ case parseLang s of
288287 Left _ -> do
289288 report $ InvalidLang s
290289 return Nothing
@@ -375,7 +374,8 @@ fetchItem :: PandocMonad m
375374fetchItem s = do
376375 mediabag <- getMediaBag
377376 case lookupMedia (T.unpack s) mediabag of
378− Just (mime, bs) -> return (BL.toStrict bs, Just mime)
377+ Just item -> return (BL.toStrict (mediaContents item),
378+ Just (mediaMimeType item))
379379 Nothing -> downloadOrRead s
380380
381381-- | Returns the content and, if available, the MIME type of a resource.
@@ -410,9 +410,10 @@ downloadOrRead s = do
410410 _ -> readLocalFile fp -- get from local file system
411411 where readLocalFile f = do
412412 resourcePath <- getResourcePath
413− cont <- if isRelative f
414− then withPaths resourcePath readFileStrict f
415− else readFileStrict f
413+ (fp', cont) <- if isRelative f
414+ then withPaths resourcePath readFileStrict f
415+ else (f,) <$> readFileStrict f
416+ report $ LoadedResource f (makeRelative "." fp')
416417 return (cont, mime)
417418 httpcolon = URI{ uriScheme = "http:",
418419 uriAuthority = Nothing,
@@ -612,7 +613,7 @@ checkExistence fn = do
612613-- | Canonicalizes a file path by removing redundant @.@ and @..@.
613614makeCanonical :: FilePath -> FilePath
614615makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
615− where transformPathParts = reverse . foldl go []
616+ where transformPathParts = reverse . foldl' go []
616617 go as "." = as
617618 go (_:as) ".." = as
618619 go as x = x : as
@@ -622,25 +623,13 @@ makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
622623-- that filepath. Returns the result of the first successful execution
623624-- of the action, or throws a @PandocResourceNotFound@ exception if the
624625-- action errors for all filepaths.
625−withPaths :: PandocMonad m => [FilePath] -> (FilePath -> m a) -> FilePath -> m a
626+withPaths :: PandocMonad m
627+ => [FilePath] -> (FilePath -> m a) -> FilePath -> m (FilePath, a)
626628withPaths [] _ fp = throwError $ PandocResourceNotFound $ T.pack fp
627629withPaths (p:ps) action fp =
628− catchError (action (p </> fp))
630+ catchError ((p </> fp,) <$> action (p </> fp))
629631 (\_ -> withPaths ps action fp)
630632
631−-- | Fetch local or remote resource (like an image) and provide data suitable
632−-- for adding it to the MediaBag.
633−fetchMediaResource :: PandocMonad m
634− => T.Text -> m (FilePath, Maybe MimeType, BL.ByteString)
635−fetchMediaResource src = do
636− (bs, mt) <- downloadOrRead src
637− let ext = fromMaybe (T.pack $ takeExtension $ T.unpack src)
638− (mt >>= extensionFromMimeType)
639− let bs' = BL.fromChunks [bs]
640− let basename = showDigest $ sha1 bs'
641− let fname = basename <.> T.unpack ext
642− return (fname, mt, bs')
643−
644633-- | Traverse tree, filling media bag for any images that
645634-- aren't already in the media bag.
646635fillMediaBag :: PandocMonad m => Pandoc -> m Pandoc
@@ -648,12 +637,13 @@ fillMediaBag d = walkM handleImage d
648637 where handleImage :: PandocMonad m => Inline -> m Inline
649638 handleImage (Image attr lab (src, tit)) = catchError
650639 (do mediabag <- getMediaBag
651− case lookupMedia (T.unpack src) mediabag of
652− Just (_, _) -> return $ Image attr lab (src, tit)
640+ let fp = T.unpack src
641+ case lookupMedia fp mediabag of
642+ Just _ -> return ()
653643 Nothing -> do
654− (fname, mt, bs) <- fetchMediaResource src
655− insertMedia fname mt bs
656− return $ Image attr lab (T.pack fname, tit))
644+ (bs, mt) <- fetchItem src
645+ insertMedia fp mt (BL.fromStrict bs)
646+ return $ Image attr lab (src, tit))
657647 (\e ->
658648 case e of
659649 PandocResourceNotFound _ -> do
src/Text/Pandoc/Error.hs+23−16
@@ -23,26 +23,27 @@ import Control.Exception (Exception, displayException)
2323import Data.Typeable (Typeable)
2424import Data.Word (Word8)
2525import Data.Text (Text)
26+import Data.List (sortOn)
2627import qualified Data.Text as T
28+import Data.Ord (Down(..))
2729import GHC.Generics (Generic)
2830import Network.HTTP.Client (HttpException)
2931import System.Exit (ExitCode (..), exitWith)
3032import System.IO (stderr)
3133import qualified Text.Pandoc.UTF8 as UTF8
34+import Text.Pandoc.Sources (Sources(..))
3235import Text.Printf (printf)
3336import Text.Parsec.Error
3437import Text.Parsec.Pos hiding (Line)
3538import Text.Pandoc.Shared (tshow)
3639import Citeproc (CiteprocError, prettyCiteprocError)
3740
38−type Input = Text
39−
4041data PandocError = PandocIOError Text IOError
4142 | PandocHttpError Text HttpException
4243 | PandocShouldNeverHappenError Text
4344 | PandocSomeError Text
4445 | PandocParseError Text
45− | PandocParsecError Input ParseError
46+ | PandocParsecError Sources ParseError
4647 | PandocMakePDFError Text
4748 | PandocOptionError Text
4849 | PandocSyntaxMapError Text
@@ -81,22 +82,28 @@ renderError e =
8182 "Please report this to pandoc's developers: " <> s
8283 PandocSomeError s -> s
8384 PandocParseError s -> s
84− PandocParsecError input err' ->
85+ PandocParsecError (Sources inputs) err' ->
8586 let errPos = errorPos err'
8687 errLine = sourceLine errPos
8788 errColumn = sourceColumn errPos
88− ls = T.lines input <> [""]
89− errorInFile = if length ls > errLine - 1
90− then T.concat ["\n", ls !! (errLine - 1)
91− ,"\n", T.replicate (errColumn - 1) " "
92− ,"^"]
93− else ""
94− in "\nError at " <> tshow err' <>
95− -- if error comes from a chunk or included file,
96− -- then we won't get the right text this way:
97− if sourceName errPos == "source"
98− then errorInFile
99− else ""
89+ errFile = sourceName errPos
90+ errorInFile =
91+ case sortOn (Down . sourceLine . fst)
92+ [ (pos,t)
93+ | (pos,t) <- inputs
94+ , sourceName pos == errFile
95+ , sourceLine pos <= errLine
96+ ] of
97+ [] -> ""
98+ ((pos,txt):_) ->
99+ let ls = T.lines txt <> [""]
100+ ln = (errLine - sourceLine pos) + 1
101+ in if length ls > ln && ln >= 1
102+ then T.concat ["\n", ls !! (ln - 1)
103+ ,"\n", T.replicate (errColumn - 1) " "
104+ ,"^"]
105+ else ""
106+ in "Error at " <> tshow err' <> errorInFile
100107 PandocMakePDFError s -> s
101108 PandocOptionError s -> s
102109 PandocSyntaxMapError s -> s
src/Text/Pandoc/Extensions.hs+9−1
@@ -34,6 +34,7 @@ module Text.Pandoc.Extensions ( Extension(..)
3434where
3535import Data.Bits (clearBit, setBit, testBit, (.|.))
3636import Data.Data (Data)
37+import Data.List (foldl')
3738import qualified Data.Text as T
3839import Data.Typeable (Typeable)
3940import GHC.Generics (Generic)
@@ -135,6 +136,8 @@ data Extension =
135136 | Ext_raw_html -- ^ Allow raw HTML
136137 | Ext_raw_tex -- ^ Allow raw TeX (other than math)
137138 | Ext_raw_markdown -- ^ Parse markdown in ipynb as raw markdown
139+ | Ext_rebase_relative_paths -- ^ Rebase relative image and link paths,
140+ -- relative to directory of containing file
138141 | Ext_shortcut_reference_links -- ^ Shortcut reference links
139142 | Ext_simple_tables -- ^ Pandoc-style simple tables
140143 | Ext_smart -- ^ "Smart" quotes, apostrophes, ellipses, dashes
@@ -353,6 +356,7 @@ getDefaultExtensions "gfm" = extensionsFromList
353356 , Ext_strikeout
354357 , Ext_task_lists
355358 , Ext_emoji
359+ , Ext_yaml_metadata_block
356360 ]
357361getDefaultExtensions "commonmark" = extensionsFromList
358362 [Ext_raw_html]
@@ -378,6 +382,7 @@ getDefaultExtensions "commonmark_x" = extensionsFromList
378382 , Ext_raw_attribute
379383 , Ext_implicit_header_references
380384 , Ext_attributes
385+ , Ext_yaml_metadata_block
381386 ]
382387getDefaultExtensions "org" = extensionsFromList
383388 [Ext_citations,
@@ -459,6 +464,7 @@ getAllExtensions f = universalExtensions <> getAll f
459464 , Ext_gutenberg
460465 , Ext_smart
461466 , Ext_literate_haskell
467+ , Ext_rebase_relative_paths
462468 ]
463469 getAll "markdown_strict" = allMarkdownExtensions
464470 getAll "markdown_phpextra" = allMarkdownExtensions
@@ -510,6 +516,8 @@ getAllExtensions f = universalExtensions <> getAll f
510516 , Ext_implicit_header_references
511517 , Ext_attributes
512518 , Ext_sourcepos
519+ , Ext_yaml_metadata_block
520+ , Ext_rebase_relative_paths
513521 ]
514522 getAll "commonmark_x" = getAll "commonmark"
515523 getAll "org" = autoIdExtensions <>
@@ -593,7 +601,7 @@ parseFormatSpec :: T.Text
593601parseFormatSpec = parse formatSpec ""
594602 where formatSpec = do
595603 name <- formatName
596− (extsToEnable, extsToDisable) <- foldl (flip ($)) ([],[]) <$>
604+ (extsToEnable, extsToDisable) <- foldl' (flip ($)) ([],[]) <$>
597605 many extMod
598606 return (T.pack name, reverse extsToEnable, reverse extsToDisable)
599607 formatName = many1 $ noneOf "-+"
src/Text/Pandoc/Highlighting.hs+5−5
@@ -52,12 +52,12 @@ highlightingStyles =
5252 ("breezedark", breezeDark),
5353 ("haddock", haddock)]
5454
55−languages :: [T.Text]
56−languages = [T.toLower (sName s) | s <- M.elems defaultSyntaxMap]
55+languages :: SyntaxMap -> [T.Text]
56+languages syntaxmap = [T.toLower (sName s) | s <- M.elems syntaxmap]
5757
58−languagesByExtension :: T.Text -> [T.Text]
59−languagesByExtension ext =
60− [T.toLower (sName s) | s <- syntaxesByExtension defaultSyntaxMap (T.unpack ext)]
58+languagesByExtension :: SyntaxMap -> T.Text -> [T.Text]
59+languagesByExtension syntaxmap ext =
60+ [T.toLower (sName s) | s <- syntaxesByExtension syntaxmap (T.unpack ext)]
6161
6262highlight :: SyntaxMap
6363 -> (FormatOptions -> [SourceLine] -> a) -- ^ Formatter
src/Text/Pandoc/ImageSize.hs+5−1
@@ -57,7 +57,8 @@ import Codec.Picture (decodeImageWithMetadata)
5757-- quick and dirty functions to get image sizes
5858-- algorithms borrowed from wwwis.pl
5959
60−data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf deriving Show
60+data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf | Tiff
61+ deriving Show
6162data Direction = Width | Height
6263instance Show Direction where
6364 show Width = "width"
@@ -100,6 +101,8 @@ imageType :: ByteString -> Maybe ImageType
100101imageType img = case B.take 4 img of
101102 "\x89\x50\x4e\x47" -> return Png
102103 "\x47\x49\x46\x38" -> return Gif
104+ "\x49\x49\x2a\x00" -> return Tiff
105+ "\x4D\x4D\x00\x2a" -> return Tiff
103106 "\xff\xd8\xff\xe0" -> return Jpeg -- JFIF
104107 "\xff\xd8\xff\xe1" -> return Jpeg -- Exif
105108 "%PDF" -> return Pdf
@@ -124,6 +127,7 @@ imageSize opts img = checkDpi <$>
124127 Just Png -> getSize img
125128 Just Gif -> getSize img
126129 Just Jpeg -> getSize img
130+ Just Tiff -> getSize img
127131 Just Svg -> mbToEither "could not determine SVG size" $ svgSize opts img
128132 Just Eps -> mbToEither "could not determine EPS size" $ epsSize img
129133 Just Pdf -> mbToEither "could not determine PDF size" $ pdfSize img
src/Text/Pandoc/Logging.hs+32−9
@@ -36,6 +36,7 @@ import Data.Typeable (Typeable)
3636import GHC.Generics (Generic)
3737import Text.Pandoc.Definition
3838import Text.Parsec.Pos
39+import Text.Pandoc.Shared (tshow)
3940
4041-- | Verbosity level.
4142data Verbosity = ERROR | WARNING | INFO
@@ -84,6 +85,7 @@ data LogMessage =
8485 | CouldNotParseCSS Text
8586 | Fetching Text
8687 | Extracting Text
88+ | LoadedResource FilePath FilePath
8789 | NoTitleElement Text
8890 | NoLangSpecified
8991 | InvalidLang Text
@@ -100,6 +102,8 @@ data LogMessage =
100102 | FilterCompleted FilePath Integer
101103 | CiteprocWarning Text
102104 | ATXHeadingInLHS Int Text
105+ | EnvironmentVariableUndefined Text
106+ | DuplicateAttribute Text Text
103107 deriving (Show, Eq, Data, Ord, Typeable, Generic)
104108
105109instance ToJSON LogMessage where
@@ -192,6 +196,9 @@ instance ToJSON LogMessage where
192196 ["path" .= fp]
193197 Extracting fp ->
194198 ["path" .= fp]
199+ LoadedResource orig found ->
200+ ["for" .= orig
201+ ,"from" .= found]
195202 NoTitleElement fallback ->
196203 ["fallback" .= fallback]
197204 NoLangSpecified -> []
@@ -229,13 +236,20 @@ instance ToJSON LogMessage where
229236 ATXHeadingInLHS lvl contents ->
230237 ["level" .= lvl
231238 ,"contents" .= contents]
239+ EnvironmentVariableUndefined var ->
240+ ["variable" .= var ]
241+ DuplicateAttribute attr val ->
242+ ["attribute" .= attr
243+ ,"value" .= val]
232244
233245showPos :: SourcePos -> Text
234246showPos pos = Text.pack $ sn ++ "line " ++
235247 show (sourceLine pos) ++ " column " ++ show (sourceColumn pos)
236− where sn = if sourceName pos == "source" || sourceName pos == ""
237− then ""
238− else sourceName pos ++ " "
248+ where
249+ sn' = sourceName pos
250+ sn = if sn' == "source" || sn' == "" || sn' == "-"
251+ then ""
252+ else sn' ++ " "
239253
240254encodeLogMessages :: [LogMessage] -> BL.ByteString
241255encodeLogMessages ms =
@@ -268,7 +282,7 @@ showLogMessage msg =
268282 ParsingUnescaped s pos ->
269283 "Parsing unescaped '" <> s <> "' at " <> showPos pos
270284 CouldNotLoadIncludeFile fp pos ->
271− "Could not load include file '" <> fp <> "' at " <> showPos pos
285+ "Could not load include file " <> fp <> " at " <> showPos pos
272286 MacroAlreadyDefined name pos ->
273287 "Macro '" <> name <> "' already defined, ignoring at " <> showPos pos
274288 InlineNotRendered il ->
@@ -280,18 +294,18 @@ showLogMessage msg =
280294 IgnoredIOError s ->
281295 "IO Error (ignored): " <> s
282296 CouldNotFetchResource fp s ->
283− "Could not fetch resource '" <> fp <> "'" <>
297+ "Could not fetch resource " <> fp <>
284298 if Text.null s then "" else ": " <> s
285299 CouldNotDetermineImageSize fp s ->
286− "Could not determine image size for '" <> fp <> "'" <>
300+ "Could not determine image size for " <> fp <>
287301 if Text.null s then "" else ": " <> s
288302 CouldNotConvertImage fp s ->
289− "Could not convert image '" <> fp <> "'" <>
303+ "Could not convert image " <> fp <>
290304 if Text.null s then "" else ": " <> s
291305 CouldNotDetermineMimeType fp ->
292− "Could not determine mime type for '" <> fp <> "'"
306+ "Could not determine mime type for " <> fp
293307 CouldNotConvertTeXMath s m ->
294− "Could not convert TeX math '" <> s <> "', rendering as TeX" <>
308+ "Could not convert TeX math " <> s <> ", rendering as TeX" <>
295309 if Text.null m then "" else ":\n" <> m
296310 CouldNotParseCSS m ->
297311 "Could not parse CSS" <> if Text.null m then "" else ":\n" <> m
@@ -299,6 +313,8 @@ showLogMessage msg =
299313 "Fetching " <> fp <> "..."
300314 Extracting fp ->
301315 "Extracting " <> fp <> "..."
316+ LoadedResource orig found ->
317+ "Loaded " <> Text.pack orig <> " from " <> Text.pack found
302318 NoTitleElement fallback ->
303319 "This document format requires a nonempty <title> element.\n" <>
304320 "Defaulting to '" <> fallback <> "' as the title.\n" <>
@@ -345,6 +361,10 @@ showLogMessage msg =
345361 if lvl < 3
346362 then " Consider using --markdown-headings=setext."
347363 else ""
364+ EnvironmentVariableUndefined var ->
365+ "Undefined environment variable " <> var <> " in defaults file."
366+ DuplicateAttribute attr val ->
367+ "Ignoring duplicate attribute " <> attr <> "=" <> tshow val <> "."
348368
349369messageVerbosity :: LogMessage -> Verbosity
350370messageVerbosity msg =
@@ -375,6 +395,7 @@ messageVerbosity msg =
375395 CouldNotParseCSS{} -> WARNING
376396 Fetching{} -> INFO
377397 Extracting{} -> INFO
398+ LoadedResource{} -> INFO
378399 NoTitleElement{} -> WARNING
379400 NoLangSpecified -> INFO
380401 InvalidLang{} -> WARNING
@@ -391,3 +412,5 @@ messageVerbosity msg =
391412 FilterCompleted{} -> INFO
392413 CiteprocWarning{} -> WARNING
393414 ATXHeadingInLHS{} -> WARNING
415+ EnvironmentVariableUndefined{}-> WARNING
416+ DuplicateAttribute{} -> WARNING
src/Text/Pandoc/Lua/Filter.hs+4−1
@@ -13,7 +13,9 @@ module Text.Pandoc.Lua.Filter ( LuaFilterFunction
1313 , LuaFilter
1414 , runFilterFile
1515 , walkInlines
16+ , walkInlineLists
1617 , walkBlocks
18+ , walkBlockLists
1719 , module Text.Pandoc.Lua.Walk
1820 ) where
1921import Control.Applicative ((<|>))
@@ -22,6 +24,7 @@ import Control.Monad.Catch (finally, try)
2224import Data.Data (Data, DataType, dataTypeConstrs, dataTypeName, dataTypeOf,
2325 showConstr, toConstr, tyconUQname)
2426import Data.Foldable (foldrM)
27+import Data.List (foldl')
2528import Data.Map (Map)
2629import Data.Maybe (fromMaybe)
2730import Foreign.Lua (Lua, Peekable, Pushable, StackIndex)
@@ -204,7 +207,7 @@ walkMeta lf (Pandoc m bs) = do
204207
205208walkPandoc :: LuaFilter -> Pandoc -> Lua Pandoc
206209walkPandoc (LuaFilter fnMap) =
207− case foldl mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
210+ case foldl' mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
208211 Just fn -> \x -> runFilterFunction fn x *> singleElement x
209212 Nothing -> return
210213
src/Text/Pandoc/Lua/Module/MediaBag.hs+3−3
@@ -73,9 +73,9 @@ lookup fp = do
7373 res <- MB.lookupMedia fp <$> getMediaBag
7474 liftPandocLua $ case res of
7575 Nothing -> 1 <$ Lua.pushnil
76− Just (mimeType, contents) -> do
77− Lua.push mimeType
78− Lua.push contents
76+ Just item -> do
77+ Lua.push $ MB.mediaMimeType item
78+ Lua.push $ MB.mediaContents item
7979 return 2
8080
8181list :: PandocLua NumResults
src/Text/Pandoc/Lua/Module/Pandoc.hs+8−3
@@ -23,8 +23,10 @@ import Foreign.Lua (Lua, NumResults, Optional, Peekable, Pushable)
2323import System.Exit (ExitCode (..))
2424import Text.Pandoc.Class.PandocIO (runIO)
2525import Text.Pandoc.Definition (Block, Inline)
26−import Text.Pandoc.Lua.Filter (walkInlines, walkBlocks, LuaFilter, SingletonsList (..))
26+import Text.Pandoc.Lua.Filter (LuaFilter, SingletonsList (..), walkInlines,
27+ walkInlineLists, walkBlocks, walkBlockLists)
2728import Text.Pandoc.Lua.Marshaling ()
29+import Text.Pandoc.Lua.Marshaling.List (List (..))
2830import Text.Pandoc.Lua.PandocLua (PandocLua, addFunction, liftPandocLua,
2931 loadDefaultModule)
3032import Text.Pandoc.Walk (Walkable)
@@ -51,9 +53,12 @@ pushModule = do
5153 return 1
5254
5355walkElement :: (Walkable (SingletonsList Inline) a,
54− Walkable (SingletonsList Block) a)
56+ Walkable (SingletonsList Block) a,
57+ Walkable (List Inline) a,
58+ Walkable (List Block) a)
5559 => a -> LuaFilter -> PandocLua a
56−walkElement x f = liftPandocLua $ walkInlines f x >>= walkBlocks f
60+walkElement x f = liftPandocLua $
61+ walkInlines f x >>= walkInlineLists f >>= walkBlocks f >>= walkBlockLists f
5762
5863walk_inline :: Inline -> LuaFilter -> PandocLua Inline
5964walk_inline = walkElement
src/Text/Pandoc/Lua/Module/Utils.hs+1−1
@@ -146,7 +146,7 @@ from_simple_table (SimpleTable capt aligns widths head' body) = do
146146 nullAttr
147147 (Caption Nothing [Plain capt])
148148 (zipWith (\a w -> (a, toColWidth w)) aligns widths)
149− (TableHead nullAttr [blockListToRow head'])
149+ (TableHead nullAttr [blockListToRow head' | not (null head') ])
150150 [TableBody nullAttr 0 [] $ map blockListToRow body]
151151 (TableFoot nullAttr [])
152152 return (NumResults 1)
src/Text/Pandoc/MIME.hs+10
@@ -43,6 +43,16 @@ getMimeTypeDef :: FilePath -> MimeType
4343getMimeTypeDef = fromMaybe "application/octet-stream" . getMimeType
4444
4545extensionFromMimeType :: MimeType -> Maybe T.Text
46+-- few special cases, where there are multiple options:
47+extensionFromMimeType "text/plain" = Just "txt"
48+extensionFromMimeType "video/quicktime" = Just "mov"
49+extensionFromMimeType "video/mpeg" = Just "mpeg"
50+extensionFromMimeType "video/dv" = Just "dv"
51+extensionFromMimeType "image/vnd.djvu" = Just "djvu"
52+extensionFromMimeType "image/tiff" = Just "tiff"
53+extensionFromMimeType "image/jpeg" = Just "jpg"
54+extensionFromMimeType "application/xml" = Just "xml"
55+extensionFromMimeType "application/ogg" = Just "ogg"
4656extensionFromMimeType mimetype =
4757 M.lookup (T.takeWhile (/=';') mimetype) reverseMimeTypes
4858 -- note: we just look up the basic mime type, dropping the content-encoding etc.
src/Text/Pandoc/MediaBag.hs+45−14
@@ -1,4 +1,5 @@
11{-# LANGUAGE CPP #-}
2+{-# LANGUAGE OverloadedStrings #-}
23{-# LANGUAGE DeriveDataTypeable #-}
34{-# LANGUAGE GeneralizedNewtypeDeriving #-}
45{- |
@@ -14,6 +15,7 @@ Definition of a MediaBag object to hold binary resources, and an
1415interface for interacting with it.
1516-}
1617module Text.Pandoc.MediaBag (
18+ MediaItem(..),
1719 MediaBag,
1820 deleteMedia,
1921 lookupMedia,
@@ -24,29 +26,43 @@ module Text.Pandoc.MediaBag (
2426import qualified Data.ByteString.Lazy as BL
2527import Data.Data (Data)
2628import qualified Data.Map as M
27−import Data.Maybe (fromMaybe)
29+import Data.Maybe (fromMaybe, isNothing)
2830import Data.Typeable (Typeable)
2931import System.FilePath
30−import qualified System.FilePath.Posix as Posix
31−import Text.Pandoc.MIME (MimeType, getMimeTypeDef)
32+import Text.Pandoc.MIME (MimeType, getMimeTypeDef, extensionFromMimeType)
33+import Data.Text (Text)
34+import qualified Data.Text as T
35+import Data.Digest.Pure.SHA (sha1, showDigest)
36+import Network.URI (URI (..), parseURI)
37+
38+data MediaItem =
39+ MediaItem
40+ { mediaMimeType :: MimeType
41+ , mediaPath :: FilePath
42+ , mediaContents :: BL.ByteString
43+ } deriving (Eq, Ord, Show, Data, Typeable)
3244
3345-- | A container for a collection of binary resources, with names and
3446-- mime types. Note that a 'MediaBag' is a Monoid, so 'mempty'
3547-- can be used for an empty 'MediaBag', and '<>' can be used to append
3648-- two 'MediaBag's.
37−newtype MediaBag = MediaBag (M.Map [FilePath] (MimeType, BL.ByteString))
49+newtype MediaBag = MediaBag (M.Map Text MediaItem)
3850 deriving (Semigroup, Monoid, Data, Typeable)
3951
4052instance Show MediaBag where
4153 show bag = "MediaBag " ++ show (mediaDirectory bag)
4254
55+-- | We represent paths with /, in normalized form.
56+canonicalize :: FilePath -> Text
57+canonicalize = T.replace "\\" "/" . T.pack . normalise
58+
4359-- | Delete a media item from a 'MediaBag', or do nothing if no item corresponds
4460-- to the given path.
4561deleteMedia :: FilePath -- ^ relative path and canonical name of resource
4662 -> MediaBag
4763 -> MediaBag
4864deleteMedia fp (MediaBag mediamap) =
49− MediaBag $ M.delete (splitDirectories fp) mediamap
65+ MediaBag $ M.delete (canonicalize fp) mediamap
5066
5167-- | Insert a media item into a 'MediaBag', replacing any existing
5268-- value with the same name.
@@ -56,26 +72,41 @@ insertMedia :: FilePath -- ^ relative path and canonical name of resource
5672 -> MediaBag
5773 -> MediaBag
5874insertMedia fp mbMime contents (MediaBag mediamap) =
59− MediaBag (M.insert (splitDirectories fp) (mime, contents) mediamap)
60− where mime = fromMaybe fallback mbMime
75+ MediaBag (M.insert fp' mediaItem mediamap)
76+ where mediaItem = MediaItem{ mediaPath = newpath
77+ , mediaContents = contents
78+ , mediaMimeType = mt }
79+ fp' = canonicalize fp
80+ uri = parseURI fp
81+ newpath = if isRelative fp
82+ && isNothing uri
83+ && ".." `notElem` splitPath fp
84+ then T.unpack fp'
85+ else showDigest (sha1 contents) <> "." <> ext
6186 fallback = case takeExtension fp of
6287 ".gz" -> getMimeTypeDef $ dropExtension fp
6388 _ -> getMimeTypeDef fp
89+ mt = fromMaybe fallback mbMime
90+ path = maybe fp uriPath uri
91+ ext = case takeExtension path of
92+ '.':e -> e
93+ _ -> maybe "" T.unpack $ extensionFromMimeType mt
94+
6495
6596-- | Lookup a media item in a 'MediaBag', returning mime type and contents.
6697lookupMedia :: FilePath
6798 -> MediaBag
68− -> Maybe (MimeType, BL.ByteString)
69−lookupMedia fp (MediaBag mediamap) = M.lookup (splitDirectories fp) mediamap
99+ -> Maybe MediaItem
100+lookupMedia fp (MediaBag mediamap) = M.lookup (canonicalize fp) mediamap
70101
71102-- | Get a list of the file paths stored in a 'MediaBag', with
72103-- their corresponding mime types and the lengths in bytes of the contents.
73104mediaDirectory :: MediaBag -> [(FilePath, MimeType, Int)]
74−mediaDirectory (MediaBag mediamap) =
75− M.foldrWithKey (\fp (mime,contents) ->
76− ((Posix.joinPath fp, mime, fromIntegral $ BL.length contents):)) [] mediamap
105+mediaDirectory mediabag =
106+ map (\(fp, mt, bs) -> (fp, mt, fromIntegral (BL.length bs)))
107+ (mediaItems mediabag)
77108
78109mediaItems :: MediaBag -> [(FilePath, MimeType, BL.ByteString)]
79110mediaItems (MediaBag mediamap) =
80− M.foldrWithKey (\fp (mime,contents) ->
81− ((Posix.joinPath fp, mime, contents):)) [] mediamap
111+ map (\item -> (mediaPath item, mediaMimeType item, mediaContents item))
112+ (M.elems mediamap)
src/Text/Pandoc/Options.hs+17−17
@@ -315,6 +315,23 @@ defaultMathJaxURL = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.j
315315defaultKaTeXURL :: Text
316316defaultKaTeXURL = "https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/"
317317
318+-- Update documentation in doc/filters.md if this is changed.
319+$(deriveJSON defaultOptions{ constructorTagModifier =
320+ camelCaseStrToHyphenated
321+ } ''TrackChanges)
322+
323+$(deriveJSON defaultOptions{ constructorTagModifier =
324+ camelCaseStrToHyphenated
325+ } ''WrapOption)
326+
327+$(deriveJSON defaultOptions{ constructorTagModifier =
328+ camelCaseStrToHyphenated . drop 8
329+ } ''TopLevelDivision)
330+
331+$(deriveJSON defaultOptions{ constructorTagModifier =
332+ camelCaseStrToHyphenated
333+ } ''ReferenceLocation)
334+
318335-- Update documentation in doc/filters.md if this is changed.
319336$(deriveJSON defaultOptions ''ReaderOptions)
320337
@@ -338,20 +355,3 @@ $(deriveJSON defaultOptions{ constructorTagModifier =
338355 } ''ObfuscationMethod)
339356
340357$(deriveJSON defaultOptions ''HTMLSlideVariant)
341−
342−-- Update documentation in doc/filters.md if this is changed.
343−$(deriveJSON defaultOptions{ constructorTagModifier =
344− camelCaseStrToHyphenated
345− } ''TrackChanges)
346−
347−$(deriveJSON defaultOptions{ constructorTagModifier =
348− camelCaseStrToHyphenated
349− } ''WrapOption)
350−
351−$(deriveJSON defaultOptions{ constructorTagModifier =
352− camelCaseStrToHyphenated . drop 8
353− } ''TopLevelDivision)
354−
355−$(deriveJSON defaultOptions{ constructorTagModifier =
356− camelCaseStrToHyphenated
357− } ''ReferenceLocation)
src/Text/Pandoc/PDF.hs+16−3
@@ -202,7 +202,7 @@ convertImage opts tmpdir fname = do
202202 Just "image/svg+xml" -> E.catch (do
203203 (exit, _) <- pipeProcess Nothing "rsvg-convert"
204204 ["-f","pdf","-a","--dpi-x",dpi,"--dpi-y",dpi,
205− "-o",pdfOut,fname] BL.empty
205+ "-o",pdfOut,svgIn] BL.empty
206206 if exit == ExitSuccess
207207 then return $ Right pdfOut
208208 else return $ Left "conversion from SVG failed")
@@ -217,6 +217,7 @@ convertImage opts tmpdir fname = do
217217 where
218218 pngOut = replaceDirectory (replaceExtension fname ".png") tmpdir
219219 pdfOut = replaceDirectory (replaceExtension fname ".pdf") tmpdir
220+ svgIn = tmpdir </> fname
220221 mime = getMimeType fname
221222 doNothing = return (Right fname)
222223
@@ -506,8 +507,20 @@ showVerboseInfo mbTmpDir program programArgs env source = do
506507 UTF8.hPutStrLn stderr $
507508 T.pack program <> " " <> T.pack (unwords (map show programArgs))
508509 UTF8.hPutStr stderr "\n"
509− UTF8.hPutStrLn stderr "[makePDF] Environment:"
510− mapM_ (UTF8.hPutStrLn stderr . tshow) env
510+ UTF8.hPutStrLn stderr "[makePDF] Relevant environment variables:"
511+ -- we filter out irrelevant stuff to avoid leaking passwords and keys!
512+ let isRelevant ("PATH",_) = True
513+ isRelevant ("TMPDIR",_) = True
514+ isRelevant ("PWD",_) = True
515+ isRelevant ("LANG",_) = True
516+ isRelevant ("HOME",_) = True
517+ isRelevant ("LUA_PATH",_) = True
518+ isRelevant ("LUA_CPATH",_) = True
519+ isRelevant ("SHELL",_) = True
520+ isRelevant ("TEXINPUTS",_) = True
521+ isRelevant ("TEXMFOUTPUT",_) = True
522+ isRelevant _ = False
523+ mapM_ (UTF8.hPutStrLn stderr . tshow) (filter isRelevant env)
511524 UTF8.hPutStr stderr "\n"
512525 UTF8.hPutStrLn stderr "[makePDF] Source:"
513526 UTF8.hPutStrLn stderr source
src/Text/Pandoc/Parsing.hs+360−242
@@ -5,7 +5,6 @@
55{-# LANGUAGE GeneralizedNewtypeDeriving #-}
66{-# LANGUAGE IncoherentInstances #-}
77{-# LANGUAGE MultiParamTypeClasses #-}
8−{-# LANGUAGE ViewPatterns #-}
98{-# LANGUAGE OverloadedStrings #-}
109{- |
1110 Module : Text.Pandoc.Parsing
@@ -19,8 +18,7 @@
1918A utility library with parsers used in pandoc readers.
2019-}
2120
22−module Text.Pandoc.Parsing ( take1WhileP,
23− takeP,
21+module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
2422 countChar,
2523 textStr,
2624 anyLine,
@@ -105,14 +103,14 @@ module Text.Pandoc.Parsing ( take1WhileP,
105103 singleQuoteEnd,
106104 doubleQuoteStart,
107105 doubleQuoteEnd,
108− ellipses,
109106 apostrophe,
107+ doubleCloseQuote,
108+ ellipses,
110109 dash,
111110 nested,
112111 citeKey,
113112 Parser,
114113 ParserT,
115− F,
116114 Future(..),
117115 runF,
118116 askF,
@@ -123,7 +121,6 @@ module Text.Pandoc.Parsing ( take1WhileP,
123121 (<+?>),
124122 extractIdClass,
125123 insertIncludedFile,
126− insertIncludedFileF,
127124 -- * Re-exports from Text.Parsec
128125 Stream,
129126 runParser,
@@ -134,22 +131,10 @@ module Text.Pandoc.Parsing ( take1WhileP,
134131 getInput,
135132 setInput,
136133 unexpected,
137− char,
138− letter,
139− digit,
140− alphaNum,
141134 skipMany,
142135 skipMany1,
143− spaces,
144− space,
145− anyChar,
146− satisfy,
147− newline,
148− string,
149136 count,
150137 eof,
151− noneOf,
152− oneOf,
153138 lookAhead,
154139 notFollowedBy,
155140 many,
@@ -174,6 +159,8 @@ module Text.Pandoc.Parsing ( take1WhileP,
174159 SourcePos,
175160 getPosition,
176161 setPosition,
162+ sourceName,
163+ setSourceName,
177164 sourceColumn,
178165 sourceLine,
179166 setSourceColumn,
@@ -189,48 +176,141 @@ module Text.Pandoc.Parsing ( take1WhileP,
189176where
190177
191178import Control.Monad.Identity
179+ ( guard,
180+ join,
181+ unless,
182+ when,
183+ void,
184+ liftM2,
185+ liftM,
186+ Identity(..),
187+ MonadPlus(mzero) )
192188import Control.Monad.Reader
189+ ( asks, runReader, MonadReader(ask), Reader, ReaderT(ReaderT) )
193190import Data.Char (chr, isAlphaNum, isAscii, isAsciiUpper, isAsciiLower,
194− isPunctuation, isSpace, ord, toLower, toUpper)
195−import Data.Default
191+ isSpace, ord, toLower, toUpper)
192+import Data.Default ( Default(..) )
196193import Data.Functor (($>))
197194import Data.List (intercalate, transpose)
198195import qualified Data.Map as M
199−import Data.Maybe (mapMaybe, fromMaybe)
196+import Data.Maybe (fromMaybe)
200197import qualified Data.Set as Set
201−import Data.String
202198import Data.Text (Text)
203199import qualified Data.Text as T
204200import Text.HTML.TagSoup.Entity (lookupEntity)
205−import Text.Pandoc.Asciify (toAsciiChar)
201+import Text.Pandoc.Asciify (toAsciiText)
206202import Text.Pandoc.Builder (Blocks, HasMeta (..), Inlines, trimInlines)
207203import qualified Text.Pandoc.Builder as B
208204import Text.Pandoc.Class.PandocMonad (PandocMonad, readFileFromDirs, report)
209205import Text.Pandoc.Definition
206+ ( Target,
207+ nullMeta,
208+ nullAttr,
209+ Meta,
210+ ColWidth(ColWidthDefault, ColWidth),
211+ TableFoot(TableFoot),
212+ TableBody(TableBody),
213+ Attr,
214+ TableHead(TableHead),
215+ Row(..),
216+ Alignment(..),
217+ Inline(Str),
218+ ListNumberDelim(..),
219+ ListAttributes,
220+ ListNumberStyle(..) )
210221import Text.Pandoc.Logging
222+ ( LogMessage(CouldNotLoadIncludeFile, DuplicateIdentifier) )
211223import Text.Pandoc.Options
224+ ( extensionEnabled,
225+ Extension(Ext_old_dashes, Ext_tex_math_dollars,
226+ Ext_tex_math_single_backslash, Ext_tex_math_double_backslash,
227+ Ext_auto_identifiers, Ext_ascii_identifiers, Ext_smart),
228+ ReaderOptions(readerTabStop, readerColumns, readerExtensions) )
212229import Text.Pandoc.Readers.LaTeX.Types (Macro)
213230import Text.Pandoc.Shared
231+ ( uniqueIdent,
232+ tshow,
233+ mapLeft,
234+ compactify,
235+ trim,
236+ trimr,
237+ splitTextByIndices,
238+ safeRead,
239+ trimMath,
240+ schemes,
241+ escapeURI )
242+import Text.Pandoc.Sources
214243import qualified Text.Pandoc.UTF8 as UTF8 (putStrLn)
215244import Text.Pandoc.XML (fromEntities)
216−import Text.Parsec hiding (token)
217−import Text.Parsec.Pos (initialPos, newPos, updatePosString)
218−
219−import Control.Monad.Except
245+import Text.Parsec
246+ ( between,
247+ setSourceName,
248+ Parsec,
249+ Column,
250+ Line,
251+ incSourceLine,
252+ incSourceColumn,
253+ setSourceLine,
254+ setSourceColumn,
255+ sourceLine,
256+ sourceColumn,
257+ sourceName,
258+ setSourceName,
259+ setPosition,
260+ getPosition,
261+ updateState,
262+ setState,
263+ getState,
264+ optionMaybe,
265+ optional,
266+ option,
267+ endBy1,
268+ endBy,
269+ sepEndBy1,
270+ sepEndBy,
271+ sepBy1,
272+ sepBy,
273+ try,
274+ choice,
275+ (<?>),
276+ (<|>),
277+ manyTill,
278+ many1,
279+ many,
280+ notFollowedBy,
281+ lookAhead,
282+ eof,
283+ count,
284+ skipMany1,
285+ skipMany,
286+ unexpected,
287+ setInput,
288+ getInput,
289+ anyToken,
290+ tokenPrim,
291+ parse,
292+ runParserT,
293+ runParser,
294+ ParseError,
295+ ParsecT,
296+ SourcePos,
297+ Stream(..) )
298+import Text.Parsec.Pos (initialPos, newPos)
299+import Control.Monad.Except ( MonadError(throwError) )
220300import Text.Pandoc.Error
301+ ( PandocError(PandocParseError, PandocParsecError) )
221302
222303type Parser t s = Parsec t s
223304
224305type ParserT = ParsecT
225306
307+
226308-- | Reader monad wrapping the parser state. This is used to possibly delay
227309-- evaluation until all relevant information has been parsed and made available
228310-- in the parser state.
229311newtype Future s a = Future { runDelayed :: Reader s a }
230312 deriving (Monad, Applicative, Functor)
231313
232−type F = Future ParserState
233−
234314runF :: Future s a -> s -> a
235315runF = runReader . runDelayed
236316
@@ -253,70 +333,48 @@ instance (Semigroup a, Monoid a) => Monoid (Future s a) where
253333 mappend = (<>)
254334
255335-- | Like @count@, but packs its result
256−countChar :: (Stream s m Char, Monad m)
336+countChar :: (Stream s m Char, UpdateSourcePos s Char, Monad m)
257337 => Int
258338 -> ParsecT s st m Char
259339 -> ParsecT s st m Text
260340countChar n = fmap T.pack . count n
261341
262342-- | Like @string@, but uses @Text@.
263−textStr :: Stream s m Char => Text -> ParsecT s u m Text
343+textStr :: (Stream s m Char, UpdateSourcePos s Char)
344+ => Text -> ParsecT s u m Text
264345textStr t = string (T.unpack t) $> t
265346
266−-- | Parse characters while a predicate is true.
267−take1WhileP :: Monad m
268− => (Char -> Bool)
269− -> ParserT Text st m Text
270−take1WhileP f = do
271− -- needed to persuade parsec that this won't match an empty string:
272− c <- satisfy f
273− inp <- getInput
274− pos <- getPosition
275− let (t, rest) = T.span f inp
276− setInput rest
277− setPosition $
278− if f '\t' || f '\n'
279− then updatePosString pos $ T.unpack t
280− else incSourceColumn pos (T.length t)
281− return $ T.singleton c <> t
282−
283−-- Parse n characters of input (or the rest of the input if
284−-- there aren't n characters).
285−takeP :: Monad m => Int -> ParserT Text st m Text
286−takeP n = do
287− guard (n > 0)
288− -- faster than 'count n anyChar'
289− inp <- getInput
290− pos <- getPosition
291− let (xs, rest) = T.splitAt n inp
292− -- needed to persuade parsec that this won't match an empty string:
293− anyChar
294− setInput rest
295− setPosition $ updatePosString pos $ T.unpack xs
296− return xs
297−
298−-- | Parse any line of text
299−anyLine :: Monad m => ParserT Text st m Text
347+
348+-- | Parse any line of text, returning the contents without the
349+-- final newline.
350+anyLine :: Monad m => ParserT Sources st m Text
300351anyLine = do
301352 -- This is much faster than:
302353 -- manyTill anyChar newline
303354 inp <- getInput
304− pos <- getPosition
305− case T.break (=='\n') inp of
306− (this, T.uncons -> Just ('\n', rest)) -> do
307− -- needed to persuade parsec that this won't match an empty string:
308− anyChar
309− setInput rest
310− setPosition $ incSourceLine (setSourceColumn pos 1) 1
311− return this
312− _ -> mzero
355+ case inp of
356+ Sources [] -> mzero
357+ Sources ((fp,t):inps) ->
358+ -- we assume that lines don't span different input files
359+ case T.break (=='\n') t of
360+ (this, rest)
361+ | T.null rest
362+ , not (null inps) ->
363+ -- line may span different input files, so do it
364+ -- character by character
365+ T.pack <$> manyTill anyChar newline
366+ | otherwise -> do -- either end of inputs or newline in rest
367+ setInput $ Sources ((fp, rest):inps)
368+ char '\n' -- needed so parsec knows we won't match empty string
369+ -- and so source pos is updated
370+ return this
313371
314372-- | Parse any line, include the final newline in the output
315−anyLineNewline :: Monad m => ParserT Text st m Text
373+anyLineNewline :: Monad m => ParserT Sources st m Text
316374anyLineNewline = (<> "\n") <$> anyLine
317375
318376-- | Parse indent by specified number of spaces (or equiv. tabs)
319−indentWith :: Stream s m Char
377+indentWith :: (Stream s m Char, UpdateSourcePos s Char)
320378 => HasReaderOptions st
321379 => Int -> ParserT s st m Text
322380indentWith num = do
@@ -401,11 +459,13 @@ notFollowedBy' p = try $ join $ do a <- try p
401459 return (return ())
402460-- (This version due to Andrew Pimlott on the Haskell mailing list.)
403461
404−oneOfStrings' :: Stream s m Char => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
462+oneOfStrings' :: (Stream s m Char, UpdateSourcePos s Char)
463+ => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
405464oneOfStrings' f = fmap T.pack . oneOfStrings'' f . fmap T.unpack
406465
407466-- TODO: This should be re-implemented in a Text-aware way
408−oneOfStrings'' :: Stream s m Char => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
467+oneOfStrings'' :: (Stream s m Char, UpdateSourcePos s Char)
468+ => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
409469oneOfStrings'' _ [] = Prelude.fail "no strings"
410470oneOfStrings'' matches strs = try $ do
411471 c <- anyChar
@@ -420,14 +480,16 @@ oneOfStrings'' matches strs = try $ do
420480-- | Parses one of a list of strings. If the list contains
421481-- two strings one of which is a prefix of the other, the longer
422482-- string will be matched if possible.
423−oneOfStrings :: Stream s m Char => [Text] -> ParserT s st m Text
483+oneOfStrings :: (Stream s m Char, UpdateSourcePos s Char)
484+ => [Text] -> ParserT s st m Text
424485oneOfStrings = oneOfStrings' (==)
425486
426487-- | Parses one of a list of strings (tried in order), case insensitive.
427488
428489-- TODO: This will not be accurate with general Unicode (neither
429490-- Text.toLower nor Text.toCaseFold can be implemented with a map)
430−oneOfStringsCI :: Stream s m Char => [Text] -> ParserT s st m Text
491+oneOfStringsCI :: (Stream s m Char, UpdateSourcePos s Char)
492+ => [Text] -> ParserT s st m Text
431493oneOfStringsCI = oneOfStrings' ciMatch
432494 where ciMatch x y = toLower' x == toLower' y
433495 -- this optimizes toLower by checking common ASCII case
@@ -438,11 +500,13 @@ oneOfStringsCI = oneOfStrings' ciMatch
438500 | otherwise = toLower c
439501
440502-- | Parses a space or tab.
441−spaceChar :: Stream s m Char => ParserT s st m Char
503+spaceChar :: (Stream s m Char, UpdateSourcePos s Char)
504+ => ParserT s st m Char
442505spaceChar = satisfy $ \c -> c == ' ' || c == '\t'
443506
444507-- | Parses a nonspace, nonnewline character.
445−nonspaceChar :: Stream s m Char => ParserT s st m Char
508+nonspaceChar :: (Stream s m Char, UpdateSourcePos s Char)
509+ => ParserT s st m Char
446510nonspaceChar = satisfy (not . isSpaceChar)
447511
448512isSpaceChar :: Char -> Bool
@@ -453,21 +517,24 @@ isSpaceChar '\r' = True
453517isSpaceChar _ = False
454518
455519-- | Skips zero or more spaces or tabs.
456−skipSpaces :: Stream s m Char => ParserT s st m ()
520+skipSpaces :: (Stream s m Char, UpdateSourcePos s Char)
521+ => ParserT s st m ()
457522skipSpaces = skipMany spaceChar
458523
459524-- | Skips zero or more spaces or tabs, then reads a newline.
460−blankline :: Stream s m Char => ParserT s st m Char
525+blankline :: (Stream s m Char, UpdateSourcePos s Char)
526+ => ParserT s st m Char
461527blankline = try $ skipSpaces >> newline
462528
463529-- | Parses one or more blank lines and returns a string of newlines.
464−blanklines :: Stream s m Char => ParserT s st m Text
530+blanklines :: (Stream s m Char, UpdateSourcePos s Char)
531+ => ParserT s st m Text
465532blanklines = T.pack <$> many1 blankline
466533
467534-- | Gobble n spaces; if tabs are encountered, expand them
468535-- and gobble some or all of their spaces, leaving the rest.
469536gobbleSpaces :: (HasReaderOptions st, Monad m)
470− => Int -> ParserT Text st m ()
537+ => Int -> ParserT Sources st m ()
471538gobbleSpaces 0 = return ()
472539gobbleSpaces n
473540 | n < 0 = error "gobbleSpaces called with negative number"
@@ -475,18 +542,26 @@ gobbleSpaces n
475542 char ' ' <|> eatOneSpaceOfTab
476543 gobbleSpaces (n - 1)
477544
478−eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Text st m Char
545+eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Sources st m Char
479546eatOneSpaceOfTab = do
480− char '\t'
547+ lookAhead (char '\t')
548+ pos <- getPosition
481549 tabstop <- getOption readerTabStop
550+ -- replace the tab on the input stream with spaces
551+ let numSpaces = tabstop - ((sourceColumn pos - 1) `mod` tabstop)
482552 inp <- getInput
483− setInput $ T.replicate (tabstop - 1) " " <> inp
484− return ' '
553+ setInput $
554+ case inp of
555+ Sources [] -> error "eatOneSpaceOfTab - empty Sources list"
556+ Sources ((fp,t):rest) ->
557+ -- drop the tab and add spaces
558+ Sources ((fp, T.replicate numSpaces " " <> T.drop 1 t):rest)
559+ char ' '
485560
486561-- | Gobble up to n spaces; if tabs are encountered, expand them
487562-- and gobble some or all of their spaces, leaving the rest.
488563gobbleAtMostSpaces :: (HasReaderOptions st, Monad m)
489− => Int -> ParserT Text st m Int
564+ => Int -> ParserT Sources st m Int
490565gobbleAtMostSpaces 0 = return 0
491566gobbleAtMostSpaces n
492567 | n < 0 = error "gobbleAtMostSpaces called with negative number"
@@ -495,7 +570,8 @@ gobbleAtMostSpaces n
495570 (+ 1) <$> gobbleAtMostSpaces (n - 1)
496571
497572-- | Parses material enclosed between start and end parsers.
498−enclosed :: (Show end, Stream s m Char) => ParserT s st m t -- ^ start parser
573+enclosed :: (Show end, Stream s m Char, UpdateSourcePos s Char)
574+ => ParserT s st m t -- ^ start parser
499575 -> ParserT s st m end -- ^ end parser
500576 -> ParserT s st m a -- ^ content parser (to be used repeatedly)
501577 -> ParserT s st m [a]
@@ -503,39 +579,41 @@ enclosed start end parser = try $
503579 start >> notFollowedBy space >> many1Till parser end
504580
505581-- | Parse string, case insensitive.
506−stringAnyCase :: Stream s m Char => Text -> ParserT s st m Text
582+stringAnyCase :: (Stream s m Char, UpdateSourcePos s Char)
583+ => Text -> ParserT s st m Text
507584stringAnyCase = fmap T.pack . stringAnyCase' . T.unpack
508585
509−stringAnyCase' :: Stream s m Char => String -> ParserT s st m String
586+stringAnyCase' :: (Stream s m Char, UpdateSourcePos s Char)
587+ => String -> ParserT s st m String
510588stringAnyCase' [] = string ""
511589stringAnyCase' (x:xs) = do
512590 firstChar <- char (toUpper x) <|> char (toLower x)
513591 rest <- stringAnyCase' xs
514592 return (firstChar:rest)
515593
594+-- TODO rewrite by just adding to Sources stream?
516595-- | Parse contents of 'str' using 'parser' and return result.
517−parseFromString :: (Stream s m Char, IsString s)
518− => ParserT s st m r
596+parseFromString :: Monad m
597+ => ParserT Sources st m r
519598 -> Text
520− -> ParserT s st m r
599+ -> ParserT Sources st m r
521600parseFromString parser str = do
522601 oldPos <- getPosition
523602 setPosition $ initialPos "chunk"
524603 oldInput <- getInput
525− setInput $ fromString $ T.unpack str
604+ setInput $ toSources str
526605 result <- parser
527606 spaces
528− eof
529607 setInput oldInput
530608 setPosition oldPos
531609 return result
532610
533611-- | Like 'parseFromString' but specialized for 'ParserState'.
534612-- This resets 'stateLastStrPos', which is almost always what we want.
535−parseFromString' :: (Stream s m Char, IsString s, HasLastStrPosition u)
536− => ParserT s u m a
613+parseFromString' :: (Monad m, HasLastStrPosition u)
614+ => ParserT Sources u m a
537615 -> Text
538− -> ParserT s u m a
616+ -> ParserT Sources u m a
539617parseFromString' parser str = do
540618 oldLastStrPos <- getLastStrPos <$> getState
541619 updateState $ setLastStrPos Nothing
@@ -544,7 +622,7 @@ parseFromString' parser str = do
544622 return res
545623
546624-- | Parse raw line block up to and including blank lines.
547−lineClump :: Monad m => ParserT Text st m Text
625+lineClump :: Monad m => ParserT Sources st m Text
548626lineClump = blanklines
549627 <|> (T.unlines <$> many1 (notFollowedBy blankline >> anyLine))
550628
@@ -553,7 +631,7 @@ lineClump = blanklines
553631-- pairs of open and close, which must be different. For example,
554632-- @charsInBalanced '(' ')' anyChar@ will parse "(hello (there))"
555633-- and return "hello (there)".
556−charsInBalanced :: Stream s m Char => Char -> Char -> ParserT s st m Char
634+charsInBalanced :: (Stream s m Char, UpdateSourcePos s Char) => Char -> Char -> ParserT s st m Char
557635 -> ParserT s st m Text
558636charsInBalanced open close parser = try $ do
559637 char open
@@ -572,7 +650,7 @@ charsInBalanced open close parser = try $ do
572650-- Auxiliary functions for romanNumeral:
573651
574652-- | Parses a roman numeral (uppercase or lowercase), returns number.
575−romanNumeral :: Stream s m Char => Bool -- ^ Uppercase if true
653+romanNumeral :: (Stream s m Char, UpdateSourcePos s Char) => Bool -- ^ Uppercase if true
576654 -> ParserT s st m Int
577655romanNumeral upperCase = do
578656 let rchar uc = char $ if upperCase then uc else toLower uc
@@ -608,20 +686,19 @@ romanNumeral upperCase = do
608686
609687-- | Parses an email address; returns original and corresponding
610688-- escaped mailto: URI.
611−emailAddress :: Stream s m Char => ParserT s st m (Text, Text)
689+emailAddress :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
612690emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
613691 where toResult mbox dom = let full = fromEntities $ T.pack $ mbox ++ '@':dom
614692 in (full, escapeURI $ "mailto:" <> full)
615693 mailbox = intercalate "." <$> (emailWord `sepBy1'` dot)
616694 domain = intercalate "." <$> (subdomain `sepBy1'` dot)
617695 dot = char '.'
618− subdomain = many1 $ alphaNum <|> innerPunct
696+ subdomain = many1 $ alphaNum <|> innerPunct (=='-')
619697 -- this excludes some valid email addresses, since an
620698 -- email could contain e.g. '__', but gives better results
621699 -- for our purposes, when combined with markdown parsing:
622− innerPunct = try (satisfy (\c -> isEmailPunct c || c == '@')
623− <* notFollowedBy space
624− <* notFollowedBy (satisfy isPunctuation))
700+ innerPunct f = try (satisfy f
701+ <* notFollowedBy (satisfy (not . isAlphaNum)))
625702 -- technically an email address could begin with a symbol,
626703 -- but allowing this creates too many problems.
627704 -- See e.g. https://github.com/jgm/pandoc/issues/2940
@@ -632,11 +709,11 @@ emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
632709 isEmailPunct c = T.any (== c) "!\"#$%&'*+-/=?^_{|}~;"
633710
634711
635−uriScheme :: Stream s m Char => ParserT s st m Text
712+uriScheme :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Text
636713uriScheme = oneOfStringsCI (Set.toList schemes)
637714
638715-- | Parses a URI. Returns pair of original and URI-escaped version.
639−uri :: Stream s m Char => ParserT s st m (Text, Text)
716+uri :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
640717uri = try $ do
641718 scheme <- uriScheme
642719 char ':'
@@ -679,7 +756,7 @@ uri = try $ do
679756 uriChunkBetween l r = try $ do chunk <- between (char l) (char r) uriChunk
680757 return (T.pack $ [l] ++ chunk ++ [r])
681758
682−mathInlineWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
759+mathInlineWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
683760mathInlineWith op cl = try $ do
684761 textStr op
685762 when (op == "$") $ notFollowedBy space
@@ -700,10 +777,10 @@ mathInlineWith op cl = try $ do
700777 notFollowedBy digit -- to prevent capture of $5
701778 return $ trimMath $ T.concat words'
702779 where
703− inBalancedBraces :: Stream s m Char => Int -> Text -> ParserT s st m Text
780+ inBalancedBraces :: (Stream s m Char, UpdateSourcePos s Char) => Int -> Text -> ParserT s st m Text
704781 inBalancedBraces n = fmap T.pack . inBalancedBraces' n . T.unpack
705782
706− inBalancedBraces' :: Stream s m Char => Int -> String -> ParserT s st m String
783+ inBalancedBraces' :: (Stream s m Char, UpdateSourcePos s Char) => Int -> String -> ParserT s st m String
707784 inBalancedBraces' 0 "" = do
708785 c <- anyChar
709786 if c == '{'
@@ -720,13 +797,13 @@ mathInlineWith op cl = try $ do
720797 '{' -> inBalancedBraces' (numOpen + 1) (c:xs)
721798 _ -> inBalancedBraces' numOpen (c:xs)
722799
723−mathDisplayWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
800+mathDisplayWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
724801mathDisplayWith op cl = try $ fmap T.pack $ do
725802 textStr op
726803 many1Till (satisfy (/= '\n') <|> (newline <* notFollowedBy' blankline))
727804 (try $ textStr cl)
728805
729−mathDisplay :: (HasReaderOptions st, Stream s m Char)
806+mathDisplay :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
730807 => ParserT s st m Text
731808mathDisplay =
732809 (guardEnabled Ext_tex_math_dollars >> mathDisplayWith "$$" "$$")
@@ -735,7 +812,7 @@ mathDisplay =
735812 <|> (guardEnabled Ext_tex_math_double_backslash >>
736813 mathDisplayWith "\\\\[" "\\\\]")
737814
738−mathInline :: (HasReaderOptions st , Stream s m Char)
815+mathInline :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
739816 => ParserT s st m Text
740817mathInline =
741818 (guardEnabled Ext_tex_math_dollars >> mathInlineWith "$" "$")
@@ -748,7 +825,7 @@ mathInline =
748825-- displacement (the difference between the source column at the end
749826-- and the source column at the beginning). Vertical displacement
750827-- (source row) is ignored.
751−withHorizDisplacement :: Stream s m Char
828+withHorizDisplacement :: (Stream s m Char, UpdateSourcePos s Char)
752829 => ParserT s st m a -- ^ Parser to apply
753830 -> ParserT s st m (a, Int) -- ^ (result, displacement)
754831withHorizDisplacement parser = do
@@ -760,30 +837,37 @@ withHorizDisplacement parser = do
760837-- | Applies a parser and returns the raw string that was parsed,
761838-- along with the value produced by the parser.
762839withRaw :: Monad m
763− => ParsecT Text st m a
764− -> ParsecT Text st m (a, Text)
840+ => ParsecT Sources st m a
841+ -> ParsecT Sources st m (a, Text)
765842withRaw parser = do
766− pos1 <- getPosition
767− inp <- getInput
843+ inps1 <- getInput
768844 result <- parser
769− pos2 <- getPosition
770− let (l1,c1) = (sourceLine pos1, sourceColumn pos1)
771− let (l2,c2) = (sourceLine pos2, sourceColumn pos2)
772− let inplines = take ((l2 - l1) + 1) $ T.lines inp
773− let raw = case inplines of
774− [] -> ""
775− [l] -> T.take (c2 - c1) l
776− ls -> T.unlines (init ls) <> T.take (c2 - 1) (last ls)
777− return (result, raw)
845+ inps2 <- getInput
846+ -- 'raw' is the difference between inps1 and inps2
847+ return (result, sourcesDifference inps1 inps2)
848+
849+sourcesDifference :: Sources -> Sources -> Text
850+sourcesDifference (Sources is1) (Sources is2) = go is1 is2
851+ where
852+ go inps1 inps2 =
853+ case (inps1, inps2) of
854+ ([], _) -> mempty
855+ (_, []) -> mconcat $ map snd inps1
856+ ((p1,t1):rest1, (p2, t2):rest2)
857+ | p1 == p2
858+ , t1 == t2 -> go rest1 rest2
859+ | p1 == p2
860+ , t1 /= t2 -> fromMaybe mempty $ T.stripSuffix t2 t1
861+ | otherwise -> t1 <> go rest1 inps2
778862
779863-- | Parses backslash, then applies character parser.
780−escaped :: Stream s m Char
864+escaped :: (Stream s m Char, UpdateSourcePos s Char)
781865 => ParserT s st m Char -- ^ Parser for character to escape
782866 -> ParserT s st m Char
783867escaped parser = try $ char '\\' >> parser
784868
785869-- | Parse character entity.
786−characterReference :: Stream s m Char => ParserT s st m Char
870+characterReference :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
787871characterReference = try $ do
788872 char '&'
789873 ent <- many1Till nonspaceChar (char ';')
@@ -796,19 +880,19 @@ characterReference = try $ do
796880 _ -> Prelude.fail "entity not found"
797881
798882-- | Parses an uppercase roman numeral and returns (UpperRoman, number).
799−upperRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
883+upperRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
800884upperRoman = do
801885 num <- romanNumeral True
802886 return (UpperRoman, num)
803887
804888-- | Parses a lowercase roman numeral and returns (LowerRoman, number).
805−lowerRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
889+lowerRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
806890lowerRoman = do
807891 num <- romanNumeral False
808892 return (LowerRoman, num)
809893
810894-- | Parses a decimal numeral and returns (Decimal, number).
811−decimal :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
895+decimal :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
812896decimal = do
813897 num <- many1 digit
814898 return (Decimal, fromMaybe 1 $ safeRead $ T.pack num)
@@ -817,7 +901,7 @@ decimal = do
817901-- returns (DefaultStyle, [next example number]). The next
818902-- example number is incremented in parser state, and the label
819903-- (if present) is added to the label table.
820−exampleNum :: Stream s m Char
904+exampleNum :: (Stream s m Char, UpdateSourcePos s Char)
821905 => ParserT s ParserState m (ListNumberStyle, Int)
822906exampleNum = do
823907 char '@'
@@ -836,37 +920,37 @@ exampleNum = do
836920 return (Example, num)
837921
838922-- | Parses a '#' returns (DefaultStyle, 1).
839−defaultNum :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
923+defaultNum :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
840924defaultNum = do
841925 char '#'
842926 return (DefaultStyle, 1)
843927
844928-- | Parses a lowercase letter and returns (LowerAlpha, number).
845−lowerAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
929+lowerAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
846930lowerAlpha = do
847931 ch <- satisfy isAsciiLower
848932 return (LowerAlpha, ord ch - ord 'a' + 1)
849933
850934-- | Parses an uppercase letter and returns (UpperAlpha, number).
851−upperAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
935+upperAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
852936upperAlpha = do
853937 ch <- satisfy isAsciiUpper
854938 return (UpperAlpha, ord ch - ord 'A' + 1)
855939
856940-- | Parses a roman numeral i or I
857−romanOne :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
941+romanOne :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
858942romanOne = (char 'i' >> return (LowerRoman, 1)) <|>
859943 (char 'I' >> return (UpperRoman, 1))
860944
861945-- | Parses an ordered list marker and returns list attributes.
862−anyOrderedListMarker :: Stream s m Char => ParserT s ParserState m ListAttributes
946+anyOrderedListMarker :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s ParserState m ListAttributes
863947anyOrderedListMarker = choice
864948 [delimParser numParser | delimParser <- [inPeriod, inOneParen, inTwoParens],
865949 numParser <- [decimal, exampleNum, defaultNum, romanOne,
866950 lowerAlpha, lowerRoman, upperAlpha, upperRoman]]
867951
868952-- | Parses a list number (num) followed by a period, returns list attributes.
869−inPeriod :: Stream s m Char
953+inPeriod :: (Stream s m Char, UpdateSourcePos s Char)
870954 => ParserT s st m (ListNumberStyle, Int)
871955 -> ParserT s st m ListAttributes
872956inPeriod num = try $ do
@@ -878,7 +962,7 @@ inPeriod num = try $ do
878962 return (start, style, delim)
879963
880964-- | Parses a list number (num) followed by a paren, returns list attributes.
881−inOneParen :: Stream s m Char
965+inOneParen :: (Stream s m Char, UpdateSourcePos s Char)
882966 => ParserT s st m (ListNumberStyle, Int)
883967 -> ParserT s st m ListAttributes
884968inOneParen num = try $ do
@@ -887,7 +971,7 @@ inOneParen num = try $ do
887971 return (start, style, OneParen)
888972
889973-- | Parses a list number (num) enclosed in parens, returns list attributes.
890−inTwoParens :: Stream s m Char
974+inTwoParens :: (Stream s m Char, UpdateSourcePos s Char)
891975 => ParserT s st m (ListNumberStyle, Int)
892976 -> ParserT s st m ListAttributes
893977inTwoParens num = try $ do
@@ -898,7 +982,7 @@ inTwoParens num = try $ do
898982
899983-- | Parses an ordered list marker with a given style and delimiter,
900984-- returns number.
901−orderedListMarker :: Stream s m Char
985+orderedListMarker :: (Stream s m Char, UpdateSourcePos s Char)
902986 => ListNumberStyle
903987 -> ListNumberDelim
904988 -> ParserT s ParserState m Int
@@ -921,10 +1005,10 @@ orderedListMarker style delim = do
9211005 return start
9221006
9231007-- | Parses a character reference and returns a Str element.
924−charRef :: Stream s m Char => ParserT s st m Inline
1008+charRef :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inline
9251009charRef = Str . T.singleton <$> characterReference
9261010
927−lineBlockLine :: Monad m => ParserT Text st m Text
1011+lineBlockLine :: Monad m => ParserT Sources st m Text
9281012lineBlockLine = try $ do
9291013 char '|'
9301014 char ' '
@@ -934,11 +1018,11 @@ lineBlockLine = try $ do
9341018 continuations <- many (try $ char ' ' >> anyLine)
9351019 return $ white <> T.unwords (line : continuations)
9361020
937−blankLineBlockLine :: Stream s m Char => ParserT s st m Char
1021+blankLineBlockLine :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
9381022blankLineBlockLine = try (char '|' >> blankline)
9391023
9401024-- | Parses an RST-style line block and returns a list of strings.
941−lineBlockLines :: Monad m => ParserT Text st m [Text]
1025+lineBlockLines :: Monad m => ParserT Sources st m [Text]
9421026lineBlockLines = try $ do
9431027 lines' <- many1 (lineBlockLine <|> (T.singleton <$> blankLineBlockLine))
9441028 skipMany blankline
@@ -946,7 +1030,8 @@ lineBlockLines = try $ do
9461030
9471031-- | Parse a table using 'headerParser', 'rowParser',
9481032-- 'lineParser', and 'footerParser'.
949−tableWith :: (Stream s m Char, HasReaderOptions st, Monad mf)
1033+tableWith :: (Stream s m Char, UpdateSourcePos s Char,
1034+ HasReaderOptions st, Monad mf)
9501035 => ParserT s st m (mf [Blocks], [Alignment], [Int])
9511036 -> ([Int] -> ParserT s st m (mf [Blocks]))
9521037 -> ParserT s st m sep
@@ -966,7 +1051,8 @@ tableWith headerParser rowParser lineParser footerParser = try $ do
9661051
9671052type TableComponents mf = ([Alignment], [Double], mf [Row], mf [Row])
9681053
969−tableWith' :: (Stream s m Char, HasReaderOptions st, Monad mf)
1054+tableWith' :: (Stream s m Char, UpdateSourcePos s Char,
1055+ HasReaderOptions st, Monad mf)
9701056 => ParserT s st m (mf [Blocks], [Alignment], [Int])
9711057 -> ([Int] -> ParserT s st m (mf [Blocks]))
9721058 -> ParserT s st m sep
@@ -1015,20 +1101,19 @@ widthsFromIndices numColumns' indices =
10151101-- (which may be grid), then the rows,
10161102-- which may be grid, separated by blank lines, and
10171103-- ending with a footer (dashed line followed by blank line).
1018−gridTableWith :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
1019− Monad mf, IsString s)
1020− => ParserT s st m (mf Blocks) -- ^ Block list parser
1104+gridTableWith :: (Monad m, HasReaderOptions st, HasLastStrPosition st, Monad mf)
1105+ => ParserT Sources st m (mf Blocks) -- ^ Block list parser
10211106 -> Bool -- ^ Headerless table
1022− -> ParserT s st m (mf Blocks)
1107+ -> ParserT Sources st m (mf Blocks)
10231108gridTableWith blocks headless =
10241109 tableWith (gridTableHeader headless blocks) (gridTableRow blocks)
10251110 (gridTableSep '-') gridTableFooter
10261111
1027−gridTableWith' :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
1028− Monad mf, IsString s)
1029− => ParserT s st m (mf Blocks) -- ^ Block list parser
1112+gridTableWith' :: (Monad m, HasReaderOptions st, HasLastStrPosition st,
1113+ Monad mf)
1114+ => ParserT Sources st m (mf Blocks) -- ^ Block list parser
10301115 -> Bool -- ^ Headerless table
1031− -> ParserT s st m (TableComponents mf)
1116+ -> ParserT Sources st m (TableComponents mf)
10321117gridTableWith' blocks headless =
10331118 tableWith' (gridTableHeader headless blocks) (gridTableRow blocks)
10341119 (gridTableSep '-') gridTableFooter
@@ -1037,7 +1122,7 @@ gridTableSplitLine :: [Int] -> Text -> [Text]
10371122gridTableSplitLine indices line = map removeFinalBar $ tail $
10381123 splitTextByIndices (init indices) $ trimr line
10391124
1040−gridPart :: Stream s m Char => Char -> ParserT s st m ((Int, Int), Alignment)
1125+gridPart :: Monad m => Char -> ParserT Sources st m ((Int, Int), Alignment)
10411126gridPart ch = do
10421127 leftColon <- option False (True <$ char ':')
10431128 dashes <- many1 (char ch)
@@ -1052,7 +1137,7 @@ gridPart ch = do
10521137 (False, False) -> AlignDefault
10531138 return ((lengthDashes, lengthDashes + 1), alignment)
10541139
1055−gridDashedLines :: Stream s m Char => Char -> ParserT s st m [((Int, Int), Alignment)]
1140+gridDashedLines :: Monad m => Char -> ParserT Sources st m [((Int, Int), Alignment)]
10561141gridDashedLines ch = try $ char '+' >> many1 (gridPart ch) <* blankline
10571142
10581143removeFinalBar :: Text -> Text
@@ -1061,14 +1146,14 @@ removeFinalBar = T.dropWhileEnd go . T.dropWhileEnd (=='|')
10611146 go c = T.any (== c) " \t"
10621147
10631148-- | Separator between rows of grid table.
1064−gridTableSep :: Stream s m Char => Char -> ParserT s st m Char
1149+gridTableSep :: Monad m => Char -> ParserT Sources st m Char
10651150gridTableSep ch = try $ gridDashedLines ch >> return '\n'
10661151
10671152-- | Parse header for a grid table.
1068−gridTableHeader :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
1153+gridTableHeader :: (Monad m, Monad mf, HasLastStrPosition st)
10691154 => Bool -- ^ Headerless table
1070− -> ParserT s st m (mf Blocks)
1071− -> ParserT s st m (mf [Blocks], [Alignment], [Int])
1155+ -> ParserT Sources st m (mf Blocks)
1156+ -> ParserT Sources st m (mf [Blocks], [Alignment], [Int])
10721157gridTableHeader True _ = do
10731158 optional blanklines
10741159 dashes <- gridDashedLines '-'
@@ -1091,17 +1176,17 @@ gridTableHeader False blocks = try $ do
10911176 heads <- sequence <$> mapM (parseFromString' blocks . trim) rawHeads
10921177 return (heads, aligns, indices)
10931178
1094−gridTableRawLine :: Stream s m Char => [Int] -> ParserT s st m [Text]
1179+gridTableRawLine :: (Stream s m Char, UpdateSourcePos s Char) => [Int] -> ParserT s st m [Text]
10951180gridTableRawLine indices = do
10961181 char '|'
10971182 line <- many1Till anyChar newline
10981183 return (gridTableSplitLine indices $ T.pack line)
10991184
11001185-- | Parse row of grid table.
1101−gridTableRow :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
1102− => ParserT s st m (mf Blocks)
1186+gridTableRow :: (Monad m, Monad mf, HasLastStrPosition st)
1187+ => ParserT Sources st m (mf Blocks)
11031188 -> [Int]
1104− -> ParserT s st m (mf [Blocks])
1189+ -> ParserT Sources st m (mf [Blocks])
11051190gridTableRow blocks indices = do
11061191 colLines <- many1 (gridTableRawLine indices)
11071192 let cols = map ((<> "\n") . T.unlines . removeOneLeadingSpace) $
@@ -1122,34 +1207,38 @@ removeOneLeadingSpace xs =
11221207 Just (c, _) -> c == ' '
11231208
11241209-- | Parse footer for a grid table.
1125−gridTableFooter :: Stream s m Char => ParserT s st m ()
1210+gridTableFooter :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
11261211gridTableFooter = optional blanklines
11271212
11281213---
11291214
11301215-- | Removes the ParsecT layer from the monad transformer stack
1131−readWithM :: (Stream s m Char, ToText s)
1132− => ParserT s st m a -- ^ parser
1133− -> st -- ^ initial state
1134− -> s -- ^ input
1216+readWithM :: (Monad m, ToSources t)
1217+ => ParserT Sources st m a -- ^ parser
1218+ -> st -- ^ initial state
1219+ -> t -- ^ input
11351220 -> m (Either PandocError a)
11361221readWithM parser state input =
1137− mapLeft (PandocParsecError $ toText input) `liftM` runParserT parser state "source" input
1222+ mapLeft (PandocParsecError sources)
1223+ <$> runParserT parser state (initialSourceName sources) sources
1224+ where
1225+ sources = toSources input
11381226
11391227-- | Parse a string with a given parser and state
1140−readWith :: Parser Text st a
1228+readWith :: ToSources t
1229+ => Parser Sources st a
11411230 -> st
1142− -> Text
1231+ -> t
11431232 -> Either PandocError a
11441233readWith p t inp = runIdentity $ readWithM p t inp
11451234
11461235-- | Parse a string with @parser@ (for testing).
11471236testStringWith :: Show a
1148− => ParserT Text ParserState Identity a
1237+ => ParserT Sources ParserState Identity a
11491238 -> Text
11501239 -> IO ()
11511240testStringWith parser str = UTF8.putStrLn $ tshow $
1152− readWith parser defaultParserState str
1241+ readWith parser defaultParserState (toSources str)
11531242
11541243-- | Parsing options.
11551244data ParserState = ParserState
@@ -1169,7 +1258,7 @@ data ParserState = ParserState
11691258 stateInNote :: Bool, -- ^ True if parsing note contents
11701259 stateNoteNumber :: Int, -- ^ Last note number for citations
11711260 stateMeta :: Meta, -- ^ Document metadata
1172− stateMeta' :: F Meta, -- ^ Document metadata
1261+ stateMeta' :: Future ParserState Meta, -- ^ Document metadata
11731262 stateCitations :: M.Map Text Text, -- ^ RST-style citations
11741263 stateHeaderTable :: [HeaderType], -- ^ Ordered list of header types used
11751264 stateIdentifiers :: Set.Set Text, -- ^ Header identifiers used
@@ -1348,7 +1437,7 @@ data QuoteContext
13481437
13491438type NoteTable = [(Text, Text)]
13501439
1351−type NoteTable' = M.Map Text (SourcePos, F Blocks)
1440+type NoteTable' = M.Map Text (SourcePos, Future ParserState Blocks)
13521441-- used in markdown reader
13531442
13541443newtype Key = Key Text deriving (Show, Read, Eq, Ord)
@@ -1383,7 +1472,7 @@ registerHeader (ident,classes,kvs) header' = do
13831472 then do
13841473 let id' = uniqueIdent exts (B.toList header') ids
13851474 let id'' = if Ext_ascii_identifiers `extensionEnabled` exts
1386− then T.pack $ mapMaybe toAsciiChar $ T.unpack id'
1475+ then toAsciiText id'
13871476 else id'
13881477 updateState $ updateIdentifierList $ Set.insert id'
13891478 updateState $ updateIdentifierList $ Set.insert id''
@@ -1396,34 +1485,42 @@ registerHeader (ident,classes,kvs) header' = do
13961485 updateState $ updateIdentifierList $ Set.insert ident
13971486 return (ident,classes,kvs)
13981487
1399−smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
1488+smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st,
1489+ HasQuoteContext st m,
1490+ Stream s m Char, UpdateSourcePos s Char)
14001491 => ParserT s st m Inlines
14011492 -> ParserT s st m Inlines
14021493smartPunctuation inlineParser = do
14031494 guardEnabled Ext_smart
1404− choice [ quoted inlineParser, apostrophe, dash, ellipses ]
1495+ choice [ quoted inlineParser, apostrophe, doubleCloseQuote, dash, ellipses ]
14051496
1406−apostrophe :: Stream s m Char => ParserT s st m Inlines
1407−apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\x2019")
1408−
1409−quoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
1497+quoted :: (HasLastStrPosition st, HasQuoteContext st m,
1498+ Stream s m Char, UpdateSourcePos s Char)
14101499 => ParserT s st m Inlines
14111500 -> ParserT s st m Inlines
14121501quoted inlineParser = doubleQuoted inlineParser <|> singleQuoted inlineParser
14131502
1414−singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
1503+singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m,
1504+ Stream s m Char, UpdateSourcePos s Char)
14151505 => ParserT s st m Inlines
14161506 -> ParserT s st m Inlines
1417−singleQuoted inlineParser = try $ B.singleQuoted . mconcat
1418− <$ singleQuoteStart
1419− <*> withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)
1420−
1421−doubleQuoted :: (HasQuoteContext st m, Stream s m Char)
1507+singleQuoted inlineParser = do
1508+ singleQuoteStart
1509+ (B.singleQuoted . mconcat <$>
1510+ try
1511+ (withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)))
1512+ <|> pure "\8217"
1513+
1514+doubleQuoted :: (HasQuoteContext st m, HasLastStrPosition st,
1515+ Stream s m Char, UpdateSourcePos s Char)
14221516 => ParserT s st m Inlines
14231517 -> ParserT s st m Inlines
1424−doubleQuoted inlineParser = try $ B.doubleQuoted . mconcat
1425− <$ doubleQuoteStart
1426− <*> withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)
1518+doubleQuoted inlineParser = do
1519+ doubleQuoteStart
1520+ (B.doubleQuoted . mconcat <$>
1521+ try
1522+ (withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)))
1523+ <|> pure (B.str "\8220")
14271524
14281525failIfInQuoteContext :: (HasQuoteContext st m, Stream s m t)
14291526 => QuoteContext
@@ -1432,13 +1529,14 @@ failIfInQuoteContext context = do
14321529 context' <- getQuoteContext
14331530 when (context' == context) $ Prelude.fail "already inside quotes"
14341531
1435−charOrRef :: Stream s m Char => [Char] -> ParserT s st m Char
1532+charOrRef :: (Stream s m Char, UpdateSourcePos s Char) => [Char] -> ParserT s st m Char
14361533charOrRef cs =
14371534 oneOf cs <|> try (do c <- characterReference
14381535 guard (c `elem` cs)
14391536 return c)
14401537
1441−singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
1538+singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m,
1539+ Stream s m Char, UpdateSourcePos s Char)
14421540 => ParserT s st m ()
14431541singleQuoteStart = do
14441542 failIfInQuoteContext InSingleQuote
@@ -1446,30 +1544,39 @@ singleQuoteStart = do
14461544 guard =<< notAfterString
14471545 try $ do
14481546 charOrRef "'\8216\145"
1449− notFollowedBy (oneOf [' ', '\t', '\n'])
1547+ void $ lookAhead (satisfy (not . isSpaceChar))
14501548
1451−singleQuoteEnd :: Stream s m Char
1549+singleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
14521550 => ParserT s st m ()
14531551singleQuoteEnd = try $ do
14541552 charOrRef "'\8217\146"
14551553 notFollowedBy alphaNum
14561554
1457−doubleQuoteStart :: (HasQuoteContext st m, Stream s m Char)
1555+doubleQuoteStart :: (HasLastStrPosition st,
1556+ HasQuoteContext st m,
1557+ Stream s m Char, UpdateSourcePos s Char)
14581558 => ParserT s st m ()
14591559doubleQuoteStart = do
14601560 failIfInQuoteContext InDoubleQuote
1561+ guard =<< notAfterString
14611562 try $ do charOrRef "\"\8220\147"
1462− notFollowedBy (oneOf [' ', '\t', '\n'])
1563+ void $ lookAhead (satisfy (not . isSpaceChar))
14631564
1464−doubleQuoteEnd :: Stream s m Char
1565+doubleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
14651566 => ParserT s st m ()
14661567doubleQuoteEnd = void (charOrRef "\"\8221\148")
14671568
1468−ellipses :: Stream s m Char
1569+apostrophe :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1570+apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\8217")
1571+
1572+doubleCloseQuote :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1573+doubleCloseQuote = B.str "\8221" <$ char '"'
1574+
1575+ellipses :: (Stream s m Char, UpdateSourcePos s Char)
14691576 => ParserT s st m Inlines
14701577ellipses = try (string "..." >> return (B.str "\8230"))
14711578
1472−dash :: (HasReaderOptions st, Stream s m Char)
1579+dash :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
14731580 => ParserT s st m Inlines
14741581dash = try $ do
14751582 oldDashes <- extensionEnabled Ext_old_dashes <$> getOption readerExtensions
@@ -1496,20 +1603,28 @@ nested p = do
14961603 updateState $ \st -> st{ stateMaxNestingLevel = nestlevel }
14971604 return res
14981605
1499−citeKey :: (Stream s m Char, HasLastStrPosition st)
1500− => ParserT s st m (Bool, Text)
1501−citeKey = try $ do
1606+citeKey :: (Stream s m Char, UpdateSourcePos s Char, HasLastStrPosition st)
1607+ => Bool -- ^ If True, allow expanded @{..} syntax.
1608+ -> ParserT s st m (Bool, Text)
1609+citeKey allowBraced = try $ do
15021610 guard =<< notAfterString
15031611 suppress_author <- option False (True <$ char '-')
15041612 char '@'
1613+ key <- simpleCiteIdentifier
1614+ <|> if allowBraced
1615+ then charsInBalanced '{' '}' (satisfy (not . isSpace))
1616+ else mzero
1617+ return (suppress_author, key)
1618+
1619+simpleCiteIdentifier :: (Stream s m Char, UpdateSourcePos s Char)
1620+ => ParserT s st m Text
1621+simpleCiteIdentifier = do
15051622 firstChar <- alphaNum <|> char '_' <|> char '*' -- @* for wildcard in nocite
15061623 let regchar = satisfy (\c -> isAlphaNum c || c == '_')
15071624 let internal p = try $ p <* lookAhead regchar
15081625 rest <- many $ regchar <|> internal (oneOf ":.#$%&-+?<>~/") <|>
15091626 try (oneOf ":/" <* lookAhead (char '/'))
1510− let key = firstChar:rest
1511− return (suppress_author, T.pack key)
1512−
1627+ return $ T.pack $ firstChar:rest
15131628
15141629token :: (Stream s m t)
15151630 => (t -> Text)
@@ -1529,12 +1644,15 @@ extractIdClass (ident, cls, kvs) = (ident', cls', kvs')
15291644 cls' = maybe cls T.words $ lookup "class" kvs
15301645 kvs' = filter (\(k,_) -> k /= "id" || k /= "class") kvs
15311646
1532−insertIncludedFile' :: (PandocMonad m, HasIncludeFiles st)
1533− => ParserT a st m (mf Blocks)
1534− -> (Text -> a)
1535− -> [FilePath] -> FilePath
1536− -> ParserT a st m (mf Blocks)
1537−insertIncludedFile' blocks totoks dirs f = do
1647+insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1648+ => ParserT a st m b -- ^ parser to apply
1649+ -> (Text -> a) -- ^ convert Text to stream type
1650+ -> [FilePath] -- ^ search path (directories)
1651+ -> FilePath -- ^ path of file to include
1652+ -> Maybe Int -- ^ start line (negative counts from end)
1653+ -> Maybe Int -- ^ end line (negative counts from end)
1654+ -> ParserT a st m b
1655+insertIncludedFile parser toStream dirs f mbstartline mbendline = do
15381656 oldPos <- getPosition
15391657 oldInput <- getInput
15401658 containers <- getIncludeFiles <$> getState
@@ -1543,32 +1661,32 @@ insertIncludedFile' blocks totoks dirs f = do
15431661 updateState $ addIncludeFile $ T.pack f
15441662 mbcontents <- readFileFromDirs dirs f
15451663 contents <- case mbcontents of
1546− Just s -> return s
1664+ Just s -> return $ exciseLines mbstartline mbendline s
15471665 Nothing -> do
15481666 report $ CouldNotLoadIncludeFile (T.pack f) oldPos
15491667 return ""
1550− setPosition $ newPos f 1 1
1551− setInput $ totoks contents
1552− bs <- blocks
1668+ setInput $ toStream contents
1669+ setPosition $ newPos f (fromMaybe 1 mbstartline) 1
1670+ result <- parser
15531671 setInput oldInput
15541672 setPosition oldPos
15551673 updateState dropLatestIncludeFile
1556− return bs
1674+ return result
1675+
1676+exciseLines :: Maybe Int -> Maybe Int -> Text -> Text
1677+exciseLines Nothing Nothing t = t
1678+exciseLines mbstartline mbendline t =
1679+ T.unlines $ take (endline' - (startline' - 1))
1680+ $ drop (startline' - 1) contentLines
1681+ where
1682+ contentLines = T.lines t
1683+ numLines = length contentLines
1684+ startline' = case mbstartline of
1685+ Nothing -> 1
1686+ Just x | x >= 0 -> x
1687+ | otherwise -> numLines + x -- negative from end
1688+ endline' = case mbendline of
1689+ Nothing -> numLines
1690+ Just x | x >= 0 -> x
1691+ | otherwise -> numLines + x -- negative from end
15571692
1558−-- | Parse content of include file as blocks. Circular includes result in an
1559−-- @PandocParseError@.
1560−insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1561− => ParserT [a] st m Blocks
1562− -> (Text -> [a])
1563− -> [FilePath] -> FilePath
1564− -> ParserT [a] st m Blocks
1565−insertIncludedFile blocks totoks dirs f =
1566− runIdentity <$> insertIncludedFile' (Identity <$> blocks) totoks dirs f
1567−
1568−-- | Parse content of include file as future blocks. Circular includes result in
1569−-- an @PandocParseError@.
1570−insertIncludedFileF :: (PandocMonad m, HasIncludeFiles st)
1571− => ParserT Text st m (Future st Blocks)
1572− -> [FilePath] -> FilePath
1573− -> ParserT Text st m (Future st Blocks)
1574−insertIncludedFileF p = insertIncludedFile' p id
src/Text/Pandoc/Readers.hs+55−48
@@ -1,4 +1,5 @@
11{-# LANGUAGE MonoLocalBinds #-}
2+{-# LANGUAGE RankNTypes #-}
23{-# LANGUAGE ScopedTypeVariables #-}
34{-# LANGUAGE OverloadedStrings #-}
45{- |
@@ -65,12 +66,14 @@ import Data.Aeson
6566import qualified Data.ByteString.Lazy as BL
6667import Data.Text (Text)
6768import qualified Data.Text as T
69+import Text.Pandoc.Shared (tshow)
6870import Text.Pandoc.Class
6971import Text.Pandoc.Definition
7072import Text.Pandoc.Error
7173import Text.Pandoc.Extensions
7274import Text.Pandoc.Options
7375import Text.Pandoc.Readers.CommonMark
76+import Text.Pandoc.Readers.Markdown
7477import Text.Pandoc.Readers.Creole
7578import Text.Pandoc.Readers.DocBook
7679import Text.Pandoc.Readers.Docx
@@ -83,7 +86,6 @@ import Text.Pandoc.Readers.HTML (readHtml)
8386import Text.Pandoc.Readers.JATS (readJATS)
8487import Text.Pandoc.Readers.Jira (readJira)
8588import Text.Pandoc.Readers.LaTeX
86−import Text.Pandoc.Readers.Markdown
8789import Text.Pandoc.Readers.MediaWiki
8890import Text.Pandoc.Readers.Muse
8991import Text.Pandoc.Readers.Native
@@ -101,59 +103,60 @@ import Text.Pandoc.Readers.CSV
101103import Text.Pandoc.Readers.CslJson
102104import Text.Pandoc.Readers.BibTeX
103105import qualified Text.Pandoc.UTF8 as UTF8
104−import Text.Parsec.Error
106+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
105107
106−data Reader m = TextReader (ReaderOptions -> Text -> m Pandoc)
108+data Reader m = TextReader (forall a . ToSources a =>
109+ ReaderOptions -> a -> m Pandoc)
107110 | ByteStringReader (ReaderOptions -> BL.ByteString -> m Pandoc)
108111
109112-- | Association list of formats and readers.
110113readers :: PandocMonad m => [(Text, Reader m)]
111−readers = [ ("native" , TextReader readNative)
112− ,("json" , TextReader readJSON)
113− ,("markdown" , TextReader readMarkdown)
114− ,("markdown_strict" , TextReader readMarkdown)
115− ,("markdown_phpextra" , TextReader readMarkdown)
116− ,("markdown_github" , TextReader readMarkdown)
117− ,("markdown_mmd", TextReader readMarkdown)
118− ,("commonmark" , TextReader readCommonMark)
119− ,("commonmark_x" , TextReader readCommonMark)
120− ,("creole" , TextReader readCreole)
121− ,("dokuwiki" , TextReader readDokuWiki)
122− ,("gfm" , TextReader readCommonMark)
123− ,("rst" , TextReader readRST)
124− ,("mediawiki" , TextReader readMediaWiki)
125− ,("vimwiki" , TextReader readVimwiki)
126− ,("docbook" , TextReader readDocBook)
127− ,("opml" , TextReader readOPML)
128− ,("org" , TextReader readOrg)
129− ,("textile" , TextReader readTextile) -- TODO : textile+lhs
130− ,("html" , TextReader readHtml)
131− ,("jats" , TextReader readJATS)
132− ,("jira" , TextReader readJira)
133− ,("latex" , TextReader readLaTeX)
134− ,("haddock" , TextReader readHaddock)
135− ,("twiki" , TextReader readTWiki)
136− ,("tikiwiki" , TextReader readTikiWiki)
137− ,("docx" , ByteStringReader readDocx)
138− ,("odt" , ByteStringReader readOdt)
139− ,("t2t" , TextReader readTxt2Tags)
140− ,("epub" , ByteStringReader readEPUB)
141− ,("muse" , TextReader readMuse)
142− ,("man" , TextReader readMan)
143− ,("fb2" , TextReader readFB2)
144− ,("ipynb" , TextReader readIpynb)
145− ,("csv" , TextReader readCSV)
146− ,("csljson" , TextReader readCslJson)
147− ,("bibtex" , TextReader readBibTeX)
148− ,("biblatex" , TextReader readBibLaTeX)
114+readers = [("native" , TextReader readNative)
115+ ,("json" , TextReader readJSON)
116+ ,("markdown" , TextReader readMarkdown)
117+ ,("markdown_strict" , TextReader readMarkdown)
118+ ,("markdown_phpextra" , TextReader readMarkdown)
119+ ,("markdown_github" , TextReader readMarkdown)
120+ ,("markdown_mmd", TextReader readMarkdown)
121+ ,("commonmark" , TextReader readCommonMark)
122+ ,("commonmark_x" , TextReader readCommonMark)
123+ ,("creole" , TextReader readCreole)
124+ ,("dokuwiki" , TextReader readDokuWiki)
125+ ,("gfm" , TextReader readCommonMark)
126+ ,("rst" , TextReader readRST)
127+ ,("mediawiki" , TextReader readMediaWiki)
128+ ,("vimwiki" , TextReader readVimwiki)
129+ ,("docbook" , TextReader readDocBook)
130+ ,("opml" , TextReader readOPML)
131+ ,("org" , TextReader readOrg)
132+ ,("textile" , TextReader readTextile) -- TODO : textile+lhs
133+ ,("html" , TextReader readHtml)
134+ ,("jats" , TextReader readJATS)
135+ ,("jira" , TextReader readJira)
136+ ,("latex" , TextReader readLaTeX)
137+ ,("haddock" , TextReader readHaddock)
138+ ,("twiki" , TextReader readTWiki)
139+ ,("tikiwiki" , TextReader readTikiWiki)
140+ ,("docx" , ByteStringReader readDocx)
141+ ,("odt" , ByteStringReader readOdt)
142+ ,("t2t" , TextReader readTxt2Tags)
143+ ,("epub" , ByteStringReader readEPUB)
144+ ,("muse" , TextReader readMuse)
145+ ,("man" , TextReader readMan)
146+ ,("fb2" , TextReader readFB2)
147+ ,("ipynb" , TextReader readIpynb)
148+ ,("csv" , TextReader readCSV)
149+ ,("csljson" , TextReader readCslJson)
150+ ,("bibtex" , TextReader readBibTeX)
151+ ,("biblatex" , TextReader readBibLaTeX)
149152 ]
150153
151154-- | Retrieve reader, extensions based on formatSpec (format+extensions).
152155getReader :: PandocMonad m => Text -> m (Reader m, Extensions)
153156getReader s =
154157 case parseFormatSpec s of
155− Left e -> throwError $ PandocAppError
156− $ T.intercalate "\n" [T.pack m | Message m <- errorMessages e]
158+ Left e -> throwError $ PandocAppError $
159+ "Error parsing reader format " <> tshow s <> ": " <> tshow e
157160 Right (readerName, extsToEnable, extsToDisable) ->
158161 case lookup readerName readers of
159162 Nothing -> throwError $ PandocUnknownReaderError
@@ -173,9 +176,13 @@ getReader s =
173176 return (r, exts)
174177
175178-- | Read pandoc document from JSON format.
176−readJSON :: PandocMonad m
177− => ReaderOptions -> Text -> m Pandoc
178−readJSON _ t =
179− case eitherDecode' . BL.fromStrict . UTF8.fromText $ t of
179+readJSON :: (PandocMonad m, ToSources a)
180+ => ReaderOptions
181+ -> a
182+ -> m Pandoc
183+readJSON _ s =
184+ case eitherDecode' . BL.fromStrict . UTF8.fromText
185+ . sourcesToText . toSources $ s of
180186 Right doc -> return doc
181− Left e -> throwError $ PandocParseError ("JSON parse error: " <> T.pack e)
187+ Left e -> throwError $ PandocParseError ("JSON parse error: "
188+ <> T.pack e)
src/Text/Pandoc/Readers/BibTeX.hs+14−8
@@ -23,41 +23,47 @@ where
2323import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
26−import Data.Text (Text)
2726import Citeproc (Lang(..), parseLang)
2827import Citeproc.Locale (getLocale)
2928import Text.Pandoc.Error (PandocError(..))
3029import Text.Pandoc.Class (PandocMonad, lookupEnv)
3130import Text.Pandoc.Citeproc.BibTeX as BibTeX
3231import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
32+import Text.Pandoc.Sources (ToSources(..))
3333import Control.Monad.Except (throwError)
3434
3535-- | Read BibTeX from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readBibTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
39+readBibTeX :: (PandocMonad m, ToSources a)
40+ => ReaderOptions -> a -> m Pandoc
4041readBibTeX = readBibTeX' BibTeX.Bibtex
4142
4243-- | Read BibLaTeX from an input string and return a Pandoc document.
4344-- The document will have only metadata, with an empty body.
4445-- The metadata will contain a `references` field with the
4546-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
46−readBibLaTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
47+readBibLaTeX :: (PandocMonad m, ToSources a)
48+ => ReaderOptions -> a -> m Pandoc
4749readBibLaTeX = readBibTeX' BibTeX.Biblatex
4850
49−readBibTeX' :: PandocMonad m => Variant -> ReaderOptions -> Text -> m Pandoc
51+readBibTeX' :: (PandocMonad m, ToSources a)
52+ => Variant -> ReaderOptions -> a -> m Pandoc
5053readBibTeX' variant _opts t = do
51− lang <- maybe (Lang "en" (Just "US")) parseLang
52− <$> lookupEnv "LANG"
54+ mblangEnv <- lookupEnv "LANG"
55+ let defaultLang = Lang "en" Nothing (Just "US") [] [] []
56+ let lang = case mblangEnv of
57+ Nothing -> defaultLang
58+ Just l -> either (const defaultLang) id $ parseLang l
5359 locale <- case getLocale lang of
5460 Left e ->
55− case getLocale (Lang "en" (Just "US")) of
61+ case getLocale (Lang "en" Nothing (Just "US") [] [] []) of
5662 Right l -> return l
5763 Left _ -> throwError $ PandocCiteprocError e
5864 Right l -> return l
5965 case BibTeX.readBibtexString variant locale (const True) t of
60− Left e -> throwError $ PandocParsecError t e
66+ Left e -> throwError $ PandocParsecError (toSources t) e
6167 Right refs -> return $ setMeta "references"
6268 (map referenceToMetaValue refs)
6369 . setMeta "nocite"
src/Text/Pandoc/Readers/CSV.hs+7−7
@@ -13,23 +13,23 @@
1313Conversion from CSV to a 'Pandoc' table.
1414-}
1515module Text.Pandoc.Readers.CSV ( readCSV ) where
16−import Data.Text (Text)
1716import qualified Data.Text as T
1817import Text.Pandoc.CSV (parseCSV, defaultCSVOptions)
1918import Text.Pandoc.Definition
2019import qualified Text.Pandoc.Builder as B
2120import Text.Pandoc.Class (PandocMonad)
22−import Text.Pandoc.Shared (crFilter)
2321import Text.Pandoc.Error
22+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2423import Text.Pandoc.Options (ReaderOptions)
2524import Control.Monad.Except (throwError)
2625
27−readCSV :: PandocMonad m
26+readCSV :: (PandocMonad m, ToSources a)
2827 => ReaderOptions -- ^ Reader options
29− -> Text -- ^ Text to parse (assuming @'\n'@ line endings)
28+ -> a
3029 -> m Pandoc
31−readCSV _opts s =
32− case parseCSV defaultCSVOptions (crFilter s) of
30+readCSV _opts s = do
31+ let txt = sourcesToText $ toSources s
32+ case parseCSV defaultCSVOptions txt of
3333 Right (r:rs) -> return $ B.doc $ B.table capt
3434 (zip aligns widths)
3535 (TableHead nullAttr hdrs)
@@ -45,4 +45,4 @@ readCSV _opts s =
4545 aligns = replicate numcols AlignDefault
4646 widths = replicate numcols ColWidthDefault
4747 Right [] -> return $ B.doc mempty
48− Left e -> throwError $ PandocParsecError s e
48+ Left e -> throwError $ PandocParsecError (toSources [("",txt)]) e
src/Text/Pandoc/Readers/CommonMark.hs+46−4
@@ -26,19 +26,59 @@ import Text.Pandoc.Definition
2626import Text.Pandoc.Builder as B
2727import Text.Pandoc.Options
2828import Text.Pandoc.Error
29+import Text.Pandoc.Readers.Metadata (yamlMetaBlock)
2930import Control.Monad.Except
3031import Data.Functor.Identity (runIdentity)
3132import Data.Typeable
33+import Text.Pandoc.Parsing (runParserT, getInput,
34+ runF, defaultParserState, option, many1, anyChar,
35+ Sources(..), ToSources(..), ParserT, Future,
36+ sourceName)
37+import qualified Data.Text as T
3238
3339-- | Parse a CommonMark formatted string into a 'Pandoc' structure.
34−readCommonMark :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
40+readCommonMark :: (PandocMonad m, ToSources a)
41+ => ReaderOptions -> a -> m Pandoc
3542readCommonMark opts s
43+ | isEnabled Ext_yaml_metadata_block opts = do
44+ let sources = toSources s
45+ let toks = concatMap sourceToToks (unSources sources)
46+ res <- runParserT (do meta <- yamlMetaBlock (metaValueParser opts)
47+ rest <- getInput
48+ return (meta, rest))
49+ defaultParserState "YAML metadata" (toSources s)
50+ case res of
51+ Left _ -> readCommonMarkBody opts sources toks
52+ Right (meta, rest) -> do
53+ -- strip off metadata section and parse body
54+ let body = concatMap sourceToToks (unSources rest)
55+ Pandoc _ bs <- readCommonMarkBody opts sources body
56+ return $ Pandoc (runF meta defaultParserState) bs
57+ | otherwise = do
58+ let sources = toSources s
59+ let toks = concatMap sourceToToks (unSources sources)
60+ readCommonMarkBody opts sources toks
61+
62+sourceToToks :: (SourcePos, Text) -> [Tok]
63+sourceToToks (pos, s) = tokenize (sourceName pos) s
64+
65+metaValueParser :: Monad m
66+ => ReaderOptions -> ParserT Sources st m (Future st MetaValue)
67+metaValueParser opts = do
68+ inp <- option "" $ T.pack <$> many1 anyChar
69+ let toks = concatMap sourceToToks (unSources (toSources inp))
70+ case runIdentity (parseCommonmarkWith (specFor opts) toks) of
71+ Left _ -> mzero
72+ Right (Cm bls :: Cm () Blocks) -> return $ return $ B.toMetaValue bls
73+
74+readCommonMarkBody :: PandocMonad m => ReaderOptions -> Sources -> [Tok] -> m Pandoc
75+readCommonMarkBody opts s toks
3676 | isEnabled Ext_sourcepos opts =
37− case runIdentity (commonmarkWith (specFor opts) "" s) of
77+ case runIdentity (parseCommonmarkWith (specFor opts) toks) of
3878 Left err -> throwError $ PandocParsecError s err
3979 Right (Cm bls :: Cm SourceRange Blocks) -> return $ B.doc bls
4080 | otherwise =
41− case runIdentity (commonmarkWith (specFor opts) "" s) of
81+ case runIdentity (parseCommonmarkWith (specFor opts) toks) of
4282 Left err -> throwError $ PandocParsecError s err
4383 Right (Cm bls :: Cm () Blocks) -> return $ B.doc bls
4484
@@ -72,5 +112,7 @@ specFor opts = foldr ($) defaultSyntaxSpec exts
72112 | isEnabled Ext_implicit_header_references opts ] ++
73113 [ (footnoteSpec <>) | isEnabled Ext_footnotes opts ] ++
74114 [ (definitionListSpec <>) | isEnabled Ext_definition_lists opts ] ++
75− [ (taskListSpec <>) | isEnabled Ext_task_lists opts ]
115+ [ (taskListSpec <>) | isEnabled Ext_task_lists opts ] ++
116+ [ (rebaseRelativePathsSpec <>)
117+ | isEnabled Ext_rebase_relative_paths opts ]
76118
src/Text/Pandoc/Readers/Creole.hs+5−6
@@ -23,21 +23,20 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad (..))
2323import Text.Pandoc.Definition
2424import Text.Pandoc.Options
2525import Text.Pandoc.Parsing hiding (enclosed)
26−import Text.Pandoc.Shared (crFilter)
27−
2826
2927-- | Read creole from an input string and return a Pandoc document.
30−readCreole :: PandocMonad m
28+readCreole :: (PandocMonad m, ToSources a)
3129 => ReaderOptions
32− -> Text
30+ -> a
3331 -> m Pandoc
3432readCreole opts s = do
35− res <- readWithM parseCreole def{ stateOptions = opts } $ crFilter s <> "\n\n"
33+ let sources = ensureFinalNewlines 2 (toSources s)
34+ res <- readWithM parseCreole def{ stateOptions = opts } sources
3635 case res of
3736 Left e -> throwError e
3837 Right d -> return d
3938
40−type CRLParser = ParserT Text ParserState
39+type CRLParser = ParserT Sources ParserState
4140
4241--
4342-- Utility functions
src/Text/Pandoc/Readers/CslJson.hs+5−4
@@ -24,21 +24,22 @@ import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
2626import qualified Text.Pandoc.UTF8 as UTF8
27−import Data.Text (Text)
2827import qualified Data.Text as T
2928import Text.Pandoc.Error (PandocError(..))
3029import Text.Pandoc.Class (PandocMonad)
3130import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
3231import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
3332import Control.Monad.Except (throwError)
33+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3434
3535-- | Read CSL JSON from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readCslJson :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
40−readCslJson _opts t =
41− case cslJsonToReferences (UTF8.fromText t) of
39+readCslJson :: (PandocMonad m, ToSources a)
40+ => ReaderOptions -> a -> m Pandoc
41+readCslJson _opts x =
42+ case cslJsonToReferences (UTF8.fromText $ sourcesToText $ toSources x) of
4243 Left e -> throwError $ PandocParseError $ T.pack e
4344 Right refs -> return $ setMeta "references"
4445 (map referenceToMetaValue refs)
src/Text/Pandoc/Readers/DocBook.hs+30−15
@@ -18,6 +18,7 @@ import Data.Either (rights)
1818import Data.Foldable (asum)
1919import Data.Generics
2020import Data.List (intersperse,elemIndex)
21+import Data.List.NonEmpty (nonEmpty)
2122import Data.Maybe (fromMaybe,mapMaybe)
2223import Data.Text (Text)
2324import qualified Data.Text as T
@@ -29,7 +30,8 @@ import Text.Pandoc.Builder
2930import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
3031import Text.Pandoc.Options
3132import Text.Pandoc.Logging (LogMessage(..))
32−import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
33+import Text.Pandoc.Shared (safeRead, extractSpaces)
34+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3335import Text.TeXMath (readMathML, writeTeX)
3436import Text.Pandoc.XML.Light
3537
@@ -132,6 +134,7 @@ List of all DocBook tags, with [x] indicating implemented,
132134[ ] corpcredit - A corporation or organization credited in a document
133135[ ] corpname - The name of a corporation
134136[ ] country - The name of a country
137+[x] danger - An admonition set off from the text indicating hazardous situation
135138[ ] database - The name of a database, or part of a database
136139[x] date - The date of publication or revision of a document
137140[ ] dedication - A wrapper for the dedication section of a book
@@ -538,11 +541,15 @@ instance Default DBState where
538541 , dbContent = [] }
539542
540543
541−readDocBook :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
544+readDocBook :: (PandocMonad m, ToSources a)
545+ => ReaderOptions
546+ -> a
547+ -> m Pandoc
542548readDocBook _ inp = do
549+ let sources = toSources inp
543550 tree <- either (throwError . PandocXMLError "") return $
544551 parseXMLContents
545− (TL.fromStrict . handleInstructions $ crFilter inp)
552+ (TL.fromStrict . handleInstructions . sourcesToText $ sources)
546553 (bs, st') <- flip runStateT (def{ dbContent = tree }) $ mapM parseBlock tree
547554 return $ Pandoc (dbMeta st') (toList . mconcat $ bs)
548555
@@ -594,16 +601,24 @@ addMetadataFromElement e = do
594601 Nothing -> return ()
595602 Just z -> addMetaField "author" z
596603 addMetaField "subtitle" e
597− addMetaField "author" e
604+ addAuthor e
598605 addMetaField "date" e
599606 addMetaField "release" e
600607 addMetaField "releaseinfo" e
601608 return mempty
602− where addMetaField fieldname elt =
603− case filterChildren (named fieldname) elt of
604− [] -> return ()
605− [z] -> getInlines z >>= addMeta fieldname
606− zs -> mapM getInlines zs >>= addMeta fieldname
609+ where
610+ addAuthor elt =
611+ case filterChildren (named "author") elt of
612+ [] -> return ()
613+ [z] -> fromAuthor z >>= addMeta "author"
614+ zs -> mapM fromAuthor zs >>= addMeta "author"
615+ fromAuthor elt =
616+ mconcat . intersperse space <$> mapM getInlines (elChildren elt)
617+ addMetaField fieldname elt =
618+ case filterChildren (named fieldname) elt of
619+ [] -> return ()
620+ [z] -> getInlines z >>= addMeta fieldname
621+ zs -> mapM getInlines zs >>= addMeta fieldname
607622
608623addMeta :: PandocMonad m => ToMetaValue a => Text -> a -> DB m ()
609624addMeta field val = modify (setMeta field val)
@@ -704,7 +719,7 @@ blockTags =
704719 ] ++ admonitionTags
705720
706721admonitionTags :: [Text]
707−admonitionTags = ["important","caution","note","tip","warning"]
722+admonitionTags = ["caution","danger","important","note","tip","warning"]
708723
709724-- Trim leading and trailing newline characters
710725trimNl :: Text -> Text
@@ -949,9 +964,8 @@ parseBlock (Elem e) =
949964 (x >= '0' && x <= '9')
950965 || x == '.') w
951966 if n > 0 then Just n else Nothing
952− let numrows = case bodyrows of
953− [] -> 0
954− xs -> maximum $ map length xs
967+ let numrows = maybe 0 maximum $ nonEmpty
968+ $ map length bodyrows
955969 let aligns = case colspecs of
956970 [] -> replicate numrows AlignDefault
957971 cs -> map toAlignment cs
@@ -1075,7 +1089,7 @@ parseInline (Elem e) =
10751089 return $ spanWith (attrValue "id" e, [], []) mempty
10761090 "phrase" -> do
10771091 let ident = attrValue "id" e
1078− let classes = T.words $ attrValue "class" e
1092+ let classes = T.words $ attrValue "role" e
10791093 if ident /= "" || classes /= []
10801094 then innerInlines (spanWith (ident,classes,[]))
10811095 else innerInlines id
@@ -1239,7 +1253,8 @@ equation e constructor =
12391253 where
12401254 mathMLEquations :: [Text]
12411255 mathMLEquations = map writeTeX $ rights $ readMath
1242− (\x -> qName (elName x) == "math" && qPrefix (elName x) == Just "mml")
1256+ (\x -> qName (elName x) == "math" &&
1257+ qURI (elName x) == Just "http://www.w3.org/1998/Math/MathML")
12431258 (readMathML . showElement)
12441259
12451260 latexEquations :: [Text]
src/Text/Pandoc/Readers/Docx.hs+59−42
@@ -61,13 +61,14 @@ module Text.Pandoc.Readers.Docx
6161import Codec.Archive.Zip
6262import Control.Monad.Reader
6363import Control.Monad.State.Strict
64+import Data.Bifunctor (bimap, first)
6465import qualified Data.ByteString.Lazy as B
6566import Data.Default (Default)
66−import Data.List (delete, intersect)
67+import Data.List (delete, intersect, foldl')
6768import Data.Char (isSpace)
6869import qualified Data.Map as M
6970import qualified Data.Text as T
70−import Data.Maybe (isJust, fromMaybe)
71+import Data.Maybe (catMaybes, isJust, fromMaybe)
7172import Data.Sequence (ViewL (..), viewl)
7273import qualified Data.Sequence as Seq
7374import qualified Data.Set as Set
@@ -85,6 +86,7 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
8586import qualified Text.Pandoc.Class.PandocMonad as P
8687import Text.Pandoc.Error
8788import Text.Pandoc.Logging
89+import Data.List.NonEmpty (nonEmpty)
8890
8991readDocx :: PandocMonad m
9092 => ReaderOptions
@@ -112,6 +114,7 @@ data DState = DState { docxAnchorMap :: M.Map T.Text T.Text
112114 -- restarting
113115 , docxListState :: M.Map (T.Text, T.Text) Integer
114116 , docxPrevPara :: Inlines
117+ , docxTableCaptions :: [Blocks]
115118 }
116119
117120instance Default DState where
@@ -122,6 +125,7 @@ instance Default DState where
122125 , docxDropCap = mempty
123126 , docxListState = M.empty
124127 , docxPrevPara = mempty
128+ , docxTableCaptions = []
125129 }
126130
127131data DEnv = DEnv { docxOptions :: ReaderOptions
@@ -490,15 +494,32 @@ singleParaToPlain blks
490494 singleton $ Plain ils
491495singleParaToPlain blks = blks
492496
493−cellToBlocks :: PandocMonad m => Docx.Cell -> DocxContext m Blocks
494−cellToBlocks (Docx.Cell bps) = do
497+cellToCell :: PandocMonad m => RowSpan -> Docx.Cell -> DocxContext m Pandoc.Cell
498+cellToCell rowSpan (Docx.Cell gridSpan _ bps) = do
495499 blks <- smushBlocks <$> mapM bodyPartToBlocks bps
496− return $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
500+ let blks' = singleParaToPlain $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
501+ return (cell AlignDefault rowSpan (ColSpan (fromIntegral gridSpan)) blks')
502+
503+rowsToRows :: PandocMonad m => [Docx.Row] -> DocxContext m [Pandoc.Row]
504+rowsToRows rows = do
505+ let rowspans = (fmap . fmap) (first RowSpan) (Docx.rowsToRowspans rows)
506+ cells <- traverse (traverse (uncurry cellToCell)) rowspans
507+ return (fmap (Pandoc.Row nullAttr) cells)
508+
509+splitHeaderRows :: Bool -> [Docx.Row] -> ([Docx.Row], [Docx.Row])
510+splitHeaderRows hasFirstRowFormatting rs = bimap reverse reverse $ fst
511+ $ if hasFirstRowFormatting
512+ then foldl' f ((take 1 rs, []), True) (drop 1 rs)
513+ else foldl' f (([], []), False) rs
514+ where
515+ f ((headerRows, bodyRows), previousRowWasHeader) r@(Docx.Row h cs)
516+ | h == HasTblHeader || (previousRowWasHeader && any isContinuationCell cs)
517+ = ((r : headerRows, bodyRows), True)
518+ | otherwise
519+ = ((headerRows, r : bodyRows), False)
520+
521+ isContinuationCell (Docx.Cell _ vm _) = vm == Docx.Continue
497522
498−rowToBlocksList :: PandocMonad m => Docx.Row -> DocxContext m [Blocks]
499−rowToBlocksList (Docx.Row cells) = do
500− blksList <- mapM cellToBlocks cells
501− return $ map singleParaToPlain blksList
502523
503524-- like trimInlines, but also take out linebreaks
504525trimSps :: Inlines -> Inlines
@@ -545,6 +566,11 @@ normalizeToClassName = T.map go . fromStyleName
545566 where go c | isSpace c = '-'
546567 | otherwise = c
547568
569+bodyPartToTableCaption :: PandocMonad m => BodyPart -> DocxContext m (Maybe Blocks)
570+bodyPartToTableCaption (TblCaption pPr parparts) =
571+ Just <$> bodyPartToBlocks (Paragraph pPr parparts)
572+bodyPartToTableCaption _ = pure Nothing
573+
548574bodyPartToBlocks :: PandocMonad m => BodyPart -> DocxContext m Blocks
549575bodyPartToBlocks (Paragraph pPr parparts)
550576 | Just True <- pBidi pPr = do
@@ -636,54 +662,43 @@ bodyPartToBlocks (ListItem pPr _ _ _ parparts) =
636662 let pPr' = pPr {pStyle = constructBogusParStyleData "list-paragraph": pStyle pPr}
637663 in
638664 bodyPartToBlocks $ Paragraph pPr' parparts
665+bodyPartToBlocks (TblCaption _ _) =
666+ return $ para mempty -- collected separately
639667bodyPartToBlocks (Tbl _ _ _ []) =
640668 return $ para mempty
641−bodyPartToBlocks (Tbl cap _ look parts@(r:rs)) = do
642− let cap' = simpleCaption $ plain $ text cap
643− (hdr, rows) = case firstRowFormatting look of
644− True | null rs -> (Nothing, [r])
645− | otherwise -> (Just r, rs)
646− False -> (Nothing, r:rs)
647−
648− cells <- mapM rowToBlocksList rows
669+bodyPartToBlocks (Tbl cap grid look parts) = do
670+ captions <- gets docxTableCaptions
671+ fullCaption <- case captions of
672+ c : cs -> do
673+ modify (\s -> s { docxTableCaptions = cs })
674+ return c
675+ [] -> return $ if T.null cap then mempty else plain (text cap)
676+ let shortCaption = if T.null cap then Nothing else Just (toList (text cap))
677+ cap' = caption shortCaption fullCaption
678+ (hdr, rows) = splitHeaderRows (firstRowFormatting look) parts
649679
650680 let width = maybe 0 maximum $ nonEmpty $ map rowLength parts
651− -- Data.List.NonEmpty is not available with ghc 7.10 so we roll out
652− -- our own, see
653− -- https://github.com/jgm/pandoc/pull/4361#issuecomment-365416155
654− nonEmpty [] = Nothing
655− nonEmpty l = Just l
656681 rowLength :: Docx.Row -> Int
657− rowLength (Docx.Row c) = length c
682+ rowLength (Docx.Row _ c) = sum (fmap (\(Docx.Cell gridSpan _ _) -> fromIntegral gridSpan) c)
658683
659− let toRow = Pandoc.Row nullAttr . map simpleCell
660− toHeaderRow l = [toRow l | not (null l)]
684+ headerCells <- rowsToRows hdr
685+ bodyCells <- rowsToRows rows
661686
662− -- pad cells. New Text.Pandoc.Builder will do that for us,
663− -- so this is for compatibility while we switch over.
664− let cells' = map (\row -> toRow $ take width (row ++ repeat mempty)) cells
665−
666− hdrCells <- case hdr of
667− Just r' -> toHeaderRow <$> rowToBlocksList r'
668− Nothing -> return []
669−
670− -- The two following variables (horizontal column alignment and
671− -- relative column widths) go to the default at the
672− -- moment. Width information is in the TblGrid field of the Tbl,
673− -- so should be possible. Alignment might be more difficult,
674− -- since there doesn't seem to be a column entity in docx.
687+ -- Horizontal column alignment goes to the default at the moment. Getting
688+ -- it might be difficult, since there doesn't seem to be a column entity
689+ -- in docx.
675690 let alignments = replicate width AlignDefault
676− widths = replicate width ColWidthDefault
691+ totalWidth = sum grid
692+ widths = (\w -> ColWidth (fromInteger w / fromInteger totalWidth)) <$> grid
677693
678694 return $ table cap'
679695 (zip alignments widths)
680− (TableHead nullAttr hdrCells)
681− [TableBody nullAttr 0 [] cells']
696+ (TableHead nullAttr headerCells)
697+ [TableBody nullAttr 0 [] bodyCells]
682698 (TableFoot nullAttr [])
683699bodyPartToBlocks (OMathPara e) =
684700 return $ para $ displayMath (writeTeX e)
685701
686−
687702-- replace targets with generated anchors.
688703rewriteLink' :: PandocMonad m => Inline -> DocxContext m Inline
689704rewriteLink' l@(Link attr ils (T.uncons -> Just ('#',target), title)) = do
@@ -719,6 +734,8 @@ bodyToOutput :: PandocMonad m => Body -> DocxContext m (Meta, [Block])
719734bodyToOutput (Body bps) = do
720735 let (metabps, blkbps) = sepBodyParts bps
721736 meta <- bodyPartsToMeta metabps
737+ captions <- catMaybes <$> mapM bodyPartToTableCaption blkbps
738+ modify (\s -> s { docxTableCaptions = captions })
722739 blks <- smushBlocks <$> mapM bodyPartToBlocks blkbps
723740 blks' <- rewriteLinks $ blocksToDefinitions $ blocksToBullets $ toList blks
724741 blks'' <- removeOrphanAnchors blks'
src/Text/Pandoc/Readers/Docx/Combine.hs+5−5
@@ -61,7 +61,7 @@ import Data.List
6161import Data.Bifunctor
6262import Data.Sequence ( ViewL (..), ViewR (..), viewl, viewr, spanr, spanl
6363 , (><), (|>) )
64−import Text.Pandoc.Builder
64+import Text.Pandoc.Builder as B
6565
6666data Modifier a = Modifier (a -> a)
6767 | AttrModifier (Attr -> a -> a) Attr
@@ -116,12 +116,12 @@ ilModifierAndInnards ils = case viewl $ unMany ils of
116116
117117inlinesL :: Inlines -> (Inlines, Inlines)
118118inlinesL ils = case viewl $ unMany ils of
119− (s :< sq) -> (singleton s, Many sq)
119+ (s :< sq) -> (B.singleton s, Many sq)
120120 _ -> (mempty, ils)
121121
122122inlinesR :: Inlines -> (Inlines, Inlines)
123123inlinesR ils = case viewr $ unMany ils of
124− (sq :> s) -> (Many sq, singleton s)
124+ (sq :> s) -> (Many sq, B.singleton s)
125125 _ -> (ils, mempty)
126126
127127combineInlines :: Inlines -> Inlines -> Inlines
@@ -182,7 +182,7 @@ isAttrModifier _ = False
182182
183183smushInlines :: [Inlines] -> Inlines
184184smushInlines xs = combineInlines xs' mempty
185− where xs' = foldl combineInlines mempty xs
185+ where xs' = foldl' combineInlines mempty xs
186186
187187smushBlocks :: [Blocks] -> Blocks
188−smushBlocks xs = foldl combineBlocks mempty xs
188+smushBlocks xs = foldl' combineBlocks mempty xs
src/Text/Pandoc/Readers/Docx/Parse.hs+121−21
@@ -33,7 +33,9 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
3333 , ParStyle
3434 , CharStyle(cStyleData)
3535 , Row(..)
36+ , TblHeader(..)
3637 , Cell(..)
38+ , VMerge(..)
3739 , TrackedChange(..)
3840 , ChangeType(..)
3941 , ChangeInfo(..)
@@ -50,6 +52,7 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
5052 , pHeading
5153 , constructBogusParStyleData
5254 , leftBiasedMergeRunStyle
55+ , rowsToRowspans
5356 ) where
5457import Text.Pandoc.Readers.Docx.Parse.Styles
5558import Codec.Archive.Zip
@@ -225,6 +228,7 @@ defaultParagraphStyle = ParagraphStyle { pStyle = []
225228data BodyPart = Paragraph ParagraphStyle [ParPart]
226229 | ListItem ParagraphStyle T.Text T.Text (Maybe Level) [ParPart]
227230 | Tbl T.Text TblGrid TblLook [Row]
231+ | TblCaption ParagraphStyle [ParPart]
228232 | OMathPara [Exp]
229233 deriving Show
230234
@@ -236,12 +240,61 @@ newtype TblLook = TblLook {firstRowFormatting::Bool}
236240defaultTblLook :: TblLook
237241defaultTblLook = TblLook{firstRowFormatting = False}
238242
239−newtype Row = Row [Cell]
240− deriving Show
243+data Row = Row TblHeader [Cell] deriving Show
244+
245+data TblHeader = HasTblHeader | NoTblHeader deriving (Show, Eq)
241246
242−newtype Cell = Cell [BodyPart]
247+data Cell = Cell GridSpan VMerge [BodyPart]
243248 deriving Show
244249
250+type GridSpan = Integer
251+
252+data VMerge = Continue
253+ -- ^ This cell should be merged with the one above it
254+ | Restart
255+ -- ^ This cell should not be merged with the one above it
256+ deriving (Show, Eq)
257+
258+rowsToRowspans :: [Row] -> [[(Int, Cell)]]
259+rowsToRowspans rows = let
260+ removeMergedCells = fmap (filter (\(_, Cell _ vmerge _) -> vmerge == Restart))
261+ in removeMergedCells (foldr f [] rows)
262+ where
263+ f :: Row -> [[(Int, Cell)]] -> [[(Int, Cell)]]
264+ f (Row _ cells) acc = let
265+ spans = g cells Nothing (listToMaybe acc)
266+ in spans : acc
267+
268+ g ::
269+ -- | The current row
270+ [Cell] ->
271+ -- | Number of columns left below
272+ Maybe Integer ->
273+ -- | (rowspan so far, cell) for the row below this one
274+ Maybe [(Int, Cell)] ->
275+ -- | (rowspan so far, cell) for this row
276+ [(Int, Cell)]
277+ g cells _ Nothing = zip (repeat 1) cells
278+ g cells columnsLeftBelow (Just rowBelow) =
279+ case cells of
280+ [] -> []
281+ thisCell@(Cell thisGridSpan _ _) : restOfRow -> case rowBelow of
282+ [] -> zip (repeat 1) cells
283+ (spanSoFarBelow, Cell gridSpanBelow vmerge _) : _ ->
284+ let spanSoFar = case vmerge of
285+ Restart -> 1
286+ Continue -> 1 + spanSoFarBelow
287+ columnsToDrop = thisGridSpan + (gridSpanBelow - fromMaybe gridSpanBelow columnsLeftBelow)
288+ (newColumnsLeftBelow, restOfRowBelow) = dropColumns columnsToDrop rowBelow
289+ in (spanSoFar, thisCell) : g restOfRow (Just newColumnsLeftBelow) (Just restOfRowBelow)
290+
291+ dropColumns :: Integer -> [(a, Cell)] -> (Integer, [(a, Cell)])
292+ dropColumns n [] = (n, [])
293+ dropColumns n cells@((_, Cell gridSpan _ _) : otherCells) =
294+ if n < gridSpan
295+ then (gridSpan - n, cells)
296+ else dropColumns (n - gridSpan) otherCells
297+
245298leftBiasedMergeRunStyle :: RunStyle -> RunStyle -> RunStyle
246299leftBiasedMergeRunStyle a b = RunStyle
247300 { isBold = isBold a <|> isBold b
@@ -426,20 +479,26 @@ filePathToRelType path docXmlPath =
426479 then Just InDocument
427480 else Nothing
428481
429−relElemToRelationship :: DocumentLocation -> Element -> Maybe Relationship
430−relElemToRelationship relType element | qName (elName element) == "Relationship" =
482+relElemToRelationship :: FilePath -> DocumentLocation -> Element
483+ -> Maybe Relationship
484+relElemToRelationship fp relType element | qName (elName element) == "Relationship" =
431485 do
432486 relId <- findAttr (QName "Id" Nothing Nothing) element
433487 target <- findAttr (QName "Target" Nothing Nothing) element
434− return $ Relationship relType relId target
435−relElemToRelationship _ _ = Nothing
488+ -- target may be relative (media/image1.jpeg) or absolute
489+ -- (/word/media/image1.jpeg); we need to relativize it (see #7374)
490+ let frontOfFp = T.pack $ takeWhile (/= '_') fp
491+ let target' = fromMaybe target $
492+ T.stripPrefix frontOfFp $ T.dropWhile (== '/') target
493+ return $ Relationship relType relId target'
494+relElemToRelationship _ _ _ = Nothing
436495
437496filePathToRelationships :: Archive -> FilePath -> FilePath -> [Relationship]
438497filePathToRelationships ar docXmlPath fp
439498 | Just relType <- filePathToRelType fp docXmlPath
440499 , Just entry <- findEntryByPath fp ar
441500 , Just relElems <- parseXMLFromEntry entry =
442− mapMaybe (relElemToRelationship relType) $ elChildren relElems
501+ mapMaybe (relElemToRelationship fp relType) $ elChildren relElems
443502filePathToRelationships _ _ _ = []
444503
445504archiveToRelationships :: Archive -> FilePath -> [Relationship]
@@ -563,7 +622,7 @@ elemToTblGrid :: NameSpaces -> Element -> D TblGrid
563622elemToTblGrid ns element | isElem ns "w" "tblGrid" element =
564623 let cols = findChildrenByName ns "w" "gridCol" element
565624 in
566− mapD (\e -> maybeToD (findAttrByName ns "w" "val" e >>= stringToInteger))
625+ mapD (\e -> maybeToD (findAttrByName ns "w" "w" e >>= stringToInteger))
567626 cols
568627elemToTblGrid _ _ = throwError WrongElem
569628
@@ -587,14 +646,31 @@ elemToRow ns element | isElem ns "w" "tr" element =
587646 do
588647 let cellElems = findChildrenByName ns "w" "tc" element
589648 cells <- mapD (elemToCell ns) cellElems
590− return $ Row cells
649+ let hasTblHeader = maybe NoTblHeader (const HasTblHeader)
650+ (findChildByName ns "w" "trPr" element
651+ >>= findChildByName ns "w" "tblHeader")
652+ return $ Row hasTblHeader cells
591653elemToRow _ _ = throwError WrongElem
592654
593655elemToCell :: NameSpaces -> Element -> D Cell
594656elemToCell ns element | isElem ns "w" "tc" element =
595657 do
658+ let properties = findChildByName ns "w" "tcPr" element
659+ let gridSpan = properties
660+ >>= findChildByName ns "w" "gridSpan"
661+ >>= findAttrByName ns "w" "val"
662+ >>= stringToInteger
663+ let vMerge = case properties >>= findChildByName ns "w" "vMerge" of
664+ Nothing -> Restart
665+ Just e ->
666+ fromMaybe Continue $ do
667+ s <- findAttrByName ns "w" "val" e
668+ case s of
669+ "continue" -> Just Continue
670+ "restart" -> Just Restart
671+ _ -> Nothing
596672 cellContents <- mapD (elemToBodyPart ns) (elChildren element)
597− return $ Cell cellContents
673+ return $ Cell (fromMaybe 1 gridSpan) vMerge cellContents
598674elemToCell _ _ = throwError WrongElem
599675
600676elemToParIndentation :: NameSpaces -> Element -> Maybe ParIndentation
@@ -626,10 +702,9 @@ pNumInfo = getParStyleField numInfo . pStyle
626702elemToBodyPart :: NameSpaces -> Element -> D BodyPart
627703elemToBodyPart ns element
628704 | isElem ns "w" "p" element
629− , (c:_) <- findChildrenByName ns "m" "oMathPara" element =
630− do
631− expsLst <- eitherToD $ readOMML $ showElement c
632− return $ OMathPara expsLst
705+ , (c:_) <- findChildrenByName ns "m" "oMathPara" element = do
706+ expsLst <- eitherToD $ readOMML $ showElement c
707+ return $ OMathPara expsLst
633708elemToBodyPart ns element
634709 | isElem ns "w" "p" element
635710 , Just (numId, lvl) <- getNumInfo ns element = do
@@ -647,13 +722,31 @@ elemToBodyPart ns element
647722 Nothing | Just (numId, lvl) <- pNumInfo parstyle -> do
648723 levelInfo <- lookupLevel numId lvl <$> asks envNumbering
649724 return $ ListItem parstyle numId lvl levelInfo parparts
650− _ -> return $ Paragraph parstyle parparts
725+ _ -> let
726+ hasCaptionStyle = elem "Caption" (pStyleId <$> pStyle parstyle)
727+
728+ hasSimpleTableField = fromMaybe False $ do
729+ fldSimple <- findChildByName ns "w" "fldSimple" element
730+ instr <- findAttrByName ns "w" "instr" fldSimple
731+ pure ("Table" `elem` T.words instr)
732+
733+ hasComplexTableField = fromMaybe False $ do
734+ instrText <- findElementByName ns "w" "instrText" element
735+ pure ("Table" `elem` T.words (strContent instrText))
736+
737+ in if hasCaptionStyle && (hasSimpleTableField || hasComplexTableField)
738+ then return $ TblCaption parstyle parparts
739+ else return $ Paragraph parstyle parparts
740+
651741elemToBodyPart ns element
652742 | isElem ns "w" "tbl" element = do
653− let caption' = findChildByName ns "w" "tblPr" element
743+ let tblProperties = findChildByName ns "w" "tblPr" element
744+ caption = fromMaybe "" $ tblProperties
654745 >>= findChildByName ns "w" "tblCaption"
655746 >>= findAttrByName ns "w" "val"
656− caption = fromMaybe "" caption'
747+ description = fromMaybe "" $ tblProperties
748+ >>= findChildByName ns "w" "tblDescription"
749+ >>= findAttrByName ns "w" "val"
657750 grid' = case findChildByName ns "w" "tblGrid" element of
658751 Just g -> elemToTblGrid ns g
659752 Nothing -> return []
@@ -666,7 +759,7 @@ elemToBodyPart ns element
666759 grid <- grid'
667760 tblLook <- tblLook'
668761 rows <- mapD (elemToRow ns) (elChildren element)
669− return $ Tbl caption grid tblLook rows
762+ return $ Tbl (caption <> description) grid tblLook rows
670763elemToBodyPart _ _ = throwError WrongElem
671764
672765lookupRelationship :: DocumentLocation -> RelId -> [Relationship] -> Maybe Target
@@ -709,7 +802,8 @@ elemToParPart ns element
709802 case drawing of
710803 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp title alt bs $ elemToExtent drawingElem)
711804 Nothing -> throwError WrongElem
712−-- The below is an attempt to deal with images in deprecated vml format.
805+-- The two cases below are an attempt to deal with images in deprecated vml format.
806+-- Todo: check out title and attr for deprecated format.
713807elemToParPart ns element
714808 | isElem ns "w" "r" element
715809 , Just _ <- findChildByName ns "w" "pict" element =
@@ -717,9 +811,15 @@ elemToParPart ns element
717811 >>= findAttrByName ns "r" "id"
718812 in
719813 case drawing of
720− -- Todo: check out title and attr for deprecated format.
721814 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
722815 Nothing -> throwError WrongElem
816+elemToParPart ns element
817+ | isElem ns "w" "r" element
818+ , Just objectElem <- findChildByName ns "w" "object" element
819+ , Just shapeElem <- findChildByName ns "v" "shape" objectElem
820+ , Just imagedataElem <- findChildByName ns "v" "imagedata" shapeElem
821+ , Just drawingId <- findAttrByName ns "r" "id" imagedataElem
822+ = expandDrawingId drawingId >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
723823-- Chart
724824elemToParPart ns element
725825 | isElem ns "w" "r" element
src/Text/Pandoc/Readers/Docx/Util.hs+7
@@ -19,6 +19,7 @@ module Text.Pandoc.Readers.Docx.Util (
1919 , elemToNameSpaces
2020 , findChildByName
2121 , findChildrenByName
22+ , findElementByName
2223 , findAttrByName
2324 ) where
2425
@@ -56,6 +57,12 @@ findChildrenByName ns pref name el =
5657 let ns' = ns <> elemToNameSpaces el
5758 in findChildren (elemName ns' pref name) el
5859
60+-- | Like 'findChildrenByName', but searches descendants.
61+findElementByName :: NameSpaces -> Text -> Text -> Element -> Maybe Element
62+findElementByName ns pref name el =
63+ let ns' = ns <> elemToNameSpaces el
64+ in findElement (elemName ns' pref name) el
65+
5966findAttrByName :: NameSpaces -> Text -> Text -> Element -> Maybe Text
6067findAttrByName ns pref name el =
6168 let ns' = ns <> elemToNameSpaces el
src/Text/Pandoc/Readers/DokuWiki.hs+9−8
@@ -29,26 +29,27 @@ import Text.Pandoc.Definition
2929import Text.Pandoc.Error (PandocError (PandocParsecError))
3030import Text.Pandoc.Options
3131import Text.Pandoc.Parsing hiding (enclosed, nested)
32−import Text.Pandoc.Shared (crFilter, trim, stringify, tshow)
32+import Text.Pandoc.Shared (trim, stringify, tshow)
3333
3434-- | Read DokuWiki from an input string and return a Pandoc document.
35−readDokuWiki :: PandocMonad m
35+readDokuWiki :: (PandocMonad m, ToSources a)
3636 => ReaderOptions
37− -> Text
37+ -> a
3838 -> m Pandoc
3939readDokuWiki opts s = do
40− let input = crFilter s
41− res <- runParserT parseDokuWiki def {stateOptions = opts } "source" input
40+ let sources = toSources s
41+ res <- runParserT parseDokuWiki def {stateOptions = opts }
42+ (initialSourceName sources) sources
4243 case res of
43− Left e -> throwError $ PandocParsecError input e
44+ Left e -> throwError $ PandocParsecError sources e
4445 Right d -> return d
4546
46−type DWParser = ParserT Text ParserState
47+type DWParser = ParserT Sources ParserState
4748
4849-- * Utility functions
4950
5051-- | Parse end-of-line, which can be either a newline or end-of-file.
51−eol :: Stream s m Char => ParserT s st m ()
52+eol :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
5253eol = void newline <|> eof
5354
5455nested :: PandocMonad m => DWParser m a -> DWParser m a
src/Text/Pandoc/Readers/FB2.hs+6−3
@@ -40,9 +40,9 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad, insertMedia, report)
4040import Text.Pandoc.Error
4141import Text.Pandoc.Logging
4242import Text.Pandoc.Options
43−import Text.Pandoc.Shared (crFilter)
4443import Text.Pandoc.XML.Light
4544import qualified Text.Pandoc.UTF8 as UTF8
45+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
4646
4747type FB2 m = StateT FB2State m
4848
@@ -63,9 +63,12 @@ instance HasMeta FB2State where
6363 setMeta field v s = s {fb2Meta = setMeta field v (fb2Meta s)}
6464 deleteMeta field s = s {fb2Meta = deleteMeta field (fb2Meta s)}
6565
66−readFB2 :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
66+readFB2 :: (PandocMonad m, ToSources a)
67+ => ReaderOptions
68+ -> a
69+ -> m Pandoc
6770readFB2 _ inp =
68− case parseXMLElement $ TL.fromStrict $ crFilter inp of
71+ case parseXMLElement $ TL.fromStrict $ sourcesToText $ toSources inp of
6972 Left msg -> throwError $ PandocXMLError "" msg
7073 Right el -> do
7174 (bs, st) <- runStateT (parseRootElement el) def
src/Text/Pandoc/Readers/HTML.hs+48−28
@@ -32,6 +32,7 @@ import Data.Char (isAlphaNum, isLetter)
3232import Data.Default (Default (..), def)
3333import Data.Foldable (for_)
3434import Data.List.Split (splitWhen)
35+import Data.List (foldl')
3536import qualified Data.Map as M
3637import Data.Maybe (fromMaybe, isJust, isNothing)
3738import Data.Monoid (First (..))
@@ -61,21 +62,21 @@ import Text.Pandoc.Options (
6162 extensionEnabled)
6263import Text.Pandoc.Parsing hiding ((<|>))
6364import Text.Pandoc.Shared (
64− addMetaField, blocksToInlines', crFilter, escapeURI, extractSpaces,
65+ addMetaField, blocksToInlines', escapeURI, extractSpaces,
6566 htmlSpanLikeElements, renderTags', safeRead, tshow)
6667import Text.Pandoc.Walk
6768import Text.Parsec.Error
6869import Text.TeXMath (readMathML, writeTeX)
6970
7071-- | Convert HTML-formatted string to 'Pandoc' document.
71−readHtml :: PandocMonad m
72+readHtml :: (PandocMonad m, ToSources a)
7273 => ReaderOptions -- ^ Reader options
73− -> Text -- ^ String to parse (assumes @'\n'@ line endings)
74+ -> a -- ^ Input to parse
7475 -> m Pandoc
7576readHtml opts inp = do
7677 let tags = stripPrefixes $ canonicalizeTags $
7778 parseTagsOptions parseOptions{ optTagPosition = True }
78− (crFilter inp)
79+ (sourcesToText $ toSources inp)
7980 parseDoc = do
8081 blocks <- fixPlains False . mconcat <$> manyTill block eof
8182 meta <- stateMeta . parserState <$> getState
@@ -204,6 +205,7 @@ block = ((do
204205 | otherwise
205206 -> pDiv
206207 "section" -> pDiv
208+ "header" -> pDiv
207209 "main" -> pDiv
208210 "figure" -> pFigure
209211 "iframe" -> pIframe
@@ -360,7 +362,10 @@ pDefListItem = try $ do
360362 terms <- many1 (try $ skipMany nonItem >> pInTags "dt" inline)
361363 defs <- many1 (try $ skipMany nonItem >> pInTags "dd" block)
362364 skipMany nonItem
363− let term = foldl1 (\x y -> x <> B.linebreak <> y) $ map trimInlines terms
365+ let term = foldl' (\x y -> if null x
366+ then trimInlines y
367+ else x <> B.linebreak <> trimInlines y)
368+ mempty terms
364369 return (term, map (fixPlains True) defs)
365370
366371fixPlains :: Bool -> Blocks -> Blocks
@@ -400,6 +405,7 @@ pLineBlock = try $ do
400405isDivLike :: Text -> Bool
401406isDivLike "div" = True
402407isDivLike "section" = True
408+isDivLike "header" = True
403409isDivLike "main" = True
404410isDivLike _ = False
405411
@@ -495,17 +501,13 @@ pHeader = try $ do
495501 tagOpen (`elem` ["h1","h2","h3","h4","h5","h6"])
496502 (const True)
497503 let attr = toStringAttr attr'
498− let bodyTitle = TagOpen tagtype attr' ~== TagOpen ("h1" :: Text)
499− [("class","title")]
500504 level <- headerLevel tagtype
501505 contents <- trimInlines . mconcat <$> manyTill inline (pCloses tagtype <|> eof)
502506 let ident = fromMaybe "" $ lookup "id" attr
503507 let classes = maybe [] T.words $ lookup "class" attr
504508 let keyvals = [(k,v) | (k,v) <- attr, k /= "class", k /= "id"]
505509 attr'' <- registerHeader (ident, classes, keyvals) contents
506− return $ if bodyTitle
507− then mempty -- skip a representation of the title in the body
508− else B.headerWith attr'' level contents
510+ return $ B.headerWith attr'' level contents
509511
510512pHrule :: PandocMonad m => TagParser m Blocks
511513pHrule = do
@@ -555,7 +557,18 @@ pFigure = try $ do
555557pCodeBlock :: PandocMonad m => TagParser m Blocks
556558pCodeBlock = try $ do
557559 TagOpen _ attr' <- pSatisfy (matchTagOpen "pre" [])
558− let attr = toAttr attr'
560+ -- if the `pre` has no attributes, try if it is followed by a `code`
561+ -- element and use those attributes if possible.
562+ attr <- case attr' of
563+ _:_ -> pure (toAttr attr')
564+ [] -> option nullAttr $ do
565+ TagOpen _ codeAttr <- pSatisfy (matchTagOpen "code" [])
566+ pure $ toAttr
567+ [ (k, v') | (k, v) <- codeAttr
568+ -- strip language from class
569+ , let v' = if k == "class"
570+ then fromMaybe v (T.stripPrefix "language-" v)
571+ else v ]
559572 contents <- manyTill pAny (pCloses "pre" <|> eof)
560573 let rawText = T.concat $ map tagToText contents
561574 -- drop leading newline if any
@@ -826,17 +839,19 @@ pInlinesInTags tagtype f = extractSpaces f <$> pInTags tagtype inline
826839
827840pTagText :: PandocMonad m => TagParser m Inlines
828841pTagText = try $ do
842+ pos <- getPosition
829843 (TagText str) <- pSatisfy isTagText
830844 st <- getState
831845 qu <- ask
832846 parsed <- lift $ lift $
833− flip runReaderT qu $ runParserT (many pTagContents) st "text" str
847+ flip runReaderT qu $ runParserT (many pTagContents) st "text"
848+ (Sources [(pos, str)])
834849 case parsed of
835850 Left _ -> throwError $ PandocParseError $
836851 "Could not parse `" <> str <> "'"
837852 Right result -> return $ mconcat result
838853
839−type InlinesParser m = HTMLParser m Text
854+type InlinesParser m = HTMLParser m Sources
840855
841856pTagContents :: PandocMonad m => InlinesParser m Inlines
842857pTagContents =
@@ -936,13 +951,15 @@ getTagName (TagClose t) = Just t
936951getTagName _ = Nothing
937952
938953isInlineTag :: Tag Text -> Bool
939−isInlineTag t =
940− isCommentTag t ||
941− case getTagName t of
942− Nothing -> False
943− Just "script" -> "math/tex" `T.isPrefixOf` fromAttrib "type" t
944− Just x -> x `Set.notMember` blockTags ||
945− T.take 1 x == "?" -- processing instr.
954+isInlineTag t = isCommentTag t || case t of
955+ TagOpen "script" _ -> "math/tex" `T.isPrefixOf` fromAttrib "type" t
956+ TagClose "script" -> True
957+ TagOpen name _ -> isInlineTagName name
958+ TagClose name -> isInlineTagName name
959+ _ -> False
960+ where isInlineTagName x =
961+ x `Set.notMember` blockTags ||
962+ T.take 1 x == "?" -- processing instr.
946963
947964isBlockTag :: Tag Text -> Bool
948965isBlockTag t = isBlockTagName || isTagComment t
@@ -966,13 +983,14 @@ isCommentTag = tagComment (const True)
966983-- | Matches a stretch of HTML in balanced tags.
967984htmlInBalanced :: Monad m
968985 => (Tag Text -> Bool)
969− -> ParserT Text st m Text
986+ -> ParserT Sources st m Text
970987htmlInBalanced f = try $ do
971988 lookAhead (char '<')
972− inp <- getInput
973− let ts = canonicalizeTags $
974− parseTagsOptions parseOptions{ optTagWarning = True,
975− optTagPosition = True } inp
989+ sources <- getInput
990+ let ts = canonicalizeTags
991+ $ parseTagsOptions parseOptions{ optTagWarning = True,
992+ optTagPosition = True }
993+ $ sourcesToText sources
976994 case ts of
977995 (TagPosition sr sc : t@(TagOpen tn _) : rest) -> do
978996 guard $ f t
@@ -1014,15 +1032,17 @@ hasTagWarning _ = False
10141032-- | Matches a tag meeting a certain condition.
10151033htmlTag :: (HasReaderOptions st, Monad m)
10161034 => (Tag Text -> Bool)
1017− -> ParserT Text st m (Tag Text, Text)
1035+ -> ParserT Sources st m (Tag Text, Text)
10181036htmlTag f = try $ do
10191037 lookAhead (char '<')
10201038 startpos <- getPosition
1021− inp <- getInput
1039+ sources <- getInput
1040+ let inp = sourcesToText sources
10221041 let ts = canonicalizeTags $ parseTagsOptions
10231042 parseOptions{ optTagWarning = False
10241043 , optTagPosition = True }
1025− (inp <> " ") -- add space to ensure that
1044+ (inp <> " ")
1045+ -- add space to ensure that
10261046 -- we get a TagPosition after the tag
10271047 (next, ln, col) <- case ts of
10281048 (TagPosition{} : next : TagPosition ln col : _)
src/Text/Pandoc/Readers/HTML/Table.hs+35−15
@@ -17,6 +17,8 @@ module Text.Pandoc.Readers.HTML.Table (pTable) where
1717
1818import Control.Applicative ((<|>))
1919import Data.Maybe (fromMaybe)
20+import Data.Either (lefts, rights)
21+import Data.List.NonEmpty (nonEmpty)
2022import Data.Text (Text)
2123import Text.HTML.TagSoup
2224import Text.Pandoc.Builder (Blocks)
@@ -32,34 +34,51 @@ import Text.Pandoc.Shared (onlySimpleTableCells, safeRead)
3234import qualified Data.Text as T
3335import qualified Text.Pandoc.Builder as B
3436
35−-- | Parses a @<col>@ element, returning the column's width. Defaults to
36−-- @'ColWidthDefault'@ if the width is not set or cannot be determined.
37−pCol :: PandocMonad m => TagParser m ColWidth
37+-- | Parses a @<col>@ element, returning the column's width.
38+-- An Either value is used: Left i means a "relative length" with
39+-- integral value i (see https://www.w3.org/TR/html4/types.html#h-6.6);
40+-- Right w means a regular width. Defaults to @'Right ColWidthDefault'@
41+-- if the width is not set or cannot be determined.
42+pCol :: PandocMonad m => TagParser m (Either Int ColWidth)
3843pCol = try $ do
3944 TagOpen _ attribs' <- pSatisfy (matchTagOpen "col" [])
4045 let attribs = toStringAttr attribs'
4146 skipMany pBlank
4247 optional $ pSatisfy (matchTagClose "col")
4348 skipMany pBlank
44− let width = case lookup "width" attribs of
49+ return $ case lookup "width" attribs of
4550 Nothing -> case lookup "style" attribs of
4651 Just (T.stripPrefix "width:" -> Just xs) | T.any (== '%') xs ->
47− fromMaybe 0.0 $ safeRead (T.filter
48− (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
49− _ -> 0.0
52+ maybe (Right ColWidthDefault) (Right . ColWidth . (/ 100.0))
53+ $ safeRead (T.filter
54+ (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
55+ _ -> Right ColWidthDefault
56+ Just (T.unsnoc -> Just (xs, '*')) ->
57+ maybe (Left 1) Left $ safeRead xs
5058 Just (T.unsnoc -> Just (xs, '%')) ->
51− fromMaybe 0.0 $ safeRead xs
52− _ -> 0.0
53− if width > 0.0
54− then return $ ColWidth $ width / 100.0
55− else return ColWidthDefault
59+ maybe (Right ColWidthDefault)
60+ (Right . ColWidth . (/ 100.0)) $ safeRead xs
61+ _ -> Right ColWidthDefault
5662
57−pColgroup :: PandocMonad m => TagParser m [ColWidth]
63+pColgroup :: PandocMonad m => TagParser m [Either Int ColWidth]
5864pColgroup = try $ do
5965 pSatisfy (matchTagOpen "colgroup" [])
6066 skipMany pBlank
6167 manyTill pCol (pCloses "colgroup" <|> eof) <* skipMany pBlank
6268
69+resolveRelativeLengths :: [Either Int ColWidth] -> [ColWidth]
70+resolveRelativeLengths ws =
71+ let remaining = 1 - sum (map getColWidth $ rights ws)
72+ relatives = sum $ lefts ws
73+ relUnit = remaining / fromIntegral relatives
74+ toColWidth (Right x) = x
75+ toColWidth (Left i) = ColWidth (fromIntegral i * relUnit)
76+ in map toColWidth ws
77+
78+getColWidth :: ColWidth -> Double
79+getColWidth ColWidthDefault = 0
80+getColWidth (ColWidth w) = w
81+
6382data CellType
6483 = HeaderCell
6584 | BodyCell
@@ -181,7 +200,8 @@ pTable :: PandocMonad m
181200pTable block = try $ do
182201 TagOpen _ attribs <- pSatisfy (matchTagOpen "table" []) <* skipMany pBlank
183202 caption <- option mempty $ pInTags "caption" block <* skipMany pBlank
184− widths <- ((mconcat <$> many1 pColgroup) <|> many pCol) <* skipMany pBlank
203+ widths <- resolveRelativeLengths <$>
204+ ((mconcat <$> many1 pColgroup) <|> many pCol) <* skipMany pBlank
185205 thead <- pTableHead block <* skipMany pBlank
186206 topfoot <- optionMaybe (pTableFoot block) <* skipMany pBlank
187207 tbodies <- many (pTableBody block) <* skipMany pBlank
@@ -216,7 +236,7 @@ normalize widths head' bodies foot = do
216236 let rows = headRows head' <> concatMap bodyRows bodies <> footRows foot
217237 let cellWidth (Cell _ _ _ (ColSpan cs) _) = cs
218238 let rowLength = foldr (\cell acc -> cellWidth cell + acc) 0 . rowCells
219− let ncols = maximum (map rowLength rows)
239+ let ncols = maybe 0 maximum $ nonEmpty $ map rowLength rows
220240 let tblType = tableType (map rowCells rows)
221241 -- fail on empty table
222242 if null rows
src/Text/Pandoc/Readers/Haddock.hs+11−7
@@ -17,8 +17,9 @@ module Text.Pandoc.Readers.Haddock
1717
1818import Control.Monad.Except (throwError)
1919import Data.List (intersperse)
20+import Data.List.NonEmpty (nonEmpty)
2021import Data.Maybe (fromMaybe)
21−import Data.Text (Text, unpack)
22+import Data.Text (unpack)
2223import qualified Data.Text as T
2324import Documentation.Haddock.Parser
2425import Documentation.Haddock.Types as H
@@ -28,15 +29,17 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
2829import Text.Pandoc.Definition
2930import Text.Pandoc.Error
3031import Text.Pandoc.Options
31−import Text.Pandoc.Shared (crFilter, splitTextBy, trim)
32+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
33+import Text.Pandoc.Shared (splitTextBy, trim)
3234
3335
3436-- | Parse Haddock markup and return a 'Pandoc' document.
35−readHaddock :: PandocMonad m
37+readHaddock :: (PandocMonad m, ToSources a)
3638 => ReaderOptions
37− -> Text
39+ -> a
3840 -> m Pandoc
39−readHaddock opts s = case readHaddockEither opts (unpack (crFilter s)) of
41+readHaddock opts s = case readHaddockEither opts
42+ (unpack . sourcesToText . toSources $ s) of
4043 Right result -> return result
4144 Left e -> throwError e
4245
@@ -92,7 +95,7 @@ docHToBlocks d' =
9295 then ([], map toCells bodyRows)
9396 else (toCells (head headerRows),
9497 map toCells (tail headerRows ++ bodyRows))
95− colspecs = replicate (maximum (map length body))
98+ colspecs = replicate (maybe 0 maximum (nonEmpty (map length body)))
9699 (AlignDefault, ColWidthDefault)
97100 in B.table B.emptyCaption
98101 colspecs
@@ -128,7 +131,8 @@ docHToInlines isCode d' =
128131 DocIdentifier s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
129132 _ -> mempty
130133 DocIdentifierUnchecked s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
131− DocModule s -> B.codeWith ("",["haskell","module"],[]) $ T.pack s
134+ DocModule s -> B.codeWith ("",["haskell","module"],[]) $
135+ T.pack (modLinkName s)
132136 DocWarning _ -> mempty -- TODO
133137 DocEmphasis d -> B.emph (docHToInlines isCode d)
134138 DocMonospaced (DocString s) -> B.code $ T.pack s
src/Text/Pandoc/Readers/Ipynb.hs+5−3
@@ -39,10 +39,12 @@ import Data.Aeson as Aeson
3939import Control.Monad.Except (throwError)
4040import Text.Pandoc.Readers.Markdown (readMarkdown)
4141import qualified Text.Pandoc.UTF8 as UTF8
42+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
4243
43−readIpynb :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
44−readIpynb opts t = do
45− let src = BL.fromStrict (TE.encodeUtf8 t)
44+readIpynb :: (PandocMonad m, ToSources a)
45+ => ReaderOptions -> a -> m Pandoc
46+readIpynb opts x = do
47+ let src = BL.fromStrict . TE.encodeUtf8 . sourcesToText $ toSources x
4648 case eitherDecode src of
4749 Right (notebook4 :: Notebook NbV4) -> notebookToPandoc opts notebook4
4850 Left _ ->
src/Text/Pandoc/Readers/JATS.hs+13−6
@@ -29,11 +29,12 @@ import Text.HTML.TagSoup.Entity (lookupEntity)
2929import Text.Pandoc.Builder
3030import Text.Pandoc.Class.PandocMonad (PandocMonad)
3131import Text.Pandoc.Options
32−import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
32+import Text.Pandoc.Shared (safeRead, extractSpaces)
3333import Text.TeXMath (readMathML, writeTeX)
3434import Text.Pandoc.XML.Light
3535import qualified Data.Set as S (fromList, member)
3636import Data.Set ((\\))
37+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3738
3839type JATS m = StateT JATSState m
3940
@@ -52,10 +53,14 @@ instance Default JATSState where
5253 , jatsContent = [] }
5354
5455
55−readJATS :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
56+readJATS :: (PandocMonad m, ToSources a)
57+ => ReaderOptions
58+ -> a
59+ -> m Pandoc
5660readJATS _ inp = do
61+ let sources = toSources inp
5762 tree <- either (throwError . PandocXMLError "") return $
58− parseXMLContents (TL.fromStrict $ crFilter inp)
63+ parseXMLContents (TL.fromStrict . sourcesToText $ sources)
5964 (bs, st') <- flip runStateT (def{ jatsContent = tree }) $ mapM parseBlock tree
6065 return $ Pandoc (jatsMeta st') (toList . mconcat $ bs)
6166
@@ -491,7 +496,8 @@ parseInline (Elem e) =
491496
492497 "disp-formula" -> formula displayMath
493498 "inline-formula" -> formula math
494− "math" | qPrefix (elName e) == Just "mml" -> return . math $ mathML e
499+ "math" | qURI (elName e) == Just "http://www.w3.org/1998/Math/MathML"
500+ -> return . math $ mathML e
495501 "tex-math" -> return . math $ textContent e
496502
497503 "email" -> return $ link ("mailto:" <> textContent e) ""
@@ -514,8 +520,9 @@ parseInline (Elem e) =
514520 filterChildren isMathML whereToLook
515521 return . mconcat . take 1 . map constructor $ texMaths ++ mathMLs
516522
517− isMathML x = qName (elName x) == "math" &&
518− qPrefix (elName x) == Just "mml"
523+ isMathML x = qName (elName x) == "math" &&
524+ qURI (elName x) ==
525+ Just "http://www.w3.org/1998/Math/MathML"
519526 removePrefix elname = elname { qPrefix = Nothing }
520527 codeWithLang = do
521528 let classes' = case attrValue "language" e of
src/Text/Pandoc/Readers/Jira.hs+13−9
@@ -20,18 +20,20 @@ import Text.Pandoc.Builder hiding (cell)
2020import Text.Pandoc.Error (PandocError (PandocParseError))
2121import Text.Pandoc.Options (ReaderOptions)
2222import Text.Pandoc.Shared (stringify)
23−
23+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2424import qualified Text.Jira.Markup as Jira
2525
2626-- | Read Jira wiki markup.
27−readJira :: PandocMonad m
27+readJira :: (PandocMonad m, ToSources a)
2828 => ReaderOptions
29− -> Text
29+ -> a
3030 -> m Pandoc
31−readJira _opts s = case parse s of
32− Right d -> return $ jiraToPandoc d
33− Left e -> throwError . PandocParseError $
34− "Jira parse error" `append` pack (show e)
31+readJira _opts inp = do
32+ let sources = toSources inp
33+ case parse (sourcesToText sources) of
34+ Right d -> return $ jiraToPandoc d
35+ Left e -> throwError . PandocParseError $
36+ "Jira parse error" `append` pack (show e)
3537
3638jiraToPandoc :: Jira.Doc -> Pandoc
3739jiraToPandoc (Jira.Doc blks) = doc $ foldMap jiraToPandocBlocks blks
@@ -71,10 +73,10 @@ toPandocCodeBlocks langMay params txt =
7173 Nothing -> []
7274 in codeBlockWith ("", classes, map paramToPair params) txt
7375
74−-- | Create a pandoc @'Div'@
76+-- | Create a pandoc @'Div'@ from a panel.
7577toPandocDiv :: [Jira.Parameter] -> [Jira.Block] -> Blocks
7678toPandocDiv params =
77− divWith ("", [], map paramToPair params) . foldMap jiraToPandocBlocks
79+ divWith ("", ["panel"], map paramToPair params) . foldMap jiraToPandocBlocks
7880
7981paramToPair :: Jira.Parameter -> (Text, Text)
8082paramToPair (Jira.Parameter key value) = (key, value)
@@ -170,6 +172,8 @@ jiraLinkToPandoc linkType alias url =
170172 Jira.Email -> link ("mailto:" <> url') "" alias'
171173 Jira.Attachment -> linkWith ("", ["attachment"], []) url' "" alias'
172174 Jira.User -> linkWith ("", ["user-account"], []) url' "" alias'
175+ Jira.SmartCard -> linkWith ("", ["smart-card"], []) url' "" alias'
176+ Jira.SmartLink -> linkWith ("", ["smart-link"], []) url' "" alias'
173177
174178-- | Get unicode representation of a Jira icon.
175179iconUnicode :: Jira.Icon -> Text
src/Text/Pandoc/Readers/LaTeX.hs+35−38
@@ -1,4 +1,3 @@
1−{-# LANGUAGE CPP #-}
21{-# LANGUAGE OverloadedStrings #-}
32{-# LANGUAGE PatternGuards #-}
43{-# LANGUAGE ScopedTypeVariables #-}
@@ -25,7 +24,7 @@ module Text.Pandoc.Readers.LaTeX ( readLaTeX,
2524import Control.Applicative (many, optional, (<|>))
2625import Control.Monad
2726import Control.Monad.Except (throwError)
28−import Data.Char (isDigit, isLetter, toUpper, chr)
27+import Data.Char (isDigit, isLetter, isAlphaNum, toUpper, chr)
2928import Data.Default
3029import Data.List (intercalate)
3130import qualified Data.Map as M
@@ -33,8 +32,9 @@ import Data.Maybe (fromMaybe, maybeToList)
3332import qualified Data.Set as Set
3433import Data.Text (Text)
3534import qualified Data.Text as T
35+import Skylighting (defaultSyntaxMap)
3636import System.FilePath (addExtension, replaceExtension, takeExtension)
37−import Text.Pandoc.BCP47 (renderLang)
37+import Text.Collate.Lang (renderLang)
3838import Text.Pandoc.Builder as B
3939import Text.Pandoc.Class.PandocPure (PandocPure)
4040import Text.Pandoc.Class.PandocMonad (PandocMonad (..), getResourcePath,
@@ -69,7 +69,7 @@ import Text.Pandoc.Readers.LaTeX.Inline (acronymCommands, refCommands,
6969 listingsLanguage)
7070import Text.Pandoc.Shared
7171import Text.Pandoc.Walk
72−import Safe
72+import Data.List.NonEmpty (nonEmpty)
7373
7474-- for debugging:
7575-- import Text.Pandoc.Extensions (getDefaultExtensions)
@@ -77,16 +77,17 @@ import Safe
7777-- import Debug.Trace (traceShowId)
7878
7979-- | Parse LaTeX from string and return 'Pandoc' document.
80−readLaTeX :: PandocMonad m
80+readLaTeX :: (PandocMonad m, ToSources a)
8181 => ReaderOptions -- ^ Reader options
82− -> Text -- ^ String to parse (assumes @'\n'@ line endings)
82+ -> a -- ^ Input to parse
8383 -> m Pandoc
8484readLaTeX opts ltx = do
85+ let sources = toSources ltx
8586 parsed <- runParserT parseLaTeX def{ sOptions = opts } "source"
86− (tokenize "source" (crFilter ltx))
87+ (tokenizeSources sources)
8788 case parsed of
8889 Right result -> return result
89− Left e -> throwError $ PandocParsecError ltx e
90+ Left e -> throwError $ PandocParsecError sources e
9091
9192parseLaTeX :: PandocMonad m => LP m Pandoc
9293parseLaTeX = do
@@ -97,11 +98,7 @@ parseLaTeX = do
9798 let doc' = doc bs
9899 let headerLevel (Header n _ _) = [n]
99100 headerLevel _ = []
100−#if MIN_VERSION_safe(0,3,18)
101− let bottomLevel = minimumBound 1 $ query headerLevel doc'
102−#else
103− let bottomLevel = minimumDef 1 $ query headerLevel doc'
104−#endif
101+ let bottomLevel = maybe 1 minimum $ nonEmpty $ query headerLevel doc'
105102 let adjustHeaders m (Header n attr ils) = Header (n+m) attr ils
106103 adjustHeaders _ x = x
107104 let (Pandoc _ bs') =
@@ -136,11 +133,11 @@ resolveRefs _ x = x
136133
137134
138135rawLaTeXBlock :: (PandocMonad m, HasMacros s, HasReaderOptions s)
139− => ParserT Text s m Text
136+ => ParserT Sources s m Text
140137rawLaTeXBlock = do
141138 lookAhead (try (char '\\' >> letter))
142139 inp <- getInput
143− let toks = tokenize "source" inp
140+ let toks = tokenizeSources inp
144141 snd <$> (rawLaTeXParser toks False (macroDef (const mempty)) blocks
145142 <|> rawLaTeXParser toks True
146143 (do choice (map controlSeq
@@ -167,11 +164,11 @@ beginOrEndCommand = try $ do
167164 (txt <> untokenize rawargs)
168165
169166rawLaTeXInline :: (PandocMonad m, HasMacros s, HasReaderOptions s)
170− => ParserT Text s m Text
167+ => ParserT Sources s m Text
171168rawLaTeXInline = do
172169 lookAhead (try (char '\\' >> letter))
173170 inp <- getInput
174− let toks = tokenize "source" inp
171+ let toks = tokenizeSources inp
175172 raw <- snd <$>
176173 ( rawLaTeXParser toks True
177174 (mempty <$ (controlSeq "input" >> skipMany rawopt >> braced))
@@ -182,11 +179,11 @@ rawLaTeXInline = do
182179 finalbraces <- mconcat <$> many (try (string "{}")) -- see #5439
183180 return $ raw <> T.pack finalbraces
184181
185−inlineCommand :: PandocMonad m => ParserT Text ParserState m Inlines
182+inlineCommand :: PandocMonad m => ParserT Sources ParserState m Inlines
186183inlineCommand = do
187184 lookAhead (try (char '\\' >> letter))
188185 inp <- getInput
189− let toks = tokenize "source" inp
186+ let toks = tokenizeSources inp
190187 fst <$> rawLaTeXParser toks True (inlineEnvironment <|> inlineCommand')
191188 inlines
192189
@@ -235,16 +232,6 @@ mkImage options (T.unpack -> src) = do
235232 _ -> return src
236233 return $ imageWith attr (T.pack src') "" alt
237234
238−doxspace :: PandocMonad m => LP m Inlines
239−doxspace =
240− (space <$ lookAhead (satisfyTok startsWithLetter)) <|> return mempty
241− where startsWithLetter (Tok _ Word t) =
242− case T.uncons t of
243− Just (c, _) | isLetter c -> True
244− _ -> False
245− startsWithLetter _ = False
246−
247−
248235removeDoubleQuotes :: Text -> Text
249236removeDoubleQuotes t =
250237 Data.Maybe.fromMaybe t $ T.stripPrefix "\"" t >>= T.stripSuffix "\""
@@ -313,7 +300,9 @@ inlineCommand' :: PandocMonad m => LP m Inlines
313300inlineCommand' = try $ do
314301 Tok _ (CtrlSeq name) cmd <- anyControlSeq
315302 guard $ name /= "begin" && name /= "end" && name /= "and"
316− star <- option "" ("*" <$ symbol '*' <* sp)
303+ star <- if T.all isAlphaNum name
304+ then option "" ("*" <$ symbol '*' <* sp)
305+ else pure ""
317306 overlay <- option "" overlaySpecification
318307 let name' = name <> star <> overlay
319308 let names = ordNub [name', name] -- check non-starred as fallback
@@ -398,17 +387,20 @@ inlineCommands = M.unions
398387 , ("footnote", skipopts >> note <$> grouped block)
399388 , ("passthrough", tok) -- \passthrough macro used by latex writer
400389 -- for listings
390+ , ("includegraphics", do options <- option [] keyvals
391+ src <- braced
392+ mkImage options .
393+ unescapeURL .
394+ removeDoubleQuotes $ untokenize src)
395+ -- hyperref
401396 , ("url", (\url -> link url "" (str url)) . unescapeURL . untokenize <$>
402397 bracedUrl)
403398 , ("nolinkurl", code . unescapeURL . untokenize <$> bracedUrl)
404399 , ("href", do url <- bracedUrl
405400 sp
406401 link (unescapeURL $ untokenize url) "" <$> tok)
407− , ("includegraphics", do options <- option [] keyvals
408− src <- braced
409− mkImage options . unescapeURL . removeDoubleQuotes $
410− untokenize src)
411402 , ("hyperlink", hyperlink)
403+ , ("hyperref", hyperref)
412404 , ("hypertarget", hypertargetInline)
413405 -- hyphenat
414406 , ("nohyphens", tok)
@@ -417,8 +409,6 @@ inlineCommands = M.unions
417409 -- LaTeX colors
418410 , ("textcolor", coloredInline "color")
419411 , ("colorbox", coloredInline "background-color")
420− -- xspace
421− , ("xspace", doxspace)
422412 -- etoolbox
423413 , ("ifstrequal", ifstrequal)
424414 , ("newtoggle", braced >>= newToggle)
@@ -463,6 +453,12 @@ hyperlink = try $ do
463453 lab <- tok
464454 return $ link ("#" <> src) "" lab
465455
456+hyperref :: PandocMonad m => LP m Inlines
457+hyperref = try $ do
458+ url <- (("#" <>) . untokenize <$> try (sp *> bracketedToks <* sp))
459+ <|> untokenize <$> (bracedUrl <* bracedUrl <* bracedUrl)
460+ link url "" <$> tok
461+
466462hypertargetBlock :: PandocMonad m => LP m Blocks
467463hypertargetBlock = try $ do
468464 ref <- untokenize <$> braced
@@ -636,7 +632,7 @@ opt = do
636632 parsed <- runParserT (mconcat <$> many inline) st "bracketed option" toks
637633 case parsed of
638634 Right result -> return result
639− Left e -> throwError $ PandocParsecError (untokenize toks) e
635+ Left e -> throwError $ PandocParsecError (toSources toks) e
640636
641637-- block elements:
642638
@@ -1166,7 +1162,8 @@ inputListing = do
11661162 let (ident,classes,kvs) = parseListingsOptions options
11671163 let classes' =
11681164 (case listingsLanguage options of
1169− Nothing -> (take 1 (languagesByExtension (T.pack $ takeExtension $ T.unpack f)) <>)
1165+ Nothing -> (take 1 (languagesByExtension defaultSyntaxMap
1166+ (T.pack $ takeExtension $ T.unpack f)) <>)
11701167 Just _ -> id) classes
11711168 let firstline = fromMaybe 1 $ lookup "firstline" options >>= safeRead
11721169 let lastline = fromMaybe (length codeLines) $
src/Text/Pandoc/Readers/LaTeX/Citation.hs+1−1
@@ -120,7 +120,7 @@ simpleCiteArgs inline = try $ do
120120 runParserT (mconcat <$> many inline) st "bracketed option" toks
121121 case parsed of
122122 Right result -> return result
123− Left e -> throwError $ PandocParsecError (untokenize toks) e
123+ Left e -> throwError $ PandocParsecError (toSources toks) e
124124
125125
126126
src/Text/Pandoc/Readers/LaTeX/Lang.hs+122−119
@@ -23,7 +23,7 @@ import qualified Data.Map as M
2323import Data.Text (Text)
2424import qualified Data.Text as T
2525import Text.Pandoc.Shared (extractSpaces)
26−import Text.Pandoc.BCP47 (Lang(..), renderLang)
26+import Text.Collate.Lang (Lang(..), renderLang)
2727import Text.Pandoc.Class (PandocMonad(..), setTranslations)
2828import Text.Pandoc.Readers.LaTeX.Parsing
2929import Text.Pandoc.Parsing (updateState, option, getState, QuoteContext(..),
@@ -99,133 +99,136 @@ setDefaultLanguage = do
9999polyglossiaLangToBCP47 :: M.Map T.Text (T.Text -> Lang)
100100polyglossiaLangToBCP47 = M.fromList
101101 [ ("arabic", \o -> case T.filter (/=' ') o of
102− "locale=algeria" -> Lang "ar" "" "DZ" []
103− "locale=mashriq" -> Lang "ar" "" "SY" []
104− "locale=libya" -> Lang "ar" "" "LY" []
105− "locale=morocco" -> Lang "ar" "" "MA" []
106− "locale=mauritania" -> Lang "ar" "" "MR" []
107− "locale=tunisia" -> Lang "ar" "" "TN" []
108− _ -> Lang "ar" "" "" [])
102+ "locale=algeria" -> Lang "ar" Nothing (Just "DZ") [] [] []
103+ "locale=mashriq" -> Lang "ar" Nothing (Just "SY") [] [] []
104+ "locale=libya" -> Lang "ar" Nothing (Just "LY") [] [] []
105+ "locale=morocco" -> Lang "ar" Nothing (Just "MA") [] [] []
106+ "locale=mauritania" -> Lang "ar" Nothing (Just "MR") [] [] []
107+ "locale=tunisia" -> Lang "ar" Nothing (Just "TN") [] [] []
108+ _ -> Lang "ar" Nothing (Just "") [] [] [])
109109 , ("german", \o -> case T.filter (/=' ') o of
110− "spelling=old" -> Lang "de" "" "DE" ["1901"]
110+ "spelling=old" -> Lang "de" Nothing (Just "DE") ["1901"] [] []
111111 "variant=austrian,spelling=old"
112− -> Lang "de" "" "AT" ["1901"]
113− "variant=austrian" -> Lang "de" "" "AT" []
112+ -> Lang "de" Nothing (Just "AT") ["1901"] [] []
113+ "variant=austrian" -> Lang "de" Nothing (Just "AT") [] [] []
114114 "variant=swiss,spelling=old"
115− -> Lang "de" "" "CH" ["1901"]
116− "variant=swiss" -> Lang "de" "" "CH" []
117− _ -> Lang "de" "" "" [])
118− , ("lsorbian", \_ -> Lang "dsb" "" "" [])
115+ -> Lang "de" Nothing (Just "CH") ["1901"] [] []
116+ "variant=swiss" -> Lang "de" Nothing (Just "CH") [] [] []
117+ _ -> Lang "de" Nothing Nothing [] [] [])
118+ , ("lsorbian", \_ -> Lang "dsb" Nothing Nothing [] [] [])
119119 , ("greek", \o -> case T.filter (/=' ') o of
120− "variant=poly" -> Lang "el" "" "polyton" []
121− "variant=ancient" -> Lang "grc" "" "" []
122− _ -> Lang "el" "" "" [])
120+ "variant=poly" -> Lang "el" Nothing (Just "polyton") [] [] []
121+ "variant=ancient" -> Lang "grc" Nothing Nothing [] [] []
122+ _ -> Lang "el" Nothing Nothing [] [] [])
123123 , ("english", \o -> case T.filter (/=' ') o of
124− "variant=australian" -> Lang "en" "" "AU" []
125− "variant=canadian" -> Lang "en" "" "CA" []
126− "variant=british" -> Lang "en" "" "GB" []
127− "variant=newzealand" -> Lang "en" "" "NZ" []
128− "variant=american" -> Lang "en" "" "US" []
129− _ -> Lang "en" "" "" [])
130− , ("usorbian", \_ -> Lang "hsb" "" "" [])
124+ "variant=australian" -> Lang "en" Nothing (Just "AU") [] [] []
125+ "variant=canadian" -> Lang "en" Nothing (Just "CA") [] [] []
126+ "variant=british" -> Lang "en" Nothing (Just "GB") [] [] []
127+ "variant=newzealand" -> Lang "en" Nothing (Just "NZ") [] [] []
128+ "variant=american" -> Lang "en" Nothing (Just "US") [] [] []
129+ _ -> Lang "en" Nothing (Just "") [] [] [])
130+ , ("usorbian", \_ -> Lang "hsb" Nothing Nothing [] [] [])
131131 , ("latin", \o -> case T.filter (/=' ') o of
132− "variant=classic" -> Lang "la" "" "" ["x-classic"]
133− _ -> Lang "la" "" "" [])
134− , ("slovenian", \_ -> Lang "sl" "" "" [])
135− , ("serbianc", \_ -> Lang "sr" "cyrl" "" [])
136− , ("pinyin", \_ -> Lang "zh" "Latn" "" ["pinyin"])
137− , ("afrikaans", \_ -> Lang "af" "" "" [])
138− , ("amharic", \_ -> Lang "am" "" "" [])
139− , ("assamese", \_ -> Lang "as" "" "" [])
140− , ("asturian", \_ -> Lang "ast" "" "" [])
141− , ("bulgarian", \_ -> Lang "bg" "" "" [])
142− , ("bengali", \_ -> Lang "bn" "" "" [])
143− , ("tibetan", \_ -> Lang "bo" "" "" [])
144− , ("breton", \_ -> Lang "br" "" "" [])
145− , ("catalan", \_ -> Lang "ca" "" "" [])
146− , ("welsh", \_ -> Lang "cy" "" "" [])
147− , ("czech", \_ -> Lang "cs" "" "" [])
148− , ("coptic", \_ -> Lang "cop" "" "" [])
149− , ("danish", \_ -> Lang "da" "" "" [])
150− , ("divehi", \_ -> Lang "dv" "" "" [])
151− , ("esperanto", \_ -> Lang "eo" "" "" [])
152− , ("spanish", \_ -> Lang "es" "" "" [])
153− , ("estonian", \_ -> Lang "et" "" "" [])
154− , ("basque", \_ -> Lang "eu" "" "" [])
155− , ("farsi", \_ -> Lang "fa" "" "" [])
156− , ("finnish", \_ -> Lang "fi" "" "" [])
157− , ("french", \_ -> Lang "fr" "" "" [])
158− , ("friulan", \_ -> Lang "fur" "" "" [])
159− , ("irish", \_ -> Lang "ga" "" "" [])
160− , ("scottish", \_ -> Lang "gd" "" "" [])
161− , ("ethiopic", \_ -> Lang "gez" "" "" [])
162− , ("galician", \_ -> Lang "gl" "" "" [])
163− , ("hebrew", \_ -> Lang "he" "" "" [])
164− , ("hindi", \_ -> Lang "hi" "" "" [])
165− , ("croatian", \_ -> Lang "hr" "" "" [])
166− , ("magyar", \_ -> Lang "hu" "" "" [])
167− , ("armenian", \_ -> Lang "hy" "" "" [])
168− , ("interlingua", \_ -> Lang "ia" "" "" [])
169− , ("indonesian", \_ -> Lang "id" "" "" [])
170− , ("icelandic", \_ -> Lang "is" "" "" [])
171− , ("italian", \_ -> Lang "it" "" "" [])
172− , ("japanese", \_ -> Lang "jp" "" "" [])
173− , ("khmer", \_ -> Lang "km" "" "" [])
174− , ("kurmanji", \_ -> Lang "kmr" "" "" [])
175− , ("kannada", \_ -> Lang "kn" "" "" [])
176− , ("korean", \_ -> Lang "ko" "" "" [])
177− , ("lao", \_ -> Lang "lo" "" "" [])
178− , ("lithuanian", \_ -> Lang "lt" "" "" [])
179− , ("latvian", \_ -> Lang "lv" "" "" [])
180− , ("malayalam", \_ -> Lang "ml" "" "" [])
181− , ("mongolian", \_ -> Lang "mn" "" "" [])
182− , ("marathi", \_ -> Lang "mr" "" "" [])
183− , ("dutch", \_ -> Lang "nl" "" "" [])
184− , ("nynorsk", \_ -> Lang "nn" "" "" [])
185− , ("norsk", \_ -> Lang "no" "" "" [])
186− , ("nko", \_ -> Lang "nqo" "" "" [])
187− , ("occitan", \_ -> Lang "oc" "" "" [])
188− , ("panjabi", \_ -> Lang "pa" "" "" [])
189− , ("polish", \_ -> Lang "pl" "" "" [])
190− , ("piedmontese", \_ -> Lang "pms" "" "" [])
191− , ("portuguese", \_ -> Lang "pt" "" "" [])
192− , ("romansh", \_ -> Lang "rm" "" "" [])
193− , ("romanian", \_ -> Lang "ro" "" "" [])
194− , ("russian", \_ -> Lang "ru" "" "" [])
195− , ("sanskrit", \_ -> Lang "sa" "" "" [])
196− , ("samin", \_ -> Lang "se" "" "" [])
197− , ("slovak", \_ -> Lang "sk" "" "" [])
198− , ("albanian", \_ -> Lang "sq" "" "" [])
199− , ("serbian", \_ -> Lang "sr" "" "" [])
200− , ("swedish", \_ -> Lang "sv" "" "" [])
201− , ("syriac", \_ -> Lang "syr" "" "" [])
202− , ("tamil", \_ -> Lang "ta" "" "" [])
203− , ("telugu", \_ -> Lang "te" "" "" [])
204− , ("thai", \_ -> Lang "th" "" "" [])
205− , ("turkmen", \_ -> Lang "tk" "" "" [])
206− , ("turkish", \_ -> Lang "tr" "" "" [])
207− , ("ukrainian", \_ -> Lang "uk" "" "" [])
208− , ("urdu", \_ -> Lang "ur" "" "" [])
209− , ("vietnamese", \_ -> Lang "vi" "" "" [])
132+ "variant=classic" -> Lang "la" Nothing Nothing ["x-classic"] [] []
133+ _ -> Lang "la" Nothing Nothing [] [] [])
134+ , ("slovenian", \_ -> Lang "sl" Nothing Nothing [] [] [])
135+ , ("serbianc", \_ -> Lang "sr" (Just "Cyrl") Nothing [] [] [])
136+ , ("pinyin", \_ -> Lang "zh" (Just "Latn") Nothing ["pinyin"] [] [])
137+ , ("afrikaans", \_ -> simpleLang "af")
138+ , ("amharic", \_ -> simpleLang "am")
139+ , ("assamese", \_ -> simpleLang "as")
140+ , ("asturian", \_ -> simpleLang "ast")
141+ , ("bulgarian", \_ -> simpleLang "bg")
142+ , ("bengali", \_ -> simpleLang "bn")
143+ , ("tibetan", \_ -> simpleLang "bo")
144+ , ("breton", \_ -> simpleLang "br")
145+ , ("catalan", \_ -> simpleLang "ca")
146+ , ("welsh", \_ -> simpleLang "cy")
147+ , ("czech", \_ -> simpleLang "cs")
148+ , ("coptic", \_ -> simpleLang "cop")
149+ , ("danish", \_ -> simpleLang "da")
150+ , ("divehi", \_ -> simpleLang "dv")
151+ , ("esperanto", \_ -> simpleLang "eo")
152+ , ("spanish", \_ -> simpleLang "es")
153+ , ("estonian", \_ -> simpleLang "et")
154+ , ("basque", \_ -> simpleLang "eu")
155+ , ("farsi", \_ -> simpleLang "fa")
156+ , ("finnish", \_ -> simpleLang "fi")
157+ , ("french", \_ -> simpleLang "fr")
158+ , ("friulan", \_ -> simpleLang "fur")
159+ , ("irish", \_ -> simpleLang "ga")
160+ , ("scottish", \_ -> simpleLang "gd")
161+ , ("ethiopic", \_ -> simpleLang "gez")
162+ , ("galician", \_ -> simpleLang "gl")
163+ , ("hebrew", \_ -> simpleLang "he")
164+ , ("hindi", \_ -> simpleLang "hi")
165+ , ("croatian", \_ -> simpleLang "hr")
166+ , ("magyar", \_ -> simpleLang "hu")
167+ , ("armenian", \_ -> simpleLang "hy")
168+ , ("interlingua", \_ -> simpleLang "ia")
169+ , ("indonesian", \_ -> simpleLang "id")
170+ , ("icelandic", \_ -> simpleLang "is")
171+ , ("italian", \_ -> simpleLang "it")
172+ , ("japanese", \_ -> simpleLang "jp")
173+ , ("khmer", \_ -> simpleLang "km")
174+ , ("kurmanji", \_ -> simpleLang "kmr")
175+ , ("kannada", \_ -> simpleLang "kn")
176+ , ("korean", \_ -> simpleLang "ko")
177+ , ("lao", \_ -> simpleLang "lo")
178+ , ("lithuanian", \_ -> simpleLang "lt")
179+ , ("latvian", \_ -> simpleLang "lv")
180+ , ("malayalam", \_ -> simpleLang "ml")
181+ , ("mongolian", \_ -> simpleLang "mn")
182+ , ("marathi", \_ -> simpleLang "mr")
183+ , ("dutch", \_ -> simpleLang "nl")
184+ , ("nynorsk", \_ -> simpleLang "nn")
185+ , ("norsk", \_ -> simpleLang "no")
186+ , ("nko", \_ -> simpleLang "nqo")
187+ , ("occitan", \_ -> simpleLang "oc")
188+ , ("panjabi", \_ -> simpleLang "pa")
189+ , ("polish", \_ -> simpleLang "pl")
190+ , ("piedmontese", \_ -> simpleLang "pms")
191+ , ("portuguese", \_ -> simpleLang "pt")
192+ , ("romansh", \_ -> simpleLang "rm")
193+ , ("romanian", \_ -> simpleLang "ro")
194+ , ("russian", \_ -> simpleLang "ru")
195+ , ("sanskrit", \_ -> simpleLang "sa")
196+ , ("samin", \_ -> simpleLang "se")
197+ , ("slovak", \_ -> simpleLang "sk")
198+ , ("albanian", \_ -> simpleLang "sq")
199+ , ("serbian", \_ -> simpleLang "sr")
200+ , ("swedish", \_ -> simpleLang "sv")
201+ , ("syriac", \_ -> simpleLang "syr")
202+ , ("tamil", \_ -> simpleLang "ta")
203+ , ("telugu", \_ -> simpleLang "te")
204+ , ("thai", \_ -> simpleLang "th")
205+ , ("turkmen", \_ -> simpleLang "tk")
206+ , ("turkish", \_ -> simpleLang "tr")
207+ , ("ukrainian", \_ -> simpleLang "uk")
208+ , ("urdu", \_ -> simpleLang "ur")
209+ , ("vietnamese", \_ -> simpleLang "vi")
210210 ]
211211
212+simpleLang :: Text -> Lang
213+simpleLang l = Lang l Nothing Nothing [] [] []
214+
212215babelLangToBCP47 :: T.Text -> Maybe Lang
213216babelLangToBCP47 s =
214217 case s of
215− "austrian" -> Just $ Lang "de" "" "AT" ["1901"]
216− "naustrian" -> Just $ Lang "de" "" "AT" []
217− "swissgerman" -> Just $ Lang "de" "" "CH" ["1901"]
218− "nswissgerman" -> Just $ Lang "de" "" "CH" []
219− "german" -> Just $ Lang "de" "" "DE" ["1901"]
220− "ngerman" -> Just $ Lang "de" "" "DE" []
221− "lowersorbian" -> Just $ Lang "dsb" "" "" []
222− "uppersorbian" -> Just $ Lang "hsb" "" "" []
223− "polutonikogreek" -> Just $ Lang "el" "" "" ["polyton"]
224− "slovene" -> Just $ Lang "sl" "" "" []
225− "australian" -> Just $ Lang "en" "" "AU" []
226− "canadian" -> Just $ Lang "en" "" "CA" []
227− "british" -> Just $ Lang "en" "" "GB" []
228− "newzealand" -> Just $ Lang "en" "" "NZ" []
229− "american" -> Just $ Lang "en" "" "US" []
230− "classiclatin" -> Just $ Lang "la" "" "" ["x-classic"]
218+ "austrian" -> Just $ Lang "de" Nothing (Just "AT") ["1901"] [] []
219+ "naustrian" -> Just $ Lang "de" Nothing (Just "AT") [] [] []
220+ "swissgerman" -> Just $ Lang "de" Nothing (Just "CH") ["1901"] [] []
221+ "nswissgerman" -> Just $ Lang "de" Nothing (Just "CH") [] [] []
222+ "german" -> Just $ Lang "de" Nothing (Just "DE") ["1901"] [] []
223+ "ngerman" -> Just $ Lang "de" Nothing (Just "DE") [] [] []
224+ "lowersorbian" -> Just $ Lang "dsb" Nothing Nothing [] [] []
225+ "uppersorbian" -> Just $ Lang "hsb" Nothing Nothing [] [] []
226+ "polutonikogreek" -> Just $ Lang "el" Nothing Nothing ["polyton"] [] []
227+ "slovene" -> Just $ simpleLang "sl"
228+ "australian" -> Just $ Lang "en" Nothing (Just "AU") [] [] []
229+ "canadian" -> Just $ Lang "en" Nothing (Just "CA") [] [] []
230+ "british" -> Just $ Lang "en" Nothing (Just "GB") [] [] []
231+ "newzealand" -> Just $ Lang "en" Nothing (Just "NZ") [] [] []
232+ "american" -> Just $ Lang "en" Nothing (Just "US") [] [] []
233+ "classiclatin" -> Just $ Lang "la" Nothing Nothing ["x-classic"] [] []
231234 _ -> ($ "") <$> M.lookup s polyglossiaLangToBCP47
src/Text/Pandoc/Readers/LaTeX/Macro.hs+45−14
@@ -14,6 +14,7 @@ import Text.Pandoc.Parsing hiding (blankline, mathDisplay, mathInline,
1414import Control.Applicative ((<|>), optional)
1515import qualified Data.Map as M
1616import Data.Text (Text)
17+import qualified Data.Text as T
1718
1819macroDef :: (PandocMonad m, Monoid a) => (Text -> a) -> LP m a
1920macroDef constructor = do
@@ -22,9 +23,11 @@ macroDef constructor = do
2223 guardDisabled Ext_latex_macros)
2324 <|> return mempty
2425 where commandDef = do
25− (name, macro') <- newcommand <|> letmacro <|> defmacro
26+ nameMacroPairs <- newcommand <|> letmacro <|> defmacro <|> newif
2627 guardDisabled Ext_latex_macros <|>
27− updateState (\s -> s{ sMacros = M.insert name macro' (sMacros s) })
28+ mapM_ (\(name, macro') ->
29+ updateState (\s -> s{ sMacros = M.insert name macro'
30+ (sMacros s) })) nameMacroPairs
2831 environmentDef = do
2932 mbenv <- newenvironment
3033 case mbenv of
@@ -40,7 +43,7 @@ macroDef constructor = do
4043 -- @\newcommand{\envname}[n-args][default]{begin}@
4144 -- @\newcommand{\endenvname}@
4245
43−letmacro :: PandocMonad m => LP m (Text, Macro)
46+letmacro :: PandocMonad m => LP m [(Text, Macro)]
4447letmacro = do
4548 controlSeq "let"
4649 (name, contents) <- withVerbatimMode $ do
@@ -53,18 +56,47 @@ letmacro = do
5356 contents <- bracedOrToken
5457 return (name, contents)
5558 contents' <- doMacros' 0 contents
56− return (name, Macro ExpandWhenDefined [] Nothing contents')
59+ return [(name, Macro ExpandWhenDefined [] Nothing contents')]
5760
58−defmacro :: PandocMonad m => LP m (Text, Macro)
59−defmacro = try $
61+defmacro :: PandocMonad m => LP m [(Text, Macro)]
62+defmacro = do
6063 -- we use withVerbatimMode, because macros are to be expanded
6164 -- at point of use, not point of definition
65+ controlSeq "def"
6266 withVerbatimMode $ do
63− controlSeq "def"
6467 Tok _ (CtrlSeq name) _ <- anyControlSeq
6568 argspecs <- many (argspecArg <|> argspecPattern)
6669 contents <- bracedOrToken
67− return (name, Macro ExpandWhenUsed argspecs Nothing contents)
70+ return [(name, Macro ExpandWhenUsed argspecs Nothing contents)]
71+
72+-- \newif\iffoo' defines:
73+-- \iffoo to be \iffalse
74+-- \footrue to be a command that defines \iffoo to be \iftrue
75+-- \foofalse to be a command that defines \iffoo to be \iffalse
76+newif :: PandocMonad m => LP m [(Text, Macro)]
77+newif = do
78+ controlSeq "newif"
79+ withVerbatimMode $ do
80+ Tok pos (CtrlSeq name) _ <- anyControlSeq
81+ -- \def\iffoo\iffalse
82+ -- \def\footrue{\def\iffoo\iftrue}
83+ -- \def\foofalse{\def\iffoo\iffalse}
84+ let base = T.drop 2 name
85+ return [ (name, Macro ExpandWhenUsed [] Nothing
86+ [Tok pos (CtrlSeq "iffalse") "\\iffalse"])
87+ , (base <> "true",
88+ Macro ExpandWhenUsed [] Nothing
89+ [ Tok pos (CtrlSeq "def") "\\def"
90+ , Tok pos (CtrlSeq name) ("\\" <> name)
91+ , Tok pos (CtrlSeq "iftrue") "\\iftrue"
92+ ])
93+ , (base <> "false",
94+ Macro ExpandWhenUsed [] Nothing
95+ [ Tok pos (CtrlSeq "def") "\\def"
96+ , Tok pos (CtrlSeq name) ("\\" <> name)
97+ , Tok pos (CtrlSeq "iffalse") "\\iffalse"
98+ ])
99+ ]
68100
69101argspecArg :: PandocMonad m => LP m ArgSpec
70102argspecArg = do
@@ -77,10 +109,9 @@ argspecPattern =
77109 (toktype' == Symbol || toktype' == Word) &&
78110 (txt /= "{" && txt /= "\\" && txt /= "}")))
79111
80−newcommand :: PandocMonad m => LP m (Text, Macro)
112+newcommand :: PandocMonad m => LP m [(Text, Macro)]
81113newcommand = do
82− pos <- getPosition
83− Tok _ (CtrlSeq mtype) _ <- controlSeq "newcommand" <|>
114+ Tok pos (CtrlSeq mtype) _ <- controlSeq "newcommand" <|>
84115 controlSeq "renewcommand" <|>
85116 controlSeq "providecommand" <|>
86117 controlSeq "DeclareMathOperator" <|>
@@ -112,9 +143,9 @@ newcommand = do
112143 Just macro
113144 | mtype == "newcommand" -> do
114145 report $ MacroAlreadyDefined txt pos
115− return (name, macro)
116− | mtype == "providecommand" -> return (name, macro)
117− _ -> return (name, Macro ExpandWhenUsed argspecs optarg contents)
146+ return [(name, macro)]
147+ | mtype == "providecommand" -> return [(name, macro)]
148+ _ -> return [(name, Macro ExpandWhenUsed argspecs optarg contents)]
118149
119150newenvironment :: PandocMonad m => LP m (Maybe (Text, Macro, Macro))
120151newenvironment = do
src/Text/Pandoc/Readers/LaTeX/Parsing.hs+48−7
@@ -27,6 +27,7 @@ module Text.Pandoc.Readers.LaTeX.Parsing
2727 , rawLaTeXParser
2828 , applyMacros
2929 , tokenize
30+ , tokenizeSources
3031 , untokenize
3132 , untoken
3233 , totoks
@@ -112,7 +113,6 @@ import Text.Pandoc.Readers.LaTeX.Types (ExpansionPoint (..), Macro (..),
112113 ArgSpec (..), Tok (..), TokType (..))
113114import Text.Pandoc.Shared
114115import Text.Parsec.Pos
115−-- import Debug.Trace
116116
117117newtype DottedNum = DottedNum [Int]
118118 deriving (Show, Eq)
@@ -248,7 +248,7 @@ withVerbatimMode parser = do
248248
249249rawLaTeXParser :: (PandocMonad m, HasMacros s, HasReaderOptions s)
250250 => [Tok] -> Bool -> LP m a -> LP m a
251− -> ParserT Text s m (a, Text)
251+ -> ParserT Sources s m (a, Text)
252252rawLaTeXParser toks retokenize parser valParser = do
253253 pstate <- getState
254254 let lstate = def{ sOptions = extractReaderOptions pstate }
@@ -268,7 +268,7 @@ rawLaTeXParser toks retokenize parser valParser = do
268268 Left _ -> mzero
269269 Right ((val, raw), st) -> do
270270 updateState (updateMacros (sMacros st <>))
271− _ <- takeP (T.length (untokenize toks'))
271+ void $ count (T.length (untokenize toks')) anyChar
272272 let result = untokenize raw
273273 -- ensure we end with space if input did, see #4442
274274 let result' =
@@ -281,7 +281,7 @@ rawLaTeXParser toks retokenize parser valParser = do
281281 return (val, result')
282282
283283applyMacros :: (PandocMonad m, HasMacros s, HasReaderOptions s)
284− => Text -> ParserT Text s m Text
284+ => Text -> ParserT Sources s m Text
285285applyMacros s = (guardDisabled Ext_latex_macros >> return s) <|>
286286 do let retokenize = untokenize <$> many (satisfyTok (const True))
287287 pstate <- getState
@@ -301,6 +301,11 @@ QuickCheck property:
301301> let t = T.pack s in untokenize (tokenize "random" t) == t
302302-}
303303
304+tokenizeSources :: Sources -> [Tok]
305+tokenizeSources = concatMap tokenizeSource . unSources
306+ where
307+ tokenizeSource (pos, t) = totoks pos t
308+
304309tokenize :: SourceName -> Text -> [Tok]
305310tokenize sourcename = totoks (initialPos sourcename)
306311
@@ -458,7 +463,7 @@ satisfyTok f = do
458463doMacros :: PandocMonad m => LP m ()
459464doMacros = do
460465 st <- getState
461− unless (sVerbatimMode st || M.null (sMacros st)) $ do
466+ unless (sVerbatimMode st) $
462467 getInput >>= doMacros' 1 >>= setInput
463468
464469doMacros' :: PandocMonad m => Int -> [Tok] -> LP m [Tok]
@@ -520,7 +525,7 @@ doMacros' n inp =
520525 $ throwError $ PandocMacroLoop name
521526 macros <- sMacros <$> getState
522527 case M.lookup name macros of
523− Nothing -> mzero
528+ Nothing -> trySpecialMacro name ts
524529 Just (Macro expansionPoint argspecs optarg newtoks) -> do
525530 let getargs' = do
526531 args <-
@@ -548,6 +553,41 @@ doMacros' n inp =
548553 ExpandWhenUsed -> doMacros' (n' + 1) result
549554 ExpandWhenDefined -> return result
550555
556+-- | Certain macros do low-level tex manipulations that can't
557+-- be represented in our Macro type, so we handle them here.
558+trySpecialMacro :: PandocMonad m => Text -> [Tok] -> LP m [Tok]
559+trySpecialMacro "xspace" ts = do
560+ ts' <- doMacros' 1 ts
561+ case ts' of
562+ Tok pos Word t : _
563+ | startsWithAlphaNum t -> return $ Tok pos Spaces " " : ts'
564+ _ -> return ts'
565+trySpecialMacro "iftrue" ts = handleIf True ts
566+trySpecialMacro "iffalse" ts = handleIf False ts
567+trySpecialMacro _ _ = mzero
568+
569+handleIf :: PandocMonad m => Bool -> [Tok] -> LP m [Tok]
570+handleIf b ts = do
571+ res' <- lift $ runParserT (ifParser b) defaultLaTeXState "tokens" ts
572+ case res' of
573+ Left _ -> Prelude.fail "Could not parse conditional"
574+ Right ts' -> return ts'
575+
576+ifParser :: PandocMonad m => Bool -> LP m [Tok]
577+ifParser b = do
578+ ifToks <- many (notFollowedBy (controlSeq "else" <|> controlSeq "fi")
579+ *> anyTok)
580+ elseToks <- (controlSeq "else" >> manyTill anyTok (controlSeq "fi"))
581+ <|> ([] <$ controlSeq "fi")
582+ rest <- getInput
583+ return $ (if b then ifToks else elseToks) ++ rest
584+
585+startsWithAlphaNum :: Text -> Bool
586+startsWithAlphaNum t =
587+ case T.uncons t of
588+ Just (c, _) | isAlphaNum c -> True
589+ _ -> False
590+
551591setpos :: SourcePos -> Tok -> Tok
552592setpos spos (Tok _ tt txt) = Tok spos tt txt
553593
@@ -783,7 +823,8 @@ withRaw parser = do
783823
784824keyval :: PandocMonad m => LP m (Text, Text)
785825keyval = try $ do
786− Tok _ Word key <- satisfyTok isWordTok
826+ key <- untokenize <$> many1 (notFollowedBy (symbol '=') >>
827+ (symbol '-' <|> symbol '_' <|> satisfyTok isWordTok))
787828 sp
788829 val <- option mempty $ do
789830 symbol '='
src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+140−66
@@ -1,3 +1,4 @@
1+{-# LANGUAGE ScopedTypeVariables #-}
12{-# LANGUAGE OverloadedStrings #-}
23module Text.Pandoc.Readers.LaTeX.SIunitx
34 ( siunitxCommands )
@@ -9,27 +10,32 @@ import Text.Pandoc.Class
910import Text.Pandoc.Parsing hiding (blankline, mathDisplay, mathInline,
1011 optional, space, spaces, withRaw, (<|>))
1112import Control.Applicative ((<|>))
13+import Control.Monad (void)
1214import qualified Data.Map as M
1315import Data.Char (isDigit)
1416import Data.Text (Text)
1517import qualified Data.Text as T
1618import Data.List (intersperse)
17−
19+import qualified Data.Sequence as Seq
20+import Text.Pandoc.Walk (walk)
1821
1922siunitxCommands :: PandocMonad m
2023 => LP m Inlines -> M.Map Text (LP m Inlines)
2124siunitxCommands tok = M.fromList
22− [ ("si", skipopts *> dosi tok)
25+ [ ("si", dosi tok)
2326 , ("SI", doSI tok)
2427 , ("SIrange", doSIrange True tok)
2528 , ("numrange", doSIrange False tok)
2629 , ("numlist", doSInumlist)
30+ , ("SIlist", doSIlist tok)
2731 , ("num", doSInum)
2832 , ("ang", doSIang)
2933 ]
3034
3135dosi :: PandocMonad m => LP m Inlines -> LP m Inlines
32−dosi tok = grouped (siUnit tok) <|> siUnit tok
36+dosi tok = do
37+ options <- option [] keyvals
38+ grouped (siUnit options tok) <|> siUnit options tok
3339
3440-- converts e.g. \SI{1}[\$]{} to "$ 1" or \SI{1}{\euro} to "1 €"
3541doSI :: PandocMonad m => LP m Inlines -> LP m Inlines
@@ -64,23 +70,50 @@ doSInumlist = do
6470 mconcat (intersperse (str "," <> space) (init xs)) <>
6571 text ", & " <> last xs
6672
73+doSIlist :: PandocMonad m => LP m Inlines -> LP m Inlines
74+doSIlist tok = do
75+ options <- option [] keyvals
76+ nums <- map tonum . T.splitOn ";" . untokenize <$> braced
77+ unit <- grouped (siUnit options tok) <|> siUnit options tok
78+ let xs = map (<> (str "\xa0" <> unit)) nums
79+ case xs of
80+ [] -> return mempty
81+ [x] -> return x
82+ _ -> return $
83+ mconcat (intersperse (str "," <> space) (init xs)) <>
84+ text ", & " <> last xs
85+
6786parseNum :: Parser Text () Inlines
6887parseNum = (mconcat <$> many parseNumPart) <* eof
6988
89+minus :: Text
90+minus = "\x2212"
91+
92+hyphenToMinus :: Inline -> Inline
93+hyphenToMinus (Str t) = Str (T.replace "-" minus t)
94+hyphenToMinus x = x
95+
7096parseNumPart :: Parser Text () Inlines
7197parseNumPart =
7298 parseDecimalNum <|>
7399 parseComma <|>
74100 parsePlusMinus <|>
101+ parsePM <|>
75102 parseI <|>
76103 parseExp <|>
77104 parseX <|>
78105 parseSpace
79106 where
80− parseDecimalNum = do
81− pref <- option mempty $ (mempty <$ char '+') <|> ("\x2212" <$ char '-')
82− basenum <- (pref <>) . T.pack
83− <$> many1 (satisfy (\c -> isDigit c || c == '.'))
107+ parseDecimalNum, parsePlusMinus, parsePM,
108+ parseComma, parseI, parseX,
109+ parseExp, parseSpace :: Parser Text () Inlines
110+ parseDecimalNum = try $ do
111+ pref <- option mempty $ (mempty <$ char '+') <|> (minus <$ char '-')
112+ basenum' <- many1 (satisfy (\c -> isDigit c || c == '.'))
113+ let basenum = pref <> T.pack
114+ (case basenum' of
115+ '.':_ -> '0':basenum'
116+ _ -> basenum')
84117 uncertainty <- option mempty $ T.pack <$> parseParens
85118 if T.null uncertainty
86119 then return $ str basenum
@@ -98,6 +131,7 @@ parseNumPart =
98131 | otherwise -> "." <> t
99132 parseComma = str "." <$ char ','
100133 parsePlusMinus = str "\xa0\xb1\xa0" <$ try (string "+-")
134+ parsePM = str "\xa0\xb1\xa0" <$ try (string "\\pm")
101135 parseParens =
102136 char '(' *> many1 (satisfy (\c -> isDigit c || c == '.')) <* char ')'
103137 parseI = str "i" <$ char 'i'
@@ -110,11 +144,14 @@ doSIang :: PandocMonad m => LP m Inlines
110144doSIang = do
111145 skipopts
112146 ps <- T.splitOn ";" . untokenize <$> braced
147+ let dropPlus t = case T.uncons t of
148+ Just ('+',t') -> t'
149+ _ -> t
113150 case ps ++ repeat "" of
114151 (d:m:s:_) -> return $
115− (if T.null d then mempty else str d <> str "\xb0") <>
116− (if T.null m then mempty else str m <> str "\x2032") <>
117− (if T.null s then mempty else str s <> str "\x2033")
152+ (if T.null d then mempty else str (dropPlus d) <> str "\xb0") <>
153+ (if T.null m then mempty else str (dropPlus m) <> str "\x2032") <>
154+ (if T.null s then mempty else str (dropPlus s) <> str "\x2033")
118155 _ -> return mempty
119156
120157-- converts e.g. \SIrange{100}{200}{\ms} to "100 ms--200 ms"
@@ -143,40 +180,99 @@ doSIrange includeUnits tok = do
143180emptyOr160 :: Inlines -> Inlines
144181emptyOr160 x = if x == mempty then x else str "\160"
145182
146−siUnit :: PandocMonad m => LP m Inlines -> LP m Inlines
147−siUnit tok = try (do
148− Tok _ (CtrlSeq name) _ <- anyControlSeq
149− case name of
150− "square" -> do
151− unit <- siUnit tok
152− return $ unit <> superscript "2"
153− "cubic" -> do
154− unit <- siUnit tok
155− return $ unit <> superscript "3"
156− "raisetothe" -> do
157− n <- tok
158− unit <- siUnit tok
159− return $ unit <> superscript n
160− _ ->
161− case M.lookup name siUnitMap of
162− Just il ->
163− option il $
164− choice
165− [ (il <> superscript "2") <$ controlSeq "squared"
166− , (il <> superscript "3") <$ controlSeq "cubed"
167− , (\n -> il <> superscript n) <$> (controlSeq "tothe" *> tok)
168− ]
169− Nothing -> fail "not an siunit unit command")
170− <|> (lookAhead anyControlSeq >> tok)
171− <|> (do Tok _ Word t <- satisfyTok isWordTok
172− return $ str t)
173− <|> (symbol '^' *> (superscript <$> tok))
174− <|> (symbol '_' *> (subscript <$> tok))
175− <|> ("\xa0" <$ symbol '.')
176− <|> ("\xa0" <$ symbol '~')
177− <|> tok
178− <|> (do Tok _ _ t <- anyTok
179− return (str t))
183+siUnit :: forall m. PandocMonad m => [(Text,Text)] -> LP m Inlines -> LP m Inlines
184+siUnit options tok = mconcat . intersperse (str "\xa0") <$> many1 siUnitPart
185+ where
186+ siUnitPart :: LP m Inlines
187+ siUnitPart = try $ do
188+ skipMany (void (symbol '.') <|> void (symbol '~') <|> spaces1)
189+ x <- ((siPrefix <*> siBase)
190+ <|> (do u <- siBase <|> tok
191+ option u $ siSuffix <*> pure u))
192+ option x (siInfix x)
193+ siInfix :: Inlines -> LP m Inlines
194+ siInfix u1 = try $
195+ (do _ <- controlSeq "per"
196+ u2 <- siUnitPart
197+ let useSlash = lookup "per-mode" options == Just "symbol"
198+ if useSlash
199+ then return (u1 <> str "/" <> u2)
200+ else return (u1 <> str "\xa0" <> negateExponent u2))
201+ <|> (do _ <- symbol '/'
202+ u2 <- siUnitPart
203+ return (u1 <> str "/" <> u2))
204+ siPrefix :: LP m (Inlines -> Inlines)
205+ siPrefix =
206+ (do _ <- controlSeq "square"
207+ skipopts
208+ return (<> superscript "2"))
209+ <|> (do _ <- controlSeq "cubic"
210+ skipopts
211+ return (<> superscript "3"))
212+ <|> (do _ <- controlSeq "raisetothe"
213+ skipopts
214+ n <- walk hyphenToMinus <$> tok
215+ return (<> superscript n))
216+ siSuffix :: LP m (Inlines -> Inlines)
217+ siSuffix =
218+ (do _ <- controlSeq "squared"
219+ skipopts
220+ return (<> superscript "2"))
221+ <|> (do _ <- controlSeq "cubed"
222+ skipopts
223+ return (<> superscript "3"))
224+ <|> (do _ <- controlSeq "tothe"
225+ skipopts
226+ n <- walk hyphenToMinus <$> tok
227+ return (<> superscript n))
228+ <|> (symbol '^' *> (do n <- walk hyphenToMinus <$> tok
229+ return (<> superscript n)))
230+ <|> (symbol '_' *> (do n <- walk hyphenToMinus <$> tok
231+ return (<> subscript n)))
232+ negateExponent :: Inlines -> Inlines
233+ negateExponent ils =
234+ case Seq.viewr (unMany ils) of
235+ xs Seq.:> Superscript ss -> (Many xs) <>
236+ superscript (str minus <> fromList ss)
237+ _ -> ils <> superscript (str (minus <> "1"))
238+ siBase :: LP m Inlines
239+ siBase =
240+ ((try
241+ (do Tok _ (CtrlSeq name) _ <- anyControlSeq
242+ case M.lookup name siUnitModifierMap of
243+ Just il -> (il <>) <$> siBase
244+ Nothing ->
245+ case M.lookup name siUnitMap of
246+ Just il -> pure il
247+ Nothing -> fail "not a unit command"))
248+ <|> (do Tok _ Word t <- satisfyTok isWordTok
249+ return $ str t)
250+ )
251+
252+siUnitModifierMap :: M.Map Text Inlines
253+siUnitModifierMap = M.fromList
254+ [ ("atto", str "a")
255+ , ("centi", str "c")
256+ , ("deca", str "d")
257+ , ("deci", str "d")
258+ , ("deka", str "d")
259+ , ("exa", str "E")
260+ , ("femto", str "f")
261+ , ("giga", str "G")
262+ , ("hecto", str "h")
263+ , ("kilo", str "k")
264+ , ("mega", str "M")
265+ , ("micro", str "μ")
266+ , ("milli", str "m")
267+ , ("nano", str "n")
268+ , ("peta", str "P")
269+ , ("pico", str "p")
270+ , ("tera", str "T")
271+ , ("yocto", str "y")
272+ , ("yotta", str "Y")
273+ , ("zepto", str "z")
274+ , ("zetta", str "Z")
275+ ]
180276
181277siUnitMap :: M.Map Text Inlines
182278siUnitMap = M.fromList
@@ -276,7 +372,6 @@ siUnitMap = M.fromList
276372 , ("arcsecond", str "″")
277373 , ("astronomicalunit", str "ua")
278374 , ("atomicmassunit", str "u")
279− , ("atto", str "a")
280375 , ("bar", str "bar")
281376 , ("barn", str "b")
282377 , ("becquerel", str "Bq")
@@ -284,51 +379,38 @@ siUnitMap = M.fromList
284379 , ("bohr", emph (str "a") <> subscript (str "0"))
285380 , ("candela", str "cd")
286381 , ("celsius", str "°C")
287− , ("centi", str "c")
288382 , ("clight", emph (str "c") <> subscript (str "0"))
289383 , ("coulomb", str "C")
290384 , ("dalton", str "Da")
291385 , ("day", str "d")
292− , ("deca", str "d")
293− , ("deci", str "d")
294386 , ("decibel", str "db")
295387 , ("degreeCelsius",str "°C")
296388 , ("degree", str "°")
297− , ("deka", str "d")
298389 , ("electronmass", emph (str "m") <> subscript (str "e"))
299390 , ("electronvolt", str "eV")
300391 , ("elementarycharge", emph (str "e"))
301− , ("exa", str "E")
302392 , ("farad", str "F")
303− , ("femto", str "f")
304− , ("giga", str "G")
305393 , ("gram", str "g")
306394 , ("gray", str "Gy")
307395 , ("hartree", emph (str "E") <> subscript (str "h"))
308396 , ("hectare", str "ha")
309− , ("hecto", str "h")
310397 , ("henry", str "H")
311398 , ("hertz", str "Hz")
312399 , ("hour", str "h")
313400 , ("joule", str "J")
314401 , ("katal", str "kat")
315402 , ("kelvin", str "K")
316− , ("kilo", str "k")
317403 , ("kilogram", str "kg")
318404 , ("knot", str "kn")
319405 , ("liter", str "L")
320406 , ("litre", str "l")
321407 , ("lumen", str "lm")
322408 , ("lux", str "lx")
323− , ("mega", str "M")
324409 , ("meter", str "m")
325410 , ("metre", str "m")
326− , ("micro", str "μ")
327− , ("milli", str "m")
328411 , ("minute", str "min")
329412 , ("mmHg", str "mmHg")
330413 , ("mole", str "mol")
331− , ("nano", str "n")
332414 , ("nauticalmile", str "M")
333415 , ("neper", str "Np")
334416 , ("newton", str "N")
@@ -336,25 +418,17 @@ siUnitMap = M.fromList
336418 , ("Pa", str "Pa")
337419 , ("pascal", str "Pa")
338420 , ("percent", str "%")
339− , ("per", str "/")
340− , ("peta", str "P")
341− , ("pico", str "p")
342421 , ("planckbar", emph (str "\x210f"))
343422 , ("radian", str "rad")
344423 , ("second", str "s")
345424 , ("siemens", str "S")
346425 , ("sievert", str "Sv")
347426 , ("steradian", str "sr")
348− , ("tera", str "T")
349427 , ("tesla", str "T")
350428 , ("tonne", str "t")
351429 , ("volt", str "V")
352430 , ("watt", str "W")
353431 , ("weber", str "Wb")
354− , ("yocto", str "y")
355− , ("yotta", str "Y")
356− , ("zepto", str "z")
357− , ("zetta", str "Z")
358432 ]
359433
360434
src/Text/Pandoc/Readers/LaTeX/Table.hs+6−1
@@ -134,6 +134,11 @@ parseTableRow :: PandocMonad m
134134 -> LP m Row
135135parseTableRow blocks inline envname prefsufs = do
136136 notFollowedBy (spaces *> end_ envname)
137+ -- contexts that can contain & that is not colsep:
138+ let canContainAmp (Tok _ (CtrlSeq "begin") _) = True
139+ canContainAmp (Tok _ (CtrlSeq "verb") _) = True
140+ canContainAmp (Tok _ (CtrlSeq "Verb") _) = True
141+ canContainAmp _ = False
137142 -- add prefixes and suffixes in token stream:
138143 let celltoks (pref, suff) = do
139144 prefpos <- getPosition
@@ -142,7 +147,7 @@ parseTableRow blocks inline envname prefsufs = do
142147 ((lookAhead (controlSeq "parbox") >>
143148 void blocks) -- #5711
144149 <|>
145− (lookAhead (controlSeq "begin") >> void inline)
150+ (lookAhead (satisfyTok canContainAmp) >> void inline)
146151 <|>
147152 (lookAhead (symbol '$') >> void inline))
148153 <|>
src/Text/Pandoc/Readers/LaTeX/Types.hs+14−1
@@ -1,3 +1,4 @@
1+{-# LANGUAGE FlexibleInstances #-}
12{- |
23 Module : Text.Pandoc.Readers.LaTeX.Types
34 Copyright : Copyright (C) 2017-2021 John MacFarlane
@@ -18,7 +19,9 @@ module Text.Pandoc.Readers.LaTeX.Types ( Tok(..)
1819 )
1920where
2021import Data.Text (Text)
21−import Text.Parsec.Pos (SourcePos)
22+import Text.Parsec.Pos (SourcePos, sourceName)
23+import Text.Pandoc.Sources
24+import Data.List (groupBy)
2225
2326data TokType = CtrlSeq Text | Spaces | Newline | Symbol | Word | Comment |
2427 Esc1 | Esc2 | Arg Int
@@ -27,6 +30,16 @@ data TokType = CtrlSeq Text | Spaces | Newline | Symbol | Word | Comment |
2730data Tok = Tok SourcePos TokType Text
2831 deriving (Eq, Ord, Show)
2932
33+instance ToSources [Tok] where
34+ toSources = Sources
35+ . map (\ts -> case ts of
36+ Tok p _ _ : _ -> (p, mconcat $ map tokToText ts)
37+ _ -> error "toSources [Tok] encountered empty group")
38+ . groupBy (\(Tok p1 _ _) (Tok p2 _ _) -> sourceName p1 == sourceName p2)
39+
40+tokToText :: Tok -> Text
41+tokToText (Tok _ _ t) = t
42+
3043data ExpansionPoint = ExpandWhenDefined | ExpandWhenUsed
3144 deriving (Eq, Ord, Show)
3245
src/Text/Pandoc/Readers/Man.hs+18−11
@@ -20,7 +20,7 @@ import Control.Monad (liftM, mzero, guard, void)
2020import Control.Monad.Trans (lift)
2121import Control.Monad.Except (throwError)
2222import Data.Maybe (catMaybes, isJust)
23−import Data.List (intersperse, intercalate)
23+import Data.List (intersperse)
2424import qualified Data.Text as T
2525import Text.Pandoc.Builder as B
2626import Text.Pandoc.Class.PandocMonad (PandocMonad(..), report)
@@ -29,9 +29,8 @@ import Text.Pandoc.Logging (LogMessage(..))
2929import Text.Pandoc.Options
3030import Text.Pandoc.Parsing
3131import Text.Pandoc.Walk (query)
32−import Text.Pandoc.Shared (crFilter, mapLeft)
32+import Text.Pandoc.Shared (mapLeft)
3333import Text.Pandoc.Readers.Roff -- TODO explicit imports
34−import Text.Parsec hiding (tokenPrim)
3534import qualified Text.Parsec as Parsec
3635import Text.Parsec.Pos (updatePosString)
3736import qualified Data.Foldable as Foldable
@@ -50,13 +49,20 @@ type ManParser m = ParserT [RoffToken] ManState m
5049
5150
5251-- | Read man (troff) from an input string and return a Pandoc document.
53−readMan :: PandocMonad m => ReaderOptions -> T.Text -> m Pandoc
54−readMan opts txt = do
55− tokenz <- lexRoff (initialPos "input") (crFilter txt)
52+readMan :: (PandocMonad m, ToSources a)
53+ => ReaderOptions
54+ -> a
55+ -> m Pandoc
56+readMan opts s = do
57+ let Sources inps = toSources s
58+ tokenz <- mconcat <$> mapM (uncurry lexRoff) inps
5659 let state = def {readerOptions = opts} :: ManState
60+ let fixError (PandocParsecError _ e) = PandocParsecError (Sources inps) e
61+ fixError e = e
5762 eitherdoc <- readWithMTokens parseMan state
5863 (Foldable.toList . unRoffTokens $ tokenz)
59− either throwError return eitherdoc
64+ either (throwError . fixError) return eitherdoc
65+
6066
6167readWithMTokens :: PandocMonad m
6268 => ParserT [RoffToken] ManState m a -- ^ parser
@@ -64,9 +70,10 @@ readWithMTokens :: PandocMonad m
6470 -> [RoffToken] -- ^ input
6571 -> m (Either PandocError a)
6672readWithMTokens parser state input =
67− let leftF = PandocParsecError . T.pack . intercalate "\n" $ show <$> input
73+ let leftF = PandocParsecError mempty
6874 in mapLeft leftF `liftM` runParserT parser state "source" input
6975
76+
7077parseMan :: PandocMonad m => ManParser m Pandoc
7178parseMan = do
7279 bs <- many parseBlock <* eof
@@ -89,7 +96,7 @@ parseBlock = choice [ parseList
8996
9097parseTable :: PandocMonad m => ManParser m Blocks
9198parseTable = do
92− modifyState $ \st -> st { tableCellsPlain = True }
99+ updateState $ \st -> st { tableCellsPlain = True }
93100 let isTbl Tbl{} = True
94101 isTbl _ = False
95102 Tbl _opts rows pos <- msatisfy isTbl
@@ -135,7 +142,7 @@ parseTable = do
135142 case res' of
136143 Left _ -> Prelude.fail "Could not parse table cell"
137144 Right x -> do
138− modifyState $ \s -> s{ tableCellsPlain = False }
145+ updateState $ \s -> s{ tableCellsPlain = False }
139146 return x
140147 Right x -> return x
141148
@@ -222,7 +229,7 @@ parseTitle = do
222229 setMeta "section" (linePartsToInlines y)
223230 [x] -> setMeta "title" (linePartsToInlines x)
224231 [] -> id
225− modifyState $ \st -> st{ metadata = adjustMeta $ metadata st }
232+ updateState $ \st -> st{ metadata = adjustMeta $ metadata st }
226233 return mempty
227234
228235linePartsToInlines :: [LinePart] -> Inlines
src/Text/Pandoc/Readers/Markdown.hs+93−73
@@ -21,15 +21,17 @@ module Text.Pandoc.Readers.Markdown (
2121import Control.Monad
2222import Control.Monad.Except (throwError)
2323import Data.Char (isAlphaNum, isPunctuation, isSpace)
24−import Data.List (transpose, elemIndex, sortOn)
24+import Data.List (transpose, elemIndex, sortOn, foldl')
25+import Data.List.NonEmpty (NonEmpty(..))
2526import qualified Data.Map as M
2627import Data.Maybe
2728import qualified Data.Set as Set
2829import Data.Text (Text)
2930import qualified Data.Text as T
30−import qualified Data.Text.Lazy as TL
3131import qualified Data.ByteString.Lazy as BL
32−import System.FilePath (addExtension, takeExtension)
32+import System.FilePath (addExtension, takeExtension, takeDirectory)
33+import qualified System.FilePath.Windows as Windows
34+import qualified System.FilePath.Posix as Posix
3335import Text.HTML.TagSoup hiding (Row)
3436import Text.Pandoc.Builder (Blocks, Inlines)
3537import qualified Text.Pandoc.Builder as B
@@ -45,20 +47,22 @@ import Text.Pandoc.Readers.HTML (htmlInBalanced, htmlTag, isBlockTag,
4547 isCommentTag, isInlineTag, isTextTag)
4648import Text.Pandoc.Readers.LaTeX (applyMacros, rawLaTeXBlock, rawLaTeXInline)
4749import Text.Pandoc.Shared
48−import qualified Text.Pandoc.UTF8 as UTF8
4950import Text.Pandoc.XML (fromEntities)
50−import Text.Pandoc.Readers.Metadata (yamlBsToMeta, yamlBsToRefs)
51+import Text.Pandoc.Readers.Metadata (yamlBsToMeta, yamlBsToRefs, yamlMetaBlock)
52+-- import Debug.Trace (traceShowId)
5153
52−type MarkdownParser m = ParserT Text ParserState m
54+type MarkdownParser m = ParserT Sources ParserState m
55+
56+type F = Future ParserState
5357
5458-- | Read markdown from an input string and return a Pandoc document.
55−readMarkdown :: PandocMonad m
59+readMarkdown :: (PandocMonad m, ToSources a)
5660 => ReaderOptions -- ^ Reader options
57− -> Text -- ^ String to parse (assuming @'\n'@ line endings)
61+ -> a -- ^ Input
5862 -> m Pandoc
5963readMarkdown opts s = do
6064 parsed <- readWithM parseMarkdown def{ stateOptions = opts }
61− (crFilter s <> "\n\n")
65+ (ensureFinalNewlines 3 (toSources s))
6266 case parsed of
6367 Right result -> return result
6468 Left e -> throwError e
@@ -79,7 +83,7 @@ yamlToMeta opts mbfp bstr = do
7983 meta <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks) bstr
8084 setPosition oldPos
8185 return $ runF meta defaultParserState
82− parsed <- readWithM parser def{ stateOptions = opts } ""
86+ parsed <- readWithM parser def{ stateOptions = opts } ("" :: Text)
8387 case parsed of
8488 Right result -> return result
8589 Left e -> throwError e
@@ -95,14 +99,12 @@ yamlToRefs :: PandocMonad m
9599 -> m [MetaValue]
96100yamlToRefs idpred opts mbfp bstr = do
97101 let parser = do
98− oldPos <- getPosition
99102 case mbfp of
100103 Nothing -> return ()
101104 Just fp -> setPosition $ initialPos fp
102105 refs <- yamlBsToRefs (fmap B.toMetaValue <$> parseBlocks) idpred bstr
103− setPosition oldPos
104106 return $ runF refs defaultParserState
105− parsed <- readWithM parser def{ stateOptions = opts } ""
107+ parsed <- readWithM parser def{ stateOptions = opts } ("" :: Text)
106108 case parsed of
107109 Right result -> return result
108110 Left e -> throwError e
@@ -145,14 +147,14 @@ inList = do
145147 ctx <- stateParserContext <$> getState
146148 guard (ctx == ListItemState)
147149
148−spnl :: PandocMonad m => ParserT Text st m ()
150+spnl :: PandocMonad m => ParserT Sources st m ()
149151spnl = try $ do
150152 skipSpaces
151153 optional newline
152154 skipSpaces
153155 notFollowedBy (char '\n')
154156
155−spnl' :: PandocMonad m => ParserT Text st m Text
157+spnl' :: PandocMonad m => ParserT Sources st m Text
156158spnl' = try $ do
157159 xs <- many spaceChar
158160 ys <- option "" $ try $ (:) <$> newline
@@ -271,25 +273,13 @@ pandocTitleBlock = do
271273 $ nullMeta
272274 updateState $ \st -> st{ stateMeta' = stateMeta' st <> meta' }
273275
274−yamlMetaBlock :: PandocMonad m => MarkdownParser m (F Blocks)
275−yamlMetaBlock = do
276+yamlMetaBlock' :: PandocMonad m => MarkdownParser m (F Blocks)
277+yamlMetaBlock' = do
276278 guardEnabled Ext_yaml_metadata_block
277− try $ do
278− string "---"
279− blankline
280− notFollowedBy blankline -- if --- is followed by a blank it's an HRULE
281− rawYamlLines <- manyTill anyLine stopLine
282− -- by including --- and ..., we allow yaml blocks with just comments:
283− let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))
284− optional blanklines
285− newMetaF <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks)
286− $ UTF8.fromTextLazy $ TL.fromStrict rawYaml
287− -- Since `<>` is left-biased, existing values are not touched:
288− updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }
289− return mempty
290−
291−stopLine :: PandocMonad m => MarkdownParser m ()
292−stopLine = try $ (string "---" <|> string "...") >> blankline >> return ()
279+ newMetaF <- yamlMetaBlock (fmap B.toMetaValue <$> parseBlocks)
280+ -- Since `<>` is left-biased, existing values are not touched:
281+ updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }
282+ return mempty
293283
294284mmdTitleBlock :: PandocMonad m => MarkdownParser m ()
295285mmdTitleBlock = do
@@ -309,7 +299,7 @@ kvPair allowEmpty = try $ do
309299 (try $ newline >> lookAhead (blankline <|> nonspaceChar))
310300 guard $ allowEmpty || not (T.null val)
311301 let key' = T.concat $ T.words $ T.toLower key
312− let val' = MetaBlocks $ B.toList $ B.plain $ B.text val
302+ let val' = MetaInlines $ B.toList $ B.text val
313303 return (key',val')
314304
315305parseMarkdown :: PandocMonad m => MarkdownParser m Pandoc
@@ -348,7 +338,9 @@ referenceKey = try $ do
348338 notFollowedBy' (() <$ reference)
349339 many1Char $ notFollowedBy space >> litChar
350340 let betweenAngles = try $ char '<' >> manyTillChar litChar (char '>')
351− src <- try betweenAngles <|> sourceURL
341+ rebase <- option False (True <$ guardEnabled Ext_rebase_relative_paths)
342+ src <- (if rebase then rebasePath pos else id) <$>
343+ (try betweenAngles <|> sourceURL)
352344 tit <- option "" referenceTitle
353345 attr <- option nullAttr $ try $
354346 do guardEnabled Ext_link_attributes
@@ -357,7 +349,7 @@ referenceKey = try $ do
357349 addKvs <- option [] $ guardEnabled Ext_mmd_link_attributes
358350 >> many (try $ spnl >> keyValAttr)
359351 blanklines
360− let attr' = extractIdClass $ foldl (\x f -> f x) attr addKvs
352+ let attr' = extractIdClass $ foldl' (\x f -> f x) attr addKvs
361353 target = (escapeURI $ trimr src, tit)
362354 st <- getState
363355 let oldkeys = stateKeys st
@@ -453,7 +445,7 @@ block :: PandocMonad m => MarkdownParser m (F Blocks)
453445block = do
454446 res <- choice [ mempty <$ blanklines
455447 , codeBlockFenced
456− , yamlMetaBlock
448+ , yamlMetaBlock'
457449 -- note: bulletList needs to be before header because of
458450 -- the possibility of empty list items: -
459451 , bulletList
@@ -579,7 +571,7 @@ registerImplicitHeader raw attr@(ident, _, _)
579571-- hrule block
580572--
581573
582−hrule :: PandocMonad m => ParserT Text st m (F Blocks)
574+hrule :: PandocMonad m => ParserT Sources st m (F Blocks)
583575hrule = try $ do
584576 skipSpaces
585577 start <- satisfy isHruleChar
@@ -599,7 +591,7 @@ indentedLine = indentSpaces >> anyLineNewline
599591blockDelimiter :: PandocMonad m
600592 => (Char -> Bool)
601593 -> Maybe Int
602− -> ParserT Text ParserState m Int
594+ -> ParserT Sources ParserState m Int
603595blockDelimiter f len = try $ do
604596 skipNonindentSpaces
605597 c <- lookAhead (satisfy f)
@@ -613,7 +605,7 @@ attributes = try $ do
613605 spnl
614606 attrs <- many (attribute <* spnl)
615607 char '}'
616− return $ foldl (\x f -> f x) nullAttr attrs
608+ return $ foldl' (\x f -> f x) nullAttr attrs
617609
618610attribute :: PandocMonad m => MarkdownParser m (Attr -> Attr)
619611attribute = identifierAttr <|> classAttr <|> keyValAttr <|> specialAttr
@@ -743,7 +735,7 @@ lhsCodeBlockBirdWith c = try $ do
743735 blanklines
744736 return $ T.intercalate "\n" lns'
745737
746−birdTrackLine :: PandocMonad m => Char -> ParserT Text st m Text
738+birdTrackLine :: PandocMonad m => Char -> ParserT Sources st m Text
747739birdTrackLine c = try $ do
748740 char c
749741 -- allow html tags on left margin:
@@ -1036,7 +1028,7 @@ para = try $ do
10361028 option (B.plain <$> result)
10371029 $ try $ do
10381030 newline
1039− (blanklines >> return mempty)
1031+ (mempty <$ blanklines)
10401032 <|> (guardDisabled Ext_blank_before_blockquote >> () <$ lookAhead blockQuote)
10411033 <|> (guardEnabled Ext_backtick_code_blocks >> () <$ lookAhead codeBlockFenced)
10421034 <|> (guardDisabled Ext_blank_before_header >> () <$ lookAhead header)
@@ -1181,7 +1173,7 @@ lineBlock = do
11811173-- and the length including trailing space.
11821174dashedLine :: PandocMonad m
11831175 => Char
1184− -> ParserT Text st m (Int, Int)
1176+ -> ParserT Sources st m (Int, Int)
11851177dashedLine ch = do
11861178 dashes <- many1 (char ch)
11871179 sp <- many spaceChar
@@ -1250,7 +1242,7 @@ rawTableLine :: PandocMonad m
12501242 -> MarkdownParser m [Text]
12511243rawTableLine indices = do
12521244 notFollowedBy' (blanklines' <|> tableFooter)
1253− line <- take1WhileP (/='\n') <* newline
1245+ line <- anyLine
12541246 return $ map trim $ tail $
12551247 splitTextByIndices (init indices) line
12561248
@@ -1364,7 +1356,7 @@ pipeTable = try $ do
13641356 lines' <- many pipeTableRow
13651357 let lines'' = map (take (length aligns) <$>) lines'
13661358 let maxlength = maximum $
1367− map (\x -> T.length . stringify $ runF x def) (heads' : lines'')
1359+ fmap (\x -> T.length . stringify $ runF x def) (heads' :| lines'')
13681360 numColumns <- getOption readerColumns
13691361 let widths = if maxlength > numColumns
13701362 then map (\len ->
@@ -1401,7 +1393,7 @@ pipeTableCell =
14011393 return $ B.plain <$> result)
14021394 <|> return mempty
14031395
1404−pipeTableHeaderPart :: PandocMonad m => ParserT Text st m (Alignment, Int)
1396+pipeTableHeaderPart :: PandocMonad m => ParserT Sources st m (Alignment, Int)
14051397pipeTableHeaderPart = try $ do
14061398 skipMany spaceChar
14071399 left <- optionMaybe (char ':')
@@ -1417,10 +1409,14 @@ pipeTableHeaderPart = try $ do
14171409 (Just _,Just _) -> AlignCenter, len)
14181410
14191411-- Succeed only if current line contains a pipe.
1420−scanForPipe :: PandocMonad m => ParserT Text st m ()
1412+scanForPipe :: PandocMonad m => ParserT Sources st m ()
14211413scanForPipe = do
1422− inp <- getInput
1423− case T.break (\c -> c == '\n' || c == '|') inp of
1414+ Sources inps <- getInput
1415+ let ln = case inps of
1416+ [] -> ""
1417+ ((_,t):(_,t'):_) | T.null t -> t'
1418+ ((_,t):_) -> t
1419+ case T.break (\c -> c == '\n' || c == '|') ln of
14241420 (_, T.uncons -> Just ('|', _)) -> return ()
14251421 _ -> mzero
14261422
@@ -1503,7 +1499,7 @@ inline = do
15031499 '$' -> math
15041500 '~' -> strikeout <|> subscript
15051501 '<' -> autoLink <|> spanHtml <|> rawHtmlInline <|> ltSign
1506− '\\' -> escapedNewline <|> escapedChar <|> rawLaTeXInline'
1502+ '\\' -> math <|> escapedNewline <|> escapedChar <|> rawLaTeXInline'
15071503 '@' -> cite <|> exampleRef
15081504 '"' -> smart
15091505 '\'' -> smart
@@ -1714,18 +1710,18 @@ whitespace = spaceChar >> return <$> (lb <|> regsp) <?> "whitespace"
17141710 where lb = spaceChar >> skipMany spaceChar >> option B.space (endline >> return B.linebreak)
17151711 regsp = skipMany spaceChar >> return B.space
17161712
1717−nonEndline :: PandocMonad m => ParserT Text st m Char
1713+nonEndline :: PandocMonad m => ParserT Sources st m Char
17181714nonEndline = satisfy (/='\n')
17191715
17201716str :: PandocMonad m => MarkdownParser m (F Inlines)
17211717str = do
17221718 result <- mconcat <$> many1
1723− ( take1WhileP isAlphaNum
1719+ ( T.pack <$> (many1 alphaNum)
17241720 <|> "." <$ try (char '.' <* notFollowedBy (char '.')) )
17251721 updateLastStrPos
17261722 (do guardEnabled Ext_smart
17271723 abbrevs <- getOption readerAbbreviations
1728− if not (T.null result) && T.last result == '.' && result `Set.member` abbrevs
1724+ if result `Set.member` abbrevs
17291725 then try (do ils <- whitespace
17301726 notFollowedBy (() <$ cite <|> () <$ note)
17311727 -- ?? lookAhead alphaNum
@@ -1844,9 +1840,12 @@ regLink :: PandocMonad m
18441840 -> MarkdownParser m (F Inlines)
18451841regLink constructor lab = try $ do
18461842 (src, tit) <- source
1843+ rebase <- option False (True <$ guardEnabled Ext_rebase_relative_paths)
1844+ pos <- getPosition
1845+ let src' = if rebase then rebasePath pos src else src
18471846 attr <- option nullAttr $
18481847 guardEnabled Ext_link_attributes >> attributes
1849− return $ constructor attr src tit <$> lab
1848+ return $ constructor attr src' tit <$> lab
18501849
18511850-- a link like [this][ref] or [this][] or [this]
18521851referenceLink :: PandocMonad m
@@ -1886,7 +1885,8 @@ referenceLink constructor (lab, raw) = do
18861885 Just ((src, tit), _) -> constructor nullAttr src tit <$> lab
18871886 Nothing -> makeFallback
18881887 else makeFallback
1889− Just ((src,tit), attr) -> constructor attr src tit <$> lab
1888+ Just ((src,tit), attr) ->
1889+ constructor attr src tit <$> lab
18901890
18911891dropBrackets :: Text -> Text
18921892dropBrackets = dropRB . dropLB
@@ -1919,15 +1919,33 @@ autoLink = try $ do
19191919 return $ return $ B.linkWith attr (src <> escapeURI extra) ""
19201920 (B.str $ orig <> extra)
19211921
1922+-- | Rebase a relative path, by adding the (relative) directory
1923+-- of the containing source position. Absolute links and URLs
1924+-- are untouched.
1925+rebasePath :: SourcePos -> Text -> Text
1926+rebasePath pos path = do
1927+ let fp = sourceName pos
1928+ isFragment = T.take 1 path == "#"
1929+ path' = T.unpack path
1930+ isAbsolutePath = Posix.isAbsolute path' || Windows.isAbsolute path'
1931+ in if T.null path || isFragment || isAbsolutePath || isURI path
1932+ then path
1933+ else
1934+ case takeDirectory fp of
1935+ "" -> path
1936+ "." -> path
1937+ d -> T.pack d <> "/" <> path
1938+
19221939image :: PandocMonad m => MarkdownParser m (F Inlines)
19231940image = try $ do
19241941 char '!'
19251942 (lab,raw) <- reference
19261943 defaultExt <- getOption readerDefaultImageExtension
1927− let constructor attr' src = case takeExtension (T.unpack src) of
1928− "" -> B.imageWith attr' (T.pack $ addExtension (T.unpack src)
1929− $ T.unpack defaultExt)
1930− _ -> B.imageWith attr' src
1944+ let constructor attr' src =
1945+ case takeExtension (T.unpack src) of
1946+ "" -> B.imageWith attr' (T.pack $ addExtension (T.unpack src)
1947+ $ T.unpack defaultExt)
1948+ _ -> B.imageWith attr' src
19311949 regLink constructor lab <|> referenceLink constructor (lab,raw)
19321950
19331951note :: PandocMonad m => MarkdownParser m (F Inlines)
@@ -1973,7 +1991,7 @@ rawLaTeXInline' = do
19731991 s <- rawLaTeXInline
19741992 return $ return $ B.rawInline "tex" s -- "tex" because it might be context
19751993
1976−rawConTeXtEnvironment :: PandocMonad m => ParserT Text st m Text
1994+rawConTeXtEnvironment :: PandocMonad m => ParserT Sources st m Text
19771995rawConTeXtEnvironment = try $ do
19781996 string "\\start"
19791997 completion <- inBrackets (letter <|> digit <|> spaceChar)
@@ -1982,7 +2000,7 @@ rawConTeXtEnvironment = try $ do
19822000 (try $ string "\\stop" >> textStr completion)
19832001 return $ "\\start" <> completion <> T.concat contents <> "\\stop" <> completion
19842002
1985−inBrackets :: PandocMonad m => ParserT Text st m Char -> ParserT Text st m Text
2003+inBrackets :: PandocMonad m => ParserT Sources st m Char -> ParserT Sources st m Text
19862004inBrackets parser = do
19872005 char '['
19882006 contents <- manyChar parser
@@ -2102,7 +2120,7 @@ cite = do
21022120
21032121textualCite :: PandocMonad m => MarkdownParser m (F Inlines)
21042122textualCite = try $ do
2105− (suppressAuthor, key) <- citeKey
2123+ (suppressAuthor, key) <- citeKey True
21062124 -- If this is a reference to an earlier example list item,
21072125 -- then don't parse it as a citation. If the example list
21082126 -- item comes later, we'll parse it here and figure out in
@@ -2182,7 +2200,7 @@ prefix = trimInlinesF . mconcat <$>
21822200 manyTill inline (char ']'
21832201 <|> lookAhead
21842202 (try $ do optional (try (char ';' >> spnl))
2185− citeKey
2203+ citeKey True
21862204 return ']'))
21872205
21882206citeList :: PandocMonad m => MarkdownParser m (F [Citation])
@@ -2191,7 +2209,7 @@ citeList = fmap sequence $ sepBy1 citation (try $ char ';' >> spnl)
21912209citation :: PandocMonad m => MarkdownParser m (F Citation)
21922210citation = try $ do
21932211 pref <- prefix
2194− (suppress_author, key) <- citeKey
2212+ (suppress_author, key) <- citeKey True
21952213 suff <- suffix
21962214 noteNum <- stateNoteNumber <$> getState
21972215 return $ do
@@ -2210,28 +2228,30 @@ citation = try $ do
22102228smart :: PandocMonad m => MarkdownParser m (F Inlines)
22112229smart = do
22122230 guardEnabled Ext_smart
2213− doubleQuoted <|> singleQuoted <|>
2214− choice (map (return <$>) [apostrophe, dash, ellipses])
2231+ doubleQuoted <|> singleQuoted <|> (return <$> doubleCloseQuote) <|>
2232+ (return <$> apostrophe) <|> (return <$> dash) <|> (return <$> ellipses)
22152233
22162234singleQuoted :: PandocMonad m => MarkdownParser m (F Inlines)
2217−singleQuoted = try $ do
2235+singleQuoted = do
22182236 singleQuoteStart
2219− withQuoteContext InSingleQuote $
2237+ (try (withQuoteContext InSingleQuote $
22202238 fmap B.singleQuoted . trimInlinesF . mconcat <$>
2221− many1Till inline singleQuoteEnd
2239+ many1Till inline singleQuoteEnd))
2240+ <|> (return (return (B.str "\8217")))
22222241
22232242-- doubleQuoted will handle regular double-quoted sections, as well
22242243-- as dialogues with an open double-quote without a close double-quote
22252244-- in the same paragraph.
22262245doubleQuoted :: PandocMonad m => MarkdownParser m (F Inlines)
2227−doubleQuoted = try $ do
2246+doubleQuoted = do
22282247 doubleQuoteStart
2229− withQuoteContext InDoubleQuote $
2248+ (try (withQuoteContext InDoubleQuote $
22302249 fmap B.doubleQuoted . trimInlinesF . mconcat <$>
2231− many1Till inline doubleQuoteEnd
2250+ many1Till inline doubleQuoteEnd))
2251+ <|> (return (return (B.str "\8220")))
22322252
22332253toRow :: [Blocks] -> Row
22342254toRow = Row nullAttr . map B.simpleCell
22352255
22362256toHeaderRow :: [Blocks] -> [Row]
2237−toHeaderRow l = [toRow l | not (null l)]
2257+toHeaderRow l = [toRow l | not (null l) && not (all null l)]
src/Text/Pandoc/Readers/MediaWiki.hs+16−7
@@ -36,17 +36,18 @@ import Text.Pandoc.Logging
3636import Text.Pandoc.Options
3737import Text.Pandoc.Parsing hiding (nested)
3838import Text.Pandoc.Readers.HTML (htmlTag, isBlockTag, isCommentTag)
39−import Text.Pandoc.Shared (crFilter, safeRead, stringify, stripTrailingNewlines,
39+import Text.Pandoc.Shared (safeRead, stringify, stripTrailingNewlines,
4040 trim, splitTextBy, tshow)
4141import Text.Pandoc.Walk (walk)
4242import Text.Pandoc.XML (fromEntities)
4343
4444-- | Read mediawiki from an input string and return a Pandoc document.
45−readMediaWiki :: PandocMonad m
46− => ReaderOptions -- ^ Reader options
47− -> Text -- ^ String to parse (assuming @'\n'@ line endings)
45+readMediaWiki :: (PandocMonad m, ToSources a)
46+ => ReaderOptions
47+ -> a
4848 -> m Pandoc
4949readMediaWiki opts s = do
50+ let sources = toSources s
5051 parsed <- readWithM parseMediaWiki MWState{ mwOptions = opts
5152 , mwMaxNestingLevel = 4
5253 , mwNextLinkNumber = 1
@@ -55,7 +56,7 @@ readMediaWiki opts s = do
5556 , mwLogMessages = []
5657 , mwInTT = False
5758 }
58− (crFilter s <> "\n")
59+ sources
5960 case parsed of
6061 Right result -> return result
6162 Left e -> throwError e
@@ -69,7 +70,7 @@ data MWState = MWState { mwOptions :: ReaderOptions
6970 , mwInTT :: Bool
7071 }
7172
72−type MWParser m = ParserT Text MWState m
73+type MWParser m = ParserT Sources MWState m
7374
7475instance HasReaderOptions MWState where
7576 extractReaderOptions = mwOptions
@@ -112,12 +113,14 @@ newBlockTags = ["haskell","syntaxhighlight","source","gallery","references"]
112113isBlockTag' :: Tag Text -> Bool
113114isBlockTag' tag@(TagOpen t _) = (isBlockTag tag || t `elem` newBlockTags) &&
114115 t `notElem` eitherBlockOrInline
116+isBlockTag' (TagClose "ref") = True -- needed so 'special' doesn't parse it
115117isBlockTag' tag@(TagClose t) = (isBlockTag tag || t `elem` newBlockTags) &&
116118 t `notElem` eitherBlockOrInline
117119isBlockTag' tag = isBlockTag tag
118120
119121isInlineTag' :: Tag Text -> Bool
120122isInlineTag' (TagComment _) = True
123+isInlineTag' (TagClose "ref") = False -- see below inlineTag
121124isInlineTag' t = not (isBlockTag' t)
122125
123126eitherBlockOrInline :: [Text]
@@ -554,11 +557,17 @@ variable = try $ do
554557 contents <- manyTillChar anyChar (try $ string "}}}")
555558 return $ "{{{" <> contents <> "}}}"
556559
560+singleParaToPlain :: Blocks -> Blocks
561+singleParaToPlain bs =
562+ case B.toList bs of
563+ [Para ils] -> B.fromList [Plain ils]
564+ _ -> bs
565+
557566inlineTag :: PandocMonad m => MWParser m Inlines
558567inlineTag = do
559568 (tag, _) <- lookAhead $ htmlTag isInlineTag'
560569 case tag of
561− TagOpen "ref" _ -> B.note . B.plain <$> inlinesInTags "ref"
570+ TagOpen "ref" _ -> B.note . singleParaToPlain <$> blocksInTags "ref"
562571 TagOpen "nowiki" _ -> try $ do
563572 (_,raw) <- htmlTag (~== tag)
564573 if T.any (== '/') raw
src/Text/Pandoc/Readers/Metadata.hs+44−17
@@ -14,6 +14,7 @@ Parse YAML/JSON metadata to 'Pandoc' 'Meta'.
1414module Text.Pandoc.Readers.Metadata (
1515 yamlBsToMeta,
1616 yamlBsToRefs,
17+ yamlMetaBlock,
1718 yamlMap ) where
1819
1920import Control.Monad
@@ -30,11 +31,13 @@ import Text.Pandoc.Definition
3031import Text.Pandoc.Error
3132import Text.Pandoc.Parsing hiding (tableWith)
3233import Text.Pandoc.Shared
34+import qualified Data.Text.Lazy as TL
35+import qualified Text.Pandoc.UTF8 as UTF8
3336
34−yamlBsToMeta :: PandocMonad m
35− => ParserT Text ParserState m (F MetaValue)
37+yamlBsToMeta :: (PandocMonad m, HasLastStrPosition st)
38+ => ParserT Sources st m (Future st MetaValue)
3639 -> BL.ByteString
37− -> ParserT Text ParserState m (F Meta)
40+ -> ParserT Sources st m (Future st Meta)
3841yamlBsToMeta pMetaValue bstr = do
3942 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of
4043 Right (YAML.Doc (YAML.Mapping _ _ o):_)
@@ -42,6 +45,9 @@ yamlBsToMeta pMetaValue bstr = do
4245 Right [] -> return . return $ mempty
4346 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]
4447 -> return . return $ mempty
48+ -- the following is what we get from a comment:
49+ Right [YAML.Doc (YAML.Scalar _ (YAML.SUnknown _ ""))]
50+ -> return . return $ mempty
4551 Right _ -> Prelude.fail "expected YAML object"
4652 Left (yamlpos, err')
4753 -> do pos <- getPosition
@@ -63,11 +69,11 @@ lookupYAML t (YAML.Mapping _ _ m) =
6369lookupYAML _ _ = Nothing
6470
6571-- Returns filtered list of references.
66−yamlBsToRefs :: PandocMonad m
67− => ParserT Text ParserState m (F MetaValue)
72+yamlBsToRefs :: (PandocMonad m, HasLastStrPosition st)
73+ => ParserT Sources st m (Future st MetaValue)
6874 -> (Text -> Bool) -- ^ Filter for id
6975 -> BL.ByteString
70− -> ParserT Text ParserState m (F [MetaValue])
76+ -> ParserT Sources st m (Future st [MetaValue])
7177yamlBsToRefs pMetaValue idpred bstr =
7278 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of
7379 Right (YAML.Doc o@YAML.Mapping{}:_)
@@ -95,8 +101,12 @@ yamlBsToRefs pMetaValue idpred bstr =
95101 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]
96102 -> return . return $ mempty
97103 Right _ -> Prelude.fail "expecting YAML object"
98− Left (_pos, err')
99− -> Prelude.fail err'
104+ Left (yamlpos, err')
105+ -> do pos <- getPosition
106+ setPosition $ incSourceLine
107+ (setSourceColumn pos (YE.posColumn yamlpos))
108+ (YE.posLine yamlpos - 1)
109+ Prelude.fail err'
100110
101111
102112nodeToKey :: YAML.Node YE.Pos -> Maybe Text
@@ -104,10 +114,10 @@ nodeToKey (YAML.Scalar _ (YAML.SStr t)) = Just t
104114nodeToKey (YAML.Scalar _ (YAML.SUnknown _ t)) = Just t
105115nodeToKey _ = Nothing
106116
107−normalizeMetaValue :: PandocMonad m
108− => ParserT Text ParserState m (F MetaValue)
117+normalizeMetaValue :: (PandocMonad m, HasLastStrPosition st)
118+ => ParserT Sources st m (Future st MetaValue)
109119 -> Text
110− -> ParserT Text ParserState m (F MetaValue)
120+ -> ParserT Sources st m (Future st MetaValue)
111121normalizeMetaValue pMetaValue x =
112122 -- Note: a standard quoted or unquoted YAML value will
113123 -- not end in a newline, but a "block" set off with
@@ -129,10 +139,10 @@ checkBoolean t
129139 | t == T.pack "false" || t == T.pack "False" || t == T.pack "FALSE" = Just False
130140 | otherwise = Nothing
131141
132−yamlToMetaValue :: PandocMonad m
133− => ParserT Text ParserState m (F MetaValue)
142+yamlToMetaValue :: (PandocMonad m, HasLastStrPosition st)
143+ => ParserT Sources st m (Future st MetaValue)
134144 -> YAML.Node YE.Pos
135− -> ParserT Text ParserState m (F MetaValue)
145+ -> ParserT Sources st m (Future st MetaValue)
136146yamlToMetaValue pMetaValue (YAML.Scalar _ x) =
137147 case x of
138148 YAML.SStr t -> normalizeMetaValue pMetaValue t
@@ -152,10 +162,10 @@ yamlToMetaValue pMetaValue (YAML.Mapping _ _ o) =
152162 fmap MetaMap <$> yamlMap pMetaValue o
153163yamlToMetaValue _ _ = return $ return $ MetaString ""
154164
155−yamlMap :: PandocMonad m
156− => ParserT Text ParserState m (F MetaValue)
165+yamlMap :: (PandocMonad m, HasLastStrPosition st)
166+ => ParserT Sources st m (Future st MetaValue)
157167 -> M.Map (YAML.Node YE.Pos) (YAML.Node YE.Pos)
158− -> ParserT Text ParserState m (F (M.Map Text MetaValue))
168+ -> ParserT Sources st m (Future st (M.Map Text MetaValue))
159169yamlMap pMetaValue o = do
160170 kvs <- forM (M.toList o) $ \(key, v) -> do
161171 k <- maybe (throwError $ PandocParseError
@@ -171,3 +181,20 @@ yamlMap pMetaValue o = do
171181 return $ do
172182 v' <- fv
173183 return (k, v')
184+
185+-- | Parse a YAML metadata block using the supplied 'MetaValue' parser.
186+yamlMetaBlock :: (HasLastStrPosition st, PandocMonad m)
187+ => ParserT Sources st m (Future st MetaValue)
188+ -> ParserT Sources st m (Future st Meta)
189+yamlMetaBlock parser = try $ do
190+ string "---"
191+ blankline
192+ notFollowedBy blankline -- if --- is followed by a blank it's an HRULE
193+ rawYamlLines <- manyTill anyLine stopLine
194+ -- by including --- and ..., we allow yaml blocks with just comments:
195+ let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))
196+ optional blanklines
197+ yamlBsToMeta parser $ UTF8.fromTextLazy $ TL.fromStrict rawYaml
198+
199+stopLine :: Monad m => ParserT Sources st m ()
200+stopLine = try $ (string "---" <|> string "...") >> blankline >> return ()
src/Text/Pandoc/Readers/Muse.hs+10−9
@@ -36,19 +36,20 @@ import Text.Pandoc.Definition
3636import Text.Pandoc.Error (PandocError (PandocParsecError))
3737import Text.Pandoc.Logging
3838import Text.Pandoc.Options
39−import Text.Pandoc.Parsing hiding (F)
40−import Text.Pandoc.Shared (crFilter, trimr, tshow)
39+import Text.Pandoc.Parsing
40+import Text.Pandoc.Shared (trimr, tshow)
4141
4242-- | Read Muse from an input string and return a Pandoc document.
43−readMuse :: PandocMonad m
43+readMuse :: (PandocMonad m, ToSources a)
4444 => ReaderOptions
45− -> Text
45+ -> a
4646 -> m Pandoc
4747readMuse opts s = do
48− let input = crFilter s
49− res <- flip runReaderT def $ runParserT parseMuse def{ museOptions = opts } "source" input
48+ let sources = toSources s
49+ res <- flip runReaderT def $ runParserT parseMuse def{ museOptions = opts }
50+ (initialSourceName sources) sources
5051 case res of
51− Left e -> throwError $ PandocParsecError input e
52+ Left e -> throwError $ PandocParsecError sources e
5253 Right d -> return d
5354
5455type F = Future MuseState
@@ -82,7 +83,7 @@ instance Default MuseEnv where
8283 , museInPara = False
8384 }
8485
85−type MuseParser m = ParserT Text MuseState (ReaderT MuseEnv m)
86+type MuseParser m = ParserT Sources MuseState (ReaderT MuseEnv m)
8687
8788instance HasReaderOptions MuseState where
8889 extractReaderOptions = museOptions
@@ -155,7 +156,7 @@ firstColumn = getPosition >>= \pos -> guard (sourceColumn pos == 1)
155156-- * Parsers
156157
157158-- | Parse end-of-line, which can be either a newline or end-of-file.
158−eol :: Stream s m Char => ParserT s st m ()
159+eol :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
159160eol = void newline <|> eof
160161
161162getIndent :: PandocMonad m
src/Text/Pandoc/Readers/Native.hs+7−5
@@ -21,6 +21,7 @@ import Control.Monad.Except (throwError)
2121import Data.Text (Text)
2222import Text.Pandoc.Class.PandocMonad (PandocMonad)
2323import Text.Pandoc.Error
24+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2425
2526-- | Read native formatted text and return a Pandoc document.
2627-- The input may be a full pandoc document, a block list, a block,
@@ -32,14 +33,15 @@ import Text.Pandoc.Error
3233--
3334-- > Pandoc nullMeta [Plain [Str "hi"]]
3435--
35−readNative :: PandocMonad m
36+readNative :: (PandocMonad m, ToSources a)
3637 => ReaderOptions
37− -> Text -- ^ String to parse (assuming @'\n'@ line endings)
38+ -> a
3839 -> m Pandoc
3940readNative _ s =
40− case maybe (Pandoc nullMeta <$> readBlocks s) Right (safeRead s) of
41− Right doc -> return doc
42− Left _ -> throwError $ PandocParseError "couldn't read native"
41+ let t = sourcesToText . toSources $ s
42+ in case maybe (Pandoc nullMeta <$> readBlocks t) Right (safeRead t) of
43+ Right doc -> return doc
44+ Left _ -> throwError $ PandocParseError "couldn't read native"
4345
4446readBlocks :: Text -> Either PandocError [Block]
4547readBlocks s = maybe ((:[]) <$> readBlock s) Right (safeRead s)
src/Text/Pandoc/Readers/OPML.hs+9−4
@@ -24,7 +24,8 @@ import Text.Pandoc.Options
2424import Text.Pandoc.Error (PandocError(..))
2525import Text.Pandoc.Readers.HTML (readHtml)
2626import Text.Pandoc.Readers.Markdown (readMarkdown)
27−import Text.Pandoc.Shared (crFilter, blocksToInlines')
27+import Text.Pandoc.Shared (blocksToInlines')
28+import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2829import Text.Pandoc.XML.Light
2930import Control.Monad.Except (throwError)
3031
@@ -46,10 +47,14 @@ instance Default OPMLState where
4647 , opmlOptions = def
4748 }
4849
49−readOPML :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
50+readOPML :: (PandocMonad m, ToSources a)
51+ => ReaderOptions
52+ -> a
53+ -> m Pandoc
5054readOPML opts inp = do
51− (bs, st') <- runStateT
52− (case parseXMLContents (TL.fromStrict (crFilter inp)) of
55+ let sources = toSources inp
56+ (bs, st') <-
57+ runStateT (case parseXMLContents (TL.fromStrict . sourcesToText $ sources) of
5358 Left msg -> throwError $ PandocXMLError "" msg
5459 Right ns -> mapM parseBlock ns)
5560 def{ opmlOptions = opts }
src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2
@@ -22,7 +22,7 @@ module Text.Pandoc.Readers.Odt.Arrows.State where
2222import Control.Arrow
2323import qualified Control.Category as Cat
2424import Control.Monad
25−
25+import Data.List (foldl')
2626import Text.Pandoc.Readers.Odt.Arrows.Utils
2727import Text.Pandoc.Readers.Odt.Generic.Fallible
2828
@@ -122,7 +122,7 @@ iterateS a = ArrowState $ \(s,f) -> foldr a' (s,mzero) f
122122iterateSL :: (Foldable f, MonadPlus m)
123123 => ArrowState s x y
124124 -> ArrowState s (f x) (m y)
125−iterateSL a = ArrowState $ \(s,f) -> foldl a' (s,mzero) f
125+iterateSL a = ArrowState $ \(s,f) -> foldl' a' (s,mzero) f
126126 where a' (s',m) x = second (mplus m.return) $ runArrowState a (s',x)
127127
128128
src/Text/Pandoc/Readers/Odt/ContentReader.hs+12−7
@@ -1,4 +1,5 @@
11{-# LANGUAGE Arrows #-}
2+{-# LANGUAGE CPP #-}
23{-# LANGUAGE DeriveFoldable #-}
34{-# LANGUAGE GeneralizedNewtypeDeriving #-}
45{-# LANGUAGE PatternGuards #-}
@@ -33,7 +34,7 @@ import Data.List (find)
3334import qualified Data.Map as M
3435import qualified Data.Text as T
3536import Data.Maybe
36−import Data.Semigroup (First(..), Option(..))
37+import Data.Monoid (Alt (..))
3738
3839import Text.TeXMath (readMathML, writeTeX)
3940import qualified Text.Pandoc.XML.Light as XML
@@ -505,13 +506,11 @@ type InlineMatcher = ElementMatcher Inlines
505506
506507type BlockMatcher = ElementMatcher Blocks
507508
508−
509−newtype FirstMatch a = FirstMatch (Option (First a))
510− deriving (Foldable, Monoid, Semigroup)
509+newtype FirstMatch a = FirstMatch (Alt Maybe a)
510+ deriving (Foldable, Monoid, Semigroup)
511511
512512firstMatch :: a -> FirstMatch a
513−firstMatch = FirstMatch . Option . Just . First
514−
513+firstMatch = FirstMatch . Alt . Just
515514
516515--
517516matchingElement :: (Monoid e)
@@ -577,7 +576,10 @@ read_spaces = matchingElement NsText "s" (
577576read_line_break :: InlineMatcher
578577read_line_break = matchingElement NsText "line-break"
579578 $ returnV linebreak
580−
579+--
580+read_tab :: InlineMatcher
581+read_tab = matchingElement NsText "tab"
582+ $ returnV space
581583--
582584read_span :: InlineMatcher
583585read_span = matchingElement NsText "span"
@@ -585,6 +587,7 @@ read_span = matchingElement NsText "span"
585587 $ matchChildContent [ read_span
586588 , read_spaces
587589 , read_line_break
590+ , read_tab
588591 , read_link
589592 , read_note
590593 , read_citation
@@ -604,6 +607,7 @@ read_paragraph = matchingElement NsText "p"
604607 $ matchChildContent [ read_span
605608 , read_spaces
606609 , read_line_break
610+ , read_tab
607611 , read_link
608612 , read_note
609613 , read_citation
@@ -630,6 +634,7 @@ read_header = matchingElement NsText "h"
630634 children <- ( matchChildContent [ read_span
631635 , read_spaces
632636 , read_line_break
637+ , read_tab
633638 , read_link
634639 , read_note
635640 , read_citation
src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+2−2
@@ -64,12 +64,12 @@ import qualified Data.Map as M
6464import Data.Text (Text)
6565import Data.Default
6666import Data.Maybe
67+import Data.List (foldl')
6768
6869import qualified Text.Pandoc.XML.Light as XML
6970
7071import Text.Pandoc.Readers.Odt.Arrows.State
7172import Text.Pandoc.Readers.Odt.Arrows.Utils
72−
7373import Text.Pandoc.Readers.Odt.Generic.Namespaces
7474import Text.Pandoc.Readers.Odt.Generic.Utils
7575import Text.Pandoc.Readers.Odt.Generic.Fallible
@@ -293,7 +293,7 @@ readNSattributes = fromState $ \state -> maybe (state, failEmpty )
293293 => XMLConverterState nsID extraState
294294 -> Maybe (XMLConverterState nsID extraState)
295295 extractNSAttrs startState
296− = foldl (\state d -> state >>= addNS d)
296+ = foldl' (\state d -> state >>= addNS d)
297297 (Just startState)
298298 nsAttribs
299299 where nsAttribs = mapMaybe readNSattr (XML.elAttribs element)
src/Text/Pandoc/Readers/Odt/StyleReader.hs+8−7
@@ -44,7 +44,7 @@ import Control.Arrow
4444
4545import Data.Default
4646import qualified Data.Foldable as F
47−import Data.List (unfoldr)
47+import Data.List (unfoldr, foldl')
4848import qualified Data.Map as M
4949import Data.Maybe
5050import Data.Text (Text)
@@ -120,7 +120,7 @@ fontPitchReader = executeInSub NsOffice "font-face-decls" (
120120 &&&
121121 lookupDefaultingAttr NsStyle "font-pitch"
122122 ))
123− >>?^ ( M.fromList . foldl accumLegalPitches [] )
123+ >>?^ ( M.fromList . foldl' accumLegalPitches [] )
124124 ) `ifFailedDo` returnV (Right M.empty)
125125 where accumLegalPitches ls (Nothing,_) = ls
126126 accumLegalPitches ls (Just n,p) = (n,p):ls
@@ -563,12 +563,13 @@ readListLevelStyle levelType = readAttr NsText "level"
563563
564564--
565565chooseMostSpecificListLevelStyle :: S.Set ListLevelStyle -> Maybe ListLevelStyle
566−chooseMostSpecificListLevelStyle ls | ls == mempty = Nothing
567− | otherwise = Just ( F.foldr1 select ls )
566+chooseMostSpecificListLevelStyle ls = F.foldr select Nothing ls
568567 where
569− select ( ListLevelStyle t1 p1 s1 f1 b1 )
570− ( ListLevelStyle t2 p2 s2 f2 _ )
571− = ListLevelStyle (select' t1 t2) (p1 <|> p2) (s1 <|> s2) (selectLinf f1 f2) b1
568+ select l Nothing = Just l
569+ select ( ListLevelStyle t1 p1 s1 f1 b1 )
570+ ( Just ( ListLevelStyle t2 p2 s2 f2 _ ))
571+ = Just $ ListLevelStyle (select' t1 t2) (p1 <|> p2) (s1 <|> s2)
572+ (selectLinf f1 f2) b1
572573 select' LltNumbered _ = LltNumbered
573574 select' _ LltNumbered = LltNumbered
574575 select' _ _ = LltBullet
src/Text/Pandoc/Readers/Org.hs+4−7
@@ -18,22 +18,19 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
1818import Text.Pandoc.Definition
1919import Text.Pandoc.Options
2020import Text.Pandoc.Parsing (reportLogMessages)
21−import Text.Pandoc.Shared (crFilter)
22−
21+import Text.Pandoc.Sources (ToSources(..), ensureFinalNewlines)
2322import Control.Monad.Except (throwError)
2423import Control.Monad.Reader (runReaderT)
2524
26−import Data.Text (Text)
27−
2825-- | Parse org-mode string and return a Pandoc document.
29−readOrg :: PandocMonad m
26+readOrg :: (PandocMonad m, ToSources a)
3027 => ReaderOptions -- ^ Reader options
31− -> Text -- ^ String to parse (assuming @'\n'@ line endings)
28+ -> a
3229 -> m Pandoc
3330readOrg opts s = do
3431 parsed <- flip runReaderT def $
3532 readWithM parseOrg (optionsToParserState opts)
36− (crFilter s <> "\n\n")
33+ (ensureFinalNewlines 2 (toSources s))
3734 case parsed of
3835 Right result -> return result
3936 Left e -> throwError e
src/Text/Pandoc/Readers/Org/Blocks.hs+7−4
@@ -39,10 +39,12 @@ import Data.Functor (($>))
3939import Data.List (foldl', intersperse)
4040import Data.Maybe (fromMaybe, isJust, isNothing)
4141import Data.Text (Text)
42−
42+import Data.List.NonEmpty (nonEmpty)
43+import System.FilePath
4344import qualified Data.Text as T
4445import qualified Text.Pandoc.Builder as B
4546import qualified Text.Pandoc.Walk as Walk
47+import Text.Pandoc.Sources (ToSources(..))
4648
4749--
4850-- parsing blocks
@@ -526,7 +528,9 @@ include = try $ do
526528 _ -> nullAttr
527529 return $ pure . B.codeBlockWith attr <$> parseRaw
528530 _ -> return $ return . B.fromList . blockFilter params <$> blockList
529− insertIncludedFileF blocksParser ["."] filename
531+ currentDir <- takeDirectory . sourceName <$> getPosition
532+ insertIncludedFile blocksParser toSources
533+ [currentDir] filename Nothing Nothing
530534 where
531535 includeTarget :: PandocMonad m => OrgParser m FilePath
532536 includeTarget = do
@@ -542,8 +546,7 @@ include = try $ do
542546 in case (minlvl >>= safeRead :: Maybe Int) of
543547 Nothing -> blks
544548 Just lvl -> let levels = Walk.query headerLevel blks
545− -- CAVE: partial function in else
546− curMin = if null levels then 0 else minimum levels
549+ curMin = maybe 0 minimum $ nonEmpty levels
547550 in Walk.walk (shiftHeader (curMin - lvl)) blks
548551
549552 headerLevel :: Block -> [Int]
src/Text/Pandoc/Readers/Org/Inlines.hs+6−5
@@ -29,6 +29,7 @@ import Text.Pandoc.Definition
2929import Text.Pandoc.Options
3030import Text.Pandoc.Readers.LaTeX (inlineCommand, rawLaTeXInline)
3131import Text.TeXMath (DisplayType (..), readTeX, writePandoc)
32+import Text.Pandoc.Sources (ToSources(..))
3233import qualified Text.TeXMath.Readers.MathML.EntityMap as MathMLEntityMap
3334
3435import Control.Monad (guard, mplus, mzero, unless, void, when)
@@ -262,7 +263,7 @@ berkeleyCitationList = try $ do
262263 where
263264 citationListPart :: PandocMonad m => OrgParser m (F Inlines)
264265 citationListPart = fmap (trimInlinesF . mconcat) . try . many1 $ do
265− notFollowedBy' citeKey
266+ notFollowedBy' $ citeKey False
266267 notFollowedBy (oneOf ";]")
267268 inline
268269
@@ -277,7 +278,7 @@ berkeleyBareTag' = try $ void (string "cite")
277278
278279berkeleyTextualCite :: PandocMonad m => OrgParser m (F [Citation])
279280berkeleyTextualCite = try $ do
280− (suppressAuthor, key) <- citeKey
281+ (suppressAuthor, key) <- citeKey False
281282 returnF . return $ Citation
282283 { citationId = key
283284 , citationPrefix = mempty
@@ -350,7 +351,7 @@ citeList = sequence <$> sepEndBy1 citation (try $ char ';' *> skipSpaces)
350351citation :: PandocMonad m => OrgParser m (F Citation)
351352citation = try $ do
352353 pref <- prefix
353− (suppress_author, key) <- citeKey
354+ (suppress_author, key) <- citeKey False
354355 suff <- suffix
355356 return $ do
356357 x <- pref
@@ -367,7 +368,7 @@ citation = try $ do
367368 }
368369 where
369370 prefix = trimInlinesF . mconcat <$>
370− manyTill inline (char ']' <|> (']' <$ lookAhead citeKey))
371+ manyTill inline (char ']' <|> (']' <$ lookAhead (citeKey False)))
371372 suffix = try $ do
372373 hasSpace <- option False (notFollowedBy nonspaceChar >> return True)
373374 skipSpaces
@@ -802,7 +803,7 @@ inlineLaTeX = try $ do
802803 parseAsInlineLaTeX :: PandocMonad m
803804 => Text -> TeXExport -> OrgParser m (Maybe Inlines)
804805 parseAsInlineLaTeX cs = \case
805− TeXExport -> maybeRight <$> runParserT inlineCommand state "" cs
806+ TeXExport -> maybeRight <$> runParserT inlineCommand state "" (toSources cs)
806807 TeXIgnore -> return (Just mempty)
807808 TeXVerbatim -> return (Just $ B.str cs)
808809
src/Text/Pandoc/Readers/Org/Parsing.hs+5−5
@@ -63,8 +63,7 @@ module Text.Pandoc.Readers.Org.Parsing
6363 , ellipses
6464 , citeKey
6565 , gridTableWith
66− , insertIncludedFileF
67− -- * Re-exports from Text.Pandoc.Parsec
66+ , insertIncludedFile
6867 , runParser
6968 , runParserT
7069 , getInput
@@ -100,21 +99,22 @@ module Text.Pandoc.Readers.Org.Parsing
10099 , getState
101100 , updateState
102101 , SourcePos
102+ , sourceName
103103 , getPosition
104104 ) where
105105
106106import Data.Text (Text)
107107import Text.Pandoc.Readers.Org.ParserState
108108
109−import Text.Pandoc.Parsing hiding (F, anyLine, blanklines, newline,
110− parseFromString)
109+import Text.Pandoc.Parsing hiding (anyLine, blanklines, newline,
110+ parseFromString)
111111import qualified Text.Pandoc.Parsing as P
112112
113113import Control.Monad (guard)
114114import Control.Monad.Reader (ReaderT)
115115
116116-- | The parser used to read org files.
117−type OrgParser m = ParserT Text OrgParserState (ReaderT OrgParserLocal m)
117+type OrgParser m = ParserT Sources OrgParserState (ReaderT OrgParserLocal m)
118118
119119--
120120-- Adaptions and specializations of parsing utilities
src/Text/Pandoc/Readers/RST.hs+78−110
@@ -27,8 +27,7 @@ import Data.Text (Text)
2727import qualified Data.Text as T
2828import Text.Pandoc.Builder (Blocks, Inlines, fromList, setMeta, trimInlines)
2929import qualified Text.Pandoc.Builder as B
30−import Text.Pandoc.Class.PandocMonad (PandocMonad, fetchItem,
31− readFileFromDirs, getTimestamp)
30+import Text.Pandoc.Class.PandocMonad (PandocMonad, fetchItem, getTimestamp)
3231import Text.Pandoc.CSV (CSVOptions (..), defaultCSVOptions, parseCSV)
3332import Text.Pandoc.Definition
3433import Text.Pandoc.Error
@@ -38,25 +37,25 @@ import Text.Pandoc.Options
3837import Text.Pandoc.Parsing
3938import Text.Pandoc.Shared
4039import qualified Text.Pandoc.UTF8 as UTF8
41−import Text.Printf (printf)
4240import Data.Time.Format
41+import System.FilePath (takeDirectory)
4342
4443-- TODO:
4544-- [ ] .. parsed-literal
4645
4746-- | Parse reStructuredText string and return Pandoc document.
48−readRST :: PandocMonad m
47+readRST :: (PandocMonad m, ToSources a)
4948 => ReaderOptions -- ^ Reader options
50− -> Text -- ^ Text to parse (assuming @'\n'@ line endings)
49+ -> a
5150 -> m Pandoc
5251readRST opts s = do
5352 parsed <- readWithM parseRST def{ stateOptions = opts }
54− (crFilter s <> "\n\n")
53+ (ensureFinalNewlines 2 (toSources s))
5554 case parsed of
5655 Right result -> return result
5756 Left e -> throwError e
5857
59−type RSTParser m = ParserT Text ParserState m
58+type RSTParser m = ParserT Sources ParserState m
6059
6160--
6261-- Constants and data structure definitions
@@ -151,11 +150,19 @@ parseRST = do
151150 startPos <- getPosition
152151 -- go through once just to get list of reference keys and notes
153152 -- docMinusKeys is the raw document with blanks where the keys were...
154− docMinusKeys <- T.concat <$>
155− manyTill (referenceKey <|> anchorDef <|>
156− noteBlock <|> citationBlock <|>
157− (snd <$> withRaw comment) <|>
158− headerBlock <|> lineClump) eof
153+ let chunk = referenceKey
154+ <|> anchorDef
155+ <|> noteBlock
156+ <|> citationBlock
157+ <|> (snd <$> withRaw comment)
158+ <|> headerBlock
159+ <|> lineClump
160+ docMinusKeys <- Sources <$>
161+ manyTill (do pos <- getPosition
162+ t <- chunk
163+ return (pos, t)) eof
164+ -- UGLY: we collapse source position information.
165+ -- TODO: fix the parser to use the F monad instead of two passes
159166 setInput docMinusKeys
160167 setPosition startPos
161168 st' <- getState
@@ -348,7 +355,7 @@ singleHeader' = try $ do
348355-- hrule block
349356--
350357
351−hrule :: Monad m => ParserT Text st m Blocks
358+hrule :: Monad m => ParserT Sources st m Blocks
352359hrule = try $ do
353360 chr <- oneOf underlineChars
354361 count 3 (char chr)
@@ -363,7 +370,7 @@ hrule = try $ do
363370
364371-- read a line indented by a given string
365372indentedLine :: (HasReaderOptions st, Monad m)
366− => Int -> ParserT Text st m Text
373+ => Int -> ParserT Sources st m Text
367374indentedLine indents = try $ do
368375 lookAhead spaceChar
369376 gobbleAtMostSpaces indents
@@ -372,7 +379,7 @@ indentedLine indents = try $ do
372379-- one or more indented lines, possibly separated by blank lines.
373380-- any amount of indentation will work.
374381indentedBlock :: (HasReaderOptions st, Monad m)
375− => ParserT Text st m Text
382+ => ParserT Sources st m Text
376383indentedBlock = try $ do
377384 indents <- length <$> lookAhead (many1 spaceChar)
378385 lns <- many1 $ try $ do b <- option "" blanklines
@@ -381,20 +388,20 @@ indentedBlock = try $ do
381388 optional blanklines
382389 return $ T.unlines lns
383390
384−quotedBlock :: Monad m => ParserT Text st m Text
391+quotedBlock :: Monad m => ParserT Sources st m Text
385392quotedBlock = try $ do
386393 quote <- lookAhead $ oneOf "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
387394 lns <- many1 $ lookAhead (char quote) >> anyLine
388395 optional blanklines
389396 return $ T.unlines lns
390397
391−codeBlockStart :: Monad m => ParserT Text st m Char
398+codeBlockStart :: Monad m => ParserT Sources st m Char
392399codeBlockStart = string "::" >> blankline >> blankline
393400
394−codeBlock :: Monad m => ParserT Text ParserState m Blocks
401+codeBlock :: Monad m => ParserT Sources ParserState m Blocks
395402codeBlock = try $ codeBlockStart >> codeBlockBody
396403
397−codeBlockBody :: Monad m => ParserT Text ParserState m Blocks
404+codeBlockBody :: Monad m => ParserT Sources ParserState m Blocks
398405codeBlockBody = do
399406 lang <- stateRstHighlight <$> getState
400407 try $ B.codeBlockWith ("", maybeToList lang, []) . stripTrailingNewlines <$>
@@ -410,14 +417,14 @@ lhsCodeBlock = try $ do
410417 return $ B.codeBlockWith ("", ["haskell","literate"], [])
411418 $ T.intercalate "\n" lns
412419
413−latexCodeBlock :: Monad m => ParserT Text st m [Text]
420+latexCodeBlock :: Monad m => ParserT Sources st m [Text]
414421latexCodeBlock = try $ do
415422 try (latexBlockLine "\\begin{code}")
416423 many1Till anyLine (try $ latexBlockLine "\\end{code}")
417424 where
418425 latexBlockLine s = skipMany spaceChar >> string s >> blankline
419426
420−birdCodeBlock :: Monad m => ParserT Text st m [Text]
427+birdCodeBlock :: Monad m => ParserT Sources st m [Text]
421428birdCodeBlock = filterSpace <$> many1 birdTrackLine
422429 where filterSpace lns =
423430 -- if (as is normal) there is always a space after >, drop it
@@ -425,7 +432,7 @@ birdCodeBlock = filterSpace <$> many1 birdTrackLine
425432 then map (T.drop 1) lns
426433 else lns
427434
428−birdTrackLine :: Monad m => ParserT Text st m Text
435+birdTrackLine :: Monad m => ParserT Sources st m Text
429436birdTrackLine = char '>' >> anyLine
430437
431438--
@@ -446,64 +453,39 @@ encoding
446453-}
447454
448455includeDirective :: PandocMonad m
449− => Text -> [(Text, Text)] -> Text
456+ => Text
457+ -> [(Text, Text)]
458+ -> Text
450459 -> RSTParser m Blocks
451460includeDirective top fields body = do
452− let f = trim top
453− guard $ not (T.null f)
461+ let f = T.unpack $ trim top
462+ guard $ not $ null f
454463 guard $ T.null (trim body)
455− -- options
456− let (startLine :: Maybe Int) = lookup "start-line" fields >>= safeRead
457− let (endLine :: Maybe Int) = lookup "end-line" fields >>= safeRead
458− oldPos <- getPosition
459− oldInput <- getInput
460− containers <- stateContainers <$> getState
461− when (f `elem` containers) $
462− throwError $ PandocParseError $ "Include file loop at " <> tshow oldPos
463− updateState $ \s -> s{ stateContainers = f : stateContainers s }
464− mbContents <- readFileFromDirs ["."] $ T.unpack f
465− contentLines <- case mbContents of
466− Just s -> return $ T.lines s
467− Nothing -> do
468− logMessage $ CouldNotLoadIncludeFile f oldPos
469− return []
470− let numLines = length contentLines
471− let startLine' = case startLine of
472− Nothing -> 1
473− Just x | x >= 0 -> x
474− | otherwise -> numLines + x -- negative from end
475− let endLine' = case endLine of
476− Nothing -> numLines + 1
477− Just x | x >= 0 -> x
478− | otherwise -> numLines + x -- negative from end
479− let contentLines' = drop (startLine' - 1)
480− $ take (endLine' - 1) contentLines
481− let contentLines'' = (case trim <$> lookup "end-before" fields of
482− Just patt -> takeWhile (not . (patt `T.isInfixOf`))
483− Nothing -> id) .
484− (case trim <$> lookup "start-after" fields of
485− Just patt -> drop 1 .
486− dropWhile (not . (patt `T.isInfixOf`))
487− Nothing -> id) $ contentLines'
488− let contents' = T.unlines contentLines''
489− case lookup "code" fields of
490− Just lang -> do
491− let classes = maybe [] T.words (lookup "class" fields)
492− let ident = maybe "" trimr $ lookup "name" fields
493− codeblock ident classes fields (trimr lang) contents' False
494− Nothing -> case lookup "literal" fields of
495− Just _ -> return $ B.rawBlock "rst" contents'
496− Nothing -> do
497− setPosition $ newPos (T.unpack f) 1 1
498− setInput $ contents' <> "\n"
499− bs <- optional blanklines >>
500− (mconcat <$> many block)
501− setInput oldInput
502− setPosition oldPos
503− updateState $ \s -> s{ stateContainers =
504− tail $ stateContainers s }
505− return bs
506−
464+ let startLine = lookup "start-line" fields >>= safeRead
465+ let endLine = lookup "end-line" fields >>= safeRead
466+ let classes = maybe [] T.words (lookup "class" fields)
467+ let ident = maybe "" trimr $ lookup "name" fields
468+ let parser =
469+ case lookup "code" fields of
470+ Just lang ->
471+ (codeblock ident classes fields (trimr lang) False
472+ . sourcesToText) <$> getInput
473+ Nothing ->
474+ case lookup "literal" fields of
475+ Just _ -> B.rawBlock "rst" . sourcesToText <$> getInput
476+ Nothing -> parseBlocks
477+ let selectLines =
478+ (case trim <$> lookup "end-before" fields of
479+ Just patt -> takeWhile (not . (patt `T.isInfixOf`))
480+ Nothing -> id) .
481+ (case trim <$> lookup "start-after" fields of
482+ Just patt -> drop 1 .
483+ dropWhile (not . (patt `T.isInfixOf`))
484+ Nothing -> id)
485+ let toStream t =
486+ toSources [(f, T.unlines . selectLines . T.lines $ t)]
487+ currentDir <- takeDirectory . sourceName <$> getPosition
488+ insertIncludedFile parser toStream [currentDir] f startLine endLine
507489
508490--
509491-- list blocks
@@ -526,7 +508,7 @@ definitionList :: PandocMonad m => RSTParser m Blocks
526508definitionList = B.definitionList <$> many1 definitionListItem
527509
528510-- parses bullet list start and returns its length (inc. following whitespace)
529−bulletListStart :: Monad m => ParserT Text st m Int
511+bulletListStart :: Monad m => ParserT Sources st m Int
530512bulletListStart = try $ do
531513 notFollowedBy' hrule -- because hrules start out just like lists
532514 marker <- oneOf bulletListMarkers
@@ -613,8 +595,9 @@ comment = try $ do
613595 string ".."
614596 skipMany1 spaceChar <|> (() <$ lookAhead newline)
615597 -- notFollowedBy' directiveLabel -- comment comes after directive so unnec.
616− manyTill anyChar blanklines
598+ _ <- anyLine
617599 optional indentedBlock
600+ optional blanklines
618601 return mempty
619602
620603directiveLabel :: Monad m => RSTParser m Text
@@ -731,8 +714,8 @@ directive' = do
731714 "" -> stateRstHighlight def
732715 lang -> Just lang })
733716 x | x == "code" || x == "code-block" || x == "sourcecode" ->
734− codeblock name classes (map (second trimr) fields)
735− (trim top) body True
717+ return $ codeblock name classes (map (second trimr) fields)
718+ (trim top) True body
736719 "aafig" -> do
737720 let attribs = (name, ["aafig"], map (second trimr) fields)
738721 return $ B.codeBlockWith attribs $ stripTrailingNewlines body
@@ -1018,10 +1001,10 @@ toChunks = dropWhile T.null
10181001 then "\\begin{aligned}\n" <> s <> "\n\\end{aligned}"
10191002 else s
10201003
1021−codeblock :: Text -> [Text] -> [(Text, Text)] -> Text -> Text -> Bool
1022− -> RSTParser m Blocks
1023−codeblock ident classes fields lang body rmTrailingNewlines =
1024− return $ B.codeBlockWith attribs $ stripTrailingNewlines' body
1004+codeblock :: Text -> [Text] -> [(Text, Text)] -> Text -> Bool -> Text
1005+ -> Blocks
1006+codeblock ident classes fields lang rmTrailingNewlines body =
1007+ B.codeBlockWith attribs $ stripTrailingNewlines' body
10251008 where stripTrailingNewlines' = if rmTrailingNewlines
10261009 then stripTrailingNewlines
10271010 else id
@@ -1102,7 +1085,7 @@ quotedReferenceName = try $ do
11021085-- plus isolated (no two adjacent) internal hyphens, underscores,
11031086-- periods, colons and plus signs; no whitespace or other characters
11041087-- are allowed.
1105−simpleReferenceName :: Monad m => ParserT Text st m Text
1088+simpleReferenceName :: Monad m => ParserT Sources st m Text
11061089simpleReferenceName = do
11071090 x <- alphaNum
11081091 xs <- many $ alphaNum
@@ -1121,7 +1104,7 @@ referenceKey = do
11211104 -- return enough blanks to replace key
11221105 return $ T.replicate (sourceLine endPos - sourceLine startPos) "\n"
11231106
1124−targetURI :: Monad m => ParserT Text st m Text
1107+targetURI :: Monad m => ParserT Sources st m Text
11251108targetURI = do
11261109 skipSpaces
11271110 optional $ try $ newline >> notFollowedBy blankline
@@ -1159,8 +1142,10 @@ anonymousKey :: Monad m => RSTParser m ()
11591142anonymousKey = try $ do
11601143 oneOfStrings [".. __:", "__"]
11611144 src <- targetURI
1162− pos <- getPosition
1163− let key = toKey $ "_" <> T.pack (printf "%09d" (sourceLine pos))
1145+ -- we need to ensure that the keys are ordered by occurrence in
1146+ -- the document.
1147+ numKeys <- M.size . stateKeys <$> getState
1148+ let key = toKey $ "_" <> T.pack (show numKeys)
11641149 updateState $ \s -> s { stateKeys = M.insert key ((src,""), nullAttr) $
11651150 stateKeys s }
11661151
@@ -1249,13 +1234,13 @@ headerBlock = do
12491234-- Grid tables TODO:
12501235-- - column spans
12511236
1252−dashedLine :: Monad m => Char -> ParserT Text st m (Int, Int)
1237+dashedLine :: Monad m => Char -> ParserT Sources st m (Int, Int)
12531238dashedLine ch = do
12541239 dashes <- many1 (char ch)
12551240 sp <- many (char ' ')
12561241 return (length dashes, length $ dashes ++ sp)
12571242
1258−simpleDashedLines :: Monad m => Char -> ParserT Text st m [(Int,Int)]
1243+simpleDashedLines :: Monad m => Char -> ParserT Sources st m [(Int,Int)]
12591244simpleDashedLines ch = try $ many1 (dashedLine ch)
12601245
12611246-- Parse a table row separator
@@ -1381,7 +1366,7 @@ hyphens = do
13811366 -- don't want to treat endline after hyphen or dash as a space
13821367 return $ B.str result
13831368
1384−escapedChar :: Monad m => ParserT Text st m Inlines
1369+escapedChar :: Monad m => ParserT Sources st m Inlines
13851370escapedChar = do c <- escaped anyChar
13861371 return $ if c == ' ' || c == '\n' || c == '\r'
13871372 -- '\ ' is null in RST
@@ -1657,21 +1642,4 @@ note = try $ do
16571642 return $ B.note contents
16581643
16591644smart :: PandocMonad m => RSTParser m Inlines
1660−smart = do
1661− guardEnabled Ext_smart
1662− doubleQuoted <|> singleQuoted <|>
1663− choice [apostrophe, dash, ellipses]
1664−
1665−singleQuoted :: PandocMonad m => RSTParser m Inlines
1666−singleQuoted = try $ do
1667− singleQuoteStart
1668− withQuoteContext InSingleQuote $
1669− B.singleQuoted . trimInlines . mconcat <$>
1670− many1Till inline singleQuoteEnd
1671−
1672−doubleQuoted :: PandocMonad m => RSTParser m Inlines
1673−doubleQuoted = try $ do
1674− doubleQuoteStart
1675− withQuoteContext InDoubleQuote $
1676− B.doubleQuoted . trimInlines . mconcat <$>
1677− many1Till inline doubleQuoteEnd
1645+smart = smartPunctuation inline
src/Text/Pandoc/Readers/Roff.hs+16−18
@@ -42,7 +42,6 @@ import Text.Pandoc.Logging (LogMessage(..))
4242import Text.Pandoc.Options
4343import Text.Pandoc.Parsing
4444import Text.Pandoc.Shared (safeRead)
45−import Text.Parsec hiding (tokenPrim)
4645import Text.Pandoc.RoffChar (characterCodes, combiningAccents)
4746import qualified Data.Sequence as Seq
4847import qualified Data.Foldable as Foldable
@@ -122,16 +121,16 @@ instance Default RoffState where
122121 , afterConditional = False
123122 }
124123
125−type RoffLexer m = ParserT T.Text RoffState m
124+type RoffLexer m = ParserT Sources RoffState m
126125
127126--
128127-- Lexer: T.Text -> RoffToken
129128--
130129
131−eofline :: Stream s m Char => ParsecT s u m ()
130+eofline :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s u m ()
132131eofline = void newline <|> eof <|> () <$ lookAhead (string "\\}")
133132
134−spacetab :: Stream s m Char => ParsecT s u m Char
133+spacetab :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s u m Char
135134spacetab = char ' ' <|> char '\t'
136135
137136characterCodeMap :: M.Map T.Text Char
@@ -303,8 +302,7 @@ expandString = try $ do
303302 char '*'
304303 cs <- escapeArg <|> countChar 1 anyChar
305304 s <- linePartsToText <$> resolveText cs pos
306− getInput >>= setInput . (s <>)
307− return ()
305+ addToInput s
308306
309307-- Parses: '..'
310308quoteArg :: PandocMonad m => RoffLexer m T.Text
@@ -316,7 +314,7 @@ escFont = do
316314 font' <- if T.null font || font == "P"
317315 then prevFont <$> getState
318316 else return $ foldr processFontLetter defaultFontSpec $ T.unpack font
319− modifyState $ \st -> st{ prevFont = currentFont st
317+ updateState $ \st -> st{ prevFont = currentFont st
320318 , currentFont = font' }
321319 return [Font font']
322320 where
@@ -372,8 +370,8 @@ lexTable pos = do
372370 spaces
373371 opts <- try tableOptions <|> [] <$ optional (char ';')
374372 case lookup "tab" opts of
375− Just (T.uncons -> Just (c, _)) -> modifyState $ \st -> st{ tableTabChar = c }
376− _ -> modifyState $ \st -> st{ tableTabChar = '\t' }
373+ Just (T.uncons -> Just (c, _)) -> updateState $ \st -> st{ tableTabChar = c }
374+ _ -> updateState $ \st -> st{ tableTabChar = '\t' }
377375 spaces
378376 skipMany lexComment
379377 spaces
@@ -489,18 +487,18 @@ lexConditional mname = do
489487 ifPart <- do
490488 optional $ try $ char '\\' >> newline
491489 lexGroup
492− <|> do modifyState $ \s -> s{ afterConditional = True }
490+ <|> do updateState $ \s -> s{ afterConditional = True }
493491 t <- manToken
494− modifyState $ \s -> s{ afterConditional = False }
492+ updateState $ \s -> s{ afterConditional = False }
495493 return t
496494 case mbtest of
497495 Nothing -> do
498− putState st -- reset state, so we don't record macros in skipped section
496+ setState st -- reset state, so we don't record macros in skipped section
499497 report $ SkippedContent (T.cons '.' mname) pos
500498 return mempty
501499 Just True -> return ifPart
502500 Just False -> do
503− putState st
501+ setState st
504502 return mempty
505503
506504expression :: PandocMonad m => RoffLexer m (Maybe Bool)
@@ -515,7 +513,7 @@ expression = do
515513 _ -> Nothing
516514 where
517515 returnValue v = do
518− modifyState $ \st -> st{ lastExpression = v }
516+ updateState $ \st -> st{ lastExpression = v }
519517 return v
520518
521519lexGroup :: PandocMonad m => RoffLexer m RoffTokens
@@ -536,7 +534,7 @@ lexIncludeFile args = do
536534 result <- readFileFromDirs dirs $ T.unpack fp
537535 case result of
538536 Nothing -> report $ CouldNotLoadIncludeFile fp pos
539− Just s -> getInput >>= setInput . (s <>)
537+ Just s -> addToInput s
540538 return mempty
541539 [] -> return mempty
542540
@@ -564,13 +562,13 @@ lexStringDef args = do -- string definition
564562 (x:ys) -> do
565563 let ts = singleTok $ TextLine (intercalate [RoffStr " " ] ys)
566564 let stringName = linePartsToText x
567− modifyState $ \st ->
565+ updateState $ \st ->
568566 st{ customMacros = M.insert stringName ts (customMacros st) }
569567 return mempty
570568
571569lexMacroDef :: PandocMonad m => [Arg] -> RoffLexer m RoffTokens
572570lexMacroDef args = do -- macro definition
573− modifyState $ \st -> st{ roffMode = CopyMode }
571+ updateState $ \st -> st{ roffMode = CopyMode }
574572 (macroName, stopMacro) <-
575573 case args of
576574 (x : y : _) -> return (linePartsToText x, linePartsToText y)
@@ -584,7 +582,7 @@ lexMacroDef args = do -- macro definition
584582 _ <- lexArgs
585583 return ()
586584 ts <- mconcat <$> manyTill manToken stop
587− modifyState $ \st ->
585+ updateState $ \st ->
588586 st{ customMacros = M.insert macroName ts (customMacros st)
589587 , roffMode = NormalMode }
590588 return mempty
src/Text/Pandoc/Readers/TWiki.hs+7−27
@@ -28,22 +28,22 @@ import Text.Pandoc.Definition
2828import Text.Pandoc.Options
2929import Text.Pandoc.Parsing hiding (enclosed, nested)
3030import Text.Pandoc.Readers.HTML (htmlTag, isCommentTag)
31−import Text.Pandoc.Shared (crFilter, tshow)
31+import Text.Pandoc.Shared (tshow)
3232import Text.Pandoc.XML (fromEntities)
3333
3434-- | Read twiki from an input string and return a Pandoc document.
35−readTWiki :: PandocMonad m
35+readTWiki :: (PandocMonad m, ToSources a)
3636 => ReaderOptions
37− -> Text
37+ -> a
3838 -> m Pandoc
3939readTWiki opts s = do
40− res <- readWithM parseTWiki def{ stateOptions = opts }
41− (crFilter s <> "\n\n")
40+ let sources = ensureFinalNewlines 2 (toSources s)
41+ res <- readWithM parseTWiki def{ stateOptions = opts } sources
4242 case res of
4343 Left e -> throwError e
4444 Right d -> return d
4545
46−type TWParser = ParserT Text ParserState
46+type TWParser = ParserT Sources ParserState
4747
4848--
4949-- utility functions
@@ -469,27 +469,7 @@ symbol :: PandocMonad m => TWParser m B.Inlines
469469symbol = B.str <$> countChar 1 nonspaceChar
470470
471471smart :: PandocMonad m => TWParser m B.Inlines
472−smart = do
473− guardEnabled Ext_smart
474− doubleQuoted <|> singleQuoted <|>
475− choice [ apostrophe
476− , dash
477− , ellipses
478− ]
479−
480−singleQuoted :: PandocMonad m => TWParser m B.Inlines
481−singleQuoted = try $ do
482− singleQuoteStart
483− withQuoteContext InSingleQuote
484− (B.singleQuoted . B.trimInlines . mconcat <$> many1Till inline singleQuoteEnd)
485−
486−doubleQuoted :: PandocMonad m => TWParser m B.Inlines
487−doubleQuoted = try $ do
488− doubleQuoteStart
489− contents <- mconcat <$> many (try $ notFollowedBy doubleQuoteEnd >> inline)
490− withQuoteContext InDoubleQuote (doubleQuoteEnd >>
491− return (B.doubleQuoted $ B.trimInlines contents))
492− <|> return (B.str "\8220" B.<> contents)
472+smart = smartPunctuation inline
493473
494474link :: PandocMonad m => TWParser m B.Inlines
495475link = try $ do
src/Text/Pandoc/Readers/Textile.hs+94−88
@@ -38,7 +38,8 @@ module Text.Pandoc.Readers.Textile ( readTextile) where
3838import Control.Monad (guard, liftM)
3939import Control.Monad.Except (throwError)
4040import Data.Char (digitToInt, isUpper)
41−import Data.List (intersperse, transpose)
41+import Data.List (intersperse, transpose, foldl')
42+import Data.List.NonEmpty (NonEmpty(..), nonEmpty)
4243import Data.Text (Text)
4344import qualified Data.Text as T
4445import Text.HTML.TagSoup (Tag (..), fromAttrib)
@@ -52,30 +53,34 @@ import Text.Pandoc.Options
5253import Text.Pandoc.Parsing
5354import Text.Pandoc.Readers.HTML (htmlTag, isBlockTag, isInlineTag)
5455import Text.Pandoc.Readers.LaTeX (rawLaTeXBlock, rawLaTeXInline)
55−import Text.Pandoc.Shared (crFilter, trim, tshow)
56+import Text.Pandoc.Shared (trim, tshow)
5657
5758-- | Parse a Textile text and return a Pandoc document.
58−readTextile :: PandocMonad m
59+readTextile :: (PandocMonad m, ToSources a)
5960 => ReaderOptions -- ^ Reader options
60− -> Text -- ^ String to parse (assuming @'\n'@ line endings)
61+ -> a
6162 -> m Pandoc
6263readTextile opts s = do
63− parsed <- readWithM parseTextile def{ stateOptions = opts }
64− (crFilter s <> "\n\n")
64+ let sources = ensureFinalNewlines 2 (toSources s)
65+ parsed <- readWithM parseTextile def{ stateOptions = opts } sources
6566 case parsed of
6667 Right result -> return result
6768 Left e -> throwError e
6869
70+type TextileParser = ParserT Sources ParserState
6971
7072-- | Generate a Pandoc ADT from a textile document
71−parseTextile :: PandocMonad m => ParserT Text ParserState m Pandoc
73+parseTextile :: PandocMonad m => TextileParser m Pandoc
7274parseTextile = do
7375 many blankline
7476 startPos <- getPosition
7577 -- go through once just to get list of reference keys and notes
7678 -- docMinusKeys is the raw document with blanks where the keys/notes were...
77− let firstPassParser = noteBlock <|> lineClump
78− manyTill firstPassParser eof >>= setInput . T.concat
79+ let firstPassParser = do
80+ pos <- getPosition
81+ t <- noteBlock <|> lineClump
82+ return (pos, t)
83+ manyTill firstPassParser eof >>= setInput . Sources
7984 setPosition startPos
8085 st' <- getState
8186 let reversedNotes = stateNotes st'
@@ -83,10 +88,10 @@ parseTextile = do
8388 -- now parse it for real...
8489 Pandoc nullMeta . B.toList <$> parseBlocks -- FIXME
8590
86−noteMarker :: PandocMonad m => ParserT Text ParserState m Text
91+noteMarker :: PandocMonad m => TextileParser m Text
8792noteMarker = skipMany spaceChar >> string "fn" >> T.pack <$> manyTill digit (char '.')
8893
89−noteBlock :: PandocMonad m => ParserT Text ParserState m Text
94+noteBlock :: PandocMonad m => TextileParser m Text
9095noteBlock = try $ do
9196 startPos <- getPosition
9297 ref <- noteMarker
@@ -101,11 +106,11 @@ noteBlock = try $ do
101106 return $ T.replicate (sourceLine endPos - sourceLine startPos) "\n"
102107
103108-- | Parse document blocks
104−parseBlocks :: PandocMonad m => ParserT Text ParserState m Blocks
109+parseBlocks :: PandocMonad m => TextileParser m Blocks
105110parseBlocks = mconcat <$> manyTill block eof
106111
107112-- | Block parsers list tried in definition order
108−blockParsers :: PandocMonad m => [ParserT Text ParserState m Blocks]
113+blockParsers :: PandocMonad m => [TextileParser m Blocks]
109114blockParsers = [ codeBlock
110115 , header
111116 , blockQuote
@@ -120,22 +125,22 @@ blockParsers = [ codeBlock
120125 ]
121126
122127-- | Any block in the order of definition of blockParsers
123−block :: PandocMonad m => ParserT Text ParserState m Blocks
128+block :: PandocMonad m => TextileParser m Blocks
124129block = do
125130 res <- choice blockParsers <?> "block"
126131 trace (T.take 60 $ tshow $ B.toList res)
127132 return res
128133
129−commentBlock :: PandocMonad m => ParserT Text ParserState m Blocks
134+commentBlock :: PandocMonad m => TextileParser m Blocks
130135commentBlock = try $ do
131136 string "###."
132137 manyTill anyLine blanklines
133138 return mempty
134139
135−codeBlock :: PandocMonad m => ParserT Text ParserState m Blocks
140+codeBlock :: PandocMonad m => TextileParser m Blocks
136141codeBlock = codeBlockTextile <|> codeBlockHtml
137142
138−codeBlockTextile :: PandocMonad m => ParserT Text ParserState m Blocks
143+codeBlockTextile :: PandocMonad m => TextileParser m Blocks
139144codeBlockTextile = try $ do
140145 string "bc." <|> string "pre."
141146 extended <- option False (True <$ char '.')
@@ -155,7 +160,7 @@ trimTrailingNewlines :: Text -> Text
155160trimTrailingNewlines = T.dropWhileEnd (=='\n')
156161
157162-- | Code Blocks in Textile are between <pre> and </pre>
158−codeBlockHtml :: PandocMonad m => ParserT Text ParserState m Blocks
163+codeBlockHtml :: PandocMonad m => TextileParser m Blocks
159164codeBlockHtml = try $ do
160165 (t@(TagOpen _ attrs),_) <- htmlTag (tagOpen (=="pre") (const True))
161166 result' <- T.pack <$> manyTill anyChar (htmlTag (tagClose (=="pre")))
@@ -173,7 +178,7 @@ codeBlockHtml = try $ do
173178 return $ B.codeBlockWith (ident,classes,kvs) result'''
174179
175180-- | Header of the form "hN. content" with N in 1..6
176−header :: PandocMonad m => ParserT Text ParserState m Blocks
181+header :: PandocMonad m => TextileParser m Blocks
177182header = try $ do
178183 char 'h'
179184 level <- digitToInt <$> oneOf "123456"
@@ -185,14 +190,14 @@ header = try $ do
185190 return $ B.headerWith attr' level name
186191
187192-- | Blockquote of the form "bq. content"
188−blockQuote :: PandocMonad m => ParserT Text ParserState m Blocks
193+blockQuote :: PandocMonad m => TextileParser m Blocks
189194blockQuote = try $ do
190195 string "bq" >> attributes >> char '.' >> whitespace
191196 B.blockQuote <$> para
192197
193198-- Horizontal rule
194199
195−hrule :: PandocMonad m => ParserT Text st m Blocks
200+hrule :: PandocMonad m => TextileParser m Blocks
196201hrule = try $ do
197202 skipSpaces
198203 start <- oneOf "-*"
@@ -207,39 +212,39 @@ hrule = try $ do
207212-- | Can be a bullet list or an ordered list. This implementation is
208213-- strict in the nesting, sublist must start at exactly "parent depth
209214-- plus one"
210−anyList :: PandocMonad m => ParserT Text ParserState m Blocks
215+anyList :: PandocMonad m => TextileParser m Blocks
211216anyList = try $ anyListAtDepth 1 <* blanklines
212217
213218-- | This allow one type of list to be nested into an other type,
214219-- provided correct nesting
215−anyListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
220+anyListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
216221anyListAtDepth depth = choice [ bulletListAtDepth depth,
217222 orderedListAtDepth depth,
218223 definitionList ]
219224
220225-- | Bullet List of given depth, depth being the number of leading '*'
221−bulletListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
226+bulletListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
222227bulletListAtDepth depth = try $ B.bulletList <$> many1 (bulletListItemAtDepth depth)
223228
224229-- | Bullet List Item of given depth, depth being the number of
225230-- leading '*'
226−bulletListItemAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
231+bulletListItemAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
227232bulletListItemAtDepth = genericListItemAtDepth '*'
228233
229234-- | Ordered List of given depth, depth being the number of
230235-- leading '#'
231−orderedListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
236+orderedListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
232237orderedListAtDepth depth = try $ do
233238 items <- many1 (orderedListItemAtDepth depth)
234239 return $ B.orderedList items
235240
236241-- | Ordered List Item of given depth, depth being the number of
237242-- leading '#'
238−orderedListItemAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
243+orderedListItemAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
239244orderedListItemAtDepth = genericListItemAtDepth '#'
240245
241246-- | Common implementation of list items
242−genericListItemAtDepth :: PandocMonad m => Char -> Int -> ParserT Text ParserState m Blocks
247+genericListItemAtDepth :: PandocMonad m => Char -> Int -> TextileParser m Blocks
243248genericListItemAtDepth c depth = try $ do
244249 count depth (char c) >> attributes >> whitespace
245250 contents <- mconcat <$> many ((B.plain . mconcat <$> many1 inline) <|>
@@ -249,25 +254,25 @@ genericListItemAtDepth c depth = try $ do
249254 return $ contents <> sublist
250255
251256-- | A definition list is a set of consecutive definition items
252−definitionList :: PandocMonad m => ParserT Text ParserState m Blocks
257+definitionList :: PandocMonad m => TextileParser m Blocks
253258definitionList = try $ B.definitionList <$> many1 definitionListItem
254259
255260-- | List start character.
256−listStart :: PandocMonad m => ParserT Text ParserState m ()
261+listStart :: PandocMonad m => TextileParser m ()
257262listStart = genericListStart '*'
258263 <|> () <$ genericListStart '#'
259264 <|> () <$ definitionListStart
260265
261−genericListStart :: PandocMonad m => Char -> ParserT Text st m ()
266+genericListStart :: PandocMonad m => Char -> TextileParser m ()
262267genericListStart c = () <$ try (many1 (char c) >> whitespace)
263268
264−basicDLStart :: PandocMonad m => ParserT Text ParserState m ()
269+basicDLStart :: PandocMonad m => TextileParser m ()
265270basicDLStart = do
266271 char '-'
267272 whitespace
268273 notFollowedBy newline
269274
270−definitionListStart :: PandocMonad m => ParserT Text ParserState m Inlines
275+definitionListStart :: PandocMonad m => TextileParser m Inlines
271276definitionListStart = try $ do
272277 basicDLStart
273278 trimInlines . mconcat <$>
@@ -280,15 +285,15 @@ definitionListStart = try $ do
280285-- the term defined, then spaces and ":=". The definition follows, on
281286-- the same single line, or spaned on multiple line, after a line
282287-- break.
283−definitionListItem :: PandocMonad m => ParserT Text ParserState m (Inlines, [Blocks])
288+definitionListItem :: PandocMonad m => TextileParser m (Inlines, [Blocks])
284289definitionListItem = try $ do
285290 term <- mconcat . intersperse B.linebreak <$> many1 definitionListStart
286291 def' <- string ":=" *> optional whitespace *> (multilineDef <|> inlineDef)
287292 return (term, def')
288− where inlineDef :: PandocMonad m => ParserT Text ParserState m [Blocks]
293+ where inlineDef :: PandocMonad m => TextileParser m [Blocks]
289294 inlineDef = liftM (\d -> [B.plain d])
290295 $ optional whitespace >> (trimInlines . mconcat <$> many inline) <* newline
291− multilineDef :: PandocMonad m => ParserT Text ParserState m [Blocks]
296+ multilineDef :: PandocMonad m => TextileParser m [Blocks]
292297 multilineDef = try $ do
293298 optional whitespace >> newline
294299 s <- T.pack <$> many1Till anyChar (try (string "=:" >> newline))
@@ -299,7 +304,7 @@ definitionListItem = try $ do
299304-- raw content
300305
301306-- | A raw Html Block, optionally followed by blanklines
302−rawHtmlBlock :: PandocMonad m => ParserT Text ParserState m Blocks
307+rawHtmlBlock :: PandocMonad m => TextileParser m Blocks
303308rawHtmlBlock = try $ do
304309 skipMany spaceChar
305310 (_,b) <- htmlTag isBlockTag
@@ -307,14 +312,14 @@ rawHtmlBlock = try $ do
307312 return $ B.rawBlock "html" b
308313
309314-- | Raw block of LaTeX content
310−rawLaTeXBlock' :: PandocMonad m => ParserT Text ParserState m Blocks
315+rawLaTeXBlock' :: PandocMonad m => TextileParser m Blocks
311316rawLaTeXBlock' = do
312317 guardEnabled Ext_raw_tex
313318 B.rawBlock "latex" <$> (rawLaTeXBlock <* spaces)
314319
315320
316321-- | In textile, paragraphs are separated by blank lines.
317−para :: PandocMonad m => ParserT Text ParserState m Blocks
322+para :: PandocMonad m => TextileParser m Blocks
318323para = B.para . trimInlines . mconcat <$> many1 inline
319324
320325-- Tables
@@ -325,7 +330,7 @@ toAlignment '>' = AlignRight
325330toAlignment '=' = AlignCenter
326331toAlignment _ = AlignDefault
327332
328−cellAttributes :: PandocMonad m => ParserT Text ParserState m (Bool, Alignment)
333+cellAttributes :: PandocMonad m => TextileParser m (Bool, Alignment)
329334cellAttributes = try $ do
330335 isHeader <- option False (True <$ char '_')
331336 -- we just ignore colspan and rowspan markers:
@@ -338,7 +343,7 @@ cellAttributes = try $ do
338343 return (isHeader, alignment)
339344
340345-- | A table cell spans until a pipe |
341−tableCell :: PandocMonad m => ParserT Text ParserState m ((Bool, Alignment), Blocks)
346+tableCell :: PandocMonad m => TextileParser m ((Bool, Alignment), Blocks)
342347tableCell = try $ do
343348 char '|'
344349 (isHeader, alignment) <- option (False, AlignDefault) cellAttributes
@@ -349,7 +354,7 @@ tableCell = try $ do
349354 return ((isHeader, alignment), B.plain content)
350355
351356-- | A table row is made of many table cells
352−tableRow :: PandocMonad m => ParserT Text ParserState m [((Bool, Alignment), Blocks)]
357+tableRow :: PandocMonad m => TextileParser m [((Bool, Alignment), Blocks)]
353358tableRow = try $ do
354359 -- skip optional row attributes
355360 optional $ try $ do
@@ -359,7 +364,7 @@ tableRow = try $ do
359364 many1 tableCell <* char '|' <* blankline
360365
361366-- | A table with an optional header.
362−table :: PandocMonad m => ParserT Text ParserState m Blocks
367+table :: PandocMonad m => TextileParser m Blocks
363368table = try $ do
364369 -- ignore table attributes
365370 caption <- option mempty $ try $ do
@@ -375,8 +380,9 @@ table = try $ do
375380 (toprow:rest) | any (fst . fst) toprow ->
376381 (toprow, rest)
377382 _ -> (mempty, rawrows)
378− let nbOfCols = maximum $ map length (headers:rows)
379− let aligns = map minimum $ transpose $ map (map (snd . fst)) (headers:rows)
383+ let nbOfCols = maximum $ fmap length (headers :| rows)
384+ let aligns = map (maybe AlignDefault minimum . nonEmpty) $
385+ transpose $ map (map (snd . fst)) (headers:rows)
380386 let toRow = Row nullAttr . map B.simpleCell
381387 toHeaderRow l = [toRow l | not (null l)]
382388 return $ B.table (B.simpleCaption $ B.plain caption)
@@ -386,7 +392,7 @@ table = try $ do
386392 (TableFoot nullAttr [])
387393
388394-- | Ignore markers for cols, thead, tfoot.
389−ignorableRow :: PandocMonad m => ParserT Text ParserState m ()
395+ignorableRow :: PandocMonad m => TextileParser m ()
390396ignorableRow = try $ do
391397 char '|'
392398 oneOf ":^-~"
@@ -395,7 +401,7 @@ ignorableRow = try $ do
395401 _ <- anyLine
396402 return ()
397403
398−explicitBlockStart :: PandocMonad m => Text -> ParserT Text ParserState m ()
404+explicitBlockStart :: PandocMonad m => Text -> TextileParser m ()
399405explicitBlockStart name = try $ do
400406 string (T.unpack name)
401407 attributes
@@ -407,8 +413,8 @@ explicitBlockStart name = try $ do
407413-- However, they can be used to set HTML/CSS attributes when needed.
408414maybeExplicitBlock :: PandocMonad m
409415 => Text -- ^ block tag name
410− -> ParserT Text ParserState m Blocks -- ^ implicit block
411− -> ParserT Text ParserState m Blocks
416+ -> TextileParser m Blocks -- ^ implicit block
417+ -> TextileParser m Blocks
412418maybeExplicitBlock name blk = try $ do
413419 optional $ explicitBlockStart name
414420 blk
@@ -421,11 +427,11 @@ maybeExplicitBlock name blk = try $ do
421427
422428
423429-- | Any inline element
424−inline :: PandocMonad m => ParserT Text ParserState m Inlines
430+inline :: PandocMonad m => TextileParser m Inlines
425431inline = choice inlineParsers <?> "inline"
426432
427433-- | Inline parsers tried in order
428−inlineParsers :: PandocMonad m => [ParserT Text ParserState m Inlines]
434+inlineParsers :: PandocMonad m => [TextileParser m Inlines]
429435inlineParsers = [ str
430436 , whitespace
431437 , endline
@@ -445,7 +451,7 @@ inlineParsers = [ str
445451 ]
446452
447453-- | Inline markups
448−inlineMarkup :: PandocMonad m => ParserT Text ParserState m Inlines
454+inlineMarkup :: PandocMonad m => TextileParser m Inlines
449455inlineMarkup = choice [ simpleInline (string "??") (B.cite [])
450456 , simpleInline (string "**") B.strong
451457 , simpleInline (string "__") B.emph
@@ -459,29 +465,29 @@ inlineMarkup = choice [ simpleInline (string "??") (B.cite [])
459465 ]
460466
461467-- | Trademark, registered, copyright
462−mark :: PandocMonad m => ParserT Text st m Inlines
468+mark :: PandocMonad m => TextileParser m Inlines
463469mark = try $ char '(' >> (try tm <|> try reg <|> copy)
464470
465−reg :: PandocMonad m => ParserT Text st m Inlines
471+reg :: PandocMonad m => TextileParser m Inlines
466472reg = do
467473 oneOf "Rr"
468474 char ')'
469475 return $ B.str "\174"
470476
471−tm :: PandocMonad m => ParserT Text st m Inlines
477+tm :: PandocMonad m => TextileParser m Inlines
472478tm = do
473479 oneOf "Tt"
474480 oneOf "Mm"
475481 char ')'
476482 return $ B.str "\8482"
477483
478−copy :: PandocMonad m => ParserT Text st m Inlines
484+copy :: PandocMonad m => TextileParser m Inlines
479485copy = do
480486 oneOf "Cc"
481487 char ')'
482488 return $ B.str "\169"
483489
484−note :: PandocMonad m => ParserT Text ParserState m Inlines
490+note :: PandocMonad m => TextileParser m Inlines
485491note = try $ do
486492 ref <- char '[' *> many1 digit <* char ']'
487493 notes <- stateNotes <$> getState
@@ -505,13 +511,13 @@ wordBoundaries :: [Char]
505511wordBoundaries = markupChars <> stringBreakers
506512
507513-- | Parse a hyphened sequence of words
508−hyphenedWords :: PandocMonad m => ParserT Text ParserState m Text
514+hyphenedWords :: PandocMonad m => TextileParser m Text
509515hyphenedWords = do
510516 x <- wordChunk
511517 xs <- many (try $ char '-' >> wordChunk)
512518 return $ T.intercalate "-" (x:xs)
513519
514−wordChunk :: PandocMonad m => ParserT Text ParserState m Text
520+wordChunk :: PandocMonad m => TextileParser m Text
515521wordChunk = try $ do
516522 hd <- noneOf wordBoundaries
517523 tl <- many ( noneOf wordBoundaries <|>
@@ -520,7 +526,7 @@ wordChunk = try $ do
520526 return $ T.pack $ hd:tl
521527
522528-- | Any string
523−str :: PandocMonad m => ParserT Text ParserState m Inlines
529+str :: PandocMonad m => TextileParser m Inlines
524530str = do
525531 baseStr <- hyphenedWords
526532 -- RedCloth compliance : if parsed word is uppercase and immediately
@@ -533,11 +539,11 @@ str = do
533539 return $ B.str fullStr
534540
535541-- | Some number of space chars
536−whitespace :: PandocMonad m => ParserT Text st m Inlines
542+whitespace :: PandocMonad m => TextileParser m Inlines
537543whitespace = many1 spaceChar >> return B.space <?> "whitespace"
538544
539545-- | In Textile, an isolated endline character is a line break
540−endline :: PandocMonad m => ParserT Text ParserState m Inlines
546+endline :: PandocMonad m => TextileParser m Inlines
541547endline = try $ do
542548 newline
543549 notFollowedBy blankline
@@ -545,18 +551,18 @@ endline = try $ do
545551 notFollowedBy rawHtmlBlock
546552 return B.linebreak
547553
548−rawHtmlInline :: PandocMonad m => ParserT Text ParserState m Inlines
554+rawHtmlInline :: PandocMonad m => TextileParser m Inlines
549555rawHtmlInline = B.rawInline "html" . snd <$> htmlTag isInlineTag
550556
551557-- | Raw LaTeX Inline
552−rawLaTeXInline' :: PandocMonad m => ParserT Text ParserState m Inlines
558+rawLaTeXInline' :: PandocMonad m => TextileParser m Inlines
553559rawLaTeXInline' = try $ do
554560 guardEnabled Ext_raw_tex
555561 B.rawInline "latex" <$> rawLaTeXInline
556562
557563-- | Textile standard link syntax is "label":target. But we
558564-- can also have ["label":target].
559−link :: PandocMonad m => ParserT Text ParserState m Inlines
565+link :: PandocMonad m => TextileParser m Inlines
560566link = try $ do
561567 bracketed <- (True <$ char '[') <|> return False
562568 char '"' *> notFollowedBy (oneOf " \t\n\r")
@@ -576,7 +582,7 @@ link = try $ do
576582 else B.spanWith attr $ B.link url "" name'
577583
578584-- | image embedding
579−image :: PandocMonad m => ParserT Text ParserState m Inlines
585+image :: PandocMonad m => TextileParser m Inlines
580586image = try $ do
581587 char '!' >> notFollowedBy space
582588 (ident, cls, kvs) <- attributes
@@ -588,51 +594,51 @@ image = try $ do
588594 char '!'
589595 return $ B.imageWith attr src alt (B.str alt)
590596
591−escapedInline :: PandocMonad m => ParserT Text ParserState m Inlines
597+escapedInline :: PandocMonad m => TextileParser m Inlines
592598escapedInline = escapedEqs <|> escapedTag
593599
594−escapedEqs :: PandocMonad m => ParserT Text ParserState m Inlines
600+escapedEqs :: PandocMonad m => TextileParser m Inlines
595601escapedEqs = B.str . T.pack <$>
596602 try (string "==" *> manyTill anyChar' (try $ string "=="))
597603
598604-- | literal text escaped btw <notextile> tags
599−escapedTag :: PandocMonad m => ParserT Text ParserState m Inlines
605+escapedTag :: PandocMonad m => TextileParser m Inlines
600606escapedTag = B.str . T.pack <$>
601607 try (string "<notextile>" *>
602608 manyTill anyChar' (try $ string "</notextile>"))
603609
604610-- | Any special symbol defined in wordBoundaries
605−symbol :: PandocMonad m => ParserT Text ParserState m Inlines
611+symbol :: PandocMonad m => TextileParser m Inlines
606612symbol = B.str . T.singleton <$> (notFollowedBy newline *>
607613 notFollowedBy rawHtmlBlock *>
608614 oneOf wordBoundaries)
609615
610616-- | Inline code
611−code :: PandocMonad m => ParserT Text ParserState m Inlines
617+code :: PandocMonad m => TextileParser m Inlines
612618code = code1 <|> code2
613619
614620-- any character except a newline before a blank line
615−anyChar' :: PandocMonad m => ParserT Text ParserState m Char
621+anyChar' :: PandocMonad m => TextileParser m Char
616622anyChar' =
617623 satisfy (/='\n') <|>
618624 try (char '\n' <* notFollowedBy blankline)
619625
620−code1 :: PandocMonad m => ParserT Text ParserState m Inlines
626+code1 :: PandocMonad m => TextileParser m Inlines
621627code1 = B.code . T.pack <$> surrounded (char '@') anyChar'
622628
623−code2 :: PandocMonad m => ParserT Text ParserState m Inlines
629+code2 :: PandocMonad m => TextileParser m Inlines
624630code2 = do
625631 htmlTag (tagOpen (=="tt") null)
626632 B.code . T.pack <$> manyTill anyChar' (try $ htmlTag $ tagClose (=="tt"))
627633
628634-- | Html / CSS attributes
629−attributes :: PandocMonad m => ParserT Text ParserState m Attr
630−attributes = foldl (flip ($)) ("",[],[]) <$>
635+attributes :: PandocMonad m => TextileParser m Attr
636+attributes = foldl' (flip ($)) ("",[],[]) <$>
631637 try (do special <- option id specialAttribute
632638 attrs <- many attribute
633639 return (special : attrs))
634640
635−specialAttribute :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
641+specialAttribute :: PandocMonad m => TextileParser m (Attr -> Attr)
636642specialAttribute = do
637643 alignStr <- ("center" <$ char '=') <|>
638644 ("justify" <$ try (string "<>")) <|>
@@ -641,11 +647,11 @@ specialAttribute = do
641647 notFollowedBy spaceChar
642648 return $ addStyle $ T.pack $ "text-align:" ++ alignStr
643649
644−attribute :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
650+attribute :: PandocMonad m => TextileParser m (Attr -> Attr)
645651attribute = try $
646652 (classIdAttr <|> styleAttr <|> langAttr) <* notFollowedBy spaceChar
647653
648−classIdAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
654+classIdAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
649655classIdAttr = try $ do -- (class class #id)
650656 char '('
651657 ws <- T.words `fmap` T.pack <$> manyTill anyChar' (char ')')
@@ -657,7 +663,7 @@ classIdAttr = try $ do -- (class class #id)
657663 classes'
658664 -> return $ \(_,_,keyvals) -> ("",classes',keyvals)
659665
660−styleAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
666+styleAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
661667styleAttr = do
662668 style <- try $ enclosed (char '{') (char '}') anyChar'
663669 return $ addStyle $ T.pack style
@@ -668,23 +674,23 @@ addStyle style (id',classes,keyvals) =
668674 where keyvals' = ("style", style') : [(k,v) | (k,v) <- keyvals, k /= "style"]
669675 style' = style <> ";" <> T.concat [v | ("style",v) <- keyvals]
670676
671−langAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
677+langAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
672678langAttr = do
673679 lang <- try $ enclosed (char '[') (char ']') alphaNum
674680 return $ \(id',classes,keyvals) -> (id',classes,("lang",T.pack lang):keyvals)
675681
676682-- | Parses material surrounded by a parser.
677683surrounded :: (PandocMonad m, Show t)
678− => ParserT Text st m t -- ^ surrounding parser
679− -> ParserT Text st m a -- ^ content parser (to be used repeatedly)
680− -> ParserT Text st m [a]
684+ => ParserT Sources st m t -- ^ surrounding parser
685+ -> ParserT Sources st m a -- ^ content parser (to be used repeatedly)
686+ -> ParserT Sources st m [a]
681687surrounded border =
682688 enclosed (border *> notFollowedBy (oneOf " \t\n\r")) (try border)
683689
684690simpleInline :: PandocMonad m
685− => ParserT Text ParserState m t -- ^ surrounding parser
691+ => TextileParser m t -- ^ surrounding parser
686692 -> (Inlines -> Inlines) -- ^ Inline constructor
687− -> ParserT Text ParserState m Inlines -- ^ content parser (to be used repeatedly)
693+ -> TextileParser m Inlines -- ^ content parser (to be used repeatedly)
688694simpleInline border construct = try $ do
689695 notAfterString
690696 border *> notFollowedBy (oneOf " \t\n\r")
@@ -698,7 +704,7 @@ simpleInline border construct = try $ do
698704 then body
699705 else B.spanWith attr body
700706
701−groupedInlineMarkup :: PandocMonad m => ParserT Text ParserState m Inlines
707+groupedInlineMarkup :: PandocMonad m => TextileParser m Inlines
702708groupedInlineMarkup = try $ do
703709 char '['
704710 sp1 <- option mempty $ B.space <$ whitespace
@@ -707,5 +713,5 @@ groupedInlineMarkup = try $ do
707713 char ']'
708714 return $ sp1 <> result <> sp2
709715
710−eof' :: Monad m => ParserT Text s m Char
716+eof' :: Monad m => ParserT Sources s m Char
711717eof' = '\n' <$ eof
src/Text/Pandoc/Readers/TikiWiki.hs+6−6
@@ -30,23 +30,23 @@ import Text.Pandoc.Definition
3030import Text.Pandoc.Logging (Verbosity (..))
3131import Text.Pandoc.Options
3232import Text.Pandoc.Parsing hiding (enclosed, nested)
33−import Text.Pandoc.Shared (crFilter, safeRead)
33+import Text.Pandoc.Shared (safeRead)
3434import Text.Pandoc.XML (fromEntities)
3535import Text.Printf (printf)
3636
3737-- | Read TikiWiki from an input string and return a Pandoc document.
38−readTikiWiki :: PandocMonad m
38+readTikiWiki :: (PandocMonad m, ToSources a)
3939 => ReaderOptions
40− -> Text
40+ -> a
4141 -> m Pandoc
4242readTikiWiki opts s = do
43− res <- readWithM parseTikiWiki def{ stateOptions = opts }
44− (crFilter s <> "\n\n")
43+ let sources = ensureFinalNewlines 2 (toSources s)
44+ res <- readWithM parseTikiWiki def{ stateOptions = opts } sources
4545 case res of
4646 Left e -> throwError e
4747 Right d -> return d
4848
49−type TikiWikiParser = ParserT Text ParserState
49+type TikiWikiParser = ParserT Sources ParserState
5050
5151--
5252-- utility functions
src/Text/Pandoc/Readers/Txt2Tags.hs+47−21
@@ -1,4 +1,5 @@
11{-# LANGUAGE OverloadedStrings #-}
2+{-# LANGUAGE LambdaCase #-}
23{- |
34 Module : Text.Pandoc.Readers.Txt2Tags
45 Copyright : Copyright (C) 2014 Matthew Pickering
@@ -19,6 +20,7 @@ import Control.Monad.Except (catchError, throwError)
1920import Control.Monad.Reader (Reader, asks, runReader)
2021import Data.Default
2122import Data.List (intercalate, transpose)
23+import Data.List.NonEmpty (nonEmpty)
2224import Data.Maybe (fromMaybe)
2325import Data.Text (Text)
2426import qualified Data.Text as T
@@ -31,9 +33,9 @@ import Data.Time (defaultTimeLocale)
3133import Text.Pandoc.Definition
3234import Text.Pandoc.Options
3335import Text.Pandoc.Parsing hiding (space, spaces, uri)
34−import Text.Pandoc.Shared (compactify, compactifyDL, crFilter, escapeURI)
36+import Text.Pandoc.Shared (compactify, compactifyDL, escapeURI)
3537
36−type T2T = ParserT Text ParserState (Reader T2TMeta)
38+type T2T = ParserT Sources ParserState (Reader T2TMeta)
3739
3840-- | An object for the T2T macros meta information
3941-- the contents of each field is simply substituted verbatim into the file
@@ -53,25 +55,28 @@ getT2TMeta = do
5355 inps <- P.getInputFiles
5456 outp <- fromMaybe "" <$> P.getOutputFile
5557 curDate <- formatTime defaultTimeLocale "%F" <$> P.getZonedTime
56− let getModTime = fmap (formatTime defaultTimeLocale "%T") .
57− P.getModificationTime
58− curMtime <- case inps of
59− [] -> formatTime defaultTimeLocale "%T" <$> P.getZonedTime
60− _ -> catchError
61− (maximum <$> mapM getModTime inps)
62− (const (return ""))
63− return $ T2TMeta (T.pack curDate) (T.pack curMtime) (intercalate ", " inps) outp
58+ curMtime <- catchError
59+ (mapM P.getModificationTime inps >>=
60+ (\case
61+ Nothing ->
62+ formatTime defaultTimeLocale "%T" <$> P.getZonedTime
63+ Just ts -> return $
64+ formatTime defaultTimeLocale "%T" $ maximum ts)
65+ . nonEmpty)
66+ (const (return ""))
67+ return $ T2TMeta (T.pack curDate) (T.pack curMtime)
68+ (intercalate ", " inps) outp
6469
6570-- | Read Txt2Tags from an input string returning a Pandoc document
66−readTxt2Tags :: PandocMonad m
71+readTxt2Tags :: (PandocMonad m, ToSources a)
6772 => ReaderOptions
68− -> Text
73+ -> a
6974 -> m Pandoc
7075readTxt2Tags opts s = do
76+ let sources = ensureFinalNewlines 2 (toSources s)
7177 meta <- getT2TMeta
7278 let parsed = flip runReader meta $
73− readWithM parseT2T (def {stateOptions = opts}) $
74− crFilter s <> "\n\n"
79+ readWithM parseT2T (def {stateOptions = opts}) sources
7580 case parsed of
7681 Right result -> return result
7782 Left e -> throwError e
@@ -261,9 +266,9 @@ table = try $ do
261266 rows <- many1 (many commentLine *> tableRow)
262267 let columns = transpose rows
263268 let ncolumns = length columns
264− let aligns = map (foldr1 findAlign . map fst) columns
269+ let aligns = map (fromMaybe AlignDefault . foldr findAlign Nothing) columns
265270 let rows' = map (map snd) rows
266− let size = maximum (map length rows')
271+ let size = maybe 0 maximum $ nonEmpty $ map length rows'
267272 let rowsPadded = map (pad size) rows'
268273 let headerPadded = if null tableHeader then mempty else pad size tableHeader
269274 let toRow = Row nullAttr . map B.simpleCell
@@ -278,10 +283,11 @@ pad :: (Monoid a) => Int -> [a] -> [a]
278283pad n xs = xs ++ replicate (n - length xs) mempty
279284
280285
281−findAlign :: Alignment -> Alignment -> Alignment
282−findAlign x y
283− | x == y = x
284− | otherwise = AlignDefault
286+findAlign :: (Alignment, a) -> Maybe Alignment -> Maybe Alignment
287+findAlign (x,_) (Just y)
288+ | x == y = Just x
289+ | otherwise = Just AlignDefault
290+findAlign (x,_) Nothing = Just x
285291
286292headerRow :: T2T [(Alignment, Blocks)]
287293headerRow = genericRow (string "||")
@@ -472,9 +478,29 @@ macro = try $ do
472478-- raw URLs in text are automatically linked
473479url :: T2T Inlines
474480url = try $ do
475− (rawUrl, escapedUrl) <- try uri <|> emailAddress
481+ (rawUrl, escapedUrl) <- try uri <|> emailAddress'
476482 return $ B.link rawUrl "" (B.str escapedUrl)
477483
484+emailAddress' :: T2T (Text, Text)
485+emailAddress' = do
486+ (base, mailURI) <- emailAddress
487+ query <- option "" emailQuery
488+ return (base <> query, mailURI <> query)
489+
490+emailQuery :: T2T Text
491+emailQuery = do
492+ char '?'
493+ parts <- kv `sepBy1` (char '&')
494+ return $ "?" <> T.intercalate "&" parts
495+
496+kv :: T2T Text
497+kv = do
498+ k <- T.pack <$> many1 alphaNum
499+ char '='
500+ let vchar = alphaNum <|> try (oneOf "%._/~:,=$@&+-;*" <* lookAhead alphaNum)
501+ v <- T.pack <$> many1 vchar
502+ return (k <> "=" <> v)
503+
478504uri :: T2T (Text, Text)
479505uri = try $ do
480506 address <- t2tURI
src/Text/Pandoc/Readers/Vimwiki.hs+12−7
@@ -74,23 +74,28 @@ import Text.Pandoc.Parsing (ParserState, ParserT, blanklines, emailAddress,
7474 many1Till, orderedListMarker, readWithM,
7575 registerHeader, spaceChar, stateMeta,
7676 stateOptions, uri, manyTillChar, manyChar, textStr,
77− many1Char, countChar, many1TillChar)
78−import Text.Pandoc.Shared (crFilter, splitTextBy, stringify, stripFirstAndLast,
77+ many1Char, countChar, many1TillChar,
78+ alphaNum, anyChar, char, newline, noneOf, oneOf,
79+ space, spaces, string)
80+import Text.Pandoc.Sources (ToSources(..), Sources)
81+import Text.Pandoc.Shared (splitTextBy, stringify, stripFirstAndLast,
7982 isURI, tshow)
80−import Text.Parsec.Char (alphaNum, anyChar, char, newline, noneOf, oneOf, space,
81− spaces, string)
8283import Text.Parsec.Combinator (between, choice, eof, lookAhead, many1,
8384 manyTill, notFollowedBy, option, skipMany1)
8485import Text.Parsec.Prim (getState, many, try, updateState, (<|>))
8586
86−readVimwiki :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
87+readVimwiki :: (PandocMonad m, ToSources a)
88+ => ReaderOptions
89+ -> a
90+ -> m Pandoc
8791readVimwiki opts s = do
88− res <- readWithM parseVimwiki def{ stateOptions = opts } $ crFilter s
92+ let sources = toSources s
93+ res <- readWithM parseVimwiki def{ stateOptions = opts } sources
8994 case res of
9095 Left e -> throwError e
9196 Right result -> return result
9297
93−type VwParser = ParserT Text ParserState
98+type VwParser = ParserT Sources ParserState
9499
95100
96101-- constants
src/Text/Pandoc/SelfContained.hs+4−7
@@ -16,7 +16,6 @@ the HTML using data URIs.
1616module Text.Pandoc.SelfContained ( makeDataURI, makeSelfContained ) where
1717import Codec.Compression.GZip as Gzip
1818import Control.Applicative ((<|>))
19−import Control.Monad.Except (throwError)
2019import Control.Monad.Trans (lift)
2120import Data.ByteString (ByteString)
2221import Data.ByteString.Base64
@@ -29,7 +28,6 @@ import System.FilePath (takeDirectory, takeExtension, (</>))
2928import Text.HTML.TagSoup
3029import Text.Pandoc.Class.PandocMonad (PandocMonad (..), fetchItem,
3130 getInputFiles, report, setInputFiles)
32−import Text.Pandoc.Error
3331import Text.Pandoc.Logging
3432import Text.Pandoc.MIME (MimeType)
3533import Text.Pandoc.Shared (isURI, renderTags', trim)
@@ -244,11 +242,10 @@ getData mimetype src
244242 let raw' = if ext `elem` [".gz", ".svgz"]
245243 then B.concat $ L.toChunks $ Gzip.decompress $ L.fromChunks [raw]
246244 else raw
247− mime <- case (mimetype, respMime) of
248− ("",Nothing) -> throwError $ PandocSomeError
249− $ "Could not determine mime type for `" <> src <> "'"
250− (x, Nothing) -> return x
251− (_, Just x ) -> return x
245+ let mime = case (mimetype, respMime) of
246+ ("",Nothing) -> "application/octet-stream"
247+ (x, Nothing) -> x
248+ (_, Just x ) -> x
252249 result <- if "text/css" `T.isPrefixOf` mime
253250 then do
254251 oldInputs <- getInputFiles
src/Text/Pandoc/Shared.hs+41−64
@@ -25,11 +25,7 @@ module Text.Pandoc.Shared (
2525 ordNub,
2626 findM,
2727 -- * Text processing
28− ToString (..),
29− ToText (..),
3028 tshow,
31− backslashEscapes,
32− escapeStringUsing,
3329 elemText,
3430 notElemText,
3531 stripTrailingNewlines,
@@ -95,7 +91,7 @@ module Text.Pandoc.Shared (
9591 safeRead,
9692 safeStrRead,
9793 -- * User data directory
98− defaultUserDataDirs,
94+ defaultUserDataDir,
9995 -- * Version
10096 pandocVersion
10197 ) where
@@ -109,7 +105,7 @@ import qualified Data.Bifunctor as Bifunctor
109105import Data.Char (isAlpha, isLower, isSpace, isUpper, toLower, isAlphaNum,
110106 generalCategory, GeneralCategory(NonSpacingMark,
111107 SpacingCombiningMark, EnclosingMark, ConnectorPunctuation))
112−import Data.List (find, intercalate, intersperse, sortOn)
108+import Data.List (find, intercalate, intersperse, sortOn, foldl')
113109import qualified Data.Map as M
114110import Data.Maybe (mapMaybe, fromMaybe)
115111import Data.Monoid (Any (..))
@@ -127,7 +123,7 @@ import Text.HTML.TagSoup (RenderOptions (..), Tag (..), renderOptions,
127123import Text.Pandoc.Builder (Blocks, Inlines, ToMetaValue (..))
128124import qualified Text.Pandoc.Builder as B
129125import Data.Time
130−import Text.Pandoc.Asciify (toAsciiChar)
126+import Text.Pandoc.Asciify (toAsciiText)
131127import Text.Pandoc.Definition
132128import Text.Pandoc.Extensions (Extensions, Extension(..), extensionEnabled)
133129import Text.Pandoc.Generic (bottomUp)
@@ -157,12 +153,20 @@ splitTextBy isSep t
157153 in first : splitTextBy isSep (T.dropWhile isSep rest)
158154
159155splitTextByIndices :: [Int] -> T.Text -> [T.Text]
160−splitTextByIndices ns = splitTextByRelIndices (zipWith (-) ns (0:ns))
156+splitTextByIndices ns = splitTextByRelIndices (zipWith (-) ns (0:ns)) . T.unpack
161157 where
162− splitTextByRelIndices [] t = [t]
163− splitTextByRelIndices (x:xs) t =
164− let (first, rest) = T.splitAt x t
165− in first : splitTextByRelIndices xs rest
158+ splitTextByRelIndices [] cs = [T.pack cs]
159+ splitTextByRelIndices (x:xs) cs =
160+ let (first, rest) = splitAt' x cs
161+ in T.pack first : splitTextByRelIndices xs rest
162+
163+-- Note: don't replace this with T.splitAt, which is not sensitive
164+-- to character widths!
165+splitAt' :: Int -> [Char] -> ([Char],[Char])
166+splitAt' _ [] = ([],[])
167+splitAt' n xs | n <= 0 = ([],xs)
168+splitAt' n (x:xs) = (x:ys,zs)
169+ where (ys,zs) = splitAt' (n - charWidth x) xs
166170
167171ordNub :: (Ord a) => [a] -> [a]
168172ordNub l = go Set.empty l
@@ -183,38 +187,9 @@ findM p = foldr go (pure Nothing)
183187-- Text processing
184188--
185189
186−class ToString a where
187− toString :: a -> String
188−
189−instance ToString String where
190− toString = id
191−
192−instance ToString T.Text where
193− toString = T.unpack
194−
195−class ToText a where
196− toText :: a -> T.Text
197−
198−instance ToText String where
199− toText = T.pack
200−
201−instance ToText T.Text where
202− toText = id
203−
204190tshow :: Show a => a -> T.Text
205191tshow = T.pack . show
206192
207−-- | Returns an association list of backslash escapes for the
208−-- designated characters.
209−backslashEscapes :: [Char] -- ^ list of special characters to escape
210− -> [(Char, T.Text)]
211−backslashEscapes = map (\ch -> (ch, T.pack ['\\',ch]))
212−
213−-- | Escape a string of characters, using an association list of
214−-- characters and strings.
215−escapeStringUsing :: [(Char, T.Text)] -> T.Text -> T.Text
216−escapeStringUsing tbl = T.concatMap $ \c -> fromMaybe (T.singleton c) $ lookup c tbl
217−
218193-- | @True@ exactly when the @Char@ appears in the @Text@.
219194elemText :: Char -> T.Text -> Bool
220195elemText c = T.any (== c)
@@ -323,6 +298,7 @@ tabFilter tabStop = T.unlines . map go . T.lines
323298 (tabStop - (T.length s1 `mod` tabStop)) (T.pack " ")
324299 <> go (T.drop 1 s2)
325300
301+{-# DEPRECATED crFilter "readers filter crs automatically" #-}
326302-- | Strip out DOS line endings.
327303crFilter :: T.Text -> T.Text
328304crFilter = T.filter (/= '\r')
@@ -464,22 +440,20 @@ plainToPara :: Block -> Block
464440plainToPara (Plain ils) = Para ils
465441plainToPara x = x
466442
443+
467444-- | Like @compactify@, but acts on items of definition lists.
468445compactifyDL :: [(Inlines, [Blocks])] -> [(Inlines, [Blocks])]
469446compactifyDL items =
470− let defs = concatMap snd items
471− in case reverse (concatMap B.toList defs) of
472− (Para x:xs)
473− | not (any isPara xs) ->
474− let (t,ds) = last items
475− lastDef = B.toList $ last ds
476− ds' = init ds ++
477− if null lastDef
478− then [B.fromList lastDef]
479− else [B.fromList $ init lastDef ++ [Plain x]]
480− in init items ++ [(t, ds')]
481− | otherwise -> items
482− _ -> items
447+ case reverse items of
448+ ((t,ds):ys) ->
449+ case reverse (map (reverse . B.toList) ds) of
450+ ((Para x:xs) : zs) | not (any isPara xs) ->
451+ reverse ys ++
452+ [(t, reverse (map B.fromList zs) ++
453+ [B.fromList (reverse (Plain x:xs))])]
454+ _ -> items
455+ _ -> items
456+
483457
484458-- | Combine a list of lines by adding hard linebreaks.
485459combineLines :: [[Inline]] -> [Inline]
@@ -513,7 +487,7 @@ inlineListToIdentifier exts =
513487 | otherwise = T.dropWhile (not . isAlpha)
514488 filterAscii
515489 | extensionEnabled Ext_ascii_identifiers exts
516− = T.pack . mapMaybe toAsciiChar . T.unpack
490+ = toAsciiText
517491 | otherwise = id
518492 toIdent
519493 | extensionEnabled Ext_gfm_auto_identifiers exts =
@@ -842,7 +816,7 @@ mapLeft = Bifunctor.first
842816-- > collapseFilePath "parent/foo/.." == "parent"
843817-- > collapseFilePath "/parent/foo/../../bar" == "/bar"
844818collapseFilePath :: FilePath -> FilePath
845−collapseFilePath = Posix.joinPath . reverse . foldl go [] . splitDirectories
819+collapseFilePath = Posix.joinPath . reverse . foldl' go [] . splitDirectories
846820 where
847821 go rs "." = rs
848822 go r@(p:rs) ".." = case p of
@@ -879,7 +853,6 @@ filteredFilesFromArchive zf f =
879853 fileAndBinary :: Archive -> FilePath -> Maybe (FilePath, BL.ByteString)
880854 fileAndBinary a fp = findEntryByPath fp a >>= \e -> Just (fp, fromEntry e)
881855
882−
883856--
884857-- IANA URIs
885858--
@@ -1012,12 +985,16 @@ safeStrRead s = case reads s of
1012985--
1013986
1014987-- | Return appropriate user data directory for platform. We use
1015−-- XDG_DATA_HOME (or its default value), but fall back to the
1016−-- legacy user data directory ($HOME/.pandoc on *nix) if this is
1017−-- missing.
1018−defaultUserDataDirs :: IO [FilePath]
1019−defaultUserDataDirs = E.catch (do
988+-- XDG_DATA_HOME (or its default value), but for backwards compatibility,
989+-- we fall back to the legacy user data directory ($HOME/.pandoc on *nix)
990+-- if the XDG_DATA_HOME is missing and this exists. If neither directory
991+-- is present, we return the XDG data directory.
992+defaultUserDataDir :: IO FilePath
993+defaultUserDataDir = do
1020994 xdgDir <- getXdgDirectory XdgData "pandoc"
1021995 legacyDir <- getAppUserDataDirectory "pandoc"
1022− return $ ordNub [xdgDir, legacyDir])
1023− (\(_ :: E.SomeException) -> return [])
996+ xdgExists <- doesDirectoryExist xdgDir
997+ legacyDirExists <- doesDirectoryExist legacyDir
998+ if not xdgExists && legacyDirExists
999+ then return legacyDir
1000+ else return xdgDir
src/Text/Pandoc/Sources.hsnew file+195
@@ -0,0 +1,195 @@
1+{-# LANGUAGE FlexibleInstances #-}
2+{-# LANGUAGE FlexibleContexts #-}
3+{-# LANGUAGE MultiParamTypeClasses #-}
4+{-# LANGUAGE GeneralizedNewtypeDeriving #-}
5+{-# LANGUAGE OverloadedStrings #-}
6+{- |
7+ Module : Text.Pandoc.Sources
8+ Copyright : Copyright (C) 2021 John MacFarlane
9+ License : GNU GPL, version 2 or above
10+
11+ Maintainer : John MacFarlane <jgm@berkeley.edu>
12+ Stability : alpha
13+ Portability : portable
14+
15+Defines Sources object to be used as input to pandoc parsers and redefines Char
16+parsers so they get source position information from it.
17+-}
18+
19+module Text.Pandoc.Sources
20+ ( Sources(..)
21+ , ToSources(..)
22+ , UpdateSourcePos(..)
23+ , sourcesToText
24+ , initialSourceName
25+ , addToSources
26+ , ensureFinalNewlines
27+ , addToInput
28+ , satisfy
29+ , oneOf
30+ , noneOf
31+ , anyChar
32+ , char
33+ , string
34+ , newline
35+ , space
36+ , spaces
37+ , letter
38+ , digit
39+ , hexDigit
40+ , alphaNum
41+ )
42+where
43+import qualified Text.Parsec as P
44+import Text.Parsec (Stream(..), ParsecT)
45+import Text.Parsec.Pos as P
46+import Data.Text (Text)
47+import qualified Data.Text as T
48+import Data.Char (isSpace, isLetter, isAlphaNum, isDigit, isHexDigit)
49+import Data.String (IsString(..))
50+import qualified Data.List.NonEmpty as NonEmpty
51+
52+-- | A list of inputs labeled with source positions. It is assumed
53+-- that the 'Text's have @\n@ line endings.
54+newtype Sources = Sources { unSources :: [(SourcePos, Text)] }
55+ deriving (Show, Semigroup, Monoid)
56+
57+instance Monad m => Stream Sources m Char where
58+ uncons (Sources []) = return Nothing
59+ uncons (Sources ((pos,t):rest)) =
60+ case T.uncons t of
61+ Nothing -> uncons (Sources rest)
62+ Just (c,t') -> return $ Just (c, Sources ((pos,t'):rest))
63+
64+instance IsString Sources where
65+ fromString s = Sources [(P.initialPos "", T.pack (filter (/='\r') s))]
66+
67+class ToSources a where
68+ toSources :: a -> Sources
69+
70+instance ToSources Text where
71+ toSources t = Sources [(P.initialPos "", T.filter (/='\r') t)]
72+
73+instance ToSources [(FilePath, Text)] where
74+ toSources = Sources
75+ . map (\(fp,t) ->
76+ (P.initialPos fp, T.snoc (T.filter (/='\r') t) '\n'))
77+
78+instance ToSources Sources where
79+ toSources = id
80+
81+sourcesToText :: Sources -> Text
82+sourcesToText (Sources xs) = mconcat $ map snd xs
83+
84+addToSources :: Monad m => SourcePos -> Text -> ParsecT Sources u m ()
85+addToSources pos t = do
86+ curpos <- P.getPosition
87+ Sources xs <- P.getInput
88+ let xs' = case xs of
89+ [] -> []
90+ ((_,t'):rest) -> (curpos,t'):rest
91+ P.setInput $ Sources ((pos, T.filter (/='\r') t):xs')
92+
93+ensureFinalNewlines :: Int -- ^ number of trailing newlines
94+ -> Sources
95+ -> Sources
96+ensureFinalNewlines n (Sources xs) =
97+ case NonEmpty.nonEmpty xs of
98+ Nothing -> Sources [(initialPos "", T.replicate n "\n")]
99+ Just lst ->
100+ case NonEmpty.last lst of
101+ (spos, t) ->
102+ case T.length (T.takeWhileEnd (=='\n') t) of
103+ len | len >= n -> Sources xs
104+ | otherwise -> Sources (NonEmpty.init lst ++
105+ [(spos,
106+ t <> T.replicate (n - len) "\n")])
107+
108+class UpdateSourcePos s c where
109+ updateSourcePos :: SourcePos -> c -> s -> SourcePos
110+
111+instance UpdateSourcePos Text Char where
112+ updateSourcePos pos c _ = updatePosChar pos c
113+
114+instance UpdateSourcePos Sources Char where
115+ updateSourcePos pos c sources =
116+ case sources of
117+ Sources [] -> updatePosChar pos c
118+ Sources ((_,t):(pos',_):_)
119+ | T.null t -> pos'
120+ Sources _ ->
121+ case c of
122+ '\n' -> incSourceLine (setSourceColumn pos 1) 1
123+ '\t' -> incSourceColumn pos (4 - ((sourceColumn pos - 1) `mod` 4))
124+ _ -> incSourceColumn pos 1
125+
126+-- | Get name of first source in 'Sources'.
127+initialSourceName :: Sources -> FilePath
128+initialSourceName (Sources []) = ""
129+initialSourceName (Sources ((pos,_):_)) = sourceName pos
130+
131+-- | Add some text to the beginning of the input sources.
132+-- This simplifies code that expands macros.
133+addToInput :: Monad m => Text -> ParsecT Sources u m ()
134+addToInput t = do
135+ Sources xs <- P.getInput
136+ case xs of
137+ [] -> P.setInput $ Sources [(initialPos "",t)]
138+ (pos,t'):rest -> P.setInput $ Sources ((pos, t <> t'):rest)
139+
140+-- We need to redefine the parsers in Text.Parsec.Char so that they
141+-- update source positions properly from the Sources stream.
142+
143+satisfy :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
144+ => (Char -> Bool) -> ParsecT s u m Char
145+satisfy f = P.tokenPrim show updateSourcePos matcher
146+ where
147+ matcher c = if f c then Just c else Nothing
148+
149+oneOf :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
150+ => [Char] -> ParsecT s u m Char
151+oneOf cs = satisfy (`elem` cs)
152+
153+noneOf :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
154+ => [Char] -> ParsecT s u m Char
155+noneOf cs = satisfy (`notElem` cs)
156+
157+anyChar :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
158+ => ParsecT s u m Char
159+anyChar = satisfy (const True)
160+
161+char :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
162+ => Char -> ParsecT s u m Char
163+char c = satisfy (== c)
164+
165+string :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
166+ => [Char] -> ParsecT s u m [Char]
167+string = mapM char
168+
169+newline :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
170+ => ParsecT s u m Char
171+newline = satisfy (== '\n')
172+
173+space :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
174+ => ParsecT s u m Char
175+space = satisfy isSpace
176+
177+spaces :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
178+ => ParsecT s u m ()
179+spaces = P.skipMany space P.<?> "white space"
180+
181+letter :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
182+ => ParsecT s u m Char
183+letter = satisfy isLetter
184+
185+alphaNum :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
186+ => ParsecT s u m Char
187+alphaNum = satisfy isAlphaNum
188+
189+digit :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
190+ => ParsecT s u m Char
191+digit = satisfy isDigit
192+
193+hexDigit :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
194+ => ParsecT s u m Char
195+hexDigit = satisfy isHexDigit
src/Text/Pandoc/Templates.hs+4
@@ -34,6 +34,7 @@ import Control.Monad.Except (catchError, throwError)
3434import Data.Text (Text)
3535import qualified Data.Text as T
3636import Text.Pandoc.Error
37+import System.IO.Error (isDoesNotExistError)
3738
3839-- | Wrap a Monad in this if you want partials to
3940-- be taken only from the default data files.
@@ -70,6 +71,9 @@ getTemplate tp = UTF8.toText <$>
7071 PandocResourceNotFound _ ->
7172 -- see #5987 on reason for takeFileName
7273 readDataFile ("templates" </> takeFileName tp)
74+ PandocIOError _ ioe | isDoesNotExistError ioe ->
75+ -- see #5987 on reason for takeFileName
76+ readDataFile ("templates" </> takeFileName tp)
7377 _ -> throwError e))
7478
7579-- | Get default template for the specified writer.
src/Text/Pandoc/Writers.hs+3−3
@@ -81,6 +81,7 @@ import Data.Aeson
8181import qualified Data.ByteString.Lazy as BL
8282import Data.Text (Text)
8383import qualified Data.Text as T
84+import Text.Pandoc.Shared (tshow)
8485import Text.Pandoc.Class
8586import Text.Pandoc.Definition
8687import Text.Pandoc.Options
@@ -122,7 +123,6 @@ import Text.Pandoc.Writers.Texinfo
122123import Text.Pandoc.Writers.Textile
123124import Text.Pandoc.Writers.XWiki
124125import Text.Pandoc.Writers.ZimWiki
125−import Text.Parsec.Error
126126
127127data Writer m = TextWriter (WriterOptions -> Pandoc -> m Text)
128128 | ByteStringWriter (WriterOptions -> Pandoc -> m BL.ByteString)
@@ -196,8 +196,8 @@ writers = [
196196getWriter :: PandocMonad m => Text -> m (Writer m, Extensions)
197197getWriter s =
198198 case parseFormatSpec s of
199− Left e -> throwError $ PandocAppError
200− $ T.intercalate "\n" [T.pack m | Message m <- errorMessages e]
199+ Left e -> throwError $ PandocAppError $
200+ "Error parsing writer format " <> tshow s <> ": " <> tshow e
201201 Right (writerName, extsToEnable, extsToDisable) ->
202202 case lookup writerName writers of
203203 Nothing -> throwError $
src/Text/Pandoc/Writers/AsciiDoc.hs+10−4
@@ -22,6 +22,7 @@ module Text.Pandoc.Writers.AsciiDoc (writeAsciiDoc, writeAsciiDoctor) where
2222import Control.Monad.State.Strict
2323import Data.Char (isPunctuation, isSpace)
2424import Data.List (intercalate, intersperse)
25+import Data.List.NonEmpty (NonEmpty(..))
2526import Data.Maybe (fromMaybe, isJust)
2627import qualified Data.Set as Set
2728import qualified Data.Text as T
@@ -104,8 +105,11 @@ pandocToAsciiDoc opts (Pandoc meta blocks) = do
104105
105106-- | Escape special characters for AsciiDoc.
106107escapeString :: Text -> Text
107−escapeString = escapeStringUsing escs
108− where escs = backslashEscapes "{"
108+escapeString t
109+ | T.any (== '{') t = T.concatMap escChar t
110+ | otherwise = t
111+ where escChar '{' = "\\{"
112+ escChar c = T.singleton c
109113
110114-- | Ordered list start parser for use in Para below.
111115olMarker :: Parser Text ParserState Char
@@ -274,7 +278,7 @@ blockToAsciiDoc opts block@(Table _ blkCapt specs thead tbody tfoot) = do
274278 let colwidth = if writerWrapText opts == WrapAuto
275279 then writerColumns opts
276280 else 100000
277− let maxwidth = maximum $ map offset (head':rows')
281+ let maxwidth = maximum $ fmap offset (head' :| rows')
278282 let body = if maxwidth > colwidth then vsep rows' else vcat rows'
279283 let border = separator <> text "==="
280284 return $
@@ -495,7 +499,9 @@ inlineToAsciiDoc opts (Quoted qt lst) = do
495499 | otherwise -> [Str "``"] ++ lst ++ [Str "''"]
496500inlineToAsciiDoc _ (Code _ str) = do
497501 isAsciidoctor <- gets asciidoctorVariant
498− let contents = literal (escapeStringUsing (backslashEscapes "`") str)
502+ let escChar '`' = "\\'"
503+ escChar c = T.singleton c
504+ let contents = literal (T.concatMap escChar str)
499505 return $
500506 if isAsciidoctor
501507 then text "`+" <> contents <> "+`"
src/Text/Pandoc/Writers/BibTeX.hs+1−1
@@ -43,7 +43,7 @@ writeBibTeX' :: PandocMonad m => Variant -> WriterOptions -> Pandoc -> m Text
4343writeBibTeX' variant opts (Pandoc meta _) = do
4444 let mblang = case lookupMetaString "lang" meta of
4545 "" -> Nothing
46− t -> Just $ parseLang t
46+ t -> either (const Nothing) Just $ parseLang t
4747 let refs = case lookupMeta "references" meta of
4848 Just (MetaList xs) -> mapMaybe metaValueToReference xs
4949 _ -> []
src/Text/Pandoc/Writers/ConTeXt.hs+44−38
@@ -20,7 +20,7 @@ import Data.Maybe (mapMaybe)
2020import Data.Text (Text)
2121import qualified Data.Text as T
2222import Network.URI (unEscapeString)
23−import Text.Pandoc.BCP47
23+import Text.Collate.Lang (Lang(..))
2424import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
2525import Text.Pandoc.Definition
2626import Text.Pandoc.ImageSize
@@ -177,8 +177,12 @@ blockToConTeXt (Para lst) = do
177177 contents <- inlineListToConTeXt lst
178178 return $ contents <> blankline
179179blockToConTeXt (LineBlock lns) = do
180− doclines <- nowrap . vcat <$> mapM inlineListToConTeXt lns
181− return $ "\\startlines" $$ doclines $$ "\\stoplines" <> blankline
180+ let emptyToBlankline doc = if isEmpty doc
181+ then blankline
182+ else doc
183+ doclines <- mapM inlineListToConTeXt lns
184+ let contextLines = vcat . map emptyToBlankline $ doclines
185+ return $ "\\startlines" $$ contextLines $$ "\\stoplines" <> blankline
182186blockToConTeXt (BlockQuote lst) = do
183187 contents <- blockListToConTeXt lst
184188 return $ "\\startblockquote" $$ nest 0 contents $$ "\\stopblockquote" <> blankline
@@ -228,13 +232,7 @@ blockToConTeXt (OrderedList (start, style', delim) lst) = do
228232 Period -> "stopper=."
229233 OneParen -> "stopper=)"
230234 TwoParens -> "left=(,stopper=)"
231− let width = maximum $ map T.length $ take (length contents)
232− (orderedListMarkers (start, style', delim))
233− let width' = (toEnum width + 1) / 2
234− let width'' = if width' > (1.5 :: Double)
235− then "width=" <> tshow width' <> "em"
236− else ""
237− let specs2Items = filter (not . T.null) [start', delim', width'']
235+ let specs2Items = filter (not . T.null) [start', delim']
238236 let specs2 = if null specs2Items
239237 then ""
240238 else "[" <> T.intercalate "," specs2Items <> "]"
@@ -248,8 +246,8 @@ blockToConTeXt (OrderedList (start, style', delim) lst) = do
248246 UpperAlpha -> 'A') :
249247 if isTightList lst then ",packed]" else "]"
250248 let specs = T.pack style'' <> specs2
251− return $ "\\startitemize" <> literal specs $$ vcat contents $$
252− "\\stopitemize" <> blankline
249+ return $ "\\startenumerate" <> literal specs $$ vcat contents $$
250+ "\\stopenumerate" <> blankline
253251blockToConTeXt (DefinitionList lst) =
254252 liftM vcat $ mapM defListItemToConTeXt lst
255253blockToConTeXt HorizontalRule = return $ "\\thinrule" <> blankline
@@ -432,9 +430,13 @@ inlineToConTeXt (Link _ txt (src, _)) = do
432430 put $ st {stNextRef = next + 1}
433431 let ref = "url" <> tshow next
434432 contents <- inlineListToConTeXt txt
433+ let escChar '#' = "\\#"
434+ escChar '%' = "\\%"
435+ escChar c = T.singleton c
436+ let escContextURL = T.concatMap escChar
435437 return $ "\\useURL"
436438 <> brackets (literal ref)
437− <> brackets (literal $ escapeStringUsing [('#',"\\#"),('%',"\\%")] src)
439+ <> brackets (literal $ escContextURL src)
438440 <> (if isAutolink
439441 then empty
440442 else brackets empty <> brackets contents)
@@ -477,7 +479,7 @@ inlineToConTeXt (Note contents) = do
477479 then literal "\\footnote{" <> nest 2 (chomp contents') <> char '}'
478480 else literal "\\startbuffer " <> nest 2 (chomp contents') <>
479481 literal "\\stopbuffer\\footnote{\\getbuffer}"
480−inlineToConTeXt (Span (_,_,kvs) ils) = do
482+inlineToConTeXt (Span (ident,_,kvs) ils) = do
481483 mblang <- fromBCP47 (lookup "lang" kvs)
482484 let wrapDir txt = case lookup "dir" kvs of
483485 Just "rtl" -> braces $ "\\righttoleft " <> txt
@@ -487,7 +489,11 @@ inlineToConTeXt (Span (_,_,kvs) ils) = do
487489 Just lng -> braces ("\\language" <>
488490 brackets (literal lng) <> txt)
489491 Nothing -> txt
490− wrapLang . wrapDir <$> inlineListToConTeXt ils
492+ addReference =
493+ if T.null ident
494+ then id
495+ else (("\\reference" <> brackets (literal ident) <> "{}") <>)
496+ addReference . wrapLang . wrapDir <$> inlineListToConTeXt ils
491497
492498-- | Craft the section header, inserting the section reference, if supplied.
493499sectionHeader :: PandocMonad m
@@ -549,26 +555,26 @@ fromBCP47 mbs = fromBCP47' <$> toLang mbs
549555-- https://tools.ietf.org/html/bcp47#section-2.1
550556-- http://wiki.contextgarden.net/Language_Codes
551557fromBCP47' :: Maybe Lang -> Maybe Text
552−fromBCP47' (Just (Lang "ar" _ "SY" _) ) = Just "ar-sy"
553−fromBCP47' (Just (Lang "ar" _ "IQ" _) ) = Just "ar-iq"
554−fromBCP47' (Just (Lang "ar" _ "JO" _) ) = Just "ar-jo"
555−fromBCP47' (Just (Lang "ar" _ "LB" _) ) = Just "ar-lb"
556−fromBCP47' (Just (Lang "ar" _ "DZ" _) ) = Just "ar-dz"
557−fromBCP47' (Just (Lang "ar" _ "MA" _) ) = Just "ar-ma"
558−fromBCP47' (Just (Lang "de" _ _ ["1901"]) ) = Just "deo"
559−fromBCP47' (Just (Lang "de" _ "DE" _) ) = Just "de-de"
560−fromBCP47' (Just (Lang "de" _ "AT" _) ) = Just "de-at"
561−fromBCP47' (Just (Lang "de" _ "CH" _) ) = Just "de-ch"
562−fromBCP47' (Just (Lang "el" _ _ ["poly"]) ) = Just "agr"
563−fromBCP47' (Just (Lang "en" _ "US" _) ) = Just "en-us"
564−fromBCP47' (Just (Lang "en" _ "GB" _) ) = Just "en-gb"
565−fromBCP47' (Just (Lang "grc"_ _ _) ) = Just "agr"
566−fromBCP47' (Just (Lang "el" _ _ _) ) = Just "gr"
567−fromBCP47' (Just (Lang "eu" _ _ _) ) = Just "ba"
568−fromBCP47' (Just (Lang "he" _ _ _) ) = Just "il"
569−fromBCP47' (Just (Lang "jp" _ _ _) ) = Just "ja"
570−fromBCP47' (Just (Lang "uk" _ _ _) ) = Just "ua"
571−fromBCP47' (Just (Lang "vi" _ _ _) ) = Just "vn"
572−fromBCP47' (Just (Lang "zh" _ _ _) ) = Just "cn"
573−fromBCP47' (Just (Lang l _ _ _) ) = Just l
574−fromBCP47' Nothing = Nothing
558+fromBCP47' (Just (Lang "ar" _ (Just "SY") _ _ _)) = Just "ar-sy"
559+fromBCP47' (Just (Lang "ar" _ (Just "IQ") _ _ _)) = Just "ar-iq"
560+fromBCP47' (Just (Lang "ar" _ (Just "JO") _ _ _)) = Just "ar-jo"
561+fromBCP47' (Just (Lang "ar" _ (Just "LB") _ _ _)) = Just "ar-lb"
562+fromBCP47' (Just (Lang "ar" _ (Just "DZ") _ _ _)) = Just "ar-dz"
563+fromBCP47' (Just (Lang "ar" _ (Just "MA") _ _ _)) = Just "ar-ma"
564+fromBCP47' (Just (Lang "de" _ _ ["1901"] _ _)) = Just "deo"
565+fromBCP47' (Just (Lang "de" _ (Just "DE") _ _ _)) = Just "de-de"
566+fromBCP47' (Just (Lang "de" _ (Just "AT") _ _ _)) = Just "de-at"
567+fromBCP47' (Just (Lang "de" _ (Just "CH") _ _ _)) = Just "de-ch"
568+fromBCP47' (Just (Lang "el" _ _ ["poly"] _ _)) = Just "agr"
569+fromBCP47' (Just (Lang "en" _ (Just "US") _ _ _)) = Just "en-us"
570+fromBCP47' (Just (Lang "en" _ (Just "GB") _ _ _)) = Just "en-gb"
571+fromBCP47' (Just (Lang "grc"_ _ _ _ _)) = Just "agr"
572+fromBCP47' (Just (Lang "el" _ _ _ _ _)) = Just "gr"
573+fromBCP47' (Just (Lang "eu" _ _ _ _ _)) = Just "ba"
574+fromBCP47' (Just (Lang "he" _ _ _ _ _)) = Just "il"
575+fromBCP47' (Just (Lang "jp" _ _ _ _ _)) = Just "ja"
576+fromBCP47' (Just (Lang "uk" _ _ _ _ _)) = Just "ua"
577+fromBCP47' (Just (Lang "vi" _ _ _ _ _)) = Just "vn"
578+fromBCP47' (Just (Lang "zh" _ _ _ _ _)) = Just "cn"
579+fromBCP47' (Just (Lang l _ _ _ _ _)) = Just l
580+fromBCP47' Nothing = Nothing
src/Text/Pandoc/Writers/CslJson.hs+4−3
@@ -34,15 +34,16 @@ import Control.Monad.Identity
3434import Citeproc.Locale (getLocale)
3535import Citeproc.CslJson
3636import Text.Pandoc.Options (WriterOptions)
37−import Data.Maybe (mapMaybe)
37+import Data.Maybe (mapMaybe, fromMaybe)
3838import Data.Aeson.Encode.Pretty (Config (..), Indent (Spaces),
3939 NumberFormat (Generic),
4040 defConfig, encodePretty')
4141
4242writeCslJson :: PandocMonad m => WriterOptions -> Pandoc -> m Text
4343writeCslJson _opts (Pandoc meta _) = do
44− let lang = maybe (Lang "en" (Just "US")) parseLang
45− (lookupMeta "lang" meta >>= metaValueToText)
44+ let lang = fromMaybe (Lang "en" Nothing (Just "US") [] [] [])
45+ (lookupMeta "lang" meta >>= metaValueToText >>=
46+ either (const Nothing) Just . parseLang)
4647 locale <- case getLocale lang of
4748 Left e -> throwError $ PandocCiteprocError e
4849 Right l -> return l
src/Text/Pandoc/Writers/Docbook.hs+50−7
@@ -168,7 +168,7 @@ blockToDocbook :: PandocMonad m => WriterOptions -> Block -> DB m (Doc Text)
168168blockToDocbook _ Null = return empty
169169-- Add ids to paragraphs in divs with ids - this is needed for
170170-- pandoc-citeproc to get link anchors in bibliographies:
171−blockToDocbook opts (Div (id',"section":_,_) (Header lvl _ ils : xs)) = do
171+blockToDocbook opts (Div (id',"section":_,_) (Header lvl (_,_,attrs) ils : xs)) = do
172172 version <- ask
173173 -- Docbook doesn't allow sections with no content, so insert some if needed
174174 let bs = if null xs
@@ -188,15 +188,17 @@ blockToDocbook opts (Div (id',"section":_,_) (Header lvl _ ils : xs)) = do
188188 -- standalone documents will include them in the template.
189189 then [("xmlns", "http://docbook.org/ns/docbook"),("xmlns:xlink", "http://www.w3.org/1999/xlink")]
190190 else []
191− attribs = nsAttr <> idAttr
191+
192+ -- Populate miscAttr with Header.Attr.attributes, filtering out non-valid DocBook section attributes, id, and xml:id
193+ miscAttr = filter (isSectionAttr version) attrs
194+ attribs = nsAttr <> idAttr <> miscAttr
192195 title' <- inlinesToDocbook opts ils
193196 contents <- blocksToDocbook opts bs
194197 return $ inTags True tag attribs $ inTagsSimple "title" title' $$ contents
195198blockToDocbook opts (Div (ident,classes,_) bs) = do
196199 version <- ask
197200 let identAttribs = [(idName version, ident) | not (T.null ident)]
198− admonitions = ["attention","caution","danger","error","hint",
199− "important","note","tip","warning"]
201+ admonitions = ["caution","danger","important","note","tip","warning"]
200202 case classes of
201203 (l:_) | l `elem` admonitions -> do
202204 let (mTitleBs, bodyBs) =
@@ -250,17 +252,18 @@ blockToDocbook opts (LineBlock lns) =
250252 blockToDocbook opts $ linesToPara lns
251253blockToDocbook opts (BlockQuote blocks) =
252254 inTagsIndented "blockquote" <$> blocksToDocbook opts blocks
253−blockToDocbook _ (CodeBlock (_,classes,_) str) = return $
255+blockToDocbook opts (CodeBlock (_,classes,_) str) = return $
254256 literal ("<programlisting" <> lang <> ">") <> cr <>
255257 flush (literal (escapeStringForXML str) <> cr <> literal "</programlisting>")
256258 where lang = if null langs
257259 then ""
258260 else " language=\"" <> escapeStringForXML (head langs) <>
259261 "\""
260− isLang l = T.toLower l `elem` map T.toLower languages
262+ syntaxMap = writerSyntaxMap opts
263+ isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
261264 langsFrom s = if isLang s
262265 then [s]
263− else languagesByExtension . T.toLower $ s
266+ else (languagesByExtension syntaxMap) . T.toLower $ s
264267 langs = concatMap langsFrom classes
265268blockToDocbook opts (BulletList lst) = do
266269 let attribs = [("spacing", "compact") | isTightList lst]
@@ -451,3 +454,43 @@ idAndRole (id',cls,_) = ident <> role
451454 where
452455 ident = [("id", id') | not (T.null id')]
453456 role = [("role", T.unwords cls) | not (null cls)]
457+
458+isSectionAttr :: DocBookVersion -> (Text, Text) -> Bool
459+isSectionAttr _ ("label",_) = True
460+isSectionAttr _ ("status",_) = True
461+isSectionAttr DocBook5 ("annotations",_) = True
462+isSectionAttr DocBook5 ("dir","ltr") = True
463+isSectionAttr DocBook5 ("dir","rtl") = True
464+isSectionAttr DocBook5 ("dir","lro") = True
465+isSectionAttr DocBook5 ("dir","rlo") = True
466+isSectionAttr _ ("remap",_) = True
467+isSectionAttr _ ("revisionflag","changed") = True
468+isSectionAttr _ ("revisionflag","added") = True
469+isSectionAttr _ ("revisionflag","deleted") = True
470+isSectionAttr _ ("revisionflag","off") = True
471+isSectionAttr _ ("role",_) = True
472+isSectionAttr DocBook5 ("version",_) = True
473+isSectionAttr DocBook5 ("xml:base",_) = True
474+isSectionAttr DocBook5 ("xml:lang",_) = True
475+isSectionAttr _ ("xreflabel",_) = True
476+isSectionAttr DocBook5 ("linkend",_) = True
477+isSectionAttr DocBook5 ("linkends",_) = True
478+isSectionAttr DocBook5 ("xlink:actuate",_) = True
479+isSectionAttr DocBook5 ("xlink:arcrole",_) = True
480+isSectionAttr DocBook5 ("xlink:from",_) = True
481+isSectionAttr DocBook5 ("xlink:href",_) = True
482+isSectionAttr DocBook5 ("xlink:label",_) = True
483+isSectionAttr DocBook5 ("xlink:role",_) = True
484+isSectionAttr DocBook5 ("xlink:show",_) = True
485+isSectionAttr DocBook5 ("xlink:title",_) = True
486+isSectionAttr DocBook5 ("xlink:to",_) = True
487+isSectionAttr DocBook5 ("xlink:type",_) = True
488+isSectionAttr DocBook4 ("arch",_) = True
489+isSectionAttr DocBook4 ("condition",_) = True
490+isSectionAttr DocBook4 ("conformance",_) = True
491+isSectionAttr DocBook4 ("lang",_) = True
492+isSectionAttr DocBook4 ("os",_) = True
493+isSectionAttr DocBook4 ("revision",_) = True
494+isSectionAttr DocBook4 ("security",_) = True
495+isSectionAttr DocBook4 ("vendor",_) = True
496+isSectionAttr _ (_,_) = False
src/Text/Pandoc/Writers/Docx.hs+60−244
@@ -22,7 +22,6 @@ import Control.Applicative ((<|>))
2222import Control.Monad.Except (catchError, throwError)
2323import Control.Monad.Reader
2424import Control.Monad.State.Strict
25−import qualified Data.ByteString as B
2625import qualified Data.ByteString.Lazy as BL
2726import Data.Char (isSpace, isLetter)
2827import Data.List (intercalate, isPrefixOf, isSuffixOf)
@@ -36,9 +35,9 @@ import qualified Data.Text.Lazy as TL
3635import Data.Time.Clock.POSIX
3736import Data.Digest.Pure.SHA (sha1, showDigest)
3837import Skylighting
39−import System.Random (randomRs, mkStdGen)
40−import Text.Pandoc.BCP47 (getLang, renderLang)
41−import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
38+import Text.Collate.Lang (renderLang)
39+import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang, translateTerm)
40+import qualified Text.Pandoc.Translations as Term
4241import qualified Text.Pandoc.Class.PandocMonad as P
4342import Data.Time
4443import Text.Pandoc.UTF8 (fromTextLazy)
@@ -48,123 +47,25 @@ import Text.Pandoc.Highlighting (highlight)
4847import Text.Pandoc.Error
4948import Text.Pandoc.ImageSize
5049import Text.Pandoc.Logging
51−import Text.Pandoc.MIME (MimeType, extensionFromMimeType, getMimeType,
52− getMimeTypeDef)
50+import Text.Pandoc.MIME (extensionFromMimeType, getMimeType, getMimeTypeDef)
5351import Text.Pandoc.Options
5452import Text.Pandoc.Writers.Docx.StyleMap
53+import Text.Pandoc.Writers.Docx.Table
54+import Text.Pandoc.Writers.Docx.Types
5555import Text.Pandoc.Shared
5656import Text.Pandoc.Walk
57+import qualified Text.Pandoc.Writers.GridTable as Grid
5758import Text.Pandoc.Writers.Math
5859import Text.Pandoc.Writers.Shared
59−import Text.Printf (printf)
6060import Text.TeXMath
6161import Text.Pandoc.Writers.OOXML
6262import Text.Pandoc.XML.Light as XML
6363import Data.Generics (mkT, everywhere)
6464
65−data ListMarker = NoMarker
66− | BulletMarker
67− | NumberMarker ListNumberStyle ListNumberDelim Int
68− deriving (Show, Read, Eq, Ord)
69−
70−listMarkerToId :: ListMarker -> Text
71−listMarkerToId NoMarker = "990"
72−listMarkerToId BulletMarker = "991"
73−listMarkerToId (NumberMarker sty delim n) = T.pack $
74− '9' : '9' : styNum : delimNum : show n
75− where styNum = case sty of
76− DefaultStyle -> '2'
77− Example -> '3'
78− Decimal -> '4'
79− LowerRoman -> '5'
80− UpperRoman -> '6'
81− LowerAlpha -> '7'
82− UpperAlpha -> '8'
83− delimNum = case delim of
84− DefaultDelim -> '0'
85− Period -> '1'
86− OneParen -> '2'
87− TwoParens -> '3'
88−
89−data EnvProps = EnvProps{ styleElement :: Maybe Element
90− , otherElements :: [Element]
91− }
92−
93−instance Semigroup EnvProps where
94− EnvProps s es <> EnvProps s' es' = EnvProps (s <|> s') (es ++ es')
95−
96−instance Monoid EnvProps where
97− mempty = EnvProps Nothing []
98− mappend = (<>)
99−
10065squashProps :: EnvProps -> [Element]
10166squashProps (EnvProps Nothing es) = es
10267squashProps (EnvProps (Just e) es) = e : es
10368
104−data WriterEnv = WriterEnv{ envTextProperties :: EnvProps
105− , envParaProperties :: EnvProps
106− , envRTL :: Bool
107− , envListLevel :: Int
108− , envListNumId :: Int
109− , envInDel :: Bool
110− , envChangesAuthor :: Text
111− , envChangesDate :: Text
112− , envPrintWidth :: Integer
113− }
114−
115−defaultWriterEnv :: WriterEnv
116−defaultWriterEnv = WriterEnv{ envTextProperties = mempty
117− , envParaProperties = mempty
118− , envRTL = False
119− , envListLevel = -1
120− , envListNumId = 1
121− , envInDel = False
122− , envChangesAuthor = "unknown"
123− , envChangesDate = "1969-12-31T19:00:00Z"
124− , envPrintWidth = 1
125− }
126−
127−data WriterState = WriterState{
128− stFootnotes :: [Element]
129− , stComments :: [([(Text, Text)], [Inline])]
130− , stSectionIds :: Set.Set Text
131− , stExternalLinks :: M.Map Text Text
132− , stImages :: M.Map FilePath (String, String, Maybe MimeType, B.ByteString)
133− , stLists :: [ListMarker]
134− , stInsId :: Int
135− , stDelId :: Int
136− , stStyleMaps :: StyleMaps
137− , stFirstPara :: Bool
138− , stInTable :: Bool
139− , stInList :: Bool
140− , stTocTitle :: [Inline]
141− , stDynamicParaProps :: Set.Set ParaStyleName
142− , stDynamicTextProps :: Set.Set CharStyleName
143− , stCurId :: Int
144− }
145−
146−defaultWriterState :: WriterState
147−defaultWriterState = WriterState{
148− stFootnotes = defaultFootnotes
149− , stComments = []
150− , stSectionIds = Set.empty
151− , stExternalLinks = M.empty
152− , stImages = M.empty
153− , stLists = [NoMarker]
154− , stInsId = 1
155− , stDelId = 1
156− , stStyleMaps = StyleMaps M.empty M.empty
157− , stFirstPara = False
158− , stInTable = False
159− , stInList = False
160− , stTocTitle = [Str "Table of Contents"]
161− , stDynamicParaProps = Set.empty
162− , stDynamicTextProps = Set.empty
163− , stCurId = 20
164− }
165−
166−type WS m = ReaderT WriterEnv (StateT WriterState m)
167−
16869renumIdMap :: Int -> [Element] -> M.Map Text Text
16970renumIdMap _ [] = M.empty
17071renumIdMap n (e:es)
@@ -569,12 +470,27 @@ writeDocx opts doc = do
569470 -- adds references to footnotes or endnotes we don't have...
570471 -- we do, however, copy some settings over from reference
571472 let settingsPath = "word/settings.xml"
572− settingsList = [ "w:autoHyphenation"
573− , "w:consecutiveHyphenLimit"
574− , "w:hyphenationZone"
575− , "w:doNotHyphenateCap"
576− , "w:evenAndOddHeaders"
577− , "w:proofState"
473+ settingsList = [ "zoom"
474+ , "embedSystemFonts"
475+ , "doNotTrackMoves"
476+ , "defaultTabStop"
477+ , "drawingGridHorizontalSpacing"
478+ , "drawingGridVerticalSpacing"
479+ , "displayHorizontalDrawingGridEvery"
480+ , "displayVerticalDrawingGridEvery"
481+ , "characterSpacingControl"
482+ , "savePreviewPicture"
483+ , "mathPr"
484+ , "themeFontLang"
485+ , "decimalSymbol"
486+ , "listSeparator"
487+ , "autoHyphenation"
488+ , "consecutiveHyphenLimit"
489+ , "hyphenationZone"
490+ , "doNotHyphenateCap"
491+ , "evenAndOddHeaders"
492+ , "proofState"
493+ , "compat"
578494 ]
579495 settingsEntry <- copyChildren refArchive distArchive settingsPath epochtime settingsList
580496
@@ -677,16 +593,17 @@ copyChildren :: (PandocMonad m)
677593copyChildren refArchive distArchive path timestamp elNames = do
678594 ref <- parseXml refArchive distArchive path
679595 dist <- parseXml distArchive distArchive path
596+ let elsToCopy =
597+ map cleanElem $ filterChildrenName (\e -> qName e `elem` elNames) ref
598+ let elsToKeep =
599+ [e | Elem e <- elContent dist, not (any (hasSameNameAs e) elsToCopy)]
680600 return $ toEntry path timestamp $ renderXml dist{
681− elContent = elContent dist ++ copyContent ref
601+ elContent = map Elem elsToKeep ++ map Elem elsToCopy
682602 }
683603 where
684− strName QName{qName=name, qPrefix=prefix}
685− | Just p <- prefix = p <> ":" <> name
686− | otherwise = name
687− shouldCopy = (`elem` elNames) . strName
688− cleanElem el@Element{elName=name} = Elem el{elName=name{qURI=Nothing}}
689− copyContent = map cleanElem . filterChildrenName shouldCopy
604+ hasSameNameAs (Element {elName = n1}) (Element {elName = n2}) =
605+ qName n1 == qName n2
606+ cleanElem el@Element{elName=name} = el{elName=name{qURI=Nothing}}
690607
691608-- this is the lowest number used for a list numId
692609baseListId :: Int
@@ -695,8 +612,7 @@ baseListId = 1000
695612mkNumbering :: [ListMarker] -> [Element]
696613mkNumbering lists =
697614 elts ++ zipWith mkNum lists [baseListId..(baseListId + length lists - 1)]
698− where elts = zipWith mkAbstractNum (ordNub lists) $
699− randomRs (0x10000000, 0xFFFFFFFF) $ mkStdGen 1848
615+ where elts = map mkAbstractNum (ordNub lists)
700616
701617maxListLevel :: Int
702618maxListLevel = 8
@@ -713,10 +629,10 @@ mkNum marker numid =
713629 $ mknode "w:startOverride" [("w:val",tshow start)] ())
714630 [0..maxListLevel]
715631
716−mkAbstractNum :: ListMarker -> Integer -> Element
717−mkAbstractNum marker nsid =
632+mkAbstractNum :: ListMarker -> Element
633+mkAbstractNum marker =
718634 mknode "w:abstractNum" [("w:abstractNumId",listMarkerToId marker)]
719− $ mknode "w:nsid" [("w:val", T.pack $ printf "%8x" nsid)] ()
635+ $ mknode "w:nsid" [("w:val", "A" <> listMarkerToId marker)] ()
720636 : mknode "w:multiLevelType" [("w:val","multilevel")] ()
721637 : map (mkLvl marker)
722638 [0..maxListLevel]
@@ -860,12 +776,6 @@ separateTables (x@Table{}:xs@(Table{}:_)) =
860776 x : RawBlock (Format "openxml") "<w:p />" : separateTables xs
861777separateTables (x:xs) = x : separateTables xs
862778
863−pStyleM :: (PandocMonad m) => ParaStyleName -> WS m XML.Element
864−pStyleM styleName = do
865− pStyleMap <- gets (smParaStyle . stStyleMaps)
866− let sty' = getStyleIdFromName styleName pStyleMap
867− return $ mknode "w:pStyle" [("w:val", fromStyleId sty')] ()
868−
869779rStyleM :: (PandocMonad m) => CharStyleName -> WS m XML.Element
870780rStyleM styleName = do
871781 cStyleMap <- gets (smCharStyle . stStyleMaps)
@@ -945,14 +855,28 @@ blockToOpenXML' opts (Plain lst) = do
945855-- title beginning with fig: indicates that the image is a figure
946856blockToOpenXML' opts (Para [Image attr alt (src,T.stripPrefix "fig:" -> Just tit)]) = do
947857 setFirstPara
858+ fignum <- gets stNextFigureNum
859+ unless (null alt) $ modify $ \st -> st{ stNextFigureNum = fignum + 1 }
860+ let figid = "fig" <> tshow fignum
861+ figname <- translateTerm Term.Figure
948862 prop <- pStyleM $
949863 if null alt
950864 then "Figure"
951865 else "Captioned Figure"
952866 paraProps <- local (\env -> env { envParaProperties = EnvProps (Just prop) [] <> envParaProperties env }) (getParaProps False)
953867 contents <- inlinesToOpenXML opts [Image attr alt (src,tit)]
954− captionNode <- withParaPropM (pStyleM "Image Caption")
955− $ blockToOpenXML opts (Para alt)
868+ captionNode <- if null alt
869+ then return []
870+ else withParaPropM (pStyleM "Image Caption")
871+ $ blockToOpenXML opts
872+ (Para $ Span (figid,[],[])
873+ [Str (figname <> "\160"),
874+ RawInline (Format "openxml")
875+ ("<w:fldSimple w:instr=\"SEQ Figure"
876+ <> " \\* ARABIC \"><w:r><w:t>"
877+ <> tshow fignum
878+ <> "</w:t></w:r></w:fldSimple>"),
879+ Str ":", Space] : alt)
956880 return $
957881 Elem (mknode "w:p" [] (map Elem paraProps ++ contents))
958882 : captionNode
@@ -997,78 +921,9 @@ blockToOpenXML' _ HorizontalRule = do
997921 $ mknode "v:rect" [("style","width:0;height:1.5pt"),
998922 ("o:hralign","center"),
999923 ("o:hrstd","t"),("o:hr","t")] () ]
1000−blockToOpenXML' opts (Table _ blkCapt specs thead tbody tfoot) = do
1001− let (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
1002− setFirstPara
1003− modify $ \s -> s { stInTable = True }
1004− let captionStr = stringify caption
1005− caption' <- if null caption
1006− then return []
1007− else withParaPropM (pStyleM "Table Caption")
1008− $ blockToOpenXML opts (Para caption)
1009− let alignmentFor al = mknode "w:jc" [("w:val",alignmentToString al)] ()
1010− -- Table cells require a <w:p> element, even an empty one!
1011− -- Not in the spec but in Word 2007, 2010. See #4953. And
1012− -- apparently the last element must be a <w:p>, see #6983.
1013− let cellToOpenXML (al, cell) = do
1014− es <- withParaProp (alignmentFor al) $ blocksToOpenXML opts cell
1015− return $
1016− case reverse (onlyElems es) of
1017− b:e:_ | qName (elName b) == "bookmarkEnd"
1018− , qName (elName e) == "p" -> es
1019− e:_ | qName (elName e) == "p" -> es
1020− _ -> es ++ [Elem $ mknode "w:p" [] ()]
1021− headers' <- mapM cellToOpenXML $ zip aligns headers
1022− rows' <- mapM (mapM cellToOpenXML . zip aligns) rows
1023− compactStyle <- pStyleM "Compact"
1024− let emptyCell' = [Elem $ mknode "w:p" [] [mknode "w:pPr" [] [compactStyle]]]
1025− let mkcell contents = mknode "w:tc" []
1026− $ if null contents
1027− then emptyCell'
1028− else contents
1029− let mkrow cells =
1030− mknode "w:tr" [] $
1031− map mkcell cells
1032− let textwidth = 7920 -- 5.5 in in twips, 1/20 pt
1033− let fullrow = 5000 -- 100% specified in pct
1034− let rowwidth = fullrow * sum widths
1035− let mkgridcol w = mknode "w:gridCol"
1036− [("w:w", tshow (floor (textwidth * w) :: Integer))] ()
1037− let hasHeader = not $ all null headers
1038− modify $ \s -> s { stInTable = False }
1039− -- for compatibility with Word <= 2007, we include a val with a bitmask
1040− -- 0×0020 Apply first row conditional formatting
1041− -- 0×0040 Apply last row conditional formatting
1042− -- 0×0080 Apply first column conditional formatting
1043− -- 0×0100 Apply last column conditional formatting
1044− -- 0×0200 Do not apply row banding conditional formatting
1045− -- 0×0400 Do not apply column banding conditional formattin
1046− let tblLookVal :: Int
1047− tblLookVal = if hasHeader then 0x20 else 0
1048− return $
1049− caption' ++
1050− [Elem $
1051− mknode "w:tbl" []
1052− ( mknode "w:tblPr" []
1053− ( mknode "w:tblStyle" [("w:val","Table")] () :
1054− mknode "w:tblW" [("w:type", "pct"), ("w:w", tshow rowwidth)] () :
1055− mknode "w:tblLook" [("w:firstRow",if hasHeader then "1" else "0")
1056− ,("w:lastRow","0")
1057− ,("w:firstColumn","0")
1058− ,("w:lastColumn","0")
1059− ,("w:noHBand","0")
1060− ,("w:noVBand","0")
1061− ,("w:val", T.pack $ printf "%04x" tblLookVal)
1062− ] () :
1063− [ mknode "w:tblCaption" [("w:val", captionStr)] ()
1064− | not (null caption) ] )
1065− : mknode "w:tblGrid" []
1066− (if all (==0) widths
1067− then []
1068− else map mkgridcol widths)
1069− : [ mkrow headers' | hasHeader ] ++
1070− map mkrow rows'
1071− )]
924+blockToOpenXML' opts (Table attr caption colspecs thead tbodies tfoot) =
925+ tableToOpenXML (blocksToOpenXML opts)
926+ (Grid.toTable attr caption colspecs thead tbodies tfoot)
1072927blockToOpenXML' opts el
1073928 | BulletList lst <- el = addOpenXMLList BulletMarker lst
1074929 | OrderedList (start, numstyle, numdelim) lst <- el
@@ -1123,13 +978,6 @@ listItemToOpenXML opts numid (first:rest) = do
1123978 modify $ \st -> st{ stInList = oldInList }
1124979 return $ first'' ++ rest''
1125980
1126−alignmentToString :: Alignment -> Text
1127−alignmentToString alignment = case alignment of
1128− AlignLeft -> "left"
1129− AlignRight -> "right"
1130− AlignCenter -> "center"
1131− AlignDefault -> "left"
1132−
1133981-- | Convert a list of inline elements to OpenXML.
1134982inlinesToOpenXML :: PandocMonad m => WriterOptions -> [Inline] -> WS m [Content]
1135983inlinesToOpenXML opts lst = concat `fmap` mapM (inlineToOpenXML opts) lst
@@ -1140,10 +988,6 @@ withNumId numid = local $ \env -> env{ envListNumId = numid }
1140988asList :: (PandocMonad m) => WS m a -> WS m a
1141989asList = local $ \env -> env{ envListLevel = envListLevel env + 1 }
1142990
1143−isStyle :: Element -> Bool
1144−isStyle e = isElem [] "w" "rStyle" e ||
1145− isElem [] "w" "pStyle" e
1146−
1147991getTextProps :: (PandocMonad m) => WS m [Element]
1148992getTextProps = do
1149993 props <- asks envTextProperties
@@ -1172,16 +1016,6 @@ getParaProps displayMathPara = do
11721016 [] -> []
11731017 ps -> [mknode "w:pPr" [] ps]
11741018
1175−withParaProp :: PandocMonad m => Element -> WS m a -> WS m a
1176−withParaProp d p =
1177− local (\env -> env {envParaProperties = ep <> envParaProperties env}) p
1178− where ep = if isStyle d then EnvProps (Just d) [] else EnvProps Nothing [d]
1179−
1180−withParaPropM :: PandocMonad m => WS m Element -> WS m a -> WS m a
1181−withParaPropM md p = do
1182− d <- md
1183− withParaProp d p
1184−
11851019formattedString :: PandocMonad m => Text -> WS m [Element]
11861020formattedString str =
11871021 -- properly handle soft hyphens
@@ -1202,9 +1036,6 @@ formattedRun els = do
12021036 props <- getTextProps
12031037 return [ mknode "w:r" [] $ props ++ els ]
12041038
1205−setFirstPara :: PandocMonad m => WS m ()
1206−setFirstPara = modify $ \s -> s { stFirstPara = True }
1207−
12081039-- | Convert an inline element to OpenXML.
12091040inlineToOpenXML :: PandocMonad m => WriterOptions -> Inline -> WS m [Content]
12101041inlineToOpenXML opts il = withDirection $ inlineToOpenXML' opts il
@@ -1473,6 +1304,7 @@ inlineToOpenXML' opts (Image attr@(imgident, _, _) alt (src, title)) = do
14731304 Just Eps -> ".eps"
14741305 Just Svg -> ".svg"
14751306 Just Emf -> ".emf"
1307+ Just Tiff -> ".tiff"
14761308 Nothing -> ""
14771309 imgpath = "media/" <> ident <> imgext
14781310 mbMimeType = mt <|> getMimeType (T.unpack imgpath)
@@ -1496,22 +1328,6 @@ inlineToOpenXML' opts (Image attr@(imgident, _, _) alt (src, title)) = do
14961328br :: Element
14971329br = mknode "w:r" [] [mknode "w:br" [] ()]
14981330
1499−-- Word will insert these footnotes into the settings.xml file
1500−-- (whether or not they're visible in the document). If they're in the
1501−-- file, but not in the footnotes.xml file, it will produce
1502−-- problems. So we want to make sure we insert them into our document.
1503−defaultFootnotes :: [Element]
1504−defaultFootnotes = [ mknode "w:footnote"
1505− [("w:type", "separator"), ("w:id", "-1")]
1506− [ mknode "w:p" []
1507− [mknode "w:r" []
1508− [ mknode "w:separator" [] ()]]]
1509− , mknode "w:footnote"
1510− [("w:type", "continuationSeparator"), ("w:id", "0")]
1511− [ mknode "w:p" []
1512− [ mknode "w:r" []
1513− [ mknode "w:continuationSeparator" [] ()]]]]
1514−
15151331
15161332withDirection :: PandocMonad m => WS m a -> WS m a
15171333withDirection x = do
src/Text/Pandoc/Writers/Docx/Table.hsnew file+227
@@ -0,0 +1,227 @@
1+{-# LANGUAGE LambdaCase #-}
2+{-# LANGUAGE OverloadedStrings #-}
3+{-# LANGUAGE ScopedTypeVariables #-}
4+{- |
5+Module : Text.Pandoc.Writers.Docx.Table
6+Copyright : Copyright (C) 2012-2021 John MacFarlane
7+License : GNU GPL, version 2 or above
8+Maintainer : John MacFarlane <jgm@berkeley.edu>
9+
10+Conversion of table blocks to docx.
11+-}
12+module Text.Pandoc.Writers.Docx.Table
13+ ( tableToOpenXML
14+ ) where
15+
16+import Control.Monad.State.Strict
17+import Data.Array
18+import Data.Text (Text)
19+import Text.Pandoc.Definition
20+import Text.Pandoc.Class.PandocMonad (PandocMonad, translateTerm)
21+import Text.Pandoc.Writers.Docx.Types
22+import Text.Pandoc.Shared
23+import Text.Printf (printf)
24+import Text.Pandoc.Writers.GridTable hiding (Table)
25+import Text.Pandoc.Writers.OOXML
26+import Text.Pandoc.XML.Light as XML hiding (Attr)
27+import qualified Data.Text as T
28+import qualified Text.Pandoc.Translations as Term
29+import qualified Text.Pandoc.Writers.GridTable as Grid
30+
31+tableToOpenXML :: PandocMonad m
32+ => ([Block] -> WS m [Content])
33+ -> Grid.Table
34+ -> WS m [Content]
35+tableToOpenXML blocksToOpenXML gridTable = do
36+ setFirstPara
37+ let (Grid.Table (ident,_,_) caption colspecs _rowheads thead tbodies tfoot) =
38+ gridTable
39+ let (Caption _maybeShortCaption captionBlocks) = caption
40+ tablenum <- gets stNextTableNum
41+ unless (null captionBlocks) $
42+ modify $ \st -> st{ stNextTableNum = tablenum + 1 }
43+ let tableid = if T.null ident
44+ then "table" <> tshow tablenum
45+ else ident
46+ tablename <- translateTerm Term.Table
47+ let captionStr = stringify captionBlocks
48+ let aligns = map fst $ elems colspecs
49+ captionXml <- if null captionBlocks
50+ then return []
51+ else withParaPropM (pStyleM "Table Caption")
52+ $ blocksToOpenXML
53+ $ addLabel tableid tablename tablenum captionBlocks
54+ -- We set "in table" after processing the caption, because we don't
55+ -- want the "Table Caption" style to be overwritten with "Compact".
56+ modify $ \s -> s { stInTable = True }
57+ head' <- cellGridToOpenXML blocksToOpenXML HeadRow aligns thead
58+ bodies <- mapM (cellGridToOpenXML blocksToOpenXML BodyRow aligns) tbodies
59+ foot' <- cellGridToOpenXML blocksToOpenXML FootRow aligns tfoot
60+
61+ let hasHeader = not . null . indices . partRowAttrs $ thead
62+ let hasFooter = not . null . indices . partRowAttrs $ tfoot
63+ -- for compatibility with Word <= 2007, we include a val with a bitmask
64+ -- 0×0020 Apply first row conditional formatting
65+ -- 0×0040 Apply last row conditional formatting
66+ -- 0×0080 Apply first column conditional formatting
67+ -- 0×0100 Apply last column conditional formatting
68+ -- 0×0200 Do not apply row banding conditional formatting
69+ -- 0×0400 Do not apply column banding conditional formattin
70+ let tblLookVal = if hasHeader then (0x20 :: Int) else 0
71+ let (gridCols, tblWattr) = tableLayout (elems colspecs)
72+ let tbl = mknode "w:tbl" []
73+ ( mknode "w:tblPr" []
74+ ( mknode "w:tblStyle" [("w:val","Table")] () :
75+ mknode "w:tblW" tblWattr () :
76+ mknode "w:tblLook" [("w:firstRow",if hasHeader then "1" else "0")
77+ ,("w:lastRow",if hasFooter then "1" else "0")
78+ ,("w:firstColumn","0")
79+ ,("w:lastColumn","0")
80+ ,("w:noHBand","0")
81+ ,("w:noVBand","0")
82+ ,("w:val", T.pack $ printf "%04x" tblLookVal)
83+ ] () :
84+ [ mknode "w:tblCaption" [("w:val", captionStr)] ()
85+ | not (T.null captionStr) ]
86+ )
87+ : mknode "w:tblGrid" [] gridCols
88+ : head' ++ mconcat bodies ++ foot'
89+ )
90+ modify $ \s -> s { stInTable = False }
91+ return $ captionXml ++ [Elem tbl]
92+
93+addLabel :: Text -> Text -> Int -> [Block] -> [Block]
94+addLabel tableid tablename tablenum bs =
95+ case bs of
96+ (Para ils : rest) -> Para (label : Space : ils) : rest
97+ (Plain ils : rest) -> Plain (label : Space : ils) : rest
98+ _ -> Para [label] : bs
99+ where
100+ label = Span (tableid,[],[])
101+ [Str (tablename <> "\160"),
102+ RawInline (Format "openxml")
103+ ("<w:fldSimple w:instr=\"SEQ Table"
104+ <> " \\* ARABIC \"><w:r><w:t>"
105+ <> tshow tablenum
106+ <> "</w:t></w:r></w:fldSimple>"),
107+ Str ":"]
108+
109+-- | Parts of a table
110+data RowType = HeadRow | BodyRow | FootRow
111+
112+alignmentToString :: Alignment -> Text
113+alignmentToString = \case
114+ AlignLeft -> "left"
115+ AlignRight -> "right"
116+ AlignCenter -> "center"
117+ AlignDefault -> "left"
118+
119+tableLayout :: [ColSpec] -> ([Element], [(Text, Text)])
120+tableLayout specs =
121+ let
122+ textwidth = 7920 -- 5.5 in in twips (1 twip == 1/20 pt)
123+ fullrow = 5000 -- 100% specified in pct (1 pct == 1/50th of a percent)
124+ ncols = length specs
125+ getWidth = \case
126+ ColWidth n -> n
127+ _ -> 0
128+ widths = map (getWidth . snd) specs
129+ rowwidth = round (fullrow * sum widths) :: Int
130+ widthToTwips w = floor (textwidth * w) :: Int
131+ mkGridCol w = mknode "w:gridCol" [("w:w", tshow (widthToTwips w))] ()
132+ in if all (== 0) widths
133+ then ( replicate ncols $ mkGridCol (1.0 / fromIntegral ncols)
134+ , [ ("w:type", "auto"), ("w:w", "0")])
135+ else ( map mkGridCol widths
136+ , [ ("w:type", "pct"), ("w:w", tshow rowwidth) ])
137+
138+cellGridToOpenXML :: PandocMonad m
139+ => ([Block] -> WS m [Content])
140+ -> RowType
141+ -> [Alignment]
142+ -> Part
143+ -> WS m [Element]
144+cellGridToOpenXML blocksToOpenXML rowType aligns part@(Part _ cellArray _) =
145+ if null (elems cellArray)
146+ then return mempty
147+ else mapM (rowToOpenXML blocksToOpenXML) $
148+ partToRows rowType aligns part
149+
150+data OOXMLCell
151+ = OOXMLCell Attr Alignment RowSpan ColSpan [Block]
152+ | OOXMLCellMerge ColSpan
153+
154+data OOXMLRow = OOXMLRow RowType Attr [OOXMLCell]
155+
156+partToRows :: RowType -> [Alignment] -> Part -> [OOXMLRow]
157+partToRows rowType aligns part =
158+ let
159+ toOOXMLCell :: Alignment -> RowIndex -> ColIndex -> GridCell -> [OOXMLCell]
160+ toOOXMLCell columnAlign ridx cidx = \case
161+ ContentCell attr align rowspan colspan blocks ->
162+ -- Respect non-default, cell specific alignment.
163+ let align' = case align of
164+ AlignDefault -> columnAlign
165+ _ -> align
166+ in [OOXMLCell attr align' rowspan colspan blocks]
167+ ContinuationCell idx'@(ridx',cidx') | ridx /= ridx', cidx == cidx' ->
168+ case (partCellArray part)!idx' of
169+ (ContentCell _ _ _ colspan _) -> [OOXMLCellMerge colspan]
170+ x -> error $ "Content cell expected, got, " ++ show x ++
171+ " at index " ++ show idx'
172+ _ -> mempty
173+ mkRow :: (RowIndex, Attr) -> OOXMLRow
174+ mkRow (ridx, attr) = OOXMLRow rowType attr
175+ . mconcat
176+ . zipWith (\align -> uncurry $ toOOXMLCell align ridx)
177+ aligns
178+ . assocs
179+ . rowArray ridx
180+ $ partCellArray part
181+ in map mkRow $ assocs (partRowAttrs part)
182+
183+rowToOpenXML :: PandocMonad m
184+ => ([Block] -> WS m [Content])
185+ -> OOXMLRow
186+ -> WS m Element
187+rowToOpenXML blocksToOpenXML (OOXMLRow rowType _attr cells) = do
188+ xmlcells <- mapM (ooxmlCellToOpenXML blocksToOpenXML) cells
189+ let addTrPr = case rowType of
190+ HeadRow -> (mknode "w:trPr" []
191+ [mknode "w:tblHeader" [("w:val", "true")] ()] :)
192+ BodyRow -> id
193+ FootRow -> id
194+ return $ mknode "w:tr" [] (addTrPr xmlcells)
195+
196+ooxmlCellToOpenXML :: PandocMonad m
197+ => ([Block] -> WS m [Content])
198+ -> OOXMLCell
199+ -> WS m Element
200+ooxmlCellToOpenXML blocksToOpenXML = \case
201+ OOXMLCellMerge (ColSpan colspan) -> do
202+ return $ mknode "w:tc" []
203+ [ mknode "w:tcPr" [] [ mknode "w:gridSpan" [("w:val", tshow colspan)] ()
204+ , mknode "w:vMerge" [("w:val", "continue")] () ]
205+ , mknode "w:p" [] [mknode "w:pPr" [] ()]]
206+ OOXMLCell _attr align rowspan (ColSpan colspan) contents -> do
207+ compactStyle <- pStyleM "Compact"
208+ es <- withParaProp (alignmentFor align) $ blocksToOpenXML contents
209+ -- Table cells require a <w:p> element, even an empty one!
210+ -- Not in the spec but in Word 2007, 2010. See #4953. And
211+ -- apparently the last element must be a <w:p>, see #6983.
212+ return . mknode "w:tc" [] $
213+ Elem
214+ (mknode "w:tcPr" [] ([ mknode "w:gridSpan" [("w:val", tshow colspan)] ()
215+ | colspan > 1] ++
216+ [ mknode "w:vMerge" [("w:val", "restart")] ()
217+ | rowspan > RowSpan 1 ])) :
218+ if null contents
219+ then [Elem $ mknode "w:p" [] [mknode "w:pPr" [] [compactStyle]]]
220+ else case reverse (onlyElems es) of
221+ b:e:_ | qName (elName b) == "bookmarkEnd" -- y tho?
222+ , qName (elName e) == "p" -> es
223+ e:_ | qName (elName e) == "p" -> es
224+ _ -> es ++ [Elem $ mknode "w:p" [] ()]
225+
226+alignmentFor :: Alignment -> Element
227+alignmentFor al = mknode "w:jc" [("w:val",alignmentToString al)] ()
src/Text/Pandoc/Writers/Docx/Types.hsnew file+185
@@ -0,0 +1,185 @@
1+{-# LANGUAGE OverloadedStrings #-}
2+{- |
3+Module : Text.Pandoc.Writers.Docx
4+Copyright : Copyright (C) 2012-2021 John MacFarlane
5+License : GNU GPL, version 2 or above
6+Maintainer : John MacFarlane <jgm@berkeley.edu>
7+
8+Conversion of table blocks to docx.
9+-}
10+module Text.Pandoc.Writers.Docx.Types
11+ ( EnvProps (..)
12+ , WriterEnv (..)
13+ , defaultWriterEnv
14+ , WriterState (..)
15+ , defaultWriterState
16+ , WS
17+ , ListMarker (..)
18+ , listMarkerToId
19+ , pStyleM
20+ , isStyle
21+ , setFirstPara
22+ , withParaProp
23+ , withParaPropM
24+ ) where
25+
26+import Control.Applicative ((<|>))
27+import Control.Monad.Reader
28+import Control.Monad.State.Strict
29+import Data.Text (Text)
30+import Text.Pandoc.Class.PandocMonad (PandocMonad)
31+import Text.Pandoc.Definition
32+import Text.Pandoc.MIME (MimeType)
33+import Text.Pandoc.Writers.Docx.StyleMap
34+import Text.Pandoc.Writers.OOXML
35+import Text.Pandoc.XML.Light as XML
36+import qualified Data.ByteString as B
37+import qualified Data.Map as M
38+import qualified Data.Set as Set
39+import qualified Data.Text as T
40+
41+data ListMarker = NoMarker
42+ | BulletMarker
43+ | NumberMarker ListNumberStyle ListNumberDelim Int
44+ deriving (Show, Read, Eq, Ord)
45+
46+listMarkerToId :: ListMarker -> Text
47+listMarkerToId NoMarker = "990"
48+listMarkerToId BulletMarker = "991"
49+listMarkerToId (NumberMarker sty delim n) = T.pack $
50+ '9' : '9' : styNum : delimNum : show n
51+ where styNum = case sty of
52+ DefaultStyle -> '2'
53+ Example -> '3'
54+ Decimal -> '4'
55+ LowerRoman -> '5'
56+ UpperRoman -> '6'
57+ LowerAlpha -> '7'
58+ UpperAlpha -> '8'
59+ delimNum = case delim of
60+ DefaultDelim -> '0'
61+ Period -> '1'
62+ OneParen -> '2'
63+ TwoParens -> '3'
64+
65+
66+data EnvProps = EnvProps{ styleElement :: Maybe Element
67+ , otherElements :: [Element]
68+ }
69+
70+instance Semigroup EnvProps where
71+ EnvProps s es <> EnvProps s' es' = EnvProps (s <|> s') (es ++ es')
72+
73+instance Monoid EnvProps where
74+ mempty = EnvProps Nothing []
75+ mappend = (<>)
76+
77+data WriterEnv = WriterEnv
78+ { envTextProperties :: EnvProps
79+ , envParaProperties :: EnvProps
80+ , envRTL :: Bool
81+ , envListLevel :: Int
82+ , envListNumId :: Int
83+ , envInDel :: Bool
84+ , envChangesAuthor :: Text
85+ , envChangesDate :: Text
86+ , envPrintWidth :: Integer
87+ }
88+
89+defaultWriterEnv :: WriterEnv
90+defaultWriterEnv = WriterEnv
91+ { envTextProperties = mempty
92+ , envParaProperties = mempty
93+ , envRTL = False
94+ , envListLevel = -1
95+ , envListNumId = 1
96+ , envInDel = False
97+ , envChangesAuthor = "unknown"
98+ , envChangesDate = "1969-12-31T19:00:00Z"
99+ , envPrintWidth = 1
100+ }
101+
102+
103+data WriterState = WriterState{
104+ stFootnotes :: [Element]
105+ , stComments :: [([(Text, Text)], [Inline])]
106+ , stSectionIds :: Set.Set Text
107+ , stExternalLinks :: M.Map Text Text
108+ , stImages :: M.Map FilePath (String, String, Maybe MimeType, B.ByteString)
109+ , stLists :: [ListMarker]
110+ , stInsId :: Int
111+ , stDelId :: Int
112+ , stStyleMaps :: StyleMaps
113+ , stFirstPara :: Bool
114+ , stInTable :: Bool
115+ , stInList :: Bool
116+ , stTocTitle :: [Inline]
117+ , stDynamicParaProps :: Set.Set ParaStyleName
118+ , stDynamicTextProps :: Set.Set CharStyleName
119+ , stCurId :: Int
120+ , stNextFigureNum :: Int
121+ , stNextTableNum :: Int
122+ }
123+
124+defaultWriterState :: WriterState
125+defaultWriterState = WriterState{
126+ stFootnotes = defaultFootnotes
127+ , stComments = []
128+ , stSectionIds = Set.empty
129+ , stExternalLinks = M.empty
130+ , stImages = M.empty
131+ , stLists = [NoMarker]
132+ , stInsId = 1
133+ , stDelId = 1
134+ , stStyleMaps = StyleMaps M.empty M.empty
135+ , stFirstPara = False
136+ , stInTable = False
137+ , stInList = False
138+ , stTocTitle = [Str "Table of Contents"]
139+ , stDynamicParaProps = Set.empty
140+ , stDynamicTextProps = Set.empty
141+ , stCurId = 20
142+ , stNextFigureNum = 1
143+ , stNextTableNum = 1
144+ }
145+
146+setFirstPara :: PandocMonad m => WS m ()
147+setFirstPara = modify $ \s -> s { stFirstPara = True }
148+
149+type WS m = ReaderT WriterEnv (StateT WriterState m)
150+
151+-- Word will insert these footnotes into the settings.xml file
152+-- (whether or not they're visible in the document). If they're in the
153+-- file, but not in the footnotes.xml file, it will produce
154+-- problems. So we want to make sure we insert them into our document.
155+defaultFootnotes :: [Element]
156+defaultFootnotes = [ mknode "w:footnote"
157+ [("w:type", "separator"), ("w:id", "-1")]
158+ [ mknode "w:p" []
159+ [mknode "w:r" []
160+ [ mknode "w:separator" [] ()]]]
161+ , mknode "w:footnote"
162+ [("w:type", "continuationSeparator"), ("w:id", "0")]
163+ [ mknode "w:p" []
164+ [ mknode "w:r" []
165+ [ mknode "w:continuationSeparator" [] ()]]]]
166+
167+pStyleM :: (PandocMonad m) => ParaStyleName -> WS m XML.Element
168+pStyleM styleName = do
169+ pStyleMap <- gets (smParaStyle . stStyleMaps)
170+ let sty' = getStyleIdFromName styleName pStyleMap
171+ return $ mknode "w:pStyle" [("w:val", fromStyleId sty')] ()
172+
173+withParaProp :: PandocMonad m => Element -> WS m a -> WS m a
174+withParaProp d p =
175+ local (\env -> env {envParaProperties = ep <> envParaProperties env}) p
176+ where ep = if isStyle d then EnvProps (Just d) [] else EnvProps Nothing [d]
177+
178+withParaPropM :: PandocMonad m => WS m Element -> WS m a -> WS m a
179+withParaPropM md p = do
180+ d <- md
181+ withParaProp d p
182+
183+isStyle :: Element -> Bool
184+isStyle e = isElem [] "w" "rStyle" e ||
185+ isElem [] "w" "pStyle" e
src/Text/Pandoc/Writers/DokuWiki.hs+3−1
@@ -27,6 +27,7 @@ import Control.Monad.Reader (ReaderT, asks, local, runReaderT)
2727import Control.Monad.State.Strict (StateT, evalStateT)
2828import Data.Default (Default (..))
2929import Data.List (transpose)
30+import Data.List.NonEmpty (nonEmpty)
3031import Data.Text (Text)
3132import qualified Data.Text as T
3233import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
@@ -172,7 +173,8 @@ blockToDokuWiki opts (Table _ blkCapt specs thead tbody tfoot) = do
172173 then return []
173174 else zipWithM (tableItemToDokuWiki opts) aligns headers
174175 rows' <- mapM (zipWithM (tableItemToDokuWiki opts) aligns) rows
175− let widths = map (maximum . map T.length) $ transpose (headers':rows')
176+ let widths = map (maybe 0 maximum . nonEmpty . map T.length)
177+ $ transpose (headers':rows')
176178 let padTo (width, al) s =
177179 case width - T.length s of
178180 x | x > 0 ->
src/Text/Pandoc/Writers/EPUB.hs+3−3
@@ -991,12 +991,12 @@ metadataElement version md currentTime =
991991 showDateTimeISO8601 currentTime | version == EPUB3 ]
992992 belongsToCollectionNodes =
993993 maybe []
994− (\belongsToCollection -> (unode "meta" ! [("property", "belongs-to-collection"), ("id", "epub-id-1")] $ belongsToCollection )
994+ (\belongsToCollection -> (unode "meta" ! [("property", "belongs-to-collection"), ("id", "epub-collection-1")] $ belongsToCollection )
995995 :
996− [unode "meta" ! [("refines", "#epub-id-1"), ("property", "collection-type")] $ ("series" :: Text) ])
996+ [unode "meta" ! [("refines", "#epub-collection-1"), ("property", "collection-type")] $ ("series" :: Text) ])
997997 (epubBelongsToCollection md)++
998998 maybe []
999− (\groupPosition -> [unode "meta" ! [("refines", "#epub-id-1"), ("property", "group-position")] $ groupPosition ])
999+ (\groupPosition -> [unode "meta" ! [("refines", "#epub-collection-1"), ("property", "group-position")] $ groupPosition ])
10001000 (epubGroupPosition md)
10011001 dcTag n s = unode ("dc:" <> n) s
10021002 dcTag' n s = [dcTag n s]
src/Text/Pandoc/Writers/GridTable.hsnew file+157
@@ -0,0 +1,157 @@
1+{-# LANGUAGE FlexibleContexts #-}
2+{-# LANGUAGE GeneralizedNewtypeDeriving #-}
3+{-# LANGUAGE LambdaCase #-}
4+{-# LANGUAGE TupleSections #-}
5+
6+{- |
7+Module : Text.Pandoc.Writers.GridTable
8+Copyright : © 2020-2021 Albert Krewinkel
9+License : GNU GPL, version 2 or above
10+
11+Maintainer : Albert Krewinkel <albert@zeitkraut.de>
12+
13+Grid representation of pandoc tables.
14+-}
15+module Text.Pandoc.Writers.GridTable
16+ ( Table (..)
17+ , GridCell (..)
18+ , RowIndex (..)
19+ , ColIndex (..)
20+ , CellIndex
21+ , Part (..)
22+ , toTable
23+ , rowArray
24+ ) where
25+
26+import Control.Monad (forM_)
27+import Control.Monad.ST
28+import Data.Array
29+import Data.Array.MArray
30+import Data.Array.ST
31+import Data.Maybe (listToMaybe)
32+import Data.STRef
33+import Text.Pandoc.Definition hiding (Table)
34+import qualified Text.Pandoc.Builder as B
35+
36+-- | A grid cell contains either a real table cell, or is the
37+-- continuation of a column or row-spanning cell. In the latter case,
38+-- the index of the continued cell is provided.
39+data GridCell
40+ = ContentCell Attr Alignment RowSpan ColSpan [Block]
41+ | ContinuationCell CellIndex
42+ deriving (Show)
43+
44+-- | Row index in a table part.
45+newtype RowIndex = RowIndex Int deriving (Enum, Eq, Ix, Ord, Show)
46+-- | Column index in a table part.
47+newtype ColIndex = ColIndex Int deriving (Enum, Eq, Ix, Ord, Show)
48+
49+-- | Index to a cell in a table part.
50+type CellIndex = (RowIndex, ColIndex)
51+
52+-- | Cells are placed on a grid. Row attributes are stored in a separate
53+-- array.
54+data Part = Part
55+ { partAttr :: Attr
56+ , partCellArray :: Array (RowIndex,ColIndex) GridCell
57+ , partRowAttrs :: Array RowIndex Attr
58+ }
59+
60+data Table = Table
61+ { tableAttr :: Attr
62+ , tableCaption :: Caption
63+ , tableColSpecs :: Array ColIndex ColSpec
64+ , tableRowHeads :: RowHeadColumns
65+ , tableHead :: Part
66+ , tableBodies :: [Part]
67+ , tableFoot :: Part
68+ }
69+
70+toTable
71+ :: B.Attr
72+ -> B.Caption
73+ -> [B.ColSpec]
74+ -> B.TableHead
75+ -> [B.TableBody]
76+ -> B.TableFoot
77+ -> Table
78+toTable attr caption colSpecs thead tbodies tfoot =
79+ Table attr caption colSpecs' rowHeads thGrid tbGrids tfGrid
80+ where
81+ colSpecs' = listArray (ColIndex 1, ColIndex $ length colSpecs) colSpecs
82+ rowHeads = case listToMaybe tbodies of
83+ Nothing -> RowHeadColumns 0
84+ Just (TableBody _attr rowHeadCols _headerRows _rows) -> rowHeadCols
85+ thGrid = let (TableHead headAttr rows) = thead
86+ in rowsToPart headAttr rows
87+ tbGrids = map bodyToGrid tbodies
88+ tfGrid = let (TableFoot footAttr rows) = tfoot
89+ in rowsToPart footAttr rows
90+ bodyToGrid (TableBody bodyAttr _rowHeadCols headRows rows) =
91+ rowsToPart bodyAttr (headRows ++ rows)
92+
93+data BuilderCell
94+ = FilledCell GridCell
95+ | FreeCell
96+
97+fromBuilderCell :: BuilderCell -> GridCell
98+fromBuilderCell = \case
99+ FilledCell c -> c
100+ FreeCell -> error "Found an unassigned cell."
101+
102+rowsToPart :: Attr -> [B.Row] -> Part
103+rowsToPart attr = \case
104+ [] -> Part
105+ attr
106+ (listArray ((RowIndex 1, ColIndex 1), (RowIndex 0, ColIndex 0)) [])
107+ (listArray (RowIndex 1, RowIndex 0) [])
108+ rows@(Row _attr firstRow:_) ->
109+ let nrows = length rows
110+ ncols = sum $ map (\(Cell _ _ _ (ColSpan cs) _) -> cs) firstRow
111+ gbounds = ((RowIndex 1, ColIndex 1), (RowIndex nrows, ColIndex ncols))
112+ mutableGrid :: ST s (STArray s CellIndex GridCell)
113+ mutableGrid = do
114+ grid <- newArray gbounds FreeCell
115+ ridx <- newSTRef (RowIndex 1)
116+ forM_ rows $ \(Row _attr cells) -> do
117+ cidx <- newSTRef (ColIndex 1)
118+ forM_ cells $ \(Cell cellAttr align rs cs blks) -> do
119+ ridx' <- readSTRef ridx
120+ let nextFreeInRow colindex@(ColIndex c) = do
121+ readArray grid (ridx', colindex) >>= \case
122+ FreeCell -> pure colindex
123+ _ -> nextFreeInRow $ ColIndex (c + 1)
124+ cidx' <- readSTRef cidx >>= nextFreeInRow
125+ writeArray grid (ridx', cidx') . FilledCell $
126+ ContentCell cellAttr align rs cs blks
127+ forM_ (continuationIndices ridx' cidx' rs cs) $ \idx -> do
128+ writeArray grid idx . FilledCell $
129+ ContinuationCell (ridx', cidx')
130+ -- go to new column
131+ writeSTRef cidx cidx'
132+ -- go to next row
133+ modifySTRef ridx (incrRowIndex 1)
134+ -- Swap BuilderCells with normal GridCells.
135+ mapArray fromBuilderCell grid
136+ in Part
137+ { partCellArray = runSTArray mutableGrid
138+ , partRowAttrs = listArray (RowIndex 1, RowIndex nrows) $
139+ map (\(Row rowAttr _) -> rowAttr) rows
140+ , partAttr = attr
141+ }
142+
143+continuationIndices :: RowIndex -> ColIndex -> RowSpan -> ColSpan -> [CellIndex]
144+continuationIndices (RowIndex ridx) (ColIndex cidx) rowspan colspan =
145+ let (RowSpan rs) = rowspan
146+ (ColSpan cs) = colspan
147+ in [ (RowIndex r, ColIndex c) | r <- [ridx..(ridx + rs - 1)]
148+ , c <- [cidx..(cidx + cs - 1)]
149+ , (r, c) /= (ridx, cidx)]
150+
151+rowArray :: RowIndex -> Array CellIndex GridCell -> Array ColIndex GridCell
152+rowArray ridx grid =
153+ let ((_minRidx, minCidx), (_maxRidx, maxCidx)) = bounds grid
154+ in ixmap (minCidx, maxCidx) (ridx,) grid
155+
156+incrRowIndex :: RowSpan -> RowIndex -> RowIndex
157+incrRowIndex (RowSpan n) (RowIndex r) = RowIndex $ r + n
src/Text/Pandoc/Writers/HTML.hs+107−34
@@ -26,13 +26,18 @@ module Text.Pandoc.Writers.HTML (
2626 writeSlideous,
2727 writeDZSlides,
2828 writeRevealJs,
29− tagWithAttributes
29+ tagWithAttributes,
30+ inlineToHtml,
31+ blockToHtml,
32+ WriterState(..),
33+ defaultWriterState
3034 ) where
35+
3136import Control.Monad.State.Strict
3237import Data.Char (ord)
33−import Data.List (intercalate, intersperse, partition, delete, (\\))
38+import Data.List (intercalate, intersperse, partition, delete, (\\), foldl')
3439import Data.List.NonEmpty (NonEmpty((:|)))
35−import Data.Maybe (fromMaybe, isJust, isNothing, mapMaybe)
40+import Data.Maybe (fromMaybe, isJust, isNothing)
3641import qualified Data.Set as Set
3742import Data.Text (Text)
3843import qualified Data.Text as T
@@ -344,6 +349,52 @@ pandocToHtml opts (Pandoc meta blocks) = do
344349 PlainMath -> defField "displaymath-css" True
345350 WebTeX _ -> defField "displaymath-css" True
346351 _ -> id) .
352+ (if slideVariant == RevealJsSlides
353+ then -- set boolean options explicitly, since
354+ -- template can't distinguish False/undefined
355+ defField "controls" True .
356+ defField "controlsTutorial" True .
357+ defField "controlsLayout" ("bottom-right" :: Text) .
358+ defField "controlsBackArrows" ("faded" :: Text) .
359+ defField "progress" True .
360+ defField "slideNumber" False .
361+ defField "showSlideNumber" ("all" :: Text) .
362+ defField "hashOneBasedIndex" False .
363+ defField "hash" False .
364+ defField "respondToHashChanges" True .
365+ defField "history" False .
366+ defField "keyboard" True .
367+ defField "overview" True .
368+ defField "disableLayout" False .
369+ defField "center" True .
370+ defField "touch" True .
371+ defField "loop" False .
372+ defField "rtl" False .
373+ defField "navigationMode" ("default" :: Text) .
374+ defField "shuffle" False .
375+ defField "fragments" True .
376+ defField "fragmentInURL" True .
377+ defField "embedded" False .
378+ defField "help" True .
379+ defField "pause" True .
380+ defField "showNotes" False .
381+ defField "autoPlayMedia" ("null" :: Text) .
382+ defField "preloadIframes" ("null" :: Text) .
383+ defField "autoSlide" ("0" :: Text) .
384+ defField "autoSlideStoppable" True .
385+ defField "autoSlideMethod" ("null" :: Text) .
386+ defField "defaultTiming" ("null" :: Text) .
387+ defField "mouseWheel" False .
388+ defField "display" ("block" :: Text) .
389+ defField "hideInactiveCursor" True .
390+ defField "hideCursorTime" ("5000" :: Text) .
391+ defField "previewLinks" False .
392+ defField "transition" ("slide" :: Text) .
393+ defField "transitionSpeed" ("default" :: Text) .
394+ defField "backgroundTransition" ("fade" :: Text) .
395+ defField "viewDistance" ("3" :: Text) .
396+ defField "mobileViewDistance" ("2" :: Text)
397+ else id) .
347398 defField "document-css" (isNothing mCss && slideVariant == NoSlides) .
348399 defField "quotes" (stQuotes st) .
349400 -- for backwards compatibility we populate toc
@@ -544,30 +595,35 @@ tagWithAttributes opts html5 selfClosing tagname attr =
544595
545596addAttrs :: PandocMonad m
546597 => WriterOptions -> Attr -> Html -> StateT WriterState m Html
547−addAttrs opts attr h = foldl (!) h <$> attrsToHtml opts attr
598+addAttrs opts attr h = foldl' (!) h <$> attrsToHtml opts attr
548599
549600toAttrs :: PandocMonad m
550601 => [(Text, Text)] -> StateT WriterState m [Attribute]
551602toAttrs kvs = do
552603 html5 <- gets stHtml5
553604 mbEpubVersion <- gets stEPUBVersion
554− return $ mapMaybe (\(x,y) ->
555− if html5
556− then
557− if x `Set.member` (html5Attributes <> rdfaAttributes)
558− || T.any (== ':') x -- e.g. epub: namespace
559− || "data-" `T.isPrefixOf` x
560− || "aria-" `T.isPrefixOf` x
561− then Just $ customAttribute (textTag x) (toValue y)
562− else Just $ customAttribute (textTag ("data-" <> x))
563− (toValue y)
564− else
565− if mbEpubVersion == Just EPUB2 &&
566− not (x `Set.member` (html4Attributes <> rdfaAttributes) ||
567− "xml:" `T.isPrefixOf` x)
568− then Nothing
569− else Just $ customAttribute (textTag x) (toValue y))
570− kvs
605+ reverse . snd <$> foldM (go html5 mbEpubVersion) (Set.empty, []) kvs
606+ where
607+ go html5 mbEpubVersion (keys, attrs) (k,v) = do
608+ if k `Set.member` keys
609+ then do
610+ report $ DuplicateAttribute k v
611+ return (keys, attrs)
612+ else return (Set.insert k keys, addAttr html5 mbEpubVersion k v attrs)
613+ addAttr html5 mbEpubVersion x y
614+ | html5
615+ = if x `Set.member` (html5Attributes <> rdfaAttributes)
616+ || T.any (== ':') x -- e.g. epub: namespace
617+ || "data-" `T.isPrefixOf` x
618+ || "aria-" `T.isPrefixOf` x
619+ then (customAttribute (textTag x) (toValue y) :)
620+ else (customAttribute (textTag ("data-" <> x)) (toValue y) :)
621+ | mbEpubVersion == Just EPUB2
622+ , not (x `Set.member` (html4Attributes <> rdfaAttributes) ||
623+ "xml:" `T.isPrefixOf` x)
624+ = id
625+ | otherwise
626+ = (customAttribute (textTag x) (toValue y) :)
571627
572628attrsToHtml :: PandocMonad m
573629 => WriterOptions -> Attr -> StateT WriterState m [Attribute]
@@ -608,17 +664,20 @@ dimensionsToAttrList attr = consolidateStyles $ go Width ++ go Height
608664figure :: PandocMonad m
609665 => WriterOptions -> Attr -> [Inline] -> (Text, Text)
610666 -> StateT WriterState m Html
611−figure opts attr txt (s,tit) = do
667+figure opts attr@(_, _, attrList) txt (s,tit) = do
612668 html5 <- gets stHtml5
613669 -- Screen-readers will normally read the @alt@ text and the figure; we
614670 -- want to avoid them reading the same text twice. With HTML5 we can
615671 -- use aria-hidden for the caption; with HTML4, we use an empty
616672 -- alt-text instead.
673+ -- When the alt text differs from the caption both should be read.
617674 let alt = if html5 then txt else [Str ""]
618675 let tocapt = if html5
619− then H5.figcaption !
620− H5.customAttribute (textTag "aria-hidden")
621− (toValue @Text "true")
676+ then (H5.figcaption !) $
677+ if isJust (lookup "alt" attrList)
678+ then mempty
679+ else H5.customAttribute (textTag "aria-hidden")
680+ (toValue @Text "true")
622681 else H.p ! A.class_ "caption"
623682 img <- inlineToHtml opts (Image attr alt (s,tit))
624683 capt <- if null txt
@@ -765,9 +824,10 @@ blockToHtml opts (Div attr@(ident, classes, kvs') bs) = do
765824 lookup "entry-spacing" kvs' >>=
766825 safeRead }
767826 let isCslBibEntry = "csl-entry" `elem` classes
768− let kvs = [(k,v) | (k,v) <- kvs', k /= "width"] ++
769− [("style", "width:" <> w <> ";") | "column" `elem` classes,
770− ("width", w) <- kvs'] ++
827+ let kvs = [(k,v) | (k,v) <- kvs'
828+ , k /= "width" || "column" `notElem` classes] ++
829+ [("style", "width:" <> w <> ";") | "column" `elem` classes
830+ , ("width", w) <- kvs'] ++
771831 [("role", "doc-bibliography") | isCslBibBody && html5] ++
772832 [("role", "doc-biblioentry") | isCslBibEntry && html5]
773833 let speakerNotes = "notes" `elem` classes
@@ -877,7 +937,7 @@ blockToHtml opts (BlockQuote blocks) = do
877937 else do
878938 contents <- blockListToHtml opts blocks
879939 return $ H.blockquote $ nl opts >> contents >> nl opts
880−blockToHtml opts (Header level attr@(_,classes,kvs) lst) = do
940+blockToHtml opts (Header level (ident,classes,kvs) lst) = do
881941 contents <- inlineListToHtml opts lst
882942 let secnum = fromMaybe mempty $ lookup "number" kvs
883943 let contents' = if writerNumberSections opts && not (T.null secnum)
@@ -885,7 +945,13 @@ blockToHtml opts (Header level attr@(_,classes,kvs) lst) = do
885945 then (H.span ! A.class_ "header-section-number"
886946 $ toHtml secnum) >> strToHtml " " >> contents
887947 else contents
888− addAttrs opts attr
948+ html5 <- gets stHtml5
949+ let kvs' = if html5
950+ then kvs
951+ else [ (k, v) | (k, v) <- kvs
952+ , k `elem` (["lang", "dir", "title", "style"
953+ , "align"] ++ intrinsicEventsHTML4)]
954+ addAttrs opts (ident,classes,kvs')
889955 $ case level of
890956 1 -> H.h1 contents'
891957 2 -> H.h2 contents'
@@ -921,7 +987,7 @@ blockToHtml opts (OrderedList (startnum, numstyle, _) lst) = do
921987 numstyle']
922988 else [])
923989 l <- ordList opts contents
924− return $ foldl (!) l attribs
990+ return $ foldl' (!) l attribs
925991blockToHtml opts (DefinitionList lst) = do
926992 contents <- mapM (\(term, defs) ->
927993 do term' <- liftM H.dt $ inlineListToHtml opts term
@@ -1375,7 +1441,7 @@ inlineToHtml opts inline = do
13751441 return $ if T.null tit
13761442 then link'
13771443 else link' ! A.title (toValue tit)
1378− (Image attr txt (s,tit)) -> do
1444+ (Image attr@(_, _, attrList) txt (s, tit)) -> do
13791445 let alternate = stringify txt
13801446 slideVariant <- gets stSlideVariant
13811447 let isReveal = slideVariant == RevealJsSlides
@@ -1388,7 +1454,8 @@ inlineToHtml opts inline = do
13881454 [A.title $ toValue tit | not (T.null tit)] ++
13891455 attrs
13901456 imageTag = (if html5 then H5.img else H.img
1391− , [A.alt $ toValue alternate | not (null txt)] )
1457+ , [A.alt $ toValue alternate | not (null txt) &&
1458+ isNothing (lookup "alt" attrList)] )
13921459 mediaTag tg fallbackTxt =
13931460 let linkTxt = if null txt
13941461 then fallbackTxt
@@ -1402,7 +1469,7 @@ inlineToHtml opts inline = do
14021469 Just "audio" -> mediaTag H5.audio "Audio"
14031470 Just _ -> (H5.embed, [])
14041471 _ -> imageTag
1405− return $ foldl (!) tag $ attributes ++ specAttrs
1472+ return $ foldl' (!) tag $ attributes ++ specAttrs
14061473 -- note: null title included, as in Markdown.pl
14071474 (Note contents) -> do
14081475 notes <- gets stNotes
@@ -1521,6 +1588,12 @@ allowsMathEnvironments MathML = True
15211588allowsMathEnvironments (WebTeX _) = True
15221589allowsMathEnvironments _ = False
15231590
1591+-- | List of intrinsic event attributes allowed on all elements in HTML4.
1592+intrinsicEventsHTML4 :: [Text]
1593+intrinsicEventsHTML4 =
1594+ [ "onclick", "ondblclick", "onmousedown", "onmouseup", "onmouseover"
1595+ , "onmouseout", "onmouseout", "onkeypress", "onkeydown", "onkeyup"]
1596+
15241597isRawHtml :: PandocMonad m => Format -> StateT WriterState m Bool
15251598isRawHtml f = do
15261599 html5 <- gets stHtml5
src/Text/Pandoc/Writers/Haddock.hs+13−2
@@ -15,6 +15,7 @@ Haddock: <http://www.haskell.org/haddock/doc/html/>
1515-}
1616module Text.Pandoc.Writers.Haddock (writeHaddock) where
1717import Control.Monad.State.Strict
18+import Data.Char (isAlphaNum)
1819import Data.Default
1920import Data.Text (Text)
2021import qualified Data.Text as T
@@ -71,8 +72,18 @@ notesToHaddock opts notes =
7172
7273-- | Escape special characters for Haddock.
7374escapeString :: Text -> Text
74−escapeString = escapeStringUsing haddockEscapes
75− where haddockEscapes = backslashEscapes "\\/'`\"@<"
75+escapeString t
76+ | T.all isAlphaNum t = t
77+ | otherwise = T.concatMap escChar t
78+ where
79+ escChar '\\' = "\\\\"
80+ escChar '/' = "\\/"
81+ escChar '\'' = "\\'"
82+ escChar '`' = "\\`"
83+ escChar '"' = "\\\""
84+ escChar '@' = "\\@"
85+ escChar '<' = "\\<"
86+ escChar c = T.singleton c
7687
7788-- | Convert Pandoc block element to haddock.
7889blockToHaddock :: PandocMonad m
src/Text/Pandoc/Writers/JATS.hs+67−44
@@ -1,6 +1,7 @@
1+{-# LANGUAGE LambdaCase #-}
2+{-# LANGUAGE OverloadedStrings #-}
13{-# LANGUAGE ScopedTypeVariables #-}
2−{-# LANGUAGE OverloadedStrings #-}
3−{-# LANGUAGE ViewPatterns #-}
4+{-# LANGUAGE ViewPatterns #-}
45{- |
56 Module : Text.Pandoc.Writers.JATS
67 Copyright : 2017-2021 John MacFarlane
@@ -20,12 +21,13 @@ module Text.Pandoc.Writers.JATS
2021 , writeJatsPublishing
2122 , writeJatsArticleAuthoring
2223 ) where
24+import Control.Applicative ((<|>))
2325import Control.Monad.Reader
2426import Control.Monad.State
2527import Data.Generics (everywhere, mkT)
2628import Data.List (partition)
2729import qualified Data.Map as M
28−import Data.Maybe (fromMaybe)
30+import Data.Maybe (fromMaybe, listToMaybe)
2931import Data.Time (toGregorian, Day, parseTimeM, defaultTimeLocale, formatTime)
3032import qualified Data.Text as T
3133import Data.Text (Text)
@@ -80,7 +82,7 @@ writeJats tagSet opts d = do
8082 let environment = JATSEnv
8183 { jatsTagSet = tagSet
8284 , jatsInlinesWriter = inlinesToJATS
83− , jatsBlockWriter = blockToJATS
85+ , jatsBlockWriter = wrappedBlocksToJATS
8486 , jatsReferences = refs
8587 }
8688 let initialState = JATSState { jatsNotes = [] }
@@ -162,11 +164,9 @@ wrappedBlocksToJATS needsWrap opts =
162164 wrappedBlockToJATS b = do
163165 inner <- blockToJATS opts b
164166 return $
165− if needsWrap b || isBlockQuote b -- see #7041
167+ if needsWrap b
166168 then inTags True "p" [("specific-use","wrapper")] inner
167169 else inner
168− isBlockQuote (BlockQuote _) = True
169− isBlockQuote _ = False
170170
171171-- | Auxiliary function to convert Plain block to Para.
172172plainToPara :: Block -> Block
@@ -225,33 +225,36 @@ imageMimeType src kvs =
225225 (T.drop 1 . T.dropWhile (/='/') <$> mbMT)
226226 in (maintype, subtype)
227227
228−languageFor :: [Text] -> Text
229−languageFor classes =
228+languageFor :: WriterOptions -> [Text] -> Text
229+languageFor opts classes =
230230 case langs of
231231 (l:_) -> escapeStringForXML l
232232 [] -> ""
233− where isLang l = T.toLower l `elem` map T.toLower languages
233+ where
234+ syntaxMap = writerSyntaxMap opts
235+ isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
234236 langsFrom s = if isLang s
235237 then [s]
236− else languagesByExtension . T.toLower $ s
238+ else (languagesByExtension syntaxMap) . T.toLower $ s
237239 langs = concatMap langsFrom classes
238240
239−codeAttr :: Attr -> (Text, [(Text, Text)])
240−codeAttr (ident,classes,kvs) = (lang, attr)
241+codeAttr :: WriterOptions -> Attr -> (Text, [(Text, Text)])
242+codeAttr opts (ident,classes,kvs) = (lang, attr)
241243 where
242− attr = [("id",ident) | not (T.null ident)] ++
244+ attr = [("id", escapeNCName ident) | not (T.null ident)] ++
243245 [("language",lang) | not (T.null lang)] ++
244246 [(k,v) | (k,v) <- kvs, k `elem` ["code-type",
245247 "code-version", "executable",
246248 "language-version", "orientation",
247249 "platforms", "position", "specific-use"]]
248− lang = languageFor classes
250+ lang = languageFor opts classes
249251
250252-- | Convert a Pandoc block element to JATS.
251253blockToJATS :: PandocMonad m => WriterOptions -> Block -> JATS m (Doc Text)
252254blockToJATS _ Null = return empty
253255blockToJATS opts (Div (id',"section":_,kvs) (Header _lvl _ ils : xs)) = do
254− let idAttr = [("id", writerIdentifierPrefix opts <> id') | not (T.null id')]
256+ let idAttr = [ ("id", writerIdentifierPrefix opts <> escapeNCName id')
257+ | not (T.null id')]
255258 let otherAttrs = ["sec-type", "specific-use"]
256259 let attribs = idAttr ++ [(k,v) | (k,v) <- kvs, k `elem` otherAttrs]
257260 title' <- inlinesToJATS opts ils
@@ -260,7 +263,7 @@ blockToJATS opts (Div (id',"section":_,kvs) (Header _lvl _ ils : xs)) = do
260263 inTagsSimple "title" title' $$ contents
261264-- Bibliography reference:
262265blockToJATS opts (Div (ident,_,_) [Para lst]) | "ref-" `T.isPrefixOf` ident =
263− inTags True "ref" [("id", ident)] .
266+ inTags True "ref" [("id", escapeNCName ident)] .
264267 inTagsSimple "mixed-citation" <$>
265268 inlinesToJATS opts lst
266269blockToJATS opts (Div ("refs",_,_) xs) = do
@@ -271,14 +274,14 @@ blockToJATS opts (Div ("refs",_,_) xs) = do
271274 return $ inTagsIndented "ref-list" contents
272275blockToJATS opts (Div (ident,[cls],kvs) bs) | cls `elem` ["fig", "caption", "table-wrap"] = do
273276 contents <- blocksToJATS opts bs
274− let attr = [("id", ident) | not (T.null ident)] ++
277+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
275278 [("xml:lang",l) | ("lang",l) <- kvs] ++
276279 [(k,v) | (k,v) <- kvs, k `elem` ["specific-use",
277280 "content-type", "orientation", "position"]]
278281 return $ inTags True cls attr contents
279282blockToJATS opts (Div (ident,_,kvs) bs) = do
280283 contents <- blocksToJATS opts bs
281− let attr = [("id", ident) | not (T.null ident)] ++
284+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
282285 [("xml:lang",l) | ("lang",l) <- kvs] ++
283286 [(k,v) | (k,v) <- kvs, k `elem` ["specific-use",
284287 "content-type", "orientation", "position"]]
@@ -296,7 +299,7 @@ blockToJATS opts (Para [Image (ident,_,kvs) txt
296299 let capt = if null txt
297300 then empty
298301 else inTagsSimple "caption" $ inTagsSimple "p" alt
299− let attr = [("id", ident) | not (T.null ident)] ++
302+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
300303 [(k,v) | (k,v) <- kvs, k `elem` ["fig-type", "orientation",
301304 "position", "specific-use"]]
302305 let graphicattr = [("mimetype",maintype),
@@ -307,7 +310,7 @@ blockToJATS opts (Para [Image (ident,_,kvs) txt
307310 capt $$ selfClosingTag "graphic" graphicattr
308311blockToJATS _ (Para [Image (ident,_,kvs) _ (src, tit)]) = do
309312 let (maintype, subtype) = imageMimeType src kvs
310− let attr = [("id", ident) | not (T.null ident)] ++
313+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
311314 [("mimetype", maintype),
312315 ("mime-subtype", subtype),
313316 ("xlink:href", src)] ++
@@ -323,13 +326,16 @@ blockToJATS opts (LineBlock lns) =
323326 blockToJATS opts $ linesToPara lns
324327blockToJATS opts (BlockQuote blocks) = do
325328 tagSet <- asks jatsTagSet
326− let blocksToJats' = if tagSet == TagSetArticleAuthoring
327− then wrappedBlocksToJATS (not . isPara)
328− else blocksToJATS
329− inTagsIndented "disp-quote" <$> blocksToJats' opts blocks
330−blockToJATS _ (CodeBlock a str) = return $
329+ let needsWrap = if tagSet == TagSetArticleAuthoring
330+ then not . isPara
331+ else \case
332+ Header{} -> True
333+ HorizontalRule -> True
334+ _ -> False
335+ inTagsIndented "disp-quote" <$> wrappedBlocksToJATS needsWrap opts blocks
336+blockToJATS opts (CodeBlock a str) = return $
331337 inTags False tag attr (flush (text (T.unpack $ escapeStringForXML str)))
332− where (lang, attr) = codeAttr a
338+ where (lang, attr) = codeAttr opts a
333339 tag = if T.null lang then "preformat" else "code"
334340blockToJATS _ (BulletList []) = return empty
335341blockToJATS opts (BulletList lst) =
@@ -409,9 +415,9 @@ inlineToJATS opts (Quoted SingleQuote lst) = do
409415inlineToJATS opts (Quoted DoubleQuote lst) = do
410416 contents <- inlinesToJATS opts lst
411417 return $ char '“' <> contents <> char '”'
412−inlineToJATS _ (Code a str) =
418+inlineToJATS opts (Code a str) =
413419 return $ inTags False tag attr $ literal (escapeStringForXML str)
414− where (lang, attr) = codeAttr a
420+ where (lang, attr) = codeAttr opts a
415421 tag = if T.null lang then "monospace" else "code"
416422inlineToJATS _ il@(RawInline f x)
417423 | f == "jats" = return $ literal x
@@ -434,7 +440,8 @@ inlineToJATS opts (Note contents) = do
434440 let notenum = case notes of
435441 (n, _):_ -> n + 1
436442 [] -> 1
437− thenote <- inTags True "fn" [("id","fn" <> tshow notenum)]
443+ thenote <- inTags True "fn" [("id", "fn" <> tshow notenum)]
444+ . (inTagsSimple "label" (literal $ tshow notenum) <>)
438445 <$> wrappedBlocksToJATS (not . isPara) opts
439446 (walk demoteHeaderAndRefs contents)
440447 modify $ \st -> st{ jatsNotes = (notenum, thenote) : notes }
@@ -442,18 +449,34 @@ inlineToJATS opts (Note contents) = do
442449 ("rid", "fn" <> tshow notenum)]
443450 $ text (show notenum)
444451inlineToJATS opts (Cite _ lst) =
445− -- TODO revisit this after examining the jats.csl pipeline
446452 inlinesToJATS opts lst
447−inlineToJATS opts (Span ("",_,[]) ils) = inlinesToJATS opts ils
448−inlineToJATS opts (Span (ident,_,kvs) ils) = do
453+inlineToJATS opts (Span (ident,classes,kvs) ils) = do
449454 contents <- inlinesToJATS opts ils
450− let attr = [("id",ident) | not (T.null ident)] ++
451− [("xml:lang",l) | ("lang",l) <- kvs] ++
452− [(k,v) | (k,v) <- kvs
453− , k `elem` ["content-type", "rationale",
454− "rid", "specific-use"]]
455− return $ selfClosingTag "milestone-start" attr <> contents <>
456− selfClosingTag "milestone-end" []
455+ let commonAttr = [("id", escapeNCName ident) | not (T.null ident)] ++
456+ [("xml:lang",l) | ("lang",l) <- kvs] ++
457+ [(k,v) | (k,v) <- kvs, k `elem` ["alt", "specific-use"]]
458+ -- A named-content element is a good fit for spans, but requires a
459+ -- content-type attribute to be present. We use either the explicit
460+ -- attribute or the first class as content type. If neither is
461+ -- available, then we fall back to using a @styled-content@ element.
462+ let (tag, specificAttr) =
463+ case lookup "content-type" kvs <|> listToMaybe classes of
464+ Just ct -> ( "named-content"
465+ , ("content-type", ct) :
466+ [(k, v) | (k, v) <- kvs
467+ , k `elem` ["rid", "vocab", "vocab-identifier",
468+ "vocab-term", "vocab-term-identifier"]])
469+ -- Fall back to styled-content
470+ Nothing -> ("styled-content"
471+ , [(k, v) | (k,v) <- kvs
472+ , k `elem` ["style", "style-type", "style-detail",
473+ "toggle"]])
474+ let attr = commonAttr ++ specificAttr
475+ -- unwrap if wrapping element would have no attributes
476+ return $
477+ if null attr
478+ then contents
479+ else inTags False tag attr contents
457480inlineToJATS _ (Math t str) = do
458481 let addPref (Xml.Attr q v)
459482 | Xml.qName q == "xmlns" = Xml.Attr q{ Xml.qName = "xmlns:mml" } v
@@ -488,9 +511,9 @@ inlineToJATS _ (Link _attr [Str t] (T.stripPrefix "mailto:" -> Just email, _))
488511 return $ inTagsSimple "email" $ literal (escapeStringForXML email)
489512inlineToJATS opts (Link (ident,_,kvs) txt (T.uncons -> Just ('#', src), _)) = do
490513 let attr = mconcat
491− [ [("id", ident) | not (T.null ident)]
514+ [ [("id", escapeNCName ident) | not (T.null ident)]
492515 , [("alt", stringify txt) | not (null txt)]
493− , [("rid", src)]
516+ , [("rid", escapeNCName src)]
494517 , [(k,v) | (k,v) <- kvs, k `elem` ["ref-type", "specific-use"]]
495518 , [("ref-type", "bibr") | "ref-" `T.isPrefixOf` src]
496519 ]
@@ -500,7 +523,7 @@ inlineToJATS opts (Link (ident,_,kvs) txt (T.uncons -> Just ('#', src), _)) = do
500523 contents <- inlinesToJATS opts txt
501524 return $ inTags False "xref" attr contents
502525inlineToJATS opts (Link (ident,_,kvs) txt (src, tit)) = do
503− let attr = [("id", ident) | not (T.null ident)] ++
526+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
504527 [("ext-link-type", "uri"),
505528 ("xlink:href", src)] ++
506529 [("xlink:title", tit) | not (T.null tit)] ++
@@ -518,7 +541,7 @@ inlineToJATS _ (Image (ident,_,kvs) _ (src, tit)) = do
518541 let subtype = fromMaybe "" $
519542 lookup "mime-subtype" kvs `mplus`
520543 (T.drop 1 . T.dropWhile (/='/') <$> mbMT)
521− let attr = [("id", ident) | not (T.null ident)] ++
544+ let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
522545 [("mimetype", maintype),
523546 ("mime-subtype", subtype),
524547 ("xlink:href", src)] ++
src/Text/Pandoc/Writers/JATS/References.hs+3−2
@@ -29,7 +29,7 @@ import Text.Pandoc.Builder (Inlines)
2929import Text.Pandoc.Options (WriterOptions)
3030import Text.Pandoc.Shared (tshow)
3131import Text.Pandoc.Writers.JATS.Types
32−import Text.Pandoc.XML (escapeStringForXML, inTags)
32+import Text.Pandoc.XML (escapeNCName, escapeStringForXML, inTags)
3333import qualified Data.Text as T
3434
3535referencesToJATS :: PandocMonad m
@@ -46,7 +46,8 @@ referenceToJATS :: PandocMonad m
4646referenceToJATS _opts ref = do
4747 let refType = referenceType ref
4848 let pubType = [("publication-type", refType) | not (T.null refType)]
49− let wrap = inTags True "ref" [("id", "ref-" <> unItemId (referenceId ref))]
49+ let ident = escapeNCName $ "ref-" <> unItemId (referenceId ref)
50+ let wrap = inTags True "ref" [("id", ident)]
5051 . inTags True "element-citation" pubType
5152 return . wrap . vcat $
5253 [ authors
src/Text/Pandoc/Writers/JATS/Table.hs+25−5
@@ -24,7 +24,7 @@ import Text.Pandoc.Definition
2424import Text.Pandoc.Options (WriterOptions)
2525import Text.Pandoc.Shared (tshow)
2626import Text.Pandoc.Writers.JATS.Types
27−import Text.Pandoc.XML (inTags, inTagsIndented, selfClosingTag)
27+import Text.Pandoc.XML (escapeNCName, inTags, inTagsIndented, selfClosingTag)
2828import qualified Data.Text as T
2929import qualified Text.Pandoc.Writers.AnnotatedTable as Ann
3030
@@ -34,13 +34,19 @@ tableToJATS :: PandocMonad m
3434 -> JATS m (Doc Text)
3535tableToJATS opts (Ann.Table attr caption colspecs thead tbodies tfoot) = do
3636 let (Caption _maybeShortCaption captionBlocks) = caption
37+ -- Only paragraphs are allowed in captions, all other blocks must be
38+ -- wrapped in @<p>@ elements.
39+ let needsWrapping = \case
40+ Plain{} -> False
41+ Para{} -> False
42+ _ -> True
3743 tbl <- captionlessTable opts attr colspecs thead tbodies tfoot
3844 captionDoc <- if null captionBlocks
3945 then return empty
4046 else do
4147 blockToJATS <- asks jatsBlockWriter
42− inTagsIndented "caption" . vcat <$>
43− mapM (blockToJATS opts) captionBlocks
48+ inTagsIndented "caption" <$>
49+ blockToJATS needsWrapping opts captionBlocks
4450 return $ inTags True "table-wrap" [] $ captionDoc $$ tbl
4551
4652captionlessTable :: PandocMonad m
@@ -216,7 +222,7 @@ cellToJats opts celltype (Ann.Cell (colspec :| _) _colNum cell) =
216222
217223toAttribs :: Attr -> [Text] -> [(Text, Text)]
218224toAttribs (ident, _classes, kvs) knownAttribs =
219− (if T.null ident then id else (("id", ident) :)) $
225+ (if T.null ident then id else (("id", escapeNCName ident) :)) $
220226 filter ((`elem` knownAttribs) . fst) kvs
221227
222228tableCellToJats :: PandocMonad m
@@ -230,7 +236,7 @@ tableCellToJats opts ctype colAlign (Cell attr align rowspan colspan item) = do
230236 inlinesToJats <- asks jatsInlinesWriter
231237 let cellContents = \case
232238 [Plain inlines] -> inlinesToJats opts inlines
233− blocks -> vcat <$> mapM (blockToJats opts) blocks
239+ blocks -> blockToJats needsWrapInCell opts blocks
234240 let tag' = case ctype of
235241 BodyCell -> "td"
236242 HeaderCell -> "th"
@@ -246,3 +252,17 @@ tableCellToJats opts ctype colAlign (Cell attr align rowspan colspan item) = do
246252 . maybeCons (colspanAttrib colspan)
247253 $ toAttribs attr validAttribs
248254 inTags False tag' attribs <$> cellContents item
255+
256+-- | Whether the JATS produced from this block should be wrapped in a
257+-- @<p>@ element when put directly below a @<td>@ element.
258+needsWrapInCell :: Block -> Bool
259+needsWrapInCell = \case
260+ Plain{} -> False -- should be unwrapped anyway
261+ Para{} -> False
262+ BulletList{} -> False
263+ OrderedList{} -> False
264+ DefinitionList{} -> False
265+ HorizontalRule -> False
266+ CodeBlock{} -> False
267+ RawBlock{} -> False -- responsibility of the user
268+ _ -> True
src/Text/Pandoc/Writers/JATS/Types.hs+12−3
@@ -37,11 +37,20 @@ newtype JATSState = JATSState
3737 { jatsNotes :: [(Int, Doc Text)]
3838 }
3939
40+-- | Environment containing all information relevant for rendering.
4041data JATSEnv m = JATSEnv
41− { jatsTagSet :: JATSTagSet
42+ { jatsTagSet :: JATSTagSet -- ^ The tag set that's being ouput
43+
44+ , jatsBlockWriter :: (Block -> Bool)
45+ -> WriterOptions -> [Block] -> JATS m (Doc Text)
46+ -- ^ Converts a block list to JATS, wrapping top-level blocks into a
47+ -- @<p>@ element if the property evaluates to @True@.
48+ -- See #7227.
49+
4250 , jatsInlinesWriter :: WriterOptions -> [Inline] -> JATS m (Doc Text)
43− , jatsBlockWriter :: WriterOptions -> Block -> JATS m (Doc Text)
44− , jatsReferences :: [Reference Inlines]
51+ -- ^ Converts an inline list to JATS.
52+
53+ , jatsReferences :: [Reference Inlines] -- ^ List of references
4554 }
4655
4756-- | JATS writer type
src/Text/Pandoc/Writers/Jira.hs+41−18
@@ -39,11 +39,17 @@ writeJira :: PandocMonad m => WriterOptions -> Pandoc -> m Text
3939writeJira opts = runDefaultConverter (writerWrapText opts) (pandocToJira opts)
4040
4141-- | State to keep track of footnotes.
42−newtype ConverterState = ConverterState { stNotes :: [Text] }
42+data ConverterState = ConverterState
43+ { stNotes :: [Text] -- ^ Footnotes to be appended to the end of the text
44+ , stInPanel :: Bool -- ^ whether we are in a @{panel}@ block
45+ }
4346
4447-- | Initial converter state.
4548startState :: ConverterState
46−startState = ConverterState { stNotes = [] }
49+startState = ConverterState
50+ { stNotes = []
51+ , stInPanel = False
52+ }
4753
4854-- | Converter monad
4955type JiraConverter m = ReaderT WrapOption (StateT ConverterState m)
@@ -126,13 +132,19 @@ toJiraCode :: PandocMonad m
126132 -> Text
127133 -> JiraConverter m [Jira.Block]
128134toJiraCode (ident, classes, _attribs) code = do
129− let lang = case find (\c -> T.toLower c `elem` knownLanguages) classes of
130− Nothing -> Jira.Language "java"
131− Just l -> Jira.Language l
132− let addAnchor b = if T.null ident
133− then b
134− else [Jira.Para (singleton (Jira.Anchor ident))] <> b
135− return . addAnchor . singleton $ Jira.Code lang mempty code
135+ return . addAnchor ident . singleton $
136+ case find (\c -> T.toLower c `elem` knownLanguages) classes of
137+ Nothing -> Jira.NoFormat mempty code
138+ Just l -> Jira.Code (Jira.Language l) mempty code
139+
140+-- | Prepends an anchor with the given identifier.
141+addAnchor :: Text -> [Jira.Block] -> [Jira.Block]
142+addAnchor ident =
143+ if T.null ident
144+ then id
145+ else \case
146+ Jira.Para xs : bs -> (Jira.Para (Jira.Anchor ident : xs) : bs)
147+ bs -> (Jira.Para (singleton (Jira.Anchor ident)) : bs)
136148
137149-- | Creates a Jira definition list
138150toJiraDefinitionList :: PandocMonad m
@@ -149,11 +161,16 @@ toJiraDefinitionList defItems = do
149161toJiraPanel :: PandocMonad m
150162 => Attr -> [Block]
151163 -> JiraConverter m [Jira.Block]
152−toJiraPanel attr blocks = do
153− jiraBlocks <- toJiraBlocks blocks
154− return $ if attr == nullAttr
155− then jiraBlocks
156− else singleton (Jira.Panel [] jiraBlocks)
164+toJiraPanel (ident, classes, attribs) blocks = do
165+ inPanel <- gets stInPanel
166+ if inPanel || ("panel" `notElem` classes && null attribs)
167+ then addAnchor ident <$> toJiraBlocks blocks
168+ else do
169+ modify $ \st -> st{ stInPanel = True }
170+ jiraBlocks <- toJiraBlocks blocks
171+ modify $ \st -> st{ stInPanel = inPanel }
172+ let params = map (uncurry Jira.Parameter) attribs
173+ return $ singleton (Jira.Panel params $ addAnchor ident jiraBlocks)
157174
158175-- | Creates a Jira header
159176toJiraHeader :: PandocMonad m
@@ -263,6 +280,8 @@ toJiraLink (_, classes, _) (url, _) alias = do
263280 | Just email <- T.stripPrefix "mailto:" url' = (Jira.Email, email)
264281 | "user-account" `elem` classes = (Jira.User, dropTilde url)
265282 | "attachment" `elem` classes = (Jira.Attachment, url)
283+ | "smart-card" `elem` classes = (Jira.SmartCard, url)
284+ | "smart-link" `elem` classes = (Jira.SmartLink, url)
266285 | otherwise = (Jira.External, url)
267286 dropTilde txt = case T.uncons txt of
268287 Just ('~', username) -> username
@@ -292,9 +311,13 @@ quotedToJira qtype xs = do
292311spanToJira :: PandocMonad m
293312 => Attr -> [Inline]
294313 -> JiraConverter m [Jira.Inline]
295−spanToJira (ident, _classes, _attribs) inls = case ident of
296− "" -> toJiraInlines inls
297− _ -> (Jira.Anchor ident :) <$> toJiraInlines inls
314+spanToJira (ident, _classes, attribs) inls =
315+ let wrap = case lookup "color" attribs of
316+ Nothing -> id
317+ Just color -> singleton . Jira.ColorInline (Jira.ColorName color)
318+ in wrap <$> case ident of
319+ "" -> toJiraInlines inls
320+ _ -> (Jira.Anchor ident :) <$> toJiraInlines inls
298321
299322registerNotes :: PandocMonad m => [Block] -> JiraConverter m [Jira.Inline]
300323registerNotes contents = do
@@ -310,7 +333,7 @@ registerNotes contents = do
310333knownLanguages :: [Text]
311334knownLanguages =
312335 [ "actionscript", "ada", "applescript", "bash", "c", "c#", "c++"
313− , "css", "erlang", "go", "groovy", "haskell", "html", "javascript"
336+ , "css", "erlang", "go", "groovy", "haskell", "html", "java", "javascript"
314337 , "json", "lua", "nyan", "objc", "perl", "php", "python", "r", "ruby"
315338 , "scala", "sql", "swift", "visualbasic", "xml", "yaml"
316339 ]
src/Text/Pandoc/Writers/LaTeX.hs+37−12
@@ -1,4 +1,5 @@
11{-# LANGUAGE OverloadedStrings #-}
2+{-# LANGUAGE LambdaCase #-}
23{-# LANGUAGE TupleSections #-}
34{-# LANGUAGE PatternGuards #-}
45{-# LANGUAGE ScopedTypeVariables #-}
@@ -28,7 +29,7 @@ import qualified Data.Text as T
2829import Network.URI (unEscapeString)
2930import Text.DocTemplates (FromContext(lookupContext), renderTemplate,
3031 Val(..), Context(..))
31−import Text.Pandoc.BCP47 (Lang (..), getLang, renderLang)
32+import Text.Collate.Lang (Lang (..), renderLang)
3233import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
3334import Text.Pandoc.Definition
3435import Text.Pandoc.Highlighting (formatLaTeXBlock, formatLaTeXInline, highlight,
@@ -145,6 +146,11 @@ pandocToLaTeX options (Pandoc meta blocks) = do
145146
146147 let dirs = query (extract "dir") blocks
147148
149+ let nociteIds = query (\case
150+ Cite cs _ -> map citationId cs
151+ _ -> [])
152+ $ lookupMetaInlines "nocite" meta
153+
148154 let context = defField "toc" (writerTableOfContents options) $
149155 defField "toc-depth" (tshow
150156 (writerTOCDepth options -
@@ -167,6 +173,7 @@ pandocToLaTeX options (Pandoc meta blocks) = do
167173 defField "has-chapters" (stHasChapters st) $
168174 defField "has-frontmatter" (documentClass `elem` frontmatterClasses) $
169175 defField "listings" (writerListings options || stLHS st) $
176+ defField "zero-width-non-joiner" (stZwnj st) $
170177 defField "beamer" beamer $
171178 (if stHighlighting st
172179 then case writerHighlightStyle options of
@@ -177,9 +184,11 @@ pandocToLaTeX options (Pandoc meta blocks) = do
177184 else id) $
178185 (case writerCiteMethod options of
179186 Natbib -> defField "biblio-title" biblioTitle .
180− defField "natbib" True
187+ defField "natbib" True .
188+ defField "nocite-ids" nociteIds
181189 Biblatex -> defField "biblio-title" biblioTitle .
182− defField "biblatex" True
190+ defField "biblatex" True .
191+ defField "nocite-ids" nociteIds
183192 _ -> id) $
184193 defField "colorlinks" (any hasStringValue
185194 ["citecolor", "urlcolor", "linkcolor", "toccolor",
@@ -282,7 +291,12 @@ blockToLaTeX :: PandocMonad m
282291 => Block -- ^ Block to convert
283292 -> LW m (Doc Text)
284293blockToLaTeX Null = return empty
285−blockToLaTeX (Div attr@(identifier,"block":_,_) (Header _ _ ils : bs)) = do
294+blockToLaTeX (Div attr@(identifier,"block":dclasses,_)
295+ (Header _ _ ils : bs)) = do
296+ let blockname
297+ | "example" `elem` dclasses = "exampleblock"
298+ | "alert" `elem` dclasses = "alertblock"
299+ | otherwise = "block"
286300 ref <- toLabel identifier
287301 let anchor = if T.null identifier
288302 then empty
@@ -290,8 +304,8 @@ blockToLaTeX (Div attr@(identifier,"block":_,_) (Header _ _ ils : bs)) = do
290304 braces (literal ref) <> braces empty
291305 title' <- inlineListToLaTeX ils
292306 contents <- blockListToLaTeX bs
293− wrapDiv attr $ ("\\begin{block}" <> braces title' <> anchor) $$
294− contents $$ "\\end{block}"
307+ wrapDiv attr $ ("\\begin" <> braces blockname <> braces title' <> anchor) $$
308+ contents $$ "\\end" <> braces blockname
295309blockToLaTeX (Div (identifier,"slide":dclasses,dkvs)
296310 (Header _ (_,hclasses,hkvs) ils : bs)) = do
297311 -- note: [fragile] is required or verbatim breaks
@@ -817,7 +831,19 @@ inlineToLaTeX (Code (_,classes,kvs) str) = do
817831 let chr = case "!\"'()*,-./:;?@" \\ T.unpack str of
818832 (c:_) -> c
819833 [] -> '!'
820− let str' = escapeStringUsing (backslashEscapes "\\{}%~_&#^") str
834+ let isEscapable '\\' = True
835+ isEscapable '{' = True
836+ isEscapable '}' = True
837+ isEscapable '%' = True
838+ isEscapable '~' = True
839+ isEscapable '_' = True
840+ isEscapable '&' = True
841+ isEscapable '#' = True
842+ isEscapable '^' = True
843+ isEscapable _ = False
844+ let escChar c | isEscapable c = T.pack ['\\',c]
845+ | otherwise = T.singleton c
846+ let str' = T.concatMap escChar str
821847 -- we always put lstinline in a dummy 'passthrough' command
822848 -- (defined in the default template) so that we don't have
823849 -- to change the way we escape characters depending on whether
@@ -868,8 +894,9 @@ inlineToLaTeX (Quoted qt lst) = do
868894 then char '`' <> inner <> char '\''
869895 else char '\x2018' <> inner <> char '\x2019'
870896 where
871− isQuoted (Quoted _ _) = True
872− isQuoted _ = False
897+ isQuoted (Span _ (x:_)) = isQuoted x
898+ isQuoted (Quoted _ _) = True
899+ isQuoted _ = False
873900inlineToLaTeX (Str str) = do
874901 setEmptyLine False
875902 liftM literal $ stringToLaTeX TextString str
@@ -892,7 +919,7 @@ inlineToLaTeX il@(RawInline f str) = do
892919inlineToLaTeX LineBreak = do
893920 emptyLine <- gets stEmptyLine
894921 setEmptyLine True
895− return $ (if emptyLine then "~" else "") <> "\\\\" <> cr
922+ return $ (if emptyLine then "\\strut " else "") <> "\\\\" <> cr
896923inlineToLaTeX SoftBreak = do
897924 wrapText <- gets (writerWrapText . stOptions)
898925 case wrapText of
@@ -1023,5 +1050,3 @@ extractInline _ _ = []
10231050-- Look up a key in an attribute and give a list of its values
10241051lookKey :: Text -> Attr -> [Text]
10251052lookKey key (_,_,kvs) = maybe [] T.words $ lookup key kvs
1026−
1027−
src/Text/Pandoc/Writers/LaTeX/Caption.hs−1
@@ -1,4 +1,3 @@
1−{-# LANGUAGE OverloadedStrings #-}
21{- |
32 Module : Text.Pandoc.Writers.LaTeX.Caption
43 Copyright : Copyright (C) 2006-2021 John MacFarlane
src/Text/Pandoc/Writers/LaTeX/Lang.hs+61−60
@@ -15,7 +15,7 @@ module Text.Pandoc.Writers.LaTeX.Lang
1515 toBabel
1616 ) where
1717import Data.Text (Text)
18−import Text.Pandoc.BCP47 (Lang (..))
18+import Text.Collate.Lang (Lang(..))
1919
2020
2121-- In environments \Arabic instead of \arabic is used
@@ -25,88 +25,89 @@ toPolyglossiaEnv l =
2525 ("arabic", o) -> ("Arabic", o)
2626 x -> x
2727
28−-- Takes a list of the constituents of a BCP 47 language code and
28+-- Takes a list of the constituents of a BCP47 language code and
2929-- converts it to a Polyglossia (language, options) tuple
3030-- http://mirrors.ctan.org/macros/latex/contrib/polyglossia/polyglossia.pdf
3131toPolyglossia :: Lang -> (Text, Text)
32−toPolyglossia (Lang "ar" _ "DZ" _) = ("arabic", "locale=algeria")
33−toPolyglossia (Lang "ar" _ "IQ" _) = ("arabic", "locale=mashriq")
34−toPolyglossia (Lang "ar" _ "JO" _) = ("arabic", "locale=mashriq")
35−toPolyglossia (Lang "ar" _ "LB" _) = ("arabic", "locale=mashriq")
36−toPolyglossia (Lang "ar" _ "LY" _) = ("arabic", "locale=libya")
37−toPolyglossia (Lang "ar" _ "MA" _) = ("arabic", "locale=morocco")
38−toPolyglossia (Lang "ar" _ "MR" _) = ("arabic", "locale=mauritania")
39−toPolyglossia (Lang "ar" _ "PS" _) = ("arabic", "locale=mashriq")
40−toPolyglossia (Lang "ar" _ "SY" _) = ("arabic", "locale=mashriq")
41−toPolyglossia (Lang "ar" _ "TN" _) = ("arabic", "locale=tunisia")
42−toPolyglossia (Lang "de" _ _ vars)
43− | "1901" `elem` vars = ("german", "spelling=old")
44−toPolyglossia (Lang "de" _ "AT" vars)
45− | "1901" `elem` vars = ("german", "variant=austrian, spelling=old")
46−toPolyglossia (Lang "de" _ "AT" _) = ("german", "variant=austrian")
47−toPolyglossia (Lang "de" _ "CH" vars)
48− | "1901" `elem` vars = ("german", "variant=swiss, spelling=old")
49−toPolyglossia (Lang "de" _ "CH" _) = ("german", "variant=swiss")
50−toPolyglossia (Lang "de" _ _ _) = ("german", "")
51−toPolyglossia (Lang "dsb" _ _ _) = ("lsorbian", "")
52−toPolyglossia (Lang "el" _ "polyton" _) = ("greek", "variant=poly")
53−toPolyglossia (Lang "en" _ "AU" _) = ("english", "variant=australian")
54−toPolyglossia (Lang "en" _ "CA" _) = ("english", "variant=canadian")
55−toPolyglossia (Lang "en" _ "GB" _) = ("english", "variant=british")
56−toPolyglossia (Lang "en" _ "NZ" _) = ("english", "variant=newzealand")
57−toPolyglossia (Lang "en" _ "UK" _) = ("english", "variant=british")
58−toPolyglossia (Lang "en" _ "US" _) = ("english", "variant=american")
59−toPolyglossia (Lang "grc" _ _ _) = ("greek", "variant=ancient")
60−toPolyglossia (Lang "hsb" _ _ _) = ("usorbian", "")
61−toPolyglossia (Lang "la" _ _ vars)
62− | "x-classic" `elem` vars = ("latin", "variant=classic")
63−toPolyglossia (Lang "pt" _ "BR" _) = ("portuguese", "variant=brazilian")
64−toPolyglossia (Lang "sl" _ _ _) = ("slovenian", "")
65−toPolyglossia x = (commonFromBcp47 x, "")
32+toPolyglossia (Lang "ar" _ (Just "DZ") _ _ _) = ("arabic", "locale=algeria")
33+toPolyglossia (Lang "ar" _ (Just "IQ") _ _ _) = ("arabic", "locale=mashriq")
34+toPolyglossia (Lang "ar" _ (Just "JO") _ _ _) = ("arabic", "locale=mashriq")
35+toPolyglossia (Lang "ar" _ (Just "LB") _ _ _) = ("arabic", "locale=mashriq")
36+toPolyglossia (Lang "ar" _ (Just "LY") _ _ _) = ("arabic", "locale=libya")
37+toPolyglossia (Lang "ar" _ (Just "MA") _ _ _) = ("arabic", "locale=morocco")
38+toPolyglossia (Lang "ar" _ (Just "MR") _ _ _) = ("arabic", "locale=mauritania")
39+toPolyglossia (Lang "ar" _ (Just "PS") _ _ _) = ("arabic", "locale=mashriq")
40+toPolyglossia (Lang "ar" _ (Just "SY") _ _ _) = ("arabic", "locale=mashriq")
41+toPolyglossia (Lang "ar" _ (Just "TN") _ _ _) = ("arabic", "locale=tunisia")
42+toPolyglossia (Lang "de" _ _ vars _ _)
43+ | "1901" `elem` vars = ("german", "spelling=old")
44+toPolyglossia (Lang "de" _ (Just "AT") vars _ _)
45+ | "1901" `elem` vars = ("german", "variant=austrian, spelling=old")
46+toPolyglossia (Lang "de" _ (Just "AT") _ _ _) = ("german", "variant=austrian")
47+toPolyglossia (Lang "de" _ (Just "CH") vars _ _)
48+ | "1901" `elem` vars = ("german", "variant=swiss, spelling=old")
49+toPolyglossia (Lang "de" _ (Just "CH") _ _ _) = ("german", "variant=swiss")
50+toPolyglossia (Lang "de" _ _ _ _ _) = ("german", "")
51+toPolyglossia (Lang "dsb" _ _ _ _ _) = ("lsorbian", "")
52+toPolyglossia (Lang "el" _ _ vars _ _)
53+ | "polyton" `elem` vars = ("greek", "variant=poly")
54+toPolyglossia (Lang "en" _ (Just "AU") _ _ _) = ("english", "variant=australian")
55+toPolyglossia (Lang "en" _ (Just "CA") _ _ _) = ("english", "variant=canadian")
56+toPolyglossia (Lang "en" _ (Just "GB") _ _ _) = ("english", "variant=british")
57+toPolyglossia (Lang "en" _ (Just "NZ") _ _ _) = ("english", "variant=newzealand")
58+toPolyglossia (Lang "en" _ (Just "UK") _ _ _) = ("english", "variant=british")
59+toPolyglossia (Lang "en" _ (Just "US") _ _ _) = ("english", "variant=american")
60+toPolyglossia (Lang "grc" _ _ _ _ _) = ("greek", "variant=ancient")
61+toPolyglossia (Lang "hsb" _ _ _ _ _) = ("usorbian", "")
62+toPolyglossia (Lang "la" _ _ vars _ _)
63+ | "x-classic" `elem` vars = ("latin", "variant=classic")
64+toPolyglossia (Lang "pt" _ (Just "BR") _ _ _) = ("portuguese", "variant=brazilian")
65+toPolyglossia (Lang "sl" _ _ _ _ _) = ("slovenian", "")
66+toPolyglossia x = (commonFromBcp47 x, "")
6667
67−-- Takes a list of the constituents of a BCP 47 language code and
68+-- Takes a list of the constituents of a BCP47 language code and
6869-- converts it to a Babel language string.
6970-- http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf
7071-- List of supported languages (slightly outdated):
7172-- http://tug.ctan.org/language/hyph-utf8/doc/generic/hyph-utf8/hyphenation.pdf
7273toBabel :: Lang -> Text
73−toBabel (Lang "de" _ "AT" vars)
74+toBabel (Lang "de" _ (Just "AT") vars _ _)
7475 | "1901" `elem` vars = "austrian"
7576 | otherwise = "naustrian"
76−toBabel (Lang "de" _ "CH" vars)
77+toBabel (Lang "de" _ (Just "CH") vars _ _)
7778 | "1901" `elem` vars = "swissgerman"
7879 | otherwise = "nswissgerman"
79−toBabel (Lang "de" _ _ vars)
80+toBabel (Lang "de" _ _ vars _ _)
8081 | "1901" `elem` vars = "german"
8182 | otherwise = "ngerman"
82−toBabel (Lang "dsb" _ _ _) = "lowersorbian"
83−toBabel (Lang "el" _ _ vars)
83+toBabel (Lang "dsb" _ _ _ _ _) = "lowersorbian"
84+toBabel (Lang "el" _ _ vars _ _)
8485 | "polyton" `elem` vars = "polutonikogreek"
85−toBabel (Lang "en" _ "AU" _) = "australian"
86−toBabel (Lang "en" _ "CA" _) = "canadian"
87−toBabel (Lang "en" _ "GB" _) = "british"
88−toBabel (Lang "en" _ "NZ" _) = "newzealand"
89−toBabel (Lang "en" _ "UK" _) = "british"
90−toBabel (Lang "en" _ "US" _) = "american"
91−toBabel (Lang "fr" _ "CA" _) = "canadien"
92−toBabel (Lang "fra" _ _ vars)
86+toBabel (Lang "en" _ (Just "AU") _ _ _) = "australian"
87+toBabel (Lang "en" _ (Just "CA") _ _ _) = "canadian"
88+toBabel (Lang "en" _ (Just "GB") _ _ _) = "british"
89+toBabel (Lang "en" _ (Just "NZ") _ _ _) = "newzealand"
90+toBabel (Lang "en" _ (Just "UK") _ _ _) = "british"
91+toBabel (Lang "en" _ (Just "US") _ _ _) = "american"
92+toBabel (Lang "fr" _ (Just "CA") _ _ _) = "canadien"
93+toBabel (Lang "fra" _ _ vars _ _)
9394 | "aca" `elem` vars = "acadian"
94−toBabel (Lang "grc" _ _ _) = "polutonikogreek"
95−toBabel (Lang "hsb" _ _ _) = "uppersorbian"
96−toBabel (Lang "la" _ _ vars)
95+toBabel (Lang "grc" _ _ _ _ _) = "polutonikogreek"
96+toBabel (Lang "hsb" _ _ _ _ _) = "uppersorbian"
97+toBabel (Lang "la" _ _ vars _ _)
9798 | "x-classic" `elem` vars = "classiclatin"
98−toBabel (Lang "pt" _ "BR" _) = "brazilian"
99−toBabel (Lang "sl" _ _ _) = "slovene"
99+toBabel (Lang "pt" _ (Just "BR") _ _ _) = "brazilian"
100+toBabel (Lang "sl" _ _ _ _ _) = "slovene"
100101toBabel x = commonFromBcp47 x
101102
102−-- Takes a list of the constituents of a BCP 47 language code
103+-- Takes a list of the constituents of a BCP47 language code
103104-- and converts it to a string shared by Babel and Polyglossia.
104105-- https://tools.ietf.org/html/bcp47#section-2.1
105106commonFromBcp47 :: Lang -> Text
106−commonFromBcp47 (Lang "sr" "Cyrl" _ _) = "serbianc"
107−commonFromBcp47 (Lang "zh" "Latn" _ vars)
108− | "pinyin" `elem` vars = "pinyin"
109−commonFromBcp47 (Lang l _ _ _) = fromIso l
107+commonFromBcp47 (Lang "sr" (Just "Cyrl") _ _ _ _) = "serbianc"
108+commonFromBcp47 (Lang "zh" (Just "Latn") _ vars _ _)
109+ | "pinyin" `elem` vars = "pinyin"
110+commonFromBcp47 (Lang l _ _ _ _ _) = fromIso l
110111 where
111112 fromIso "af" = "afrikaans"
112113 fromIso "am" = "amharic"
src/Text/Pandoc/Writers/LaTeX/Table.hs+21−8
@@ -16,6 +16,7 @@ module Text.Pandoc.Writers.LaTeX.Table
1616 ) where
1717import Control.Monad.State.Strict
1818import Data.List (intersperse)
19+import qualified Data.List.NonEmpty as NonEmpty
1920import Data.List.NonEmpty (NonEmpty ((:|)))
2021import Data.Text (Text)
2122import qualified Data.Text as T
@@ -25,7 +26,8 @@ import Text.DocLayout
2526 ( Doc, braces, cr, empty, hcat, hsep, isEmpty, literal, nest
2627 , text, vcat, ($$) )
2728import Text.Pandoc.Shared (blocksToInlines, splitBy, tshow)
28−import Text.Pandoc.Walk (walk)
29+import Text.Pandoc.Walk (walk, query)
30+import Data.Monoid (Any(..))
2931import Text.Pandoc.Writers.LaTeX.Caption (getCaption)
3032import Text.Pandoc.Writers.LaTeX.Notes (notesToLaTeX)
3133import Text.Pandoc.Writers.LaTeX.Types
@@ -219,9 +221,7 @@ footRows (Ann.TableFoot _attr rows) = map headerRowCells rows
219221-- we need to go to some lengths to get line breaks working:
220222-- as LineBreak bs = \vtop{\hbox{\strut as}\hbox{\strut bs}}.
221223fixLineBreaks :: Block -> Block
222−fixLineBreaks (Para ils) = Para $ fixLineBreaks' ils
223−fixLineBreaks (Plain ils) = Plain $ fixLineBreaks' ils
224−fixLineBreaks x = x
224+fixLineBreaks = walk fixLineBreaks'
225225
226226fixLineBreaks' :: [Inline] -> [Inline]
227227fixLineBreaks' ils = case splitBy (== LineBreak) ils of
@@ -245,8 +245,13 @@ cellToLaTeX :: PandocMonad m
245245 -> Ann.Cell
246246 -> LW m (Doc Text)
247247cellToLaTeX blockListToLaTeX celltype annotatedCell = do
248− let (Ann.Cell _specs _colnum cell) = annotatedCell
249− let (Cell _attr align rowspan colspan blocks) = cell
248+ let (Ann.Cell specs _colnum cell) = annotatedCell
249+ let hasWidths = snd (NonEmpty.head specs) /= ColWidthDefault
250+ let specAlign = fst (NonEmpty.head specs)
251+ let (Cell _attr align' rowspan colspan blocks) = cell
252+ let align = case align' of
253+ AlignDefault -> specAlign
254+ _ -> align'
250255 beamer <- gets stBeamer
251256 externalNotes <- gets stExternalNotes
252257 inMinipage <- gets stInMinipage
@@ -257,8 +262,13 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
257262 Para{} -> True
258263 Plain{} -> True
259264 _ -> False
265+ let hasLineBreak LineBreak = Any True
266+ hasLineBreak _ = Any False
267+ let hasLineBreaks = getAny $ query hasLineBreak blocks
260268 result <-
261− if all isPlainOrPara blocks
269+ if not hasWidths || (celltype /= HeaderCell
270+ && all isPlainOrPara blocks
271+ && not hasLineBreaks)
262272 then
263273 blockListToLaTeX $ walk fixLineBreaks $ walk displayMathToInline blocks
264274 else do
@@ -271,7 +281,9 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
271281 let halign = literal $ alignCommand align
272282 return $ "\\begin{minipage}" <> valign <>
273283 braces "\\linewidth" <> halign <> cr <>
274− cellContents <> cr <>
284+ cellContents <>
285+ (if hasLineBreaks then "\\strut" else mempty)
286+ <> cr <>
275287 "\\end{minipage}"
276288 modify $ \st -> st{ stExternalNotes = externalNotes }
277289 when (rowspan /= RowSpan 1) $
@@ -292,3 +304,4 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
292304data CellType
293305 = HeaderCell
294306 | BodyCell
307+ deriving Eq
src/Text/Pandoc/Writers/LaTeX/Types.hs+2−1
@@ -40,7 +40,7 @@ data WriterState =
4040 , stCsquotes :: Bool -- ^ true if document uses csquotes
4141 , stHighlighting :: Bool -- ^ true if document has highlighted code
4242 , stIncremental :: Bool -- ^ true if beamer lists should be
43− -- displayed bit by bit
43+ , stZwnj :: Bool -- ^ true if document has a ZWNJ character
4444 , stInternalLinks :: [Text] -- ^ list of internal link targets
4545 , stBeamer :: Bool -- ^ produce beamer
4646 , stEmptyLine :: Bool -- ^ true if no content on line
@@ -74,6 +74,7 @@ startingState options =
7474 , stCsquotes = False
7575 , stHighlighting = False
7676 , stIncremental = writerIncremental options
77+ , stZwnj = False
7778 , stInternalLinks = []
7879 , stBeamer = False
7980 , stEmptyLine = True
src/Text/Pandoc/Writers/LaTeX/Util.hs+4−3
@@ -22,6 +22,7 @@ module Text.Pandoc.Writers.LaTeX.Util (
2222where
2323
2424import Control.Applicative ((<|>))
25+import Control.Monad (when)
2526import Text.Pandoc.Class (PandocMonad, toLang)
2627import Text.Pandoc.Options (WriterOptions(..), isEnabled)
2728import Text.Pandoc.Writers.LaTeX.Types (LW, WriterState(..))
@@ -30,7 +31,7 @@ import Text.Pandoc.Highlighting (toListingsLanguage)
3031import Text.DocLayout
3132import Text.Pandoc.Definition
3233import Text.Pandoc.ImageSize (showFl)
33−import Control.Monad.State.Strict (gets)
34+import Control.Monad.State.Strict (gets, modify)
3435import Data.Text (Text)
3536import qualified Data.Text as T
3637import Text.Pandoc.Extensions (Extension(Ext_smart))
@@ -49,6 +50,8 @@ data StringContext = TextString
4950stringToLaTeX :: PandocMonad m => StringContext -> Text -> LW m Text
5051stringToLaTeX context zs = do
5152 opts <- gets stOptions
53+ when ('\x200c' `elemText` zs) $
54+ modify (\s -> s { stZwnj = True })
5255 return $ T.pack $
5356 foldr (go opts context) mempty $ T.unpack $
5457 if writerPreferAscii opts
@@ -270,5 +273,3 @@ mbBraced :: Text -> Text
270273mbBraced x = if not (T.all isAlphaNum x)
271274 then "{" <> x <> "}"
272275 else x
273−
274−
src/Text/Pandoc/Writers/Man.hs+2−2
@@ -16,6 +16,7 @@ Conversion of 'Pandoc' documents to roff man page format.
1616module Text.Pandoc.Writers.Man ( writeMan ) where
1717import Control.Monad.State.Strict
1818import Data.List (intersperse)
19+import Data.List.NonEmpty (nonEmpty)
1920import Data.Maybe (fromMaybe)
2021import Data.Text (Text)
2122import qualified Data.Text as T
@@ -175,8 +176,7 @@ blockToMan opts (BulletList items) = do
175176 return (vcat contents)
176177blockToMan opts (OrderedList attribs items) = do
177178 let markers = take (length items) $ orderedListMarkers attribs
178− let indent = 1 +
179− maximum (map T.length markers)
179+ let indent = 1 + maybe 0 maximum (nonEmpty (map T.length markers))
180180 contents <- mapM (\(num, item) -> orderedListItemToMan opts num indent item) $
181181 zip markers items
182182 return (vcat contents)
src/Text/Pandoc/Writers/Markdown.hs+54−56
@@ -24,8 +24,9 @@ import Control.Monad.Reader
2424import Control.Monad.State.Strict
2525import Data.Default
2626import Data.List (intersperse, sortOn, transpose)
27+import Data.List.NonEmpty (nonEmpty, NonEmpty(..))
2728import qualified Data.Map as M
28−import Data.Maybe (fromMaybe, mapMaybe)
29+import Data.Maybe (fromMaybe, mapMaybe, isNothing)
2930import qualified Data.Set as Set
3031import Data.Text (Text)
3132import qualified Data.Text as T
@@ -42,7 +43,7 @@ import Text.Pandoc.Templates (renderTemplate)
4243import Text.DocTemplates (Val(..), Context(..), FromContext(..))
4344import Text.Pandoc.Walk
4445import Text.Pandoc.Writers.HTML (writeHtml5String)
45−import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown)
46+import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown, linkAttributes, attrsToMarkdown)
4647import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(..),
4748 WriterState(..),
4849 WriterEnv(..),
@@ -66,7 +67,16 @@ writePlain opts document =
6667-- | Convert Pandoc to Commonmark.
6768writeCommonMark :: PandocMonad m => WriterOptions -> Pandoc -> m Text
6869writeCommonMark opts document =
69− evalMD (pandocToMarkdown opts document) def{ envVariant = Commonmark } def
70+ evalMD (pandocToMarkdown opts' document) def{ envVariant = Commonmark } def
71+ where
72+ opts' = opts{ writerExtensions =
73+ -- These extensions can't be enabled or disabled
74+ -- for commonmark because they're part of the core;
75+ -- we set them here so that escapeText will behave
76+ -- properly.
77+ enableExtension Ext_all_symbols_escapable $
78+ enableExtension Ext_intraword_underscores $
79+ writerExtensions opts }
7080
7181pandocTitleBlock :: Doc Text -> [Doc Text] -> Doc Text -> Doc Text
7282pandocTitleBlock tit auths dat =
@@ -131,18 +141,17 @@ valToYaml (SimpleVal x)
131141 | otherwise =
132142 if hasNewlines x
133143 then hang 0 ("|" <> cr) x
134− else if fst $ foldr needsDoubleQuotes (False, True) x
144+ else if isNothing $ foldM needsDoubleQuotes True x
135145 then "\"" <> fmap escapeInDoubleQuotes x <> "\""
136146 else x
137147 where
138− needsDoubleQuotes t (positive, isFirst)
148+ needsDoubleQuotes isFirst t
139149 = if T.any isBadAnywhere t ||
140150 (isFirst && T.any isYamlPunct (T.take 1 t))
141− then (True, False)
142− else (positive, False)
151+ then Nothing
152+ else Just False
143153 isBadAnywhere '#' = True
144154 isBadAnywhere ':' = True
145− isBadAnywhere '`' = False
146155 isBadAnywhere _ = False
147156 hasNewlines NewLine = True
148157 hasNewlines BlankLines{} = True
@@ -247,32 +256,11 @@ noteToMarkdown opts num blocks = do
247256 then hang (writerTabStop opts) (marker <> spacer) contents
248257 else marker <> spacer <> contents
249258
250−attrsToMarkdown :: Attr -> Doc Text
251−attrsToMarkdown attribs = braces $ hsep [attribId, attribClasses, attribKeys]
252− where attribId = case attribs of
253− ("",_,_) -> empty
254− (i,_,_) -> "#" <> escAttr i
255− attribClasses = case attribs of
256− (_,[],_) -> empty
257− (_,cs,_) -> hsep $
258− map (escAttr . ("."<>))
259− cs
260− attribKeys = case attribs of
261− (_,_,[]) -> empty
262− (_,_,ks) -> hsep $
263− map (\(k,v) -> escAttr k
264− <> "=\"" <>
265− escAttr v <> "\"") ks
266− escAttr = mconcat . map escAttrChar . T.unpack
267− escAttrChar '"' = literal "\\\""
268− escAttrChar '\\' = literal "\\\\"
269− escAttrChar c = literal $ T.singleton c
270−
271−linkAttributes :: WriterOptions -> Attr -> Doc Text
272−linkAttributes opts attr =
273− if isEnabled Ext_link_attributes opts && attr /= nullAttr
274− then attrsToMarkdown attr
275− else empty
259+-- | (Code) blocks with a single class and no attributes can just use it
260+-- standalone, no need to bother with curly braces.
261+classOrAttrsToMarkdown :: Attr -> Doc Text
262+classOrAttrsToMarkdown ("",[cls],[]) = literal cls
263+classOrAttrsToMarkdown attrs = attrsToMarkdown attrs
276264
277265-- | Ordered list start parser for use in Para below.
278266olMarker :: Parser Text ParserState ()
@@ -333,9 +321,12 @@ blockToMarkdown' opts (Div attrs ils) = do
333321 case () of
334322 _ | isEnabled Ext_fenced_divs opts &&
335323 attrs /= nullAttr ->
336− nowrap (literal ":::" <+> attrsToMarkdown attrs) $$
337− chomp contents $$
338− literal ":::" <> blankline
324+ let attrsToMd = if variant == Commonmark
325+ then attrsToMarkdown
326+ else classOrAttrsToMarkdown
327+ in nowrap (literal ":::" <+> attrsToMd attrs) $$
328+ chomp contents $$
329+ literal ":::" <> blankline
339330 | isEnabled Ext_native_divs opts ||
340331 (isEnabled Ext_raw_html opts &&
341332 (variant == Commonmark ||
@@ -377,7 +368,7 @@ blockToMarkdown' opts (Plain inlines) = do
377368-- title beginning with fig: indicates figure
378369blockToMarkdown' opts (Para [Image attr alt (src,tgt@(T.stripPrefix "fig:" -> Just tit))])
379370 | isEnabled Ext_raw_html opts &&
380− not (isEnabled Ext_link_attributes opts) &&
371+ not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) &&
381372 attr /= nullAttr = -- use raw HTML
382373 (<> blankline) . literal . T.strip <$>
383374 writeHtml5String opts{ writerTemplate = Nothing }
@@ -442,7 +433,8 @@ blockToMarkdown' opts (Header level attr inlines) = do
442433 && id' == autoId -> empty
443434 (id',_,_) | isEnabled Ext_mmd_header_identifiers opts ->
444435 space <> brackets (literal id')
445− _ | isEnabled Ext_header_attributes opts ->
436+ _ | isEnabled Ext_header_attributes opts ||
437+ isEnabled Ext_attributes opts ->
446438 space <> attrsToMarkdown attr
447439 | otherwise -> empty
448440 contents <- inlineListToMarkdown opts $
@@ -492,19 +484,21 @@ blockToMarkdown' opts (CodeBlock attribs str) = do
492484 | isEnabled Ext_fenced_code_blocks opts ->
493485 tildes <> attrs <> cr <> literal str <> cr <> tildes <> blankline
494486 _ -> nest (writerTabStop opts) (literal str) <> blankline
495− where endline c = literal $ case [T.length ln
496− | ln <- map trim (T.lines str)
497− , T.pack [c,c,c] `T.isPrefixOf` ln
498− , T.all (== c) ln] of
499− [] -> T.replicate 3 $ T.singleton c
500− xs -> T.replicate (maximum xs + 1) $ T.singleton c
501− backticks = endline '`'
502− tildes = endline '~'
503− attrs = if isEnabled Ext_fenced_code_attributes opts
504− then nowrap $ " " <> attrsToMarkdown attribs
505− else case attribs of
506− (_,cls:_,_) -> " " <> literal cls
507− _ -> empty
487+ where
488+ endlineLen c = maybe 3 ((+1) . maximum) $ nonEmpty $
489+ [T.length ln
490+ | ln <- map trim (T.lines str)
491+ , T.pack [c,c,c] `T.isPrefixOf` ln
492+ , T.all (== c) ln]
493+ endline c = literal $ T.replicate (endlineLen c) $ T.singleton c
494+ backticks = endline '`'
495+ tildes = endline '~'
496+ attrs = if isEnabled Ext_fenced_code_attributes opts ||
497+ isEnabled Ext_attributes opts
498+ then nowrap $ " " <> classOrAttrsToMarkdown attribs
499+ else case attribs of
500+ (_,cls:_,_) -> " " <> literal cls
501+ _ -> empty
508502blockToMarkdown' opts (BlockQuote blocks) = do
509503 variant <- asks envVariant
510504 -- if we're writing literate haskell, put a space before the bird tracks
@@ -517,7 +511,7 @@ blockToMarkdown' opts (BlockQuote blocks) = do
517511 return $ prefixed leader contents <> blankline
518512blockToMarkdown' opts t@(Table _ blkCapt specs thead tbody tfoot) = do
519513 let (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
520− let numcols = maximum (length aligns : length widths :
514+ let numcols = maximum (length aligns :| length widths :
521515 map length (headers:rows))
522516 caption' <- inlineListToMarkdown opts caption
523517 let caption''
@@ -619,7 +613,8 @@ pipeTable headless aligns rawHeaders rawRows = do
619613 blockFor AlignCenter x y = cblock (x + 2) (sp <> y <> sp) <> lblock 0 empty
620614 blockFor AlignRight x y = rblock (x + 2) (y <> sp) <> lblock 0 empty
621615 blockFor _ x y = lblock (x + 2) (sp <> y) <> lblock 0 empty
622− let widths = map (max 3 . maximum . map offset) $ transpose (rawHeaders : rawRows)
616+ let widths = map (max 3 . maybe 3 maximum . nonEmpty . map offset) $
617+ transpose (rawHeaders : rawRows)
623618 let torow cs = nowrap $ literal "|" <>
624619 hcat (intersperse (literal "|") $
625620 zipWith3 blockFor aligns widths (map chomp cs))
@@ -653,11 +648,11 @@ pandocTable opts multiline headless aligns widths rawHeaders rawRows = do
653648 -- Number of characters per column necessary to output every cell
654649 -- without requiring a line break.
655650 -- The @+2@ is needed for specifying the alignment.
656− let numChars = (+ 2) . maximum . map offset
651+ let numChars = (+ 2) . maybe 0 maximum . nonEmpty . map offset
657652 -- Number of characters per column necessary to output every cell
658653 -- without requiring a line break *inside a word*.
659654 -- The @+2@ is needed for specifying the alignment.
660− let minNumChars = (+ 2) . maximum . map minOffset
655+ let minNumChars = (+ 2) . maybe 0 maximum . nonEmpty . map minOffset
661656 let columns = transpose (rawHeaders : rawRows)
662657 -- minimal column width without wrapping a single word
663658 let relWidth w col =
@@ -796,6 +791,9 @@ blockListToMarkdown opts blocks = do
796791 b1 : commentSep : fixBlocks (b2:bs)
797792 fixBlocks (Plain ils : bs@(RawBlock{}:_)) =
798793 Plain ils : fixBlocks bs
794+ fixBlocks (Plain ils : bs@(Div{}:_))
795+ | isEnabled Ext_fenced_divs opts =
796+ Para ils : fixBlocks bs
799797 fixBlocks (Plain ils : bs) | inlist =
800798 Plain ils : fixBlocks bs
801799 fixBlocks (Plain ils : bs) =
src/Text/Pandoc/Writers/Markdown/Inline.hs+36−21
@@ -11,12 +11,15 @@
1111 Portability : portable
1212-}
1313module Text.Pandoc.Writers.Markdown.Inline (
14− inlineListToMarkdown
14+ inlineListToMarkdown,
15+ linkAttributes,
16+ attrsToMarkdown
1517 ) where
1618import Control.Monad.Reader
1719import Control.Monad.State.Strict
1820import Data.Char (isAlphaNum, isDigit)
1921import Data.List (find, intersperse)
22+import Data.List.NonEmpty (nonEmpty)
2023import qualified Data.Map as M
2124import Data.Maybe (fromMaybe)
2225import Data.Text (Text)
@@ -42,7 +45,11 @@ import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(..),
4245-- | Escape special characters for Markdown.
4346escapeText :: WriterOptions -> Text -> Text
4447escapeText opts = T.pack . go . T.unpack
45− where
48+ where
49+ startsWithSpace (' ':_) = True
50+ startsWithSpace ('\t':_) = True
51+ startsWithSpace [] = True
52+ startsWithSpace _ = False
4653 go [] = []
4754 go (c:cs) =
4855 case c of
@@ -55,10 +62,13 @@ escapeText opts = T.pack . go . T.unpack
5562 '@' | isEnabled Ext_citations opts ->
5663 case cs of
5764 (d:_)
58− | isAlphaNum d || d == '_'
65+ | isAlphaNum d || d == '_' || d == '{'
5966 -> '\\':'@':go cs
6067 _ -> '@':go cs
61− _ | c `elem` ['\\','`','*','_','[',']','#'] ->
68+ '#' | isEnabled Ext_space_in_atx_header opts
69+ , startsWithSpace cs
70+ -> '\\':'#':go cs
71+ _ | c `elem` ['\\','`','*','_','[',']'] ->
6272 '\\':c:go cs
6373 '|' | isEnabled Ext_pipe_tables opts -> '\\':'|':go cs
6474 '^' | isEnabled Ext_superscript opts -> '\\':'^':go cs
@@ -80,6 +90,8 @@ escapeText opts = T.pack . go . T.unpack
8090 | isEnabled Ext_intraword_underscores opts
8191 , isAlphaNum c
8292 , isAlphaNum x -> c : '_' : x : go xs
93+ '#':xs -> c : '#' : go xs
94+ '>':xs -> c : '>' : go xs
8395 _ -> c : go cs
8496
8597attrsToMarkdown :: Attr -> Doc Text
@@ -105,7 +117,7 @@ attrsToMarkdown attribs = braces $ hsep [attribId, attribClasses, attribKeys]
105117
106118linkAttributes :: WriterOptions -> Attr -> Doc Text
107119linkAttributes opts attr =
108− if isEnabled Ext_link_attributes opts && attr /= nullAttr
120+ if (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) && attr /= nullAttr
109121 then attrsToMarkdown attr
110122 else empty
111123
@@ -382,15 +394,15 @@ inlineToMarkdown opts (Quoted DoubleQuote lst) = do
382394 then "&ldquo;" <> contents <> "&rdquo;"
383395 else "“" <> contents <> "”"
384396inlineToMarkdown opts (Code attr str) = do
385− let tickGroups = filter (T.any (== '`')) $ T.group str
386− let longest = if null tickGroups
387− then 0
388− else maximum $ map T.length tickGroups
389− let marker = T.replicate (longest + 1) "`"
390− let spacer = if longest == 0 then "" else " "
391− let attrs = if isEnabled Ext_inline_code_attributes opts && attr /= nullAttr
392− then attrsToMarkdown attr
393− else empty
397+ let tickGroups = filter (T.any (== '`')) $ T.group str
398+ let longest = maybe 0 maximum $ nonEmpty $ map T.length tickGroups
399+ let marker = T.replicate (longest + 1) "`"
400+ let spacer = if longest == 0 then "" else " "
401+ let attrsEnabled = isEnabled Ext_inline_code_attributes opts ||
402+ isEnabled Ext_attributes opts
403+ let attrs = if attrsEnabled && attr /= nullAttr
404+ then attrsToMarkdown attr
405+ else empty
394406 variant <- asks envVariant
395407 case variant of
396408 PlainText -> return $ literal str
@@ -438,9 +450,7 @@ inlineToMarkdown opts (Math DisplayMath str) =
438450 (texMathToInlines DisplayMath str >>= inlineListToMarkdown opts)
439451inlineToMarkdown opts il@(RawInline f str) = do
440452 let tickGroups = filter (T.any (== '`')) $ T.group str
441− let numticks = if null tickGroups
442− then 1
443− else 1 + maximum (map T.length tickGroups)
453+ let numticks = 1 + maybe 0 maximum (nonEmpty (map T.length tickGroups))
444454 variant <- asks envVariant
445455 let Format fmt = f
446456 let rawAttribInline = return $
@@ -494,11 +504,16 @@ inlineToMarkdown opts (Cite (c:cs) lst)
494504 rest <- mapM convertOne cs
495505 let inbr = suffs <+> joincits rest
496506 br = if isEmpty inbr then empty else char '[' <> inbr <> char ']'
497− return $ literal ("@" <> citationId c) <+> br
507+ return $ literal ("@" <> maybeInBraces (citationId c)) <+> br
498508 else do
499509 cits <- mapM convertOne (c:cs)
500510 return $ literal "[" <> joincits cits <> literal "]"
501511 where
512+ maybeInBraces key =
513+ case readWith (citeKey False >> spaces >> eof)
514+ defaultParserState ("@" <> key) of
515+ Left _ -> "{" <> key <> "}"
516+ Right _ -> key
502517 joincits = hcat . intersperse (literal "; ") . filter (not . isEmpty)
503518 convertOne Citation { citationId = k
504519 , citationPrefix = pinlines
@@ -507,7 +522,7 @@ inlineToMarkdown opts (Cite (c:cs) lst)
507522 = do
508523 pdoc <- inlineListToMarkdown opts pinlines
509524 sdoc <- inlineListToMarkdown opts sinlines
510− let k' = literal (modekey m <> "@" <> k)
525+ let k' = literal (modekey m <> "@" <> maybeInBraces k)
511526 r = case sinlines of
512527 Str (T.uncons -> Just (y,_)):_ | y `elem` (",;]@" :: String) -> k' <> sdoc
513528 _ -> k' <+> sdoc
@@ -546,7 +561,7 @@ inlineToMarkdown opts lnk@(Link attr txt (src, tit)) = do
546561 else "[" <> reftext <> "]"
547562 in return $ first <> second
548563 | isEnabled Ext_raw_html opts
549− , not (isEnabled Ext_link_attributes opts)
564+ , not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts)
550565 , attr /= nullAttr -> -- use raw HTML to render attributes
551566 literal . T.strip <$>
552567 writeHtml5String opts{ writerTemplate = Nothing }
@@ -556,7 +571,7 @@ inlineToMarkdown opts lnk@(Link attr txt (src, tit)) = do
556571 linkAttributes opts attr
557572inlineToMarkdown opts img@(Image attr alternate (source, tit))
558573 | isEnabled Ext_raw_html opts &&
559− not (isEnabled Ext_link_attributes opts) &&
574+ not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) &&
560575 attr /= nullAttr = -- use raw HTML
561576 literal . T.strip <$>
562577 writeHtml5String opts{ writerTemplate = Nothing } (Pandoc nullMeta [Plain [img]])
src/Text/Pandoc/Writers/Ms.hs+24−8
@@ -23,6 +23,7 @@ module Text.Pandoc.Writers.Ms ( writeMs ) where
2323import Control.Monad.State.Strict
2424import Data.Char (isLower, isUpper, ord)
2525import Data.List (intercalate, intersperse)
26+import Data.List.NonEmpty (nonEmpty)
2627import qualified Data.Map as Map
2728import Data.Maybe (catMaybes)
2829import Data.Text (Text)
@@ -244,13 +245,17 @@ blockToMs opts (Table _ blkCapt specs thead tbody tfoot) =
244245 aligncode AlignDefault = "l"
245246 in do
246247 caption' <- inlineListToMs' opts caption
247− let iwidths = if all (== 0) widths
248− then repeat ""
249− else map (T.pack . printf "w(%0.1fn)" . (70 *)) widths
248+ let isSimple = all (== 0) widths
249+ let totalWidth = 70
250250 -- 78n default width - 8n indent = 70n
251251 let coldescriptions = literal $ T.unwords
252− (zipWith (\align width -> aligncode align <> width)
253− alignments iwidths) <> "."
252+ (zipWith (\align width -> aligncode align <>
253+ if width == 0
254+ then ""
255+ else T.pack $
256+ printf "w(%0.1fn)"
257+ (totalWidth * width))
258+ alignments widths) <> "."
254259 colheadings <- mapM (blockListToMs opts) headers
255260 let makeRow cols = literal "T{" $$
256261 vcat (intersperse (literal "T}\tT{") cols) $$
@@ -259,13 +264,25 @@ blockToMs opts (Table _ blkCapt specs thead tbody tfoot) =
259264 then empty
260265 else makeRow colheadings $$ char '_'
261266 body <- mapM (\row -> do
262− cols <- mapM (blockListToMs opts) row
267+ cols <- mapM (\(cell, w) ->
268+ (if isSimple
269+ then id
270+ else (literal (".nr LL " <>
271+ T.pack (printf "%0.1fn"
272+ (w * totalWidth))) $$)) <$>
273+ blockListToMs opts cell) (zip row widths)
263274 return $ makeRow cols) rows
264275 setFirstPara
265276 return $ literal ".PP" $$ caption' $$
266277 literal ".na" $$ -- we don't want justification in table cells
278+ (if isSimple
279+ then ""
280+ else ".nr LLold \\n[LL]") $$
267281 literal ".TS" $$ literal "delim(@@) tab(\t);" $$ coldescriptions $$
268282 colheadings' $$ vcat body $$ literal ".TE" $$
283+ (if isSimple
284+ then ""
285+ else ".nr LL \\n[LLold]") $$
269286 literal ".ad"
270287
271288blockToMs opts (BulletList items) = do
@@ -274,8 +291,7 @@ blockToMs opts (BulletList items) = do
274291 return (vcat contents)
275292blockToMs opts (OrderedList attribs items) = do
276293 let markers = take (length items) $ orderedListMarkers attribs
277− let indent = 2 +
278− maximum (map T.length markers)
294+ let indent = 2 + maybe 0 maximum (nonEmpty (map T.length markers))
279295 contents <- mapM (\(num, item) -> orderedListItemToMs opts num indent item) $
280296 zip markers items
281297 setFirstPara
src/Text/Pandoc/Writers/Muse.hs+6−3
@@ -31,6 +31,7 @@ import Control.Monad.State.Strict
3131import Data.Char (isAlphaNum, isAsciiLower, isAsciiUpper, isDigit, isSpace)
3232import Data.Default
3333import Data.List (intersperse, transpose)
34+import Data.List.NonEmpty (nonEmpty, NonEmpty(..))
3435import qualified Data.Set as Set
3536import qualified Data.Text as T
3637import Data.Text (Text)
@@ -158,7 +159,8 @@ simpleTable caption headers rows = do
158159 caption' <- inlineListToMuse caption
159160 headers' <- mapM blockListToMuse headers
160161 rows' <- mapM (mapM blockListToMuse) rows
161− let widthsInChars = maximum . map offset <$> transpose (headers' : rows')
162+ let widthsInChars = maybe 0 maximum . nonEmpty . map offset <$>
163+ transpose (headers' : rows')
162164 let hpipeBlocks sep blocks = hcat $ intersperse sep' blocks
163165 where sep' = lblock (T.length sep) $ literal sep
164166 let makeRow sep = hpipeBlocks sep . zipWith lblock widthsInChars
@@ -238,7 +240,7 @@ blockToMuse (DefinitionList items) = do
238240 label' <- local (\env -> env { envOneLine = True, envAfterSpace = True }) $ inlineListToMuse' label
239241 let ind = offset' label' -- using Text.DocLayout.offset results in round trip failures
240242 hang ind (nowrap label') . vcat <$> mapM descriptionToMuse defs
241− where offset' d = maximum (0: map T.length
243+ where offset' d = maximum (0 :| map T.length
242244 (T.lines $ render Nothing d))
243245 descriptionToMuse :: PandocMonad m
244246 => [Block]
@@ -269,7 +271,8 @@ blockToMuse (Table _ blkCapt specs thead tbody tfoot) =
269271 (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
270272 blocksToDoc opts blocks =
271273 local (\env -> env { envOptions = opts }) $ blockListToMuse blocks
272− numcols = maximum (length aligns : length widths : map length (headers:rows))
274+ numcols = maximum
275+ (length aligns :| length widths : map length (headers:rows))
273276 isSimple = onlySimpleTableCells (headers : rows) && all (== 0) widths
274277blockToMuse (Div _ bs) = flatBlockListToMuse bs
275278blockToMuse Null = return empty
src/Text/Pandoc/Writers/ODT.hs+4−3
@@ -16,6 +16,7 @@ import Codec.Archive.Zip
1616import Control.Monad.Except (catchError, throwError)
1717import Control.Monad.State.Strict
1818import qualified Data.ByteString.Lazy as B
19+import Data.Maybe (fromMaybe)
1920import Data.Generics (everywhere', mkT)
2021import Data.List (isPrefixOf)
2122import qualified Data.Map as Map
@@ -23,7 +24,7 @@ import qualified Data.Text as T
2324import qualified Data.Text.Lazy as TL
2425import Data.Time
2526import System.FilePath (takeDirectory, takeExtension, (<.>))
26−import Text.Pandoc.BCP47 (Lang (..), getLang, renderLang)
27+import Text.Collate.Lang (Lang (..), renderLang)
2728import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
2829import qualified Text.Pandoc.Class.PandocMonad as P
2930import Text.Pandoc.Definition
@@ -35,7 +36,7 @@ import Text.Pandoc.Options (WrapOption (..), WriterOptions (..))
3536import Text.DocLayout
3637import Text.Pandoc.Shared (stringify, pandocVersion, tshow)
3738import Text.Pandoc.Writers.Shared (lookupMetaString, lookupMetaBlocks,
38− fixDisplayMath)
39+ fixDisplayMath, getLang)
3940import Text.Pandoc.UTF8 (fromStringLazy, fromTextLazy, toTextLazy)
4041import Text.Pandoc.Walk
4142import Text.Pandoc.Writers.OpenDocument (writeOpenDocument)
@@ -194,7 +195,7 @@ addLang lang = everywhere' (mkT updateLangAttr)
194195 where updateLangAttr (Attr n@(QName "language" _ (Just "fo")) _)
195196 = Attr n (langLanguage lang)
196197 updateLangAttr (Attr n@(QName "country" _ (Just "fo")) _)
197− = Attr n (langRegion lang)
198+ = Attr n (fromMaybe "" $ langRegion lang)
198199 updateLangAttr x = x
199200
200201-- | transform both Image and Math elements
src/Text/Pandoc/Writers/OpenDocument.hs+50−54
@@ -1,4 +1,5 @@
11{-# LANGUAGE FlexibleContexts #-}
2+{-# LANGUAGE LambdaCase #-}
23{-# LANGUAGE OverloadedStrings #-}
34{-# LANGUAGE PatternGuards #-}
45{-# LANGUAGE ViewPatterns #-}
@@ -25,7 +26,7 @@ import Data.Ord (comparing)
2526import qualified Data.Set as Set
2627import Data.Text (Text)
2728import qualified Data.Text as T
28−import Text.Pandoc.BCP47 (Lang (..), parseBCP47)
29+import Text.Collate.Lang (Lang (..), parseLang)
2930import Text.Pandoc.Class.PandocMonad (PandocMonad, report, translateTerm,
3031 setTranslations, toLang)
3132import Text.Pandoc.Definition
@@ -236,7 +237,7 @@ handleSpaces s = case T.uncons s of
236237-- | Convert Pandoc document to string in OpenDocument format.
237238writeOpenDocument :: PandocMonad m => WriterOptions -> Pandoc -> m Text
238239writeOpenDocument opts (Pandoc meta blocks) = do
239− let defLang = Lang "en" "US" "" []
240+ let defLang = Lang "en" (Just "US") Nothing [] [] []
240241 lang <- case lookupMetaString "lang" meta of
241242 "" -> pure defLang
242243 s -> fromMaybe defLang <$> toLang (Just s)
@@ -372,36 +373,32 @@ blocksToOpenDocument o b = vcat <$> mapM (blockToOpenDocument o) b
372373
373374-- | Convert a Pandoc block element to OpenDocument.
374375blockToOpenDocument :: PandocMonad m => WriterOptions -> Block -> OD m (Doc Text)
375−blockToOpenDocument o bs
376− | Plain b <- bs = if null b
377− then return empty
378− else inParagraphTags =<< inlinesToOpenDocument o b
379− | Para [Image attr c (s,T.stripPrefix "fig:" -> Just t)] <- bs
380− = figure attr c s t
381− | Para b <- bs = if null b &&
382− not (isEnabled Ext_empty_paragraphs o)
383− then return empty
384− else inParagraphTags =<< inlinesToOpenDocument o b
385− | LineBlock b <- bs = blockToOpenDocument o $ linesToPara b
386− | Div attr xs <- bs = mkDiv attr xs
387− | Header i (ident,_,_) b
388− <- bs = setFirstPara >> (inHeaderTags i ident
389− =<< inlinesToOpenDocument o b)
390− | BlockQuote b <- bs = setFirstPara >> mkBlockQuote b
391− | DefinitionList b <- bs = setFirstPara >> defList b
392− | BulletList b <- bs = setFirstPara >> bulletListToOpenDocument o b
393− | OrderedList a b <- bs = setFirstPara >> orderedList a b
394− | CodeBlock _ s <- bs = setFirstPara >> preformatted s
395− | Table a bc s th tb tf <- bs = setFirstPara >> table (Ann.toTable a bc s th tb tf)
396− | HorizontalRule <- bs = setFirstPara >> return (selfClosingTag "text:p"
397− [ ("text:style-name", "Horizontal_20_Line") ])
398− | RawBlock f s <- bs = if f == Format "opendocument"
399− then return $ text $ T.unpack s
400− else do
401− report $ BlockNotRendered bs
402− return empty
403− | Null <- bs = return empty
404− | otherwise = return empty
376+blockToOpenDocument o = \case
377+ Plain b -> if null b
378+ then return empty
379+ else inParagraphTags =<< inlinesToOpenDocument o b
380+ Para [Image attr c (s,T.stripPrefix "fig:" -> Just t)] -> figure attr c s t
381+ Para b -> if null b &&
382+ not (isEnabled Ext_empty_paragraphs o)
383+ then return empty
384+ else inParagraphTags =<< inlinesToOpenDocument o b
385+ LineBlock b -> blockToOpenDocument o $ linesToPara b
386+ Div attr xs -> mkDiv attr xs
387+ Header i (ident,_,_) b -> do
388+ setFirstPara
389+ inHeaderTags i ident =<< inlinesToOpenDocument o b
390+ BlockQuote b -> setFirstPara >> mkBlockQuote b
391+ DefinitionList b -> setFirstPara >> defList b
392+ BulletList b -> setFirstPara >> bulletListToOpenDocument o b
393+ OrderedList a b -> setFirstPara >> orderedList a b
394+ CodeBlock _ s -> setFirstPara >> preformatted s
395+ Table a bc s th tb tf -> setFirstPara >> table (Ann.toTable a bc s th tb tf)
396+ HorizontalRule -> setFirstPara >> return (selfClosingTag "text:p"
397+ [ ("text:style-name", "Horizontal_20_Line") ])
398+ b@(RawBlock f s) -> if f == Format "opendocument"
399+ then return $ text $ T.unpack s
400+ else empty <$ report (BlockNotRendered b)
401+ Null -> return empty
405402 where
406403 defList b = do setInDefinitionList True
407404 r <- vcat <$> mapM (deflistItemToOpenDocument o) b
@@ -874,34 +871,33 @@ data TextStyle = Italic
874871textStyleAttr :: Map.Map Text Text
875872 -> TextStyle
876873 -> Map.Map Text Text
877−textStyleAttr m s
878− | Italic <- s = Map.insert "fo:font-style" "italic" .
879− Map.insert "style:font-style-asian" "italic" .
880− Map.insert "style:font-style-complex" "italic" $ m
881− | Bold <- s = Map.insert "fo:font-weight" "bold" .
882− Map.insert "style:font-weight-asian" "bold" .
883− Map.insert "style:font-weight-complex" "bold" $ m
884− | Under <- s = Map.insert "style:text-underline-style" "solid" .
885− Map.insert "style:text-underline-width" "auto" .
886− Map.insert "style:text-underline-color" "font-color" $ m
887− | Strike <- s = Map.insert "style:text-line-through-style" "solid" m
888− | Sub <- s = Map.insert "style:text-position" "sub 58%" m
889− | Sup <- s = Map.insert "style:text-position" "super 58%" m
890− | SmallC <- s = Map.insert "fo:font-variant" "small-caps" m
891− | Pre <- s = Map.insert "style:font-name" "Courier New" .
892− Map.insert "style:font-name-asian" "Courier New" .
893− Map.insert "style:font-name-complex" "Courier New" $ m
894− | Language lang <- s
895− = Map.insert "fo:language" (langLanguage lang) .
896− Map.insert "fo:country" (langRegion lang) $ m
897− | otherwise = m
874+textStyleAttr m = \case
875+ Italic -> Map.insert "fo:font-style" "italic" .
876+ Map.insert "style:font-style-asian" "italic" .
877+ Map.insert "style:font-style-complex" "italic" $ m
878+ Bold -> Map.insert "fo:font-weight" "bold" .
879+ Map.insert "style:font-weight-asian" "bold" .
880+ Map.insert "style:font-weight-complex" "bold" $ m
881+ Under -> Map.insert "style:text-underline-style" "solid" .
882+ Map.insert "style:text-underline-width" "auto" .
883+ Map.insert "style:text-underline-color" "font-color" $ m
884+ Strike -> Map.insert "style:text-line-through-style" "solid" m
885+ Sub -> Map.insert "style:text-position" "sub 58%" m
886+ Sup -> Map.insert "style:text-position" "super 58%" m
887+ SmallC -> Map.insert "fo:font-variant" "small-caps" m
888+ Pre -> Map.insert "style:font-name" "Courier New" .
889+ Map.insert "style:font-name-asian" "Courier New" .
890+ Map.insert "style:font-name-complex" "Courier New" $ m
891+ Language lang ->
892+ Map.insert "fo:language" (langLanguage lang) .
893+ maybe id (Map.insert "fo:country") (langRegion lang) $ m
898894
899895withLangFromAttr :: PandocMonad m => Attr -> OD m a -> OD m a
900896withLangFromAttr (_,_,kvs) action =
901897 case lookup "lang" kvs of
902898 Nothing -> action
903899 Just l ->
904− case parseBCP47 l of
900+ case parseLang l of
905901 Right lang -> withTextStyle (Language lang) action
906902 Left _ -> do
907903 report $ InvalidLang l
src/Text/Pandoc/Writers/Org.hs+24−14
@@ -17,8 +17,9 @@ Org-Mode: <http://orgmode.org>
1717-}
1818module Text.Pandoc.Writers.Org (writeOrg) where
1919import Control.Monad.State.Strict
20−import Data.Char (isAlphaNum)
20+import Data.Char (isAlphaNum, isDigit)
2121import Data.List (intersect, intersperse, partition, transpose)
22+import Data.List.NonEmpty (nonEmpty)
2223import Data.Text (Text)
2324import qualified Data.Text as T
2425import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
@@ -83,12 +84,15 @@ noteToOrg num note = do
8384
8485-- | Escape special characters for Org.
8586escapeString :: Text -> Text
86−escapeString = escapeStringUsing
87− [ ('\x2014',"---")
88− , ('\x2013',"--")
89− , ('\x2019',"'")
90− , ('\x2026',"...")
91− ]
87+escapeString t
88+ | T.all (\c -> c < '\x2013' || c > '\x2026') t = t
89+ | otherwise = T.concatMap escChar t
90+ where
91+ escChar '\x2013' = "--"
92+ escChar '\x2014' = "---"
93+ escChar '\x2019' = "'"
94+ escChar '\x2026' = "..."
95+ escChar c = T.singleton c
9296
9397isRawFormat :: Format -> Bool
9498isRawFormat f =
@@ -163,7 +167,7 @@ blockToOrg (Table _ blkCapt specs thead tbody tfoot) = do
163167 else "#+caption: " <> caption''
164168 headers' <- mapM blockListToOrg headers
165169 rawRows <- mapM (mapM blockListToOrg) rows
166− let numChars = maximum . map offset
170+ let numChars = maybe 0 maximum . nonEmpty . map offset
167171 -- FIXME: width is not being used.
168172 let widthsInChars =
169173 map numChars $ transpose (headers' : rawRows)
@@ -198,7 +202,7 @@ blockToOrg (OrderedList (start, _, delim) items) = do
198202 x -> x
199203 let markers = take (length items) $ orderedListMarkers
200204 (start, Decimal, delim')
201− let maxMarkerLength = maximum $ map T.length markers
205+ let maxMarkerLength = maybe 0 maximum . nonEmpty $ map T.length markers
202206 let markers' = map (\m -> let s = maxMarkerLength - T.length m
203207 in m <> T.replicate s " ") markers
204208 contents <- zipWithM orderedListItemToOrg markers' items
@@ -347,16 +351,20 @@ inlineListToOrg :: PandocMonad m
347351 => [Inline]
348352 -> Org m (Doc Text)
349353inlineListToOrg lst = hcat <$> mapM inlineToOrg (fixMarkers lst)
350− where fixMarkers [] = [] -- prevent note refs and list markers from wrapping, see #4171
354+ where -- Prevent note refs and list markers from wrapping, see #4171
355+ -- and #7132.
356+ fixMarkers [] = []
351357 fixMarkers (Space : x : rest) | shouldFix x =
352358 Str " " : x : fixMarkers rest
353359 fixMarkers (SoftBreak : x : rest) | shouldFix x =
354360 Str " " : x : fixMarkers rest
355361 fixMarkers (x : rest) = x : fixMarkers rest
356362
357− shouldFix Note{} = True -- Prevent footnotes
363+ shouldFix Note{} = True -- Prevent footnotes
358364 shouldFix (Str "-") = True -- Prevent bullet list items
359− -- TODO: prevent ordered list items
365+ shouldFix (Str x) -- Prevent ordered list items
366+ | Just (cs, c) <- T.unsnoc x = T.all isDigit cs &&
367+ (c == '.' || c == ')')
360368 shouldFix _ = False
361369
362370-- | Convert Pandoc inline element to Org.
@@ -396,9 +404,11 @@ inlineToOrg (Str str) = return . literal $ escapeString str
396404inlineToOrg (Math t str) = do
397405 modify $ \st -> st{ stHasMath = True }
398406 return $ if t == InlineMath
399− then "$" <> literal str <> "$"
400− else "$$" <> literal str <> "$$"
407+ then "\\(" <> literal str <> "\\)"
408+ else "\\[" <> literal str <> "\\]"
401409inlineToOrg il@(RawInline f str)
410+ | elem f ["tex", "latex"] && T.isPrefixOf "\\begin" str =
411+ return $ cr <> literal str <> cr
402412 | isRawFormat f = return $ literal str
403413 | otherwise = do
404414 report $ InlineNotRendered il
src/Text/Pandoc/Writers/Powerpoint.hs+1−1
@@ -39,5 +39,5 @@ writePowerpoint opts (Pandoc meta blks) = do
3939 let blks' = walk fixDisplayMath blks
4040 let (pres, logMsgs) = documentToPresentation opts (Pandoc meta blks')
4141 mapM_ report logMsgs
42− archv <- presentationToArchive opts pres
42+ archv <- presentationToArchive opts meta pres
4343 return $ fromArchive archv
src/Text/Pandoc/Writers/Powerpoint/Output.hs+43−31
@@ -38,18 +38,21 @@ import qualified Text.Pandoc.Class.PandocMonad as P
3838import Text.Pandoc.Options
3939import Text.Pandoc.MIME
4040import qualified Data.ByteString.Lazy as BL
41+import Text.Pandoc.Writers.Shared (metaToContext)
4142import Text.Pandoc.Writers.OOXML
4243import qualified Data.Map as M
4344import Data.Maybe (mapMaybe, listToMaybe, fromMaybe, maybeToList, catMaybes, isJust)
4445import Text.Pandoc.ImageSize
4546import Control.Applicative ((<|>))
4647import System.FilePath.Glob
47−import Text.DocTemplates (FromContext(lookupContext))
48+import Text.DocTemplates (FromContext(lookupContext), Context)
49+import Text.DocLayout (literal)
4850import Text.TeXMath
4951import Text.Pandoc.Writers.Math (convertMath)
5052import Text.Pandoc.Writers.Powerpoint.Presentation
51−import Text.Pandoc.Shared (tshow)
53+import Text.Pandoc.Shared (tshow, stringify)
5254import Skylighting (fromColor)
55+import Data.List.NonEmpty (nonEmpty)
5356
5457-- |The 'EMU' type is used to specify sizes in English Metric Units.
5558type EMU = Integer
@@ -96,6 +99,7 @@ data WriterEnv = WriterEnv { envRefArchive :: Archive
9699 , envDistArchive :: Archive
97100 , envUTCTime :: UTCTime
98101 , envOpts :: WriterOptions
102+ , envContext :: Context Text
99103 , envPresentationSize :: (Integer, Integer)
100104 , envSlideHasHeader :: Bool
101105 , envInList :: Bool
@@ -121,6 +125,7 @@ instance Default WriterEnv where
121125 , envDistArchive = emptyArchive
122126 , envUTCTime = posixSecondsToUTCTime 0
123127 , envOpts = def
128+ , envContext = mempty
124129 , envPresentationSize = (720, 540)
125130 , envSlideHasHeader = False
126131 , envInList = False
@@ -167,7 +172,7 @@ runP env st p = evalStateT (runReaderT p env) st
167172
168173monospaceFont :: Monad m => P m T.Text
169174monospaceFont = do
170− vars <- writerVariables <$> asks envOpts
175+ vars <- asks envContext
171176 case lookupContext "monofont" vars of
172177 Just s -> return s
173178 Nothing -> return "Courier"
@@ -303,8 +308,9 @@ makeSpeakerNotesMap (Presentation _ slides) =
303308 then Nothing
304309 else Just n
305310
306−presentationToArchive :: PandocMonad m => WriterOptions -> Presentation -> m Archive
307−presentationToArchive opts pres = do
311+presentationToArchive :: PandocMonad m
312+ => WriterOptions -> Meta -> Presentation -> m Archive
313+presentationToArchive opts meta pres = do
308314 distArchive <- toArchive . BL.fromStrict <$>
309315 P.readDefaultDataFile "reference.pptx"
310316 refArchive <- case writerReferenceDoc opts of
@@ -320,10 +326,18 @@ presentationToArchive opts pres = do
320326 PandocSomeError
321327 "Could not determine presentation size"
322328
329+ -- note, we need writerTemplate to be Just _ or metaToContext does
330+ -- nothing
331+ context <- metaToContext opts{ writerTemplate =
332+ writerTemplate opts <|> Just mempty }
333+ (return . literal . stringify)
334+ (return . literal . stringify) meta
335+
323336 let env = def { envRefArchive = refArchive
324337 , envDistArchive = distArchive
325338 , envUTCTime = utctime
326339 , envOpts = opts
340+ , envContext = context
327341 , envPresentationSize = presSize
328342 , envSlideIdMap = makeSlideIdMap pres
329343 , envSpeakerNotesIdMap = makeSpeakerNotesMap pres
@@ -474,15 +488,16 @@ registerLink link = do
474488 linkReg <- gets stLinkIds
475489 mediaReg <- gets stMediaIds
476490 hasSpeakerNotes <- curSlideHasSpeakerNotes
477− let maxLinkId = case M.lookup curSlideId linkReg of
478− Just mp -> case M.keys mp of
479− [] -> if hasSpeakerNotes then 2 else 1
480− ks -> maximum ks
481− Nothing -> if hasSpeakerNotes then 2 else 1
482− maxMediaId = case M.lookup curSlideId mediaReg of
483− Just [] -> if hasSpeakerNotes then 2 else 1
484− Just mInfos -> maximum $ map mInfoLocalId mInfos
485− Nothing -> if hasSpeakerNotes then 2 else 1
491+ let maxLinkId = case M.lookup curSlideId linkReg >>= nonEmpty . M.keys of
492+ Just xs -> maximum xs
493+ Nothing
494+ | hasSpeakerNotes -> 2
495+ | otherwise -> 1
496+ maxMediaId = case M.lookup curSlideId mediaReg >>= nonEmpty of
497+ Just mInfos -> maximum $ fmap mInfoLocalId mInfos
498+ Nothing
499+ | hasSpeakerNotes -> 2
500+ | otherwise -> 1
486501 maxId = max maxLinkId maxMediaId
487502 slideLinks = case M.lookup curSlideId linkReg of
488503 Just mp -> M.insert (maxId + 1) link mp
@@ -497,20 +512,19 @@ registerMedia fp caption = do
497512 mediaReg <- gets stMediaIds
498513 globalIds <- gets stMediaGlobalIds
499514 hasSpeakerNotes <- curSlideHasSpeakerNotes
500− let maxLinkId = case M.lookup curSlideId linkReg of
501− Just mp -> case M.keys mp of
502− [] -> if hasSpeakerNotes then 2 else 1
503− ks -> maximum ks
504− Nothing -> if hasSpeakerNotes then 2 else 1
505− maxMediaId = case M.lookup curSlideId mediaReg of
506− Just [] -> if hasSpeakerNotes then 2 else 1
507− Just mInfos -> maximum $ map mInfoLocalId mInfos
508− Nothing -> if hasSpeakerNotes then 2 else 1
515+ let maxLinkId = case M.lookup curSlideId linkReg >>= nonEmpty . M.keys of
516+ Just ks -> maximum ks
517+ Nothing
518+ | hasSpeakerNotes -> 2
519+ | otherwise -> 1
520+ maxMediaId = case M.lookup curSlideId mediaReg >>= nonEmpty of
521+ Just mInfos -> maximum $ fmap mInfoLocalId mInfos
522+ Nothing
523+ | hasSpeakerNotes -> 2
524+ | otherwise -> 1
509525 maxLocalId = max maxLinkId maxMediaId
510526
511− maxGlobalId = case M.elems globalIds of
512− [] -> 0
513− ids -> maximum ids
527+ maxGlobalId = maybe 0 maximum $ nonEmpty $ M.elems globalIds
514528
515529 (imgBytes, mbMt) <- P.fetchItem $ T.pack fp
516530 let imgExt = (mbMt >>= extensionFromMimeType >>= (\x -> return $ "." <> x))
@@ -523,6 +537,7 @@ registerMedia fp caption = do
523537 Just Eps -> Just ".eps"
524538 Just Svg -> Just ".svg"
525539 Just Emf -> Just ".emf"
540+ Just Tiff -> Just ".tiff"
526541 Nothing -> Nothing
527542
528543 let newGlobalId = fromMaybe (maxGlobalId + 1) (M.lookup fp globalIds)
@@ -1427,11 +1442,8 @@ presentationToRels pres@(Presentation _ slides) = do
14271442 -- all relWithoutSlide rels (unless they're 1)
14281443 -- 3. If we have a notesmaster slide, we make space for that as well.
14291444
1430− let minRelNotOne = case filter (1<) $ map relId relsWeKeep of
1431− [] -> 0 -- doesn't matter in this case, since
1432− -- there will be nothing to map the
1433− -- function over
1434− l -> minimum l
1445+ let minRelNotOne = maybe 0 minimum $ nonEmpty
1446+ $ filter (1 <) $ map relId relsWeKeep
14351447
14361448 modifyRelNum :: Int -> Int
14371449 modifyRelNum 1 = 1
src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+2−3
@@ -45,6 +45,7 @@ module Text.Pandoc.Writers.Powerpoint.Presentation ( documentToPresentation
4545import Control.Monad.Reader
4646import Control.Monad.State
4747import Data.List (intercalate)
48+import Data.List.NonEmpty (nonEmpty)
4849import Data.Default
4950import Text.Pandoc.Definition
5051import Text.Pandoc.ImageSize
@@ -363,9 +364,7 @@ inlineToParElems (Note blks) = do
363364 then return []
364365 else do
365366 notes <- gets stNoteIds
366− let maxNoteId = case M.keys notes of
367− [] -> 0
368− lst -> maximum lst
367+ let maxNoteId = maybe 0 maximum $ nonEmpty $ M.keys notes
369368 curNoteId = maxNoteId + 1
370369 modify $ \st -> st { stNoteIds = M.insert curNoteId blks notes }
371370 local (\env -> env{envRunProps = (envRunProps env){rLink = Just $ InternalTarget endNotesSlideId}}) $
src/Text/Pandoc/Writers/RST.hs+14−10
@@ -16,7 +16,8 @@ reStructuredText: <http://docutils.sourceforge.net/rst.html>
1616module Text.Pandoc.Writers.RST ( writeRST, flatten ) where
1717import Control.Monad.State.Strict
1818import Data.Char (isSpace)
19−import Data.List (transpose, intersperse)
19+import Data.List (transpose, intersperse, foldl')
20+import qualified Data.List.NonEmpty as NE
2021import Data.Maybe (fromMaybe)
2122import qualified Data.Text as T
2223import Data.Text (Text)
@@ -335,7 +336,7 @@ blockToRST (OrderedList (start, style', delim) items) = do
335336 then replicate (length items) "#."
336337 else take (length items) $ orderedListMarkers
337338 (start, style', delim)
338− let maxMarkerLength = maximum $ map T.length markers
339+ let maxMarkerLength = maybe 0 maximum $ NE.nonEmpty $ map T.length markers
339340 let markers' = map (\m -> let s = maxMarkerLength - T.length m
340341 in m <> T.replicate s " ") markers
341342 contents <- zipWithM orderedListItemToRST markers' items
@@ -509,7 +510,7 @@ flatten outer
509510 | null contents = [outer]
510511 | otherwise = combineAll contents
511512 where contents = dropInlineParent outer
512− combineAll = foldl combine []
513+ combineAll = foldl' combine []
513514
514515 combine :: [Inline] -> Inline -> [Inline]
515516 combine f i =
@@ -537,11 +538,15 @@ flatten outer
537538 collapse f i = appendToLast f $ dropInlineParent i
538539
539540 appendToLast :: [Inline] -> [Inline] -> [Inline]
540− appendToLast [] toAppend = [setInlineChildren outer toAppend]
541− appendToLast flattened toAppend
542− | isOuter lastFlat = init flattened <> [appendTo lastFlat toAppend]
543− | otherwise = flattened <> [setInlineChildren outer toAppend]
544− where lastFlat = last flattened
541+ appendToLast flattened toAppend =
542+ case NE.nonEmpty flattened of
543+ Nothing -> [setInlineChildren outer toAppend]
544+ Just xs ->
545+ if isOuter lastFlat
546+ then NE.init xs <> [appendTo lastFlat toAppend]
547+ else flattened <> [setInlineChildren outer toAppend]
548+ where
549+ lastFlat = NE.last xs
545550 appendTo o i = mapNested (<> i) o
546551 isOuter i = emptyParent i == emptyParent outer
547552 emptyParent i = setInlineChildren i []
@@ -761,8 +766,7 @@ simpleTable opts blocksToDoc headers rows = do
761766 then return []
762767 else fixEmpties <$> mapM (blocksToDoc opts) headers
763768 rowDocs <- mapM (fmap fixEmpties . mapM (blocksToDoc opts)) rows
764− let numChars [] = 0
765− numChars xs = maximum . map offset $ xs
769+ let numChars = maybe 0 maximum . NE.nonEmpty . map offset
766770 let colWidths = map numChars $ transpose (headerDocs : rowDocs)
767771 let toRow = mconcat . intersperse (lblock 1 " ") . zipWith lblock colWidths
768772 let hline = nowrap $ hsep (map (\n -> literal (T.replicate n "=")) colWidths)
src/Text/Pandoc/Writers/RTF.hs+16−10
@@ -16,7 +16,7 @@ module Text.Pandoc.Writers.RTF ( writeRTF
1616import Control.Monad.Except (catchError, throwError)
1717import Control.Monad
1818import qualified Data.ByteString as B
19−import Data.Char (chr, isDigit, ord)
19+import Data.Char (chr, isDigit, ord, isAlphaNum)
2020import qualified Data.Map as M
2121import Data.Text (Text)
2222import qualified Data.Text as T
@@ -137,15 +137,21 @@ handleUnicode = T.concatMap $ \c ->
137137
138138-- | Escape special characters.
139139escapeSpecial :: Text -> Text
140−escapeSpecial = escapeStringUsing $
141− [ ('\t',"\\tab ")
142− , ('\8216',"\\u8216'")
143− , ('\8217',"\\u8217'")
144− , ('\8220',"\\u8220\"")
145− , ('\8221',"\\u8221\"")
146− , ('\8211',"\\u8211-")
147− , ('\8212',"\\u8212-")
148− ] <> backslashEscapes "{\\}"
140+escapeSpecial t
141+ | T.all isAlphaNum t = t
142+ | otherwise = T.concatMap escChar t
143+ where
144+ escChar '\t' = "\\tab "
145+ escChar '\8216' = "\\u8216'"
146+ escChar '\8217' = "\\u8217'"
147+ escChar '\8220' = "\\u8220\""
148+ escChar '\8221' = "\\u8221\""
149+ escChar '\8211' = "\\u8211-"
150+ escChar '\8212' = "\\u8212-"
151+ escChar '{' = "\\{"
152+ escChar '}' = "\\}"
153+ escChar '\\' = "\\\\"
154+ escChar c = T.singleton c
149155
150156-- | Escape strings as needed for rich text format.
151157stringToRTF :: Text -> Text
src/Text/Pandoc/Writers/Shared.hs+18−3
@@ -20,6 +20,7 @@ module Text.Pandoc.Writers.Shared (
2020 , setField
2121 , resetField
2222 , defField
23+ , getLang
2324 , tagWithAttrs
2425 , isDisplayMath
2526 , fixDisplayMath
@@ -44,6 +45,7 @@ import Control.Monad (zipWithM)
4445import Data.Aeson (ToJSON (..), encode)
4546import Data.Char (chr, ord, isSpace)
4647import Data.List (groupBy, intersperse, transpose, foldl')
48+import Data.List.NonEmpty (NonEmpty(..), nonEmpty)
4749import Data.Text.Conversions (FromText(..))
4850import qualified Data.Map as M
4951import qualified Data.Text as T
@@ -146,6 +148,19 @@ defField field val (Context m) =
146148 where
147149 f _newval oldval = oldval
148150
151+-- | Get the contents of the `lang` metadata field or variable.
152+getLang :: WriterOptions -> Meta -> Maybe T.Text
153+getLang opts meta =
154+ case lookupContext "lang" (writerVariables opts) of
155+ Just s -> Just s
156+ _ ->
157+ case lookupMeta "lang" meta of
158+ Just (MetaBlocks [Para [Str s]]) -> Just s
159+ Just (MetaBlocks [Plain [Str s]]) -> Just s
160+ Just (MetaInlines [Str s]) -> Just s
161+ Just (MetaString s) -> Just s
162+ _ -> Nothing
163+
149164-- | Produce an HTML tag with the given pandoc attributes.
150165tagWithAttrs :: HasChars a => T.Text -> Attr -> Doc a
151166tagWithAttrs tag (ident,classes,kvs) = hsep
@@ -224,7 +239,7 @@ gridTable :: (Monad m, HasChars a)
224239 -> m (Doc a)
225240gridTable opts blocksToDoc headless aligns widths headers rows = do
226241 -- the number of columns will be used in case of even widths
227− let numcols = maximum (length aligns : length widths :
242+ let numcols = maximum (length aligns :| length widths :
228243 map length (headers:rows))
229244 let officialWidthsInChars widths' = map (
230245 (\x -> if x < 1 then 1 else x) .
@@ -253,8 +268,7 @@ gridTable opts blocksToDoc headless aligns widths headers rows = do
253268 let handleFullWidths widths' = do
254269 rawHeaders' <- mapM (blocksToDoc opts) headers
255270 rawRows' <- mapM (mapM (blocksToDoc opts)) rows
256− let numChars [] = 0
257− numChars xs = maximum . map offset $ xs
271+ let numChars = maybe 0 maximum . nonEmpty . map offset
258272 let minWidthsInChars =
259273 map numChars $ transpose (rawHeaders' : rawRows')
260274 let widthsInChars' = zipWith max
@@ -380,6 +394,7 @@ toSuperscript '2' = Just '\x00B2'
380394toSuperscript '3' = Just '\x00B3'
381395toSuperscript '+' = Just '\x207A'
382396toSuperscript '-' = Just '\x207B'
397+toSuperscript '\x2212' = Just '\x207B' -- unicode minus
383398toSuperscript '=' = Just '\x207C'
384399toSuperscript '(' = Just '\x207D'
385400toSuperscript ')' = Just '\x207E'
src/Text/Pandoc/Writers/TEI.hs+4−3
@@ -146,16 +146,17 @@ blockToTEI opts (LineBlock lns) =
146146 blockToTEI opts $ linesToPara lns
147147blockToTEI opts (BlockQuote blocks) =
148148 inTagsIndented "quote" <$> blocksToTEI opts blocks
149−blockToTEI _ (CodeBlock (_,classes,_) str) =
149+blockToTEI opts (CodeBlock (_,classes,_) str) =
150150 return $ literal ("<ab type='codeblock " <> lang <> "'>") <> cr <>
151151 flush (literal (escapeStringForXML str) <> cr <> text "</ab>")
152152 where lang = if null langs
153153 then ""
154154 else escapeStringForXML (head langs)
155− isLang l = T.toLower l `elem` map T.toLower languages
155+ syntaxMap = writerSyntaxMap opts
156+ isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
156157 langsFrom s = if isLang s
157158 then [s]
158− else languagesByExtension . T.toLower $ s
159+ else (languagesByExtension syntaxMap) . T.toLower $ s
159160 langs = concatMap langsFrom classes
160161blockToTEI opts (BulletList lst) = do
161162 let attribs = [("type", "unordered")]
src/Text/Pandoc/Writers/Texinfo.hs+22−15
@@ -14,8 +14,9 @@ Conversion of 'Pandoc' format into Texinfo.
1414module Text.Pandoc.Writers.Texinfo ( writeTexinfo ) where
1515import Control.Monad.Except (throwError)
1616import Control.Monad.State.Strict
17−import Data.Char (chr, ord)
18−import Data.List (maximumBy, transpose)
17+import Data.Char (chr, ord, isAlphaNum)
18+import Data.List (maximumBy, transpose, foldl')
19+import Data.List.NonEmpty (nonEmpty)
1920import Data.Ord (comparing)
2021import qualified Data.Set as Set
2122import Data.Text (Text)
@@ -84,16 +85,18 @@ pandocToTexinfo options (Pandoc meta blocks) = do
8485
8586-- | Escape things as needed for Texinfo.
8687stringToTexinfo :: Text -> Text
87−stringToTexinfo = escapeStringUsing texinfoEscapes
88− where texinfoEscapes = [ ('{', "@{")
89− , ('}', "@}")
90− , ('@', "@@")
91− , ('\160', "@ ")
92− , ('\x2014', "---")
93− , ('\x2013', "--")
94− , ('\x2026', "@dots{}")
95− , ('\x2019', "'")
96− ]
88+stringToTexinfo t
89+ | T.all isAlphaNum t = t
90+ | otherwise = T.concatMap escChar t
91+ where escChar '{' = "@{"
92+ escChar '}' = "@}"
93+ escChar '@' = "@@"
94+ escChar '\160' = "@ "
95+ escChar '\x2014' = "---"
96+ escChar '\x2013' = "--"
97+ escChar '\x2026' = "@dots{}"
98+ escChar '\x2019' = "'"
99+ escChar c = T.singleton c
97100
98101escapeCommas :: PandocMonad m => TI m (Doc Text) -> TI m (Doc Text)
99102escapeCommas parser = do
@@ -238,9 +241,13 @@ blockToTexinfo (Table _ blkCapt specs thead tbody tfoot) = do
238241 colDescriptors <-
239242 if all (== 0) widths
240243 then do -- use longest entry instead of column widths
241− cols <- mapM (mapM (liftM (T.unpack . render Nothing . hcat) . mapM blockToTexinfo)) $
244+ cols <- mapM (mapM (fmap (T.unpack . render Nothing . hcat) .
245+ mapM blockToTexinfo)) $
242246 transpose $ heads : rows
243− return $ concatMap ((\x -> "{"++x++"} ") . maximumBy (comparing length)) cols
247+ return $ concatMap
248+ ((\x -> "{"++x++"} ") .
249+ maybe "" (maximumBy (comparing length)) . nonEmpty)
250+ cols
244251 else return $ "@columnfractions " ++ concatMap (printf "%.2f ") widths
245252 let tableBody = text ("@multitable " ++ colDescriptors) $$
246253 headers $$
@@ -271,7 +278,7 @@ tableAnyRowToTexinfo :: PandocMonad m
271278 -> [[Block]]
272279 -> TI m (Doc Text)
273280tableAnyRowToTexinfo itemtype aligns cols =
274− (literal itemtype $$) . foldl (\row item -> row $$
281+ (literal itemtype $$) . foldl' (\row item -> row $$
275282 (if isEmpty row then empty else text " @tab ") <> item) empty <$> zipWithM alignedBlock aligns cols
276283
277284alignedBlock :: PandocMonad m
src/Text/Pandoc/Writers/ZimWiki.hs+4−2
@@ -20,6 +20,7 @@ import Control.Monad (zipWithM)
2020import Control.Monad.State.Strict (StateT, evalStateT, gets, modify)
2121import Data.Default (Default (..))
2222import Data.List (transpose)
23+import Data.List.NonEmpty (nonEmpty)
2324import qualified Data.Map as Map
2425import Text.DocLayout (render, literal)
2526import Data.Maybe (fromMaybe)
@@ -115,7 +116,7 @@ blockToZimWiki opts b@(RawBlock f str)
115116blockToZimWiki _ HorizontalRule = return "\n----\n"
116117
117118blockToZimWiki opts (Header level _ inlines) = do
118− contents <- inlineListToZimWiki opts $ removeFormatting inlines -- emphasis, links etc. not allowed in headers
119+ contents <- inlineListToZimWiki opts inlines
119120 let eqs = T.replicate ( 7 - level ) "="
120121 return $ eqs <> " " <> contents <> " " <> eqs <> "\n"
121122
@@ -143,7 +144,8 @@ blockToZimWiki opts (Table _ blkCapt specs thead tbody tfoot) = do
143144 then zipWithM (tableItemToZimWiki opts) aligns (head rows)
144145 else mapM (inlineListToZimWiki opts . removeFormatting)headers -- emphasis, links etc. are not allowed in table headers
145146 rows' <- mapM (zipWithM (tableItemToZimWiki opts) aligns) rows
146− let widths = map (maximum . map T.length) $ transpose (headers':rows')
147+ let widths = map (maybe 0 maximum . nonEmpty . map T.length) $
148+ transpose (headers':rows')
147149 let padTo (width, al) s =
148150 case width - T.length s of
149151 x | x > 0 ->
src/Text/Pandoc/XML.hs+28−2
@@ -13,6 +13,7 @@ Functions for escaping and formatting XML.
1313-}
1414module Text.Pandoc.XML ( escapeCharForXML,
1515 escapeStringForXML,
16+ escapeNCName,
1617 inTags,
1718 selfClosingTag,
1819 inTagsSimple,
@@ -24,7 +25,7 @@ module Text.Pandoc.XML ( escapeCharForXML,
2425 html5Attributes,
2526 rdfaAttributes ) where
2627
27−import Data.Char (isAscii, isSpace, ord)
28+import Data.Char (isAscii, isSpace, ord, isLetter, isDigit)
2829import Data.Text (Text)
2930import qualified Data.Text as T
3031import Text.HTML.TagSoup.Entity (lookupEntity, htmlEntities)
@@ -119,8 +120,33 @@ html5EntityMap = foldr go mempty htmlEntities
119120 where ent' = T.takeWhile (/=';') (T.pack ent)
120121 _ -> entmap
121122
123+-- | Converts a string into an NCName, i.e., an XML name without colons.
124+-- Disallowed characters are escaped using @ux%x@, where @%x@ is the
125+-- hexadecimal unicode identifier of the escaped character.
126+escapeNCName :: Text -> Text
127+escapeNCName t = case T.uncons t of
128+ Nothing -> T.empty
129+ Just (c, cs) -> escapeStartChar c <> T.concatMap escapeNCNameChar cs
130+ where
131+ escapeStartChar :: Char -> Text
132+ escapeStartChar c = if isLetter c || c == '_'
133+ then T.singleton c
134+ else escapeChar c
122135
123−-- Unescapes XML entities
136+ escapeNCNameChar :: Char -> Text
137+ escapeNCNameChar c = if isNCNameChar c
138+ then T.singleton c
139+ else escapeChar c
140+
141+ isNCNameChar :: Char -> Bool
142+ isNCNameChar c = isLetter c || c `elem` ("_-.·" :: String) || isDigit c
143+ || '\x0300' <= c && c <= '\x036f'
144+ || '\x203f' <= c && c <= '\x2040'
145+
146+ escapeChar :: Char -> Text
147+ escapeChar = T.pack . printf "U%04X" . ord
148+
149+-- | Unescapes XML entities
124150fromEntities :: Text -> Text
125151fromEntities t
126152 = let (x, y) = T.break (== '&') t
src/Text/Pandoc/XML/Light/Proc.hs−1
@@ -1,5 +1,4 @@
11{-# LANGUAGE FlexibleInstances #-}
2−{-# LANGUAGE OverloadedStrings #-}
32{- |
43 Module : Text.Pandoc.XML.Light.Proc
54 Copyright : Copyright (C) 2007 Galois, Inc., 2021 John MacFarlane
src/Text/Pandoc/XML/Light/Types.hs+4−4
@@ -53,7 +53,7 @@ type Line = Integer
5353data Content = Elem Element
5454 | Text CData
5555 | CRef Text
56− deriving (Show, Typeable, Data)
56+ deriving (Show, Typeable, Data, Ord, Eq)
5757
5858-- | XML elements
5959data Element = Element {
@@ -61,7 +61,7 @@ data Element = Element {
6161 elAttribs :: [Attr],
6262 elContent :: [Content],
6363 elLine :: Maybe Line
64− } deriving (Show, Typeable, Data)
64+ } deriving (Show, Typeable, Data, Ord, Eq)
6565
6666-- | XML attributes
6767data Attr = Attr {
@@ -74,13 +74,13 @@ data CData = CData {
7474 cdVerbatim :: CDataKind,
7575 cdData :: Text,
7676 cdLine :: Maybe Line
77− } deriving (Show, Typeable, Data)
77+ } deriving (Show, Typeable, Data, Ord, Eq)
7878
7979data CDataKind
8080 = CDataText -- ^ Ordinary character data; pretty printer escapes &, < etc.
8181 | CDataVerbatim -- ^ Unescaped character data; pretty printer embeds it in <![CDATA[..
8282 | CDataRaw -- ^ As-is character data; pretty printer passes it along without any escaping or CDATA wrap-up.
83− deriving ( Eq, Show, Typeable, Data )
83+ deriving ( Eq, Ord, Show, Typeable, Data )
8484
8585-- | XML qualified names
8686data QName = QName {
stack.yaml+11−6
@@ -2,16 +2,21 @@ flags:
22 pandoc:
33 trypandoc: false
44 embed_data_files: true
5+ QuickCheck:
6+ old-random: false
57packages:
68- '.'
79extra-deps:
8−- hslua-1.3.0
9−- hslua-module-path-0.1.0
10−#- citeproc:
11−# git: https://github.com/jgm/citeproc.git
12−# commit: feb3b7580c6738eec3b23921f7c1739cfba611aa
10+- commonmark-0.2.1
11+- commonmark-pandoc-0.2.1.1
12+- skylighting-core-0.10.5.2
13+- skylighting-0.10.5.2
14+- git: https://github.com/jgm/doctemplates
15+ commit: 428c26d5303cf7a2b1051fe1ffd9aafe9ba71c81
16+- git: https://github.com/jgm/citeproc
17+ commit: 4ce0501cd6f9c86eee087afcf878c256e49b9615
1318ghc-options:
1419 "$locals": -fhide-source-paths -Wno-missing-home-modules
15−resolver: lts-17.5
20+resolver: lts-18.0
1621nix:
1722 packages: [zlib]
test/Tests/Command.hs+7−14
@@ -1,4 +1,3 @@
1−{-# LANGUAGE TupleSections #-}
21{- |
32 Module : Tests.Command
43 Copyright : © 2006-2021 John MacFarlane
@@ -14,14 +13,13 @@ module Tests.Command (runTest, tests)
1413where
1514
1615import Data.Algorithm.Diff
17−import System.Environment.Executable (getExecutablePath)
16+import System.Environment (getExecutablePath)
1817import qualified Data.ByteString as BS
1918import qualified Data.Text as T
2019import Data.List (isSuffixOf)
2120import System.Directory
22−import qualified System.Environment as Env
2321import System.Exit
24−import System.FilePath (takeDirectory, (</>))
22+import System.FilePath ((</>))
2523import System.IO (hPutStr, stderr)
2624import System.IO.Unsafe (unsafePerformIO)
2725import System.Process
@@ -38,15 +36,7 @@ execTest :: String -- ^ Path to test executable
3836 -> String -- ^ Input text
3937 -> IO (ExitCode, String) -- ^ Exit code and actual output
4038execTest testExePath cmd inp = do
41− mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
42− mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
43− let env' = ("PATH",takeDirectory testExePath) :
44− ("TMP",".") :
45− ("LANG","en_US.UTF-8") :
46− ("HOME", "./") :
47− ("pandoc_datadir", "..") :
48− maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
49− maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
39+ env' <- setupEnvironment testExePath
5040 let pr = (shell (pandocToEmulate True cmd)){ env = Just env' }
5141 (ec, out', err') <- readCreateProcessWithExitCode pr inp
5242 -- filter \r so the tests will work on Windows machines
@@ -140,4 +130,7 @@ extractCommandTest testExePath fp = unsafePerformIO $ do
140130 def{ readerExtensions = pandocExtensions } contents)
141131 let codeblocks = map extractCode $ filter isCodeBlock blocks
142132 let cases = zipWith (runCommandTest testExePath fp) [1..] codeblocks
143− return $ testGroup fp cases
133+ return $ testGroup fp
134+ $ if null cases
135+ then [testCase "!!" $ assertFailure "No command tests defined"]
136+ else cases
test/Tests/Helpers.hs+23
@@ -1,4 +1,5 @@
11{-# LANGUAGE CPP #-}
2+{-# LANGUAGE TupleSections #-}
23{-# LANGUAGE FlexibleInstances #-}
34{- |
45 Module : Tests.Helpers
@@ -13,6 +14,7 @@ Utility functions for the test suite.
1314-}
1415module Tests.Helpers ( test
1516 , TestResult(..)
17+ , setupEnvironment
1618 , showDiff
1719 , (=?>)
1820 , purely
@@ -25,6 +27,8 @@ import Data.Algorithm.Diff
2527import qualified Data.Map as M
2628import Data.Text (Text, unpack)
2729import System.Exit
30+import System.FilePath (takeDirectory)
31+import qualified System.Environment as Env
2832import Test.Tasty
2933import Test.Tasty.HUnit
3034import Text.Pandoc.Builder (Blocks, Inlines, doc, plain)
@@ -57,6 +61,25 @@ test fn name (input, expected) =
5761 dashes "" = replicate 72 '-'
5862 dashes x = replicate (72 - length x - 5) '-' ++ " " ++ x ++ " ---"
5963
64+-- | Set up environment for pandoc command tests.
65+setupEnvironment :: FilePath -> IO [(String, String)]
66+setupEnvironment testExePath = do
67+ mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
68+ mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
69+ mpdd <- Env.lookupEnv "pandoc_datadir"
70+ -- Note that Cabal sets the pandoc_datadir environment variable
71+ -- to point to the source directory, since otherwise getDataFilename
72+ -- will look in the data directory into which pandoc will be installed
73+ -- (but has not yet been). So when we spawn a new process with
74+ -- pandoc, we need to make sure this environment variable is set.
75+ return $ ("PATH",takeDirectory testExePath) :
76+ ("TMP",".") :
77+ ("LANG","en_US.UTF-8") :
78+ ("HOME", "./") :
79+ maybe [] ((:[]) . ("pandoc_datadir",)) mpdd ++
80+ maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
81+ maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
82+
6083data TestResult = TestPassed
6184 | TestError ExitCode
6285 | TestFailed String FilePath [Diff String]
test/Tests/Old.hs+3−12
@@ -1,4 +1,3 @@
1−{-# LANGUAGE TupleSections #-}
21{- |
32 Module : Tests.Old
43 Copyright : © 2006-2021 John MacFarlane
@@ -15,8 +14,7 @@ module Tests.Old (tests) where
1514import Data.Algorithm.Diff
1615import System.Exit
1716import System.FilePath ((<.>), (</>))
18−import qualified System.Environment as Env
19−import System.Environment.Executable (getExecutablePath)
17+import System.Environment (getExecutablePath)
2018import Text.Pandoc.Process (pipeProcess)
2119import Test.Tasty (TestTree, testGroup)
2220import Test.Tasty.Golden.Advanced (goldenTest)
@@ -319,14 +317,7 @@ testWithNormalize normalizer pandocPath testname opts inp norm =
319317 (compareValues norm options) updateGolden
320318 where getExpected = normalizer <$> readFile' norm
321319 getActual = do
322− mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
323− mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
324− let env = ("TMP",".") :
325− ("LANG","en_US.UTF-8") :
326− ("HOME", "./") :
327− maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
328− maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
329−
320+ env <- setupEnvironment pandocPath
330321 (ec, out) <- pipeProcess (Just env) pandocPath
331322 ("--emulate":options) mempty
332323 if ec == ExitSuccess
@@ -335,7 +326,7 @@ testWithNormalize normalizer pandocPath testname opts inp norm =
335326 -- filter \r so the tests will work on Windows machines
336327 else fail $ "Pandoc failed with error code " ++ show ec
337328 updateGolden = UTF8.writeFile norm . T.pack
338− options = ["--data-dir=../data","--quiet"] ++ [inp] ++ opts
329+ options = ["--quiet"] ++ [inp] ++ opts
339330
340331compareValues :: FilePath -> [String] -> String -> String -> IO (Maybe String)
341332compareValues norm options expected actual = do
test/Tests/Readers/Docx.hs+25−5
@@ -24,7 +24,7 @@ import Test.Tasty.HUnit
2424import Tests.Helpers
2525import Text.Pandoc
2626import qualified Text.Pandoc.Class as P
27−import Text.Pandoc.MediaBag (MediaBag, lookupMedia, mediaDirectory)
27+import qualified Text.Pandoc.MediaBag as MB
2828import Text.Pandoc.UTF8 as UTF8
2929
3030-- We define a wrapper around pandoc that doesn't normalize in the
@@ -91,11 +91,11 @@ getMedia :: FilePath -> FilePath -> IO (Maybe B.ByteString)
9191getMedia archivePath mediaPath = fmap fromEntry . findEntryByPath
9292 ("word/" ++ mediaPath) . toArchive <$> B.readFile archivePath
9393
94−compareMediaPathIO :: FilePath -> MediaBag -> FilePath -> IO Bool
94+compareMediaPathIO :: FilePath -> MB.MediaBag -> FilePath -> IO Bool
9595compareMediaPathIO mediaPath mediaBag docxPath = do
9696 docxMedia <- getMedia docxPath mediaPath
97− let mbBS = case lookupMedia mediaPath mediaBag of
98− Just (_, bs) -> bs
97+ let mbBS = case MB.lookupMedia mediaPath mediaBag of
98+ Just item -> MB.mediaContents item
9999 Nothing -> error ("couldn't find " ++
100100 mediaPath ++
101101 " in media bag")
@@ -110,7 +110,7 @@ compareMediaBagIO docxFile = do
110110 mb <- runIOorExplode $ readDocx defopts df >> P.getMediaBag
111111 bools <- mapM
112112 (\(fp, _, _) -> compareMediaPathIO fp mb docxFile)
113− (mediaDirectory mb)
113+ (MB.mediaDirectory mb)
114114 return $ and bools
115115
116116testMediaBagIO :: String -> FilePath -> IO TestTree
@@ -155,6 +155,10 @@ tests = [ testGroup "document"
155155 "VML image"
156156 "docx/image_vml.docx"
157157 "docx/image_vml.native"
158+ , testCompare
159+ "VML image as object"
160+ "docx/image_vml_as_object.docx"
161+ "docx/image_vml_as_object.native"
158162 , testCompare
159163 "inline image in links"
160164 "docx/inline_images.docx"
@@ -313,14 +317,30 @@ tests = [ testGroup "document"
313317 "tables with lists in cells"
314318 "docx/table_with_list_cell.docx"
315319 "docx/table_with_list_cell.native"
320+ , testCompare
321+ "a table with a header which contains rowspans greater than 1"
322+ "docx/table_header_rowspan.docx"
323+ "docx/table_header_rowspan.native"
316324 , testCompare
317325 "tables with one row"
318326 "docx/table_one_row.docx"
319327 "docx/table_one_row.native"
328+ , testCompare
329+ "tables with just one row, which is a header"
330+ "docx/table_one_header_row.docx"
331+ "docx/table_one_header_row.native"
320332 , testCompare
321333 "tables with variable width"
322334 "docx/table_variable_width.docx"
323335 "docx/table_variable_width.native"
336+ , testCompare
337+ "tables with captions which contain a Table field"
338+ "docx/table_captions_with_field.docx"
339+ "docx/table_captions_with_field.native"
340+ , testCompare
341+ "tables with captions which don't contain a Table field"
342+ "docx/table_captions_no_field.docx"
343+ "docx/table_captions_no_field.native"
324344 , testCompare
325345 "code block"
326346 "docx/codeblock.docx"
test/Tests/Readers/EPUB.hs+1−1
@@ -47,7 +47,7 @@ featuresBag = [("img/check.gif","image/gif",1340)
4747
4848-- with additional meta tag for cover in EPUB2 format
4949epub3CoverBag :: [(String, String, Int)]
50−epub3CoverBag = [("wasteland-cover.jpg","image/jpeg",103477)]
50+epub3CoverBag = [("wasteland-cover.jpg","image/jpeg", 16586)]
5151
5252epub3NoCoverBag :: [(String, String, Int)]
5353epub3NoCoverBag = [("img/check.gif","image/gif",1340)
test/Tests/Readers/HTML.hs+20−5
@@ -99,11 +99,26 @@ tests = [ testGroup "base tag"
9999 plain (codeWith ("",["sample"],[]) "Answer is 42")
100100 ]
101101 , testGroup "var"
102− [
103− test html "inline var block" $
104− "<var>result</var>" =?>
105− plain (codeWith ("",["variable"],[]) "result")
106− ]
102+ [ test html "inline var block" $
103+ "<var>result</var>" =?>
104+ plain (codeWith ("",["variable"],[]) "result")
105+ ]
106+ , testGroup "header"
107+ [ test htmlNativeDivs "<header> is parsed as a div" $
108+ "<header id=\"title\">Title</header>" =?>
109+ divWith ("title", mempty, mempty) (plain "Title")
110+ ]
111+ , testGroup "code block"
112+ [ test html "attributes in pre > code element" $
113+ "<pre><code id=\"a\" class=\"python\">\nprint('hi')\n</code></pre>"
114+ =?>
115+ codeBlockWith ("a", ["python"], []) "print('hi')"
116+
117+ , test html "attributes in pre take precendence" $
118+ "<pre id=\"c\"><code id=\"d\">\nprint('hi mom!')\n</code></pre>"
119+ =?>
120+ codeBlockWith ("c", [], []) "print('hi mom!')"
121+ ]
107122 , askOption $ \(QuickCheckTests numtests) ->
108123 testProperty "Round trip" $
109124 withMaxSuccess (if QuickCheckTests numtests == defaultValue
test/Tests/Readers/Jira.hs+14
@@ -96,6 +96,12 @@ tests =
9696 simpleTable [para "Name"] [[para "Test"]]
9797 ]
9898
99+ , testGroup "panel"
100+ [ "simple panel" =:
101+ "{panel}\nInterviewer: Jane Doe{panel}\n" =?>
102+ divWith ("", ["panel"], []) (para "Interviewer: Jane Doe")
103+ ]
104+
99105 , testGroup "inlines"
100106 [ "emphasis" =:
101107 "*quid pro quo*" =?>
@@ -161,6 +167,14 @@ tests =
161167 , "user with description" =:
162168 "[John Doe|~johndoe]" =?>
163169 para (linkWith ("", ["user-account"], []) "~johndoe" "" "John Doe")
170+
171+ , "'smart' link" =:
172+ "[x|http://example.com|smart-link]" =?>
173+ para (linkWith ("", ["smart-link"], []) "http://example.com" "" "x")
174+
175+ , "'smart' card" =:
176+ "[x|http://example.com|smart-card]" =?>
177+ para (linkWith ("", ["smart-card"], []) "http://example.com" "" "x")
164178 ]
165179
166180 , "image" =:
test/Tests/Readers/Markdown.hs+3−3
@@ -358,7 +358,7 @@ tests = [ testGroup "inline code"
358358 para (text "The value of the " <> math "x" <> text "\8217s and the systems\8217 condition.")
359359 , test markdownSmart "unclosed double quote"
360360 ("**this should \"be bold**"
361− =?> para (strong "this should \"be bold"))
361+ =?> para (strong "this should \8220be bold"))
362362 ]
363363 , testGroup "footnotes"
364364 [ "indent followed by newline and flush-left text" =:
@@ -374,8 +374,8 @@ tests = [ testGroup "inline code"
374374 , testGroup "lhs"
375375 [ test (purely $ readMarkdown def{ readerExtensions = enableExtension
376376 Ext_literate_haskell pandocExtensions })
377− "inverse bird tracks and html" $
378− "> a\n\n< b\n\n<div>\n"
377+ "inverse bird tracks and html"
378+ $ ("> a\n\n< b\n\n<div>\n" :: Text)
379379 =?> codeBlockWith ("",["haskell","literate"],[]) "a"
380380 <>
381381 codeBlockWith ("",["haskell"],[]) "b"
test/Tests/Readers/Odt.hs+1
@@ -176,6 +176,7 @@ namesOfTestsComparingToNative = [ "blockquote"
176176 , "referenceToText"
177177 , "simpleTable"
178178 , "simpleTableWithCaption"
179+ , "tab"
179180-- , "table"
180181 , "textMixedStyles"
181182 , "tableWithContents"
test/Tests/Writers/ConTeXt.hs+122−113
@@ -39,116 +39,125 @@ infix 4 =:
3939(=:) = test context
4040
4141tests :: [TestTree]
42−tests = [ testGroup "inline code"
43− [ "with '}'" =: code "}" =?> "\\mono{\\}}"
44− , "without '}'" =: code "]" =?> "\\type{]}"
45− , testProperty "code property" $ \s -> null s || '\n' `elem` s ||
46− if '{' `elem` s || '}' `elem` s
47− then context' (code $ pack s) == "\\mono{" ++
48− context' (str $ pack s) ++ "}"
49− else context' (code $ pack s) == "\\type{" ++ s ++ "}"
50− ]
51− , testGroup "headers"
52− [ "level 1" =:
53− headerWith ("my-header",[],[]) 1 "My header" =?> "\\section[title={My header},reference={my-header}]"
54− , test contextDiv "section-divs" $
55− ( headerWith ("header1", [], []) 1 (text "Header1")
56− <> headerWith ("header2", [], []) 2 (text "Header2")
57− <> headerWith ("header3", [], []) 3 (text "Header3")
58− <> headerWith ("header4", [], []) 4 (text "Header4")
59− <> headerWith ("header5", [], []) 5 (text "Header5")
60− <> headerWith ("header6", [], []) 6 (text "Header6"))
61− =?>
62− unlines [ "\\startsection[title={Header1},reference={header1}]\n"
63− , "\\startsubsection[title={Header2},reference={header2}]\n"
64− , "\\startsubsubsection[title={Header3},reference={header3}]\n"
65− , "\\startsubsubsubsection[title={Header4},reference={header4}]\n"
66− , "\\startsubsubsubsubsection[title={Header5},reference={header5}]\n"
67− , "\\startsubsubsubsubsubsection[title={Header6},reference={header6}]\n"
68− , "\\stopsubsubsubsubsubsection\n"
69− , "\\stopsubsubsubsubsection\n"
70− , "\\stopsubsubsubsection\n"
71− , "\\stopsubsubsection\n"
72− , "\\stopsubsection\n"
73− , "\\stopsection" ]
74− ]
75− , testGroup "bullet lists"
76− [ "nested" =:
77− bulletList [
78− plain (text "top")
79− <> bulletList [
80− plain (text "next")
81− <> bulletList [plain (text "bot")]
82− ]
83− ] =?> unlines
84− [ "\\startitemize[packed]"
85− , "\\item"
86− , " top"
87− , " \\startitemize[packed]"
88− , " \\item"
89− , " next"
90− , " \\startitemize[packed]"
91− , " \\item"
92− , " bot"
93− , " \\stopitemize"
94− , " \\stopitemize"
95− , "\\stopitemize" ]
96− ]
97− , testGroup "natural tables"
98− [ test contextNtb "table with header and caption" $
99− let capt = text "Table 1"
100− aligns = [(AlignRight, ColWidthDefault), (AlignLeft, ColWidthDefault), (AlignCenter, ColWidthDefault), (AlignDefault, ColWidthDefault)]
101− headers = [plain $ text "Right",
102− plain $ text "Left",
103− plain $ text "Center",
104− plain $ text "Default"]
105− rows = [[plain $ text "1.1",
106− plain $ text "1.2",
107− plain $ text "1.3",
108− plain $ text "1.4"]
109− ,[plain $ text "2.1",
110− plain $ text "2.2",
111− plain $ text "2.3",
112− plain $ text "2.4"]
113− ,[plain $ text "3.1",
114− plain $ text "3.2",
115− plain $ text "3.3",
116− plain $ text "3.4"]]
117− toRow = Row nullAttr . map simpleCell
118− in table (simpleCaption $ plain capt)
119− aligns
120− (TableHead nullAttr [toRow headers])
121− [TableBody nullAttr 0 [] $ map toRow rows]
122− (TableFoot nullAttr [])
123− =?> unlines [ "\\startplacetable[title={Table 1}]"
124− , "\\startTABLE"
125− , "\\startTABLEhead"
126− , "\\NC[align=left] Right"
127− , "\\NC[align=right] Left"
128− , "\\NC[align=middle] Center"
129− , "\\NC Default"
130− , "\\NC\\NR"
131− , "\\stopTABLEhead"
132− , "\\startTABLEbody"
133− , "\\NC[align=left] 1.1"
134− , "\\NC[align=right] 1.2"
135− , "\\NC[align=middle] 1.3"
136− , "\\NC 1.4"
137− , "\\NC\\NR"
138− , "\\NC[align=left] 2.1"
139− , "\\NC[align=right] 2.2"
140− , "\\NC[align=middle] 2.3"
141− , "\\NC 2.4"
142− , "\\NC\\NR"
143− , "\\stopTABLEbody"
144− , "\\startTABLEfoot"
145− , "\\NC[align=left] 3.1"
146− , "\\NC[align=right] 3.2"
147− , "\\NC[align=middle] 3.3"
148− , "\\NC 3.4"
149− , "\\NC\\NR"
150− , "\\stopTABLEfoot"
151− , "\\stopTABLE"
152− , "\\stopplacetable" ]
153− ]
154− ]
42+tests =
43+ [ testGroup "inline code"
44+ [ "with '}'" =: code "}" =?> "\\mono{\\}}"
45+ , "without '}'" =: code "]" =?> "\\type{]}"
46+ , "span with ID" =:
47+ spanWith ("city", [], []) "Berlin" =?>
48+ "\\reference[city]{}Berlin"
49+ , testProperty "code property" $ \s -> null s || '\n' `elem` s ||
50+ if '{' `elem` s || '}' `elem` s
51+ then context' (code $ pack s) == "\\mono{" ++
52+ context' (str $ pack s) ++ "}"
53+ else context' (code $ pack s) == "\\type{" ++ s ++ "}"
54+ ]
55+ , testGroup "headers"
56+ [ "level 1" =:
57+ headerWith ("my-header",[],[]) 1 "My header" =?>
58+ "\\section[title={My header},reference={my-header}]"
59+ , test contextDiv "section-divs" $
60+ ( headerWith ("header1", [], []) 1 (text "Header1")
61+ <> headerWith ("header2", [], []) 2 (text "Header2")
62+ <> headerWith ("header3", [], []) 3 (text "Header3")
63+ <> headerWith ("header4", [], []) 4 (text "Header4")
64+ <> headerWith ("header5", [], []) 5 (text "Header5")
65+ <> headerWith ("header6", [], []) 6 (text "Header6"))
66+ =?>
67+ unlines
68+ [ "\\startsection[title={Header1},reference={header1}]\n"
69+ , "\\startsubsection[title={Header2},reference={header2}]\n"
70+ , "\\startsubsubsection[title={Header3},reference={header3}]\n"
71+ , "\\startsubsubsubsection[title={Header4},reference={header4}]\n"
72+ , "\\startsubsubsubsubsection[title={Header5},reference={header5}]\n"
73+ , "\\startsubsubsubsubsubsection[title={Header6},reference={header6}]\n"
74+ , "\\stopsubsubsubsubsubsection\n"
75+ , "\\stopsubsubsubsubsection\n"
76+ , "\\stopsubsubsubsection\n"
77+ , "\\stopsubsubsection\n"
78+ , "\\stopsubsection\n"
79+ , "\\stopsection" ]
80+ ]
81+ , testGroup "bullet lists"
82+ [ "nested" =:
83+ bulletList [
84+ plain (text "top")
85+ <> bulletList [
86+ plain (text "next")
87+ <> bulletList [plain (text "bot")]
88+ ]
89+ ] =?> unlines
90+ [ "\\startitemize[packed]"
91+ , "\\item"
92+ , " top"
93+ , " \\startitemize[packed]"
94+ , " \\item"
95+ , " next"
96+ , " \\startitemize[packed]"
97+ , " \\item"
98+ , " bot"
99+ , " \\stopitemize"
100+ , " \\stopitemize"
101+ , "\\stopitemize" ]
102+ ]
103+ , testGroup "natural tables"
104+ [ test contextNtb "table with header and caption" $
105+ let capt = text "Table 1"
106+ aligns = [ (AlignRight, ColWidthDefault)
107+ , (AlignLeft, ColWidthDefault)
108+ , (AlignCenter, ColWidthDefault)
109+ , (AlignDefault, ColWidthDefault) ]
110+ headers = [plain $ text "Right",
111+ plain $ text "Left",
112+ plain $ text "Center",
113+ plain $ text "Default"]
114+ rows = [[plain $ text "1.1",
115+ plain $ text "1.2",
116+ plain $ text "1.3",
117+ plain $ text "1.4"]
118+ ,[plain $ text "2.1",
119+ plain $ text "2.2",
120+ plain $ text "2.3",
121+ plain $ text "2.4"]
122+ ,[plain $ text "3.1",
123+ plain $ text "3.2",
124+ plain $ text "3.3",
125+ plain $ text "3.4"]]
126+ toRow = Row nullAttr . map simpleCell
127+ in table (simpleCaption $ plain capt)
128+ aligns
129+ (TableHead nullAttr [toRow headers])
130+ [TableBody nullAttr 0 [] $ map toRow rows]
131+ (TableFoot nullAttr [])
132+ =?> unlines [ "\\startplacetable[title={Table 1}]"
133+ , "\\startTABLE"
134+ , "\\startTABLEhead"
135+ , "\\NC[align=left] Right"
136+ , "\\NC[align=right] Left"
137+ , "\\NC[align=middle] Center"
138+ , "\\NC Default"
139+ , "\\NC\\NR"
140+ , "\\stopTABLEhead"
141+ , "\\startTABLEbody"
142+ , "\\NC[align=left] 1.1"
143+ , "\\NC[align=right] 1.2"
144+ , "\\NC[align=middle] 1.3"
145+ , "\\NC 1.4"
146+ , "\\NC\\NR"
147+ , "\\NC[align=left] 2.1"
148+ , "\\NC[align=right] 2.2"
149+ , "\\NC[align=middle] 2.3"
150+ , "\\NC 2.4"
151+ , "\\NC\\NR"
152+ , "\\stopTABLEbody"
153+ , "\\startTABLEfoot"
154+ , "\\NC[align=left] 3.1"
155+ , "\\NC[align=right] 3.2"
156+ , "\\NC[align=middle] 3.3"
157+ , "\\NC 3.4"
158+ , "\\NC\\NR"
159+ , "\\stopTABLEfoot"
160+ , "\\stopTABLE"
161+ , "\\stopplacetable" ]
162+ ]
163+ ]
test/Tests/Writers/Docbook.hs+43−6
@@ -11,9 +11,14 @@ import Text.Pandoc.Builder
1111docbook :: (ToPandoc a) => a -> String
1212docbook = docbookWithOpts def{ writerWrapText = WrapNone }
1313
14+docbook5 :: (ToPandoc a) => a -> String
15+docbook5 = docbook5WithOpts def{ writerWrapText = WrapNone }
16+
1417docbookWithOpts :: ToPandoc a => WriterOptions -> a -> String
1518docbookWithOpts opts = unpack . purely (writeDocbook4 opts) . toPandoc
1619
20+docbook5WithOpts :: ToPandoc a => WriterOptions -> a -> String
21+docbook5WithOpts opts = unpack . purely (writeDocbook5 opts) . toPandoc
1722{-
1823 "my test" =: X =?> Y
1924
@@ -78,32 +83,32 @@ tests = [ testGroup "line blocks"
7883 , "</warning>"
7984 ]
8085 , "admonition-with-title" =:
81− divWith ("foo", ["attention"], []) (
86+ divWith ("foo", ["note"], []) (
8287 divWith ("foo", ["title"], [])
8388 (plain (text "This is title")) <>
8489 para "This is a test"
8590 )
8691 =?> unlines
87− [ "<attention id=\"foo\">"
92+ [ "<note id=\"foo\">"
8893 , " <title>This is title</title>"
8994 , " <para>"
9095 , " This is a test"
9196 , " </para>"
92− , "</attention>"
97+ , "</note>"
9398 ]
9499 , "admonition-with-title-in-para" =:
95− divWith ("foo", ["attention"], []) (
100+ divWith ("foo", ["note"], []) (
96101 divWith ("foo", ["title"], [])
97102 (para "This is title") <>
98103 para "This is a test"
99104 )
100105 =?> unlines
101− [ "<attention id=\"foo\">"
106+ [ "<note id=\"foo\">"
102107 , " <title>This is title</title>"
103108 , " <para>"
104109 , " This is a test"
105110 , " </para>"
106− , "</attention>"
111+ , "</note>"
107112 ]
108113 , "single-child" =:
109114 divWith ("foo", [], []) (para "This is a test")
@@ -366,4 +371,36 @@ tests = [ testGroup "line blocks"
366371 ]
367372 ]
368373 ]
374+ , testGroup "section attributes" $
375+ let
376+ headers = headerWith ("myid1",[],[("role","internal"),("xml:id","anotherid"),("dir","rtl")]) 1 "header1"
377+ <> headerWith ("myid2",[],[("invalidname","value"),("arch","linux"),("dir","invaliddir")]) 1 "header2"
378+ in
379+ [ test docbook5 "sections with attributes (db5)" $
380+ headers =?>
381+ unlines [ "<section xmlns=\"http://docbook.org/ns/docbook\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:id=\"myid1\" role=\"internal\" dir=\"rtl\">"
382+ , " <title>header1</title>"
383+ , " <para>"
384+ , " </para>"
385+ , "</section>"
386+ , "<section xmlns=\"http://docbook.org/ns/docbook\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:id=\"myid2\">"
387+ , " <title>header2</title>"
388+ , " <para>"
389+ , " </para>"
390+ , "</section>"
391+ ]
392+ , test docbook "sections with attributes (db4)" $
393+ headers =?>
394+ unlines [ "<sect1 id=\"myid1\" role=\"internal\">"
395+ , " <title>header1</title>"
396+ , " <para>"
397+ , " </para>"
398+ , "</sect1>"
399+ , "<sect1 id=\"myid2\" arch=\"linux\">"
400+ , " <title>header2</title>"
401+ , " <para>"
402+ , " </para>"
403+ , "</sect1>"
404+ ]
405+ ]
369406 ]
test/Tests/Writers/Docx.hs+5
@@ -111,6 +111,11 @@ tests = [ testGroup "inlines"
111111 def
112112 "docx/tables.native"
113113 "docx/golden/tables.docx"
114+ , docxTest
115+ "tables without explicit column widths"
116+ def
117+ "docx/tables-default-widths.native"
118+ "docx/golden/tables-default-widths.docx"
114119 , docxTest
115120 "tables with lists in cells"
116121 def
test/Tests/Writers/HTML.hs+57−52
@@ -34,55 +34,60 @@ infix 4 =:
3434(=:) = test html
3535
3636tests :: [TestTree]
37−tests = [ testGroup "inline code"
38− [ "basic" =: code "@&" =?> "<code>@&amp;</code>"
39− , "haskell" =: codeWith ("",["haskell"],[]) ">>="
40− =?> "<code class=\"sourceCode haskell\"><span class=\"op\">&gt;&gt;=</span></code>"
41− , "nolanguage" =: codeWith ("",["nolanguage"],[]) ">>="
42− =?> "<code class=\"nolanguage\">&gt;&gt;=</code>"
43− ]
44− , testGroup "images"
45− [ "alt with formatting" =:
46− image "/url" "title" ("my " <> emph "image")
47− =?> "<img src=\"/url\" title=\"title\" alt=\"my image\" />"
48− ]
49− , testGroup "blocks"
50− [ "definition list with empty <dt>" =:
51− definitionList [(mempty, [para $ text "foo bar"])]
52− =?> "<dl><dt></dt><dd><p>foo bar</p></dd></dl>"
53− ]
54− , testGroup "quotes"
55− [ "quote with cite attribute (without q-tags)" =:
56− doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
57− =?> "“<span cite=\"http://example.org\">examples</span>”"
58− , tQ "quote with cite attribute (with q-tags)" $
59− doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
60− =?> "<q cite=\"http://example.org\">examples</q>"
61− ]
62− , testGroup "sample"
63− [ "sample should be rendered correctly" =:
64− plain (codeWith ("",["sample"],[]) "Answer is 42") =?>
65− "<samp>Answer is 42</samp>"
66− ]
67− , testGroup "variable"
68− [ "variable should be rendered correctly" =:
69− plain (codeWith ("",["variable"],[]) "result") =?>
70− "<var>result</var>"
71− ]
72− , testGroup "sample with style"
73− [ "samp should wrap highlighted code" =:
74− codeWith ("",["sample","haskell"],[]) ">>="
75− =?> ("<samp><code class=\"sourceCode haskell\">" ++
76− "<span class=\"op\">&gt;&gt;=</span></code></samp>")
77− ]
78− , testGroup "variable with style"
79− [ "var should wrap highlighted code" =:
80− codeWith ("",["haskell","variable"],[]) ">>="
81− =?> ("<var><code class=\"sourceCode haskell\">" ++
82− "<span class=\"op\">&gt;&gt;=</span></code></var>")
83− ]
84− ]
85− where
86− tQ :: (ToString a, ToPandoc a)
87− => String -> (a, String) -> TestTree
88− tQ = test htmlQTags
37+tests =
38+ [ testGroup "inline code"
39+ [ "basic" =: code "@&" =?> "<code>@&amp;</code>"
40+ , "haskell" =: codeWith ("",["haskell"],[]) ">>="
41+ =?> "<code class=\"sourceCode haskell\"><span class=\"op\">&gt;&gt;=</span></code>"
42+ , "nolanguage" =: codeWith ("",["nolanguage"],[]) ">>="
43+ =?> "<code class=\"nolanguage\">&gt;&gt;=</code>"
44+ ]
45+ , testGroup "images"
46+ [ "alt with formatting" =:
47+ image "/url" "title" ("my " <> emph "image")
48+ =?> "<img src=\"/url\" title=\"title\" alt=\"my image\" />"
49+ ]
50+ , testGroup "blocks"
51+ [ "definition list with empty <dt>" =:
52+ definitionList [(mempty, [para $ text "foo bar"])]
53+ =?> "<dl><dt></dt><dd><p>foo bar</p></dd></dl>"
54+ , "heading with disallowed attributes" =:
55+ headerWith ("", [], [("invalid","1"), ("lang", "en")]) 1 "test"
56+ =?>
57+ "<h1 lang=\"en\">test</h1>"
58+ ]
59+ , testGroup "quotes"
60+ [ "quote with cite attribute (without q-tags)" =:
61+ doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
62+ =?> "“<span cite=\"http://example.org\">examples</span>”"
63+ , tQ "quote with cite attribute (with q-tags)" $
64+ doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
65+ =?> "<q cite=\"http://example.org\">examples</q>"
66+ ]
67+ , testGroup "sample"
68+ [ "sample should be rendered correctly" =:
69+ plain (codeWith ("",["sample"],[]) "Answer is 42") =?>
70+ "<samp>Answer is 42</samp>"
71+ ]
72+ , testGroup "variable"
73+ [ "variable should be rendered correctly" =:
74+ plain (codeWith ("",["variable"],[]) "result") =?>
75+ "<var>result</var>"
76+ ]
77+ , testGroup "sample with style"
78+ [ "samp should wrap highlighted code" =:
79+ codeWith ("",["sample","haskell"],[]) ">>="
80+ =?> ("<samp><code class=\"sourceCode haskell\">" ++
81+ "<span class=\"op\">&gt;&gt;=</span></code></samp>")
82+ ]
83+ , testGroup "variable with style"
84+ [ "var should wrap highlighted code" =:
85+ codeWith ("",["haskell","variable"],[]) ">>="
86+ =?> ("<var><code class=\"sourceCode haskell\">" ++
87+ "<span class=\"op\">&gt;&gt;=</span></code></var>")
88+ ]
89+ ]
90+ where
91+ tQ :: (ToString a, ToPandoc a)
92+ => String -> (a, String) -> TestTree
93+ tQ = test htmlQTags
test/Tests/Writers/JATS.hs+135−91
@@ -1,21 +1,21 @@
11{-# LANGUAGE OverloadedStrings #-}
22module Tests.Writers.JATS (tests) where
33
4−import Data.Text (unpack)
4+import Data.Text (Text)
55import Test.Tasty
66import Tests.Helpers
77import Text.Pandoc
88import Text.Pandoc.Arbitrary ()
99import Text.Pandoc.Builder
10+import qualified Data.Text as T
1011
11−jats :: (ToPandoc a) => a -> String
12−jats = unpack
13− . purely (writeJATS def{ writerWrapText = WrapNone })
14− . toPandoc
12+jats :: (ToPandoc a) => a -> Text
13+jats = purely (writeJATS def{ writerWrapText = WrapNone })
14+ . toPandoc
1515
16−jatsArticleAuthoring :: (ToPandoc a) => a -> String
17−jatsArticleAuthoring = unpack
18− . purely (writeJatsArticleAuthoring def{ writerWrapText = WrapNone })
16+jatsArticleAuthoring :: (ToPandoc a) => a -> Text
17+jatsArticleAuthoring =
18+ purely (writeJatsArticleAuthoring def{ writerWrapText = WrapNone })
1919 . toPandoc
2020
2121{-
@@ -32,89 +32,133 @@ which is in turn shorthand for
3232
3333infix 4 =:
3434(=:) :: (ToString a, ToPandoc a)
35− => String -> (a, String) -> TestTree
35+ => String -> (a, Text) -> TestTree
3636(=:) = test jats
3737
3838tests :: [TestTree]
39−tests = [ testGroup "inline code"
40− [ "basic" =: code "@&" =?> "<p><monospace>@&amp;</monospace></p>"
41− , "lang" =: codeWith ("", ["c"], []) "@&" =?> "<p><code language=\"c\">@&amp;</code></p>"
42− ]
43− , testGroup "block code"
44− [ "basic" =: codeBlock "@&" =?> "<preformat>@&amp;</preformat>"
45− , "lang" =: codeBlockWith ("", ["c"], []) "@&" =?> "<code language=\"c\">@&amp;</code>"
46− ]
47− , testGroup "images"
48− [ "basic" =:
49− image "/url" "title" mempty
50− =?> "<graphic mimetype=\"image\" mime-subtype=\"\" xlink:href=\"/url\" xlink:title=\"title\" />"
51− ]
52− , testGroup "inlines"
53− [ "Emphasis" =: emph "emphasized"
54− =?> "<p><italic>emphasized</italic></p>"
55−
56− , test jatsArticleAuthoring "footnote in articleauthoring tag set"
57− ("test" <> note (para "footnote") =?>
58− unlines [ "<p>test<fn>"
59− , " <p>footnote</p>"
60− , "</fn></p>"
61− ])
62− ]
63− , "bullet list" =: bulletList [ plain $ text "first"
64− , plain $ text "second"
65− , plain $ text "third"
66− ]
67− =?> "<list list-type=\"bullet\">\n\
68− \ <list-item>\n\
69− \ <p>first</p>\n\
70− \ </list-item>\n\
71− \ <list-item>\n\
72− \ <p>second</p>\n\
73− \ </list-item>\n\
74− \ <list-item>\n\
75− \ <p>third</p>\n\
76− \ </list-item>\n\
77− \</list>"
78− , testGroup "definition lists"
79− [ "with internal link" =: definitionList [(link "#go" "" (str "testing"),
80− [plain (text "hi there")])] =?>
81− "<def-list>\n\
82− \ <def-item>\n\
83− \ <term><xref alt=\"testing\" rid=\"go\">testing</xref></term>\n\
84− \ <def>\n\
85− \ <p>hi there</p>\n\
86− \ </def>\n\
87− \ </def-item>\n\
88− \</def-list>"
89− ]
90− , testGroup "math"
91− [ "escape |" =: para (math "\\sigma|_{\\{x\\}}") =?>
92− "<p><inline-formula><alternatives>\n\
93− \<tex-math><![CDATA[\\sigma|_{\\{x\\}}]]></tex-math>\n\
94− \<mml:math display=\"inline\" xmlns:mml=\"http://www.w3.org/1998/Math/MathML\"><mml:mrow><mml:mi>σ</mml:mi><mml:msub><mml:mo stretchy=\"false\" form=\"prefix\">|</mml:mo><mml:mrow><mml:mo stretchy=\"false\" form=\"prefix\">{</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy=\"false\" form=\"postfix\">}</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></alternatives></inline-formula></p>"
95− ]
96− , testGroup "headers"
97− [ "unnumbered header" =:
98− headerWith ("foo",["unnumbered"],[]) 1
99− (text "Header 1" <> note (plain $ text "note")) =?>
100− "<sec id=\"foo\">\n\
101− \ <title>Header 1<xref ref-type=\"fn\" rid=\"fn1\">1</xref></title>\n\
102− \</sec>"
103− , "unnumbered sub header" =:
104− headerWith ("foo",["unnumbered"],[]) 1
105− (text "Header")
106− <> headerWith ("foo",["unnumbered"],[]) 2
107− (text "Sub-Header") =?>
108− "<sec id=\"foo\">\n\
109− \ <title>Header</title>\n\
110− \ <sec id=\"foo\">\n\
111− \ <title>Sub-Header</title>\n\
112− \ </sec>\n\
113− \</sec>"
114− , "containing image" =:
115− header 1 (image "imgs/foo.jpg" "" (text "Alt text")) =?>
116− "<sec>\n\
117− \ <title><inline-graphic mimetype=\"image\" mime-subtype=\"jpeg\" xlink:href=\"imgs/foo.jpg\" /></title>\n\
118− \</sec>"
119− ]
120− ]
39+tests =
40+ [ testGroup "inline code"
41+ [ "basic" =: code "@&" =?> "<p><monospace>@&amp;</monospace></p>"
42+ , "lang" =: codeWith ("", ["c"], []) "@&" =?> "<p><code language=\"c\">@&amp;</code></p>"
43+ ]
44+ , testGroup "block code"
45+ [ "basic" =: codeBlock "@&" =?> "<preformat>@&amp;</preformat>"
46+ , "lang" =: codeBlockWith ("", ["c"], []) "@&" =?> "<code language=\"c\">@&amp;</code>"
47+ ]
48+ , testGroup "images"
49+ [ "basic" =:
50+ image "/url" "title" mempty
51+ =?> "<graphic mimetype=\"image\" mime-subtype=\"\" xlink:href=\"/url\" xlink:title=\"title\" />"
52+ ]
53+ , testGroup "inlines"
54+ [ "Emphasis" =: emph "emphasized"
55+ =?> "<p><italic>emphasized</italic></p>"
56+
57+ , test jatsArticleAuthoring "footnote in articleauthoring tag set"
58+ ("test" <> note (para "footnote") =?>
59+ unlines [ "<p>test<fn>"
60+ , " <p>footnote</p>"
61+ , "</fn></p>"
62+ ])
63+ ]
64+ , "bullet list" =: bulletList [ plain $ text "first"
65+ , plain $ text "second"
66+ , plain $ text "third"
67+ ]
68+ =?> "<list list-type=\"bullet\">\n\
69+ \ <list-item>\n\
70+ \ <p>first</p>\n\
71+ \ </list-item>\n\
72+ \ <list-item>\n\
73+ \ <p>second</p>\n\
74+ \ </list-item>\n\
75+ \ <list-item>\n\
76+ \ <p>third</p>\n\
77+ \ </list-item>\n\
78+ \</list>"
79+ , testGroup "definition lists"
80+ [ "with internal link" =: definitionList [(link "#go" "" (str "testing"),
81+ [plain (text "hi there")])] =?>
82+ "<def-list>\n\
83+ \ <def-item>\n\
84+ \ <term><xref alt=\"testing\" rid=\"go\">testing</xref></term>\n\
85+ \ <def>\n\
86+ \ <p>hi there</p>\n\
87+ \ </def>\n\
88+ \ </def-item>\n\
89+ \</def-list>"
90+ ]
91+ , testGroup "math"
92+ [ "escape |" =: para (math "\\sigma|_{\\{x\\}}") =?>
93+ "<p><inline-formula><alternatives>\n\
94+ \<tex-math><![CDATA[\\sigma|_{\\{x\\}}]]></tex-math>\n\
95+ \<mml:math display=\"inline\" xmlns:mml=\"http://www.w3.org/1998/Math/MathML\"><mml:mrow><mml:mi>σ</mml:mi><mml:msub><mml:mo stretchy=\"false\" form=\"prefix\">|</mml:mo><mml:mrow><mml:mo stretchy=\"false\" form=\"prefix\">{</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy=\"false\" form=\"postfix\">}</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></alternatives></inline-formula></p>"
96+ ]
97+ , testGroup "headers"
98+ [ "unnumbered header" =:
99+ headerWith ("foo",["unnumbered"],[]) 1
100+ (text "Header 1" <> note (plain $ text "note")) =?>
101+ "<sec id=\"foo\">\n\
102+ \ <title>Header 1<xref ref-type=\"fn\" rid=\"fn1\">1</xref></title>\n\
103+ \</sec>"
104+ , "unnumbered sub header" =:
105+ headerWith ("foo",["unnumbered"],[]) 1
106+ (text "Header")
107+ <> headerWith ("foo",["unnumbered"],[]) 2
108+ (text "Sub-Header") =?>
109+ "<sec id=\"foo\">\n\
110+ \ <title>Header</title>\n\
111+ \ <sec id=\"foo\">\n\
112+ \ <title>Sub-Header</title>\n\
113+ \ </sec>\n\
114+ \</sec>"
115+ , "containing image" =:
116+ header 1 (image "imgs/foo.jpg" "" (text "Alt text")) =?>
117+ "<sec>\n\
118+ \ <title><inline-graphic mimetype=\"image\" mime-subtype=\"jpeg\" xlink:href=\"imgs/foo.jpg\" /></title>\n\
119+ \</sec>"
120+ ]
121+
122+ , testGroup "ids"
123+ [ "non-ASCII in header ID" =:
124+ headerWith ("smørbrød",[],[]) 1 (text "smørbrød") =?>
125+ T.unlines [ "<sec id=\"smørbrød\">"
126+ , " <title>smørbrød</title>"
127+ , "</sec>"
128+ ]
129+
130+ , "disallowed symbol in header id" =:
131+ headerWith ("i/o",[],[]) 1 (text "I/O") =?>
132+ T.unlines [ "<sec id=\"iU002Fo\">"
133+ , " <title>I/O</title>"
134+ , "</sec>"
135+ ]
136+
137+ , "disallowed symbols in internal link target" =:
138+ link "#foo:bar" "" "baz" =?>
139+ "<p><xref alt=\"baz\" rid=\"fooU003Abar\">baz</xref></p>"
140+
141+ , "code id starting with a number" =:
142+ codeWith ("7y",[],[]) "print 5" =?>
143+ "<p><monospace id=\"U0037y\">print 5</monospace></p>"
144+ ]
145+
146+ , testGroup "spans"
147+ [ "unwrapped if no attributes given" =:
148+ spanWith nullAttr "text in span" =?>
149+ "<p>text in span</p>"
150+
151+ , "converted to named-content element if class given" =:
152+ spanWith ("a", ["genus-species"], [("alt", "aa")]) "C. elegans" =?>
153+ ("<p><named-content id=\"a\" alt=\"aa\" content-type=\"genus-species\">"
154+ <> "C. elegans</named-content></p>")
155+
156+ , "unwrapped if styled-content element would have no attributes" =:
157+ spanWith ("", [], [("hidden", "true")]) "text in span" =?>
158+ "<p>text in span</p>"
159+
160+ , "use content-type attribute if present" =:
161+ spanWith ("", [], [("content-type", "species")]) "E. coli" =?>
162+ "<p><named-content content-type=\"species\">E. coli</named-content></p>"
163+ ]
164+ ]
test/Tests/Writers/Jira.hs+52
@@ -61,12 +61,64 @@ tests =
6161 , "user link with user as description" =:
6262 linkWith ("", ["user-account"], []) "~johndoe" "" "~johndoe" =?>
6363 "[~johndoe]"
64+
65+ , "'smart' link" =:
66+ para (linkWith ("", ["smart-link"], []) "http://example.com" "" "x") =?>
67+ "[x|http://example.com|smart-link]"
68+
69+ , "'smart' card" =:
70+ para (linkWith ("", ["smart-card"], []) "http://example.org" "" "x") =?>
71+ "[x|http://example.org|smart-card]"
6472 ]
6573
6674 , testGroup "spans"
6775 [ "id is used as anchor" =:
6876 spanWith ("unicorn", [], []) (str "Unicorn") =?>
6977 "{anchor:unicorn}Unicorn"
78+
79+ , "use `color` attribute" =:
80+ spanWith ("",[],[("color","red")]) "ruby" =?>
81+ "{color:red}ruby{color}"
82+ ]
83+
84+ , testGroup "code"
85+ [ "code block with known language" =:
86+ codeBlockWith ("", ["java"], []) "Book book = new Book(\"Algebra\")" =?>
87+ "{code:java}\nBook book = new Book(\"Algebra\")\n{code}"
88+
89+ , "code block without language" =:
90+ codeBlockWith ("", [], []) "preformatted\n text.\n" =?>
91+ "{noformat}\npreformatted\n text.\n{noformat}"
92+ ]
93+ ]
94+
95+ , testGroup "blocks"
96+ [ testGroup "div"
97+ [ "empty attributes" =:
98+ divWith nullAttr (para "interesting text") =?>
99+ "interesting text"
100+
101+ , "just identifier" =:
102+ divWith ("a", [], []) (para "interesting text") =?>
103+ "{anchor:a}interesting text"
104+
105+ , "with class 'panel'" =:
106+ divWith ("", ["panel"], []) (para "Contents!") =?>
107+ "{panel}\nContents\\!\n{panel}\n"
108+
109+ , "panel with id" =:
110+ divWith ("b", ["panel"], []) (para "text") =?>
111+ "{panel}\n{anchor:b}text\n{panel}\n"
112+
113+ , "title attribute" =:
114+ divWith ("", [], [("title", "Gimme!")]) (para "Contents!") =?>
115+ "{panel:title=Gimme!}\nContents\\!\n{panel}\n"
116+
117+ , "nested panels" =:
118+ let panelAttr = ("", ["panel"], [])
119+ in divWith panelAttr (para "hi" <>
120+ divWith panelAttr (para "wassup?")) =?>
121+ "{panel}\nhi\n\nwassup?\n{panel}\n"
70122 ]
71123 ]
72124 ]
test/Tests/Writers/OOXML.hs+3−1
@@ -55,7 +55,9 @@ testArchive :: (WriterOptions -> Pandoc -> PandocIO BL.ByteString)
5555 -> IO Archive
5656testArchive writerFn opts fp = do
5757 txt <- T.readFile fp
58− bs <- runIOorExplode $ readNative def txt >>= writerFn opts
58+ bs <- runIOorExplode $ do
59+ setTranslations "en-US"
60+ readNative def txt >>= writerFn opts
5961 return $ toArchive bs
6062
6163compareFileList :: FilePath -> Archive -> Archive -> Maybe String
test/command/1279.md+1−1
@@ -9,7 +9,7 @@ title: My Article
99[^1]: Dept. of This and That
1010^D
1111---
12−author: "John Doe[^1]"
12+author: John Doe[^1]
1313date: 2014
1414title: My Article
1515---
test/command/2103.mdnew file+8
@@ -0,0 +1,8 @@
1+```
2+pandoc -t latex
3+| A happy pandoc user said "fix this bug please
4+| or I'll go crazy!"
5+^D
6+A happy pandoc user said ``fix this bug please\\
7+or I'll go crazy!''
8+```
test/command/2874.md+2−2
@@ -3,12 +3,12 @@
33<a></a>
44<br/>
55^D
6−{}~\\
6+{}\strut \\
77```
88
99```
1010% pandoc -f html -t latex
1111<a name="foo"></a><br/>
1212^D
13−\protect\hypertarget{foo}{}{}~\\
13+\protect\hypertarget{foo}{}{}\strut \\
1414```
test/command/3752.mdnew file+60
@@ -0,0 +1,60 @@
1+```
2+% pandoc command/chap1/text.md command/chap2/text.md -f markdown+rebase_relative_paths --verbose -t docx -o - | pandoc -f docx -t plain
3+^D
4+[INFO] Loaded command/chap1/spider.png from command/chap1/spider.png
5+[INFO] Loaded command/chap2/spider.png from command/chap2/spider.png
6+[INFO] Loaded command/chap1/../../lalune.jpg from command/chap1/../../lalune.jpg
7+Chapter one
8+
9+A spider: [spider]
10+
11+Another spider: [another spider]
12+
13+The moon: [moon]
14+
15+Link to spider picture.
16+
17+URL left alone: manual.
18+
19+Absolute path left alone: absolute.
20+
21+Link to fragment: chapter two.
22+
23+Empty path: empty.
24+
25+Chapter two
26+
27+A spider: [spider]
28+```
29+
30+```
31+% pandoc command/chap1/text.md command/chap2/text.md -f markdown+rebase_relative_paths -t html
32+^D
33+<h1 id="chapter-one">Chapter one</h1>
34+<p>A spider: <img src="command/chap1/spider.png" alt="spider" /></p>
35+<p>Another spider: <img src="command/chap2/spider.png" alt="another spider" /></p>
36+<p>The moon: <img src="command/chap1/../../lalune.jpg" alt="moon" /></p>
37+<p>Link to <a href="command/chap1/spider.png">spider picture</a>.</p>
38+<p>URL left alone: <a href="https://pandoc.org/MANUAL.html">manual</a>.</p>
39+<p>Absolute path left alone: <a href="/foo/bar/baz.png">absolute</a>.</p>
40+<p>Link to fragment: <a href="#chapter-two">chapter two</a>.</p>
41+<p>Empty path: <a href="">empty</a>.</p>
42+<h1 id="chapter-two">Chapter two</h1>
43+<p>A spider: <img src="command/chap2/spider.png" alt="spider" /></p>
44+```
45+
46+```
47+% pandoc command/chap1/text.md command/chap2/text.md -f commonmark+rebase_relative_paths -t html
48+^D
49+<h1>Chapter one</h1>
50+<p>A spider: <img src="command/chap1/spider.png" alt="spider" /></p>
51+<p>Another spider: <img src="command/chap2/spider.png" alt="another spider" /></p>
52+<p>The moon: <img src="command/chap1/../../lalune.jpg" alt="moon" /></p>
53+<p>Link to <a href="command/chap1/spider.png">spider picture</a>.</p>
54+<p>URL left alone: <a href="https://pandoc.org/MANUAL.html">manual</a>.</p>
55+<p>Absolute path left alone: <a href="/foo/bar/baz.png">absolute</a>.</p>
56+<p>Link to fragment: <a href="#chapter-two">chapter two</a>.</p>
57+<p>Empty path: <a href="">empty</a>.</p>
58+<h1>Chapter two</h1>
59+<p>A spider: <img src="command/chap2/spider.png" alt="spider" /></p>
60+```
test/command/3792.md+1−1
@@ -6,7 +6,7 @@ and properly escaped.
66ok
77^D
88---
9−title: \<this\> \*that\*
9+title: \<this> \*that\*
1010---
1111
1212ok
test/command/4063.mdnew file+29
@@ -0,0 +1,29 @@
1+```
2+% pandoc -f html -t native
3+<table>
4+<colgroup>
5+ <col width="30%" />
6+ <col width="*" />
7+</colgroup>
8+<tr>
9+ <td>1</td>
10+ <td>2</td>
11+</tr>
12+</table>
13+^D
14+[Table ("",[],[]) (Caption Nothing
15+ [])
16+ [(AlignDefault,ColWidth 0.3)
17+ ,(AlignDefault,ColWidth 0.7)]
18+ (TableHead ("",[],[])
19+ [])
20+ [(TableBody ("",[],[]) (RowHeadColumns 0)
21+ []
22+ [Row ("",[],[])
23+ [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
24+ [Plain [Str "1"]]
25+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
26+ [Plain [Str "2"]]]])]
27+ (TableFoot ("",[],[])
28+ [])]
29+```
test/command/4164.md+1−1
@@ -26,6 +26,6 @@ Here is inline html:
2626
2727Here is inline html:
2828
29−\<div\> \<balise\> bla bla \</div\>
29+\<div> \<balise> bla bla \</div>
3030
3131```
test/command/4442.md+1−1
@@ -5,5 +5,5 @@
55^D
66\newcommand{\myFruit}{Mango\xspace}
77
8−Mango\xspace is the king of fruits.
8+Mango is the king of fruits.
99```
test/command/4465.mdnew file+15
@@ -0,0 +1,15 @@
1+```
2+% pandoc -f html -t markdown
3+<ol>
4+ <li>An ordered list can contain block-level elements ind html, it means that divs are also allowed.</li>
5+ <li>Let's see the problem! <div class="example">This is an example.</div></li>
6+</ol>
7+^D
8+1. An ordered list can contain block-level elements ind html, it means
9+ that divs are also allowed.
10+2. Let\'s see the problem!
11+
12+ ::: example
13+ This is an example.
14+ :::
15+```
test/command/5304.md+1−1
@@ -11,7 +11,7 @@
1111...
1212```
1313^D
14−``` {.markdown}
14+``` markdown
1515`«sträng»`
1616
1717`` «sträng» ``
test/command/5367.md+6−2
@@ -24,11 +24,15 @@ hello\footnote{doc footnote}
2424 >{\centering\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{0.17}}@{}}
2525\caption[Sample table.]{Sample table.\footnote{caption footnote}}\tabularnewline
2626\toprule
27−Fruit\footnote{header footnote} \\
27+\begin{minipage}[b]{\linewidth}\centering
28+Fruit\footnote{header footnote}
29+\end{minipage} \\
2830\midrule
2931\endfirsthead
3032\toprule
31−Fruit{} \\
33+\begin{minipage}[b]{\linewidth}\centering
34+Fruit{}
35+\end{minipage} \\
3236\midrule
3337\endhead
3438Bans\footnote{table cell footnote} \\
test/command/5519.md+1−1
@@ -6,7 +6,7 @@
66 ```
77``````
88^D
9−```` {.attr}
9+```` attr
1010 ```
1111 code
1212 ```
test/command/6026.mdnew file+19
@@ -0,0 +1,19 @@
1+```
2+% pandoc -t native
3+@{https://openreview.net/forum?id=HkwoSDPgg}
4+
5+@https://openreview.net/forum?id=HkwoSDPgg
6+^D
7+[Para [Cite [Citation {citationId = "https://openreview.net/forum?id=HkwoSDPgg", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 1, citationHash = 0}] [Str "@https://openreview.net/forum?id=HkwoSDPgg"]]
8+,Para [Cite [Citation {citationId = "https://openreview.net/forum?id", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "@https://openreview.net/forum?id"],Str "=HkwoSDPgg"]]
9+```
10+```
11+% pandoc -t markdown
12+@{https://openreview.net/forum?id=HkwoSDPgg}
13+
14+@https://openreview.net/forum?id=HkwoSDPgg
15+^D
16+@{https://openreview.net/forum?id=HkwoSDPgg}
17+
18+@https://openreview.net/forum?id=HkwoSDPgg
19+```
test/command/6541.mdnew file+27
@@ -0,0 +1,27 @@
1+```
2+% pandoc -f docbook -t markdown -s
3+<?xml version="1.0"?>
4+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
5+ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
6+<book>
7+
8+<bookinfo>
9+<title>Title</title>
10+<author>
11+<firstname>Firstname</firstname><surname>Lastname</surname>
12+</author>
13+<releaseinfo>1.17</releaseinfo>
14+</bookinfo>
15+
16+<para>Text.</para>
17+
18+</book>
19+^D
20+---
21+author: Firstname Lastname
22+releaseinfo: 1.17
23+title: Title
24+---
25+
26+Text.
27+```
test/command/6620.md+3
@@ -7,9 +7,12 @@
77\SI{0.135(21)}{\m}
88
99\SI{12.3(60)}{\m}
10+
11+\SI{10.0 \pm 3.3}{\ms}
1012^D
1113<p>23 ± 2 m</p>
1214<p>125 ± 12 m</p>
1315<p>0.135 ± 0.021 m</p>
1416<p>12.3 ± 6 m</p>
17+<p>10.0 ± 3.3 ms</p>
1518```
test/command/6658.mdnew file+75
@@ -0,0 +1,75 @@
1+```
2+pandoc -f latex -t html
3+\num{12345,67890}
4+
5+\num{1+-2i}
6+
7+\num{.3e45}
8+
9+\num{1.654 x 2.34 x 3.430}
10+
11+\si{kg.m.s^{-1}}
12+
13+\si{\kilogram\metre\per\second}
14+
15+\si[per-mode=symbol]{\kilogram\metre\per\second}
16+
17+\si[per-mode=symbol]{\kilogram\metre\per\ampere\per\second}
18+
19+\numlist{10;20;30}
20+
21+\SIlist{0.13;0.67;0.80}{\milli\metre}
22+
23+\numrange{10}{20}
24+
25+\SIrange{0.13}{0.67}{\milli\metre}
26+
27+\ang{10}
28+
29+\ang{1;2;3}
30+
31+\ang{;;1}
32+
33+\ang{+10;;}
34+
35+\ang{-0;1;}
36+
37+\si{kg.m/s^2}
38+
39+\si{g_{polymer}~mol_{cat}.s^{-1}}
40+
41+\si{\kilo\gram\metre\per\square\second}
42+
43+\si{\gram\per\cubic\centi\metre}
44+
45+\si{\square\volt\cubic\lumen\per\farad}
46+
47+\si{\metre\squared\per\gray\cubic\lux}
48+
49+\si{\henry\second}
50+^D
51+<p>12345.67890</p>
52+<p>1 ± 2i</p>
53+<p>0.3 × 10<sup>45</sup></p>
54+<p>1.654 × 2.34 × 3.430</p>
55+<p>kg m s<sup>−1</sup></p>
56+<p>kg m s<sup>−1</sup></p>
57+<p>kg m/s</p>
58+<p>kg m/A/s</p>
59+<p>10, 20, &amp; 30</p>
60+<p>0.13 mm, 0.67 mm, &amp; 0.80 mm</p>
61+<p>10–20</p>
62+<p>0.13 mm–0.67 mm</p>
63+<p>10°</p>
64+<p>1°2′3″</p>
65+<p>1″</p>
66+<p>10°</p>
67+<p>-0°1′</p>
68+<p>kg m/s<sup>2</sup></p>
69+<p>g<sub>polymer</sub> mol<sub>cat</sub> s<sup>−1</sup></p>
70+<p>kg m s<sup>−2</sup></p>
71+<p>g cm<sup>−3</sup></p>
72+<p>V<sup>2</sup> lm<sup>3</sup> F<sup>−1</sup></p>
73+<p>m<sup>2</sup> Gy<sup>−1</sup> lx<sup>3</sup></p>
74+<p>H s</p>
75+```
test/command/6709.mdnew file+11
@@ -0,0 +1,11 @@
1+Tabs must be expanded even if --file-scope is used
2+````
3+% pandoc -t native --file-scope --tab-stop=2
4+```
5+if true; then
6+ echo "yup"
7+fi
8+```
9+^D
10+[CodeBlock ("",[],[]) "if true; then\n echo \"yup\"\nfi"]
11+````
test/command/6890.md+2−2
@@ -23,12 +23,12 @@ references:
2323
2424@fruchtel-sozialer-2013a
2525
26−Some text [^1].
26+Some text.[^1]
2727
2828[^1]: @fruchtel-sozialer-2013a
2929^D
3030[Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 1, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]
31−,Para [Str "Some",Space,Str "text",Str ".",Note [Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]],Str ""]
31+,Para [Str "Some",Space,Str "text.",Note [Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]]]
3232,Div ("refs",["references","csl-bib-body","hanging-indent"],[])
3333 [Div ("ref-fruchtel-sozialer-2013a",["csl-entry"],[])
3434 [Para [Str "Fr\252chtel,",Space,Str "Frank,",Space,Str "Wolfgang",Space,Str "Budde,",Space,Str "and",Space,Str "Gudrun",Space,Str "Cyprian.",Space,Str "2013.",Space,Emph [Str "Sozialer",Space,Str "Raum",Space,Str "und",Space,Str "Soziale",Space,Str "Arbeit",Space,Str "Fieldbook:",Space,Str "Methoden",Space,Str "und",Space,Str "Techniken"],Str ".",Space,Str "3rd",Space,Str "ed.",Space,Str "Wiesbaden,",Space,Str "Germany:",Space,Str "Springer",Space,Str "VS."]]]]
test/command/6925.md+2−2
@@ -20,13 +20,13 @@ a
2020\end{thm2}
2121\end{document}
2222^D
23−::: {.thm}
23+::: thm
2424**Theorem 1**. *a*
2525
2626![image](1.png)
2727:::
2828
29−::: {.thm2}
29+::: thm2
3030**Theorem 1**. a
3131
3232![image](1.png)
test/command/6958.mdnew file+10
@@ -0,0 +1,10 @@
1+Add thin space between single and double quotes.
2+```
3+% pandoc -t latex+smart
4+---
5+lang: en-GB
6+---
7+'["On the Outside"]{}: Constructing Cycling Citizenship.'
8+^D
9+`\,{``On the Outside''}: Constructing Cycling Citizenship.'
10+```
test/command/7041.mdnew file+23
@@ -0,0 +1,23 @@
1+```
2+% pandoc -f html -t jats
3+<table>
4+ <tr><td><blockquote>Fly, you fools!</blockquote></td></tr>
5+</table>
6+^D
7+<table-wrap>
8+ <table>
9+ <colgroup>
10+ <col width="100%" />
11+ </colgroup>
12+ <tbody>
13+ <tr>
14+ <td><p specific-use="wrapper">
15+ <disp-quote>
16+ <p>Fly, you fools!</p>
17+ </disp-quote>
18+ </p></td>
19+ </tr>
20+ </tbody>
21+ </table>
22+</table-wrap>
23+```
test/command/7099.md+1−1
@@ -11,7 +11,7 @@
1111<iframe src="h:invalid@url"></iframe>
1212^D
1313[INFO] Fetching h:invalid@url...
14−[WARNING] Could not fetch resource 'h:invalid@url': Could not fetch h:invalid@url
14+[WARNING] Could not fetch resource h:invalid@url: Could not fetch h:invalid@url
1515 InvalidUrlException "h:invalid@url" "Invalid scheme"
1616[INFO] Skipped '<iframe src="h:invalid@url"></iframe>' at input line 1 column 29
1717[]
test/command/7129.mdnew file+27
@@ -0,0 +1,27 @@
1+```
2+% pandoc -f latex -t native
3+\begin{tabular}{ll} \hline
4+ FOO & BAR \\ \hline
5+ foo & \verb|b&r| \\ \hline
6+\end{tabular}
7+^D
8+[Table ("",[],[]) (Caption Nothing
9+ [])
10+ [(AlignLeft,ColWidthDefault)
11+ ,(AlignLeft,ColWidthDefault)]
12+ (TableHead ("",[],[])
13+ [Row ("",[],[])
14+ [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
15+ [Plain [Str "FOO"]]
16+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
17+ [Plain [Str "BAR"]]]])
18+ [(TableBody ("",[],[]) (RowHeadColumns 0)
19+ []
20+ [Row ("",[],[])
21+ [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
22+ [Plain [Str "foo"]]
23+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
24+ [Plain [Code ("",[],[]) "b&r"]]]])]
25+ (TableFoot ("",[],[])
26+ [])]
27+```
test/command/7132.mdnew file+10
@@ -0,0 +1,10 @@
1+```
2+% pandoc -f markdown -t org --columns=72
3+- This line has exactly the wrong number of characters before the number 5.
4+- Long line ending with a number (this time it is in parentheses and a 23)
5+^D
6+- This line has exactly the wrong number of characters before the
7+ number 5.
8+- Long line ending with a number (this time it is in parentheses and
9+ a 23)
10+```
test/command/7134.mdnew file+16
@@ -0,0 +1,16 @@
1+```
2+% pandoc -f rst -t native
3+This is a paragraph.
4+
5+ This is a block quote.
6+
7+..
8+
9+ This should be a second block quote.
10+^D
11+[Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "paragraph."]
12+,BlockQuote
13+ [Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "block",Space,Str "quote."]]
14+,BlockQuote
15+ [Para [Str "This",Space,Str "should",Space,Str "be",Space,Str "a",Space,Str "second",Space,Str "block",Space,Str "quote."]]]
16+```
test/command/7145.mdnew file+12
@@ -0,0 +1,12 @@
1+```
2+% pandoc -f mediawiki -t native
3+Maecenas at sapien tempor, pretium turpis ut, imperdiet augue.<ref>This is a multiline
4+
5+reference
6+<i>with</i>
7+empty
8+
9+linebreaks</ref> Nulla ut massa eget ex venenatis lobortis id in eros.
10+^D
11+[Para [Str "Maecenas",Space,Str "at",Space,Str "sapien",Space,Str "tempor,",Space,Str "pretium",Space,Str "turpis",Space,Str "ut,",Space,Str "imperdiet",Space,Str "augue.",Note [Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "multiline"],Para [Str "reference",SoftBreak,RawInline (Format "html") "<i>",Str "with",RawInline (Format "html") "</i>",SoftBreak,Str "empty"],Para [Str "linebreaks"]],Space,Str "Nulla",Space,Str "ut",Space,Str "massa",Space,Str "eget",Space,Str "ex",Space,Str "venenatis",Space,Str "lobortis",Space,Str "id",Space,Str "in",Space,Str "eros."]]
12+```
test/command/7155.mdnew file+15
@@ -0,0 +1,15 @@
1+```
2+% pandoc -f markdown+tex_math_single_backslash -t native
3+\(x\) \[x\]
4+\\(x\\) \\[x\\]
5+^D
6+[Para [Math InlineMath "x",Space,Math DisplayMath "x",SoftBreak,Str "\\(x\\)",Space,Str "\\[x\\]"]]
7+```
8+
9+```
10+% pandoc -f markdown+tex_math_double_backslash -t native
11+\(x\) \[x\]
12+\\(x\\) \\[x\\]
13+^D
14+[Para [Str "(x)",Space,Str "[x]",SoftBreak,Math InlineMath "x",Space,Math DisplayMath "x"]]
15+```
test/command/7173.mdnew file+137
@@ -0,0 +1,137 @@
1+```
2+% pandoc -f docbook -t latex
3+<?xml version="1.0" encoding="UTF-8"?>
4+<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
5+<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
6+<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
7+<article xmlns="http://docbook.org/ns/docbook"
8+ xmlns:xlink="http://www.w3.org/1999/xlink"
9+ version="5.1">
10+ <title>Untitled Document</title>
11+ <para>Word</para>
12+ <informalequation>
13+ <alt role='tex'>1+2</alt>
14+ <m:math xmlns:m="http://www.w3.org/1998/Math/MathML">
15+ <m:mrow>
16+ <m:mrow><m:mn>1</m:mn><m:mo>+</m:mo><m:mn>2</m:mn>
17+ </m:mrow>
18+ </m:mrow>
19+ </m:math>
20+ </informalequation>
21+</article>
22+^D
23+Word
24+
25+\[1 + 2\]
26+```
27+
28+```
29+% pandoc -f docbook -t latex
30+<?xml version="1.0" encoding="UTF-8"?>
31+<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
32+<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
33+<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
34+<article xmlns="http://docbook.org/ns/docbook"
35+ xmlns:xlink="http://www.w3.org/1999/xlink"
36+ version="5.1">
37+ <title>Untitled Document</title>
38+ <para>Word</para>
39+ <informalequation>
40+ <alt role='tex'>1+2</alt>
41+ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML">
42+ <mml:mrow>
43+ <mml:mrow><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mn>2</mml:mn>
44+ </mml:mrow>
45+ </mml:mrow>
46+ </mml:math>
47+ </informalequation>
48+</article>
49+^D
50+Word
51+
52+\[1 + 2\]
53+```
54+
55+```
56+% pandoc -f docbook -t latex
57+<?xml version="1.0" encoding="UTF-8"?>
58+<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
59+<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
60+<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
61+<article xmlns="http://docbook.org/ns/docbook"
62+ xmlns:xlink="http://www.w3.org/1999/xlink"
63+ version="5.1">
64+ <title>Untitled Document</title>
65+ <para>Word</para>
66+ <informalequation>
67+ <alt role='tex'>1+2</alt>
68+ <math xmlns="http://www.w3.org/1998/Math/MathML">
69+ <mrow>
70+ <mrow><mn>1</mn><mo>+</mo><mn>2</mn>
71+ </mrow>
72+ </mrow>
73+ </math>
74+ </informalequation>
75+</article>
76+^D
77+Word
78+
79+\[1 + 2\]
80+```
81+
82+```
83+% pandoc -f docbook -t latex
84+<?xml version="1.0" encoding="UTF-8"?>
85+<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
86+<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
87+<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
88+<article xmlns="http://docbook.org/ns/docbook"
89+ xmlns:xlink="http://www.w3.org/1999/xlink"
90+ xmlns:m="http://www.w3.org/1998/Math/MathML"
91+ version="5.1">
92+ <title>Untitled Document</title>
93+ <para>Word</para>
94+ <informalequation>
95+ <alt role='tex'>1+2</alt>
96+ <m:math>
97+ <m:mrow>
98+ <m:mrow><m:mn>1</m:mn><m:mo>+</m:mo><m:mn>2</m:mn>
99+ </m:mrow>
100+ </m:mrow>
101+ </m:math>
102+ </informalequation>
103+</article>
104+^D
105+Word
106+
107+\[1 + 2\]
108+```
109+
110+```
111+% pandoc -f docbook -t latex
112+<?xml version="1.0" encoding="UTF-8"?>
113+<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
114+<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
115+<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
116+<article xmlns="http://docbook.org/ns/docbook"
117+ xmlns:xlink="http://www.w3.org/1999/xlink"
118+ xmlns:mml="http://www.w3.org/1998/Math/MathML"
119+ version="5.1">
120+ <title>Untitled Document</title>
121+ <para>Word</para>
122+ <informalequation>
123+ <alt role='tex'>1+2</alt>
124+ <mml:math>
125+ <mml:mrow>
126+ <mml:mrow><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mn>2</mml:mn>
127+ </mml:mrow>
128+ </mml:mrow>
129+ </mml:math>
130+ </informalequation>
131+</article>
132+^D
133+Word
134+
135+\[1 + 2\]
136+```
137+
test/command/7208.mdnew file+6
@@ -0,0 +1,6 @@
1+```
2+% pandoc -t gfm
3+\<hi\>
4+^D
5+\<hi>
6+```
test/command/7214.mdnew file+28
@@ -0,0 +1,28 @@
1+```
2+% pandoc
3++------------+----------+------------------+
4+|日本語 | の文字列 | words in english |
5++------------+----------+------------------+
6+|abc defghij | def | xyz |
7++------------+----------+------------------+
8+^D
9+<table style="width:60%;">
10+<colgroup>
11+<col style="width: 18%" />
12+<col style="width: 15%" />
13+<col style="width: 26%" />
14+</colgroup>
15+<tbody>
16+<tr class="odd">
17+<td>日本語</td>
18+<td>の文字列</td>
19+<td>words in english</td>
20+</tr>
21+<tr class="even">
22+<td>abc defghij</td>
23+<td>def</td>
24+<td>xyz</td>
25+</tr>
26+</tbody>
27+</table>
28+```
test/command/7216.mdnew file+19
@@ -0,0 +1,19 @@
1+```
2+pandoc -t latex
3+"This is some text in quotes. Another paragraph by the same speaker follows. The first paragraph should have no close quote.
4+
5+"The second paragraph should have open and close quotes."
6+
7+| "Open quote on this line,
8+| Close quote on the next line."
9+| "Quotes on the same line."
10+^D
11+``This is some text in quotes. Another paragraph by the same speaker
12+follows. The first paragraph should have no close quote.
13+
14+``The second paragraph should have open and close quotes.''
15+
16+``Open quote on this line,\\
17+Close quote on the next line.''\\
18+``Quotes on the same line.''
19+```
test/command/7266.mdnew file+10
@@ -0,0 +1,10 @@
1+```
2+% pandoc -f biblatex -t biblatex -s
3+@article{id,
4+ annote = "annotation"
5+}
6+^D
7+@article{id,
8+ annote = {annotation}
9+}
10+```
test/command/7272.mdnew file+27
@@ -0,0 +1,27 @@
1+```
2+% pandoc -t latex -f html
3+<table>
4+ <tbody>
5+ <tr>
6+ <td>
7+ <span>
8+ text
9+ <br />
10+ text2
11+ </span>
12+ </td>
13+ </tr>
14+ </tbody>
15+</table>
16+^D
17+\begin{longtable}[]{@{}
18+ >{\raggedright\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{1.00}}@{}}
19+\toprule
20+\endhead
21+\begin{minipage}[t]{\linewidth}\raggedright
22+{ text\\
23+text2 }\strut
24+\end{minipage} \\
25+\bottomrule
26+\end{longtable}
27+```
test/command/7278.mdnew file+38
@@ -0,0 +1,38 @@
1+```
2+% pandoc -t beamer
3+# Slide
4+
5+Some blocks:
6+
7+## example block title {.example}
8+
9+text in block
10+
11+## alert block title {.alert}
12+
13+text in block
14+
15+## block title
16+
17+text in block
18+^D
19+\begin{frame}{Slide}
20+\protect\hypertarget{slide}{}
21+Some blocks:
22+
23+\begin{exampleblock}{example block title}
24+\protect\hypertarget{example-block-title}{}
25+text in block
26+\end{exampleblock}
27+
28+\begin{alertblock}{alert block title}
29+\protect\hypertarget{alert-block-title}{}
30+text in block
31+\end{alertblock}
32+
33+\begin{block}{block title}
34+\protect\hypertarget{block-title}{}
35+text in block
36+\end{block}
37+\end{frame}
38+```
test/command/7282.mdnew file+7
@@ -0,0 +1,7 @@
1+Don't crash on unmatched closing tag.
2+```
3+% pandoc -f html -t native
4+</script>
5+^D
6+[]
7+```
test/command/7288.mdnew file+40
@@ -0,0 +1,40 @@
1+```
2+% pandoc -f rst -t ms
3+.. list-table::
4+ :widths: 50 50
5+ :header-rows: 1
6+
7+ * - Left
8+ - Right
9+ * - Long text that should be easy to break up into multiple lines
10+ - Another long text that should be easy to break up into multiple lines
11+
12+ Bar
13+^D
14+.PP
15+.na
16+.nr LLold \n[LL]
17+.TS
18+delim(@@) tab( );
19+lw(35.0n) lw(35.0n).
20+T{
21+Left
22+T} T{
23+Right
24+T}
25+_
26+T{
27+.nr LL 35.0n
28+.LP
29+Long text that should be easy to break up into multiple lines
30+T} T{
31+.nr LL 35.0n
32+.PP
33+Another long text that should be easy to break up into multiple lines
34+.PP
35+Bar
36+T}
37+.TE
38+.nr LL \n[LLold]
39+.ad
40+```
test/command/7299.mdnew file+23
@@ -0,0 +1,23 @@
1+```
2+% pandoc -f latex -t plain
3+$1-{\ensuremath{r}\xspace}$
4+^D
5+1 − r
6+```
7+
8+```
9+% pandoc -f latex -t plain
10+\newcommand{\foo}{Foo\xspace}
11+
12+$\text{\foo bar}$
13+^D
14+Foo bar
15+```
16+
17+```
18+% pandoc -f latex -t plain
19+a\xspace b
20+^D
21+a b
22+```
23+
test/command/7321.mdnew file+24
@@ -0,0 +1,24 @@
1+```
2+% pandoc -t plain --citeproc --wrap=none
3+---
4+references:
5+- id: fenner2012a
6+ title: One-click science marketing
7+ author:
8+ - family: Fenner
9+ given: Martin
10+ container-title: Nature Materials
11+ volume: 11
12+ issue: 4
13+ publisher: Nature Publishing Group
14+ type: article-journal
15+ issued:
16+ year: 2012
17+---
18+
19+[@fenner2012a, $a^2+b^2=c^2$]
20+^D
21+(Fenner 2012, a² + b² = c²)
22+
23+Fenner, Martin. 2012. “One-Click Science Marketing.” Nature Materials 11 (4).
24+```
test/command/7323.mdnew file+29
@@ -0,0 +1,29 @@
1+```
2+% pandoc --citeproc -t plain
3+---
4+references:
5+- id: smith
6+ author: John Smith
7+ issued: 2019
8+ title: Insects
9+ type: book
10+...
11+
12+@smith [chap. 6]
13+
14+@smith [chapter 6]
15+
16+@smith [Chap. 6]
17+
18+@smith [Chapter 6]
19+^D
20+John Smith (2019, chap. 6)
21+
22+John Smith (2019, chap. 6)
23+
24+John Smith (2019, chap. 6)
25+
26+John Smith (2019, chap. 6)
27+
28+John Smith. 2019. Insects.
29+```
test/command/7324.mdnew file+25
@@ -0,0 +1,25 @@
1+```
2+% pandoc --citeproc -t plain
3+---
4+references:
5+- id: smith
6+ author: John Smith
7+ issued: 2019
8+ title: Insects
9+ type: book
10+...
11+
12+@smith [, among others]
13+
14+@smith [ among others]
15+
16+@smith [among others]
17+^D
18+John Smith (2019, among others)
19+
20+John Smith (2019 among others)
21+
22+John Smith (2019, among others)
23+
24+John Smith. 2019. Insects.
25+```
test/command/7339.mdnew file+11
@@ -0,0 +1,11 @@
1+```
2+% pandoc -f gfm -s -t native
3+---
4+title: Test
5+---
6+
7+Hi
8+^D
9+Pandoc (Meta {unMeta = fromList [("title",MetaInlines [Str "Test"])]})
10+[Para [Str "Hi"]]
11+```
test/command/7340.mdnew file+6
@@ -0,0 +1,6 @@
1+```
2+% pandoc -f latex -t native
3+\(*\)
4+^D
5+[Para [Math InlineMath "*"]]
6+```
test/command/7376.mdnew file+16
@@ -0,0 +1,16 @@
1+```
2+% pandoc --citeproc -t plain
3+---
4+references:
5+- id: item1
6+ type: book
7+ author:
8+ - family: Doe
9+ given: Jane
10+ issued: 2020
11+ title: The title
12+...
13+^D
14+
15+
16+```
test/command/7394.mdnew file+85
@@ -0,0 +1,85 @@
1+```
2+% pandoc -f markdown -t plain --citeproc
3+---
4+csl: command/chicago-fullnote-bibliography.csl
5+references:
6+- author:
7+ - family: Wandt
8+ given: Manfred
9+ edition: 6
10+ id: wandt2014ges-sv
11+ issued: 2014
12+ publisher: Franz Vahlen
13+ publisher-place: München
14+ title: Gesetzliche schuldverhältnisse
15+ title-short: Gesetzl SV
16+ type: book
17+- author:
18+ - family: Smith
19+ given: Zenda
20+ edition: 6
21+ id: smith2015
22+ issued: 2015
23+ publisher: Macmillan
24+ publisher-place: New York
25+ title: A verb and a noun
26+ type: book
27+---
28+
29+Hi^[@wandt2014ges-sv.].
30+
31+Hi^[[@wandt2014ges-sv].].
32+
33+Hi^[[See also @wandt2014ges-sv].].
34+
35+Hi^[See also @wandt2014ges-sv.].
36+
37+Hi^[@wandt2014ges-sv [@smith2015].].
38+
39+Hi^[[@wandt2014ges-sv; @smith2015].].
40+
41+Hi [@wandt2014ges-sv].
42+
43+Hi [see also @wandt2014ges-sv].
44+
45+^D
46+Hi[1].
47+
48+Hi[2].
49+
50+Hi[3].
51+
52+Hi[4].
53+
54+Hi[5].
55+
56+Hi[6].
57+
58+Hi.[7]
59+
60+Hi.[8]
61+
62+Smith, Zenda. A Verb and a Noun. 6th ed. New York: Macmillan, 2015.
63+
64+Wandt, Manfred. Gesetzliche Schuldverhältnisse. 6th ed. München: Franz
65+Vahlen, 2014.
66+
67+[1] Manfred Wandt, Gesetzliche Schuldverhältnisse, 6th ed. (München:
68+Franz Vahlen, 2014).
69+
70+[2] Wandt.
71+
72+[3] See also Wandt.
73+
74+[4] See also Wandt.
75+
76+[5] Wandt, Zenda Smith, A Verb and a Noun, 6th ed. (New York: Macmillan,
77+2015).
78+
79+[6] Wandt, Gesetzl SV; Smith, A Verb and a Noun.
80+
81+[7] Wandt, Gesetzl SV.
82+
83+[8] See also Wandt.
84+```
85+
test/command/7397.mdnew file+14
@@ -0,0 +1,14 @@
1+````
2+% pandoc -t markdown
3+~~~~ { .haskell startFrom="100"}
4+qsort [] = []
5+qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++
6+ qsort (filter (>= x) xs)
7+~~~~
8+^D
9+``` {.haskell startFrom="100"}
10+qsort [] = []
11+qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++
12+ qsort (filter (>= x) xs)
13+```
14+````
test/command/7400.mdnew file+9
@@ -0,0 +1,9 @@
1+```
2+% pandoc -t native -s
3+---
4+# Comment only
5+...
6+^D
7+Pandoc (Meta {unMeta = fromList []})
8+[]
9+```
test/command/7416.mdnew file+19
@@ -0,0 +1,19 @@
1+```
2+% pandoc -f markdown -t html
3+![caption](../media/rId25.jpg "title"){alt="alt"}
4+
5+^D
6+<figure>
7+<img src="../media/rId25.jpg" title="title" alt="alt" /><figcaption>caption</figcaption>
8+</figure>
9+```
10+
11+```
12+% pandoc -f markdown -t html
13+![caption](../media/rId25.jpg "title")
14+
15+^D
16+<figure>
17+<img src="../media/rId25.jpg" title="title" alt="caption" /><figcaption aria-hidden="true">caption</figcaption>
18+</figure>
19+```
test/command/averroes.bibnew file+95
@@ -0,0 +1,95 @@
1+@string{anch-ie = {Angew.~Chem. Int.~Ed.}}
2+@string{cup = {Cambridge University Press}}
3+@string{dtv = {Deutscher Taschenbuch-Verlag}}
4+@string{hup = {Harvard University Press}}
5+@string{jams = {J.~Amer. Math. Soc.}}
6+@string{jchph = {J.~Chem. Phys.}}
7+@string{jomch = {J.~Organomet. Chem.}}
8+@string{pup = {Princeton University Press}}
9+
10+@book{averroes/bland,
11+ author = {Averroes},
12+ title = {The Epistle on the Possibility of Conjunction with the Active
13+ Intellect by {Ibn Rushd} with the Commentary of {Moses Narboni}},
14+ date = 1982,
15+ editor = {Bland, Kalman P.},
16+ translator = {Bland, Kalman P.},
17+ series = {Moreshet: Studies in {Jewish} History, Literature and Thought},
18+ number = 7,
19+ publisher = {Jewish Theological Seminary of America},
20+ location = {New York},
21+ keywords = {primary},
22+ langid = {english},
23+ langidopts = {variant=american},
24+ indextitle = {Epistle on the Possibility of Conjunction, The},
25+ shorttitle = {Possibility of Conjunction},
26+ annotation = {A \texttt{book} entry with a \texttt{series} and a
27+ \texttt{number}. Note the concatenation of the \texttt{editor}
28+ and \texttt{translator} fields as well as the
29+ \texttt{indextitle} field},
30+}
31+
32+@book{averroes/hannes,
33+ author = {Averroes},
34+ title = {Des Averro{\"e}s Abhandlung: \mkbibquote{{\"U}ber die
35+ M{\"o}glichkeit der Conjunktion} oder \mkbibquote{{\"U}ber den
36+ materiellen Intellekt}},
37+ date = 1892,
38+ editor = {Hannes, Ludwig},
39+ translator = {Hannes, Ludwig},
40+ annotator = {Hannes, Ludwig},
41+ publisher = {C.~A. Kaemmerer},
42+ location = {Halle an der Saale},
43+ keywords = {primary},
44+ langid = {german},
45+ sorttitle = {Uber die Moglichkeit der Conjunktion},
46+ indexsorttitle= {Uber die Moglichkeit der Conjunktion},
47+ indextitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
48+ shorttitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
49+ annotation = {An annotated edition. Note the concatenation of the
50+ \texttt{editor}, \texttt{translator}, and \texttt{annotator}
51+ fields. Also note the \texttt{shorttitle},
52+ \texttt{indextitle}, \texttt{sorttitle}, and
53+ \texttt{indexsorttitle} fields},
54+}
55+
56+@book{averroes/hercz,
57+ author = {Averroes},
58+ title = {Drei Abhandlungen {\"u}ber die Conjunction des separaten
59+ Intellects mit dem Menschen},
60+ date = 1869,
61+ editor = {Hercz, J.},
62+ translator = {Hercz, J.},
63+ publisher = {S.~Hermann},
64+ location = {Berlin},
65+ keywords = {primary},
66+ langid = {german},
67+ indexsorttitle= {Drei Abhandlungen uber die Conjunction},
68+ indextitle = {Drei Abhandlungen {\"u}ber die Conjunction},
69+ subtitle = {Von Averroes (Vater und Sohn), aus dem Arabischen
70+ {\"u}bersetzt von Samuel Ibn Tibbon},
71+ shorttitle = {Drei Abhandlungen},
72+ annotation = {A \texttt{book} entry. Note the concatenation of the
73+ \texttt{editor} and \texttt{translator} fields as well as the
74+ \texttt{indextitle} and \texttt{indexsorttitle} fields},
75+}
76+
77+@book{cicero,
78+ author = {Cicero, Marcus Tullius},
79+ title = {De natura deorum. {\"U}ber das Wesen der G{\"o}tter},
80+ date = 1995,
81+ editor = {Blank-Sangmeister, Ursula},
82+ translator = {Blank-Sangmeister, Ursula},
83+ afterword = {Thraede, Klaus},
84+ language = {langlatin and langgerman},
85+ publisher = {Reclam},
86+ location = {Stuttgart},
87+ langid = {german},
88+ indextitle = {De natura deorum},
89+ shorttitle = {De natura deorum},
90+ annotation = {A bilingual edition of Cicero's \emph{De natura deorum}, with
91+ a German translation. Note the format of the \texttt{language}
92+ field in the database file, the concatenation of the
93+ \texttt{editor} and \texttt{translator} fields, and the
94+ \texttt{afterword} field},
95+}
test/command/biblatex-basic.md+5−3
@@ -1,3 +1,5 @@
1+```
2+% pandoc -f biblatex -t markdown -s
13@Book{item1,
24author="John Doe",
35title="First Book",
@@ -27,7 +29,7 @@ year="2007"
2729
2830^D
2931---
30−nocite: '[@*]'
32+nocite: "[@*]"
3133references:
3234- author:
3335 - family: Doe
@@ -44,9 +46,9 @@ references:
4446 container-title: Journal of Generic Studies
4547 id: item2
4648 issued: 2006
47− page: '33-34'
49+ page: 33-34
4850 title: Article
49− type: 'article-journal'
51+ type: article-journal
5052 volume: 6
5153- author:
5254 - family: Doe
test/command/biblatex-examples.bibdeleted−1674
@@ -1,1674 +0,0 @@
1−@string{anch-ie = {Angew.~Chem. Int.~Ed.}}
2−@string{cup = {Cambridge University Press}}
3−@string{dtv = {Deutscher Taschenbuch-Verlag}}
4−@string{hup = {Harvard University Press}}
5−@string{jams = {J.~Amer. Math. Soc.}}
6−@string{jchph = {J.~Chem. Phys.}}
7−@string{jomch = {J.~Organomet. Chem.}}
8−@string{pup = {Princeton University Press}}
9−
10−@incollection{westfahl:space,
11− author = {Westfahl, Gary},
12− title = {The True Frontier},
13− subtitle = {Confronting and Avoiding the Realities of Space in {American}
14− Science Fiction Films},
15− pages = {55-65},
16− crossref = {westfahl:frontier},
17− langid = {english},
18− langidopts = {variant=american},
19− indextitle = {True Frontier, The},
20− annotation = {A cross-referenced article from a \texttt{collection}. This is
21− an \texttt{incollection} entry with a \texttt{crossref}
22− field. Note the \texttt{subtitle} and \texttt{indextitle}
23− fields},
24−}
25−
26−@set{set,
27− entryset = {herrmann,aksin,yoon},
28− annotation = {A \texttt{set} with three members. The \texttt{crossref} field
29− in the \texttt{@set} entry and the \texttt{entryset} field in
30− each set member entry is needed only when using BibTeX as the
31− backend},
32−}
33−
34−@set{stdmodel,
35− entryset = {glashow,weinberg,salam},
36− annotation = {A \texttt{set} with three members discussing the standard
37− model of particle physics. The \texttt{crossref} field
38− in the \texttt{@set} entry and the \texttt{entryset} field in
39− each set member entry is needed only when using BibTeX as the
40− backend},
41−}
42−
43−@article{aksin,
44− author = {Aks{\i}n, {\"O}zge and T{\"u}rkmen, Hayati and Artok, Levent
45− and {\c{C}}etinkaya, Bekir and Ni, Chaoying and
46− B{\"u}y{\"u}kg{\"u}ng{\"o}r, Orhan and {\"O}zkal, Erhan},
47− title = {Effect of immobilization on catalytic characteristics of
48− saturated {Pd-N}-heterocyclic carbenes in {Mizoroki-Heck}
49− reactions},
50− journaltitle = jomch,
51− date = 2006,
52− volume = 691,
53− number = 13,
54− pages = {3027-3036},
55− indextitle = {Effect of immobilization on catalytic characteristics},
56−}
57−
58−@article{angenendt,
59− author = {Angenendt, Arnold},
60− title = {In Honore Salvatoris~-- Vom Sinn und Unsinn der
61− Patrozinienkunde},
62− journaltitle = {Revue d'Histoire Eccl{\'e}siastique},
63− date = 2002,
64− volume = 97,
65− pages = {431--456, 791--823},
66− langid = {german},
67− indextitle = {In Honore Salvatoris},
68− shorttitle = {In Honore Salvatoris},
69− annotation = {A German article in a French journal. Apart from that, a
70− typical \texttt{article} entry. Note the \texttt{indextitle}
71− field},
72−}
73−
74−@article{baez/article,
75− author = {Baez, John C. and Lauda, Aaron D.},
76− title = {Higher-Dimensional Algebra {V}: 2-Groups},
77− journaltitle = {Theory and Applications of Categories},
78− date = 2004,
79− volume = 12,
80− pages = {423-491},
81− version = 3,
82− eprint = {math/0307200v3},
83− eprinttype = {arxiv},
84− langid = {english},
85− langidopts = {variant=american},
86− annotation = {An \texttt{article} with \texttt{eprint} and
87− \texttt{eprinttype} fields. Note that the arXiv reference is
88− transformed into a clickable link if \texttt{hyperref} support
89− has been enabled. Compare \texttt{baez\slash online}, which
90− is the same item given as an \texttt{online} entry},
91−}
92−
93−@article{bertram,
94− author = {Bertram, Aaron and Wentworth, Richard},
95− title = {Gromov invariants for holomorphic maps on {Riemann} surfaces},
96− journaltitle = jams,
97− date = 1996,
98− volume = 9,
99− number = 2,
100− pages = {529-571},
101− langid = {english},
102− langidopts = {variant=american},
103− shorttitle = {Gromov invariants},
104− annotation = {An \texttt{article} entry with a \texttt{volume} and a
105− \texttt{number} field},
106−}
107−
108−@article{doody,
109− author = {Doody, Terrence},
110− title = {Hemingway's Style and {Jake's} Narration},
111− year = 1974,
112− volume = 4,
113− number = 3,
114− pages = {212-225},
115− langid = {english},
116− langidopts = {variant=american},
117− related = {matuz:doody},
118− relatedstring= {\autocap{e}xcerpt in},
119− journal = {The Journal of Narrative Technique},
120− annotation = {An \texttt{article} entry cited as an excerpt from a
121− \texttt{collection} entry. Note the format of the
122− \texttt{related} and \texttt{relatedstring} fields},
123−}
124−
125−@collection{matuz:doody,
126− editor = {Matuz, Roger},
127− title = {Contemporary Literary Criticism},
128− year = 1990,
129− volume = 61,
130− publisher = {Gale},
131− location = {Detroit},
132− pages = {204-208},
133− langid = {english},
134− langidopts = {variant=american},
135− annotation = {A \texttt{collection} entry providing the excerpt information
136− for the \texttt{doody} entry. Note the format of the
137− \texttt{pages} field},
138−}
139−
140−@article{gillies,
141− author = {Gillies, Alexander},
142− title = {Herder and the Preparation of {Goethe's} Idea of World
143− Literature},
144− journaltitle = {Publications of the English Goethe Society},
145− date = 1933,
146− series = {newseries},
147− volume = 9,
148− pages = {46-67},
149− langid = {english},
150− langidopts = {variant=british},
151− annotation = {An \texttt{article} entry with a \texttt{series} and a
152− \texttt{volume} field. Note that format of the \texttt{series}
153− field in the database file},
154−}
155−
156−@article{glashow,
157− author = {Glashow, Sheldon},
158− title = {Partial Symmetries of Weak Interactions},
159− journaltitle = {Nucl.~Phys.},
160− date = 1961,
161− volume = 22,
162− pages = {579-588},
163−}
164−
165−@article{herrmann,
166− author = {Herrmann, Wolfgang A. and {\"O}fele, Karl and Schneider,
167− Sabine K. and Herdtweck, Eberhardt and Hoffmann, Stephan D.},
168− title = {A carbocyclic carbene as an efficient catalyst ligand for {C--C}
169− coupling reactions},
170− journaltitle = anch-ie,
171− date = 2006,
172− volume = 45,
173− number = 23,
174− pages = {3859-3862},
175− indextitle = {Carbocyclic carbene as an efficient catalyst, A},
176−}
177−
178−@article{kastenholz,
179− author = {Kastenholz, M. A. and H{\"u}nenberger, Philippe H.},
180− title = {Computation of methodology\hyphen independent ionic solvation
181− free energies from molecular simulations},
182− journaltitle = jchph,
183− date = 2006,
184− subtitle = {{I}. {The} electrostatic potential in molecular liquids},
185− volume = 124,
186− eid = 124106,
187− doi = {10.1063/1.2172593},
188− langid = {english},
189− langidopts = {variant=american},
190− indextitle = {Computation of ionic solvation free energies},
191− annotation = {An \texttt{article} entry with an \texttt{eid} and a
192− \texttt{doi} field. Note that the \textsc{doi} is transformed
193− into a clickable link if \texttt{hyperref} support has been
194− enabled},
195− abstract = {The computation of ionic solvation free energies from
196− atomistic simulations is a surprisingly difficult problem that
197− has found no satisfactory solution for more than 15 years. The
198− reason is that the charging free energies evaluated from such
199− simulations are affected by very large errors. One of these is
200− related to the choice of a specific convention for summing up
201− the contributions of solvent charges to the electrostatic
202− potential in the ionic cavity, namely, on the basis of point
203− charges within entire solvent molecules (M scheme) or on the
204− basis of individual point charges (P scheme). The use of an
205− inappropriate convention may lead to a charge-independent
206− offset in the calculated potential, which depends on the
207− details of the summation scheme, on the quadrupole-moment
208− trace of the solvent molecule, and on the approximate form
209− used to represent electrostatic interactions in the
210− system. However, whether the M or P scheme (if any) represents
211− the appropriate convention is still a matter of on-going
212− debate. The goal of the present article is to settle this
213− long-standing controversy by carefully analyzing (both
214− analytically and numerically) the properties of the
215− electrostatic potential in molecular liquids (and inside
216− cavities within them).},
217−}
218−
219−@article{murray,
220− author = {Hostetler, Michael J. and Wingate, Julia E. and Zhong,
221− Chuan-Jian and Harris, Jay E. and Vachet, Richard W. and
222− Clark, Michael R. and Londono, J. David and Green, Stephen
223− J. and Stokes, Jennifer J. and Wignall, George D. and Glish,
224− Gary L. and Porter, Marc D. and Evans, Neal D. and Murray,
225− Royce W.},
226− title = {Alkanethiolate gold cluster molecules with core diameters from
227− 1.5 to 5.2~{nm}},
228− journaltitle = {Langmuir},
229− date = 1998,
230− subtitle = {Core and monolayer properties as a function of core size},
231− volume = 14,
232− number = 1,
233− pages = {17-30},
234− langid = {english},
235− langidopts = {variant=american},
236− indextitle = {Alkanethiolate gold cluster molecules},
237− shorttitle = {Alkanethiolate gold cluster molecules},
238− annotation = {An \texttt{article} entry with \arabic{author} authors. By
239− default, long author and editor lists are automatically
240− truncated. This is configurable},
241−}
242−
243−@article{reese,
244− author = {Reese, Trevor R.},
245− title = {Georgia in {Anglo-Spanish} Diplomacy, 1736--1739},
246− journaltitle = {William and Mary Quarterly},
247− date = 1958,
248− series = 3,
249− volume = 15,
250− pages = {168-190},
251− langid = {english},
252− langidopts = {variant=american},
253− annotation = {An \texttt{article} entry with a \texttt{series} and a
254− \texttt{volume} field. Note the format of the series. If the
255− value of the \texttt{series} field is an integer, this number
256− is printed as an ordinal and the string \enquote*{series} is
257− appended automatically},
258−}
259−
260−@article{sarfraz,
261− author = {M. Sarfraz and M. F. A. Razzak},
262− title = {Technical section: {An} algorithm for automatic capturing of
263− the font outlines},
264− year = 2002,
265− volume = 26,
266− number = 5,
267− pages = {795-804},
268− issn = {0097-8493},
269− journal = {Computers and Graphics},
270− annotation = {An \texttt{article} entry with an \texttt{issn} field},
271−}
272−
273−@article{shore,
274− author = {Shore, Bradd},
275− title = {Twice-Born, Once Conceived},
276− journaltitle = {American Anthropologist},
277− date = {1991-03},
278− subtitle = {Meaning Construction and Cultural Cognition},
279− series = {newseries},
280− volume = 93,
281− number = 1,
282− pages = {9-27},
283− annotation = {An \texttt{article} entry with \texttt{series},
284− \texttt{volume}, and \texttt{number} fields. Note the format
285− of the \texttt{series} which is a localization key},
286−}
287−
288−@article{sigfridsson,
289− author = {Sigfridsson, Emma and Ryde, Ulf},
290− title = {Comparison of methods for deriving atomic charges from the
291− electrostatic potential and moments},
292− journaltitle = {Journal of Computational Chemistry},
293− date = 1998,
294− volume = 19,
295− number = 4,
296− pages = {377-395},
297− doi = {10.1002/(SICI)1096-987X(199803)19:4<377::AID-JCC1>3.0.CO;2-P},
298− langid = {english},
299− langidopts = {variant=american},
300− indextitle = {Methods for deriving atomic charges},
301− annotation = {An \texttt{article} entry with \texttt{volume},
302− \texttt{number}, and \texttt{doi} fields. Note that the
303− \textsc{doi} is transformed into a clickable link if
304− \texttt{hyperref} support has been enabled},
305− abstract = {Four methods for deriving partial atomic charges from the
306− quantum chemical electrostatic potential (CHELP, CHELPG,
307− Merz-Kollman, and RESP) have been compared and critically
308− evaluated. It is shown that charges strongly depend on how and
309− where the potential points are selected. Two alternative
310− methods are suggested to avoid the arbitrariness in the
311− point-selection schemes and van der Waals exclusion radii:
312− CHELP-BOW, which also estimates the charges from the
313− electrostatic potential, but with potential points that are
314− Boltzmann-weighted after their occurrence in actual
315− simulations using the energy function of the program in which
316− the charges will be used, and CHELMO, which estimates the
317− charges directly from the electrostatic multipole
318− moments. Different criteria for the quality of the charges are
319− discussed.},
320−}
321−
322−@article{spiegelberg,
323− author = {Spiegelberg, Herbert},
324− title = {\mkbibquote{Intention} und \mkbibquote{Intentionalit{\"a}t} in
325− der Scholastik, bei Brentano und Husserl},
326− journaltitle = {Studia Philosophica},
327− date = 1969,
328− volume = 29,
329− pages = {189-216},
330− langid = {german},
331− sorttitle = {Intention und Intentionalitat in der Scholastik, bei Brentano
332− und Husserl},
333− indexsorttitle= {Intention und Intentionalitat in der Scholastik, bei Brentano
334− und Husserl},
335− shorttitle = {Intention und Intentionalit{\"a}t},
336− annotation = {An \texttt{article} entry. Note the \texttt{sorttitle} and
337− \texttt{indexsorttitle} fields and the markup of the quotes in
338− the database file},
339−}
340−
341−@article{springer,
342− author = {Springer, Otto},
343− title = {Mediaeval Pilgrim Routes from {Scandinavia} to {Rome}},
344− journaltitle = {Mediaeval Studies},
345− date = 1950,
346− volume = 12,
347− pages = {92-122},
348− langid = {english},
349− langidopts = {variant=british},
350− shorttitle = {Mediaeval Pilgrim Routes},
351− annotation = {A plain \texttt{article} entry},
352−}
353−
354−@article{weinberg,
355− author = {Weinberg, Steven},
356− title = {A Model of Leptons},
357− journaltitle = {Phys.~Rev.~Lett.},
358− date = 1967,
359− volume = 19,
360− pages = {1264-1266},
361−}
362−
363−@article{yoon,
364− author = {Yoon, Myeong S. and Ryu, Dowook and Kim, Jeongryul and Ahn,
365− Kyo Han},
366− title = {Palladium pincer complexes with reduced bond angle strain:
367− efficient catalysts for the {Heck} reaction},
368− journaltitle = {Organometallics},
369− date = 2006,
370− volume = 25,
371− number = 10,
372− pages = {2409-2411},
373− indextitle = {Palladium pincer complexes},
374−}
375−
376−@book{aristotle:anima,
377− author = {Aristotle},
378− title = {De Anima},
379− date = 1907,
380− editor = {Hicks, Robert Drew},
381− publisher = cup,
382− location = {Cambridge},
383− keywords = {primary},
384− langid = {english},
385− langidopts = {variant=british},
386− annotation = {A \texttt{book} entry with an \texttt{author} and an
387− \texttt{editor}},
388−}
389−
390−@book{aristotle:physics,
391− author = {Aristotle},
392− title = {Physics},
393− date = 1929,
394− translator = {Wicksteed, P. H. and Cornford, F. M.},
395− publisher = {G. P. Putnam},
396− location = {New York},
397− keywords = {primary},
398− langid = {english},
399− langidopts = {variant=american},
400− shorttitle = {Physics},
401− annotation = {A \texttt{book} entry with a \texttt{translator} field},
402−}
403−
404−@book{aristotle:poetics,
405− author = {Aristotle},
406− title = {Poetics},
407− date = 1968,
408− editor = {Lucas, D. W.},
409− series = {Clarendon {Aristotle}},
410− publisher = {Clarendon Press},
411− location = {Oxford},
412− keywords = {primary},
413− langid = {english},
414− langidopts = {variant=british},
415− shorttitle = {Poetics},
416− annotation = {A \texttt{book} entry with an \texttt{author} and an
417− \texttt{editor} as well as a \texttt{series} field},
418−}
419−
420−@book{aristotle:rhetoric,
421− author = {Aristotle},
422− title = {The Rhetoric of {Aristotle} with a commentary by the late {Edward
423− Meredith Cope}},
424− date = 1877,
425− editor = {Cope, Edward Meredith},
426− commentator = {Cope, Edward Meredith},
427− volumes = 3,
428− publisher = cup,
429− keywords = {primary},
430− langid = {english},
431− langidopts = {variant=british},
432− sorttitle = {Rhetoric of Aristotle},
433− indextitle = {Rhetoric of {Aristotle}, The},
434− shorttitle = {Rhetoric},
435− annotation = {A commented edition. Note the concatenation of the
436− \texttt{editor} and \texttt{commentator} fields as well as the
437− \texttt{volumes}, \texttt{sorttitle}, and \texttt{indextitle}
438− fields},
439−}
440−
441−@book{augustine,
442− author = {Augustine, Robert L.},
443− title = {Heterogeneous catalysis for the synthetic chemist},
444− date = 1995,
445− publisher = {Marcel Dekker},
446− location = {New York},
447− langid = {english},
448− langidopts = {variant=american},
449− shorttitle = {Heterogeneous catalysis},
450− annotation = {A plain \texttt{book} entry},
451−}
452−
453−@book{averroes/bland,
454− author = {Averroes},
455− title = {The Epistle on the Possibility of Conjunction with the Active
456− Intellect by {Ibn Rushd} with the Commentary of {Moses Narboni}},
457− date = 1982,
458− editor = {Bland, Kalman P.},
459− translator = {Bland, Kalman P.},
460− series = {Moreshet: Studies in {Jewish} History, Literature and Thought},
461− number = 7,
462− publisher = {Jewish Theological Seminary of America},
463− location = {New York},
464− keywords = {primary},
465− langid = {english},
466− langidopts = {variant=american},
467− indextitle = {Epistle on the Possibility of Conjunction, The},
468− shorttitle = {Possibility of Conjunction},
469− annotation = {A \texttt{book} entry with a \texttt{series} and a
470− \texttt{number}. Note the concatenation of the \texttt{editor}
471− and \texttt{translator} fields as well as the
472− \texttt{indextitle} field},
473−}
474−
475−@book{averroes/hannes,
476− author = {Averroes},
477− title = {Des Averro{\"e}s Abhandlung: \mkbibquote{{\"U}ber die
478− M{\"o}glichkeit der Conjunktion} oder \mkbibquote{{\"U}ber den
479− materiellen Intellekt}},
480− date = 1892,
481− editor = {Hannes, Ludwig},
482− translator = {Hannes, Ludwig},
483− annotator = {Hannes, Ludwig},
484− publisher = {C.~A. Kaemmerer},
485− location = {Halle an der Saale},
486− keywords = {primary},
487− langid = {german},
488− sorttitle = {Uber die Moglichkeit der Conjunktion},
489− indexsorttitle= {Uber die Moglichkeit der Conjunktion},
490− indextitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
491− shorttitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
492− annotation = {An annotated edition. Note the concatenation of the
493− \texttt{editor}, \texttt{translator}, and \texttt{annotator}
494− fields. Also note the \texttt{shorttitle},
495− \texttt{indextitle}, \texttt{sorttitle}, and
496− \texttt{indexsorttitle} fields},
497−}
498−
499−@book{averroes/hercz,
500− author = {Averroes},
501− title = {Drei Abhandlungen {\"u}ber die Conjunction des separaten
502− Intellects mit dem Menschen},
503− date = 1869,
504− editor = {Hercz, J.},
505− translator = {Hercz, J.},
506− publisher = {S.~Hermann},
507− location = {Berlin},
508− keywords = {primary},
509− langid = {german},
510− indexsorttitle= {Drei Abhandlungen uber die Conjunction},
511− indextitle = {Drei Abhandlungen {\"u}ber die Conjunction},
512− subtitle = {Von Averroes (Vater und Sohn), aus dem Arabischen
513− {\"u}bersetzt von Samuel Ibn Tibbon},
514− shorttitle = {Drei Abhandlungen},
515− annotation = {A \texttt{book} entry. Note the concatenation of the
516− \texttt{editor} and \texttt{translator} fields as well as the
517− \texttt{indextitle} and \texttt{indexsorttitle} fields},
518−}
519−
520−@book{cicero,
521− author = {Cicero, Marcus Tullius},
522− title = {De natura deorum. {\"U}ber das Wesen der G{\"o}tter},
523− date = 1995,
524− editor = {Blank-Sangmeister, Ursula},
525− translator = {Blank-Sangmeister, Ursula},
526− afterword = {Thraede, Klaus},
527− language = {langlatin and langgerman},
528− publisher = {Reclam},
529− location = {Stuttgart},
530− langid = {german},
531− indextitle = {De natura deorum},
532− shorttitle = {De natura deorum},
533− annotation = {A bilingual edition of Cicero's \emph{De natura deorum}, with
534− a German translation. Note the format of the \texttt{language}
535− field in the database file, the concatenation of the
536− \texttt{editor} and \texttt{translator} fields, and the
537− \texttt{afterword} field},
538−}
539−
540−@book{coleridge,
541− author = {Coleridge, Samuel Taylor},
542− title = {Biographia literaria, or {Biographical} sketches of my literary
543− life and opinions},
544− date = 1983,
545− editor = {Coburn, Kathleen and Engell, James and Bate, W. Jackson},
546− maintitle = {The collected works of {Samuel Taylor Coleridge}},
547− volume = 7,
548− part = 2,
549− series = {Bollingen Series},
550− number = 75,
551− publisher = {Routledge {and} Kegan Paul},
552− location = {London},
553− langid = {english},
554− langidopts = {variant=british},
555− indextitle = {Biographia literaria},
556− shorttitle = {Biographia literaria},
557− annotation = {One (partial) volume of a multivolume book. This is a
558− \texttt{book} entry with a \texttt{volume} and a \texttt{part}
559− field which explicitly refers to the second (physical) part of
560− the seventh (logical) volume. Also note the \texttt{series}
561− and \texttt{number} fields},
562−}
563−
564−@book{companion,
565− author = {Goossens, Michel and Mittelbach, Frank and Samarin, Alexander},
566− title = {The {LaTeX} Companion},
567− date = 1994,
568− edition = 1,
569− publisher = {Addison-Wesley},
570− location = {Reading, Mass.},
571− pagetotal = 528,
572− langid = {english},
573− langidopts = {variant=american},
574− sorttitle = {LaTeX Companion},
575− indextitle = {LaTeX Companion, The},
576− shorttitle = {LaTeX Companion},
577− annotation = {A book with three authors. Note the formatting of the author
578− list. By default, only the first name is reversed in the
579− bibliography},
580−}
581−
582−@book{cotton,
583− author = {Cotton, Frank Albert and Wilkinson, Geoffrey and Murillio,
584− Carlos A. and Bochmann, Manfred},
585− title = {Advanced inorganic chemistry},
586− date = 1999,
587− edition = 6,
588− publisher = {Wiley},
589− location = {Chichester},
590− langid = {english},
591− langidopts = {variant=british},
592− annotation = {A \texttt{book} entry with \arabic{author} authors and an
593− \texttt{edition} field. By default, long \texttt{author} and
594− \texttt{editor} lists are automatically truncated. This is
595− configurable},
596−}
597−
598−@book{gerhardt,
599− author = {Gerhardt, Michael J.},
600− title = {The Federal Appointments Process},
601− date = 2000,
602− publisher = {Duke University Press},
603− location = {Durham and London},
604− langid = {english},
605− langidopts = {variant=american},
606− sorttitle = {Federal Appointments Process},
607− indextitle = {Federal Appointments Process, The},
608− subtitle = {A Constitutional and Historical Analysis},
609− shorttitle = {Federal Appointments Process},
610− annotation = {This is a \texttt{book} entry. Note the format of the
611− \texttt{location} field as well as the \texttt{sorttitle} and
612− \texttt{indextitle} fields},
613−}
614−
615−@book{gonzalez,
616− author = {Gonzalez, Ray},
617− title = {The Ghost of {John Wayne} and Other Stories},
618− date = 2001,
619− publisher = {The University of Arizona Press},
620− location = {Tucson},
621− isbn = {0-816-52066-6},
622− langid = {english},
623− langidopts = {variant=american},
624− sorttitle = {Ghost of John Wayne and Other Stories},
625− indextitle = {Ghost of {John Wayne} and Other Stories, The},
626− shorttitle = {Ghost of {John Wayne}},
627− annotation = {A collection of short stories. This is a \texttt{book} entry.
628− Note the \texttt{sorttitle} and \texttt{indextitle} fields in
629− the database file. There's also an \texttt{isbn} field},
630−}
631−
632−@book{hammond,
633− author = {Hammond, Christopher},
634− title = {The basics of crystallography and diffraction},
635− date = 1997,
636− publisher = {International Union of Crystallography and Oxford University
637− Press},
638− location = {Oxford},
639− langid = {english},
640− langidopts = {variant=british},
641− sorttitle = {Basics of crystallography and diffraction},
642− indextitle = {Basics of crystallography and diffraction, The},
643− shorttitle = {Crystallography and diffraction},
644− annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
645− \texttt{indextitle} fields as well as the format of the
646− \texttt{publisher} field},
647−}
648−
649−@book{iliad,
650− author = {Homer},
651− title = {Die Ilias},
652− date = 2004,
653− translator = {Schadewaldt, Wolfgang},
654− introduction = {Latacz, Joachim},
655− edition = 3,
656− publisher = {Artemis \& Winkler},
657− location = {D{\"u}sseldorf and Z{\"u}rich},
658− langid = {german},
659− sorttitle = {Ilias},
660− indextitle = {Ilias, Die},
661− shorttitle = {Ilias},
662− annotation = {A German translation of the \emph{Iliad}. Note the
663− \texttt{translator} and \texttt{introduction} fields and the
664− format of the \texttt{location} field in the database
665− file. Also note the \texttt{sorttitle} and \texttt{indextitle}
666− fields},
667−}
668−
669−@book{knuth:ct,
670− author = {Knuth, Donald E.},
671− title = {Computers \& Typesetting},
672− date = {1984/1986},
673− volumes = 5,
674− publisher = {Addison-Wesley},
675− location = {Reading, Mass.},
676− langid = {english},
677− langidopts = {variant=american},
678− sortyear = {1984-0},
679− sorttitle = {Computers & Typesetting},
680− indexsorttitle= {Computers & Typesetting},
681− annotation = {A five-volume book cited as a whole. This is a \texttt{book}
682− entry, note the \texttt{volumes} field},
683−}
684−
685−@book{knuth:ct:a,
686− author = {Knuth, Donald E.},
687− title = {The {\TeX} book},
688− date = 1984,
689− maintitle = {Computers \& Typesetting},
690− volume = {A},
691− publisher = {Addison-Wesley},
692− location = {Reading, Mass.},
693− langid = {english},
694− langidopts = {variant=american},
695− sortyear = {1984-1},
696− sorttitle = {Computers & Typesetting A},
697− indexsorttitle= {The TeXbook},
698− indextitle = {\protect\TeX book, The},
699− shorttitle = {\TeX book},
700− annotation = {The first volume of a five-volume book. Note the
701− \texttt{sorttitle} and \texttt{sortyear} fields. We want this
702− volume to be listed after the entry referring to the entire
703− five-volume set. Also note the \texttt{indextitle} and
704− \texttt{indexsorttitle} fields. Indexing packages that don't
705− generate robust index entries require some control sequences
706− to be protected from expansion},
707−}
708−
709−@book{knuth:ct:b,
710− author = {Knuth, Donald E.},
711− title = {\TeX: The Program},
712− date = 1986,
713− maintitle = {Computers \& Typesetting},
714− volume = {B},
715− publisher = {Addison-Wesley},
716− location = {Reading, Mass.},
717− langid = {english},
718− langidopts = {variant=american},
719− sortyear = {1986-1},
720− sorttitle = {Computers & Typesetting B},
721− indexsorttitle= {TeX: The Program},
722− shorttitle = {\TeX},
723− annotation = {The second volume of a five-volume book. Note the
724− \texttt{sorttitle} and \texttt{sortyear} fields. Also note the
725− \texttt{indexsorttitle} field},
726−}
727−
728−@book{knuth:ct:c,
729− author = {Knuth, Donald E.},
730− title = {The {METAFONTbook}},
731− date = 1986,
732− maintitle = {Computers \& Typesetting},
733− volume = {C},
734− publisher = {Addison-Wesley},
735− location = {Reading, Mass.},
736− langid = {english},
737− langidopts = {variant=american},
738− sortyear = {1986-2},
739− sorttitle = {Computers & Typesetting C},
740− indextitle = {METAFONTbook, The},
741− shorttitle = {METAFONTbook},
742− annotation = {The third volume of a five-volume book. Note the
743− \texttt{sorttitle} and \texttt{sortyear} fields as well as the
744− \texttt{indextitle} field},
745−}
746−
747−@book{knuth:ct:d,
748− author = {Knuth, Donald E.},
749− title = {METAFONT: The Program},
750− date = 1986,
751− maintitle = {Computers \& Typesetting},
752− volume = {D},
753− publisher = {Addison-Wesley},
754− location = {Reading, Mass.},
755− langid = {english},
756− langidopts = {variant=american},
757− sortyear = {1986-3},
758− sorttitle = {Computers & Typesetting D},
759− shorttitle = {METAFONT},
760− annotation = {The fourth volume of a five-volume book. Note the
761− \texttt{sorttitle} and \texttt{sortyear} fields},
762−}
763−
764−@book{knuth:ct:e,
765− author = {Knuth, Donald E.},
766− title = {{Computer Modern} Typefaces},
767− date = 1986,
768− maintitle = {Computers \& Typesetting},
769− volume = {E},
770− publisher = {Addison-Wesley},
771− location = {Reading, Mass.},
772− langid = {english},
773− langidopts = {variant=american},
774− sortyear = {1986-4},
775− sorttitle = {Computers & Typesetting E},
776− annotation = {The fifth volume of a five-volume book. Note the
777− \texttt{sorttitle} and \texttt{sortyear} fields},
778−}
779−
780−@book{knuth:ct:related,
781− author = {Knuth, Donald E.},
782− title = {Computers \& Typesetting},
783− date = {1984/1986},
784− volumes = 5,
785− publisher = {Addison-Wesley},
786− location = {Reading, Mass.},
787− langid = {english},
788− langidopts = {variant=american},
789− sortyear = {1984-0},
790− sorttitle = {Computers & Typesetting},
791− indexsorttitle= {Computers & Typesetting},
792− related = {knuth:ct:a,knuth:ct:b,knuth:ct:c,knuth:ct:d,knuth:ct:e},
793− relatedtype = {multivolume},
794− annotation = {A five-volume book cited as a whole and related to its
795− individual volumes. Note the \texttt{related} and
796− \texttt{relatedtype} fields},
797−}
798−
799−@book{kullback,
800− author = {Kullback, Solomon},
801− title = {Information Theory and Statistics},
802− year = 1959,
803− publisher = {John Wiley \& Sons},
804− location = {New York},
805− langid = {english},
806− langidopts = {variant=american},
807−}
808−
809−@book{kullback:reprint,
810− author = {Kullback, Solomon},
811− title = {Information Theory and Statistics},
812− year = 1997,
813− publisher = {Dover Publications},
814− location = {New York},
815− origyear = 1959,
816− origpublisher= {John Wiley \& Sons},
817− langid = {english},
818− langidopts = {variant=american},
819− annotation = {A reprint of the \texttt{kullback} entry. Note the format of
820− \texttt{origyear} and \texttt{origpublisher}. These fields are
821− not used by the standard bibliography styles},
822−}
823−
824−@book{kullback:related,
825− author = {Kullback, Solomon},
826− title = {Information Theory and Statistics},
827− year = 1997,
828− publisher = {Dover Publications},
829− location = {New York},
830− langid = {english},
831− langidopts = {variant=american},
832− related = {kullback},
833− relatedtype = {origpubin},
834− annotation = {A reprint of the \texttt{kullback} entry. Note the format of
835− the \texttt{related} and \texttt{relatedtype} fields},
836−}
837−
838−@book{malinowski,
839− author = {Malinowski, Bronis{\l}aw},
840− title = {Argonauts of the {Western Pacific}},
841− date = 1972,
842− edition = 8,
843− publisher = {Routledge {and} Kegan Paul},
844− location = {London},
845− langid = {english},
846− langidopts = {variant=british},
847− subtitle = {An account of native enterprise and adventure in the
848− Archipelagoes of {Melanesian New Guinea}},
849− shorttitle = {Argonauts},
850− annotation = {This is a \texttt{book} entry. Note the format of the
851− \texttt{publisher} and \texttt{edition} fields as well as the
852− \texttt{subtitle} field},
853−}
854−
855−@book{maron,
856− author = {Maron, Monika},
857− title = {Animal Triste},
858− date = 2000,
859− translator = {Brigitte Goldstein},
860− origlanguage = {german},
861− publisher = {University of Nebraska Press},
862− location = {Lincoln},
863− langid = {english},
864− langidopts = {variant=american},
865− shorttitle = {Animal Triste},
866− annotation = {An English translation of a German novel with a French title.
867− In other words: a \texttt{book} entry with a
868− \texttt{translator} field. Note the \texttt{origlanguage}
869− field which is concatenated with the \texttt{translator}},
870−}
871−
872−@book{massa,
873− author = {Werner Massa},
874− title = {Crystal structure determination},
875− date = 2004,
876− edition = 2,
877− publisher = {Spinger},
878− location = {Berlin},
879− langid = {english},
880− langidopts = {variant=british},
881− annotation = {A \texttt{book} entry with an \texttt{edition} field},
882−}
883−
884−@article{moore,
885− author = {Moore, Gordon E.},
886− title = {Cramming more components onto integrated circuits},
887− journaltitle = {Electronics},
888− year = 1965,
889− volume = 38,
890− number = 8,
891− pages = {114-117},
892− langid = {english},
893− langidopts = {variant=american},
894−}
895−
896−@article{moore:related,
897− author = {Moore, Gordon E.},
898− title = {Cramming more components onto integrated circuits},
899− journaltitle = {Proceedings of the {IEEE}},
900− year = 1998,
901− volume = 86,
902− number = 1,
903− pages = {82-85},
904− langid = {english},
905− langidopts = {variant=american},
906− related = {moore},
907− relatedtype = {reprintfrom},
908− annotation = {A reprint of Moore's law. Note the \texttt{related} and
909− \texttt{relatedtype} fields},
910−}
911−
912−@book{nietzsche:ksa,
913− author = {Nietzsche, Friedrich},
914− title = {S{\"a}mtliche Werke},
915− date = 1988,
916− editor = {Colli, Giorgio and Montinari, Mazzino},
917− edition = 2,
918− volumes = 15,
919− publisher = dtv # { and Walter de Gruyter},
920− location = {M{\"u}nchen and Berlin and New York},
921− langid = {german},
922− sortyear = {1988-0},
923− sorttitle = {Werke-00-000},
924− indexsorttitle= {Samtliche Werke},
925− subtitle = {Kritische Studienausgabe},
926− annotation = {The critical edition of Nietzsche's works. This is a
927− \texttt{book} entry referring to a 15-volume work as a
928− whole. Note the \texttt{volumes} field and the format of the
929− \texttt{publisher} and \texttt{location} fields in the
930− database file. Also note the \texttt{sorttitle} and
931− \texttt{sortyear} fields which are used to fine-tune the
932− sorting order of the bibliography. We want this item listed
933− first in the bibliography},
934−}
935−
936−@book{nietzsche:ksa1,
937− author = {Nietzsche, Friedrich},
938− title = {Die Geburt der Trag{\"o}die. Unzeitgem{\"a}{\ss}e
939− Betrachtungen I--IV. Nachgelassene Schriften 1870--1973},
940− date = 1988,
941− editor = {Colli, Giorgio and Montinari, Mazzino},
942− maintitle = {S{\"a}mtliche Werke},
943− mainsubtitle = {Kritische Studienausgabe},
944− volume = 1,
945− edition = 2,
946− publisher = dtv # { and Walter de Gruyter},
947− location = {M{\"u}nchen and Berlin and New York},
948− langid = {german},
949− sortyear = {1988-1},
950− sorttitle = {Werke-01-000},
951− indexsorttitle= {Samtliche Werke I},
952− bookauthor = {Nietzsche, Friedrich},
953− indextitle = {S{\"a}mtliche Werke I},
954− shorttitle = {S{\"a}mtliche Werke I},
955− annotation = {A single volume from the critical edition of Nietzsche's
956− works. This \texttt{book} entry explicitly refers to the first
957− volume only. Note the \texttt{title} and \texttt{maintitle}
958− fields. Also note the \texttt{sorttitle} and \texttt{sortyear}
959− fields. We want this entry to be listed after the entry
960− referring to the entire edition},
961−}
962−
963−@book{nussbaum,
964− author = {Nussbaum, Martha},
965− title = {Aristotle's \mkbibquote{De Motu Animalium}},
966− date = 1978,
967− publisher = pup,
968− location = {Princeton},
969− keywords = {secondary},
970− langid = {english},
971− langidopts = {variant=american},
972− sorttitle = {Aristotle's De Motu Animalium},
973− indexsorttitle= {Aristotle's De Motu Animalium},
974− annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
975− \texttt{indexsorttitle} fields and the markup of the quotes in
976− the database file},
977−}
978−
979−@book{piccato,
980− author = {Piccato, Pablo},
981− title = {City of Suspects},
982− date = 2001,
983− publisher = {Duke University Press},
984− location = {Durham and London},
985− langid = {english},
986− langidopts = {variant=american},
987− subtitle = {Crime in {Mexico City}, 1900--1931},
988− shorttitle = {City of Suspects},
989− annotation = {This is a \texttt{book} entry. Note the format of the
990− \texttt{location} field in the database file},
991−}
992−
993−@book{vangennep,
994− author = {van Gennep, Arnold},
995− title = {Les rites de passage},
996− date = 1909,
997− publisher = {Nourry},
998− location = {Paris},
999− options = {useprefix},
1000− langid = {french},
1001− sorttitle = {Rites de passage},
1002− indextitle = {Rites de passage, Les},
1003− shorttitle = {Rites de passage},
1004− annotation = {A \texttt{book} entry. Note the format of the printed name and
1005− compare the \texttt{useprefix} option in the \texttt{options}
1006− field as well as \texttt{brandt} and \texttt{geer}},
1007−}
1008−
1009−@book{vangennep:trans,
1010− author = {van Gennep, Arnold},
1011− title = {The Rites of Passage},
1012− year = 1960,
1013− translator = {Vizedom, Monika B. and Caffee, Gabrielle L.},
1014− language = {english},
1015− origlanguage = {french},
1016− publisher = {University of Chicago Press},
1017− options = {useprefix},
1018− indextitle = {Rites of Passage, The},
1019− sorttitle = {Rites of Passage},
1020− shorttitle = {Rites of Passage},
1021− langid = {english},
1022− langidopts = {variant=american},
1023− annotation = {A translation of the \texttt{vangennep} entry. Note the
1024− \texttt{translator} and \texttt{origlanguage} fields. Compare
1025− with the \texttt{vangennep:related} entry.},
1026−}
1027−
1028−@book{vangennep:related,
1029− author = {van Gennep, Arnold},
1030− title = {Les rites de passage},
1031− date = 1909,
1032− publisher = {Nourry},
1033− location = {Paris},
1034− options = {useprefix},
1035− langid = {french},
1036− related = {vizedom:related},
1037− relatedtype = {bytranslator},
1038− sorttitle = {Rites de passage},
1039− indextitle = {Rites de passage, Les},
1040− shorttitle = {Rites de passage},
1041− annotation = {A variant of the \texttt{vangennep} entry related to its
1042− translation. Note the format of the \texttt{related} and
1043− \texttt{relatedtype} fields},
1044−}
1045−
1046−@book{vizedom:related,
1047− title = {The Rites of Passage},
1048− year = 1960,
1049− translator = {Vizedom, Monika B. and Caffee, Gabrielle L.},
1050− language = {english},
1051− publisher = {University of Chicago Press},
1052− langid = {english},
1053− langidopts = {variant=american},
1054− options = {usetranslator},
1055− related = {vangennep},
1056− relatedtype = {translationof},
1057− indextitle = {Rites of Passage, The},
1058− sorttitle = {Rites of Passage},
1059− shorttitle = {Rites of Passage},
1060− annotation = {A translated work from \texttt{vangennep}. Note the format of
1061− the \texttt{related} and \texttt{relatedtype} fields},
1062−}
1063−
1064−@book{vazques-de-parga,
1065− author = {V{\'a}zques{ de }Parga, Luis and Lacarra, Jos{\'e} Mar{\'i}a
1066− and Ur{\'i}a R{\'i}u, Juan},
1067− title = {Las Peregrinaciones a Santiago de Compostela},
1068− date = 1993,
1069− volumes = 3,
1070− note = {Ed. facs. de la realizada en 1948--49},
1071− publisher = {Iberdrola},
1072− location = {Pamplona},
1073− langid = {spanish},
1074− sorttitle = {Peregrinaciones a Santiago de Compostela},
1075− indextitle = {Peregrinaciones a Santiago de Compostela, Las},
1076− shorttitle = {Peregrinaciones},
1077− annotation = {A multivolume book cited as a whole. This is a \texttt{book}
1078− entry with \texttt{volumes}, \texttt{note},
1079− \texttt{sorttitle}, and \texttt{indextitle} fields},
1080−}
1081−
1082−@book{wilde,
1083− author = {Wilde, Oscar},
1084− title = {The Importance of Being Earnest: A Trivial Comedy for Serious
1085− People},
1086− year = 1899,
1087− series = {English and {American} drama of the Nineteenth Century},
1088− publisher = {Leonard Smithers {and} Company},
1089− eprint = {4HIWAAAAYAAJ},
1090− eprinttype = {googlebooks},
1091− annotation = {A \texttt{book} with \texttt{eprint} and \texttt{eprinttype}
1092− fields.},
1093−}
1094−
1095−@book{worman,
1096− author = {Worman, Nancy},
1097− title = {The Cast of Character},
1098− date = 2002,
1099− publisher = {University of Texas Press},
1100− location = {Austin},
1101− langid = {english},
1102− langidopts = {variant=american},
1103− sorttitle = {Cast of Character},
1104− indextitle = {Cast of Character, The},
1105− subtitle = {Style in {Greek} Literature},
1106− shorttitle = {Cast of Character},
1107− annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
1108− \texttt{indextitle} fields},
1109−}
1110−
1111−@collection{britannica,
1112− editor = {Preece, Warren E.},
1113− title = {The {New Encyclop{\ae}dia Britannica}},
1114− date = 2003,
1115− edition = 15,
1116− volumes = 32,
1117− publisher = {Encyclop{\ae}dia Britannica},
1118− location = {Chicago, Ill.},
1119− options = {useeditor=false},
1120− label = {EB},
1121− langid = {english},
1122− langidopts = {variant=british},
1123− sorttitle = {Encyclop{\ae}dia Britannica},
1124− indextitle = {{Encyclop{\ae}dia Britannica}, The {New}},
1125− shorttitle = {{Encyclop{\ae}dia Britannica}},
1126− annotation = {This is a \texttt{collection} entry for an encyclopedia. Note
1127− the \texttt{useeditor} option in the \texttt{options} field as
1128− well as the \texttt{sorttitle} field. We want this entry to be
1129− cited and alphabetized by title even though there is an
1130− editor. In addition to that, we want the title to be
1131− alphabetized under \enquote*{E} rather than \enquote*{T}. Also
1132− note the \texttt{label} field which is provided for
1133− author-year citation styles},
1134−}
1135−
1136−@collection{gaonkar,
1137− editor = {Gaonkar, Dilip Parameshwar},
1138− title = {Alternative Modernities},
1139− date = 2001,
1140− publisher = {Duke University Press},
1141− location = {Durham and London},
1142− isbn = {0-822-32714-7},
1143− langid = {english},
1144− langidopts = {variant=american},
1145− annotation = {This is a \texttt{collection} entry. Note the format of the
1146− \texttt{location} field in the database file as well as the
1147− \texttt{isbn} field},
1148−}
1149−
1150−@InCollection{gaonkar:in,
1151− author = {Gaonkar, Dilip Parameshwar},
1152− editor = {Gaonkar, Dilip Parameshwar},
1153− title = {On Alternative Modernities},
1154− date = 2001,
1155− booktitle = {Alternative Modernities},
1156− publisher = {Duke University Press},
1157− location = {Durham and London},
1158− isbn = {0-822-32714-7},
1159− pages = {1-23},
1160−}
1161−
1162−@collection{jaffe,
1163− editor = {Jaff{\'e}, Philipp},
1164− title = {Regesta Pontificum Romanorum ab condita ecclesia ad annum post
1165− Christum natum \textsc{mcxcviii}},
1166− date = {1885/1888},
1167− editora = {Loewenfeld, Samuel and Kaltenbrunner, Ferdinand and Ewald,
1168− Paul},
1169− edition = 2,
1170− volumes = 2,
1171− location = {Leipzig},
1172− langid = {latin},
1173− editoratype = {redactor},
1174− indextitle = {Regesta Pontificum Romanorum},
1175− shorttitle = {Regesta Pontificum Romanorum},
1176− annotation = {A \texttt{collection} entry with \texttt{edition} and
1177− \texttt{volumes} fields. Note the \texttt{editora} and
1178− \texttt{editoratype} fields},
1179−}
1180−
1181−@collection{westfahl:frontier,
1182− editor = {Westfahl, Gary},
1183− title = {Space and Beyond},
1184− date = 2000,
1185− subtitle = {The Frontier Theme in Science Fiction},
1186− publisher = {Greenwood},
1187− location = {Westport, Conn. and London},
1188− langid = {english},
1189− langidopts = {variant=american},
1190− booktitle = {Space and Beyond},
1191− booksubtitle = {The Frontier Theme in Science Fiction},
1192− annotation = {This is a \texttt{collection} entry. Note the format of the
1193− \texttt{location} field as well as the \texttt{subtitle} and
1194− \texttt{booksubtitle} fields},
1195−}
1196−
1197−@inbook{kant:kpv,
1198− title = {Kritik der praktischen Vernunft},
1199− date = 1968,
1200− author = {Kant, Immanuel},
1201− booktitle = {Kritik der praktischen Vernunft. Kritik der Urtheilskraft},
1202− bookauthor = {Kant, Immanuel},
1203− maintitle = {Kants Werke. Akademie Textausgabe},
1204− volume = 5,
1205− publisher = {Walter de Gruyter},
1206− location = {Berlin},
1207− pages = {1-163},
1208− shorthand = {KpV},
1209− langid = {german},
1210− shorttitle = {Kritik der praktischen Vernunft},
1211− annotation = {An edition of Kant's \emph{Collected Works}, volume five. This
1212− is an \texttt{inbook} entry which explicitly refers to the
1213− \emph{Critique of Practical Reason} only, not to the entire
1214− fifth volume. Note the \texttt{author} and \texttt{bookauthor}
1215− fields in the database file. By default, the
1216− \texttt{bookauthor} is omitted if the values of the
1217− \texttt{author} and \texttt{bookauthor} fields are identical},
1218−}
1219−
1220−@inbook{kant:ku,
1221− title = {Kritik der Urtheilskraft},
1222− date = 1968,
1223− author = {Kant, Immanuel},
1224− booktitle = {Kritik der praktischen Vernunft. Kritik der Urtheilskraft},
1225− bookauthor = {Kant, Immanuel},
1226− maintitle = {Kants Werke. Akademie Textausgabe},
1227− volume = 5,
1228− publisher = {Walter de Gruyter},
1229− location = {Berlin},
1230− pages = {165-485},
1231− shorthand = {KU},
1232− langid = {german},
1233− annotation = {An edition of Kant's \emph{Collected Works}, volume five. This
1234− is an \texttt{inbook} entry which explicitly refers to the
1235− \emph{Critique of Judgment} only, not to the entire fifth
1236− volume},
1237−}
1238−
1239−@inbook{nietzsche:historie,
1240− title = {Unzeitgem{\"a}sse Betrachtungen. Zweites St{\"u}ck},
1241− date = 1988,
1242− author = {Nietzsche, Friedrich},
1243− booktitle = {Die Geburt der Trag{\"o}die. Unzeitgem{\"a}{\ss}e
1244− Betrachtungen I--IV. Nachgelassene Schriften 1870--1973},
1245− bookauthor = {Nietzsche, Friedrich},
1246− editor = {Colli, Giorgio and Montinari, Mazzino},
1247− subtitle = {Vom Nutzen und Nachtheil der Historie f{\"u}r das Leben},
1248− maintitle = {S{\"a}mtliche Werke},
1249− mainsubtitle = {Kritische Studienausgabe},
1250− volume = 1,
1251− publisher = dtv # { and Walter de Gruyter},
1252− location = {M{\"u}nchen and Berlin and New York},
1253− pages = {243-334},
1254− langid = {german},
1255− sortyear = {1988-2},
1256− sorttitle = {Werke-01-243},
1257− indexsorttitle= {Vom Nutzen und Nachtheil der Historie fur das Leben},
1258− indextitle = {Vom Nutzen und Nachtheil der Historie f{\"u}r das Leben},
1259− shorttitle = {Vom Nutzen und Nachtheil der Historie},
1260− annotation = {A single essay from the critical edition of Nietzsche's works.
1261− This \texttt{inbook} entry explicitly refers to an essay found
1262− in the first volume. Note the \texttt{title},
1263− \texttt{booktitle}, and \texttt{maintitle} fields. Also note
1264− the \texttt{sorttitle} and \texttt{sortyear} fields. We want
1265− this entry to be listed after the entry referring to the
1266− entire first volume},
1267−}
1268−
1269−@incollection{brandt,
1270− author = {von Brandt, Ahasver and Erich Hoffmann},
1271− editor = {Ferdinand Seibt},
1272− title = {Die nordischen L{\"a}nder von der Mitte des 11.~Jahrhunderts
1273− bis 1448},
1274− date = 1987,
1275− booktitle = {Europa im Hoch- und Sp{\"a}tmittelalter},
1276− series = {Handbuch der europ{\"a}ischen Geschichte},
1277− number = 2,
1278− publisher = {Klett-Cotta},
1279− location = {Stuttgart},
1280− pages = {884-917},
1281− options = {useprefix=false},
1282− langid = {german},
1283− indexsorttitle= {Nordischen Lander von der Mitte des 11. Jahrhunderts bis
1284− 1448},
1285− indextitle = {Nordischen L{\"a}nder von der Mitte des 11.~Jahrhunderts bis
1286− 1448, Die},
1287− shorttitle = {Die nordischen L{\"a}nder},
1288− annotation = {An \texttt{incollection} entry with a \texttt{series} and a
1289− \texttt{number}. Note the format of the printed name and
1290− compare the \texttt{useprefix} option in the \texttt{options}
1291− field as well as \texttt{vangennep}. Also note the
1292− \texttt{indextitle, and \texttt{indexsorttitle} fields}},
1293−}
1294−
1295−@incollection{hyman,
1296− author = {Arthur Hyman},
1297− editor = {O'Meara, Dominic J.},
1298− title = {Aristotle's Theory of the Intellect and its Interpretation by
1299− {Averroes}},
1300− date = 1981,
1301− booktitle = {Studies in {Aristotle}},
1302− series = {Studies in Philosophy and the History of Philosophy},
1303− number = 9,
1304− publisher = {The Catholic University of America Press},
1305− location = {Washington, D.C.},
1306− pages = {161-191},
1307− keywords = {secondary},
1308− langid = {english},
1309− langidopts = {variant=american},
1310− indextitle = {Aristotle's Theory of the Intellect},
1311− shorttitle = {Aristotle's Theory of the Intellect},
1312− annotation = {An \texttt{incollection} entry with a \texttt{series} and
1313− \texttt{number} field},
1314−}
1315−
1316−@incollection{pines,
1317− author = {Pines, Shlomo},
1318− editor = {Twersky, Isadore},
1319− title = {The Limitations of Human Knowledge According to {Al-Farabi}, {ibn
1320− Bajja}, and {Maimonides}},
1321− date = 1979,
1322− booktitle = {Studies in Medieval {Jewish} History and Literature},
1323− publisher = hup,
1324− location = {Cambridge, Mass.},
1325− pages = {82-109},
1326− keywords = {secondary},
1327− langid = {english},
1328− langidopts = {variant=american},
1329− indextitle = {Limitations of Human Knowledge According to {Al-Farabi}, {ibn
1330− Bajja}, and {Maimonides}, The},
1331− shorttitle = {Limitations of Human Knowledge},
1332− annotation = {A typical \texttt{incollection} entry. Note the
1333− \texttt{indextitle} field},
1334−}
1335−
1336−@inproceedings{moraux,
1337− author = {Moraux, Paul},
1338− editor = {Lloyd, G. E. R. and Owen, G. E. L.},
1339− title = {Le \emph{De Anima} dans la tradition gr{\`e}cque},
1340− date = 1979,
1341− booktitle = {Aristotle on Mind and the Senses},
1342− subtitle = {Quelques aspects de l'interpretation du trait{\'e}, de
1343− Theophraste {\`a} Themistius},
1344− booktitleaddon= {Proceedings of the Seventh Symposium Aristotelicum},
1345− eventdate = 1975,
1346− publisher = cup,
1347− location = {Cambridge},
1348− pages = {281-324},
1349− keywords = {secondary},
1350− langid = {french},
1351− indexsorttitle= {De Anima dans la tradition grecque},
1352− indextitle = {\emph{De Anima} dans la tradition gr{\`e}cque, Le},
1353− shorttitle = {\emph{De Anima} dans la tradition gr{\`e}cque},
1354− annotation = {This is a typical \texttt{inproceedings} entry. Note the
1355− \texttt{booksubtitle}, \texttt{shorttitle},
1356− \texttt{indextitle}, and \texttt{indexsorttitle} fields. Also
1357− note the \texttt{eventdate} field.},
1358−}
1359−
1360−@inproceedings{salam,
1361− author = {Salam, Abdus},
1362− editor = {Svartholm, Nils},
1363− title = {Weak and Electromagnetic Interactions},
1364− date = 1968,
1365− booktitle = {Elementary particle theory},
1366− booksubtitle = {Relativistic groups and analyticity},
1367− booktitleaddon= {Proceedings of the {Eighth Nobel Symposium}},
1368− eventdate = {1968-05-19/1968-05-25},
1369− venue = {Aspen{\"a}sgarden, Lerum},
1370− publisher = {Almquist \& Wiksell},
1371− location = {Stockholm},
1372− pages = {367-377},
1373−}
1374−
1375−@manual{cms,
1376− title = {The {Chicago} Manual of Style},
1377− date = 2003,
1378− subtitle = {The Essential Guide for Writers, Editors, and Publishers},
1379− edition = 15,
1380− publisher = {University of Chicago Press},
1381− location = {Chicago, Ill.},
1382− isbn = {0-226-10403-6},
1383− label = {CMS},
1384− langid = {english},
1385− langidopts = {variant=american},
1386− sorttitle = {Chicago Manual of Style},
1387− indextitle = {Chicago Manual of Style, The},
1388− shorttitle = {Chicago Manual of Style},
1389− annotation = {This is a \texttt{manual} entry without an \texttt{author} or
1390− \texttt{editor}. Note the \texttt{label} field in the database
1391− file which is provided for author-year citation styles. Also
1392− note the \texttt{sorttitle} and \texttt{indextitle} fields. By
1393− default, all entries without an \texttt{author} or
1394− \texttt{editor} are alphabetized by \texttt{title} but we want
1395− this entry to be alphabetized under \enquote*{C} rather than
1396− \enquote*{T}. There's also an \texttt{isbn} field},
1397−}
1398−
1399−@online{baez/online,
1400− author = {Baez, John C. and Lauda, Aaron D.},
1401− title = {Higher-Dimensional Algebra {V}: 2-Groups},
1402− date = {2004-10-27},
1403− version = 3,
1404− langid = {english},
1405− langidopts = {variant=american},
1406− eprinttype = {arxiv},
1407− eprint = {math/0307200v3},
1408− annotation = {An \texttt{online} reference from arXiv. Note the
1409− \texttt{eprint} and \texttt{eprinttype} fields. Compare
1410− \texttt{baez\slash article} which is the same item given as an
1411− \texttt{article} entry with eprint information},
1412−}
1413−
1414−@online{ctan,
1415− title = {CTAN},
1416− date = 2006,
1417− url = {http://www.ctan.org},
1418− subtitle = {The {Comprehensive TeX Archive Network}},
1419− urldate = {2006-10-01},
1420− label = {CTAN},
1421− langid = {english},
1422− langidopts = {variant=american},
1423− annotation = {This is an \texttt{online} entry. The \textsc{url}, which is
1424− given in the \texttt{url} field, is transformed into a
1425− clickable link if \texttt{hyperref} support has been
1426− enabled. Note the format of the \texttt{urldate} field
1427− (\texttt{yyyy-mm-dd}) in the database file. Also note the
1428− \texttt{label} field which may be used as a fallback by
1429− citation styles which need an \texttt{author} and\slash or a
1430− \texttt{year}},
1431−}
1432−
1433−@online{itzhaki,
1434− author = {Itzhaki, Nissan},
1435− title = {Some remarks on {'t Hooft's} {S}-matrix for black holes},
1436− date = {1996-03-11},
1437− version = 1,
1438− langid = {english},
1439− langidopts = {variant=american},
1440− eprinttype = {arxiv},
1441− eprint = {hep-th/9603067},
1442− annotation = {An \texttt{online} reference from arXiv. Note the
1443− \texttt{eprint} and \texttt{eprinttype} fields. Also note that
1444− the arXiv reference is transformed into a clickable link if
1445− \texttt{hyperref} support has been enabled},
1446− abstract = {We discuss the limitations of 't Hooft's proposal for the
1447− black hole S-matrix. We find that the validity of the S-matrix
1448− implies violation of the semi-classical approximation at
1449− scales large compared to the Planck scale. We also show that
1450− the effect of the centrifugal barrier on the S-matrix is
1451− crucial even for large transverse distances.},
1452−}
1453−
1454−@online{markey,
1455− author = {Markey, Nicolas},
1456− title = {Tame the {BeaST}},
1457− date = {2005-10-16},
1458− url = {http://mirror.ctan.org/info/bibtex/tamethebeast/ttb_en.pdf},
1459− subtitle = {The {B} to {X} of {BibTeX}},
1460− version = {1.3},
1461− urldate = {2006-10-01},
1462− langid = {english},
1463− langidopts = {variant=american},
1464− sorttitle = {Tame the Beast},
1465− annotation = {An \texttt{online} entry for a tutorial. Note the format of
1466− the \texttt{date} field (\texttt{yyyy-mm-dd}) in the database
1467− file.},
1468−}
1469−
1470−@online{wassenberg,
1471− author = {Wassenberg, Jan and Sanders, Peter},
1472− title = {Faster Radix Sort via Virtual Memory and Write-Combining},
1473− date = {2010-08-17},
1474− version = 1,
1475− langid = {english},
1476− langidopts = {variant=american},
1477− eprinttype = {arxiv},
1478− eprintclass = {cs.DS},
1479− eprint = {1008.2849v1},
1480− annotation = {A recent \texttt{online} reference from arXiv using the new
1481− (April 2007 onward) identifier format. Note the
1482− \texttt{eprint}, \texttt{eprinttype}, and \texttt{eprintclass}
1483− fields. Also note that the arXiv reference is transformed into
1484− a clickable link if \texttt{hyperref} support has been
1485− enabled},
1486− abstract = {Sorting algorithms are the deciding factor for the performance
1487− of common operations such as removal of duplicates or database
1488− sort-merge joins. This work focuses on 32-bit integer keys,
1489− optionally paired with a 32-bit value. We present a fast radix
1490− sorting algorithm that builds upon a microarchitecture-aware
1491− variant of counting sort},
1492−}
1493−
1494−@patent{almendro,
1495− author = {Almendro, Jos{\'e} L. and Mart{\'i}n, Jacinto and S{\'a}nchez,
1496− Alberto and Nozal, Fernando},
1497− title = {Elektromagnetisches Signalhorn},
1498− number = {EU-29702195U},
1499− date = 1998,
1500− location = {countryfr and countryuk and countryde},
1501− langid = {german},
1502− annotation = {This is a \texttt{patent} entry with a \texttt{location}
1503− field. The number is given in the \texttt{number} field. Note
1504− the format of the \texttt{location} field in the database
1505− file. Compare \texttt{laufenberg}, \texttt{sorace}, and
1506− \texttt{kowalik}},
1507−}
1508−
1509−@patent{kowalik,
1510− author = {Kowalik, F. and Isard, M.},
1511− title = {Estimateur d'un d{\'e}faut de fonctionnement d'un modulateur
1512− en quadrature et {\'e}tage de modulation l'utilisant},
1513− number = 9500261,
1514− date = {1995-01-11},
1515− type = {patreqfr},
1516− langid = {french},
1517− indextitle = {Estimateur d'un d{\'e}faut de fonctionnement},
1518− annotation = {This is a \texttt{patent} entry for a French patent request
1519− with a full date. The number is given in the \texttt{number}
1520− field. Note the format of the \texttt{type} and \texttt{date}
1521− fields in the database file. Compare \texttt{almendro},
1522− \texttt{laufenberg}, and \texttt{sorace}},
1523−}
1524−
1525−@patent{laufenberg,
1526− author = {Laufenberg, Xaver and Eynius, Dominique and Suelzle, Helmut
1527− and Usbeck, Stephan and Spaeth, Matthias and Neuser-Hoffmann,
1528− Miriam and Myrzik, Christian and Schmid, Manfred and Nietfeld,
1529− Franz and Thiel, Alexander and Braun, Harald and Ebner,
1530− Norbert},
1531− title = {Elektrische Einrichtung und Betriebsverfahren},
1532− number = 1700367,
1533− date = {2006-09-13},
1534− holder = {{Robert Bosch GmbH} and {Daimler Chrysler AG} and {Bayerische
1535− Motoren Werke AG}},
1536− type = {patenteu},
1537− langid = {german},
1538− annotation = {This is a \texttt{patent} entry with a \texttt{holder} field.
1539− Note the format of the \texttt{type} and \texttt{location}
1540− fields in the database file. Compare \texttt{almendro},
1541− \texttt{sorace}, and \texttt{kowalik}},
1542− abstract = {The invention relates to an electric device comprising a
1543− generator, in particular for use in the vehicle electric
1544− system of a motor vehicle and a controller for controlling the
1545− generator voltage. The device is equipped with a control zone,
1546− in which the voltage is controlled and zones, in which the
1547− torque is controlled. The invention also relates to methods
1548− for operating a device of this type.},
1549− file = {http://v3.espacenet.com/textdoc?IDX=EP1700367},
1550−}
1551−
1552−@patent{sorace,
1553− author = {Sorace, Ronald E. and Reinhardt, Victor S. and Vaughn, Steven
1554− A.},
1555− title = {High-Speed Digital-to-{RF} Converter},
1556− number = 5668842,
1557− date = {1997-09-16},
1558− holder = {{Hughes Aircraft Company}},
1559− type = {patentus},
1560− langid = {english},
1561− langidopts = {variant=american},
1562− annotation = {This is a \texttt{patent} entry with a \texttt{holder} field.
1563− Note the format of the \texttt{type} and \texttt{date} fields
1564− in the database file. Compare \texttt{almendro},
1565− \texttt{laufenberg}, and \texttt{kowalik}},
1566−}
1567−
1568−@periodical{jcg,
1569− title = {Computers and Graphics},
1570− year = 2011,
1571− issuetitle = {Semantic {3D} Media and Content},
1572− volume = 35,
1573− number = 4,
1574− issn = {0097-8493},
1575− annotation = {This is a \texttt{periodical} entry with an \texttt{issn}
1576− field.},
1577−}
1578−
1579−@report{chiu,
1580− author = {Chiu, Willy W. and Chow, We Min},
1581− title = {A Hybrid Hierarchical Model of a {Multiple Virtual Storage}
1582− ({MVS}) Operating System},
1583− type = {resreport},
1584− institution = {IBM},
1585− date = 1978,
1586− number = {RC-6947},
1587− langid = {english},
1588− langidopts = {variant=american},
1589− sorttitle = {Hybrid Hierarchical Model of a Multiple Virtual Storage (MVS)
1590− Operating System},
1591− indextitle = {Hybrid Hierarchical Model, A},
1592− annotation = {This is a \texttt{report} entry for a research report. Note
1593− the format of the \texttt{type} field in the database file
1594− which uses a localization key. The number of the report is
1595− given in the \texttt{number} field. Also note the
1596− \texttt{sorttitle} and \texttt{indextitle} fields},
1597−}
1598−
1599−@report{padhye,
1600− author = {Padhye, Jitendra and Firoiu, Victor and Towsley, Don},
1601− title = {A Stochastic Model of {TCP Reno} Congestion Avoidance and
1602− Control},
1603− type = {techreport},
1604− institution = {University of Massachusetts},
1605− date = 1999,
1606− number = {99-02},
1607− location = {Amherst, Mass.},
1608− langid = {english},
1609− langidopts = {variant=american},
1610− sorttitle = {A Stochastic Model of TCP Reno Congestion Avoidance and
1611− Control},
1612− indextitle = {Stochastic Model of {TCP Reno} Congestion Avoidance and Control,
1613− A},
1614− annotation = {This is a \texttt{report} entry for a technical report. Note
1615− the format of the \texttt{type} field in the database file
1616− which uses a localization key. The number of the report is
1617− given in the \texttt{number} field. Also note the
1618− \texttt{sorttitle} and \texttt{indextitle} fields},
1619− abstract = {The steady state performance of a bulk transfer TCP flow
1620− (i.e. a flow with a large amount of data to send, such as FTP
1621− transfers) may be characterized by three quantities. The first
1622− is the send rate, which is the amount of data sent by the
1623− sender in unit time. The second is the throughput, which is
1624− the amount of data received by the receiver in unit time. Note
1625− that the throughput will always be less than or equal to the
1626− send rate due to losses. Finally, the number of non-duplicate
1627− packets received by the receiver in unit time gives us the
1628− goodput of the connection. The goodput is always less than or
1629− equal to the throughput, since the receiver may receive two
1630− copies of the same packet due to retransmissions by the
1631− sender. In a previous paper, we presented a simple model for
1632− predicting the steady state send rate of a bulk transfer TCP
1633− flow as a function of loss rate and round trip time. In this
1634− paper, we extend that work in two ways. First, we analyze the
1635− performance of bulk transfer TCP flows using more precise,
1636− stochastic analysis. Second, we build upon the previous
1637− analysis to provide both an approximate formula as well as a
1638− more accurate stochastic model for the steady state throughput
1639− of a bulk transfer TCP flow.},
1640− file = {ftp://gaia.cs.umass.edu/pub/Padhey99-markov.ps},
1641−}
1642−
1643−@thesis{geer,
1644− author = {de Geer, Ingrid},
1645− title = {Earl, Saint, Bishop, Skald~-- and Music},
1646− type = {phdthesis},
1647− institution = {Uppsala Universitet},
1648− date = 1985,
1649− subtitle = {The {Orkney Earldom} of the Twelfth Century. {A} Musicological
1650− Study},
1651− location = {Uppsala},
1652− options = {useprefix=false},
1653− langid = {english},
1654− langidopts = {variant=british},
1655− annotation = {This is a typical \texttt{thesis} entry for a PhD thesis. Note
1656− the \texttt{type} field in the database file which uses a
1657− localization key. Also note the format of the printed name and
1658− compare the \texttt{useprefix} option in the \texttt{options}
1659− field as well as \texttt{vangennep}},
1660−}
1661−
1662−@thesis{loh,
1663− author = {Loh, Nin C.},
1664− title = {High-Resolution Micromachined Interferometric Accelerometer},
1665− type = {mathesis},
1666− institution = {Massachusetts Institute of Technology},
1667− date = 1992,
1668− location = {Cambridge, Mass.},
1669− langid = {english},
1670− langidopts = {variant=american},
1671− annotation = {This is a typical \texttt{thesis} entry for an MA thesis. Note
1672− the \texttt{type} field in the database file which uses a
1673− localization key},
1674−}
test/command/biblatex-inproceedings.md+1−1
@@ -63,7 +63,7 @@ references:
6363 publisher-place: Cambridge
6464 title: "Le *De Anima* dans la tradition grècque: Quelques aspects de
6565 l'interpretation du traité, de Theophraste à Themistius"
66− title-short: *De Anima* dans la tradition grècque
66+ title-short: "*De Anima* dans la tradition grècque"
6767 type: paper-conference
6868- author:
6969 - family: Salam
test/command/biblatex-jaffe.md+2−2
@@ -72,8 +72,8 @@ references:
7272 language: la
7373 number-of-volumes: 2
7474 publisher-place: Leipzig
75− title: "Regesta Pontificum Romanorum ab condita ecclesia ad annum post
76− Christum natum [mcxcviii]{.smallcaps}"
75+ title: Regesta Pontificum Romanorum ab condita ecclesia ad annum post
76+ Christum natum [mcxcviii]{.smallcaps}
7777 title-short: Regesta Pontificum Romanorum
7878 type: book
7979---
test/command/biblatex-moraux.md+1−1
@@ -86,7 +86,7 @@ references:
8686 publisher-place: Cambridge
8787 title: "Le *De Anima* dans la tradition grècque: Quelques aspects de
8888 l'interpretation du traité, de Theophraste à Themistius"
89− title-short: *De Anima* dans la tradition grècque
89+ title-short: "*De Anima* dans la tradition grècque"
9090 type: paper-conference
9191---
9292
test/command/biblatex-nussbaum.mddeleted−63
@@ -1,63 +0,0 @@
1−```
2−% pandoc -f biblatex -t markdown -s
3−@comment{
4−
5−Adapted from biblatex-example.bib
6−
7−
8−Formatted with pandoc and chicago-author-date.csl, 2013-10-23:
9−
10−(Nussbaum 1978)
11−
12−Nussbaum, Martha. 1978. *Aristotle’s “De Motu Animalium”*. Princeton:
13−Princeton University Press.
14−
15−
16−Formatted with pandoc and apa.csl, 2013-10-23:
17−
18−(Nussbaum, 1978)
19−
20−Nussbaum, M. (1978). *Aristotle’s “De Motu Animalium”*. Princeton:
21−Princeton University Press.
22−
23−
24−}
25−
26−@string{ pup = {Princeton University Press} }
27−
28−@Book{nussbaum,
29− author = {Nussbaum, Martha},
30− title = {Aristotle's \mkbibquote{De Motu Animalium}},
31− date = 1978,
32− publisher = pup,
33− location = {Princeton},
34− keywords = {secondary},
35− hyphenation = {american},
36− sorttitle = {Aristotle's De Motu Animalium},
37− indexsorttitle= {Aristotle's De Motu Animalium},
38− annotation = {A book entry. Note the sorttitle and
39− indexsorttitle fields and the markup of the quotes in
40− the database file},
41−}
42−
43−^D
44−---
45−nocite: "[@*]"
46−references:
47−- annote: A book entry. Note the sorttitle and indexsorttitle fields and
48− the markup of the quotes in the database file
49− author:
50− - family: Nussbaum
51− given: Martha
52− id: nussbaum
53− issued: 1978
54− keyword: secondary
55− language: en-US
56− publisher: Princeton University Press
57− publisher-place: Princeton
58− title: "Aristotle's \"De Motu Animalium\""
59− type: book
60−---
61−
62−
63−```
test/command/biblatex-quotes.md+1−1
@@ -36,7 +36,7 @@ references:
3636 language: en-US
3737 publisher: Princeton University Press
3838 publisher-place: Princeton
39− title: "Aristotle's \"De Motu Animalium\""
39+ title: Aristotle's "De Motu Animalium"
4040 type: book
4141---
4242
test/command/biblatex-sigfridsson.md+1−1
@@ -90,7 +90,7 @@ references:
9090 - family: Ryde
9191 given: Ulf
9292 container-title: Journal of Computational Chemistry
93− doi: "10.1002/(SICI)1096-987X(199803)19:4\\<377::AID-JCC1\\>3.0.CO;2-P"
93+ doi: "10.1002/(SICI)1096-987X(199803)19:4\\<377::AID-JCC1>3.0.CO;2-P"
9494 id: sigfridsson
9595 issue: 4
9696 issued: 1998
test/command/biblatex-spiegelberg.md+2−2
@@ -63,8 +63,8 @@ references:
6363 issued: 1969
6464 language: de-DE
6565 page: 189-216
66− title: "Intention" und "Intentionalität" in der Scholastik, bei
67− Brentano und Husserl
66+ title: "\"Intention\" und \"Intentionalität\" in der Scholastik, bei
67+ Brentano und Husserl"
6868 title-short: Intention und Intentionalität
6969 type: article-journal
7070 volume: 29
test/command/biblatex-test-case-conversion.md+2−2
@@ -63,9 +63,9 @@ references:
6363 id: item1
6464 issued: 2013
6565 language: en-US
66− title: "A title, in English, with a Proper Name and an ACRONYM and a
66+ title: A title, in English, with a Proper Name and an ACRONYM and a
6767 [camelCase]{.nocase} word and some units, 400 [nm]{.nocase},
68− 3 [cm]{.nocase}, and a quote, *Alea [iacta est]{.nocase}*"
68+ 3 [cm]{.nocase}, and a quote, *Alea [iacta est]{.nocase}*
6969 type: article-journal
7070---
7171
test/command/biblatex-textnormal.md+1−1
@@ -9,7 +9,7 @@
99nocite: "[@*]"
1010references:
1111- id: item1
12− title: "The title [of this book]{.nodecor}"
12+ title: The title [of this book]{.nodecor}
1313 type: book
1414---
1515
test/command/chap1/spider.pngnew file

Binary file not shown.

test/command/chap1/text.mdnew file+17
@@ -0,0 +1,17 @@
1+# Chapter one
2+
3+A spider: ![spider](spider.png)
4+
5+Another spider: ![another spider][refspider]
6+
7+The moon: ![moon](../../lalune.jpg)
8+
9+Link to [spider picture](spider.png).
10+
11+URL left alone: [manual](https://pandoc.org/MANUAL.html).
12+
13+Absolute path left alone: [absolute](/foo/bar/baz.png).
14+
15+Link to fragment: [chapter two](#chapter-two).
16+
17+Empty path: [empty]().
test/command/chap2/spider.pngnew file

Binary file not shown.

test/command/chap2/text.mdnew file+5
@@ -0,0 +1,5 @@
1+# Chapter two
2+
3+A spider: ![spider](spider.png)
4+
5+[refspider]: spider.png
test/command/defaults-inheritance-2.md+1−1
@@ -1,5 +1,5 @@
11```
2−% pandoc -d command/defaults6
2+% pandoc -d command/defaults6 2>&1
33^D
44Error: Circular defaults file reference in 'command/defaults7.yaml'
55```
test/command/duplicate_attributes.mdnew file+7
@@ -0,0 +1,7 @@
1+```
2+% pandoc
3+[span]{.foobar style="color:blue" class="zip" style="color:red"}
4+^D
5+[WARNING] Ignoring duplicate attribute style="color:red".
6+<p><span class="foobar zip" style="color:blue">span</span></p>
7+```
test/command/mmd-metadata.mdnew file+20
@@ -0,0 +1,20 @@
1+```
2+% pandoc -f markdown_mmd -t markdown -s
3+Title: Blah blah blah
4+Author: Doo de Doo
5+Base Header Level: 1
6+Bibliography: Pubs.bib
7+Lang: en-GB
8+
9+body
10+^D
11+---
12+author: Doo de Doo
13+baseheaderlevel: 1
14+bibliography: Pubs.bib
15+lang: en-GB
16+title: Blah blah blah
17+---
18+
19+body
20+```
test/command/newif.mdnew file+55
@@ -0,0 +1,55 @@
1+```
2+% pandoc -f latex -t plain
3+\iftrue
4+should print
5+\iftrue
6+should print
7+\else
8+should not print
9+\fi
10+\else
11+should not print
12+\fi
13+
14+\iffalse
15+should not print
16+\else
17+\iftrue
18+should print
19+\else
20+should not print
21+\fi
22+\fi
23+
24+\newif\ifepub
25+
26+\ifepub
27+should not print
28+\fi
29+
30+\epubtrue
31+
32+\ifepub
33+should print
34+\else
35+should not print
36+\fi
37+
38+\epubfalse
39+
40+\ifepub
41+should not print
42+\else
43+should print
44+\fi
45+^D
46+should print
47+
48+should print
49+
50+should print
51+
52+should print
53+
54+should print
55+```
test/command/pandoc-citeproc-119.md+1−1
@@ -1,7 +1,7 @@
11```
22% pandoc --citeproc -t markdown-citations
33---
4−bibliography: 'command/biblatex-examples.bib'
4+bibliography: 'command/averroes.bib'
55csl: command/apa.csl
66---
77
test/command/pandoc-citeproc-320a.md+3−3
@@ -49,7 +49,7 @@ references:
4949Foo [@item1; @item2; @item3; @item4; @item5; @item6; @item7; @item8].
5050^D
5151Foo (al-ʾUdhrī, n.d.; al-ʿUdhrī, n.d.; al-\'Udhrī, n.d.; al-'Udhrī,
52−n.d.; al-'Udhrī, n.d.; Uch, n.d.; Uebel, n.d.; Zzz, n.d.).
52+n.d.a, n.d.b; Uch, n.d.; Uebel, n.d.; Zzz, n.d.).
5353
5454::: {#refs .references .csl-bib-body .hanging-indent}
5555::: {#ref-item6 .csl-entry}
@@ -69,11 +69,11 @@ Uch, Ann. n.d.
6969:::
7070
7171::: {#ref-item4 .csl-entry}
72−'Udhrī, Jamīl al-. n.d.
72+'Udhrī, Jamīl al-. n.d.a.
7373:::
7474
7575::: {#ref-item5 .csl-entry}
76−'Udhrī, Jamīl al-. n.d.
76+---------. n.d.b.
7777:::
7878
7979::: {#ref-item7 .csl-entry}
test/command/pandoc-citeproc-322.md+1−1
@@ -19,7 +19,7 @@ references:
1919 type: 'article-journal'
2020---
2121
22−Foo[@timmory__justice_1950].
22+Foo [@timmory__justice_1950].
2323^D
2424Foo.[^1]
2525
test/command/pandoc-citeproc-356.md+2−7
@@ -15,11 +15,6 @@ references:
1515
1616[@bar]
1717^D
18−(Alice 2042)
19−
20−::: {#refs .references .csl-bib-body .hanging-indent}
21−::: {#ref-foo .csl-entry}
22−Alice. 2042.
23−:::
24−:::
18+[WARNING] Citeproc: citation bar not found
19+(**bar?**)
2520```
test/command/pandoc-citeproc-65.md+1−1
@@ -14,7 +14,7 @@ references:
1414 id: 'stotz:1996handbuch'
1515 issued:
1616 literal: 1996_2004
17− language: German
17+ language: de
1818 number-of-volumes: 5
1919 publisher: Beck
2020 publisher-place: Munich
test/command/pandoc-citeproc-70.md+2−2
@@ -22,7 +22,7 @@ references:
2222 issued:
2323 date-parts:
2424 - - 2009
25− language: French
25+ language: fr
2626 page: '155--174'
2727 publisher: 'Fédération Internationale des Instituts d''Études
2828 Médiévales'
@@ -42,7 +42,7 @@ references:
4242 issued:
4343 date-parts:
4444 - - 1955
45− language: Latin
45+ language: la
4646 note: 'ArticleType: research-article / Full publication date: 1955 /
4747 Copyright © 1955 Fordham University'
4848 page: '163--193'
test/command/pandoc-citeproc-chicago-author-date.md+7−7
@@ -54,14 +54,14 @@ References {#references .unnumbered}
5454
5555([**nonexistent?**](#ref-nonexistent))
5656
57−[Doe](#ref-item1) ([2005](#ref-item1)) says blah.
57+Doe ([2005](#ref-item1)) says blah.
5858
59−[Doe](#ref-item1) ([2005, 30](#ref-item1)) says blah.
59+Doe ([2005, 30](#ref-item1)) says blah.
6060
61−[Doe](#ref-item1) ([2005, 30](#ref-item1), with suffix) says blah.
61+Doe ([2005, 30](#ref-item1), with suffix) says blah.
6262
63−[Doe](#ref-item1) ([2005](#ref-item1); [2006, 30](#ref-item2); see also
64−[Doe and Roe 2007](#ref-пункт3)) says blah.
63+Doe ([2005](#ref-item1); [2006, 30](#ref-item2); see also [Doe and Roe
64+2007](#ref-пункт3)) says blah.
6565
6666In a note.[^1]
6767
@@ -98,8 +98,8 @@ edited by Sam Smith. Oxford: Oxford University Press.
9898:::
9999:::
100100
101−[^1]: [Doe and Roe](#ref-пункт3) ([2007, 12](#ref-пункт3)) and a
102− citation without locators ([Doe and Roe 2007](#ref-пункт3)).
101+[^1]: Doe and Roe ([2007, 12](#ref-пункт3)) and a citation without
102+ locators ([Doe and Roe 2007](#ref-пункт3)).
103103
104104[^2]: Some citations (see [Doe 2005, chap. 3](#ref-item1);
105105 [2006](#ref-item2); [Doe and Roe 2007](#ref-пункт3)).
test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+6−7
@@ -55,13 +55,13 @@ References {#references .unnumbered}
5555
5656[^2]
5757
58−[John Doe](#ref-item1)[^3] says blah.
58+John Doe[^3] says blah.
5959
60−[Doe](#ref-item1)[^4] says blah.
60+Doe[^4] says blah.
6161
62−[Doe](#ref-item1)[^5] says blah.
62+Doe[^5] says blah.
6363
64−[Doe](#ref-item1)[^6] says blah.
64+Doe[^6] says blah.
6565
6666In a note.[^7]
6767
@@ -112,9 +112,8 @@ Sam Smith. Oxford: Oxford University Press, 2007.
112112 Roe, "Why Water Is Wet," in *Third Book*, ed. Sam Smith (Oxford:
113113 Oxford University Press, 2007)](#ref-пункт3).
114114
115−[^7]: [Doe and Roe](#ref-пункт3), ["Why Water Is Wet," 12](#ref-пункт3)
116− and a citation without locators ([Doe and Roe, "Why Water Is
117− Wet"](#ref-пункт3)).
115+[^7]: Doe and Roe, ["Why Water Is Wet," 12](#ref-пункт3) and a citation
116+ without locators ([Doe and Roe, "Why Water Is Wet"](#ref-пункт3)).
118117
119118[^8]: See [Doe, *First Book*, chap. 3](#ref-item1); also [Doe and Roe,
120119 "Why Water Is Wet," 34--35](#ref-пункт3).
test/command/svg.md+1−1
@@ -2,7 +2,7 @@
22% pandoc -f latex -t icml
33\includegraphics{command/corrupt.svg}
44^D
5−[WARNING] Could not determine image size for 'command/corrupt.svg': could not determine image type
5+[WARNING] Could not determine image size for command/corrupt.svg: could not determine image type
66<ParagraphStyleRange AppliedParagraphStyle="ParagraphStyle/Paragraph">
77 <CharacterStyleRange AppliedCharacterStyle="$ID/NormalCharacterStyle">
88 <Rectangle Self="uec" StrokeWeight="0" ItemTransform="1 0 0 1 150 -100">
test/command/toc.md+2−2

This diff is too big to show here.

test/command/unicode-collation.mdnew file+130

This diff is too big to show here.

test/docbook-reader.docbook+3−1

This diff is too big to show here.

test/docx/0_level_headers.native+2−2

This diff is too big to show here.

test/docx/golden/block_quotes.docx

Binary file not shown.

test/docx/golden/codeblock.docx

Binary file not shown.

test/docx/golden/comments.docx

Binary file not shown.

test/docx/golden/custom_style_no_reference.docx

Binary file not shown.

test/docx/golden/custom_style_preserve.docx

Binary file not shown.

test/docx/golden/custom_style_reference.docx

Binary file not shown.

test/docx/golden/definition_list.docx

Binary file not shown.

test/docx/golden/document-properties-short-desc.docx

Binary file not shown.

test/docx/golden/document-properties.docx

Binary file not shown.

test/docx/golden/headers.docx

Binary file not shown.

test/docx/golden/image.docx

Binary file not shown.

test/docx/golden/inline_code.docx

Binary file not shown.

test/docx/golden/inline_formatting.docx

Binary file not shown.

test/docx/golden/inline_images.docx

Binary file not shown.

test/docx/golden/link_in_notes.docx

Binary file not shown.

test/docx/golden/links.docx

Binary file not shown.

test/docx/golden/lists.docx

Binary file not shown.

test/docx/golden/lists_continuing.docx

Binary file not shown.

test/docx/golden/lists_multiple_initial.docx

Binary file not shown.

test/docx/golden/lists_restarting.docx

Binary file not shown.

test/docx/golden/nested_anchors_in_header.docx

Binary file not shown.

test/docx/golden/notes.docx

Binary file not shown.

test/docx/golden/raw-blocks.docx

Binary file not shown.

test/docx/golden/raw-bookmarks.docx

Binary file not shown.

test/docx/golden/table_one_row.docx

Binary file not shown.

test/docx/golden/table_with_list_cell.docx

Binary file not shown.

test/docx/golden/tables-default-widths.docxnew file

Binary file not shown.

test/docx/golden/tables.docx

Binary file not shown.

test/docx/golden/track_changes_deletion.docx

Binary file not shown.

test/docx/golden/track_changes_insertion.docx

Binary file not shown.

test/docx/golden/track_changes_move.docx

Binary file not shown.

test/docx/golden/track_changes_scrubbed_metadata.docx

Binary file not shown.

test/docx/golden/unicode.docx

Binary file not shown.

test/docx/golden/verbatim_subsuper.docx

Binary file not shown.

test/docx/image_vml_as_object.docxnew file

Binary file not shown.

test/docx/image_vml_as_object.nativenew file+2

This diff is too big to show here.

test/docx/sdt_elements.native+9−10

This diff is too big to show here.

test/docx/table_captions_no_field.docxnew file

Binary file not shown.

test/docx/table_captions_no_field.nativenew file+34

This diff is too big to show here.

test/docx/table_captions_with_field.docxnew file

Binary file not shown.

test/docx/table_captions_with_field.nativenew file+54

This diff is too big to show here.

test/docx/table_header_rowspan.docxnew file

Binary file not shown.

test/docx/table_header_rowspan.nativenew file+189

This diff is too big to show here.

test/docx/table_one_header_row.docxnew file

Binary file not shown.

test/docx/table_one_header_row.nativenew file+18

This diff is too big to show here.

test/docx/table_one_row.docx

Binary file not shown.

test/docx/table_one_row.native+4−4

This diff is too big to show here.

test/docx/table_variable_width.native+12−17

This diff is too big to show here.

test/docx/table_with_list_cell.native+3−3

This diff is too big to show here.

test/docx/tables-default-widths.nativenew file+92

This diff is too big to show here.

test/docx/tables.native+9−9

This diff is too big to show here.

test/epub/features.native+2−2

This diff is too big to show here.

test/epub/wasteland.epub

Binary file not shown.

test/fb2/images.fb2+1−1

This diff is too big to show here.

test/fb2/test.jpg

Binary file not shown.

test/html-reader.native+2−1

This diff is too big to show here.

test/jira-reader.native+1−1

This diff is too big to show here.

test/lhs-test.html+2−1

This diff is too big to show here.

test/lhs-test.html+lhs+2−1

This diff is too big to show here.

test/odt/native/tab.nativenew file+1

This diff is too big to show here.

test/odt/odt/tab.odtnew file

Binary file not shown.

test/pipe-tables.native+2−10

This diff is too big to show here.

test/pptx/code-custom_templated.pptx

Binary file not shown.

test/pptx/code_templated.pptx

Binary file not shown.

test/pptx/document-properties-short-desc_templated.pptx

Binary file not shown.

test/pptx/document-properties_templated.pptx

Binary file not shown.

test/pptx/endnotes_templated.pptx

Binary file not shown.

test/pptx/endnotes_toc_templated.pptx

Binary file not shown.

test/pptx/images_templated.pptx

Binary file not shown.

test/pptx/inline_formatting_templated.pptx

Binary file not shown.

test/pptx/lists_templated.pptx

Binary file not shown.

test/pptx/raw_ooxml_templated.pptx

Binary file not shown.

test/pptx/reference_depth.pptx

Binary file not shown.

test/pptx/remove_empty_slides_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_slide_level_1_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_toc_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_after_metadata_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterheader_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterseps_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_templated.pptx

Binary file not shown.

test/pptx/start_numbering_at_templated.pptx

Binary file not shown.

test/pptx/tables_templated.pptx

Binary file not shown.

test/pptx/two_column_templated.pptx

Binary file not shown.

test/tables.latex+27−3

This diff is too big to show here.

test/tables.ms+30

This diff is too big to show here.

test/tables/nordics.latex+22−4

This diff is too big to show here.

test/tables/planets.latex+4−4

This diff is too big to show here.

test/tables/students.latex+10−2

This diff is too big to show here.

test/test-pandoc.hs+1−2

This diff is too big to show here.

test/writer.context+42−38

This diff is too big to show here.

test/writer.html4+2−1

This diff is too big to show here.

test/writer.html5+2−1

This diff is too big to show here.

test/writer.jats_archiving+37−47

This diff is too big to show here.

test/writer.jats_articleauthoring+41−49

This diff is too big to show here.

test/writer.jats_publishing+37−47

This diff is too big to show here.

test/writer.jira+23−35

This diff is too big to show here.

test/writer.ms+1−1

This diff is too big to show here.

test/writer.opml+1−1

This diff is too big to show here.

test/writer.org+7−7

This diff is too big to show here.

test/writer.zimwiki+3−3

This diff is too big to show here.

test/writers-lang-and-dir.context+4

This diff is too big to show here.

tools/build-arm.shnew file+110

This diff is too big to show here.