git.iko.soy/typeable/pandoc
on 65d4e02 help sign in
364 files changed+5544−8240
since splitsnapshots
unifiedsplit
.circleci/config.yml+1−1 .editorconfig−8 .github/ISSUE_TEMPLATE+11 .github/ISSUE_TEMPLATE/bug_report.md−25 .github/ISSUE_TEMPLATE/config.yml−9 .github/ISSUE_TEMPLATE/feature_request.md−21 .github/workflows/ci.yml+6−21 .github/workflows/nightly.yml+2−2 .hlint.yaml+5−3 AUTHORS.md−14 CONTRIBUTING.md+11−16 INSTALL.md−4 MANUAL.txt+62−176 Makefile+12−19 RELEASE-CHECKLIST−1 SECURITY.md−12 benchmark/benchmark-pandoc.hs+12−13 cabal.project+8−9 changelog.md+8−795 data/docx/word/settings.xml+16−6 data/templates/article.jats_publishing+1−1 data/templates/default.context+3−7 data/templates/default.dzslides+1−1 data/templates/default.jats_articleauthoring+1−1 data/templates/default.latex+25−50 data/templates/default.revealjs+129−79 data/templates/default.s5−3 data/templates/default.slideous−7 data/templates/default.slidy−5 data/templates/styles.html+1−2 doc/epub.md+2−9 doc/faqs.md−128 doc/filters.md−2 doc/jats.md−9 doc/libraries.md−4 doc/lua-filters.md+2−2 doc/org.md+2−16 doc/using-the-pandoc-api.md+12−25 linux/make_artifacts.sh+4−14 man/pandoc.1+49−233 pandoc.cabal+30−40 src/Text/Pandoc/App.hs+13−19 src/Text/Pandoc/App/CommandLineOptions.hs+2−2 src/Text/Pandoc/App/FormatHeuristics.hs+7−13 src/Text/Pandoc/App/Opt.hs+1−2 src/Text/Pandoc/Asciify.hs+390−13 src/Text/Pandoc/BCP47.hs+112 src/Text/Pandoc/Citeproc.hs+56−75 src/Text/Pandoc/Citeproc/BibTeX.hs+15−25 src/Text/Pandoc/Citeproc/Data.hs+6−7 src/Text/Pandoc/Citeproc/Locator.hs+4−20 src/Text/Pandoc/Class/CommonState.hs+1−1 src/Text/Pandoc/Class/IO.hs+22−21 src/Text/Pandoc/Class/PandocMonad.hs+34−24 src/Text/Pandoc/Error.hs+16−23 src/Text/Pandoc/Extensions.hs+1−9 src/Text/Pandoc/Highlighting.hs+5−5 src/Text/Pandoc/ImageSize.hs+1−5 src/Text/Pandoc/Logging.hs+9−26 src/Text/Pandoc/Lua/Filter.hs+1−4 src/Text/Pandoc/Lua/Module/MediaBag.hs+3−3 src/Text/Pandoc/Lua/Module/Pandoc.hs+3−8 src/Text/Pandoc/Lua/Module/Utils.hs+1−1 src/Text/Pandoc/MIME.hs−10 src/Text/Pandoc/MediaBag.hs+14−45 src/Text/Pandoc/Options.hs+17−17 src/Text/Pandoc/PDF.hs+3−16 src/Text/Pandoc/Parsing.hs+242−360 src/Text/Pandoc/Readers.hs+48−55 src/Text/Pandoc/Readers/BibTeX.hs+8−14 src/Text/Pandoc/Readers/CSV.hs+7−7 src/Text/Pandoc/Readers/CommonMark.hs+4−46 src/Text/Pandoc/Readers/Creole.hs+6−5 src/Text/Pandoc/Readers/CslJson.hs+4−5 src/Text/Pandoc/Readers/DocBook.hs+15−30 src/Text/Pandoc/Readers/Docx.hs+42−59 src/Text/Pandoc/Readers/Docx/Combine.hs+5−5 src/Text/Pandoc/Readers/Docx/Parse.hs+21−121 src/Text/Pandoc/Readers/Docx/Util.hs−7 src/Text/Pandoc/Readers/DokuWiki.hs+8−9 src/Text/Pandoc/Readers/FB2.hs+3−6 src/Text/Pandoc/Readers/HTML.hs+28−48 src/Text/Pandoc/Readers/HTML/Table.hs+15−35 src/Text/Pandoc/Readers/Haddock.hs+7−11 src/Text/Pandoc/Readers/Ipynb.hs+3−5 src/Text/Pandoc/Readers/JATS.hs+6−13 src/Text/Pandoc/Readers/Jira.hs+9−13 src/Text/Pandoc/Readers/LaTeX.hs+34−22 src/Text/Pandoc/Readers/LaTeX/Citation.hs+1−1 src/Text/Pandoc/Readers/LaTeX/Lang.hs+119−122 src/Text/Pandoc/Readers/LaTeX/Macro.hs+14−45 src/Text/Pandoc/Readers/LaTeX/Parsing.hs+7−48 src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+66−140 src/Text/Pandoc/Readers/LaTeX/Types.hs+1−14 src/Text/Pandoc/Readers/Man.hs+11−18 src/Text/Pandoc/Readers/Markdown.hs+72−92 src/Text/Pandoc/Readers/MediaWiki.hs+7−16 src/Text/Pandoc/Readers/Metadata.hs+17−44 src/Text/Pandoc/Readers/Muse.hs+9−10 src/Text/Pandoc/Readers/Native.hs+5−7 src/Text/Pandoc/Readers/OPML.hs+4−9 src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2 src/Text/Pandoc/Readers/Odt/ContentReader.hs+7−12 src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+2−2 src/Text/Pandoc/Readers/Odt/StyleReader.hs+7−8 src/Text/Pandoc/Readers/Org.hs+7−4 src/Text/Pandoc/Readers/Org/Blocks.hs+4−7 src/Text/Pandoc/Readers/Org/Inlines.hs+5−6 src/Text/Pandoc/Readers/Org/Parsing.hs+5−5 src/Text/Pandoc/Readers/RST.hs+110−78 src/Text/Pandoc/Readers/Roff.hs+18−16 src/Text/Pandoc/Readers/TWiki.hs+27−7 src/Text/Pandoc/Readers/Textile.hs+88−94 src/Text/Pandoc/Readers/TikiWiki.hs+6−6 src/Text/Pandoc/Readers/Txt2Tags.hs+21−47 src/Text/Pandoc/Readers/Vimwiki.hs+7−12 src/Text/Pandoc/SelfContained.hs+7−4 src/Text/Pandoc/Shared.hs+56−29 src/Text/Pandoc/Sources.hs−195 src/Text/Pandoc/Templates.hs−4 src/Text/Pandoc/Writers.hs+3−3 src/Text/Pandoc/Writers/AsciiDoc.hs+4−10 src/Text/Pandoc/Writers/BibTeX.hs+1−1 src/Text/Pandoc/Writers/ConTeXt.hs+38−44 src/Text/Pandoc/Writers/CslJson.hs+3−4 src/Text/Pandoc/Writers/Docbook.hs+7−50 src/Text/Pandoc/Writers/Docx.hs+244−60 src/Text/Pandoc/Writers/Docx/Table.hs−227 src/Text/Pandoc/Writers/Docx/Types.hs−185 src/Text/Pandoc/Writers/DokuWiki.hs+1−3 src/Text/Pandoc/Writers/EPUB.hs+3−3 src/Text/Pandoc/Writers/GridTable.hs−157 src/Text/Pandoc/Writers/HTML.hs+34−107 src/Text/Pandoc/Writers/Haddock.hs+2−13 src/Text/Pandoc/Writers/JATS.hs+44−67 src/Text/Pandoc/Writers/JATS/References.hs+2−3 src/Text/Pandoc/Writers/JATS/Table.hs+5−25 src/Text/Pandoc/Writers/JATS/Types.hs+3−12 src/Text/Pandoc/Writers/Jira.hs+18−41 src/Text/Pandoc/Writers/LaTeX.hs+12−37 src/Text/Pandoc/Writers/LaTeX/Caption.hs+1 src/Text/Pandoc/Writers/LaTeX/Lang.hs+60−61 src/Text/Pandoc/Writers/LaTeX/Table.hs+8−21 src/Text/Pandoc/Writers/LaTeX/Types.hs+1−2 src/Text/Pandoc/Writers/LaTeX/Util.hs+3−4 src/Text/Pandoc/Writers/Man.hs+2−2 src/Text/Pandoc/Writers/Markdown.hs+56−54 src/Text/Pandoc/Writers/Markdown/Inline.hs+21−36 src/Text/Pandoc/Writers/Ms.hs+8−24 src/Text/Pandoc/Writers/Muse.hs+3−6 src/Text/Pandoc/Writers/ODT.hs+3−4 src/Text/Pandoc/Writers/OpenDocument.hs+54−50 src/Text/Pandoc/Writers/Org.hs+14−24 src/Text/Pandoc/Writers/Powerpoint.hs+1−1 src/Text/Pandoc/Writers/Powerpoint/Output.hs+31−43 src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+3−2 src/Text/Pandoc/Writers/RST.hs+10−14 src/Text/Pandoc/Writers/RTF.hs+10−16 src/Text/Pandoc/Writers/Shared.hs+3−18 src/Text/Pandoc/Writers/TEI.hs+3−4 src/Text/Pandoc/Writers/Texinfo.hs+15−22 src/Text/Pandoc/Writers/ZimWiki.hs+2−4 src/Text/Pandoc/XML.hs+2−28 src/Text/Pandoc/XML/Light/Proc.hs+1 src/Text/Pandoc/XML/Light/Types.hs+4−4 stack.yaml+6−11 test/Tests/Command.hs+14−7 test/Tests/Helpers.hs−23 test/Tests/Old.hs+12−3 test/Tests/Readers/Docx.hs+5−25 test/Tests/Readers/EPUB.hs+1−1 test/Tests/Readers/HTML.hs+5−20 test/Tests/Readers/Jira.hs−14 test/Tests/Readers/Markdown.hs+3−3 test/Tests/Readers/Odt.hs−1 test/Tests/Writers/ConTeXt.hs+113−122 test/Tests/Writers/Docbook.hs+6−43 test/Tests/Writers/Docx.hs−5 test/Tests/Writers/HTML.hs+52−57 test/Tests/Writers/JATS.hs+91−135 test/Tests/Writers/Jira.hs−52 test/Tests/Writers/OOXML.hs+1−3 test/command/1279.md+1−1 test/command/2103.md−8 test/command/2874.md+2−2 test/command/3752.md−60 test/command/3792.md+1−1 test/command/4063.md−29 test/command/4164.md+1−1 test/command/4442.md+1−1 test/command/4465.md−15 test/command/5304.md+1−1 test/command/5367.md+2−6 test/command/5519.md+1−1 test/command/6026.md−19 test/command/6541.md−27 test/command/6620.md−3 test/command/6658.md−75 test/command/6709.md−11 test/command/6890.md+2−2 test/command/6925.md+2−2 test/command/6958.md−10 test/command/7041.md−23 test/command/7099.md+1−1 test/command/7132.md−10 test/command/7134.md−16 test/command/7145.md−12 test/command/7155.md−15 test/command/7173.md−137 test/command/7208.md−6 test/command/7214.md−28 test/command/7216.md−19 test/command/7266.md−10 test/command/7272.md−27 test/command/7278.md−38 test/command/7282.md−7 test/command/7288.md−40 test/command/7299.md−23 test/command/7321.md−24 test/command/7323.md−29 test/command/7324.md−25 test/command/7339.md−11 test/command/7340.md−6 test/command/7376.md−16 test/command/7394.md−85 test/command/7397.md−14 test/command/7400.md−9 test/command/7416.md−19 test/command/averroes.bib−95 test/command/biblatex-basic.md+3−5 test/command/biblatex-examples.bib+1674 test/command/biblatex-inproceedings.md+1−1 test/command/biblatex-jaffe.md+2−2 test/command/biblatex-moraux.md+1−1 test/command/biblatex-nussbaum.md+63 test/command/biblatex-quotes.md+1−1 test/command/biblatex-sigfridsson.md+1−1 test/command/biblatex-spiegelberg.md+2−2 test/command/biblatex-test-case-conversion.md+2−2 test/command/biblatex-textnormal.md+1−1 test/command/chap1/spider.png test/command/chap1/text.md−17 test/command/chap2/spider.png test/command/chap2/text.md−5 test/command/defaults-inheritance-2.md+1−1 test/command/duplicate_attributes.md−7 test/command/mmd-metadata.md−20 test/command/newif.md−55 test/command/pandoc-citeproc-119.md+1−1 test/command/pandoc-citeproc-320a.md+3−3 test/command/pandoc-citeproc-322.md+1−1 test/command/pandoc-citeproc-356.md+7−2 test/command/pandoc-citeproc-65.md+1−1 test/command/pandoc-citeproc-70.md+2−2 test/command/pandoc-citeproc-chicago-author-date.md+7−7 test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+7−6 test/command/svg.md+1−1 test/command/toc.md+2−2 test/command/unicode-collation.md−130 test/docbook-reader.docbook+1−3 test/docx/0_level_headers.native+2−2 test/docx/golden/block_quotes.docx test/docx/golden/codeblock.docx test/docx/golden/comments.docx test/docx/golden/custom_style_no_reference.docx test/docx/golden/custom_style_preserve.docx test/docx/golden/custom_style_reference.docx test/docx/golden/definition_list.docx test/docx/golden/document-properties-short-desc.docx test/docx/golden/document-properties.docx test/docx/golden/headers.docx test/docx/golden/image.docx test/docx/golden/inline_code.docx test/docx/golden/inline_formatting.docx test/docx/golden/inline_images.docx test/docx/golden/link_in_notes.docx test/docx/golden/links.docx test/docx/golden/lists.docx test/docx/golden/lists_continuing.docx test/docx/golden/lists_multiple_initial.docx test/docx/golden/lists_restarting.docx test/docx/golden/nested_anchors_in_header.docx test/docx/golden/notes.docx test/docx/golden/raw-blocks.docx test/docx/golden/raw-bookmarks.docx test/docx/golden/table_one_row.docx test/docx/golden/table_with_list_cell.docx test/docx/golden/tables-default-widths.docx test/docx/golden/tables.docx test/docx/golden/track_changes_deletion.docx test/docx/golden/track_changes_insertion.docx test/docx/golden/track_changes_move.docx test/docx/golden/track_changes_scrubbed_metadata.docx test/docx/golden/unicode.docx test/docx/golden/verbatim_subsuper.docx test/docx/image_vml_as_object.docx test/docx/image_vml_as_object.native−2 test/docx/sdt_elements.native+10−9 test/docx/table_captions_no_field.docx test/docx/table_captions_no_field.native−34 test/docx/table_captions_with_field.docx test/docx/table_captions_with_field.native−54 test/docx/table_header_rowspan.docx test/docx/table_header_rowspan.native−189 test/docx/table_one_header_row.docx test/docx/table_one_header_row.native−18 test/docx/table_one_row.docx test/docx/table_one_row.native+4−4 test/docx/table_variable_width.native+17−12 test/docx/table_with_list_cell.native+3−3 test/docx/tables-default-widths.native−92 test/docx/tables.native+9−9 test/epub/features.native+2−2 test/epub/wasteland.epub test/fb2/images.fb2+1−1 test/fb2/test.jpg test/html-reader.native+1−2 test/jira-reader.native+1−1 test/lhs-test.html+1−2 test/lhs-test.html+lhs+1−2 test/odt/native/tab.native−1 test/odt/odt/tab.odt test/pipe-tables.native+10−2 test/pptx/code-custom_templated.pptx test/pptx/code_templated.pptx test/pptx/document-properties-short-desc_templated.pptx test/pptx/document-properties_templated.pptx test/pptx/endnotes_templated.pptx test/pptx/endnotes_toc_templated.pptx test/pptx/images_templated.pptx test/pptx/inline_formatting_templated.pptx test/pptx/lists_templated.pptx test/pptx/raw_ooxml_templated.pptx test/pptx/reference_depth.pptx test/pptx/remove_empty_slides_templated.pptx test/pptx/slide_breaks_slide_level_1_templated.pptx test/pptx/slide_breaks_templated.pptx test/pptx/slide_breaks_toc_templated.pptx test/pptx/speaker_notes_after_metadata_templated.pptx test/pptx/speaker_notes_afterheader_templated.pptx test/pptx/speaker_notes_afterseps_templated.pptx test/pptx/speaker_notes_templated.pptx test/pptx/start_numbering_at_templated.pptx test/pptx/tables_templated.pptx test/pptx/two_column_templated.pptx test/tables.latex+3−27 test/tables.ms−30 test/tables/nordics.latex+4−22 test/tables/planets.latex+4−4 test/tables/students.latex+2−10 test/test-pandoc.hs+2−1 test/writer.context+38−42 test/writer.html4+1−2 test/writer.html5+1−2 test/writer.jats_archiving+47−37 test/writer.jats_articleauthoring+49−41 test/writer.jats_publishing+47−37 test/writer.jira+35−23 test/writer.ms+1−1 test/writer.opml+1−1 test/writer.org+7−7 test/writer.zimwiki+3−3 test/writers-lang-and-dir.context−4 tools/build-arm.sh+5−17
.circleci/config.yml+1−1
@@ -2,7 +2,7 @@ version: "2.1"
22jobs:
33 build:
44 macos:
5− xcode: 11.1.0
5+ xcode: 9.4.1
66 environment:
77 HOMEBREW_NO_AUTO_UPDATE: 1
88 steps:
.editorconfig−8
@@ -8,14 +8,6 @@ indent_size = 2
88insert_final_newline = true
99trim_trailing_whitespace = true
1010
11−[*.{zip,docx}]
12−charset =
13−end_of_line =
14−indent_style =
15−indent_size =
16−insert_final_newline =
17−trim_trailing_whitespace =
18−
1911[*.{markdown,md}]
2012trim_trailing_whitespace = false
2113
.github/ISSUE_TEMPLATEnew file+11
@@ -0,0 +1,11 @@
1+Asking a question?
2+Please use the pandoc-discuss mailing list instead:
3+https://groups.google.com/forum/#!forum/pandoc-discuss
4+This tracker is for bug reports and enhancement requests.
5+
6+Reporting a bug?
7+Please include your pandoc version and an example that allows us
8+to reproduce the problem (complete input, exact command line
9+used, expected and actual output). The smaller the example, the
10+better.
11+
.github/ISSUE_TEMPLATE/bug_report.mddeleted−25
@@ -1,25 +0,0 @@
1−---
2−name: Bug report
3−about: Create a report to help us improve
4−title: ''
5−labels: 'bug'
6−assignees: ''
7−
8−---
9−
10−<!--
11−Thank you for reporting an issue! Before you continue, please make sure that you have
12−
13−- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
14−- searched the pandoc-discuss mailing list for solutions: https://groups.google.com/forum/#!forum/pandoc-discuss
15−- reproduced your issue on the latest release of pandoc, or on https://pandoc.org/try
16−
17−Note that this bug tracker is for reporting bugs, not asking questions. For questions, use the pandoc-discuss mailing list.
18−-->
19−
20−**Explain the problem.**
21−Include the exact command line you used and all inputs necessary to reproduce the issue. Please create as minimal an example as possible, to help the maintainers isolate the problem. Explain the output you received and how it differs from what you expected.
22−
23−**Pandoc version?**
24−What version of pandoc are you using, on what OS?
25−
.github/ISSUE_TEMPLATE/config.ymldeleted−9
@@ -1,9 +0,0 @@
1−blank_issues_enabled: false
2−contact_links:
3− - name: Ask a question
4− url: https://groups.google.com/forum/#!forum/pandoc-discuss
5− about: "Discussion forum for pandoc (may be used as an email list)"
6− - name: Contributing Guidelines
7− url: https://pandoc.org/CONTRIBUTING.html
8− about: "Guidelines for submitting bug reports and code improvements"
9−
.github/ISSUE_TEMPLATE/feature_request.mddeleted−21
@@ -1,21 +0,0 @@
1−---
2−name: Feature request
3−about: Suggest an idea for this project
4−title: ''
5−labels: 'enhancement'
6−assignees: ''
7−
8−---
9−
10−<!--
11−Thank you for suggesting a feature! Before you continue, please make sure that you have
12−
13−- read the contributing guidelines: https://pandoc.org/CONTRIBUTING.html
14−- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
15−- searched the pandoc-discuss mailing list for relevant discussions: https://groups.google.com/forum/#!forum/pandoc-discuss
16−-->
17−
18−**Describe your proposed improvement and the problem it solves.**
19−
20−**Describe alternatives you've considered.**
21−
.github/workflows/ci.yml+6−21
@@ -44,47 +44,32 @@ jobs:
4444 matrix:
4545 versions:
4646 - ghc: '8.0.2'
47− cabal: '2.4'
48− prefix: 'v2-'
49− cabalopts: ''
47+ cabal: '2.2'
48+ prefix: 'new-'
5049 testopts: ''
5150 - ghc: '8.2.2'
5251 cabal: '2.4'
5352 prefix: 'v2-'
54− cabalopts: ''
5553 testopts: ''
5654 - ghc: '8.4.4'
5755 cabal: '2.4'
5856 prefix: 'v2-'
59− cabalopts: '-f-embed_data_files'
6057 testopts: ''
6158 - ghc: '8.6.5'
6259 cabal: '3.2'
6360 prefix: ''
64− cabalopts: ''
6561 testopts: '--test-option=--hide-successes'
6662 - ghc: '8.8.4'
6763 cabal: '3.2'
6864 prefix: ''
69− cabalopts: ''
7065 testopts: '--test-option=--hide-successes'
71− - ghc: '8.10.4'
66+ - ghc: '8.10.2'
7267 cabal: '3.2'
7368 prefix: ''
74− cabalopts: '-ftrypandoc'
75− testopts: '--test-option=--hide-successes'
76− - ghc: '9.0.1'
77− cabal: '3.4'
78− prefix: ''
79− cabalopts: ''
8069 testopts: '--test-option=--hide-successes'
8170 steps:
8271 - uses: actions/checkout@v2
8372
84− # needed by memory
85− - name: Install numa
86− run: sudo apt-get install libnuma-dev
87−
8873 # need to install older cabal/ghc versions from ppa repository
8974
9075 - name: Install recent cabal/ghc
@@ -116,14 +101,14 @@ jobs:
116101 - name: Install dependencies
117102 run: |
118103 cabal ${{ matrix.versions.prefix }}update
119− cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --dependencies-only --enable-tests --disable-optimization
104+ cabal ${{ matrix.versions.prefix }}build --dependencies-only --enable-tests --disable-optimization
120105
121106 - name: Build and test
122107 run: |
123− cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --enable-tests --disable-optimization 2>&1 | tee build.log
108+ cabal ${{ matrix.versions.prefix }}build --enable-tests --disable-optimization 2>&1 | tee build.log
124109 # fail if warnings in local build
125110 ! grep -q ": *[Ww]arning:" build.log || exit 1
126− cabal ${{ matrix.versions.prefix }}test ${{ matrix.versions.cabalopts }} --disable-optimization ${{ matrix.versions.testopts }}
111+ cabal ${{ matrix.versions.prefix }}test --disable-optimization ${{ matrix.versions.testopts }}
127112
128113 windows:
129114
.github/workflows/nightly.yml+2−2
@@ -22,10 +22,10 @@ jobs:
2222 ghc --version
2323 cabal --version
2424 cabal v2-update
25− cabal v2-build --dependencies-only
25+ cabal v2-build --dependencies-only all
2626 - name: Build
2727 run: |
28− cabal v2-install
28+ cabal v2-install all
2929 strip $HOME/.cabal/bin/pandoc
3030 - name: Install artifact
3131 run: |
.hlint.yaml+5−3
@@ -9,7 +9,6 @@
99# Ignore some builtin hints
1010#
1111- ignore: {name: "Avoid lambda"}
12−- ignore: {name: "Use bimap"}
1312- ignore: {name: "Eta reduce"}
1413- ignore: {name: "Evaluate"}
1514- ignore: {name: "Reduce duplication"} # TODO: could be more fine-grained
@@ -40,10 +39,13 @@
4039 name: "Redundant return"
4140 within: Text.Pandoc.Citeproc.BibTeX
4241
43−# With recent hlint this rule has given false positives on TH
42+# TODO: check
4443- ignore:
4544 name: "Redundant bracket"
46−
45+ within:
46+ - Text.Pandoc.Citeproc
47+ - Text.Pandoc.Citeproc.BibTeX
48+ - Text.Pandoc.Citeproc.MetaValue
4749
4850- ignore:
4951 name: "Use <$>"
AUTHORS.md−14
@@ -20,7 +20,6 @@
2020- Andreas Lööw
2121- Andrew Dunning
2222- Andy Morris
23−- Anti-Distinctlyminty
2423- Antoine Latter
2524- Antonio Terceiro
2625- Arata Mizuki
@@ -36,7 +35,6 @@
3635- Benoit Schweblin
3736- Bjorn Buckwalter
3837- Blake Eryx
39−- Bodigrim
4038- Bradley Kuhn
4139- Brent Yorgey
4240- Brian Leung
@@ -47,7 +45,6 @@
4745- Carlos Sosa
4846- Cécile Chemin
4947- Cédric Couralet
50−- Charanjit Singh
5148- Chris Black
5249- Christian Conkle
5350- Christian Despres
@@ -78,10 +75,8 @@
7875- Eigil Rischel
7976- Emanuel Evans
8077- Emerson Harkin
81−- Emily Bourke
8278- Emily Eisenberg
8379- Eric Kow
84−- Erik Rask
8580- Eric Seidel
8681- Eric Schrijver
8782- Ethan Riley
@@ -91,7 +86,6 @@
9186- Florian Beeres
9287- Florian Eitel
9388- Florian Klink
94−- Florian Kohrt
9589- Francesco Occhipinti
9690- François Gannaz
9791- Frederik Elwert
@@ -152,7 +146,6 @@
152146- Josef Svenningsson
153147- Joseph C. Sible
154148- Julien Cretel
155−- Julien Dutant
156149- Juliette Fourcot
157150- Juliusz Gonera
158151- Justin Bogner
@@ -235,7 +228,6 @@
235228- Raymond Ehlers
236229- Recai Oktaş
237230- Roland Hieber
238−- Roman Beránek
239231- RyanGlScott
240232- Salim B
241233- Sascha Wilde
@@ -253,7 +245,6 @@
253245- Stefan Dresselhaus
254246- Sumit Sahrawat
255247- Tarik Graba
256−- Tatiana Porras
257248- Thenaesh Elango
258249- Thomas Hodgson
259250- Thomas Weißschuh
@@ -287,7 +278,6 @@
287278- a-vrma
288279- andrebauer
289280- argent0
290−- badumont
291281- blmage
292282- bucklereed
293283- bumper314
@@ -303,11 +293,8 @@
303293- kaizshang91
304294- lux-lth
305295- lwolfsonkin
306−- mbrackeantidot
307296- nkalvi
308297- niszet
309−- nuew
310−- obcat
311298- oltolm
312299- oquechy
313300- quasicomputational
@@ -319,7 +306,6 @@
319306- shreevatsa.public
320307- takahashim
321308- taotieren
322−- tecosaur
323309- TEC
324310- tgkokk
325311- the-solipsist
CONTRIBUTING.md+11−16
@@ -244,16 +244,9 @@ Or with stack:
244244
245245 stack test --test-arguments='-p markdown'
246246
247−It is often helpful to add `-j4` (run tests in parallel) and
248−`--hide-successes` (don't clutter output with successes) to the test
249−arguments as well. Collecting all options in a `cabal.project.local`
250−file in the project's root directory can help to keep `cabal`
251−commands short. E.g.:
252−
253− flags: +embed_data_files
254− tests: True
255− test-show-details: direct
256− test-options: -j4 --hide-successes
247+It is often helpful to add `-j4` (run tests in parallel)
248+and `--hide-successes` (don't clutter output with successes)
249+to the test arguments as well.
257250
258251If you add a new feature to pandoc, please add tests as well, following
259252the pattern of the existing tests. The test suite code is in
@@ -318,6 +311,9 @@ With stack:
318311
319312 stack bench
320313
314+You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
315+to get some statistics on memory usage. (Eventually this should
316+be incorporated into the benchmark suite.)
321317
322318Using the REPL
323319--------------
@@ -406,11 +402,11 @@ The library is structured as follows:
406402 tree automatically.
407403 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
408404 are the writers.
409− - `Text.Pandoc.Citeproc.*` contain the code for citation handling,
410− including an interface to the [citeproc] library.
405+ - `Text.Pandoc.Biblio` is a utility module for formatting citations
406+ using citeproc-hs.
411407 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
412− cabal flag is used.
413− - `Text.Pandoc.Emoji` is a thin wrapper around [emojis].
408+ cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
409+ the preprocessor [hsb2hs].
414410 - `Text.Pandoc.Highlighting` contains the interface to the
415411 skylighting library, which is used for code syntax highlighting.
416412 - `Text.Pandoc.ImageSize` is a utility module containing functions for
@@ -421,6 +417,7 @@ The library is structured as follows:
421417 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
422418 LaTeX source.
423419 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
420+ - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
424421 the needs of pandoc.
425422 - `Text.Pandoc.SelfContained` contains functions for making an HTML
426423 file "self-contained," by importing remotely linked images, CSS,
@@ -455,8 +452,6 @@ you may want to consider submitting a pull request to the
455452[EditorConfig]: https://editorconfig.org/
456453[Haskell platform]: https://www.haskell.org/platform/
457454[hlint]: https://hackage.haskell.org/package/hlint
458−[citeproc]: https://hackage.haskell.org/package/citeproc
459−[emojis]: https://hackage.haskell.org/package/emojis
460455[hsb2hs]: https://hackage.haskell.org/package/hsb2hs
461456[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
462457[GitHub labels]: https://github.com/jgm/pandoc/labels
INSTALL.md−4
@@ -34,10 +34,6 @@ without SVG support), [Python] (to use Pandoc filters), and
3434
3535 choco install rsvg-convert python miktex
3636
37−Using multiple installation methods can result in two separate
38−installations of pandoc; it is recommended to properly uninstall
39−pandoc before switching to an alternative installation method.
40−
4137By default, Pandoc creates PDFs using LaTeX.
4238We recommend installing it via [MiKTeX].
4339
MANUAL.txt+62−176
@@ -1,7 +1,7 @@
11---
22title: Pandoc User's Guide
33author: John MacFarlane
4−date: June 20, 2021
4+date: March 4, 2021
55---
66
77# Synopsis
@@ -675,12 +675,12 @@ header when requesting a document from a URL:
675675: Extract images and other media contained in or linked from
676676 the source document to the path *DIR*, creating it if
677677 necessary, and adjust the images references in the document
678− so they point to the extracted files. Media are downloaded,
679− read from the file system, or extracted from a binary
680− container (e.g. docx), as needed. The original file paths
681− are used if they are relative paths not containing `..`.
682− Otherwise filenames are constructed from the SHA1 hash of
683− the contents.
678+ so they point to the extracted files. If the source format is
679+ a binary container (docx, epub, or odt), the media is
680+ extracted from the container and the original
681+ filenames are used. Otherwise the media is read from the
682+ file system or downloaded, and new filenames are constructed
683+ based on SHA1 hashes of the contents.
684684
685685`--abbreviations=`*FILE*
686686
@@ -1415,12 +1415,13 @@ results only for basic math, usually you will want to use
14151415`--gladtex`
14161416
14171417: Enclose TeX math in `<eq>` tags in HTML output. The resulting HTML
1418− can then be processed by [GladTeX] to produce SVG images of the typeset
1419− formulas and an HTML file with these images embedded.
1418+ can then be processed by [GladTeX] to produce images of the typeset
1419+ formulas and an HTML file with links to these images.
1420+ So, the procedure is:
14201421
14211422 pandoc -s --gladtex input.md -o myfile.htex
1422− gladtex -d image_dir myfile.htex
1423− # produces myfile.html and images in image_dir
1423+ gladtex -d myfile-images myfile.htex
1424+ # produces myfile.html and images in myfile-images
14241425
14251426[MathML]: https://www.w3.org/Math/
14261427[MathJax]: https://www.mathjax.org
@@ -2314,8 +2315,8 @@ math equations flush left using [YAML metadata](#layout) or with
23142315
23152316These affect HTML output when [producing slide shows with pandoc].
23162317
2317−`institute`
2318−: author affiliations: can be a list when there are multiple authors
2318+All [reveal.js configuration options] are available as variables.
2319+To turn off boolean flags that default to true in reveal.js, use `0`.
23192320
23202321`revealjs-url`
23212322: base URL for reveal.js documents (defaults to
@@ -2335,9 +2336,6 @@ These affect HTML output when [producing slide shows with pandoc].
23352336: additional attributes for the title slide of reveal.js slide shows.
23362337 See [background in reveal.js and beamer] for an example.
23372338
2338−All [reveal.js configuration options] are available as variables.
2339−To turn off boolean flags that default to true in reveal.js, use `0`.
2340−
23412339[reveal.js configuration options]: https://revealjs.com/config/
23422340
23432341### Variables for Beamer slides
@@ -3754,7 +3752,7 @@ definition:
37543752Note that space between items in a definition list is required.
37553753(A variant that loosens this requirement, but disallows "lazy"
37563754hard wrapping, can be activated with `compact_definition_lists`: see
3757−[Non-default extensions], below.)
3755+[Non-pandoc extensions], below.)
37583756
37593757[^3]: I have been influenced by the suggestions of [David
37603758 Wheeler](https://justatheory.com/2009/02/modest-markdown-proposal/).
@@ -4179,12 +4177,10 @@ option is used. All of the metadata will appear in a single block
41794177at the beginning of the document.
41804178
41814179Note that [YAML] escaping rules must be followed. Thus, for example,
4182−if a title contains a colon, it must be quoted, and if it contains a
4183−backslash escape, then it must be ensured that it is not treated as a
4184−[YAML escape sequence]. The pipe character (`|`) can be used to begin
4185−an indented block that will be interpreted literally, without need for
4186−escaping. This form is necessary when the field contains blank lines
4187−or block-level formatting:
4180+if a title contains a colon, it must be quoted. The pipe character
4181+(`|`) can be used to begin an indented block that will be interpreted
4182+literally, without need for escaping. This form is necessary
4183+when the field contains blank lines or block-level formatting:
41884184
41894185 ---
41904186 title: 'This is the title: it contains a colon'
@@ -4198,12 +4194,6 @@ or block-level formatting:
41984194 It consists of two paragraphs.
41994195 ...
42004196
4201−The literal block after the `|` must be indented relative to the
4202−line containing the `|`. If it is not, the YAML will be invalid
4203−and pandoc will not interpret it as metadata. For an overview
4204−of the complex rules governing YAML, see the [Wikipedia entry on
4205−YAML syntax].
4206−
42074197Template variables will be set automatically from the metadata. Thus, for
42084198example, in writing HTML, the variable `abstract` will be set to the HTML
42094199equivalent of the Markdown in the `abstract` field:
@@ -4250,24 +4240,6 @@ will be interpreted as markdown. For example:
42504240 \renewcommand{\section}[1]{\clearpage\oldsection{#1}}
42514241 ```
42524242
4253−Note: the `yaml_metadata_block` extension works with
4254−`commonmark` as well as `markdown` (and it is enabled by default
4255−in `gfm` and `commonmark_x`). However, in these formats the
4256−following restrictions apply:
4257−
4258−- The YAML metadata block must occur at the beginning of the
4259− document (and there can be only one). If multiple files are
4260− given as arguments to pandoc, only the first can be a YAML
4261− metadata block.
4262−
4263−- The leaf nodes of the YAML structure are parsed in isolation from
4264− each other and from the rest of the document. So, for
4265− example, you can't use a reference link in these contexts
4266− if the link definition is somewhere else in the document.
4267−
4268−[YAML escape sequence]: https://yaml.org/spec/1.2/spec.html#id2776092
4269−[Wikipedia entry on YAML syntax]: https://en.m.wikipedia.org/wiki/YAML#Syntax
4270−
42714243## Backslash escapes
42724244
42734245#### Extension: `all_symbols_escapable` ####
@@ -4960,68 +4932,37 @@ Inline and regular footnotes may be mixed freely.
49604932
49614933#### Extension: `citations` ####
49624934
4963−To cite a bibliographic item with an identifier foo, use the
4964−syntax `@foo`. Normal citations should be included in square
4965−brackets, with semicolons separating distinct items:
4966−
4967− Blah blah [@doe99; @smith2000; @smith2004].
4968−
4969−How this is rendered depends on the citation style. In an
4970−author-date style, it might render as
4971−
4972− Blah blah (Doe 1999, Smith 2000, 2004).
4973−
4974−In a footnote style, it might render as
4975−
4976− Blah blah.[^1]
4977−
4978− [^1]: John Doe, "Frogs," *Journal of Amphibians* 44 (1999);
4979− Susan Smith, "Flies," *Journal of Insects* (2000);
4980− Susan Smith, "Bees," *Journal of Insects* (2004).
4981−
4982−See the [CSL user documentation] for more information about CSL
4983−styles and how they affect rendering.
4984−
4985−Unless a citation key start with a letter, digit, or `_`,
4986−and contains only alphanumerics and internal punctuation
4987−characters (`:.#$%&-+?<>~/`), it must be surrounded
4988−by curly braces, which are not considered part of the key.
4989−In `@Foo_bar.baz.`, the key is `Foo_bar.baz`. The final
4990−period is not *internal* punctuation, so it is not included in
4991−the key. In `@{Foo_bar.baz.}`, the key is `Foo_bar.baz.`, including
4992−the final period. The curly braces are recommended if you use URLs as
4993−keys: `[@{https://example.com/bib?name=foobar&date=2000}, p. 33]`.
4994−
4995−Citation items may optionally include a prefix, a locator, and
4996−a suffix. In
4997−
4998− Blah blah [see @doe99, pp. 33-35 and *passim*; @smith04, chap. 1].
4999−
5000−The first item (`doe99`) has prefix `see `, locator `pp. 33-35`,
5001−and suffix `and *passim*`. The second item (`smith04`) has
5002−locator `chap. 1` and no prefix or suffix.
5003−
5004−Pandoc uses some heuristics to separate the locator from the
5005−rest of the subject. It is sensitive to the locator terms
5006−defined in the [CSL locale files]. Either abbreviated or
5007−unabbreviated forms are accepted. In the `en-US` locale, locator
5008−terms can be written in either singular or plural forms, as
5009−`book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
5010−`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`,
5011−`fol.`/`fols.`; `number`, `no.`/`nos.`; `line`, `l.`/`ll.`;
5012−`note`, `n.`/`nn.`; `opus`, `op.`/`opp.`; `page`, `p.`/`pp.`;
5013−`paragraph`, `para.`/`paras.`; `part`, `pt.`/`pts.`; `section`,
5014−`sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`; `verse`,
5015−`v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
4935+Markdown citations go inside square brackets and are separated
4936+by semicolons. Each citation must have a key, composed of '@' +
4937+the citation identifier from the database, and may optionally
4938+have a prefix, a locator, and a suffix. The citation key must
4939+begin with a letter, digit, or `_`, and may contain
4940+alphanumerics, `_`, and internal punctuation characters
4941+(`:.#$%&-+?<>~/`). Here are some examples:
4942+
4943+ Blah blah [see @doe99, pp. 33-35; also @smith04, chap. 1].
4944+
4945+ Blah blah [@doe99, pp. 33-35, 38-39 and *passim*].
4946+
4947+ Blah blah [@smith04; @doe99].
4948+
4949+`pandoc` detects locator terms in the [CSL locale files].
4950+Either abbreviated or unabbreviated forms are accepted. In the `en-US`
4951+locale, locator terms can be written in either singular or plural forms,
4952+as `book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
4953+`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`, `fol.`/`fols.`;
4954+`number`, `no.`/`nos.`; `line`, `l.`/`ll.`; `note`, `n.`/`nn.`; `opus`,
4955+`op.`/`opp.`; `page`, `p.`/`pp.`; `paragraph`, `para.`/`paras.`; `part`,
4956+`pt.`/`pts.`; `section`, `sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`;
4957+`verse`, `v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
50164958locator term is used, "page" is assumed.
50174959
5018−In complex cases, you can force something to be treated as
5019−a locator by enclosing it in curly braces or prevent parsing
5020−the suffix as locator by prepending curly braces:
4960+`pandoc` will use heuristics to distinguish the locator
4961+from the suffix. In complex cases, the locator can be enclosed
4962+in curly braces:
50214963
50224964 [@smith{ii, A, D-Z}, with a suffix]
50234965 [@smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5024− [@smith{}, 99 years later]
50254966
50264967A minus sign (`-`) before the `@` will suppress mention of
50274968the author in the citation. This can be useful when the
@@ -5029,26 +4970,13 @@ author is already mentioned in the text:
50294970
50304971 Smith says blah [-@smith04].
50314972
5032−You can also write an author-in-text citation, by omitting the
5033−square brackets:
4973+You can also write an in-text citation, as follows:
50344974
50354975 @smith04 says blah.
50364976
50374977 @smith04 [p. 33] says blah.
50384978
5039−This will cause the author's name to be rendered, followed by
5040−the bibliographical details. Use this form when you want to
5041−make the citation the subject of a sentence.
50424979
5043−When you are using a note style, it is usually better to let
5044−citeproc create the footnotes from citations rather than writing
5045−an explicit note. If you do write an explicit note that
5046−contains a citation, note that normal citations will be put in
5047−parentheses, while author-in-text citations will not. For
5048−this reason, it is sometimes preferable to use the
5049−author-in-text style inside notes when using a note style.
5050−
5051−[CSL user documentation]: https://citationstyles.org/authors/
50524980[CSL]: https://docs.citationstyles.org/en/stable/specification.html
50534981[CSL markup specs]: https://docs.citationstyles.org/en/1.0/release-notes.html#rich-text-markup-within-fields
50544982[Chicago Manual of Style]: https://chicagomanualofstyle.org
@@ -5057,44 +4985,13 @@ author-in-text style inside notes when using a note style.
50574985[finding and editing styles]: https://citationstyles.org/authors/
50584986[CSL locale files]: https://github.com/citation-style-language/locales
50594987
5060−## Non-default extensions
4988+## Non-pandoc extensions
50614989
50624990The following Markdown syntax extensions are not enabled by default
50634991in pandoc, but may be enabled by adding `+EXTENSION` to the format
50644992name, where `EXTENSION` is the name of the extension. Thus, for
50654993example, `markdown+hard_line_breaks` is Markdown with hard line breaks.
50664994
5067−#### Extension: `rebase_relative_paths` ####
5068−
5069−Rewrite relative paths for Markdown links and images, depending
5070−on the path of the file containing the link or image link. For
5071−each link or image, pandoc will compute the directory of the
5072−containing file, relative to the working directory, and prepend
5073−the resulting path to the link or image path.
5074−
5075−The use of this extension is best understood by example.
5076−Suppose you have a a subdirectory for each chapter of a book,
5077−`chap1`, `chap2`, `chap3`. Each contains a file `text.md` and a
5078−number of images used in the chapter. You would like to have
5079−`![image](spider.jpg)` in `chap1/text.md` refer to
5080−`chap1/spider.jpg` and `![image](spider.jpg)` in `chap2/text.md`
5081−refer to `chap2/spider.jpg`. To do this, use
5082−
5083− pandoc chap*/*.md -f markdown+rebase_relative_paths
5084−
5085−Without this extension, you would have to use
5086−`![image](chap1/spider.jpg)` in `chap1/text.md` and
5087−`![image](chap2/spider.jpg)` in `chap2/text.md`. Links with
5088−relative paths will be rewritten in the same way as images.
5089−
5090−Absolute paths and URLs are not changed. Neither are empty
5091−paths or paths consisting entirely of a fragment, e.g., `#foo`.
5092−
5093−Note that relative paths in reference links and images will
5094−be rewritten relative to the file containing the link
5095−reference definition, not the file containing the reference link
5096−or image itself, if these differ.
5097−
50984995#### Extension: `attributes` ####
50994996
51004997Allows attributes to be attached to any inline or block-level
@@ -5628,23 +5525,10 @@ A few other metadata fields affect bibliography formatting:
56285525
56295526`lang`
56305527: The `lang` field will affect how the style is localized,
5631− for example in the translation of labels, the use
5632− of quotation marks, and the way items are sorted.
5633− (For backwards compatibility, `locale` may be used instead
5634− of `lang`, but this use is deprecated.)
5635−
5636− A BCP 47 language tag is expected: for example, `en`,
5637− `de`, `en-US`, `fr-CA`, `ug-Cyrl`. The unicode extension
5638− syntax (after `-u-`) may be used to specify options for
5639− collation (sorting) more precisely. Here are some examples:
5640−
5641− - `zh-u-co-pinyin` -- Chinese with the Pinyin collation.
5642− - `es-u-co-trad` -- Spanish with the traditional collation
5643− (with `Ch` sorting after `C`).
5644− - `fr-u-kb` -- French with "backwards" accent sorting
5645− (with `coté` sorting after `côte`).
5646− - `en-US-u-kf-upper` -- English with uppercase letters sorting
5647− before lower (default is lower before upper).
5528+ for example in the translation of labels and the use
5529+ of quotation marks. (For backwards compatibility,
5530+ `locale` may be used instead of `lang`, but this use
5531+ is deprecated.)
56485532
56495533`notes-after-punctuation`
56505534: If true (the default), pandoc will put footnote citations
@@ -5656,6 +5540,12 @@ A few other metadata fields affect bibliography formatting:
56565540 punctuation.
56575541
56585542
5543+
5544+
5545+
5546+
5547+
5548+
56595549# Slide shows
56605550
56615551You can use pandoc to produce an HTML + JavaScript slide presentation
@@ -5748,11 +5638,7 @@ rules:
57485638 * A heading at the slide level always starts a new slide.
57495639
57505640 * Headings *below* the slide level in the hierarchy create
5751− headings *within* a slide. (In beamer, a "block" will be
5752− created. If the heading has the class `example`, an
5753− `exampleblock` environment will be used; if it has the class
5754− `alert`, an `alertblock` will be used; otherwise a regular
5755− `block` will be used.)
5641+ headings *within* a slide.
57565642
57575643 * Headings *above* the slide level in the hierarchy create
57585644 "title slides," which just contain the section title
@@ -6330,7 +6216,7 @@ Then edit `my.theme` and use it like this:
63306216If you are not satisfied with the built-in highlighting, or you
63316217want highlight a language that isn't supported, you can use the
63326218`--syntax-definition` option to load a [KDE-style XML syntax definition
6333−file](https://docs.kde.org/stable5/en/kate/katepart/highlight.html).
6219+file](https://docs.kde.org/stable5/en/applications/katepart/highlight.html).
63346220Before writing your own, have a look at KDE's [repository of syntax
63356221definitions](https://github.com/KDE/syntax-highlighting/tree/master/data/syntax).
63366222
@@ -6504,9 +6390,9 @@ application, here are some things to keep in mind:
650463904. The HTML generated by pandoc is not guaranteed to be safe.
65056391 If `raw_html` is enabled for the Markdown input, users can
65066392 inject arbitrary HTML. Even if `raw_html` is disabled,
6507− users can include dangerous content in URLs and attributes.
6508− To be safe, you should run all the generated HTML through
6509− an HTML sanitizer.
6393+ users can include dangerous content in attributes for
6394+ headings, spans, and code blocks. To be safe, you should
6395+ run all the generated HTML through an HTML sanitizer.
65106396
65116397# Authors
65126398
Makefile+12−19
@@ -7,26 +7,22 @@ DOCKERIMAGE=registry.gitlab.b-data.ch/ghc/ghc4pandoc:8.10.4
77COMMIT=$(shell git rev-parse --short HEAD)
88TIMESTAMP=$(shell date "+%Y%m%d_%H%M")
99LATESTBENCH=$(word 1,$(shell ls -t bench_*.csv 2>/dev/null))
10−BASELINE?=$(LATESTBENCH)
11−ifeq ($(BASELINE),)
12−BASELINECMD=
10+ifeq ($(LATESTBENCH),)
11+BASELINE=
1312else
14−BASELINECMD=--baseline $(BASELINE)
13+BASELINE=--baseline $(LATESTBENCH)
1514endif
16−GHCOPTS=-fdiagnostics-color=always -j4 +RTS -A8m -RTS
15+GHCOPTS=-fdiagnostics-color=always
1716WEBSITE=../../web/pandoc.org
1817REVISION?=1
19−# For gauge:
20−# BENCHARGS?=--small --ci=0.90 --match=pattern $(PATTERN)
21−# For tasty-bench:
22−BENCHARGS?=--csv bench_$(TIMESTAMP).csv $(BASELINECMD) --timeout=6 +RTS -T -RTS $(if $(PATTERN),--pattern "$(PATTERN)",)
18+BENCHARGS?=--timeout=6 +RTS -T -RTS $(if $(PATTERN),--pattern "$(PATTERN)",)
2319
2420quick:
25− stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='$(GHCOPTS)' --test-arguments='-j4 --hide-successes $(TESTARGS)'
21+ stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='-j +RTS -A64m -RTS' --test-arguments='-j4 --hide-successes $(TESTARGS)'
2622
2723quick-cabal:
2824 cabal new-configure . --ghc-options '$(GHCOPTS)' --disable-optimization --enable-tests
29− cabal new-build -j4 . --disable-optimization
25+ cabal new-build . --disable-optimization
3026 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
3127
3228full-cabal:
@@ -35,7 +31,7 @@ full-cabal:
3531 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
3632
3733full:
38− stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 $(GHCOPTS)'
34+ stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 -j4 $(GHCOPTS)'
3935
4036ghci:
4137 stack ghci --flag 'pandoc:embed_data_files'
@@ -51,14 +47,11 @@ test:
5147ghcid:
5248 ghcid -c "stack repl --flag 'pandoc:embed_data_files'"
5349
54−ghcid-test:
55− ghcid -c "stack repl --ghc-options=-XNoImplicitPrelude --flag 'pandoc:embed_data_files' --ghci-options=-fobject-code pandoc:lib pandoc:test-pandoc"
56−
5750bench:
58− stack bench \
59− --ghc-options '$(GHCOPTS)' \
60− --benchmark-arguments='$(BENCHARGS)' 2>&1 | \
61− tee "bench_latest.txt"
51+ stack bench --benchmark-arguments='$(BENCHARGS) $(BASELINE) --csv bench_$(TIMESTAMP).csv' --ghc-options '$(GHCOPTS)'
52+
53+weigh:
54+ stack build --ghc-options '$(GHCOPTS)' pandoc:weigh-pandoc && stack exec weigh-pandoc
6255
6356reformat:
6457 for f in $(SOURCEFILES); do echo $$f; stylish-haskell -i $$f ; done
RELEASE-CHECKLIST−1
@@ -4,7 +4,6 @@
44[ ] Finalize changelog
55[ ] push release candidate branch rc/VERSION and (if it builds
66 successfully) download artifacts
7−[ ] run tools/build-arm.sh to create and download arm64 linux package
87[ ] download macos artifacts from CircleCI
98[ ] Use 'make' in macos and windows artifacts to sign code
109[ ] make update-website
SECURITY.mddeleted−12
@@ -1,12 +0,0 @@
1−# Security Policy
2−
3−## Supported Versions
4−
5−Only the most recent version of pandoc is supported with security updates.
6−
7−## Reporting a Vulnerability
8−
9−To report a vulnerability, email the maintainer, jgm@berkeley.edu.
10−But first please read the section of the manual entitled
11−[A note on security](https://pandoc.org/MANUAL.html#a-note-on-security),
12−which describes some security guarantees pandoc does NOT make.
benchmark/benchmark-pandoc.hs+12−13
@@ -18,13 +18,12 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
1818-}
1919import Text.Pandoc
2020import Text.Pandoc.MIME
21−import Control.DeepSeq (force)
21+import Control.Monad (when)
2222import Control.Monad.Except (throwError)
2323import qualified Text.Pandoc.UTF8 as UTF8
2424import qualified Data.ByteString as B
2525import qualified Data.Text as T
2626import Test.Tasty.Bench
27−-- import Gauge
2827import qualified Data.ByteString.Lazy as BL
2928import Data.Maybe (mapMaybe)
3029import Data.List (sortOn)
@@ -32,10 +31,10 @@ import Data.List (sortOn)
3231readerBench :: Pandoc
3332 -> T.Text
3433 -> Maybe Benchmark
35−readerBench _ name
36− | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
3734readerBench doc name = either (const Nothing) Just $
3835 runPure $ do
36+ when (name `elem` ["bibtex", "biblatex", "csljson"]) $
37+ throwError $ PandocSomeError $ name <> " not supported for benchmark"
3938 (rdr, rexts) <- getReader name
4039 (wtr, wexts) <- getWriter name
4140 case (rdr, wtr) of
@@ -57,19 +56,19 @@ readerBench doc name = either (const Nothing) Just $
5756
5857getImages :: IO [(FilePath, MimeType, BL.ByteString)]
5958getImages = do
60− ll <- B.readFile "test/lalune.jpg"
61− mv <- B.readFile "test/movie.jpg"
62− return [("lalune.jpg", "image/jpg", BL.fromStrict ll)
63− ,("movie.jpg", "image/jpg", BL.fromStrict mv)]
59+ ll <- BL.readFile "test/lalune.jpg"
60+ mv <- BL.readFile "test/movie.jpg"
61+ return [("lalune.jpg", "image/jpg", ll)
62+ ,("movie.jpg", "image/jpg", mv)]
6463
6564writerBench :: [(FilePath, MimeType, BL.ByteString)]
6665 -> Pandoc
6766 -> T.Text
6867 -> Maybe Benchmark
69−writerBench _ _ name
70− | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
7168writerBench imgs doc name = either (const Nothing) Just $
7269 runPure $ do
70+ when (name `elem` ["bibtex", "biblatex", "csljson"]) $
71+ throwError $ PandocSomeError $ name <> " not supported for benchmark"
7372 (wtr, wexts) <- getWriter name
7473 case wtr of
7574 TextWriter writerFun ->
@@ -91,10 +90,10 @@ main :: IO ()
9190main = do
9291 inp <- UTF8.toText <$> B.readFile "test/testsuite.txt"
9392 let opts = def
94− let doc = either (error . show) force $ runPure $ readMarkdown opts inp
93+ let doc = either (error . show) id $ runPure $ readMarkdown opts inp
94+ imgs <- getImages
9595 defaultMain
96− [ env getImages $ \imgs ->
97− bgroup "writers" $ mapMaybe (writerBench imgs doc . fst)
96+ [ bgroup "writers" $ mapMaybe (writerBench imgs doc . fst)
9897 (sortOn fst
9998 writers :: [(T.Text, Writer PandocPure)])
10099 , bgroup "readers" $ mapMaybe (readerBench doc . fst)
cabal.project+8−9
@@ -1,13 +1,12 @@
11packages: pandoc.cabal
2+
23tests: True
3−flags: +embed_data_files
44
5−source-repository-package
6− type: git
7− location: https://github.com/jgm/doctemplates
8− tag: 428c26d5303cf7a2b1051fe1ffd9aafe9ba71c81
5+package pandoc
6+ flags: +embed_data_files
7+ ghc-options: -j +RTS -A256m -RTS
98
10−source-repository-package
11− type: git
12− location: https://github.com/jgm/citeproc
13− tag: 4ce0501cd6f9c86eee087afcf878c256e49b9615
9+-- source-repository-package
10+-- type: git
11+-- location: https://github.com/jgm/citeproc
12+-- tag: d44e24696ab444090d0e63e321c3a573f68b2e74
changelog.md+8−795
@@ -1,783 +1,12 @@
11# Revision history for pandoc
22
3−## pandoc 2.14.0.3 (2021-06-22)
4−
5− * Text.Pandoc.MediaBag `insertMediaBag`: ensure we get a sane mediaPath
6− for URLs (#7391). In earlier 2.14.x versions, we'd get
7− incorrect paths for resources downloaded from URLs when the
8− media are extracted (including in PDF production).
9− * Text.Pandoc.Parsing: improve `emailAddress` (#7398).
10− Previously the parser would accept characters in domains
11− that are illegal in domains, and this sometimes caused it
12− to gobble bits of the following text.
13− * txt2tags reader: modify the email address parser so
14− it still includes form parameters, even after the change to
15− `emailAddress` in Text.Pandoc.Parsing.
16− * Text.Pandoc.Readers.Metadata: Fix regression with comment-only YAML
17− metadata blocks (#7400).
18− * reveal.js writer and template: better handling of options. Previously
19− it was impossible to specify false values for options that default to
20− true (e.g. `center`); setting the option to false just caused the portion
21− of the template setting the option to be omitted. Now we prepopulate
22− all the variables with their default values, including them all
23− unconditionally and allowing them to be overridden.
24− * Markdown writer: Fix regression in code blocks with attributes (#7397).
25− Code blocks with a single class but nonempty attributes
26− were having attributes drop as a result of #7242.
27− * LaTeX writer:
28− + Add strut at end of minipage if it contains line breaks.
29− Without them, the last line is not as tall as it should be in
30− some cases.
31− + Always use a minipage for cells with line breaks, when
32− width information is available (#7393). Otherwise the way we treat them
33− can lead to content that overflows a cell.
34− + Use `\strut` instead of `~` before `\\` in empty line.
35− * Use lts-18.0 stack resolver.
36− * Require skylighting 0.10.5.2 (adding support for Swift).
37− * Require commonmark 0.2.1.
38− * Rephrase section on unsafe HTML in manual.
39− * Create SECURITY.md
40−
41−## pandoc 2.14.0.2 (2021-06-13)
42−
43− * Fix MediaBag regressions (#7345). iIn the 2.14 release `--extract-media`
44− stopped working as before; there could be mismatches between the
45− paths in the rendered document and the extracted media.
46− This patch makes several changes that restore the earlier behavior
47− (while keeping the same API). The `mediaPath` in 2.14 was always
48− constructed from the SHA1 hash of the media contents. Now, we
49− preserve the original path unless it's an absolute path or contains
50− `..` segments (in that case we use a path based on the SHA1 hash of
51− the contents).
52−
53− In Text.Pandoc.MediaBag, `mediaDirectory` and `mediaItems` now use the
54− `mediaPath`, rather than the mediabag key, for the first component of the
55− tuple. This makes more sense, I think, and fits with the documentation of
56− these functions; eventually, though, we should rework the API so that
57− `mediaItems` returns both the keys and the MediaItems.
58−
59− In Text.Pandoc.Class.IO, rewriting of source paths in `extractMedia` has
60− been fixed.
61−
62− In Text.Pandoc.Class.PandocMonad, `fillMediaBag` has been modified so that
63− it doesn't modify image paths (that was part of the problem in #7345).
64−
65− We now do path normalization (e.g. `\` separators on Windows) in
66− writing the media.
67−
68− * Text.Pandoc.PDF:
69−
70− + Text.Pandoc.PDF: Fix regression in 2.14 for generation of PDFs with
71− SVGs (#7344).
72− + Only print relevant part of environment on `--verbose`. Since
73− `--verbose` output might be put in an issue, we want to avoid
74− spilling out secrets in environment variables.
75−
76− * Markdown reader: fix pipe table regression in 2.11.4 (#7343).
77− Previously pipe tables with empty headers (that is, a header
78− line with all empty cells) would be rendered as headerless
79− tables. This broke in 2.11.4. The fix here is to produce an
80− AST with an empty table head when a pipe table has all empty
81− header cells.
82−
83− * LaTeX reader: don't allow optional `*` on symbol control sequences
84− (#7340). Generally we allow optional starred variants of LaTeX commands
85− (since many allow them, and if we don't accept these explicitly,
86− ignoring the star usually gives acceptable results). But we
87− don't want to do this for `\(*\)` and similar cases.
88−
89− * Docx reader: handle absolute URIs in Relationship Target (#7374).
90−
91− * Docx writer: fix handling of empty table headers (Albert Krewinkel,
92− #7369). A table header which does not contain any cells is now treated as
93− an empty header.
94−
95− * LaTeX writer: Fix regression in table header position (#7347).
96− In recent versions the table headers were no longer bottom-aligned
97− (if more than one line). This patch fixes that by using minipages
98− for table headers in non-simple tables.
99−
100− * CommonMark writer:
101−
102− + Do not use simple class for fenced-divs (Jan Tojnar, amends #7242.)
103− + Do not throw away attributes when `Ext_attributes` is enabled.
104− `Ext_attributes` covers at least the following:
105− `Ext_fenced_code_attributes`, `Ext_header_attributes`,
106− `Ext_inline_code_attributes`, `Ext_link_attributes`.
107−
108− * Markdown writer:
109−
110− + Allow `pipe_tables` to be disabled for commonmark formats
111− (`commonmark_x`, `gfm`) (#7375).
112− + Re-use functions from Text.Pandoc.Markdown.Inline (Jan Tojnar).
113−
114− * DocBook writer: Remove non-existent admonitions (Jan Tojnar).
115− `attention`, `error` and `hint` are reStructuredText specific.
116−
117− * HTML writer: Don't omit width attribute on div (#7342).
118−
119− * Text.Pandoc.MIME, `extensionFromMimeType`: add a few special cases.
120− When we do a reverse lookup in the MIME table, we just get the
121− last match, so when the same mime type is associated with several
122− different extensions, we sometimes got weird results, e.g. `.vs`
123− for `text/plain`. These special cases help us get the most standard
124− extensions for mime types like `text/plain`.
125−
126− * Lua utils: fix handling of table headers in `from_simple_table` (Albert
127− Krewinkel, #7369). Passing an empty list of header cells now results
128− in an empty table header.
129−
130− * Text.Pandoc.Citeproc:
131−
132− + Avoid duplicate classes and attributes on references div.
133− + Fix regression in citeproc processing (#7376). If inline
134− references are used (in the metadata `references` field), we
135− should still only include in the bibliography items that are
136− actually cited (unless `nocite` is used).
137−
138− * Require citeproc 0.4.0.1. This fixes a bug which led to doubled
139− "et al." in some (rare) circumstances.
140−
141− * MANUAL.txt:
142−
143− + Mention GladTeX for EPUB export (Sebastian Humenda).
144− This updates the manual and the web site about the GladTeX usage.
145− + More details and a useful link for YAML syntax.
146−
147− * CONTRIBUTING.md: update modules overview (Albert Krewinkel).
148−
149− * using-the-pandoc-api.md: switch from String to Text (Albert Krewinkel).
150−
151−
152−## pandoc 2.14.0.1 (2021-06-01)
153−
154− * Commonmark reader: Fix regression in 2.14 with YAML metdata block parsing,
155− which could cause the document body to be omitted after metadata (#7339).
156−
157− * HTML reader: fix column width regression in 2.14 (#7334).
158− Column widths specified with a style attribute were off by a factor of 100.
159−
160− * Markdown reader: in `rebasePaths`, check for both Windows and Posix
161− absolute paths. Previously Windows pandoc was treating
162− `/foo/bar.jpg` as non-absolute.
163−
164− * Text.Pandoc.Logging: In rendering `LoadedResource`, use relative paths.
165−
166− * Docx writer: fix regression on captions (#7328). The "Table Caption"
167− style was no longer getting applied. (It was overwritten by "Compact.")
168−
169− * Use commonmark-extensions 0.2.1.2
170−
171−## pandoc 2.14 (2021-05-28)
172−
173− * Change reader types, allowing better tracking of source positions
174− [API change]. Previously, when multiple file arguments were provided,
175− pandoc simply concatenated them and passed the contents to the readers,
176− which took a Text argument. As a result, the readers had no way of knowing
177− which file was the source of any particular bit of text. This meant that
178− we couldn't report accurate source positions on errors or include accurate
179− source positions as attributes in the AST. More seriously, it meant that
180− we couldn't resolve resource paths relative to the files containing them
181− (see e.g. #5501, #6632, #6384, #3752).
182−
183− * Add `rebase_relative_paths` extension (#3752). When enabled, this
184− extension rewrites relative image and link paths by prepending
185− the (relative) directory of the containing file. This
186− behavior is useful when your input sources are split
187− into multiple files, across several directories, with files
188− referring to images stored in the same directory. The
189− extension can be enabled for all markdown and commonmark-based formats.
190−
191− * Add Text.Pandoc.Sources (exported module), with a `Sources` type and a
192− `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs
193− [API change]. A parsec `Stream` instance is provided for `Sources`. The
194− module also exports versions of parsec's `satisfy` and other Char parsers
195− that track source positions accurately from a `Sources` stream (or any
196− instance of the new `UpdateSourcePos` class).
197−
198− * Text.Pandoc.Parsing
199−
200− + Export the modified Char parsers defined in Text.Pandoc.Sources
201− instead of the ones parsec provides. Modified parsers to use a
202− `Sources` as stream [API change].
203− + Improve include file functions [API change]. Remove old
204− `insertIncludedFileF`. Give `insertIncludedFile` a more general type,
205− allowing it to be used where `insertIncludedFileF` was.
206− + Add parameter to the `citeKey` parser from
207− Text.Pandoc.Parsing, which controls whether the `@{..}`
208− syntax is allowed [API change].
209−
210− * Text.Pandoc.Error: Modified the constructor `PandocParsecError` to take a
211− `Sources` rather than a `Text` as first argument, so parse error locations
212− can be accurately reported.
213−
214− * Fix source position reporting for YAML bibliographies (#7273).
215−
216− * Issue error message when reader or writer format is malformed
217− (#7231). Previously we exited with an error status but (due to a bug)
218− no message.
219−
220− * Smarter smart quotes (#7216, #2103). Treat a leading `"` with no
221− closing `"` as a left curly quote. This supports the practice, in
222− fiction, of continuing paragraphs quoting the same speaker without an
223− end quote. It also helps with quotes that break over lines in line blocks.
224−
225− * Markdown reader:
226−
227− + Use MetaInlines not MetaBlocks for multimarkdown metadata fields.
228− This gives better results in converting to e.g. pandoc markdown.
229− + Implement curly-brace syntax for Markdown citation keys (#6026).
230− The change provides a way to use citation keys that contain
231− special characters not usable with the standard citation key syntax.
232− Example: `@{foo_bar{x}'}` for the key `foo_bar{x}`. It also allows
233− separating citation keys from immediately following text, e.g. `@{foo}A`.
234−
235− * RST reader:
236−
237− + Seek include files in the directory of the file
238− containing the include directive, as RST requires (#6632).
239− + Use `insertIncludedFile` from Text.Pandoc.Parsing
240− instead of reproducing much of its code.
241−
242− * Org reader: Resolve org includes relative to the directory containing the
243− file containing the INCLUDE directive (#5501).
244−
245− * ODT reader: Treat tabs as spaces (#7185, niszet).
246−
247− * Docx reader:
248−
249− + Add handling of vml image objects (#7257, mbrackeantidot).
250− + Support new table features (Emily Bourke, #6316): column
251− spans, row spans, multiple header rows, table description
252− (parsed as a simple caption), captions, column widths.
253−
254− * LaTeX reader:
255−
256− + Improved siunitx support (#6658, #6620).
257− + Better support for `\xspace` (#7299).
258− + Improve parsing of `\def` macros. We previously set "verbatim mode"
259− even for parsing the initial `\def`; this caused problems
260− for `\def` nested inside another `\def`.
261− + Implement `\newif`.
262−
263− * ConTeXt writer: improve ordered lists (#5016, Denis Maier).
264− Change ordered list from itemize to enumerate. Add new
265− itemgroup for ordered lists. Remove manual insertion of
266− width attributes. Use tabular figures in ordered list
267− enumerators.
268−
269− * HTML reader:
270−
271− + Don't fail on unmatched closing "script" tag (Albert Krenkel, #7282).
272− + Keep h1 tags as normal headers (#2293, Albert Krewinkel).
273− The tags `<title>` and `<h1 class="title">` often contain the same
274− information, so the latter was dropped from the document. However, as
275− this can lead to loss of information, the heading is now always
276− retained. Use `--shift-heading-level-by=-1` to turn the `<h1>`
277− into the document title, or a filter to restore the previous behavior.
278− + Handle relative lengths (e.g. `2*`) in HTML column widths (#4063).
279− See <https://www.w3.org/TR/html4/types.html#h-6.6>.
280−
281− * DocBook/JATS readers:
282−
283− + Fix mathml regression caused by the switch in XML libraries (#7173).
284− + Fix "phrase" in DocBook: take classes from "role" not "class" (#7195).
285−
286− * DocBook reader: ensure that first and last names are separated (#6541).
287−
288− * Jira reader (Albert Krewinkel, #7218):
289−
290− + Support "smart" links: `[alias|https://example.com|smart-card]` syntax.
291− + Allow spaces and most unicode characters in attachment links.
292− + No longer require a newline character after `{noformat}`.
293− + Only allow URI path segment characters in bare links.
294− + The `file:` schema is no longer allowed in bare links; these
295− rarely make sense.
296−
297− * Plain writer: handle superscript unicode minus (#7276).
298−
299− * LaTeX writer:
300−
301− + Better handling of line breaks in simple tables (#7272).
302− Now we also handle the case where they're embedded in other elements,
303− e.g. spans.
304− + For beamer output, support `exampleblock` and `alertblock` (#7278).
305− A block will be rendered as an `exampleblock` if the heading
306− has class `example` and an `alertblock` if it has class `alert`.
307− + Separate successive quote chars with thin space (#6958,
308− Albert Krewinkel). Successive quote characters are separated with
309− a thin space to improve readability and to prevent unwanted ligatures.
310− Detection of these quotes sometimes had failed if the second quote
311− was nested in a span element.
312− + Separate successive quote chars with thin space (#6958, Albert
313− Krewinkel).
314−
315− * EPUB Writer: Fix belongs-to-collection XML id choice (#7267, nuew).
316− The epub writer previously used the same XML id for both the book
317− identifier and the epub collection. This causes an error on epubcheck.
318−
319− * BibTeX/BibLaTeX writer: Handle `annote` field (#7266).
320−
321− * ZimWiki writer: allow links and emphasis in headers (#6605,
322− Albert Krewinkel).
323−
324− * ConTeXt writer:
325−
326− + Support blank lines in line blocks (#6564, Albert Krewinkel,
327− thanks to @denismaier).
328− + Use span identifiers as reference anchors (#7246, Albert Krewinkel).
329−
330− * HTML writer:
331−
332− + Keep attributes from code nested below `pre` tag (#7221,
333− Albert Krewinkel). If a code block is defined with `<pre><code
334− class="language-x">…</code></pre>`, where the `<pre>` element has no
335− attributes, then the attributes from the `<code>` element are used
336− instead. Any leading `language-` prefix is dropped in the code's
337− `class` attribute are dropped to improve syntax highlighting.
338− + Ensure headings only have valid attribs in HTML4 (#5944, Albert
339− Krewinkel).
340− + Parse `<header>` as a Div (Albert Krewinkel).
341−
342− * Org writer:
343−
344− + Inline latex envs need newlines (#7252, tecosaur).
345− As specified in https://orgmode.org/manual/LaTeX-fragments.html, an
346− inline \begin{}...\end{} LaTeX block must start on a new line.
347− + Use LaTeX style maths deliminators (#7196, tecosaur).
348−
349− * JATS writer (Albert Krewinkel):
350−
351− + Use either styled-content or named-content for spans (#7211).
352− If the element has a content-type attribute, or at least one class,
353− then that value is used as `content-type` and the span is put inside
354− a `<named-content>` element. Otherwise a `<styled-content>` element
355− is used instead.
356− + Reduce unnecessary use of `<p>` elements for wrapping (#7227).
357− The `<p>` element is used for wrapping in cases were the contents
358− would otherwise not be allowed in a certain context. Unnecessary
359− wrapping is avoided, especially around quotes (`<disp-quote>` elements).
360− + Convert spans to `<named-content>` elements (#7211). Spans with
361− attributes are converted to `<named-content>` elements instead of
362− being wrapped with `<milestone-start/>` and `<milestone-end>`
363− elements. Milestone elements are not allowed in documents using the
364− articleauthoring tag set, so this change ensures the creation of valid
365− documents.
366− + Add footnote number as label in backmatter (#7210). Footnotes in the
367− backmatter are given the footnote's number as a label. The
368− articleauthoring output is unaffected from this change, as footnotes
369− are placed inline there.
370− + Escape disallows chars in identifiers. XML identifiers must start
371− with an underscore or letter, and can contain only a limited set
372− of punctuation characters. Any IDs not adhering to these rules are
373− rewritten by writing the offending characters as `Uxxxx`,
374− where `xxxx` is the character's hex code.
375−
376− * Jira writer: use `{color}` when span has a color attribute
377− (Albert Krewinkel, tarleb/jira-wiki-markup#10).
378−
379− * Docx writer:
380−
381− + Autoset table width if no column has an explicit width (Albert
382− Krewinkel).
383− + Extract Table handling into separate module (Albert Krewinkel).
384− + Support colspans and rowspans in tables (Albert Krewinkel, #6315).
385− + Support multirow table headers (Albert Krewinkel).
386− + Improve integration of settings from reference.docx (#1209).
387− This change allows users to create a reference.docx that
388− sets `w:proofState` for spelling or grammar to `dirty`,
389− so that spell/grammar checking will be triggered on the
390− generated docx.
391− + Copy over more settings from reference.docx (#7240). From settings.xml
392− in the reference-doc, we now include: `zoom`, `embedSystemFonts`,
393− `doNotTrackMoves`, `defaultTabStop`, `drawingGridHorizontalSpacing`,
394− `drawingGridVerticalSpacing`, `displayHorizontalDrawingGridEvery`,
395− `displayVerticalDrawingGridEvery`, `characterSpacingControl`,
396− `savePreviewPicture`, `mathPr`, `themeFontLang`, `decimalSymbol`,
397− `listSeparator`, `autoHyphenation`, `compat`.
398− + Set zoom to 100% by default in settings.xml.
399− + Align math options more with current Word defaults (e.g. Cambria Math
400− font).
401− + Remove `rsid`s from default settings.xml. Word will add these
402− when revisions are made.
403−
404− * Ms writer: Handle tables with multiple paragraphs (#7288).
405− Previously they overflowed the table cell width. We now set line lengths
406− per-cell and restore them after the table has been written.
407−
408− * Markdown writer:
409−
410− + Use cleaner braceless syntax for code blocks with a
411− single class (#7242, Jan Tojnar).
412− + Add quotes properly in markdown YAML metadata fields (#7245).
413− This fixes a bug, which caused the writer to look at the *last*
414− rather than the *first* character in determining whether quotes
415− were needed. So we got spurious quotes in some cases and
416− didn't get necessary quotes in others.
417− + Use `@{..}` syntax for citations when needed.
418− + Use fewer unneeded escapes for `#` (see #6259).
419− + Improve escaping of `@`. We need to escape literal `@` before
420− `{` because of the new citation syntax.
421−
422− * Commonmark writer: Use backslash escapes for `<` and `|`...
423− instead of entities (#7208).
424−
425− * Powerpoint writer: allow `monofont` to be specified in metadata
426− (#7187).
427−
428− * LaTeX template:
429−
430− + Use non-starred names for xcolor color names (#6109).
431− This should make svgnames and x11names work properly.
432− + Fix bad vertical spacing after bibliography (#7234, badumont).
433− + List of figures before list of tables (#7235, Julien Dutant).
434− + Move CSL macro definitions before header-includes so they can be
435− overridden (#7286).
436− + Improve treatment of CSL `entry-spacing` (#7296).
437− Previously with the default template settings (`indent` variable
438− not set), we would get interparagraph spaces separating bib
439− entries even with `entry-spacing="0"`. On the other hand,
440− setting `entry-spacing="2"` gave ridiculously large spacing.
441− This change makes the spacing caused by `entry-spacing` a multiple
442− of `\parskip` by default, which gives aesthetically reasonable
443− output. Those who want a larger or smaller unit (e.g. because
444− they use `indent` which sets `\parskip` to 0) may
445− `\setlength{\cslentryspacingunit}{10pt}` in header-includes
446− to override the defaults.
447− + Move title, author, date up to top of preamble (#7295).
448− This allows header-includes to use them, and puts them
449− in a position where you can see them immediately.
450− + Define commands for zero width non-joiner character
451− (#6639, Albert Krewinkel). The zero-width non-joiner character
452− is used to avoid ligatures (e.g. in German).
453−
454− * ConTeXt template:
455−
456− + Define `enumerate` itemgroup (#5016, Denis Maier).
457− + List of figures before list of tables (#7235, Julien Dutant).
458−
459− * reveal.js template:
460−
461− + Support `toc-title` (#7171, Florian Kohrt).
462− + Use `hash: true` by default rather than `history: true` (#6968).
463−
464− * HTML-based slide shows: add support for `institute` (#7289, Thomas
465− Hodgson).
466−
467− * Text.Pandoc.Extensions: Add constructor `Ext_rebase_relative_paths` to
468− `Extensions` [API change].
469−
470− * Text.Pandoc.XML.Light: add Eq, Ord instances for Content,
471− Element, Attr, CDataKind [API change].
472−
473− * Text.Pandoc.MediaBag:
474−
475− + Change type to use a `Text` key instead of `[FilePath]`.
476− We normalize the path and use `/` separators for consistency.
477− + Export `MediaItem` type [API change].
478− + Change `MediaBag` type to a map from Text to MediaItem [API change].
479− + `lookupMedia` now returns a `MediaItem` [API change].
480− + Change `insertMedia` so it sets the `mediaPath` to a filename based on
481− the SHA1 hash of the contents. This will be used when contents
482− are extracted.
483−
484− * Text.Pandoc.Class.PandocMonad:
485−
486− + Remove `fetchMediaResource` [API change]. Use `fetchItem` to get
487− resources in `fillMediaBag`.
488− + Add informational message in `downloadOrRead` indicating what path
489− local resources have been loaded from.
490−
491− * Text.Pandoc.Logging:
492−
493− + Remove single quotes around paths in messages.
494− + Add LoadedResource constructor to LogMessage [API change].
495− This is for INFO-level messages telling where image data has been
496− loaded from. (This can vary because of the resource path.)
497−
498− * Text.Pandoc.Asciify: simplify code and export `toAsciiText` [API change].
499− Instead of encoding a giant (and incomplete) map, we now
500− just use unicode-transforms to normalize the text to
501− a canonical decomposition, and manipulate the result.
502−
503− * App: allow tabs expansion even if file-scope is used (Albert Krewinkel,
504− #6709). Tabs in plain-text inputs are now handled correctly, even if
505− the `--file-scope` flag is used.
506−
507− * Add new internal module Text.Pandoc.Writers.GridTable (Albert Krewinkel).
508−
509− * Text.Pandoc.Highlighting: Change type of `languagesByExtension`, adding
510− a parameter for a `SyntaxMap` [API change] (Jan Tojnar, #7241).
511− Languages defined using `--syntax-definition` were not recognized by
512− `languagesByExtension`. This patch corrects that, allowing the writers
513− to see all custom definitions. The LaTeX writer still uses the default
514− syntax map, but that's okay in that context, since
515− `--syntax-definition` won't create new listings styles.
516−
517− * Text.Pandoc.Citeproc:
518−
519− + Ensure that CSL-related attributes are passed on to a Div with id
520− 'refs'. Otherwise things like `entry-spacing` won't work when
521− such Divs are used.
522− + Use metadata's `lang` for the lang parameter of citeproc, overriding
523− `localeLanguage`.
524− + Recognize locators spelled with a capital letter (#7323).
525− + Add a comma and a space in front of the suffix if it doesn't start
526− with space or punctuation (#7324).
527− + Don't detect math elements as locators (#7321).
528−
529− * Remove Text.Pandoc.BCP47 module [API change]. Use types and functions
530− from UnicodeCollation.Lang instead. This is a richer implementation
531− of BCP 47.
532−
533− * Text.Pandoc.Shared:
534−
535− + Fix regression in grid tables for wide characters (#7214).
536− In the translation from String to Text, a char-width-sensitive
537− `splitAt'` was dropped. This commit reinstates it and uses it to make
538− `splitTextByInstances` char-width sensitive.
539− + Add `getLang` (formerly in the now-removed BCP47) [API change].
540−
541− * Text.Pandoc.SelfContained: use `application/octet-stream`
542− for unknown mime types instead of halting with an error (#7202).
543−
544− * Lua filters: respect Inlines/Blocks filter functions in `pandoc.walk_*`
545− (Albert Krewinkel).
546−
547− * Add text as build-depend for trypandoc (#7193, Roman Beránek).
548−
549− * Bump upper-bounds for network-uri, time, attoparsec.
550−
551− * Use citeproc 0.4.
552−
553− * Use texmath 0.12.3.
554−
555− * Use jira-wiki-markup 1.3.5 (Albert Krewinkel).
556−
557− * Require latest skylighting (fixes a bug in XML syntax highlighting).
558−
559− * Use latest xml-conduit.
560−
561− * Use latest commonmark, commonmark-extensions, commonmark-pandoc.
562−
563− * Use haddock-library-1.10.0 (Albert Krewinkel).
564−
565− * Allow compilation with base 4.15 (Albert Krewinkel).
566−
567− * MANUAL:
568−
569− + Add information about `lang` and bibliography sorting.
570− + Add info about YAML escape sequences, link to spec (#7152,
571− Albert Krewinkel).
572− + Note that `institute` variable works for HTML-based slides.
573− + Update documentation on citation syntax.
574− + Add citation example for locators and suffixes (Tristan Stenner)
575−
576− * Updated and fixed typos in documentation (Charanjit Singh,
577− Anti-Distinctlyminty, Tatiana Porras, obcat).
578−
579− * Add instructions for installing pandoc-types before compiling filter.
580−
581− * INSTALL: add note that parallel installations should be avoided
582− (#6865).
583−
584− * Remove `biblatex-nussbaum.md` test. It is basically the same
585− as `biblaetx-quotes.md`.
586−
587− * Command tests: fail if a file contains no tests---and fix a
588− test that failed in that way!
589−
590− * Use smaller images in tests, reducing the size of the source tarball by 8 MB.
591−
592−
593−## pandoc 2.13 (2021-03-21)
594−
595− * Support `yaml_metadata_block` extension for `commonmark`, `gfm` (#6537).
596− This supported is a bit more limited than with pandoc's
597− `markdown`. The YAML block must be the first thing in the input,
598− and the leaf notes are parsed in isolation from the rest of
599− the document. So, for example, you can't use reference
600− links if the references are defined later in the document.
601−
602− * Fix fallback to default partials when custom templates are used.
603− If the directory containing a template does not contain the partial,
604− it should be sought in the default templates, but this was not
605− working properly (#7164).
606−
607− * Handle `nocite` better with `--biblatex` and `--natbib` (#4585).
608− Previously the nocite metadata field was ignored with these formats.
609− Now it populates a `nocite-ids` template variable and causes a
610− `\nocite` command to be issued.
611−
612− * Text.Pandoc.Citeproc: apply `fixLinks` correctly (#7130). This is code
613− that incorporates a prefix like `https://doi.org/` into a following link
614− when appropriate.
615−
616− * Text.Pandoc.Shared:
617−
618− + Remove `backslashEscapes`, `escapeStringUsing` [API change]. Replace
619− these inefficient association list lookups with more efficient escaping
620− functions in the writers that used them (for a 10-25% performance boost
621− in org, haddock, rtf, texinfo writers).
622− + Remove `ToString`, `ToText` typeclasses [API change]. These were needed
623− for the transition from String to Text, but they are no longer used and
624− may clash with other things.
625− + Simplify `compactDL`.
626−
627− * Text.Pandoc.Parsing:
628−
629− + Change type of `readWithM` so that it is no longer polymorphic
630− [API change]. The `ToText` class has been removed, and now that we've
631− completed the transition to Text we no longer need this to operate
632− on Strings.
633− + Remove `F` type synonym [API change]. Muse and Org were defining their
634− own `F` anyway.
635−
636− * Text.Pandoc.Readers.Metadata:
637−
638− + Export `yamlMetaBlock` [API change].
639− + Make `yamlBsToMeta`, `yamlBsToRefs` polymorphic on the parser state
640− [API change].
641−
642− * Markdown reader: Fix regression with `tex_math_backslash` (#7155).
643−
644− * MediaWiki reader: Allow block-level content in notes (ref) (#7145).
645−
646− * Jira reader (Albert Krewinkel):
647−
648− + Fixed parsing of autolinks (i.e., of bare URLs in the text).
649− Previously an autolink would take up the rest of a line, as spaces
650− were allowed characters in these items.
651− + Emoji character sequences no longer cause parsing failures. This was
652− due to missing backtracking when emoji parsing fails.
653− + Mark divs created from panels with class "panel".
654−
655− * RST reader: fix logic for ending comments (#7134). Previously comments
656− sometimes got extended too far.
657−
658− * DocBook writer: include Header attributes as XML attributes on
659− section (Erik Rask). Attributes with key names that are not allowed
660− as XML attributes are dropped, as are attributes with invalid values
661− and `xml:id` (DocBook 5) and `id` (DocBook 4).
662−
663− * Docx writer:
664−
665− + Make `nsid` in `abstractNum` deterministic. Previously we assigned
666− a random number, but we don't need random values, so now we just
667− assign a value based on the list marker.
668− + Use integral values for `w:tblW` (#7141).
669−
670− * Jira writer (Albert Krewinkel):
671−
672− + Block quotes are only rendered as `bq.` if they do not contain a
673− linebreak.
674− + Jira writer: improve div/panel handling. Include div attributes in
675− panels, always render divs with class `panel` as panels, and
676− avoid nesting of panels.
677−
678− * HTML writer: Add warnings on duplicate attribute values.
679− This prevents emitting invalid HTML. Ultimately it would be good to
680− prevent this in the types themselves, but this is better for now.
681−
682− * Org writer: Prevent unintended creation of ordered list items (#7132,
683− Albert Krewinkel). Adjust line wrapping if default wrapping would cause
684− a line to be read as an ordered list item.
685−
686− * JATS templates: support 'equal-contrib' attrib for authors (Albert
687− Krewinkel). Authors who contributed equally to a paper may be marked
688− with `equal-contrib`.
689−
690− * reveal.js template: replace JS comment with HTML (#7154, Florian Kohrt).
691−
692− * Text.Pandoc.Logging: Add `DuplicateAttribute` constructor to `LogMessage`.
693− [API change]
694−
695− * Use `-j4` for linux release build. This speeds up the build dramatically
696− on arm.
697−
698− * cabal.project: remove ghcoptions. Move flags to top level, so they can
699− be set differently on the command line.
700−
701− * Require latest texmath, skylighting, citeproc, jira-wiki-markup.
702− (The latest skylighting fixes a bad bug with Haskell syntax highlighting.)
703− Narrow version bounds for texmath, skylighting, and citeproc, since
704− the test output depend on them.
705−
706− * Use doclayout 0.3.0.2. This significantly reduces the time and memory
707− needed to compile pandoc.
708−
709− * Use `foldl'` instead of `foldl` everywhere.
710−
711− * Update bounds for random (#7156, Alexey Kuleshevich).
712−
713− * Remove uses of some partial functions.
714−
715− * Don't bake in a larger stack size for the executable.
716−
717− * Test improvements:
718−
719− + Use `getExecutablePath` from base, avoiding the dependency on
720− `executable-path`.
721− + Factor out `setupEnvironment` in Helpers, to avoid code duplication.
722− + Fix finding of data files by setting teh `pandoc_datadir` environment
723− variable when we shell out to pandoc. This avoids the need to use
724− `--data-dir` for the tests, which caused problems finding `pandoc.lua`
725− when compiling without the `embed_data_files` flag (#7163).
726−
727− * Benchmark improvements:
728−
729− + Build `+RTS -A8m -RTS` into default ghc-options for benchmark.
730− This is necessary to get accurate benchmark results; otherwise we
731− are largely measuring garbage collecting, some not related to the
732− current benchmark.
733− + Allow specifying BASELINE file in 'make bench' for comparison
734− (otherwise the latest benchmark is chosen by default).
735− + Force `readFile` in benchmarks early (Bodigrim).
736−
737− * CONTRIBUTING: suggest using a `cabal.project.local` file (#7153,
738− Albert Krewinkel).
739−
740− * Add ghcid-test to Makefile. This loads the test suite in ghcid.
741−
742−
743−## pandoc 2.12 (2021-03-08)
744−
745− * `--resource-path` now accumulates if specified multiple
746− times (#6152). Resource paths specified later on the command line are
747− prepended to those specified earlier. Thus,
748− `--resource-path foo --resource-path bar:baz` is equivalent to
749− `--resource-path bar:bas:foo`. (The previous behavior was
750− for the last `--resource-path` to replace all the rest.)
751− `resource-path` in defaults files behaves the same way: it
752− will be prepended to the resource path set by earlier
753− command line options or defaults files. This change
754− facilitates the use of multiple defaults files: each can
755− specify a directory containing resources it refers to
756− without clobbering the resource paths set by the others.
757−
758− * Allow defaults files to refer to the home directory, the
759− user data directory, and the directory containing the defaults file
760− itself (#5871, #5982, #5977). In fields that expect file paths
761− (and only in these fields),
762−
763− + `${VARIABLE}` will expand to the value of the environment variable
764− `VARIABLE` (and in particular `${HOME}` will expand to the path
765− of the home directory). A warning will be raised for undefined
766− variables.
767− + `${USERDATA}` will expand to the path of the user data
768− directory in force when the defaults file is being processed.
769− + `${.}` will expand to the directory containing the defaults file.
770− (This allows default files to be placed in a directory containing
771− resources they make use of.)
3+## pandoc 2.12 (UNRELEASED -- PROVISIONAL)
7724
7735 * When downloading content from URL arguments, be sensitive to
7746 the character encoding (#5600). We can properly handle UTF-8 and latin1
7757 (ISO-8859-1); for others we raise an error. Fall back to latin1 if
7768 no charset is given in the mime type and UTF-8 decoding fails.
7779
778− * Allow abbreviations that don't end in a period to be
779− specified using `--abbreviations` (#7124).
780−
78110 * Add new unexported module Text.Pandoc.XML.Light, as well
78211 as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc,
78312 Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).
@@ -873,7 +102,6 @@
873102 Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Inline}.
874103 Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map
875104 instead of individual commands.
876− + Handle table cells containing `&` in `\verb` (#7129).
877105
878106 * Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change].
879107
@@ -911,10 +139,9 @@
911139 * Jira reader:
912140
913141 + Modified the Doc parser to skip leading blank lines. This fixes
914− parsing of documents which start with multiple blank lines (Albert
915− Krewinkel, #7095).
916− + Prevent URLs within link aliases to be treated as autolinks (Albert
917− Krewinkel, #6944).
142+ parsing of documents which start with multiple blank lines (#7095).
143+ + Prevent URLs within link aliases to be treated as autolinks
144+ (#6944).
918145
919146 * Text.Pandoc.Shared
920147
@@ -923,10 +150,6 @@
923150 and `underlineSpan` (which had been deprecated in April 2020)
924151 [API change].
925152 + Export `handleTaskListItem` (Albert Krewinkel) [API change].
926− + Change `defaultUserDataDirs` to `defaultUserDataDir` [API
927− change]. We determine what is the default user data directory
928− by seeing whether the XDG directory and/or legacy
929− directory exist.
930153
931154 * BibTeX writer:
932155
@@ -961,10 +184,7 @@
961184
962185 * Jira writer:
963186
964− + Use Span identifiers as anchors (Albert Krewinkel).
965− + Use `{noformat}` instead of `{code}` for unknown languages (Albert
966− Krewinkel). Code blocks which are not marked as a language supported
967− by Jira are rendered as preformatted text via `{noformat}` blocks.
187+ + Use Span identifiers as anchors (Albert Krewinkel,
968188
969189 * LaTeX writer:
970190
@@ -1042,8 +262,6 @@
1042262 * Text.Pandoc.App
1043263
1044264 + Add `parseOptionsFromArgs` [API change, new exported function].
1045− + Add fields for CSL options to `Opt` [API change]:
1046− `optCSL`, `optbibliography`, `optCitationAbbreviations`.
1047265
1048266 * Text.Pandoc.Citeproc.BibTeX
1049267
@@ -1087,6 +305,7 @@
1087305 + Export `renderError` [API change].
1088306 + Refactor `handleError` to use `renderError`. This allows us render
1089307 error messages without exiting.
308+ + Remove unused variables (Albert Krewinkel)
1090309
1091310 * Text.Pandoc.Extensions
1092311
@@ -1225,9 +444,6 @@
1225444 * Remove `weigh-pandoc`. It's not really useful any more, now that our
1226445 regular benchmarks include data on allocation.
1227446
1228− * Improve linux package build process and add script to
1229− automate building an arm64 binary package.
1230−
1231447
1232448## pandoc 2.11.4 (2021-01-22)
1233449
@@ -1820,11 +1036,8 @@
18201036 translated to `C:\/foo/bar`, which caused problems.
18211037 With this fix, the backslashes are removed.
18221038
1823− * Text.Pandoc.Logging:
1824−
1825− + Add constructor `ATXHeadingInLHS` to `LogMessage` [API change].
1826− + Add constructor `EnvironmentVariableUndefined` to
1827− `LogMessage` [API change].
1039+ * Text.Pandoc.Logging: Add constructor `ATXHeadingInLHS` constructor
1040+ to `LogMessage` [API change].
18281041
18291042 * Fix error that is given when people specify `doc` output (#6834,
18301043 gison93).
data/docx/word/settings.xml+16−6
@@ -1,6 +1,6 @@
11<?xml version="1.0" encoding="UTF-8"?>
22<w:settings xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:w10="urn:schemas-microsoft-com:office:word" xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:sl="http://schemas.openxmlformats.org/schemaLibrary/2006/main">
3− <w:zoom w:percent="100" />
3+ <w:zoom w:percent="90" />
44 <w:embedSystemFonts />
55 <w:proofState w:spelling="clean" w:grammar="clean" />
66 <w:stylePaneFormatFilter w:val="0004" />
@@ -17,21 +17,31 @@
1717 <w:characterSpacingControl w:val="doNotCompress" />
1818 <w:savePreviewPicture />
1919 <w:rsids>
20+ <w:rsidRoot w:val="00590D07" />
21+ <w:rsid w:val="00011C8B" />
22+ <w:rsid w:val="004E29B3" />
23+ <w:rsid w:val="00590D07" />
24+ <w:rsid w:val="00784D58" />
25+ <w:rsid w:val="008D6863" />
26+ <w:rsid w:val="00B86B75" />
27+ <w:rsid w:val="00BC48D5" />
28+ <w:rsid w:val="00C36279" />
29+ <w:rsid w:val="00E315A3" />
2030 </w:rsids>
2131 <m:mathPr>
22− <m:mathFont m:val="Cambria Math" />
32+ <m:mathFont m:val="Lucida Grande" />
2333 <m:brkBin m:val="before" />
2434 <m:brkBinSub m:val="--" />
25− <m:smallFrac m:val="0" />
26− <m:dispDef />
35+ <m:smallFrac m:val="false" />
36+ <m:dispDef m:val="false" />
2737 <m:lMargin m:val="0" />
2838 <m:rMargin m:val="0" />
2939 <m:wrapRight />
3040 <m:intLim m:val="subSup" />
31− <m:naryLim m:val="undOvr" />
41+ <m:naryLim m:val="subSup" />
3242 </m:mathPr>
3343 <w:themeFontLang w:val="en-US" />
3444 <w:clrSchemeMapping w:bg1="light1" w:t1="dark1" w:bg2="light2" w:t2="dark2" w:accent1="accent1" w:accent2="accent2" w:accent3="accent3" w:accent4="accent4" w:accent5="accent5" w:accent6="accent6" w:hyperlink="hyperlink" w:followedHyperlink="followedHyperlink" />
3545 <w:decimalSymbol w:val="." />
3646 <w:listSeparator w:val="," />
37−</w:settings>
47+</w:settings>
data/templates/article.jats_publishing+1−1
@@ -86,7 +86,7 @@ $endif$
8686$if(author)$
8787<contrib-group>
8888$for(author)$
89−<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
89+<contrib contrib-type="author">
9090$if(author.orcid)$
9191<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
9292$endif$
data/templates/default.context+3−7
@@ -51,7 +51,6 @@ $endif$
5151
5252% use microtypography
5353\definefontfeature[default][default][script=latn, protrusion=quality, expansion=quality, itlc=yes, textitalics=yes, onum=yes, pnum=yes]
54−\definefontfeature[default:tnum][default][tnum=yes, pnum=no]
5554\definefontfeature[smallcaps][script=latn, protrusion=quality, expansion=quality, smcp=yes, onum=yes, pnum=yes]
5655\setupalign[hz,hanging]
5756\setupitaliccorrection[global, always]
@@ -98,9 +97,6 @@ $endif$
9897\setupitemize[autointro] % prevent orphan list intro
9998\setupitemize[indentnext=no]
10099
101−\defineitemgroup[enumerate]
102−\setupenumerate[each][fit][itemalign=left,distance=.5em,style={\feature[+][default:tnum]}]
103−
104100\setupfloat[figure][default={here,nonumber}]
105101\setupfloat[table][default={here,nonumber}]
106102
@@ -164,12 +160,12 @@ $endfor$
164160$if(toc)$
165161\completecontent
166162$endif$
167−$if(lof)$
168−\completelistoffigures
169−$endif$
170163$if(lot)$
171164\completelistoftables
172165$endif$
166+$if(lof)$
167+\completelistoffigures
168+$endif$
173169
174170$body$
175171
data/templates/default.dzslides+1−1
@@ -171,7 +171,7 @@ $if(subtitle)$
171171 <h1 class="subtitle">$subtitle$</h1>
172172$endif$
173173 <footer>
174− $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span> · $endif$$if(institute)$<span class="institute">$for(institute)$$institute$$sep$, $endfor$</span> · $endif$$if(date)$<span class="date">$date$</span>$endif$
174+ $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span>$endif$ · $if(date)$<span class="date">$date$</span>$endif$
175175 </footer>
176176</section>
177177$endif$
data/templates/default.jats_articleauthoring+1−1
@@ -19,7 +19,7 @@ $endif$
1919$if(author)$
2020<contrib-group>
2121$for(author)$
22−<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
22+<contrib contrib-type="author">
2323$if(author.orcid)$
2424<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
2525$endif$
data/templates/default.latex+25−50
@@ -2,7 +2,7 @@
22\PassOptionsToPackage{unicode$for(hyperrefoptions)$,$hyperrefoptions$$endfor$}{hyperref}
33\PassOptionsToPackage{hyphens}{url}
44$if(colorlinks)$
5−\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}
5+\PassOptionsToPackage{dvipsnames,svgnames*,x11names*}{xcolor}
66$endif$
77$if(dir)$
88$if(latex-dir-rtl)$
@@ -156,25 +156,6 @@ $if(CJKmainfont)$
156156 \fi
157157$endif$
158158\fi
159−$if(zero-width-non-joiner)$
160−%% Support for zero-width non-joiner characters.
161−\makeatletter
162−\def\zerowidthnonjoiner{%
163− % Prevent ligatures and adjust kerning, but still support hyphenating.
164− \texorpdfstring{%
165− \textormath{\nobreak\discretionary{-}{}{\kern.03em}%
166− \ifvmode\else\nobreak\hskip\z@skip\fi}{}%
167− }{}%
168−}
169−\makeatother
170−\ifPDFTeX
171− \DeclareUnicodeCharacter{200C}{\zerowidthnonjoiner}
172−\else
173− \catcode`^^^^200c=\active
174− \protected\def ^^^^200c{\zerowidthnonjoiner}
175−\fi
176−%% End of ZWNJ support
177−$endif$
178159$if(beamer)$
179160$if(theme)$
180161\usetheme[$for(themeoptions)$$themeoptions$$sep$,$endfor$]{$theme$}
@@ -344,31 +325,6 @@ $endif$
344325$if(pagestyle)$
345326\pagestyle{$pagestyle$}
346327$endif$
347−$if(csl-refs)$
348−\newlength{\cslhangindent}
349−\setlength{\cslhangindent}{1.5em}
350−\newlength{\csllabelwidth}
351−\setlength{\csllabelwidth}{3em}
352−\newlength{\cslentryspacingunit} % times entry-spacing
353−\setlength{\cslentryspacingunit}{\parskip}
354−\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
355− {% don't indent paragraphs
356− \setlength{\parindent}{0pt}
357− % turn on hanging indent if param 1 is 1
358− \ifodd #1
359− \let\oldpar\par
360− \def\par{\hangindent=\cslhangindent\oldpar}
361− \fi
362− % set entry spacing
363− \setlength{\parskip}{#2\cslentryspacingunit}
364− }%
365− {}
366−\usepackage{calc}
367−\newcommand{\CSLBlock}[1]{#1\hfill\break}
368−\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
369−\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
370−\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
371−$endif$
372328$for(header-includes)$
373329$header-includes$
374330$endfor$
@@ -416,8 +372,27 @@ $for(bibliography)$
416372\addbibresource{$bibliography$}
417373$endfor$
418374$endif$
419−$if(nocite-ids)$
420−\nocite{$for(nocite-ids)$$it$$sep$, $endfor$}
375+$if(csl-refs)$
376+\newlength{\cslhangindent}
377+\setlength{\cslhangindent}{1.5em}
378+\newlength{\csllabelwidth}
379+\setlength{\csllabelwidth}{3em}
380+\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
381+ {% don't indent paragraphs
382+ \setlength{\parindent}{0pt}
383+ % turn on hanging indent if param 1 is 1
384+ \ifodd #1 \everypar{\setlength{\hangindent}{\cslhangindent}}\ignorespaces\fi
385+ % set entry spacing
386+ \ifnum #2 > 0
387+ \setlength{\parskip}{#2\baselineskip}
388+ \fi
389+ }%
390+ {}
391+\usepackage{calc}
392+\newcommand{\CSLBlock}[1]{#1\hfill\break}
393+\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
394+\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
395+\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
421396$endif$
422397$if(csquotes)$
423398\usepackage{csquotes}
@@ -494,12 +469,12 @@ $endif$
494469}
495470$endif$
496471$endif$
497−$if(lof)$
498−\listoffigures
499−$endif$
500472$if(lot)$
501473\listoftables
502474$endif$
475+$if(lof)$
476+\listoffigures
477+$endif$
503478$if(linestretch)$
504479\setstretch{$linestretch$}
505480$endif$
data/templates/default.revealjs+129−79
@@ -52,9 +52,6 @@ $endif$
5252$for(author)$
5353 <p class="author">$author$</p>
5454$endfor$
55−$for(institute)$
56− <p class="institute">$institute$</p>
57−$endfor$
5855$if(date)$
5956 <p class="date">$date$</p>
6057$endif$
@@ -62,12 +59,7 @@ $endif$
6259$endif$
6360$if(toc)$
6461<section id="$idprefix$TOC">
65−<nav role="doc-toc">
66−$if(toc-title)$
67−<h2 id="$idprefix$toc-title">$toc-title$</h2>
68−$endif$
6962$table-of-contents$
70−</nav>
7163</section>
7264$endif$
7365
@@ -77,7 +69,7 @@ $body$
7769
7870 <script src="$revealjs-url$/dist/reveal.js"></script>
7971
80− <!-- reveal.js plugins -->
72+ // reveal.js plugins
8173 <script src="$revealjs-url$/plugin/notes/notes.js"></script>
8274 <script src="$revealjs-url$/plugin/search/search.js"></script>
8375 <script src="$revealjs-url$/plugin/zoom/zoom.js"></script>
@@ -90,198 +82,256 @@ $endif$
9082 // Full list of configuration options available at:
9183 // https://revealjs.com/config/
9284 Reveal.initialize({
85+$if(center)$
86+ // Determines whether slide content should be vertically centered
87+ center: $center$,
88+$endif$
89+$if(controls)$
9390 // Display controls in the bottom right corner
9491 controls: $controls$,
95−
92+$endif$
93+$if(controlsTutorial)$
9694 // Help the user learn the controls by providing hints, for example by
9795 // bouncing the down arrow when they first encounter a vertical slide
9896 controlsTutorial: $controlsTutorial$,
99−
97+$endif$
98+$if(controlsLayout)$
10099 // Determines where controls appear, "edges" or "bottom-right"
101100 controlsLayout: '$controlsLayout$',
102−
101+$endif$
102+$if(controlsBackArrows)$
103103 // Visibility rule for backwards navigation arrows; "faded", "hidden"
104104 // or "visible"
105105 controlsBackArrows: '$controlsBackArrows$',
106−
106+$endif$
107+$if(progress)$
107108 // Display a presentation progress bar
108109 progress: $progress$,
109−
110+$endif$
111+$if(slideNumber)$
110112 // Display the page number of the current slide
111113 slideNumber: $slideNumber$,
112−
114+$endif$
115+$if(showSlideNumber)$
113116 // 'all', 'print', or 'speaker'
114117 showSlideNumber: '$showSlideNumber$',
115−
118+$endif$
119+$if(hash)$
116120 // Add the current slide number to the URL hash so that reloading the
117121 // page/copying the URL will return you to the same slide
118122 hash: $hash$,
119−
123+$endif$
124+$if(hashOneBasedIndex)$
120125 // Start with 1 for the hash rather than 0
121126 hashOneBasedIndex: $hashOneBasedIndex$,
122−
123− // Flags if we should monitor the hash and change slides accordingly
124− respondToHashChanges: $respondToHashChanges$,
125−
127+$endif$
126128 // Push each slide change to the browser history
127− history: $history$,
128−
129+ history: $if(history)$$history$$else$true$endif$,
130+$if(keyboard)$
129131 // Enable keyboard shortcuts for navigation
130132 keyboard: $keyboard$,
131−
133+$endif$
134+$if(overview)$
132135 // Enable the slide overview mode
133136 overview: $overview$,
134−
135− // Disables the default reveal.js slide layout (scaling and centering)
136− // so that you can use custom CSS layout
137− disableLayout: false,
138−
137+$endif$
138+$if(center)$
139139 // Vertical centering of slides
140140 center: $center$,
141−
141+$endif$
142+$if(touch)$
142143 // Enables touch navigation on devices with touch input
143144 touch: $touch$,
144−
145+$endif$
146+$if(loop)$
145147 // Loop the presentation
146148 loop: $loop$,
147−
149+$endif$
150+$if(rtl)$
148151 // Change the presentation direction to be RTL
149152 rtl: $rtl$,
150−
153+$endif$
154+$if(navigationMode)$
151155 // see https://revealjs.com/vertical-slides/#navigation-mode
152156 navigationMode: '$navigationMode$',
153−
157+$endif$
158+$if(shuffle)$
154159 // Randomizes the order of slides each time the presentation loads
155160 shuffle: $shuffle$,
156−
161+$endif$
162+$if(fragments)$
157163 // Turns fragments on and off globally
158164 fragments: $fragments$,
159−
165+$endif$
166+$if(fragmentInURL)$
160167 // Flags whether to include the current fragment in the URL,
161168 // so that reloading brings you to the same fragment position
162169 fragmentInURL: $fragmentInURL$,
163−
170+$endif$
171+$if(embedded)$
164172 // Flags if the presentation is running in an embedded mode,
165173 // i.e. contained within a limited portion of the screen
166174 embedded: $embedded$,
167−
175+$endif$
176+$if(help)$
168177 // Flags if we should show a help overlay when the questionmark
169178 // key is pressed
170179 help: $help$,
171−
172− // Flags if it should be possible to pause the presentation (blackout)
173− pause: $pause$,
174−
180+$endif$
181+$if(pause)$
182+ // Flags if it should be possible to pause the presentation (blackout)
183+ pause: $pause$,
184+$endif$
185+$if(showNotes)$
175186 // Flags if speaker notes should be visible to all viewers
176187 showNotes: $showNotes$,
177−
178− // Global override for autoplaying embedded media (null/true/false)
188+$endif$
189+$if(autoPlayMedia)$
190+ // Global override for autoplaying embedded media (video/audio/iframe)
191+ // - null: Media will only autoplay if data-autoplay is present
192+ // - true: All media will autoplay, regardless of individual setting
193+ // - false: No media will autoplay, regardless of individual setting
179194 autoPlayMedia: $autoPlayMedia$,
180−
181− // Global override for preloading lazy-loaded iframes (null/true/false)
195+$endif$
196+$if(preloadIframes)$
197+ // Global override for preloading lazy-loaded iframes
198+ // - null: Iframes with data-src AND data-preload will be loaded when within
199+ // the viewDistance, iframes with only data-src will be loaded when visible
200+ // - true: All iframes with data-src will be loaded when within the viewDistance
201+ // - false: All iframes with data-src will be loaded only when visible
182202 preloadIframes: $preloadIframes$,
183−
203+$endif$
204+$if(autoSlide)$
184205 // Number of milliseconds between automatically proceeding to the
185206 // next slide, disabled when set to 0, this value can be overwritten
186207 // by using a data-autoslide attribute on your slides
187208 autoSlide: $autoSlide$,
188−
209+$endif$
210+$if(autoSlideStoppable)$
189211 // Stop auto-sliding after user input
190212 autoSlideStoppable: $autoSlideStoppable$,
191−
213+$endif$
214+$if(autoSlideMethod)$
192215 // Use this method for navigation when auto-sliding
193216 autoSlideMethod: $autoSlideMethod$,
194−
217+$endif$
218+$if(defaultTiming)$
195219 // Specify the average time in seconds that you think you will spend
196220 // presenting each slide. This is used to show a pacing timer in the
197221 // speaker view
198222 defaultTiming: $defaultTiming$,
199−
223+$endif$
224+$if(totalTime)$
225+ // Specify the total time in seconds that is available to
226+ // present. If this is set to a nonzero value, the pacing
227+ // timer will work out the time available for each slide,
228+ // instead of using the defaultTiming value
229+ totalTime: $totalTime$,
230+$endif$
231+$if(minimumTimePerSlide)$
232+ // Specify the minimum amount of time you want to allot to
233+ // each slide, if using the totalTime calculation method. If
234+ // the automated time allocation causes slide pacing to fall
235+ // below this threshold, then you will see an alert in the
236+ // speaker notes window
237+ minimumTimePerSlide: $minimumTimePerSlide$,
238+$endif$
239+$if(mouseWheel)$
200240 // Enable slide navigation via mouse wheel
201241 mouseWheel: $mouseWheel$,
202−
203− // The display mode that will be used to show slides
204− display: '$display$',
205−
242+$endif$
243+$if(rollingLinks)$
244+ // Apply a 3D roll to links on hover
245+ rollingLinks: $rollingLinks$,
246+$endif$
247+$if(hideInactiveCursor)$
206248 // Hide cursor if inactive
207249 hideInactiveCursor: $hideInactiveCursor$,
208−
250+$endif$
251+$if(hideCursorTime)$
209252 // Time before the cursor is hidden (in ms)
210253 hideCursorTime: $hideCursorTime$,
211−
254+$endif$
255+$if(hideAddressBar)$
256+ // Hides the address bar on mobile devices
257+ hideAddressBar: $hideAddressBar$,
258+$endif$
259+$if(previewLinks)$
212260 // Opens links in an iframe preview overlay
213261 previewLinks: $previewLinks$,
214−
215− // Transition style (none/fade/slide/convex/concave/zoom)
216− transition: '$transition$',
217−
218− // Transition speed (default/fast/slow)
219− transitionSpeed: '$transitionSpeed$',
220−
262+$endif$
263+$if(transition)$
264+ // Transition style
265+ transition: '$transition$', // none/fade/slide/convex/concave/zoom
266+$endif$
267+$if(transitionSpeed)$
268+ // Transition speed
269+ transitionSpeed: '$transitionSpeed$', // default/fast/slow
270+$endif$
271+$if(backgroundTransition)$
221272 // Transition style for full page slide backgrounds
222− // (none/fade/slide/convex/concave/zoom)
223− backgroundTransition: '$backgroundTransition$',
224−
273+ backgroundTransition: '$backgroundTransition$', // none/fade/slide/convex/concave/zoom
274+$endif$
275+$if(viewDistance)$
225276 // Number of slides away from the current that are visible
226277 viewDistance: $viewDistance$,
227−
278+$endif$
279+$if(mobileViewDistance)$
228280 // Number of slides away from the current that are visible on mobile
229281 // devices. It is advisable to set this to a lower number than
230282 // viewDistance in order to save resources.
231283 mobileViewDistance: $mobileViewDistance$,
284+$endif$
232285$if(parallaxBackgroundImage)$
233−
234286 // Parallax background image
235287 parallaxBackgroundImage: '$parallaxBackgroundImage$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
236288$else$
237289$if(background-image)$
238−
239290 // Parallax background image
240291 parallaxBackgroundImage: '$background-image$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
241292$endif$
242293$endif$
243294$if(parallaxBackgroundSize)$
244−
245295 // Parallax background size
246296 parallaxBackgroundSize: '$parallaxBackgroundSize$', // CSS syntax, e.g. "2100px 900px"
247297$endif$
248298$if(parallaxBackgroundHorizontal)$
249−
250299 // Amount to move parallax background (horizontal and vertical) on slide change
251300 // Number, e.g. 100
252301 parallaxBackgroundHorizontal: $parallaxBackgroundHorizontal$,
253302$endif$
254303$if(parallaxBackgroundVertical)$
255−
256304 parallaxBackgroundVertical: $parallaxBackgroundVertical$,
257305$endif$
258306$if(width)$
259−
260307 // The "normal" size of the presentation, aspect ratio will be preserved
261308 // when the presentation is scaled to fit different resolutions. Can be
262309 // specified using percentage units.
263310 width: $width$,
264311$endif$
265312$if(height)$
266−
267313 height: $height$,
268314$endif$
269315$if(margin)$
270−
271316 // Factor of the display size that should remain empty around the content
272317 margin: $margin$,
273318$endif$
274319$if(minScale)$
275−
276320 // Bounds for smallest/largest possible scale to apply to content
277321 minScale: $minScale$,
278322$endif$
279323$if(maxScale)$
280−
281324 maxScale: $maxScale$,
282325$endif$
326+$if(zoomKey)$
327+ // Modifier key used to click-zoom to part of the slide
328+ zoomKey: '$zoomKey$',
329+$endif$
330+$if(display)$
331+ // The display mode that will be used to show slides
332+ display: '$display$',
333+$endif$
283334$if(mathjax)$
284−
285335 math: {
286336 mathjax: '$mathjaxurl$',
287337 config: 'TeX-AMS_HTML-full',
data/templates/default.s5−3
@@ -61,9 +61,6 @@ $endif$
6161$if(author)$
6262 <h3 class="author">$for(author)$$author$$sep$<br/>$endfor$</h3>
6363$endif$
64−$if(institute)$
65− <h3 class="institute">$for(institute)$$institute$$sep$<br/>$endfor$</h3>
66−$endif$
6764$if(date)$
6865 <h4 class="date">$date$</h4>
6966$endif$
data/templates/default.slideous−7
@@ -63,16 +63,9 @@ $if(title)$
6363$if(subtitle)$
6464 <h1 class="subtitle">$subtitle$</h1>
6565$endif$
66−$if(author)$
6766 <p class="author">
6867$for(author)$$author$$sep$<br/>$endfor$
6968 </p>
70−$endif$
71−$if(institute)$
72− <p class="institute">
73−$for(institute)$$institute$$sep$<br/>$endfor$
74− </p>
75−$endif$
7669$if(date)$
7770 <p class="date">$date$</p>
7871$endif$
data/templates/default.slidy−5
@@ -52,11 +52,6 @@ $if(author)$
5252$for(author)$$author$$sep$<br/>$endfor$
5353 </p>
5454$endif$
55−$if(institute)$
56− <p class="institute">
57−$for(institute)$$institute$$sep$<br/>$endfor$
58− </p>
59−$endif$
6055$if(date)$
6156 <p class="date">$date$</p>
6257$endif$
data/templates/styles.html+1−2
@@ -14,7 +14,7 @@ body {
1414 padding-top: $if(margin-top)$$margin-top$$else$50px$endif$;
1515 padding-bottom: $if(margin-bottom)$$margin-bottom$$else$50px$endif$;
1616 hyphens: auto;
17− overflow-wrap: break-word;
17+ word-wrap: break-word;
1818 text-rendering: optimizeLegibility;
1919 font-kerning: normal;
2020}
@@ -93,7 +93,6 @@ $endif$
9393pre code {
9494 padding: 0;
9595 overflow: visible;
96− overflow-wrap: normal;
9796}
9897.sourceCode {
9998 background-color: transparent;
doc/epub.md+2−9
@@ -141,15 +141,8 @@ Pandoc has an EPUB3 writer. It renders LaTeX math into MathML, which
141141EPUB3 readers are supposed to support (but unfortunately few do).
142142
143143Of course, this isn't much help if you want EPUB2 output (`pandoc -t epub2`)
144−or target readers that don't support MathML. Then you have two options:
145−
146−1. Use the [`--webtex`](https://pandoc.org/MANUAL.html#option--webtex) option,
147− which will use a web service to convert the TeX to an image.
148−2. Use the [`--gladtex`](https://pandoc.org/MANUAL.html#option--gladtex) option
149− to convert maths into SVG images on your local machine.
150−
151−Both GladTeX and WebTeX add the LaTeX source of the formula as alternative text
152−of the image, increasing accessibility for blind users.
144+or target readers that don't support MathML. Then you should try using the
145+`--webtex` option, which will use a web service to convert the TeX to an image.
153146
154147[KindleGen]: https://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000765211
155148[EPUB]: https://en.wikipedia.org/wiki/EPUB
doc/faqs.mddeleted−128
@@ -1,128 +0,0 @@
1−% FAQs
2−
3−::: faqs
4−
5−## How can I convert a whole directory of files from Markdown to RTF?
6−
7−On linux or OSX:
8−
9− for f in *.txt; do pandoc "$f" -s -o "${f%.txt}.rtf"; done
10−
11−
12−In Windows Powershell:
13−
14− gci -r -i *.txt |foreach{$rtf=$_.directoryname+"\"+$_.basename+".rtf";pandoc -f markdown -s $_.name -o $rtf}
15−
16−## I used pandoc to convert a document to ICML (or OPML or RTF), and when I try to open it I'm told it's invalid. What have I done wrong?
17−
18−Be sure to use the `-s` or `--standalone` flag, or you just get a
19−fragment, not a full document with the required header:
20−
21− pandoc -s -f markdown -t icml -o my.icml my.md
22−
23−## I get a blank document when I try to convert a markdown document in Chinese to PDF.
24−
25−By default, pandoc uses pdflatex to generate the PDF, and pdflatex
26−doesn't handle Chinese characters. But you can change the default to
27−use xelatex instead. You should also make sure you're using a font
28−with Chinese glyphs. For example:
29−
30− pandoc -o c.pdf --pdf-engine=xelatex -V mainfont='Adobe Ming Std'
31−
32−## The Windows installer does a single user install, rather than installing pandoc for all users. How can I install pandoc for all users?
33−
34−Run the following command as admin:
35−
36− msiexec /i pandoc-VERSION.msi ALLUSERS=1
37−
38−This will put pandoc in `C:\Program Files\Pandoc`.
39−You can install Pandoc to a different directory by setting APPLICATIONFOLDER parameter,
40−for example:
41−
42− msiexec /i pandoc-1.11.1.msi ALLUSERS=1 APPLICATIONFOLDER="C:\Pandoc"
43−
44−## How do I change the margins in PDF output?
45−
46−The option
47−
48− -V geometry:margin=1in
49−
50−will set the margins to one inch on each side. If you don't want uniform
51−margins, you can do something like
52−
53− -V geometry:"top=2cm, bottom=1.5cm, left=1cm, right=1cm"
54−
55−Or
56−
57− -V geometry:"left=3cm, width=10cm"
58−
59−For more options, see the documentation for the LaTeX [geometry
60−package](https://www.ctan.org/pkg/geometry).
61−
62−## How does pandoc compare to multimarkdown?
63−
64−Here is a [wiki
65−page](https://github.com/jgm/pandoc/wiki/Pandoc-vs-Multimarkdown)
66−comparing the two.
67−
68−## When I specify an image width of 50% and convert to LaTeX, pandoc sets the height to textheight and the aspect ratio isn't preserved. How can I prevent this?
69−
70−For example, if you convert an image with `{width="50%"}`, the LaTeX produced
71−will be `\includegraphics[width=0.5\textwidth,height=\textheight]`.
72−
73−This output presupposes the following code in pandoc's default latex
74−template:
75−
76−```
77−% Scale images if necessary, so that they will not overflow the page
78−% margins by default, and it is still possible to overwrite the defaults
79−% using explicit options in \includegraphics[width, height, ...]{}
80−\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
81−```
82−
83−If you don't have this in your custom template, you should
84−add it. If we didn't set the `height` explicitly in this way,
85−the image would not be resized correctly unless it was
86−being resized to smaller than its original size.
87−
88−## Pandoc sometimes uses too much memory. How can I limit the memory used by pandoc?
89−
90−`pandoc +RTS -M30m -RTS` will limit heap memory to 30MB.
91−When converting a document requires more than this, an out of
92−memory error will be issued.
93−
94−## When using `--include-in-header` with PDF or LaTeX output, how do I reference tex declarations coming after `$header-includes$` in the default template?
95−
96−For various reasons, the `$header-includes$` are not at the very
97−end of the LaTeX preamble. This poses a problem when the code
98−you are inserting depends on declarations in the preamble coming
99−after the `$header-includes$` location. For example, you might
100−want to reference the `\author` and `\title` metadata values
101−(set at the very bottom of the preamble) and present them in
102−margins. In that case you can wrap your code in `etoolbox`'s
103−`\AtEndPreamble`. The technique is demonstrated in [this
104−gist](https://gist.github.com/JohnLukeBentley/9dda6166b9ee5c4127afd2b8cd16b70a).
105−When using `\AtEndPreamble`, keep any `makeatletter` or
106−`makeatother` outside of the `\AtEndPreamble`, as shown in the
107−example.
108−
109−## How can I convert PDFs to other formats using pandoc?
110−
111−You can't. You can try opening the PDF in Word or Google Docs
112−and saving in a format from which pandoc can convert directly.
113−
114−## Do I really need to install a 1 GB TeX installation to produce a PDF using pandoc?
115−
116−No. You can get by with a relatively small TeX installation,
117−for example, by starting with MacTeX's Basic TeX distribution
118−and using the `tlmgr` tool to install a few packages required by pandoc
119−(see https://pandoc.org/MANUAL.html#creating-a-pdf).
120−
121−Or, you can produce PDFs via HTML and `wkhtmltopdf`,
122−or via groff ms and `pdfroff`. (These don't produce as nice
123−topography as TeX, particularly when it comes to math, but they
124−may be fine for many purposes.)
125−
126−
127−:::
128−
doc/filters.md−2
@@ -394,8 +394,6 @@ automatically assign it `Just` the target format or `Nothing`.
394394
395395We compile our script:
396396
397− # first, make sure pandoc-types is installed:
398− cabal install --lib pandoc-types --package-env .
399397 ghc --make handleRuby
400398
401399Then run it:
doc/jats.md−9
@@ -50,13 +50,6 @@ Metadata Values
5050 list must also occur as the `id` of an affiliation listed in
5151 the top-level `affiliation` list.
5252
53− `equal-contrib`
54− : boolean attribute used to mark authors who contributed
55− equally to the work. The
56− [`equal-contrib`][attr:equal-contrib] attribute is added
57− to the author's [`<contrib>`] element if this is set to a
58− truthy value.
59−
6053 `cor-id`
6154 : identifier linking to the contributor's correspondence
6255 information. The info itself must be stored in as an item in
@@ -362,7 +355,6 @@ Required metadata values:
362355[Ringgold]: https://ringgold.com/
363356[attr:content-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/content-type.html
364357[attr:date-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/date-type.html
365−[attr:equal-contrib]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/equal-contrib.html
366358[attr:fn-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/fn-type.html
367359[attr:institution-id-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/institution-id-type.html
368360[attr:iso-8601-date]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/iso-8601-date.html
@@ -406,7 +398,6 @@ Required metadata values:
406398[`<addr-line>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/addr-line.html
407399[`<aff>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/aff.html
408400[`<city>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/city.html
409−[`<contrib>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/contrib.html
410401[`<country>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/country.html
411402[`<institution-id>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-id.html
412403[`<institution-wrap>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-wrap.html
doc/libraries.md−4
@@ -15,9 +15,6 @@ pandoc:
1515[texmath]
1616: Conversion of math between tex, Word equation, MathML, and GNU eqn.
1717
18−[unicode-collation]
19−: Proper Unicode collation (sorting).
20−
2118[doclayout]
2219: Combinators for laying out a textual document, with support
2320 for line wrapping, tabular layout, and more.
@@ -59,5 +56,4 @@ pandoc:
5956[rfc5051]: https://hackage.haskell.org/package/rfc5051
6057[emojis]: https://hackage.haskell.org/package/emojis
6158[jira-wiki-markup]: https://hackage.haskell.org/package/jira-wiki-markup
62−[unicode-collation]: https://hackage.haskell.org/package/unicode-collation
6359
doc/lua-filters.md+2−2
@@ -109,7 +109,7 @@ element filtering function. In other words, filter entries will
109109be called for each corresponding element in the document,
110110getting the respective element as input.
111111
112−The return of a filter function must be one of the following:
112+The return of a filter function must one of the following:
113113
114114- nil: this means that the object should remain unchanged.
115115- a pandoc object: this must be of the same type as the input
@@ -3205,7 +3205,7 @@ Usage:
32053205
32063206# Module pandoc.List
32073207
3208−This module defines pandoc's list type. It comes with useful
3208+The this module defines pandoc's list type. It comes with useful
32093209methods and convenience functions.
32103210
32113211## Constructor
doc/org.md+2−16
@@ -25,7 +25,8 @@ The following export keywords are supported:
2525- LANGUAGE: document language; included as plain-text metadata
2626 field `lang`. The value should be a [BCP47 language tag].
2727
28−- SELECT_TAGS: tags which select a tree for export.
28+- SELECT_TAGS: tags which select a tree for export. Currently
29+ *unsupported*.
2930
3031- EXCLUDE\_TAGS: tags which prevent a subtree from being
3132 exported. Fully supported.
@@ -36,21 +37,6 @@ The following export keywords are supported:
3637 defaults to stdout unless a target has to be given as a command
3738 line option.
3839
39−::: {.alert .alert-info}
40−Pandoc tries to be compatible with org-mode when exporting an org document. If
41−you find some behavior confusing, please do refer to org-mode
42−[Export-Settings](https://orgmode.org/manual/Export-Settings.html)
43−documentation. For example, a common confusion
44−([#3214](https://github.com/jgm/pandoc/issues/3214 "Problem with headers lower
45−then 3 in org-mode reader"), [#5169](https://github.com/jgm/pandoc/issues/5169
46−"org mode headings past level three converted to numbered outline list"),
47−[#6145](https://github.com/jgm/pandoc/issues/6145 "Headers 4 levels deep render
48−differently"), [#7236](https://github.com/jgm/pandoc/issues/7236 "In Org mode,
49−Header with level > 3 are not recognized as headers")) is treatment of headers
50−with level > 3 differently because org-mode sets `org-export-headline-levels`
51−(configurable with `#+OPTIONS: H:3`) to 3 by default.
52−:::
53−
5440[BCP47 language tag]: https://tools.ietf.org/html/bcp47
5541
5642Format-specific options
doc/using-the-pandoc-api.md+12−25
@@ -14,7 +14,7 @@ and types is available at
1414Pandoc is structured as a set of *readers*, which translate
1515various input formats into an abstract syntax tree (the
1616Pandoc AST) representing a structured document, and a set of
17−*writers*, which render this AST into various output formats.
17+*writers*, which render this AST into various input formats.
1818Pictorially:
1919
2020```
@@ -90,14 +90,8 @@ Some notes:
9090Let's look at the types of `readMarkdown` and `writeRST`:
9191
9292```haskell
93−readMarkdown :: (PandocMonad m, ToSources a)
94− => ReaderOptions
95− -> a
96− -> m Pandoc
97−writeRST :: PandocMonad m
98− => WriterOptions
99− -> Pandoc
100− -> m Text
93+readMarkdown :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
94+writeRST :: PandocMonad m => WriterOptions -> Pandoc -> m Text
10195```
10296
10397The `PandocMonad m =>` part is a typeclass constraint.
@@ -139,7 +133,7 @@ report :: PandocMonad m => LogMessage -> m ()
139133-- | Fetch an image or other item from the local filesystem or the net.
140134-- Returns raw content and maybe mime type.
141135fetchItem :: PandocMonad m
142− => Text
136+ => String
143137 -> m (B.ByteString, Maybe MimeType)
144138
145139-- Set the resource path searched by 'fetchItem'.
@@ -161,13 +155,6 @@ Note that `PandocIO` is an instance of `MonadIO`, so you can
161155use `liftIO` to perform arbitrary IO operations inside a pandoc
162156conversion chain.
163157
164−`readMarkdown` is polymorphic in its second argument, which
165−can be any type that is an instance of the `ToSources`
166−typeclass. You can use `Text`, as in the example above.
167−But you can also use `[(FilePath, Text)]`, if the input comes
168−from multiple files and you want to track source positions
169−accurately.
170−
171158# Options
172159
173160The first argument of each reader or writer is for
@@ -213,8 +200,8 @@ of the Monoid typeclass and can easily be concatenated:
213200import Text.Pandoc.Builder
214201
215202mydoc :: Pandoc
216−mydoc = doc $ header 1 (text (T.pack "Hello!"))
217− <> para (emph (text (T.pack "hello world")) <> text (T.pack "."))
203+mydoc = doc $ header 1 (text "Hello!")
204+ <> para (emph (text "hello world") <> text ".")
218205
219206main :: IO ()
220207main = print mydoc
@@ -261,16 +248,16 @@ import qualified Data.Text as T
261248import Data.List (intersperse)
262249
263250data Station = Station{
264− address :: T.Text
265− , name :: T.Text
266− , cardsAccepted :: [T.Text]
251+ address :: String
252+ , name :: String
253+ , cardsAccepted :: [String]
267254 } deriving Show
268255
269256instance FromJSON Station where
270257 parseJSON (Object v) = Station <$>
271258 v .: "street_address" <*>
272259 v .: "station_name" <*>
273− (T.words <$> (v .:? "cards_accepted" .!= ""))
260+ (words <$> (v .:? "cards_accepted" .!= ""))
274261 parseJSON _ = mzero
275262
276263createLetter :: [Station] -> Pandoc
@@ -328,7 +315,7 @@ users to override the system defaults. If you want to disable
328315this behavior, use `setUserDataDir Nothing`.
329316
330317To render a template, use `renderTemplate'`, which takes two
331−arguments, a template (Text) and a context (any instance
318+arguments, a template (String) and a context (any instance
332319of ToJSON). If you want to create a context from the metadata
333320part of a Pandoc document, use `metaToJSON'` from
334321[Text.Pandoc.Writers.Shared]. If you also want to incorporate
@@ -426,7 +413,7 @@ concatenated together. Here's an example that returns a
426413list of the URLs linked to in a document:
427414
428415```haskell
429−listURLs :: Pandoc -> [Text]
416+listURLs :: Pandoc -> [String]
430417listURLs = query urls
431418 where urls (Link _ _ (src, _)) = [src]
432419 urls _ = []
linux/make_artifacts.sh+4−14
@@ -11,14 +11,6 @@ esac
1111
1212ARTIFACTS="${ARTIFACTS:-/artifacts}"
1313
14−# This is our sentinel that tells us when we're done.
15−rm -f $ARTIFACTS/DONE
16−
17−clean_up() {
18− echo "All done!" > "$ARTIFACTS/DONE"
19−}
20−trap clean_up EXIT
21−
2214# build binaries
2315
2416cabal --version
@@ -26,9 +18,9 @@ ghc --version
2618
2719cabal v2-update
2820cabal v2-clean
29−cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-j4 +RTS -A256m -RTS -split-sections -optc-Os -optl=-pthread' pandoc
30−cabal v2-build -j4
31−cabal v2-test -j4
21+cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-split-sections -optc-Os -optl=-pthread' pandoc
22+cabal v2-build
23+cabal v2-test -j1
3224for f in $(find dist-newstyle -name 'pandoc' -type f -perm /400); do cp $f /artifacts/; done
3325
3426# make deb
@@ -78,7 +70,5 @@ mv pandoc $TARGET/bin
7870strip $TARGET/bin/pandoc
7971gzip -9 $TARGET/share/man/man1/pandoc.1
8072
81−tar cvzf $TARGET-linux-$ARCHITECTURE.tar.gz $TARGET
73+tar cvzf $TARGET-linux-amd64.tar.gz $TARGET
8274rm -r $TARGET
83−
84−exit 0
man/pandoc.1+49−233
@@ -1,7 +1,7 @@
11'\" t
2−.\" Automatically generated by Pandoc 2.14.0.1
2+.\" Automatically generated by Pandoc 2.11.4
33.\"
4−.TH "Pandoc User\[cq]s Guide" "" "June 20, 2021" "pandoc 2.14.0.3" ""
4+.TH "Pandoc User\[cq]s Guide" "" "March 4, 2021" "pandoc 2.12" ""
55.hy
66.SH NAME
77pandoc - general markup converter
@@ -457,15 +457,16 @@ On *nix and macOS systems this will be the \f[C]pandoc\f[R] subdirectory
457457of the XDG data directory (by default, \f[C]$HOME/.local/share\f[R],
458458overridable by setting the \f[C]XDG_DATA_HOME\f[R] environment
459459variable).
460−If that directory does not exist and \f[C]$HOME/.pandoc\f[R] exists, it
461−will be used (for backwards compatibility).
462−On Windows the default user data directory is
460+If that directory does not exist, \f[C]$HOME/.pandoc\f[R] will be used
461+(for backwards compatibility).
462+In Windows the default user data directory is
463463\f[C]C:\[rs]Users\[rs]USERNAME\[rs]AppData\[rs]Roaming\[rs]pandoc\f[R].
464464You can find the default user data directory on your system by looking
465465at the output of \f[C]pandoc --version\f[R].
466−Data files placed in this directory (for example,
467−\f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
468−\f[C]templates\f[R]) will override pandoc\[cq]s normal defaults.
466+A \f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
467+\f[C]templates\f[R], \f[C]slidy\f[R], \f[C]slideous\f[R], or
468+\f[C]s5\f[R] directory placed in this directory will override
469+pandoc\[cq]s normal defaults.
469470.TP
470471\f[B]\f[CB]-d\f[B]\f[R] \f[I]FILE\f[R], \f[B]\f[CB]--defaults=\f[B]\f[R]\f[I]FILE\f[R]
471472Specify a set of default option settings.
@@ -716,11 +717,11 @@ Extract images and other media contained in or linked from the source
716717document to the path \f[I]DIR\f[R], creating it if necessary, and adjust
717718the images references in the document so they point to the extracted
718719files.
719−Media are downloaded, read from the file system, or extracted from a
720−binary container (e.g.\ docx), as needed.
721−The original file paths are used if they are relative paths not
722−containing \f[C]..\f[R].
723−Otherwise filenames are constructed from the SHA1 hash of the contents.
720+If the source format is a binary container (docx, epub, or odt), the
721+media is extracted from the container and the original filenames are
722+used.
723+Otherwise the media is read from the file system or downloaded, and new
724+filenames are constructed based on SHA1 hashes of the contents.
724725.TP
725726\f[B]\f[CB]--abbreviations=\f[B]\f[R]\f[I]FILE\f[R]
726727Specifies a custom abbreviations file, with abbreviations one to a line.
@@ -730,9 +731,9 @@ system default.
730731To see the system default, use
731732\f[C]pandoc --print-default-data-file=abbreviations\f[R].
732733The only use pandoc makes of this list is in the Markdown reader.
733−Strings found in this list will be followed by a nonbreaking space, and
734−the period will not produce sentence-ending space in formats like LaTeX.
735−The strings may not contain spaces.
734+Strings ending in a period that are found in this list will be followed
735+by a nonbreaking space, so that the period will not produce
736+sentence-ending space in formats like LaTeX.
736737.SS General writer options
737738.TP
738739\f[B]\f[CB]-s\f[B]\f[R], \f[B]\f[CB]--standalone\f[B]\f[R]
@@ -923,11 +924,6 @@ Note that, if \f[C]--resource-path\f[R] is specified, the working
923924directory must be explicitly listed or it will not be searched.
924925For example: \f[C]--resource-path=.:test\f[R] will search the working
925926directory and the \f[C]test\f[R] subdirectory, in that order.
926−This option can be used repeatedly.
927−Search path components that come later on the command line will be
928−searched before those that come earlier, so
929−\f[C]--resource-path foo:bar --resource-path baz:bim\f[R] is equivalent
930−to \f[C]--resource-path baz:bim:foo:bar\f[R].
931927.TP
932928\f[B]\f[CB]--request-header=\f[B]\f[R]\f[I]NAME\f[R]\f[B]\f[CB]:\f[B]\f[R]\f[I]VAL\f[R]
933929Set the request header \f[I]NAME\f[R] to the value \f[I]VAL\f[R] when
@@ -1504,16 +1500,16 @@ inserted.
15041500.TP
15051501\f[B]\f[CB]--gladtex\f[B]\f[R]
15061502Enclose TeX math in \f[C]<eq>\f[R] tags in HTML output.
1507−The resulting HTML can then be processed by GladTeX to produce SVG
1508−images of the typeset formulas and an HTML file with these images
1509−embedded.
1503+The resulting HTML can then be processed by GladTeX to produce images of
1504+the typeset formulas and an HTML file with links to these images.
1505+So, the procedure is:
15101506.RS
15111507.IP
15121508.nf
15131509\f[C]
15141510pandoc -s --gladtex input.md -o myfile.htex
1515−gladtex -d image_dir myfile.htex
1516−# produces myfile.html and images in image_dir
1511+gladtex -d myfile-images myfile.htex
1512+# produces myfile.html and images in myfile-images
15171513\f[R]
15181514.fi
15191515.RE
@@ -1891,38 +1887,6 @@ verbosity: INFO
18911887\f[R]
18921888.fi
18931889.PP
1894−In fields that expect a file path (or list of file paths), the following
1895−syntax may be used to interpolate environment variables:
1896−.IP
1897−.nf
1898−\f[C]
1899−csl: ${HOME}/mycsldir/special.csl
1900−\f[R]
1901−.fi
1902−.PP
1903−\f[C]${USERDATA}\f[R] may also be used; this will always resolve to the
1904−user data directory that is current when the defaults file is parsed,
1905−regardless of the setting of the environment variable
1906−\f[C]USERDATA\f[R].
1907−.PP
1908−\f[C]${.}\f[R] will resolve to the directory containing the default file
1909−itself.
1910−This allows you to refer to resources contained in that directory:
1911−.IP
1912−.nf
1913−\f[C]
1914−epub-cover-image: ${.}/cover.jpg
1915−epub-metadata: ${.}/meta.xml
1916−resource-path:
1917−- . # the working directory from which pandoc is run
1918−- ${.}/images # the images subdirectory of the directory
1919− # containing this defaults file
1920−\f[R]
1921−.fi
1922−.PP
1923−This environment variable interpolation syntax \f[I]only\f[R] works in
1924−fields that expect file paths.
1925−.PP
19261890Default files can be placed in the \f[C]defaults\f[R] subdirectory of
19271891the user data directory and used from any directory.
19281892For example, one could create a file specifying defaults for writing
@@ -2591,9 +2555,10 @@ YAML metadata or with \f[C]-M classoption=fleqn\f[R].
25912555.SS Variables for HTML slides
25922556.PP
25932557These affect HTML output when [producing slide shows with pandoc].
2594−.TP
2595−\f[B]\f[CB]institute\f[B]\f[R]
2596−author affiliations: can be a list when there are multiple authors
2558+.PP
2559+All reveal.js configuration options are available as variables.
2560+To turn off boolean flags that default to true in reveal.js, use
2561+\f[C]0\f[R].
25972562.TP
25982563\f[B]\f[CB]revealjs-url\f[B]\f[R]
25992564base URL for reveal.js documents (defaults to
@@ -2612,10 +2577,6 @@ base URL for Slideous documents (defaults to \f[C]slideous\f[R])
26122577\f[B]\f[CB]title-slide-attributes\f[B]\f[R]
26132578additional attributes for the title slide of reveal.js slide shows.
26142579See background in reveal.js and beamer for an example.
2615−.PP
2616−All reveal.js configuration options are available as variables.
2617−To turn off boolean flags that default to true in reveal.js, use
2618−\f[C]0\f[R].
26192580.SS Variables for Beamer slides
26202581.PP
26212582These variables change the appearance of PDF slides using
@@ -4275,7 +4236,7 @@ Term 2
42754236Note that space between items in a definition list is required.
42764237(A variant that loosens this requirement, but disallows \[lq]lazy\[rq]
42774238hard wrapping, can be activated with \f[C]compact_definition_lists\f[R]:
4278−see Non-default extensions, below.)
4239+see Non-pandoc extensions, below.)
42794240.SS Numbered example lists
42804241.SS Extension: \f[C]example_lists\f[R]
42814242.PP
@@ -4796,9 +4757,7 @@ All of the metadata will appear in a single block at the beginning of
47964757the document.
47974758.PP
47984759Note that YAML escaping rules must be followed.
4799−Thus, for example, if a title contains a colon, it must be quoted, and
4800−if it contains a backslash escape, then it must be ensured that it is
4801−not treated as a YAML escape sequence.
4760+Thus, for example, if a title contains a colon, it must be quoted.
48024761The pipe character (\f[C]|\f[R]) can be used to begin an indented block
48034762that will be interpreted literally, without need for escaping.
48044763This form is necessary when the field contains blank lines or
@@ -4820,13 +4779,6 @@ abstract: |
48204779\f[R]
48214780.fi
48224781.PP
4823−The literal block after the \f[C]|\f[R] must be indented relative to the
4824−line containing the \f[C]|\f[R].
4825−If it is not, the YAML will be invalid and pandoc will not interpret it
4826−as metadata.
4827−For an overview of the complex rules governing YAML, see the Wikipedia
4828−entry on YAML syntax.
4829−.PP
48304782Template variables will be set automatically from the metadata.
48314783Thus, for example, in writing HTML, the variable \f[C]abstract\f[R] will
48324784be set to the HTML equivalent of the Markdown in the \f[C]abstract\f[R]
@@ -4892,21 +4844,6 @@ header-includes:
48924844 \[ga]\[ga]\[ga]
48934845\f[R]
48944846.fi
4895−.PP
4896−Note: the \f[C]yaml_metadata_block\f[R] extension works with
4897−\f[C]commonmark\f[R] as well as \f[C]markdown\f[R] (and it is enabled by
4898−default in \f[C]gfm\f[R] and \f[C]commonmark_x\f[R]).
4899−However, in these formats the following restrictions apply:
4900−.IP \[bu] 2
4901−The YAML metadata block must occur at the beginning of the document (and
4902−there can be only one).
4903−If multiple files are given as arguments to pandoc, only the first can
4904−be a YAML metadata block.
4905−.IP \[bu] 2
4906−The leaf nodes of the YAML structure are parsed in isolation from each
4907−other and from the rest of the document.
4908−So, for example, you can\[cq]t use a reference link in these contexts if
4909−the link definition is somewhere else in the document.
49104847.SS Backslash escapes
49114848.SS Extension: \f[C]all_symbols_escapable\f[R]
49124849.PP
@@ -5787,70 +5724,27 @@ Inline and regular footnotes may be mixed freely.
57875724.SS Citation syntax
57885725.SS Extension: \f[C]citations\f[R]
57895726.PP
5790−To cite a bibliographic item with an identifier foo, use the syntax
5791−\f[C]\[at]foo\f[R].
5792−Normal citations should be included in square brackets, with semicolons
5793−separating distinct items:
5794−.IP
5795−.nf
5796−\f[C]
5797−Blah blah [\[at]doe99; \[at]smith2000; \[at]smith2004].
5798−\f[R]
5799−.fi
5800−.PP
5801−How this is rendered depends on the citation style.
5802−In an author-date style, it might render as
5803−.IP
5804−.nf
5805−\f[C]
5806−Blah blah (Doe 1999, Smith 2000, 2004).
5807−\f[R]
5808−.fi
5809−.PP
5810−In a footnote style, it might render as
5727+Markdown citations go inside square brackets and are separated by
5728+semicolons.
5729+Each citation must have a key, composed of `\[at]' + the citation
5730+identifier from the database, and may optionally have a prefix, a
5731+locator, and a suffix.
5732+The citation key must begin with a letter, digit, or \f[C]_\f[R], and
5733+may contain alphanumerics, \f[C]_\f[R], and internal punctuation
5734+characters (\f[C]:.#$%&-+?<>\[ti]/\f[R]).
5735+Here are some examples:
58115736.IP
58125737.nf
58135738\f[C]
5814−Blah blah.[\[ha]1]
5739+Blah blah [see \[at]doe99, pp. 33-35; also \[at]smith04, chap. 1].
58155740
5816−[\[ha]1]: John Doe, \[dq]Frogs,\[dq] *Journal of Amphibians* 44 (1999);
5817−Susan Smith, \[dq]Flies,\[dq] *Journal of Insects* (2000);
5818−Susan Smith, \[dq]Bees,\[dq] *Journal of Insects* (2004).
5819−\f[R]
5820−.fi
5821−.PP
5822−See the CSL user documentation for more information about CSL styles and
5823−how they affect rendering.
5824−.PP
5825−Unless a citation key start with a letter, digit, or \f[C]_\f[R], and
5826−contains only alphanumerics and internal punctuation characters
5827−(\f[C]:.#$%&-+?<>\[ti]/\f[R]), it must be surrounded by curly braces,
5828−which are not considered part of the key.
5829−In \f[C]\[at]Foo_bar.baz.\f[R], the key is \f[C]Foo_bar.baz\f[R].
5830−The final period is not \f[I]internal\f[R] punctuation, so it is not
5831−included in the key.
5832−In \f[C]\[at]{Foo_bar.baz.}\f[R], the key is \f[C]Foo_bar.baz.\f[R],
5833−including the final period.
5834−The curly braces are recommended if you use URLs as keys:
5835−\f[C][\[at]{https://example.com/bib?name=foobar&date=2000}, p. 33]\f[R].
5836−.PP
5837−Citation items may optionally include a prefix, a locator, and a suffix.
5838−In
5839−.IP
5840−.nf
5841−\f[C]
5842−Blah blah [see \[at]doe99, pp. 33-35 and *passim*; \[at]smith04, chap. 1].
5741+Blah blah [\[at]doe99, pp. 33-35, 38-39 and *passim*].
5742+
5743+Blah blah [\[at]smith04; \[at]doe99].
58435744\f[R]
58445745.fi
58455746.PP
5846−The first item (\f[C]doe99\f[R]) has prefix \f[C]see\f[R], locator
5847−\f[C]pp. 33-35\f[R], and suffix \f[C]and *passim*\f[R].
5848−The second item (\f[C]smith04\f[R]) has locator \f[C]chap. 1\f[R] and no
5849−prefix or suffix.
5850−.PP
5851−Pandoc uses some heuristics to separate the locator from the rest of the
5852−subject.
5853−It is sensitive to the locator terms defined in the CSL locale files.
5747+\f[C]pandoc\f[R] detects locator terms in the CSL locale files.
58545748Either abbreviated or unabbreviated forms are accepted.
58555749In the \f[C]en-US\f[R] locale, locator terms can be written in either
58565750singular or plural forms, as \f[C]book\f[R],
@@ -5872,15 +5766,14 @@ singular or plural forms, as \f[C]book\f[R],
58725766\f[C]\[sc]\f[R]/\f[C]\[sc]\[sc]\f[R].
58735767If no locator term is used, \[lq]page\[rq] is assumed.
58745768.PP
5875−In complex cases, you can force something to be treated as a locator by
5876−enclosing it in curly braces or prevent parsing the suffix as locator by
5877−prepending curly braces:
5769+\f[C]pandoc\f[R] will use heuristics to distinguish the locator from the
5770+suffix.
5771+In complex cases, the locator can be enclosed in curly braces:
58785772.IP
58795773.nf
58805774\f[C]
58815775[\[at]smith{ii, A, D-Z}, with a suffix]
58825776[\[at]smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5883−[\[at]smith{}, 99 years later]
58845777\f[R]
58855778.fi
58865779.PP
@@ -5894,8 +5787,7 @@ Smith says blah [-\[at]smith04].
58945787\f[R]
58955788.fi
58965789.PP
5897−You can also write an author-in-text citation, by omitting the square
5898−brackets:
5790+You can also write an in-text citation, as follows:
58995791.IP
59005792.nf
59015793\f[C]
@@ -5904,64 +5796,13 @@ brackets:
59045796\[at]smith04 [p. 33] says blah.
59055797\f[R]
59065798.fi
5907−.PP
5908−This will cause the author\[cq]s name to be rendered, followed by the
5909−bibliographical details.
5910−Use this form when you want to make the citation the subject of a
5911−sentence.
5912−.PP
5913−When you are using a note style, it is usually better to let citeproc
5914−create the footnotes from citations rather than writing an explicit
5915−note.
5916−If you do write an explicit note that contains a citation, note that
5917−normal citations will be put in parentheses, while author-in-text
5918−citations will not.
5919−For this reason, it is sometimes preferable to use the author-in-text
5920−style inside notes when using a note style.
5921−.SS Non-default extensions
5799+.SS Non-pandoc extensions
59225800.PP
59235801The following Markdown syntax extensions are not enabled by default in
59245802pandoc, but may be enabled by adding \f[C]+EXTENSION\f[R] to the format
59255803name, where \f[C]EXTENSION\f[R] is the name of the extension.
59265804Thus, for example, \f[C]markdown+hard_line_breaks\f[R] is Markdown with
59275805hard line breaks.
5928−.SS Extension: \f[C]rebase_relative_paths\f[R]
5929−.PP
5930−Rewrite relative paths for Markdown links and images, depending on the
5931−path of the file containing the link or image link.
5932−For each link or image, pandoc will compute the directory of the
5933−containing file, relative to the working directory, and prepend the
5934−resulting path to the link or image path.
5935−.PP
5936−The use of this extension is best understood by example.
5937−Suppose you have a a subdirectory for each chapter of a book,
5938−\f[C]chap1\f[R], \f[C]chap2\f[R], \f[C]chap3\f[R].
5939−Each contains a file \f[C]text.md\f[R] and a number of images used in
5940−the chapter.
5941−You would like to have \f[C]![image](spider.jpg)\f[R] in
5942−\f[C]chap1/text.md\f[R] refer to \f[C]chap1/spider.jpg\f[R] and
5943−\f[C]![image](spider.jpg)\f[R] in \f[C]chap2/text.md\f[R] refer to
5944−\f[C]chap2/spider.jpg\f[R].
5945−To do this, use
5946−.IP
5947−.nf
5948−\f[C]
5949−pandoc chap*/*.md -f markdown+rebase_relative_paths
5950−\f[R]
5951−.fi
5952−.PP
5953−Without this extension, you would have to use
5954−\f[C]![image](chap1/spider.jpg)\f[R] in \f[C]chap1/text.md\f[R] and
5955−\f[C]![image](chap2/spider.jpg)\f[R] in \f[C]chap2/text.md\f[R].
5956−Links with relative paths will be rewritten in the same way as images.
5957−.PP
5958−Absolute paths and URLs are not changed.
5959−Neither are empty paths or paths consisting entirely of a fragment,
5960−e.g., \f[C]#foo\f[R].
5961−.PP
5962−Note that relative paths in reference links and images will be rewritten
5963−relative to the file containing the link reference definition, not the
5964−file containing the reference link or image itself, if these differ.
59655806.SS Extension: \f[C]attributes\f[R]
59665807.PP
59675808Allows attributes to be attached to any inline or block-level element.
@@ -6591,29 +6432,9 @@ entries (for author-date and numerical styles only).
65916432.TP
65926433\f[B]\f[CB]lang\f[B]\f[R]
65936434The \f[C]lang\f[R] field will affect how the style is localized, for
6594−example in the translation of labels, the use of quotation marks, and
6595−the way items are sorted.
6435+example in the translation of labels and the use of quotation marks.
65966436(For backwards compatibility, \f[C]locale\f[R] may be used instead of
65976437\f[C]lang\f[R], but this use is deprecated.)
6598−.RS
6599−.PP
6600−A BCP 47 language tag is expected: for example, \f[C]en\f[R],
6601−\f[C]de\f[R], \f[C]en-US\f[R], \f[C]fr-CA\f[R], \f[C]ug-Cyrl\f[R].
6602−The unicode extension syntax (after \f[C]-u-\f[R]) may be used to
6603−specify options for collation (sorting) more precisely.
6604−Here are some examples:
6605−.IP \[bu] 2
6606−\f[C]zh-u-co-pinyin\f[R] \[en] Chinese with the Pinyin collation.
6607−.IP \[bu] 2
6608−\f[C]es-u-co-trad\f[R] \[en] Spanish with the traditional collation
6609−(with \f[C]Ch\f[R] sorting after \f[C]C\f[R]).
6610−.IP \[bu] 2
6611−\f[C]fr-u-kb\f[R] \[en] French with \[lq]backwards\[rq] accent sorting
6612−(with \f[C]cot\['e]\f[R] sorting after \f[C]c\[^o]te\f[R]).
6613−.IP \[bu] 2
6614−\f[C]en-US-u-kf-upper\f[R] \[en] English with uppercase letters sorting
6615−before lower (default is lower before upper).
6616−.RE
66176438.TP
66186439\f[B]\f[CB]notes-after-punctuation\f[B]\f[R]
66196440If true (the default), pandoc will put footnote citations after
@@ -6734,11 +6555,6 @@ A heading at the slide level always starts a new slide.
67346555.IP \[bu] 2
67356556Headings \f[I]below\f[R] the slide level in the hierarchy create
67366557headings \f[I]within\f[R] a slide.
6737−(In beamer, a \[lq]block\[rq] will be created.
6738−If the heading has the class \f[C]example\f[R], an
6739−\f[C]exampleblock\f[R] environment will be used; if it has the class
6740−\f[C]alert\f[R], an \f[C]alertblock\f[R] will be used; otherwise a
6741−regular \f[C]block\f[R] will be used.)
67426558.IP \[bu] 2
67436559Headings \f[I]above\f[R] the slide level in the hierarchy create
67446560\[lq]title slides,\[rq] which just contain the section title and help to
@@ -7708,7 +7524,7 @@ The HTML generated by pandoc is not guaranteed to be safe.
77087524If \f[C]raw_html\f[R] is enabled for the Markdown input, users can
77097525inject arbitrary HTML.
77107526Even if \f[C]raw_html\f[R] is disabled, users can include dangerous
7711−content in URLs and attributes.
7527+content in attributes for headings, spans, and code blocks.
77127528To be safe, you should run all the generated HTML through an HTML
77137529sanitizer.
77147530.SH AUTHORS
pandoc.cabal+30−40
@@ -1,6 +1,6 @@
11cabal-version: 2.2
22name: pandoc
3−version: 2.14.0.3
3+version: 2.12
44build-type: Simple
55license: GPL-2.0-or-later
66license-file: COPYING.md
@@ -12,7 +12,7 @@ stability: alpha
1212homepage: https://pandoc.org
1313category: Text
1414tested-with: GHC == 8.2.2, GHC == 8.4.4, GHC == 8.6.5,
15− GHC == 8.8.4, GHC == 8.10.2, GHC == 9.0.1
15+ GHC == 8.8.4, GHC == 8.10.2
1616synopsis: Conversion between markup formats
1717description: Pandoc is a Haskell library for converting from one markup
1818 format to another, and a command-line tool that uses
@@ -208,16 +208,12 @@ extra-source-files:
208208 test/command/*.md
209209 test/command/*.csl
210210 test/command/biblio.bib
211− test/command/averroes.bib
211+ test/command/biblatex-examples.bib
212212 test/command/A.txt
213213 test/command/B.txt
214214 test/command/C.txt
215215 test/command/D.txt
216216 test/command/01.csv
217− test/command/chap1/spider.png
218− test/command/chap2/spider.png
219− test/command/chap1/text.md
220− test/command/chap2/text.md
221217 test/command/defaults1.yaml
222218 test/command/defaults2.yaml
223219 test/command/defaults3.yaml
@@ -402,7 +398,8 @@ flag trypandoc
402398
403399common common-options
404400 default-language: Haskell2010
405− build-depends: base >= 4.9 && < 5
401+ build-depends: base >= 4.9 && < 5,
402+ text >= 1.1.1.0 && < 1.3
406403 ghc-options: -Wall -fno-warn-unused-do-bind
407404 -Wincomplete-record-updates
408405 -Wnoncanonical-monad-instances
@@ -430,7 +427,7 @@ common common-options
430427common common-executable
431428 import: common-options
432429 build-depends: pandoc
433− ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
430+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
434431
435432
436433library
@@ -442,30 +439,29 @@ library
442439 SHA >= 1.6 && < 1.7,
443440 aeson >= 0.7 && < 1.6,
444441 aeson-pretty >= 0.8.5 && < 0.9,
445− array >= 0.5 && < 0.6,
446− attoparsec >= 0.12 && < 0.15,
442+ attoparsec >= 0.12 && < 0.14,
447443 base64-bytestring >= 0.1 && < 1.3,
448444 binary >= 0.7 && < 0.11,
449445 blaze-html >= 0.9 && < 0.10,
450446 blaze-markup >= 0.8 && < 0.9,
451447 bytestring >= 0.9 && < 0.12,
452448 case-insensitive >= 1.2 && < 1.3,
453− citeproc >= 0.4.0.1 && < 0.4.1,
454− commonmark >= 0.2.1 && < 0.3,
455− commonmark-extensions >= 0.2.1.2 && < 0.3,
456− commonmark-pandoc >= 0.2.1.1 && < 0.3,
449+ citeproc >= 0.3.0.7 && < 0.4,
450+ commonmark >= 0.1.1.3 && < 0.2,
451+ commonmark-extensions >= 0.2.0.4 && < 0.3,
452+ commonmark-pandoc >= 0.2 && < 0.3,
457453 connection >= 0.3.1,
458454 containers >= 0.4.2.1 && < 0.7,
459455 data-default >= 0.4 && < 0.8,
460456 deepseq >= 1.3 && < 1.5,
461457 directory >= 1.2.3 && < 1.4,
462− doclayout >= 0.3.0.1 && < 0.4,
458+ doclayout >= 0.3 && < 0.4,
463459 doctemplates >= 0.9 && < 0.10,
464460 emojis >= 0.1 && < 0.2,
465461 exceptions >= 0.8 && < 0.11,
466462 file-embed >= 0.0 && < 0.1,
467463 filepath >= 1.1 && < 1.5,
468− haddock-library >= 1.10 && < 1.11,
464+ haddock-library >= 1.8 && < 1.10,
469465 hslua >= 1.1 && < 1.4,
470466 hslua-module-path >= 0.1.0 && < 0.2.0,
471467 hslua-module-system >= 0.2 && < 0.3,
@@ -474,31 +470,30 @@ library
474470 http-client-tls >= 0.2.4 && < 0.4,
475471 http-types >= 0.8 && < 0.13,
476472 ipynb >= 0.1 && < 0.2,
477− jira-wiki-markup >= 1.4 && < 1.5,
473+ jira-wiki-markup >= 1.3.3 && < 1.4,
478474 mtl >= 2.2 && < 2.3,
479475 network >= 2.6,
480− network-uri >= 2.6 && < 2.8,
476+ network-uri >= 2.6 && < 2.7,
481477 pandoc-types >= 1.22 && < 1.23,
482478 parsec >= 3.1 && < 3.2,
483479 process >= 1.2.3 && < 1.7,
484− random >= 1 && < 1.3,
485− safe >= 0.3.18 && < 0.4,
480+ random >= 1 && < 1.2,
481+ safe >= 0.3 && < 0.4,
486482 scientific >= 0.3 && < 0.4,
487− skylighting >= 0.10.5.2 && < 0.10.6,
488− skylighting-core >= 0.10.5.2 && < 0.10.6,
483+ skylighting >= 0.10.4 && < 0.11,
484+ skylighting-core >= 0.10.4 && < 0.11,
489485 split >= 0.2 && < 0.3,
490486 syb >= 0.1 && < 0.8,
491487 tagsoup >= 0.14.6 && < 0.15,
492488 temporary >= 1.1 && < 1.4,
493− texmath >= 0.12.3 && < 0.12.4,
489+ texmath >= 0.12.1 && < 0.13,
494490 text >= 1.1.1.0 && < 1.3,
495491 text-conversions >= 0.3 && < 0.4,
496− time >= 1.5 && < 1.12,
492+ time >= 1.5 && < 1.10,
497493 unicode-transforms >= 0.3 && < 0.4,
498494 unordered-containers >= 0.2 && < 0.3,
499495 xml >= 1.3.12 && < 1.4,
500− xml-conduit >= 1.9.1.1 && < 1.10,
501− unicode-collation >= 0.1.1 && < 0.2,
496+ xml-conduit >= 1.7 && < 1.10,
502497 zip-archive >= 0.2.3.4 && < 0.5,
503498 zlib >= 0.5 && < 0.7
504499 if os(windows) && arch(i386)
@@ -517,7 +512,6 @@ library
517512 Text.Pandoc.Options,
518513 Text.Pandoc.Extensions,
519514 Text.Pandoc.Shared,
520− Text.Pandoc.Sources,
521515 Text.Pandoc.MediaBag,
522516 Text.Pandoc.Error,
523517 Text.Pandoc.Filter,
@@ -607,6 +601,7 @@ library
607601 Text.Pandoc.Asciify,
608602 Text.Pandoc.Emoji,
609603 Text.Pandoc.ImageSize,
604+ Text.Pandoc.BCP47,
610605 Text.Pandoc.Class,
611606 Text.Pandoc.Citeproc
612607 other-modules: Text.Pandoc.App.CommandLineOptions,
@@ -663,9 +658,6 @@ library
663658 Text.Pandoc.Readers.Metadata,
664659 Text.Pandoc.Readers.Roff,
665660 Text.Pandoc.Writers.Docx.StyleMap,
666− Text.Pandoc.Writers.Docx.Table,
667− Text.Pandoc.Writers.Docx.Types,
668− Text.Pandoc.Writers.GridTable
669661 Text.Pandoc.Writers.JATS.References,
670662 Text.Pandoc.Writers.JATS.Table,
671663 Text.Pandoc.Writers.JATS.Types,
@@ -738,7 +730,7 @@ executable trypandoc
738730 main-is: trypandoc.hs
739731 hs-source-dirs: trypandoc
740732 if flag(trypandoc)
741− build-depends: aeson, http-types, text, wai >= 0.3, wai-extra >= 3.0.24
733+ build-depends: aeson, http-types, wai >= 0.3, wai-extra >= 3.0.24
742734 buildable: True
743735 else
744736 buildable: False
@@ -752,11 +744,13 @@ test-suite test-pandoc
752744 Diff >= 0.2 && < 0.5,
753745 Glob >= 0.7 && < 0.11,
754746 QuickCheck >= 2.4 && < 2.15,
747+ base64-bytestring >= 0.1 && < 1.3,
755748 bytestring >= 0.9 && < 0.12,
756749 containers >= 0.4.2.1 && < 0.7,
757750 directory >= 1.2.3 && < 1.4,
758751 doctemplates >= 0.9 && < 0.10,
759752 exceptions >= 0.8 && < 0.11,
753+ executable-path >= 0.0 && < 0.1,
760754 filepath >= 1.1 && < 1.5,
761755 hslua >= 1.1 && < 1.4,
762756 mtl >= 2.2 && < 2.3,
@@ -767,8 +761,8 @@ test-suite test-pandoc
767761 tasty-hunit >= 0.9 && < 0.11,
768762 tasty-lua >= 0.2 && < 0.3,
769763 tasty-quickcheck >= 0.8 && < 0.11,
770− text >= 1.1.1.0 && < 1.3,
771− time >= 1.5 && < 1.12,
764+ temporary >= 1.1 && < 1.4,
765+ time >= 1.5 && < 1.10,
772766 xml >= 1.3.12 && < 1.4,
773767 zip-archive >= 0.2.3.4 && < 0.5
774768 other-modules: Tests.Old
@@ -834,11 +828,7 @@ benchmark benchmark-pandoc
834828 hs-source-dirs: benchmark
835829 build-depends: bytestring,
836830 containers,
837− -- gauge >= 0.2 && < 0.3,
831+ tasty,
838832 tasty-bench >= 0.2 && <= 0.3,
839833 mtl >= 2.2 && < 2.3,
840− text >= 1.1.1.0 && < 1.3,
841− time,
842− deepseq
843− -- we increase heap size to avoid benchmarking garbage collection:
844− ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
834+ time
src/Text/Pandoc/App.hs+13−19
@@ -55,7 +55,7 @@ import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), defaultOpts,
5555import Text.Pandoc.App.CommandLineOptions (parseOptions, parseOptionsFromArgs,
5656 options)
5757import Text.Pandoc.App.OutputSettings (OutputSettings (..), optToOutputSettings)
58−import Text.Collate.Lang (Lang (..), parseLang)
58+import Text.Pandoc.BCP47 (Lang (..), parseBCP47)
5959import Text.Pandoc.Filter (Filter (JSONFilter, LuaFilter), applyFilters)
6060import Text.Pandoc.PDF (makePDF)
6161import Text.Pandoc.SelfContained (makeSelfContained)
@@ -160,11 +160,9 @@ convertWithOpts opts = do
160160 else optTabStop opts)
161161
162162
163− let readSources :: [FilePath] -> PandocIO [(FilePath, Text)]
164− readSources srcs =
165− mapM (\fp -> do
166− t <- readSource fp
167− return (if fp == "-" then "" else fp, convertTabs t)) srcs
163+ let readSources :: [FilePath] -> PandocIO Text
164+ readSources srcs = convertTabs . T.intercalate (T.pack "\n") <$>
165+ mapM readSource srcs
168166
169167
170168 outputSettings <- optToOutputSettings opts
@@ -202,8 +200,8 @@ convertWithOpts opts = do
202200 Just f -> readFileStrict f
203201
204202 case lookupMetaString "lang" (optMetadata opts) of
205− "" -> setTranslations $ Lang "en" Nothing (Just "US") [] [] []
206− l -> case parseLang l of
203+ "" -> setTranslations $ Lang "en" "" "US" []
204+ l -> case parseBCP47 l of
207205 Left _ -> report $ InvalidLang l
208206 Right l' -> setTranslations l'
209207
@@ -257,17 +255,13 @@ convertWithOpts opts = do
257255 let sourceToDoc :: [FilePath] -> PandocIO Pandoc
258256 sourceToDoc sources' =
259257 case reader of
260− TextReader r
261− | readerNameBase == "json" ->
262− mconcat <$> mapM (readSource >=> r readerOpts) sources'
263− | optFileScope opts ->
264− -- Read source and convert tabs (see #6709)
265− let readSource' = fmap convertTabs . readSource
266− in mconcat <$> mapM (readSource' >=> r readerOpts) sources'
267− | otherwise ->
268− readSources sources' >>= r readerOpts
269− ByteStringReader r ->
270− mconcat <$> mapM (readFile' >=> r readerOpts) sources'
258+ TextReader r
259+ | optFileScope opts || readerNameBase == "json" ->
260+ mconcat <$> mapM (readSource >=> r readerOpts) sources'
261+ | otherwise ->
262+ readSources sources' >>= r readerOpts
263+ ByteStringReader r ->
264+ mconcat <$> mapM (readFile' >=> r readerOpts) sources'
271265
272266
273267 when (readerNameBase == "markdown_github" ||
src/Text/Pandoc/App/CommandLineOptions.hs+2−2
@@ -31,7 +31,7 @@ import Data.Aeson.Encode.Pretty (encodePretty', Config(..), keyOrder,
3131 defConfig, Indent(..), NumberFormat(..))
3232import Data.Bifunctor (second)
3333import Data.Char (toLower)
34−import Data.List (intercalate, sort, foldl')
34+import Data.List (intercalate, sort)
3535#ifdef _WINDOWS
3636#if MIN_VERSION_base(4,12,0)
3737import Data.List (isPrefixOf)
@@ -93,7 +93,7 @@ parseOptionsFromArgs options' defaults prg rawArgs = do
9393 ("Try " ++ prg ++ " --help for more information.")
9494
9595 -- thread option data structure through all supplied option actions
96− opts <- foldl' (>>=) (return defaults) actions
96+ opts <- foldl (>>=) (return defaults) actions
9797 let mbArgs = case args of
9898 [] -> Nothing
9999 xs -> Just xs
src/Text/Pandoc/App/FormatHeuristics.hs+7−13
@@ -15,24 +15,18 @@ module Text.Pandoc.App.FormatHeuristics
1515 ) where
1616
1717import Data.Char (toLower)
18−import Data.Foldable (asum)
1918import Data.Text (Text)
2019import System.FilePath (takeExtension)
2120
22−-- | Determines default format based on file extensions; uses the format
23−-- of the first extension that's associated with a format.
24−--
25−-- Examples:
26−--
27−-- > formatFromFilePaths ["text.unknown", "no-extension"]
28−-- Nothing
29−--
30−-- > formatFromFilePaths ["my.md", "other.rst"]
31−-- Just "markdown"
21+-- Determine default format based on file extensions.
3222formatFromFilePaths :: [FilePath] -> Maybe Text
33−formatFromFilePaths = asum . map formatFromFilePath
23+formatFromFilePaths [] = Nothing
24+formatFromFilePaths (x:xs) =
25+ case formatFromFilePath x of
26+ Just f -> Just f
27+ Nothing -> formatFromFilePaths xs
3428
35−-- | Determines format based on file extension.
29+-- Determine format based on file extension
3630formatFromFilePath :: FilePath -> Maybe Text
3731formatFromFilePath x =
3832 case takeExtension (map toLower x) of
src/Text/Pandoc/App/Opt.hs+1−2
@@ -686,8 +686,7 @@ yamlToMeta (Mapping _ _ m) =
686686 either (fail . show) return $ runEverything (yamlMap pMetaString m)
687687 where
688688 pMetaString = pure . MetaString <$> P.manyChar P.anyChar
689− runEverything p =
690− runPure (P.readWithM p (def :: P.ParserState) ("" :: Text))
689+ runEverything p = runPure (P.readWithM p def "")
691690 >>= fmap (Meta . flip P.runF def)
692691yamlToMeta _ = return mempty
693692
src/Text/Pandoc/Asciify.hs+390−13
@@ -10,19 +10,396 @@
1010Function to convert accented latin letters to their unaccented
1111ascii equivalents (used in constructing HTML identifiers).
1212-}
13−module Text.Pandoc.Asciify (toAsciiChar, toAsciiText)
13+module Text.Pandoc.Asciify (toAsciiChar)
1414where
15−import Data.Char (isAscii, isMark)
16−import qualified Data.Text.Normalize as TN
17−import Data.Text (Text)
18−import qualified Data.Text as T
19−
20−toAsciiText :: Text -> Text
21−toAsciiText = T.filter isAscii . TN.normalize (TN.NFD)
15+import Data.Char (isAscii)
16+import qualified Data.Map as M
2217
2318toAsciiChar :: Char -> Maybe Char
24−toAsciiChar c = case T.unpack (TN.normalize TN.NFD (T.singleton c)) of
25− (x:xs) | isAscii x
26− , all isMark xs
27− -> Just x
28− _ -> Nothing
19+toAsciiChar c | isAscii c = Just c
20+ | otherwise = M.lookup c asciiMap
21+
22+asciiMap :: M.Map Char Char
23+asciiMap = M.fromList
24+ [('\192','A')
25+ ,('\193','A')
26+ ,('\194','A')
27+ ,('\195','A')
28+ ,('\196','A')
29+ ,('\197','A')
30+ ,('\199','C')
31+ ,('\200','E')
32+ ,('\201','E')
33+ ,('\202','E')
34+ ,('\203','E')
35+ ,('\204','I')
36+ ,('\205','I')
37+ ,('\206','I')
38+ ,('\207','I')
39+ ,('\209','N')
40+ ,('\210','O')
41+ ,('\211','O')
42+ ,('\212','O')
43+ ,('\213','O')
44+ ,('\214','O')
45+ ,('\217','U')
46+ ,('\218','U')
47+ ,('\219','U')
48+ ,('\220','U')
49+ ,('\221','Y')
50+ ,('\224','a')
51+ ,('\225','a')
52+ ,('\226','a')
53+ ,('\227','a')
54+ ,('\228','a')
55+ ,('\229','a')
56+ ,('\231','c')
57+ ,('\232','e')
58+ ,('\233','e')
59+ ,('\234','e')
60+ ,('\235','e')
61+ ,('\236','i')
62+ ,('\237','i')
63+ ,('\238','i')
64+ ,('\239','i')
65+ ,('\241','n')
66+ ,('\242','o')
67+ ,('\243','o')
68+ ,('\244','o')
69+ ,('\245','o')
70+ ,('\246','o')
71+ ,('\249','u')
72+ ,('\250','u')
73+ ,('\251','u')
74+ ,('\252','u')
75+ ,('\253','y')
76+ ,('\255','y')
77+ ,('\256','A')
78+ ,('\257','a')
79+ ,('\258','A')
80+ ,('\259','a')
81+ ,('\260','A')
82+ ,('\261','a')
83+ ,('\262','C')
84+ ,('\263','c')
85+ ,('\264','C')
86+ ,('\265','c')
87+ ,('\266','C')
88+ ,('\267','c')
89+ ,('\268','C')
90+ ,('\269','c')
91+ ,('\270','D')
92+ ,('\271','d')
93+ ,('\274','E')
94+ ,('\275','e')
95+ ,('\276','E')
96+ ,('\277','e')
97+ ,('\278','E')
98+ ,('\279','e')
99+ ,('\280','E')
100+ ,('\281','e')
101+ ,('\282','E')
102+ ,('\283','e')
103+ ,('\284','G')
104+ ,('\285','g')
105+ ,('\286','G')
106+ ,('\287','g')
107+ ,('\288','G')
108+ ,('\289','g')
109+ ,('\290','G')
110+ ,('\291','g')
111+ ,('\292','H')
112+ ,('\293','h')
113+ ,('\296','I')
114+ ,('\297','i')
115+ ,('\298','I')
116+ ,('\299','i')
117+ ,('\300','I')
118+ ,('\301','i')
119+ ,('\302','I')
120+ ,('\303','i')
121+ ,('\304','I')
122+ ,('\305','i')
123+ ,('\308','J')
124+ ,('\309','j')
125+ ,('\310','K')
126+ ,('\311','k')
127+ ,('\313','L')
128+ ,('\314','l')
129+ ,('\315','L')
130+ ,('\316','l')
131+ ,('\317','L')
132+ ,('\318','l')
133+ ,('\323','N')
134+ ,('\324','n')
135+ ,('\325','N')
136+ ,('\326','n')
137+ ,('\327','N')
138+ ,('\328','n')
139+ ,('\332','O')
140+ ,('\333','o')
141+ ,('\334','O')
142+ ,('\335','o')
143+ ,('\336','O')
144+ ,('\337','o')
145+ ,('\340','R')
146+ ,('\341','r')
147+ ,('\342','R')
148+ ,('\343','r')
149+ ,('\344','R')
150+ ,('\345','r')
151+ ,('\346','S')
152+ ,('\347','s')
153+ ,('\348','S')
154+ ,('\349','s')
155+ ,('\350','S')
156+ ,('\351','s')
157+ ,('\352','S')
158+ ,('\353','s')
159+ ,('\354','T')
160+ ,('\355','t')
161+ ,('\356','T')
162+ ,('\357','t')
163+ ,('\360','U')
164+ ,('\361','u')
165+ ,('\362','U')
166+ ,('\363','u')
167+ ,('\364','U')
168+ ,('\365','u')
169+ ,('\366','U')
170+ ,('\367','u')
171+ ,('\368','U')
172+ ,('\369','u')
173+ ,('\370','U')
174+ ,('\371','u')
175+ ,('\372','W')
176+ ,('\373','w')
177+ ,('\374','Y')
178+ ,('\375','y')
179+ ,('\376','Y')
180+ ,('\377','Z')
181+ ,('\378','z')
182+ ,('\379','Z')
183+ ,('\380','z')
184+ ,('\381','Z')
185+ ,('\382','z')
186+ ,('\416','O')
187+ ,('\417','o')
188+ ,('\431','U')
189+ ,('\432','u')
190+ ,('\461','A')
191+ ,('\462','a')
192+ ,('\463','I')
193+ ,('\464','i')
194+ ,('\465','O')
195+ ,('\466','o')
196+ ,('\467','U')
197+ ,('\468','u')
198+ ,('\486','G')
199+ ,('\487','g')
200+ ,('\488','K')
201+ ,('\489','k')
202+ ,('\490','O')
203+ ,('\491','o')
204+ ,('\496','j')
205+ ,('\500','G')
206+ ,('\501','g')
207+ ,('\504','N')
208+ ,('\505','n')
209+ ,('\512','A')
210+ ,('\513','a')
211+ ,('\514','A')
212+ ,('\515','a')
213+ ,('\516','E')
214+ ,('\517','e')
215+ ,('\518','E')
216+ ,('\519','e')
217+ ,('\520','I')
218+ ,('\521','i')
219+ ,('\522','I')
220+ ,('\523','i')
221+ ,('\524','O')
222+ ,('\525','o')
223+ ,('\526','O')
224+ ,('\527','o')
225+ ,('\528','R')
226+ ,('\529','r')
227+ ,('\530','R')
228+ ,('\531','r')
229+ ,('\532','U')
230+ ,('\533','u')
231+ ,('\534','U')
232+ ,('\535','u')
233+ ,('\536','S')
234+ ,('\537','s')
235+ ,('\538','T')
236+ ,('\539','t')
237+ ,('\542','H')
238+ ,('\543','h')
239+ ,('\550','A')
240+ ,('\551','a')
241+ ,('\552','E')
242+ ,('\553','e')
243+ ,('\558','O')
244+ ,('\559','o')
245+ ,('\562','Y')
246+ ,('\563','y')
247+ ,('\894',';')
248+ ,('\7680','A')
249+ ,('\7681','a')
250+ ,('\7682','B')
251+ ,('\7683','b')
252+ ,('\7684','B')
253+ ,('\7685','b')
254+ ,('\7686','B')
255+ ,('\7687','b')
256+ ,('\7690','D')
257+ ,('\7691','d')
258+ ,('\7692','D')
259+ ,('\7693','d')
260+ ,('\7694','D')
261+ ,('\7695','d')
262+ ,('\7696','D')
263+ ,('\7697','d')
264+ ,('\7698','D')
265+ ,('\7699','d')
266+ ,('\7704','E')
267+ ,('\7705','e')
268+ ,('\7706','E')
269+ ,('\7707','e')
270+ ,('\7710','F')
271+ ,('\7711','f')
272+ ,('\7712','G')
273+ ,('\7713','g')
274+ ,('\7714','H')
275+ ,('\7715','h')
276+ ,('\7716','H')
277+ ,('\7717','h')
278+ ,('\7718','H')
279+ ,('\7719','h')
280+ ,('\7720','H')
281+ ,('\7721','h')
282+ ,('\7722','H')
283+ ,('\7723','h')
284+ ,('\7724','I')
285+ ,('\7725','i')
286+ ,('\7728','K')
287+ ,('\7729','k')
288+ ,('\7730','K')
289+ ,('\7731','k')
290+ ,('\7732','K')
291+ ,('\7733','k')
292+ ,('\7734','L')
293+ ,('\7735','l')
294+ ,('\7738','L')
295+ ,('\7739','l')
296+ ,('\7740','L')
297+ ,('\7741','l')
298+ ,('\7742','M')
299+ ,('\7743','m')
300+ ,('\7744','M')
301+ ,('\7745','m')
302+ ,('\7746','M')
303+ ,('\7747','m')
304+ ,('\7748','N')
305+ ,('\7749','n')
306+ ,('\7750','N')
307+ ,('\7751','n')
308+ ,('\7752','N')
309+ ,('\7753','n')
310+ ,('\7754','N')
311+ ,('\7755','n')
312+ ,('\7764','P')
313+ ,('\7765','p')
314+ ,('\7766','P')
315+ ,('\7767','p')
316+ ,('\7768','R')
317+ ,('\7769','r')
318+ ,('\7770','R')
319+ ,('\7771','r')
320+ ,('\7774','R')
321+ ,('\7775','r')
322+ ,('\7776','S')
323+ ,('\7777','s')
324+ ,('\7778','S')
325+ ,('\7779','s')
326+ ,('\7786','T')
327+ ,('\7787','t')
328+ ,('\7788','T')
329+ ,('\7789','t')
330+ ,('\7790','T')
331+ ,('\7791','t')
332+ ,('\7792','T')
333+ ,('\7793','t')
334+ ,('\7794','U')
335+ ,('\7795','u')
336+ ,('\7796','U')
337+ ,('\7797','u')
338+ ,('\7798','U')
339+ ,('\7799','u')
340+ ,('\7804','V')
341+ ,('\7805','v')
342+ ,('\7806','V')
343+ ,('\7807','v')
344+ ,('\7808','W')
345+ ,('\7809','w')
346+ ,('\7810','W')
347+ ,('\7811','w')
348+ ,('\7812','W')
349+ ,('\7813','w')
350+ ,('\7814','W')
351+ ,('\7815','w')
352+ ,('\7816','W')
353+ ,('\7817','w')
354+ ,('\7818','X')
355+ ,('\7819','x')
356+ ,('\7820','X')
357+ ,('\7821','x')
358+ ,('\7822','Y')
359+ ,('\7823','y')
360+ ,('\7824','Z')
361+ ,('\7825','z')
362+ ,('\7826','Z')
363+ ,('\7827','z')
364+ ,('\7828','Z')
365+ ,('\7829','z')
366+ ,('\7830','h')
367+ ,('\7831','t')
368+ ,('\7832','w')
369+ ,('\7833','y')
370+ ,('\7840','A')
371+ ,('\7841','a')
372+ ,('\7842','A')
373+ ,('\7843','a')
374+ ,('\7864','E')
375+ ,('\7865','e')
376+ ,('\7866','E')
377+ ,('\7867','e')
378+ ,('\7868','E')
379+ ,('\7869','e')
380+ ,('\7880','I')
381+ ,('\7881','i')
382+ ,('\7882','I')
383+ ,('\7883','i')
384+ ,('\7884','O')
385+ ,('\7885','o')
386+ ,('\7886','O')
387+ ,('\7887','o')
388+ ,('\7908','U')
389+ ,('\7909','u')
390+ ,('\7910','U')
391+ ,('\7911','u')
392+ ,('\7922','Y')
393+ ,('\7923','y')
394+ ,('\7924','Y')
395+ ,('\7925','y')
396+ ,('\7926','Y')
397+ ,('\7927','y')
398+ ,('\7928','Y')
399+ ,('\7929','y')
400+ ,('\8175','`')
401+ ,('\8490','K')
402+ ,('\8800','=')
403+ ,('\8814','<')
404+ ,('\8815','>')
405+ ]
src/Text/Pandoc/BCP47.hsnew file+112
@@ -0,0 +1,112 @@
1+{-# LANGUAGE OverloadedStrings #-}
2+{- |
3+ Module : Text.Pandoc.BCP47
4+ Copyright : Copyright (C) 2017-2021 John MacFarlane
5+ License : GNU GPL, version 2 or above
6+
7+ Maintainer : John MacFarlane <jgm@berkeley.edu>
8+ Stability : alpha
9+ Portability : portable
10+
11+Functions for parsing and rendering BCP47 language identifiers.
12+-}
13+module Text.Pandoc.BCP47 (
14+ getLang
15+ , parseBCP47
16+ , Lang(..)
17+ , renderLang
18+ )
19+where
20+import Control.Monad (guard)
21+import Data.Char (isAlphaNum, isAscii, isLetter, isLower, isUpper)
22+import Text.Pandoc.Definition
23+import Text.Pandoc.Options
24+import Text.DocTemplates (FromContext(..))
25+import qualified Data.Text as T
26+import qualified Text.Parsec as P
27+
28+-- | Represents BCP 47 language/country code.
29+data Lang = Lang{ langLanguage :: T.Text
30+ , langScript :: T.Text
31+ , langRegion :: T.Text
32+ , langVariants :: [T.Text] }
33+ deriving (Eq, Ord, Show)
34+
35+-- | Render a Lang as BCP 47.
36+renderLang :: Lang -> T.Text
37+renderLang lang = T.intercalate "-" (langLanguage lang : filter (not . T.null)
38+ ([langScript lang, langRegion lang] ++ langVariants lang))
39+
40+-- | Get the contents of the `lang` metadata field or variable.
41+getLang :: WriterOptions -> Meta -> Maybe T.Text
42+getLang opts meta =
43+ case lookupContext "lang" (writerVariables opts) of
44+ Just s -> Just s
45+ _ ->
46+ case lookupMeta "lang" meta of
47+ Just (MetaBlocks [Para [Str s]]) -> Just s
48+ Just (MetaBlocks [Plain [Str s]]) -> Just s
49+ Just (MetaInlines [Str s]) -> Just s
50+ Just (MetaString s) -> Just s
51+ _ -> Nothing
52+
53+-- | Parse a BCP 47 string as a Lang. Currently we parse
54+-- extensions and private-use fields as "variants," even
55+-- though officially they aren't.
56+parseBCP47 :: T.Text -> Either T.Text Lang
57+parseBCP47 lang =
58+ case P.parse bcp47 "lang" lang of
59+ Right r -> Right r
60+ Left e -> Left $ T.pack $ show e
61+ where bcp47 = do
62+ language <- pLanguage
63+ script <- P.option "" pScript
64+ region <- P.option "" pRegion
65+ variants <- P.many (pVariant P.<|> pExtension P.<|> pPrivateUse)
66+ P.eof
67+ return Lang{ langLanguage = language
68+ , langScript = script
69+ , langRegion = region
70+ , langVariants = variants }
71+ asciiLetter = P.satisfy (\c -> isAscii c && isLetter c)
72+ pLanguage = do
73+ cs <- P.many1 asciiLetter
74+ let lcs = length cs
75+ guard $ lcs == 2 || lcs == 3
76+ return $ T.toLower $ T.pack cs
77+ pScript = P.try $ do
78+ P.char '-'
79+ x <- P.satisfy (\c -> isAscii c && isLetter c && isUpper c)
80+ xs <- P.count 3
81+ (P.satisfy (\c -> isAscii c && isLetter c && isLower c))
82+ return $ T.toLower $ T.pack (x:xs)
83+ pRegion = P.try $ do
84+ P.char '-'
85+ cs <- P.many1 asciiLetter
86+ let lcs = length cs
87+ guard $ lcs == 2 || lcs == 3
88+ return $ T.toUpper $ T.pack cs
89+ pVariant = P.try $ do
90+ P.char '-'
91+ ds <- P.option "" (P.count 1 P.digit)
92+ cs <- P.many1 asciiLetter
93+ let var = ds ++ cs
94+ lv = length var
95+ guard $ if null ds
96+ then lv >= 5 && lv <= 8
97+ else lv == 4
98+ return $ T.toLower $ T.pack var
99+ pExtension = P.try $ do
100+ P.char '-'
101+ cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
102+ let lcs = length cs
103+ guard $ lcs >= 2 && lcs <= 8
104+ return $ T.toLower $ T.pack cs
105+ pPrivateUse = P.try $ do
106+ P.char '-'
107+ P.char 'x'
108+ P.char '-'
109+ cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
110+ guard $ not (null cs) && length cs <= 8
111+ let var = "x-" ++ cs
112+ return $ T.toLower $ T.pack var
src/Text/Pandoc/Citeproc.hs+56−75
@@ -18,6 +18,7 @@ import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
1818import Text.Pandoc.Citeproc.BibTeX (readBibtexString, Variant(..))
1919import Text.Pandoc.Citeproc.MetaValue (metaValueToReference, metaValueToText)
2020import Text.Pandoc.Readers.Markdown (yamlToRefs)
21+import qualified Text.Pandoc.BCP47 as BCP47
2122import Text.Pandoc.Builder (Inlines, Many(..), deleteMeta, setMeta)
2223import qualified Text.Pandoc.Builder as B
2324import Text.Pandoc.Definition as Pandoc
@@ -27,7 +28,7 @@ import Text.Pandoc.Error (PandocError(..))
2728import Text.Pandoc.Extensions (pandocExtensions)
2829import Text.Pandoc.Logging (LogMessage(..))
2930import Text.Pandoc.Options (ReaderOptions(..))
30−import Text.Pandoc.Shared (stringify, ordNub, tshow)
31+import Text.Pandoc.Shared (stringify, ordNub, blocksToInlines, tshow)
3132import qualified Text.Pandoc.UTF8 as UTF8
3233import Text.Pandoc.Walk (query, walk, walkM)
3334import Control.Applicative ((<|>))
@@ -48,6 +49,7 @@ import Data.Text (Text)
4849import qualified Data.Text as T
4950import System.FilePath (takeExtension)
5051import Safe (lastMay, initSafe)
52+-- import Debug.Trace as Trace (trace, traceShowId)
5153
5254
5355processCitations :: PandocMonad m => Pandoc -> m Pandoc
@@ -74,7 +76,8 @@ processCitations (Pandoc meta bs) = do
7476
7577 let linkCites = maybe False truish $ lookupMeta "link-citations" meta
7678 let opts = defaultCiteprocOptions{ linkCitations = linkCites }
77− let result = Citeproc.citeproc opts style mblang refs citations
79+ let result = Citeproc.citeproc opts style (localeLanguage locale)
80+ refs citations
7881 mapM_ (report . CiteprocWarning) (resultWarnings result)
7982 let sopts = styleOptions style
8083 let classes = "references" : -- TODO remove this or keep for compatibility?
@@ -88,21 +91,24 @@ processCitations (Pandoc meta bs) = do
8891 _ -> id) $ []
8992 let bibs = mconcat $ map (\(ident, out) ->
9093 B.divWith ("ref-" <> ident,["csl-entry"],[]) . B.para .
91− walk (convertQuotes locale) .
92− insertSpace $ out)
94+ walk (convertQuotes locale) . insertSpace $ out)
9395 (resultBibliography result)
94− let moveNotes = styleIsNoteStyle sopts &&
95− maybe True truish (lookupMeta "notes-after-punctuation" meta)
96− let cits = map (walk (convertQuotes locale)) $
96+ let moveNotes = maybe True truish $
97+ lookupMeta "notes-after-punctuation" meta
98+ let cits = map (walk fixLinks . walk (convertQuotes locale)) $
9799 resultCitations result
98100
101+ let fixQuotes = case localePunctuationInQuote locale of
102+ Just True ->
103+ B.toList . movePunctuationInsideQuotes . B.fromList
104+ _ -> id
105+
99106 let metanocites = lookupMeta "nocite" meta
100107 let Pandoc meta'' bs' =
101108 maybe id (setMeta "nocite") metanocites .
102− walk (mvPunct moveNotes locale) .
103− (if styleIsNoteStyle sopts
104− then walk addNote . walk deNote
105− else id) .
109+ walk (map capitalizeNoteCitation .
110+ fixQuotes . mvPunct moveNotes locale) .
111+ walk deNote .
106112 evalState (walkM insertResolvedCitations $ Pandoc meta' bs)
107113 $ cits
108114 return $ Pandoc meta''
@@ -191,9 +197,7 @@ getReferences mblocale (Pandoc meta bs) = do
191197 then const True
192198 else (`Set.member` citeIds)
193199 let inlineRefs = case lookupMeta "references" meta of
194− Just (MetaList rs) ->
195− filter (idpred . unItemId . referenceId)
196− $ mapMaybe metaValueToReference rs
200+ Just (MetaList rs) -> mapMaybe metaValueToReference rs
197201 _ -> []
198202 externalRefs <- case lookupMeta "bibliography" meta of
199203 Just (MetaList xs) ->
@@ -290,7 +294,7 @@ insertResolvedCitations (Cite cs ils) = do
290294 [] -> return (Cite cs ils)
291295 (x:xs) -> do
292296 put xs
293− return $ Cite cs (walk fixLinks $ B.toList x)
297+ return $ Cite cs (B.toList x)
294298insertResolvedCitations x = return x
295299
296300getCitations :: Locale
@@ -372,6 +376,7 @@ formatFromExtension fp = case dropWhile (== '.') $ takeExtension fp of
372376
373377
374378isNote :: Inline -> Bool
379+isNote (Note _) = True
375380isNote (Cite _ [Note _]) = True
376381 -- the following allows citation styles that are "in-text" but use superscript
377382 -- references to be treated as if they are "notes" for the purposes of moving
@@ -384,12 +389,6 @@ isSpacy Space = True
384389isSpacy SoftBreak = True
385390isSpacy _ = False
386391
387−movePunctInsideQuotes :: Locale -> [Inline] -> [Inline]
388−movePunctInsideQuotes locale
389− | localePunctuationInQuote locale == Just True
390− = B.toList . movePunctuationInsideQuotes . B.fromList
391− | otherwise
392− = id
393392
394393mvPunct :: Bool -> Locale -> [Inline] -> [Inline]
395394mvPunct moveNotes locale (x : xs)
@@ -402,8 +401,7 @@ mvPunct moveNotes locale (q : s : x : ys)
402401 in if moveNotes
403402 then if T.null spunct
404403 then q : x : mvPunct moveNotes locale ys
405− else movePunctInsideQuotes locale
406− [q , Str spunct , x] ++ mvPunct moveNotes locale
404+ else q : Str spunct : x : mvPunct moveNotes locale
407405 (B.toList
408406 (dropTextWhile isPunctuation (B.fromList ys)))
409407 else q : x : mvPunct moveNotes locale ys
@@ -415,10 +413,9 @@ mvPunct moveNotes locale (Cite cs ils : ys)
415413 , moveNotes
416414 = let s = stringify ys
417415 spunct = T.takeWhile isPunctuation s
418− in Cite cs (movePunctInsideQuotes locale $
419− init ils
420− ++ [Str spunct | not (endWithPunct False (init ils))]
421− ++ [last ils]) :
416+ in Cite cs (init ils
417+ ++ [Str spunct | not (endWithPunct False (init ils))]
418+ ++ [last ils]) :
422419 mvPunct moveNotes locale
423420 (B.toList (dropTextWhile isPunctuation (B.fromList ys)))
424421mvPunct moveNotes locale (s : x : ys) | isSpacy s, isNote x =
@@ -434,7 +431,7 @@ mvPunct _ _ [] = []
434431-- move https://doi.org etc. prefix inside link text (#6723):
435432fixLinks :: [Inline] -> [Inline]
436433fixLinks (Str t : Link attr [Str u1] (u2,tit) : xs)
437− | u2 == t <> u1
434+ | t <> u1 == u2
438435 = Link attr [Str (t <> u1)] (u2,tit) : fixLinks xs
439436fixLinks (x:xs) = x : fixLinks xs
440437fixLinks [] = []
@@ -504,8 +501,7 @@ insertRefs refkvs refclasses meta refs bs =
504501 put True
505502 -- refHeader isn't used if you have an explicit references div
506503 let cs' = ordNub $ cs ++ refclasses
507− let kvs' = ordNub $ kvs ++ refkvs
508− return $ Div ("refs",cs',kvs') (xs ++ refs)
504+ return $ Div ("refs",cs',kvs) (xs ++ refs)
509505 go x = return x
510506
511507refTitle :: Meta -> Maybe [Inline]
@@ -564,62 +560,42 @@ extractText (FancyVal x) = toText x
564560extractText (NumVal n) = T.pack (show n)
565561extractText _ = mempty
566562
567−-- Here we take the Spans with class csl-note that are left
568−-- after deNote has removed nested ones, and convert them
569−-- into real notes.
570−addNote :: Inline -> Inline
571−addNote (Span ("",["csl-note"],[]) ils) =
572− Note [Para $
573− B.toList . addTextCase Nothing CapitalizeFirst . B.fromList $ ils]
574−addNote x = x
575−
576−-- Here we handle citation notes that occur inside footnotes
577−-- or other citation notes, in a note style. We don't want
578−-- notes inside notes, so we convert these to parenthesized
579−-- or comma-separated citations.
580−deNote :: Inline -> Inline
581−deNote (Note bs) =
582− case bs of
583− [Para (cit@(Cite (c:_) _) : ils)]
584− | citationMode c /= AuthorInText ->
585− -- if citation is first in note, no need to parenthesize.
586− Note [Para (walk removeNotes $ cit : walk addParens ils)]
587− _ -> Note (walk removeNotes . walk addParens $ bs)
588−
563+capitalizeNoteCitation :: Inline -> Inline
564+capitalizeNoteCitation (Cite cs [Note [Para ils]]) =
565+ Cite cs
566+ [Note [Para $ B.toList $ addTextCase Nothing CapitalizeFirst
567+ $ B.fromList ils]]
568+capitalizeNoteCitation x = x
569+
570+deNote :: [Inline] -> [Inline]
571+deNote [] = []
572+deNote (Note bs:rest) =
573+ Note (walk go bs) : deNote rest
589574 where
590− addParens [] = []
591− addParens (Cite (c:cs) ils : zs)
575+ go [] = []
576+ go (Cite (c:cs) ils : zs)
592577 | citationMode c == AuthorInText
593− = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) :
594− addParens zs
578+ = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) : go zs
595579 | otherwise
596− = Cite (c:cs) (concatMap noteInParens ils) : addParens zs
597− addParens (x:xs) = x : addParens xs
598−
599− removeNotes (Span ("",["csl-note"],[]) ils) = Span ("",[],[]) ils
600− removeNotes x = x
601−
580+ = Cite (c:cs) (concatMap noteInParens ils) : go zs
581+ go (x:xs) = x : go xs
602582 needsPeriod [] = True
603583 needsPeriod (Str t:_) = case T.uncons t of
604584 Nothing -> False
605585 Just (c,_) -> isUpper c
606586 needsPeriod (Space:zs) = needsPeriod zs
607587 needsPeriod _ = False
608−
609− noteInParens (Span ("",["csl-note"],[]) ils)
588+ noteInParens (Note bs')
610589 = Space : Str "(" :
611− removeFinalPeriod ils ++ [Str ")"]
590+ removeFinalPeriod (blocksToInlines bs') ++ [Str ")"]
612591 noteInParens x = [x]
613−
614− noteAfterComma needsPer (Span ("",["csl-note"],[]) ils)
615− | not (null ils)
592+ noteAfterComma needsPer (Note bs')
616593 = Str "," : Space :
617− if needsPer
618− then ils
619− else removeFinalPeriod ils
594+ (if needsPer
595+ then id
596+ else removeFinalPeriod) (blocksToInlines bs')
620597 noteAfterComma _ x = [x]
621−
622−deNote x = x
598+deNote (x:xs) = x : deNote xs
623599
624600-- Note: we can't use dropTextWhileEnd indiscriminately,
625601-- because this would remove the final period on abbreviations like Ibid.
@@ -654,8 +630,13 @@ removeFinalPeriod ils =
654630
655631bcp47LangToIETF :: PandocMonad m => Text -> m (Maybe Lang)
656632bcp47LangToIETF bcplang =
657− case parseLang bcplang of
633+ case BCP47.parseBCP47 bcplang of
658634 Left _ -> do
659635 report $ InvalidLang bcplang
660636 return Nothing
661− Right lang -> return $ Just lang
637+ Right lang ->
638+ return $ Just
639+ $ Lang (BCP47.langLanguage lang)
640+ (if T.null (BCP47.langRegion lang)
641+ then Nothing
642+ else Just (BCP47.langRegion lang))
src/Text/Pandoc/Citeproc/BibTeX.hs+15−25
@@ -59,11 +59,10 @@ data Variant = Bibtex | Biblatex
5959 deriving (Show, Eq, Ord)
6060
6161-- | Parse BibTeX or BibLaTeX into a list of 'Reference's.
62−readBibtexString :: ToSources a
63− => Variant -- ^ bibtex or biblatex
62+readBibtexString :: Variant -- ^ bibtex or biblatex
6463 -> Locale -- ^ Locale
6564 -> (Text -> Bool) -- ^ Filter on citation ids
66− -> a -- ^ bibtex/biblatex text
65+ -> Text -- ^ bibtex/biblatex text
6766 -> Either ParseError [Reference Inlines]
6867readBibtexString variant locale idpred contents = do
6968 case runParser (((resolveCrossRefs variant <$> bibEntries) <* eof) >>=
@@ -71,11 +70,9 @@ readBibtexString variant locale idpred contents = do
7170 filter (\item -> idpred (identifier item) &&
7271 entryType item /= "xdata"))
7372 (fromMaybe defaultLang $ localeLanguage locale, Map.empty)
74− (initialSourceName sources) sources of
73+ "" contents of
7574 Left err -> Left err
7675 Right xs -> return xs
77− where
78− sources = toSources contents
7976
8077-- | Write BibTeX or BibLaTeX given given a 'Reference'.
8178writeBibtexString :: WriterOptions -- ^ options (for writing LaTex)
@@ -158,7 +155,6 @@ writeBibtexString opts variant mblang ref =
158155 , "langid"
159156 , "abstract"
160157 , "keywords"
161− , "annote"
162158 ]
163159 Bibtex ->
164160 [ "author"
@@ -178,7 +174,6 @@ writeBibtexString opts variant mblang ref =
178174 , "address"
179175 , "type"
180176 , "note"
181− , "annote"
182177 ]
183178
184179 valToInlines (TextVal t) = B.text t
@@ -210,13 +205,10 @@ writeBibtexString opts variant mblang ref =
210205 [ (", " <>) <$> nameGiven name,
211206 nameDroppingParticle name ]
212207
213− mblang' = case getVariableAsText "language" of
214− Just l -> either (const Nothing) Just $ parseLang l
215− Nothing -> mblang
208+ mblang' = (parseLang <$> getVariableAsText "language") <|> mblang
216209
217210 titlecase = case mblang' of
218− Just lang | langLanguage lang == "en"
219− -> titlecase'
211+ Just (Lang "en" _) -> titlecase'
220212 Nothing -> titlecase'
221213 _ ->
222214 case variant of
@@ -339,12 +331,12 @@ writeBibtexString opts variant mblang ref =
339331 renderFields = mconcat . intersperse ("," <> cr) . mapMaybe renderField
340332
341333defaultLang :: Lang
342−defaultLang = Lang "en" Nothing (Just "US") [] [] []
334+defaultLang = Lang "en" (Just "US")
343335
344336-- a map of bibtex "string" macros
345337type StringMap = Map.Map Text Text
346338
347−type BibParser = Parser Sources (Lang, StringMap)
339+type BibParser = Parser Text (Lang, StringMap)
348340
349341data Item = Item{ identifier :: Text
350342 , sourcePos :: SourcePos
@@ -359,7 +351,9 @@ itemToReference locale variant item = do
359351 bib item $ do
360352 let lang = fromMaybe defaultLang $ localeLanguage locale
361353 modify $ \st -> st{ localeLang = lang,
362− untitlecase = langLanguage lang == "en" }
354+ untitlecase = case lang of
355+ (Lang "en" _) -> True
356+ _ -> False }
363357
364358 id' <- asks identifier
365359 otherIds <- (Just <$> getRawField "ids")
@@ -717,7 +711,7 @@ itemToReference locale variant item = do
717711
718712
719713bib :: Item -> Bib a -> BibParser a
720−bib entry m = fst <$> evalRWST m entry (BibState True defaultLang)
714+bib entry m = fst <$> evalRWST m entry (BibState True (Lang "en" (Just "US")))
721715
722716resolveCrossRefs :: Variant -> [Item] -> [Item]
723717resolveCrossRefs variant entries =
@@ -809,7 +803,7 @@ bibEntries = do
809803 (bibComment <|> bibPreamble <|> bibString))
810804
811805bibSkip :: BibParser ()
812−bibSkip = skipMany1 (satisfy (/='@'))
806+bibSkip = () <$ take1WhileP (/='@')
813807
814808bibComment :: BibParser ()
815809bibComment = do
@@ -834,9 +828,6 @@ bibString = do
834828 updateState (\(l,m) -> (l, Map.insert k v m))
835829 return ()
836830
837−take1WhileP :: Monad m => (Char -> Bool) -> ParserT Sources u m Text
838−take1WhileP f = T.pack <$> many1 (satisfy f)
839−
840831inBraces :: BibParser Text
841832inBraces = do
842833 char '{'
@@ -868,7 +859,7 @@ fieldName = resolveAlias . T.toLower
868859
869860isBibtexKeyChar :: Char -> Bool
870861isBibtexKeyChar c =
871− isAlphaNum c || c `elem` (".:;?!`'()$/*@_+=-[]*&" :: [Char])
862+ isAlphaNum c || c `elem` (".:;?!`'()/*@_+=-[]*&" :: [Char])
872863
873864bibItem :: BibParser Item
874865bibItem = do
@@ -1465,9 +1456,8 @@ resolveKey lang ils = Walk.walk go ils
14651456 go x = x
14661457
14671458resolveKey' :: Lang -> Text -> Text
1468−resolveKey' lang k =
1469− case Map.lookup (langLanguage lang) biblatexStringMap >>=
1470− Map.lookup (T.toLower k) of
1459+resolveKey' lang@(Lang l _) k =
1460+ case Map.lookup l biblatexStringMap >>= Map.lookup (T.toLower k) of
14711461 Nothing -> k
14721462 Just (x, _) -> either (const k) stringify $ parseLaTeX lang x
14731463
src/Text/Pandoc/Citeproc/Data.hs+6−7
@@ -10,7 +10,7 @@ import qualified Data.Text.Encoding as TE
1010import qualified Data.Text as T
1111import Data.Text (Text)
1212import Text.Pandoc.Citeproc.Util (toIETF)
13−import Text.Collate.Lang (Lang(..), parseLang)
13+import Citeproc (Lang(..), parseLang)
1414
1515biblatexLocalizations :: [(FilePath, ByteString)]
1616biblatexLocalizations = $(embedDir "citeproc/biblatex-localization")
@@ -21,12 +21,11 @@ biblatexStringMap :: M.Map Text (M.Map Text (Text, Text))
2121biblatexStringMap = foldr go mempty biblatexLocalizations
2222 where
2323 go (fp, bs) =
24− let ls = T.lines $ TE.decodeUtf8 bs
25− in case parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp) of
26− Right lang | length ls > 4
27− -> M.insert (langLanguage lang)
28− (toStringMap $ map (T.splitOn "|") ls)
29− _ -> id
24+ let Lang lang _ = parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp)
25+ ls = T.lines $ TE.decodeUtf8 bs
26+ in if length ls > 4
27+ then M.insert lang (toStringMap $ map (T.splitOn "|") ls)
28+ else id
3029 toStringMap = foldr go' mempty
3130 go' [term, x, y] = M.insert term (x, y)
3231 go' _ = id
src/Text/Pandoc/Citeproc/Locator.hs+4−20
@@ -7,7 +7,6 @@ where
77import Citeproc.Types
88import Data.Text (Text)
99import qualified Data.Text as T
10−import Data.List (foldl')
1110import Text.Parsec
1211import Text.Pandoc.Definition
1312import Text.Pandoc.Parsing (romanNumeral)
@@ -20,7 +19,7 @@ parseLocator :: Locale -> [Inline] -> (Maybe (Text, Text), [Inline])
2019parseLocator locale inp =
2120 case parse (pLocatorWords (toLocatorMap locale)) "suffix" $ splitInp inp of
2221 Right r -> r
23− Left _ -> (Nothing, maybeAddComma inp)
22+ Left _ -> (Nothing, inp)
2423
2524splitInp :: [Inline] -> [Inline]
2625splitInp = splitStrWhen (\c -> isSpace c || (isPunctuation c && c /= ':'))
@@ -42,17 +41,9 @@ pLocatorWords locMap = do
4241 -- i.e. the first one will be " 9"
4342 return $
4443 if T.null la && T.null lo
45− then (Nothing, maybeAddComma s)
44+ then (Nothing, s)
4645 else (Just (la, T.strip lo), s)
4746
48−maybeAddComma :: [Inline] -> [Inline]
49−maybeAddComma [] = []
50−maybeAddComma ils@(Space : _) = ils
51−maybeAddComma ils@(Str t : _)
52− | Just (c, _) <- T.uncons t
53− , isPunctuation c = ils
54−maybeAddComma ils = Str "," : Space : ils
55−
5647pLocatorDelimited :: LocatorMap -> LocatorParser (Text, Text)
5748pLocatorDelimited locMap = try $ do
5849 _ <- pMatchChar "{" (== '{')
@@ -105,7 +96,7 @@ pLocatorLabel' locMap lim = go ""
10596 t <- anyToken
10697 ts <- manyTill anyToken (try $ lookAhead lim)
10798 let s = acc <> stringify (t:ts)
108− case M.lookup (T.toCaseFold $ T.strip s) locMap of
99+ case M.lookup (T.strip s) locMap of
109100 -- try to find a longer one, or return this one
110101 Just l -> go s <|> return (l, False)
111102 Nothing -> go s
@@ -148,7 +139,7 @@ pBalancedBraces braces p = try $ do
148139 where
149140 except = notFollowedBy pBraces >> p
150141 -- outer and inner
151− surround = foldl' (\a (open, close) -> sur open close except <|> a)
142+ surround = foldl (\a (open, close) -> sur open close except <|> a)
152143 except
153144 braces
154145
@@ -189,7 +180,6 @@ pPageUnit = roman <|> plainUnit
189180 plainUnit = do
190181 ts <- many1 (notFollowedBy pSpace >>
191182 notFollowedBy pLocatorPunct >>
192− notFollowedBy pMath >>
193183 anyToken)
194184 let s = stringify ts
195185 -- otherwise look for actual digits or -s
@@ -220,12 +210,6 @@ pMatchChar msg f = satisfyTok f' <?> msg
220210pSpace :: LocatorParser Inline
221211pSpace = satisfyTok (\t -> isSpacey t || t == Str "\160") <?> "space"
222212
223−pMath :: LocatorParser Inline
224−pMath = satisfyTok isMath
225− where
226− isMath (Math{}) = True
227− isMath _ = False
228−
229213satisfyTok :: (Inline -> Bool) -> LocatorParser Inline
230214satisfyTok f = tokenPrim show (\sp _ _ -> sp) (\tok -> if f tok
231215 then Just tok
src/Text/Pandoc/Class/CommonState.hs+1−1
@@ -19,7 +19,7 @@ where
1919
2020import Data.Default (Default (def))
2121import Data.Text (Text)
22−import Text.Collate.Lang (Lang)
22+import Text.Pandoc.BCP47 (Lang)
2323import Text.Pandoc.MediaBag (MediaBag)
2424import Text.Pandoc.Logging (LogMessage, Verbosity (WARNING))
2525import Text.Pandoc.Translations (Translations)
src/Text/Pandoc/Class/IO.hs+22−21
@@ -62,7 +62,7 @@ import Text.Pandoc.Definition (Pandoc, Inline (Image))
6262import Text.Pandoc.Error (PandocError (..))
6363import Text.Pandoc.Logging (LogMessage (..), messageVerbosity, showLogMessage)
6464import Text.Pandoc.MIME (MimeType)
65−import Text.Pandoc.MediaBag (MediaBag, MediaItem(..), lookupMedia, mediaItems)
65+import Text.Pandoc.MediaBag (MediaBag, lookupMedia, mediaDirectory)
6666import Text.Pandoc.Walk (walk)
6767import qualified Control.Exception as E
6868import qualified Data.ByteString as B
@@ -200,32 +200,33 @@ alertIndent (l:ls) = do
200200extractMedia :: (PandocMonad m, MonadIO m) => FilePath -> Pandoc -> m Pandoc
201201extractMedia dir d = do
202202 media <- getMediaBag
203− let items = mediaItems media
204− if null items
205− then return d
206− else do
207− mapM_ (writeMedia dir) items
208− return $ walk (adjustImagePath dir media) d
203+ case [fp | (fp, _, _) <- mediaDirectory media] of
204+ [] -> return d
205+ fps -> do
206+ mapM_ (writeMedia dir media) fps
207+ return $ walk (adjustImagePath dir fps) d
209208
210209-- | Write the contents of a media bag to a path.
211210writeMedia :: (PandocMonad m, MonadIO m)
212− => FilePath
213− -> (FilePath, MimeType, BL.ByteString)
211+ => FilePath -> MediaBag -> FilePath
214212 -> m ()
215−writeMedia dir (fp, _mt, bs) = do
216− -- we normalize to get proper path separators for the platform
217− let fullpath = normalise $ dir </> fp
218− liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
219− logIOError $ BL.writeFile fullpath bs
213+writeMedia dir mediabag subpath = do
214+ -- we join and split to convert a/b/c to a\b\c on Windows;
215+ -- in zip containers all paths use /
216+ let fullpath = dir </> unEscapeString (normalise subpath)
217+ let mbcontents = lookupMedia subpath mediabag
218+ case mbcontents of
219+ Nothing -> throwError $ PandocResourceNotFound $ pack subpath
220+ Just (_, bs) -> do
221+ report $ Extracting $ pack fullpath
222+ liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
223+ logIOError $ BL.writeFile fullpath bs
220224
221225-- | If the given Inline element is an image with a @src@ path equal to
222226-- one in the list of @paths@, then prepends @dir@ to the image source;
223227-- returns the element unchanged otherwise.
224−adjustImagePath :: FilePath -> MediaBag -> Inline -> Inline
225−adjustImagePath dir mediabag (Image attr lab (src, tit)) =
226− case lookupMedia (T.unpack src) mediabag of
227− Nothing -> Image attr lab (src, tit)
228− Just item ->
229− let fullpath = dir </> mediaPath item
230− in Image attr lab (T.pack fullpath, tit)
228+adjustImagePath :: FilePath -> [FilePath] -> Inline -> Inline
229+adjustImagePath dir paths (Image attr lab (src, tit))
230+ | unpack src `elem` paths
231+ = Image attr lab (pack (normalise $ dir </> unpack src), tit)
231232adjustImagePath _ _ x = x
src/Text/Pandoc/Class/PandocMonad.hs+34−24
@@ -1,5 +1,4 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TupleSections #-}
32{-# LANGUAGE FlexibleContexts #-}
43{-# LANGUAGE FlexibleInstances #-}
54{-# LANGUAGE MultiParamTypeClasses #-}
@@ -38,6 +37,7 @@ module Text.Pandoc.Class.PandocMonad
3837 , setUserDataDir
3938 , getUserDataDir
4039 , fetchItem
40+ , fetchMediaResource
4141 , getInputFiles
4242 , setInputFiles
4343 , getOutputFile
@@ -57,7 +57,8 @@ module Text.Pandoc.Class.PandocMonad
5757import Codec.Archive.Zip
5858import Control.Monad.Except (MonadError (catchError, throwError),
5959 MonadTrans, lift, when)
60−import Data.List (foldl')
60+import Data.Digest.Pure.SHA (sha1, showDigest)
61+import Data.Maybe (fromMaybe)
6162import Data.Time (UTCTime)
6263import Data.Time.Clock.POSIX (POSIXTime, utcTimeToPOSIXSeconds,
6364 posixSecondsToUTCTime)
@@ -65,16 +66,16 @@ import Data.Time.LocalTime (TimeZone, ZonedTime, utcToZonedTime)
6566import Network.URI ( escapeURIString, nonStrictRelativeTo,
6667 unEscapeString, parseURIReference, isAllowedInURI,
6768 parseURI, URI(..) )
68−import System.FilePath ((</>), takeExtension, dropExtension,
69− isRelative, splitDirectories, makeRelative)
69+import System.FilePath ((</>), (<.>), takeExtension, dropExtension,
70+ isRelative, splitDirectories)
7071import System.Random (StdGen)
71−import Text.Collate.Lang (Lang(..), parseLang, renderLang)
72+import Text.Pandoc.BCP47 (Lang(..), parseBCP47, renderLang)
7273import Text.Pandoc.Class.CommonState (CommonState (..))
7374import Text.Pandoc.Definition
7475import Text.Pandoc.Error
7576import Text.Pandoc.Logging
76−import Text.Pandoc.MIME (MimeType, getMimeType)
77−import Text.Pandoc.MediaBag (MediaBag, lookupMedia, MediaItem(..))
77+import Text.Pandoc.MIME (MimeType, getMimeType, extensionFromMimeType)
78+import Text.Pandoc.MediaBag (MediaBag, lookupMedia)
7879import Text.Pandoc.Shared (uriPathToPath, safeRead)
7980import Text.Pandoc.Translations (Term(..), Translations, lookupTerm,
8081 readTranslations)
@@ -283,7 +284,7 @@ readFileFromDirs (d:ds) f = catchError
283284toLang :: PandocMonad m => Maybe T.Text -> m (Maybe Lang)
284285toLang Nothing = return Nothing
285286toLang (Just s) =
286− case parseLang s of
287+ case parseBCP47 s of
287288 Left _ -> do
288289 report $ InvalidLang s
289290 return Nothing
@@ -374,8 +375,7 @@ fetchItem :: PandocMonad m
374375fetchItem s = do
375376 mediabag <- getMediaBag
376377 case lookupMedia (T.unpack s) mediabag of
377− Just item -> return (BL.toStrict (mediaContents item),
378− Just (mediaMimeType item))
378+ Just (mime, bs) -> return (BL.toStrict bs, Just mime)
379379 Nothing -> downloadOrRead s
380380
381381-- | Returns the content and, if available, the MIME type of a resource.
@@ -410,10 +410,9 @@ downloadOrRead s = do
410410 _ -> readLocalFile fp -- get from local file system
411411 where readLocalFile f = do
412412 resourcePath <- getResourcePath
413− (fp', cont) <- if isRelative f
414− then withPaths resourcePath readFileStrict f
415− else (f,) <$> readFileStrict f
416− report $ LoadedResource f (makeRelative "." fp')
413+ cont <- if isRelative f
414+ then withPaths resourcePath readFileStrict f
415+ else readFileStrict f
417416 return (cont, mime)
418417 httpcolon = URI{ uriScheme = "http:",
419418 uriAuthority = Nothing,
@@ -613,7 +612,7 @@ checkExistence fn = do
613612-- | Canonicalizes a file path by removing redundant @.@ and @..@.
614613makeCanonical :: FilePath -> FilePath
615614makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
616− where transformPathParts = reverse . foldl' go []
615+ where transformPathParts = reverse . foldl go []
617616 go as "." = as
618617 go (_:as) ".." = as
619618 go as x = x : as
@@ -623,13 +622,25 @@ makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
623622-- that filepath. Returns the result of the first successful execution
624623-- of the action, or throws a @PandocResourceNotFound@ exception if the
625624-- action errors for all filepaths.
626−withPaths :: PandocMonad m
627− => [FilePath] -> (FilePath -> m a) -> FilePath -> m (FilePath, a)
625+withPaths :: PandocMonad m => [FilePath] -> (FilePath -> m a) -> FilePath -> m a
628626withPaths [] _ fp = throwError $ PandocResourceNotFound $ T.pack fp
629627withPaths (p:ps) action fp =
630− catchError ((p </> fp,) <$> action (p </> fp))
628+ catchError (action (p </> fp))
631629 (\_ -> withPaths ps action fp)
632630
631+-- | Fetch local or remote resource (like an image) and provide data suitable
632+-- for adding it to the MediaBag.
633+fetchMediaResource :: PandocMonad m
634+ => T.Text -> m (FilePath, Maybe MimeType, BL.ByteString)
635+fetchMediaResource src = do
636+ (bs, mt) <- downloadOrRead src
637+ let ext = fromMaybe (T.pack $ takeExtension $ T.unpack src)
638+ (mt >>= extensionFromMimeType)
639+ let bs' = BL.fromChunks [bs]
640+ let basename = showDigest $ sha1 bs'
641+ let fname = basename <.> T.unpack ext
642+ return (fname, mt, bs')
643+
633644-- | Traverse tree, filling media bag for any images that
634645-- aren't already in the media bag.
635646fillMediaBag :: PandocMonad m => Pandoc -> m Pandoc
@@ -637,13 +648,12 @@ fillMediaBag d = walkM handleImage d
637648 where handleImage :: PandocMonad m => Inline -> m Inline
638649 handleImage (Image attr lab (src, tit)) = catchError
639650 (do mediabag <- getMediaBag
640− let fp = T.unpack src
641− case lookupMedia fp mediabag of
642− Just _ -> return ()
651+ case lookupMedia (T.unpack src) mediabag of
652+ Just (_, _) -> return $ Image attr lab (src, tit)
643653 Nothing -> do
644− (bs, mt) <- fetchItem src
645− insertMedia fp mt (BL.fromStrict bs)
646− return $ Image attr lab (src, tit))
654+ (fname, mt, bs) <- fetchMediaResource src
655+ insertMedia fname mt bs
656+ return $ Image attr lab (T.pack fname, tit))
647657 (\e ->
648658 case e of
649659 PandocResourceNotFound _ -> do
src/Text/Pandoc/Error.hs+16−23
@@ -23,27 +23,26 @@ import Control.Exception (Exception, displayException)
2323import Data.Typeable (Typeable)
2424import Data.Word (Word8)
2525import Data.Text (Text)
26−import Data.List (sortOn)
2726import qualified Data.Text as T
28−import Data.Ord (Down(..))
2927import GHC.Generics (Generic)
3028import Network.HTTP.Client (HttpException)
3129import System.Exit (ExitCode (..), exitWith)
3230import System.IO (stderr)
3331import qualified Text.Pandoc.UTF8 as UTF8
34−import Text.Pandoc.Sources (Sources(..))
3532import Text.Printf (printf)
3633import Text.Parsec.Error
3734import Text.Parsec.Pos hiding (Line)
3835import Text.Pandoc.Shared (tshow)
3936import Citeproc (CiteprocError, prettyCiteprocError)
4037
38+type Input = Text
39+
4140data PandocError = PandocIOError Text IOError
4241 | PandocHttpError Text HttpException
4342 | PandocShouldNeverHappenError Text
4443 | PandocSomeError Text
4544 | PandocParseError Text
46− | PandocParsecError Sources ParseError
45+ | PandocParsecError Input ParseError
4746 | PandocMakePDFError Text
4847 | PandocOptionError Text
4948 | PandocSyntaxMapError Text
@@ -82,28 +81,22 @@ renderError e =
8281 "Please report this to pandoc's developers: " <> s
8382 PandocSomeError s -> s
8483 PandocParseError s -> s
85− PandocParsecError (Sources inputs) err' ->
84+ PandocParsecError input err' ->
8685 let errPos = errorPos err'
8786 errLine = sourceLine errPos
8887 errColumn = sourceColumn errPos
89− errFile = sourceName errPos
90− errorInFile =
91− case sortOn (Down . sourceLine . fst)
92− [ (pos,t)
93− | (pos,t) <- inputs
94− , sourceName pos == errFile
95− , sourceLine pos <= errLine
96− ] of
97− [] -> ""
98− ((pos,txt):_) ->
99− let ls = T.lines txt <> [""]
100− ln = (errLine - sourceLine pos) + 1
101− in if length ls > ln && ln >= 1
102− then T.concat ["\n", ls !! (ln - 1)
103− ,"\n", T.replicate (errColumn - 1) " "
104− ,"^"]
105− else ""
106− in "Error at " <> tshow err' <> errorInFile
88+ ls = T.lines input <> [""]
89+ errorInFile = if length ls > errLine - 1
90+ then T.concat ["\n", ls !! (errLine - 1)
91+ ,"\n", T.replicate (errColumn - 1) " "
92+ ,"^"]
93+ else ""
94+ in "\nError at " <> tshow err' <>
95+ -- if error comes from a chunk or included file,
96+ -- then we won't get the right text this way:
97+ if sourceName errPos == "source"
98+ then errorInFile
99+ else ""
107100 PandocMakePDFError s -> s
108101 PandocOptionError s -> s
109102 PandocSyntaxMapError s -> s
src/Text/Pandoc/Extensions.hs+1−9
@@ -34,7 +34,6 @@ module Text.Pandoc.Extensions ( Extension(..)
3434where
3535import Data.Bits (clearBit, setBit, testBit, (.|.))
3636import Data.Data (Data)
37−import Data.List (foldl')
3837import qualified Data.Text as T
3938import Data.Typeable (Typeable)
4039import GHC.Generics (Generic)
@@ -136,8 +135,6 @@ data Extension =
136135 | Ext_raw_html -- ^ Allow raw HTML
137136 | Ext_raw_tex -- ^ Allow raw TeX (other than math)
138137 | Ext_raw_markdown -- ^ Parse markdown in ipynb as raw markdown
139− | Ext_rebase_relative_paths -- ^ Rebase relative image and link paths,
140− -- relative to directory of containing file
141138 | Ext_shortcut_reference_links -- ^ Shortcut reference links
142139 | Ext_simple_tables -- ^ Pandoc-style simple tables
143140 | Ext_smart -- ^ "Smart" quotes, apostrophes, ellipses, dashes
@@ -356,7 +353,6 @@ getDefaultExtensions "gfm" = extensionsFromList
356353 , Ext_strikeout
357354 , Ext_task_lists
358355 , Ext_emoji
359− , Ext_yaml_metadata_block
360356 ]
361357getDefaultExtensions "commonmark" = extensionsFromList
362358 [Ext_raw_html]
@@ -382,7 +378,6 @@ getDefaultExtensions "commonmark_x" = extensionsFromList
382378 , Ext_raw_attribute
383379 , Ext_implicit_header_references
384380 , Ext_attributes
385− , Ext_yaml_metadata_block
386381 ]
387382getDefaultExtensions "org" = extensionsFromList
388383 [Ext_citations,
@@ -464,7 +459,6 @@ getAllExtensions f = universalExtensions <> getAll f
464459 , Ext_gutenberg
465460 , Ext_smart
466461 , Ext_literate_haskell
467− , Ext_rebase_relative_paths
468462 ]
469463 getAll "markdown_strict" = allMarkdownExtensions
470464 getAll "markdown_phpextra" = allMarkdownExtensions
@@ -516,8 +510,6 @@ getAllExtensions f = universalExtensions <> getAll f
516510 , Ext_implicit_header_references
517511 , Ext_attributes
518512 , Ext_sourcepos
519− , Ext_yaml_metadata_block
520− , Ext_rebase_relative_paths
521513 ]
522514 getAll "commonmark_x" = getAll "commonmark"
523515 getAll "org" = autoIdExtensions <>
@@ -601,7 +593,7 @@ parseFormatSpec :: T.Text
601593parseFormatSpec = parse formatSpec ""
602594 where formatSpec = do
603595 name <- formatName
604− (extsToEnable, extsToDisable) <- foldl' (flip ($)) ([],[]) <$>
596+ (extsToEnable, extsToDisable) <- foldl (flip ($)) ([],[]) <$>
605597 many extMod
606598 return (T.pack name, reverse extsToEnable, reverse extsToDisable)
607599 formatName = many1 $ noneOf "-+"
src/Text/Pandoc/Highlighting.hs+5−5
@@ -52,12 +52,12 @@ highlightingStyles =
5252 ("breezedark", breezeDark),
5353 ("haddock", haddock)]
5454
55−languages :: SyntaxMap -> [T.Text]
56−languages syntaxmap = [T.toLower (sName s) | s <- M.elems syntaxmap]
55+languages :: [T.Text]
56+languages = [T.toLower (sName s) | s <- M.elems defaultSyntaxMap]
5757
58−languagesByExtension :: SyntaxMap -> T.Text -> [T.Text]
59−languagesByExtension syntaxmap ext =
60− [T.toLower (sName s) | s <- syntaxesByExtension syntaxmap (T.unpack ext)]
58+languagesByExtension :: T.Text -> [T.Text]
59+languagesByExtension ext =
60+ [T.toLower (sName s) | s <- syntaxesByExtension defaultSyntaxMap (T.unpack ext)]
6161
6262highlight :: SyntaxMap
6363 -> (FormatOptions -> [SourceLine] -> a) -- ^ Formatter
src/Text/Pandoc/ImageSize.hs+1−5
@@ -57,8 +57,7 @@ import Codec.Picture (decodeImageWithMetadata)
5757-- quick and dirty functions to get image sizes
5858-- algorithms borrowed from wwwis.pl
5959
60−data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf | Tiff
61− deriving Show
60+data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf deriving Show
6261data Direction = Width | Height
6362instance Show Direction where
6463 show Width = "width"
@@ -101,8 +100,6 @@ imageType :: ByteString -> Maybe ImageType
101100imageType img = case B.take 4 img of
102101 "\x89\x50\x4e\x47" -> return Png
103102 "\x47\x49\x46\x38" -> return Gif
104− "\x49\x49\x2a\x00" -> return Tiff
105− "\x4D\x4D\x00\x2a" -> return Tiff
106103 "\xff\xd8\xff\xe0" -> return Jpeg -- JFIF
107104 "\xff\xd8\xff\xe1" -> return Jpeg -- Exif
108105 "%PDF" -> return Pdf
@@ -127,7 +124,6 @@ imageSize opts img = checkDpi <$>
127124 Just Png -> getSize img
128125 Just Gif -> getSize img
129126 Just Jpeg -> getSize img
130− Just Tiff -> getSize img
131127 Just Svg -> mbToEither "could not determine SVG size" $ svgSize opts img
132128 Just Eps -> mbToEither "could not determine EPS size" $ epsSize img
133129 Just Pdf -> mbToEither "could not determine PDF size" $ pdfSize img
src/Text/Pandoc/Logging.hs+9−26
@@ -36,7 +36,6 @@ import Data.Typeable (Typeable)
3636import GHC.Generics (Generic)
3737import Text.Pandoc.Definition
3838import Text.Parsec.Pos
39−import Text.Pandoc.Shared (tshow)
4039
4140-- | Verbosity level.
4241data Verbosity = ERROR | WARNING | INFO
@@ -85,7 +84,6 @@ data LogMessage =
8584 | CouldNotParseCSS Text
8685 | Fetching Text
8786 | Extracting Text
88− | LoadedResource FilePath FilePath
8987 | NoTitleElement Text
9088 | NoLangSpecified
9189 | InvalidLang Text
@@ -103,7 +101,6 @@ data LogMessage =
103101 | CiteprocWarning Text
104102 | ATXHeadingInLHS Int Text
105103 | EnvironmentVariableUndefined Text
106− | DuplicateAttribute Text Text
107104 deriving (Show, Eq, Data, Ord, Typeable, Generic)
108105
109106instance ToJSON LogMessage where
@@ -196,9 +193,6 @@ instance ToJSON LogMessage where
196193 ["path" .= fp]
197194 Extracting fp ->
198195 ["path" .= fp]
199− LoadedResource orig found ->
200− ["for" .= orig
201− ,"from" .= found]
202196 NoTitleElement fallback ->
203197 ["fallback" .= fallback]
204198 NoLangSpecified -> []
@@ -238,18 +232,13 @@ instance ToJSON LogMessage where
238232 ,"contents" .= contents]
239233 EnvironmentVariableUndefined var ->
240234 ["variable" .= var ]
241− DuplicateAttribute attr val ->
242− ["attribute" .= attr
243− ,"value" .= val]
244235
245236showPos :: SourcePos -> Text
246237showPos pos = Text.pack $ sn ++ "line " ++
247238 show (sourceLine pos) ++ " column " ++ show (sourceColumn pos)
248− where
249− sn' = sourceName pos
250− sn = if sn' == "source" || sn' == "" || sn' == "-"
251− then ""
252− else sn' ++ " "
239+ where sn = if sourceName pos == "source" || sourceName pos == ""
240+ then ""
241+ else sourceName pos ++ " "
253242
254243encodeLogMessages :: [LogMessage] -> BL.ByteString
255244encodeLogMessages ms =
@@ -282,7 +271,7 @@ showLogMessage msg =
282271 ParsingUnescaped s pos ->
283272 "Parsing unescaped '" <> s <> "' at " <> showPos pos
284273 CouldNotLoadIncludeFile fp pos ->
285− "Could not load include file " <> fp <> " at " <> showPos pos
274+ "Could not load include file '" <> fp <> "' at " <> showPos pos
286275 MacroAlreadyDefined name pos ->
287276 "Macro '" <> name <> "' already defined, ignoring at " <> showPos pos
288277 InlineNotRendered il ->
@@ -294,18 +283,18 @@ showLogMessage msg =
294283 IgnoredIOError s ->
295284 "IO Error (ignored): " <> s
296285 CouldNotFetchResource fp s ->
297− "Could not fetch resource " <> fp <>
286+ "Could not fetch resource '" <> fp <> "'" <>
298287 if Text.null s then "" else ": " <> s
299288 CouldNotDetermineImageSize fp s ->
300− "Could not determine image size for " <> fp <>
289+ "Could not determine image size for '" <> fp <> "'" <>
301290 if Text.null s then "" else ": " <> s
302291 CouldNotConvertImage fp s ->
303− "Could not convert image " <> fp <>
292+ "Could not convert image '" <> fp <> "'" <>
304293 if Text.null s then "" else ": " <> s
305294 CouldNotDetermineMimeType fp ->
306− "Could not determine mime type for " <> fp
295+ "Could not determine mime type for '" <> fp <> "'"
307296 CouldNotConvertTeXMath s m ->
308− "Could not convert TeX math " <> s <> ", rendering as TeX" <>
297+ "Could not convert TeX math '" <> s <> "', rendering as TeX" <>
309298 if Text.null m then "" else ":\n" <> m
310299 CouldNotParseCSS m ->
311300 "Could not parse CSS" <> if Text.null m then "" else ":\n" <> m
@@ -313,8 +302,6 @@ showLogMessage msg =
313302 "Fetching " <> fp <> "..."
314303 Extracting fp ->
315304 "Extracting " <> fp <> "..."
316− LoadedResource orig found ->
317− "Loaded " <> Text.pack orig <> " from " <> Text.pack found
318305 NoTitleElement fallback ->
319306 "This document format requires a nonempty <title> element.\n" <>
320307 "Defaulting to '" <> fallback <> "' as the title.\n" <>
@@ -363,8 +350,6 @@ showLogMessage msg =
363350 else ""
364351 EnvironmentVariableUndefined var ->
365352 "Undefined environment variable " <> var <> " in defaults file."
366− DuplicateAttribute attr val ->
367− "Ignoring duplicate attribute " <> attr <> "=" <> tshow val <> "."
368353
369354messageVerbosity :: LogMessage -> Verbosity
370355messageVerbosity msg =
@@ -395,7 +380,6 @@ messageVerbosity msg =
395380 CouldNotParseCSS{} -> WARNING
396381 Fetching{} -> INFO
397382 Extracting{} -> INFO
398− LoadedResource{} -> INFO
399383 NoTitleElement{} -> WARNING
400384 NoLangSpecified -> INFO
401385 InvalidLang{} -> WARNING
@@ -413,4 +397,3 @@ messageVerbosity msg =
413397 CiteprocWarning{} -> WARNING
414398 ATXHeadingInLHS{} -> WARNING
415399 EnvironmentVariableUndefined{}-> WARNING
416− DuplicateAttribute{} -> WARNING
src/Text/Pandoc/Lua/Filter.hs+1−4
@@ -13,9 +13,7 @@ module Text.Pandoc.Lua.Filter ( LuaFilterFunction
1313 , LuaFilter
1414 , runFilterFile
1515 , walkInlines
16− , walkInlineLists
1716 , walkBlocks
18− , walkBlockLists
1917 , module Text.Pandoc.Lua.Walk
2018 ) where
2119import Control.Applicative ((<|>))
@@ -24,7 +22,6 @@ import Control.Monad.Catch (finally, try)
2422import Data.Data (Data, DataType, dataTypeConstrs, dataTypeName, dataTypeOf,
2523 showConstr, toConstr, tyconUQname)
2624import Data.Foldable (foldrM)
27−import Data.List (foldl')
2825import Data.Map (Map)
2926import Data.Maybe (fromMaybe)
3027import Foreign.Lua (Lua, Peekable, Pushable, StackIndex)
@@ -207,7 +204,7 @@ walkMeta lf (Pandoc m bs) = do
207204
208205walkPandoc :: LuaFilter -> Pandoc -> Lua Pandoc
209206walkPandoc (LuaFilter fnMap) =
210− case foldl' mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
207+ case foldl mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
211208 Just fn -> \x -> runFilterFunction fn x *> singleElement x
212209 Nothing -> return
213210
src/Text/Pandoc/Lua/Module/MediaBag.hs+3−3
@@ -73,9 +73,9 @@ lookup fp = do
7373 res <- MB.lookupMedia fp <$> getMediaBag
7474 liftPandocLua $ case res of
7575 Nothing -> 1 <$ Lua.pushnil
76− Just item -> do
77− Lua.push $ MB.mediaMimeType item
78− Lua.push $ MB.mediaContents item
76+ Just (mimeType, contents) -> do
77+ Lua.push mimeType
78+ Lua.push contents
7979 return 2
8080
8181list :: PandocLua NumResults
src/Text/Pandoc/Lua/Module/Pandoc.hs+3−8
@@ -23,10 +23,8 @@ import Foreign.Lua (Lua, NumResults, Optional, Peekable, Pushable)
2323import System.Exit (ExitCode (..))
2424import Text.Pandoc.Class.PandocIO (runIO)
2525import Text.Pandoc.Definition (Block, Inline)
26−import Text.Pandoc.Lua.Filter (LuaFilter, SingletonsList (..), walkInlines,
27− walkInlineLists, walkBlocks, walkBlockLists)
26+import Text.Pandoc.Lua.Filter (walkInlines, walkBlocks, LuaFilter, SingletonsList (..))
2827import Text.Pandoc.Lua.Marshaling ()
29−import Text.Pandoc.Lua.Marshaling.List (List (..))
3028import Text.Pandoc.Lua.PandocLua (PandocLua, addFunction, liftPandocLua,
3129 loadDefaultModule)
3230import Text.Pandoc.Walk (Walkable)
@@ -53,12 +51,9 @@ pushModule = do
5351 return 1
5452
5553walkElement :: (Walkable (SingletonsList Inline) a,
56− Walkable (SingletonsList Block) a,
57− Walkable (List Inline) a,
58− Walkable (List Block) a)
54+ Walkable (SingletonsList Block) a)
5955 => a -> LuaFilter -> PandocLua a
60−walkElement x f = liftPandocLua $
61− walkInlines f x >>= walkInlineLists f >>= walkBlocks f >>= walkBlockLists f
56+walkElement x f = liftPandocLua $ walkInlines f x >>= walkBlocks f
6257
6358walk_inline :: Inline -> LuaFilter -> PandocLua Inline
6459walk_inline = walkElement
src/Text/Pandoc/Lua/Module/Utils.hs+1−1
@@ -146,7 +146,7 @@ from_simple_table (SimpleTable capt aligns widths head' body) = do
146146 nullAttr
147147 (Caption Nothing [Plain capt])
148148 (zipWith (\a w -> (a, toColWidth w)) aligns widths)
149− (TableHead nullAttr [blockListToRow head' | not (null head') ])
149+ (TableHead nullAttr [blockListToRow head'])
150150 [TableBody nullAttr 0 [] $ map blockListToRow body]
151151 (TableFoot nullAttr [])
152152 return (NumResults 1)
src/Text/Pandoc/MIME.hs−10
@@ -43,16 +43,6 @@ getMimeTypeDef :: FilePath -> MimeType
4343getMimeTypeDef = fromMaybe "application/octet-stream" . getMimeType
4444
4545extensionFromMimeType :: MimeType -> Maybe T.Text
46−-- few special cases, where there are multiple options:
47−extensionFromMimeType "text/plain" = Just "txt"
48−extensionFromMimeType "video/quicktime" = Just "mov"
49−extensionFromMimeType "video/mpeg" = Just "mpeg"
50−extensionFromMimeType "video/dv" = Just "dv"
51−extensionFromMimeType "image/vnd.djvu" = Just "djvu"
52−extensionFromMimeType "image/tiff" = Just "tiff"
53−extensionFromMimeType "image/jpeg" = Just "jpg"
54−extensionFromMimeType "application/xml" = Just "xml"
55−extensionFromMimeType "application/ogg" = Just "ogg"
5646extensionFromMimeType mimetype =
5747 M.lookup (T.takeWhile (/=';') mimetype) reverseMimeTypes
5848 -- note: we just look up the basic mime type, dropping the content-encoding etc.
src/Text/Pandoc/MediaBag.hs+14−45
@@ -1,5 +1,4 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE OverloadedStrings #-}
32{-# LANGUAGE DeriveDataTypeable #-}
43{-# LANGUAGE GeneralizedNewtypeDeriving #-}
54{- |
@@ -15,7 +14,6 @@ Definition of a MediaBag object to hold binary resources, and an
1514interface for interacting with it.
1615-}
1716module Text.Pandoc.MediaBag (
18− MediaItem(..),
1917 MediaBag,
2018 deleteMedia,
2119 lookupMedia,
@@ -26,43 +24,29 @@ module Text.Pandoc.MediaBag (
2624import qualified Data.ByteString.Lazy as BL
2725import Data.Data (Data)
2826import qualified Data.Map as M
29−import Data.Maybe (fromMaybe, isNothing)
27+import Data.Maybe (fromMaybe)
3028import Data.Typeable (Typeable)
3129import System.FilePath
32−import Text.Pandoc.MIME (MimeType, getMimeTypeDef, extensionFromMimeType)
33−import Data.Text (Text)
34−import qualified Data.Text as T
35−import Data.Digest.Pure.SHA (sha1, showDigest)
36−import Network.URI (URI (..), parseURI)
37−
38−data MediaItem =
39− MediaItem
40− { mediaMimeType :: MimeType
41− , mediaPath :: FilePath
42− , mediaContents :: BL.ByteString
43− } deriving (Eq, Ord, Show, Data, Typeable)
30+import qualified System.FilePath.Posix as Posix
31+import Text.Pandoc.MIME (MimeType, getMimeTypeDef)
4432
4533-- | A container for a collection of binary resources, with names and
4634-- mime types. Note that a 'MediaBag' is a Monoid, so 'mempty'
4735-- can be used for an empty 'MediaBag', and '<>' can be used to append
4836-- two 'MediaBag's.
49−newtype MediaBag = MediaBag (M.Map Text MediaItem)
37+newtype MediaBag = MediaBag (M.Map [FilePath] (MimeType, BL.ByteString))
5038 deriving (Semigroup, Monoid, Data, Typeable)
5139
5240instance Show MediaBag where
5341 show bag = "MediaBag " ++ show (mediaDirectory bag)
5442
55−-- | We represent paths with /, in normalized form.
56−canonicalize :: FilePath -> Text
57−canonicalize = T.replace "\\" "/" . T.pack . normalise
58−
5943-- | Delete a media item from a 'MediaBag', or do nothing if no item corresponds
6044-- to the given path.
6145deleteMedia :: FilePath -- ^ relative path and canonical name of resource
6246 -> MediaBag
6347 -> MediaBag
6448deleteMedia fp (MediaBag mediamap) =
65− MediaBag $ M.delete (canonicalize fp) mediamap
49+ MediaBag $ M.delete (splitDirectories fp) mediamap
6650
6751-- | Insert a media item into a 'MediaBag', replacing any existing
6852-- value with the same name.
@@ -72,41 +56,26 @@ insertMedia :: FilePath -- ^ relative path and canonical name of resource
7256 -> MediaBag
7357 -> MediaBag
7458insertMedia fp mbMime contents (MediaBag mediamap) =
75− MediaBag (M.insert fp' mediaItem mediamap)
76− where mediaItem = MediaItem{ mediaPath = newpath
77− , mediaContents = contents
78− , mediaMimeType = mt }
79− fp' = canonicalize fp
80− uri = parseURI fp
81− newpath = if isRelative fp
82− && isNothing uri
83− && ".." `notElem` splitPath fp
84− then T.unpack fp'
85− else showDigest (sha1 contents) <> "." <> ext
59+ MediaBag (M.insert (splitDirectories fp) (mime, contents) mediamap)
60+ where mime = fromMaybe fallback mbMime
8661 fallback = case takeExtension fp of
8762 ".gz" -> getMimeTypeDef $ dropExtension fp
8863 _ -> getMimeTypeDef fp
89− mt = fromMaybe fallback mbMime
90− path = maybe fp uriPath uri
91− ext = case takeExtension path of
92− '.':e -> e
93− _ -> maybe "" T.unpack $ extensionFromMimeType mt
94−
9564
9665-- | Lookup a media item in a 'MediaBag', returning mime type and contents.
9766lookupMedia :: FilePath
9867 -> MediaBag
99− -> Maybe MediaItem
100−lookupMedia fp (MediaBag mediamap) = M.lookup (canonicalize fp) mediamap
68+ -> Maybe (MimeType, BL.ByteString)
69+lookupMedia fp (MediaBag mediamap) = M.lookup (splitDirectories fp) mediamap
10170
10271-- | Get a list of the file paths stored in a 'MediaBag', with
10372-- their corresponding mime types and the lengths in bytes of the contents.
10473mediaDirectory :: MediaBag -> [(FilePath, MimeType, Int)]
105−mediaDirectory mediabag =
106− map (\(fp, mt, bs) -> (fp, mt, fromIntegral (BL.length bs)))
107− (mediaItems mediabag)
74+mediaDirectory (MediaBag mediamap) =
75+ M.foldrWithKey (\fp (mime,contents) ->
76+ ((Posix.joinPath fp, mime, fromIntegral $ BL.length contents):)) [] mediamap
10877
10978mediaItems :: MediaBag -> [(FilePath, MimeType, BL.ByteString)]
11079mediaItems (MediaBag mediamap) =
111− map (\item -> (mediaPath item, mediaMimeType item, mediaContents item))
112− (M.elems mediamap)
80+ M.foldrWithKey (\fp (mime,contents) ->
81+ ((Posix.joinPath fp, mime, contents):)) [] mediamap
src/Text/Pandoc/Options.hs+17−17
@@ -315,23 +315,6 @@ defaultMathJaxURL = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.j
315315defaultKaTeXURL :: Text
316316defaultKaTeXURL = "https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/"
317317
318−-- Update documentation in doc/filters.md if this is changed.
319−$(deriveJSON defaultOptions{ constructorTagModifier =
320− camelCaseStrToHyphenated
321− } ''TrackChanges)
322−
323−$(deriveJSON defaultOptions{ constructorTagModifier =
324− camelCaseStrToHyphenated
325− } ''WrapOption)
326−
327−$(deriveJSON defaultOptions{ constructorTagModifier =
328− camelCaseStrToHyphenated . drop 8
329− } ''TopLevelDivision)
330−
331−$(deriveJSON defaultOptions{ constructorTagModifier =
332− camelCaseStrToHyphenated
333− } ''ReferenceLocation)
334−
335318-- Update documentation in doc/filters.md if this is changed.
336319$(deriveJSON defaultOptions ''ReaderOptions)
337320
@@ -355,3 +338,20 @@ $(deriveJSON defaultOptions{ constructorTagModifier =
355338 } ''ObfuscationMethod)
356339
357340$(deriveJSON defaultOptions ''HTMLSlideVariant)
341+
342+-- Update documentation in doc/filters.md if this is changed.
343+$(deriveJSON defaultOptions{ constructorTagModifier =
344+ camelCaseStrToHyphenated
345+ } ''TrackChanges)
346+
347+$(deriveJSON defaultOptions{ constructorTagModifier =
348+ camelCaseStrToHyphenated
349+ } ''WrapOption)
350+
351+$(deriveJSON defaultOptions{ constructorTagModifier =
352+ camelCaseStrToHyphenated . drop 8
353+ } ''TopLevelDivision)
354+
355+$(deriveJSON defaultOptions{ constructorTagModifier =
356+ camelCaseStrToHyphenated
357+ } ''ReferenceLocation)
src/Text/Pandoc/PDF.hs+3−16
@@ -202,7 +202,7 @@ convertImage opts tmpdir fname = do
202202 Just "image/svg+xml" -> E.catch (do
203203 (exit, _) <- pipeProcess Nothing "rsvg-convert"
204204 ["-f","pdf","-a","--dpi-x",dpi,"--dpi-y",dpi,
205− "-o",pdfOut,svgIn] BL.empty
205+ "-o",pdfOut,fname] BL.empty
206206 if exit == ExitSuccess
207207 then return $ Right pdfOut
208208 else return $ Left "conversion from SVG failed")
@@ -217,7 +217,6 @@ convertImage opts tmpdir fname = do
217217 where
218218 pngOut = replaceDirectory (replaceExtension fname ".png") tmpdir
219219 pdfOut = replaceDirectory (replaceExtension fname ".pdf") tmpdir
220− svgIn = tmpdir </> fname
221220 mime = getMimeType fname
222221 doNothing = return (Right fname)
223222
@@ -507,20 +506,8 @@ showVerboseInfo mbTmpDir program programArgs env source = do
507506 UTF8.hPutStrLn stderr $
508507 T.pack program <> " " <> T.pack (unwords (map show programArgs))
509508 UTF8.hPutStr stderr "\n"
510− UTF8.hPutStrLn stderr "[makePDF] Relevant environment variables:"
511− -- we filter out irrelevant stuff to avoid leaking passwords and keys!
512− let isRelevant ("PATH",_) = True
513− isRelevant ("TMPDIR",_) = True
514− isRelevant ("PWD",_) = True
515− isRelevant ("LANG",_) = True
516− isRelevant ("HOME",_) = True
517− isRelevant ("LUA_PATH",_) = True
518− isRelevant ("LUA_CPATH",_) = True
519− isRelevant ("SHELL",_) = True
520− isRelevant ("TEXINPUTS",_) = True
521− isRelevant ("TEXMFOUTPUT",_) = True
522− isRelevant _ = False
523− mapM_ (UTF8.hPutStrLn stderr . tshow) (filter isRelevant env)
509+ UTF8.hPutStrLn stderr "[makePDF] Environment:"
510+ mapM_ (UTF8.hPutStrLn stderr . tshow) env
524511 UTF8.hPutStr stderr "\n"
525512 UTF8.hPutStrLn stderr "[makePDF] Source:"
526513 UTF8.hPutStrLn stderr source
src/Text/Pandoc/Parsing.hs+242−360
@@ -5,6 +5,7 @@
55{-# LANGUAGE GeneralizedNewtypeDeriving #-}
66{-# LANGUAGE IncoherentInstances #-}
77{-# LANGUAGE MultiParamTypeClasses #-}
8+{-# LANGUAGE ViewPatterns #-}
89{-# LANGUAGE OverloadedStrings #-}
910{- |
1011 Module : Text.Pandoc.Parsing
@@ -18,7 +19,8 @@
1819A utility library with parsers used in pandoc readers.
1920-}
2021
21−module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
22+module Text.Pandoc.Parsing ( take1WhileP,
23+ takeP,
2224 countChar,
2325 textStr,
2426 anyLine,
@@ -103,14 +105,14 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
103105 singleQuoteEnd,
104106 doubleQuoteStart,
105107 doubleQuoteEnd,
106− apostrophe,
107− doubleCloseQuote,
108108 ellipses,
109+ apostrophe,
109110 dash,
110111 nested,
111112 citeKey,
112113 Parser,
113114 ParserT,
115+ F,
114116 Future(..),
115117 runF,
116118 askF,
@@ -121,6 +123,7 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
121123 (<+?>),
122124 extractIdClass,
123125 insertIncludedFile,
126+ insertIncludedFileF,
124127 -- * Re-exports from Text.Parsec
125128 Stream,
126129 runParser,
@@ -131,10 +134,22 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
131134 getInput,
132135 setInput,
133136 unexpected,
137+ char,
138+ letter,
139+ digit,
140+ alphaNum,
134141 skipMany,
135142 skipMany1,
143+ spaces,
144+ space,
145+ anyChar,
146+ satisfy,
147+ newline,
148+ string,
136149 count,
137150 eof,
151+ noneOf,
152+ oneOf,
138153 lookAhead,
139154 notFollowedBy,
140155 many,
@@ -159,8 +174,6 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
159174 SourcePos,
160175 getPosition,
161176 setPosition,
162− sourceName,
163− setSourceName,
164177 sourceColumn,
165178 sourceLine,
166179 setSourceColumn,
@@ -176,141 +189,48 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
176189where
177190
178191import Control.Monad.Identity
179− ( guard,
180− join,
181− unless,
182− when,
183− void,
184− liftM2,
185− liftM,
186− Identity(..),
187− MonadPlus(mzero) )
188192import Control.Monad.Reader
189− ( asks, runReader, MonadReader(ask), Reader, ReaderT(ReaderT) )
190193import Data.Char (chr, isAlphaNum, isAscii, isAsciiUpper, isAsciiLower,
191− isSpace, ord, toLower, toUpper)
192−import Data.Default ( Default(..) )
194+ isPunctuation, isSpace, ord, toLower, toUpper)
195+import Data.Default
193196import Data.Functor (($>))
194197import Data.List (intercalate, transpose)
195198import qualified Data.Map as M
196−import Data.Maybe (fromMaybe)
199+import Data.Maybe (mapMaybe, fromMaybe)
197200import qualified Data.Set as Set
201+import Data.String
198202import Data.Text (Text)
199203import qualified Data.Text as T
200204import Text.HTML.TagSoup.Entity (lookupEntity)
201−import Text.Pandoc.Asciify (toAsciiText)
205+import Text.Pandoc.Asciify (toAsciiChar)
202206import Text.Pandoc.Builder (Blocks, HasMeta (..), Inlines, trimInlines)
203207import qualified Text.Pandoc.Builder as B
204208import Text.Pandoc.Class.PandocMonad (PandocMonad, readFileFromDirs, report)
205209import Text.Pandoc.Definition
206− ( Target,
207− nullMeta,
208− nullAttr,
209− Meta,
210− ColWidth(ColWidthDefault, ColWidth),
211− TableFoot(TableFoot),
212− TableBody(TableBody),
213− Attr,
214− TableHead(TableHead),
215− Row(..),
216− Alignment(..),
217− Inline(Str),
218− ListNumberDelim(..),
219− ListAttributes,
220− ListNumberStyle(..) )
221210import Text.Pandoc.Logging
222− ( LogMessage(CouldNotLoadIncludeFile, DuplicateIdentifier) )
223211import Text.Pandoc.Options
224− ( extensionEnabled,
225− Extension(Ext_old_dashes, Ext_tex_math_dollars,
226− Ext_tex_math_single_backslash, Ext_tex_math_double_backslash,
227− Ext_auto_identifiers, Ext_ascii_identifiers, Ext_smart),
228− ReaderOptions(readerTabStop, readerColumns, readerExtensions) )
229212import Text.Pandoc.Readers.LaTeX.Types (Macro)
230213import Text.Pandoc.Shared
231− ( uniqueIdent,
232− tshow,
233− mapLeft,
234− compactify,
235− trim,
236− trimr,
237− splitTextByIndices,
238− safeRead,
239− trimMath,
240− schemes,
241− escapeURI )
242−import Text.Pandoc.Sources
243214import qualified Text.Pandoc.UTF8 as UTF8 (putStrLn)
244215import Text.Pandoc.XML (fromEntities)
245−import Text.Parsec
246− ( between,
247− setSourceName,
248− Parsec,
249− Column,
250− Line,
251− incSourceLine,
252− incSourceColumn,
253− setSourceLine,
254− setSourceColumn,
255− sourceLine,
256− sourceColumn,
257− sourceName,
258− setSourceName,
259− setPosition,
260− getPosition,
261− updateState,
262− setState,
263− getState,
264− optionMaybe,
265− optional,
266− option,
267− endBy1,
268− endBy,
269− sepEndBy1,
270− sepEndBy,
271− sepBy1,
272− sepBy,
273− try,
274− choice,
275− (<?>),
276− (<|>),
277− manyTill,
278− many1,
279− many,
280− notFollowedBy,
281− lookAhead,
282− eof,
283− count,
284− skipMany1,
285− skipMany,
286− unexpected,
287− setInput,
288− getInput,
289− anyToken,
290− tokenPrim,
291− parse,
292− runParserT,
293− runParser,
294− ParseError,
295− ParsecT,
296− SourcePos,
297− Stream(..) )
298−import Text.Parsec.Pos (initialPos, newPos)
299−import Control.Monad.Except ( MonadError(throwError) )
216+import Text.Parsec hiding (token)
217+import Text.Parsec.Pos (initialPos, newPos, updatePosString)
218+
219+import Control.Monad.Except
300220import Text.Pandoc.Error
301− ( PandocError(PandocParseError, PandocParsecError) )
302221
303222type Parser t s = Parsec t s
304223
305224type ParserT = ParsecT
306225
307−
308226-- | Reader monad wrapping the parser state. This is used to possibly delay
309227-- evaluation until all relevant information has been parsed and made available
310228-- in the parser state.
311229newtype Future s a = Future { runDelayed :: Reader s a }
312230 deriving (Monad, Applicative, Functor)
313231
232+type F = Future ParserState
233+
314234runF :: Future s a -> s -> a
315235runF = runReader . runDelayed
316236
@@ -333,48 +253,70 @@ instance (Semigroup a, Monoid a) => Monoid (Future s a) where
333253 mappend = (<>)
334254
335255-- | Like @count@, but packs its result
336−countChar :: (Stream s m Char, UpdateSourcePos s Char, Monad m)
256+countChar :: (Stream s m Char, Monad m)
337257 => Int
338258 -> ParsecT s st m Char
339259 -> ParsecT s st m Text
340260countChar n = fmap T.pack . count n
341261
342262-- | Like @string@, but uses @Text@.
343−textStr :: (Stream s m Char, UpdateSourcePos s Char)
344− => Text -> ParsecT s u m Text
263+textStr :: Stream s m Char => Text -> ParsecT s u m Text
345264textStr t = string (T.unpack t) $> t
346265
347−
348−-- | Parse any line of text, returning the contents without the
349−-- final newline.
350−anyLine :: Monad m => ParserT Sources st m Text
266+-- | Parse characters while a predicate is true.
267+take1WhileP :: Monad m
268+ => (Char -> Bool)
269+ -> ParserT Text st m Text
270+take1WhileP f = do
271+ -- needed to persuade parsec that this won't match an empty string:
272+ c <- satisfy f
273+ inp <- getInput
274+ pos <- getPosition
275+ let (t, rest) = T.span f inp
276+ setInput rest
277+ setPosition $
278+ if f '\t' || f '\n'
279+ then updatePosString pos $ T.unpack t
280+ else incSourceColumn pos (T.length t)
281+ return $ T.singleton c <> t
282+
283+-- Parse n characters of input (or the rest of the input if
284+-- there aren't n characters).
285+takeP :: Monad m => Int -> ParserT Text st m Text
286+takeP n = do
287+ guard (n > 0)
288+ -- faster than 'count n anyChar'
289+ inp <- getInput
290+ pos <- getPosition
291+ let (xs, rest) = T.splitAt n inp
292+ -- needed to persuade parsec that this won't match an empty string:
293+ anyChar
294+ setInput rest
295+ setPosition $ updatePosString pos $ T.unpack xs
296+ return xs
297+
298+-- | Parse any line of text
299+anyLine :: Monad m => ParserT Text st m Text
351300anyLine = do
352301 -- This is much faster than:
353302 -- manyTill anyChar newline
354303 inp <- getInput
355− case inp of
356− Sources [] -> mzero
357− Sources ((fp,t):inps) ->
358− -- we assume that lines don't span different input files
359− case T.break (=='\n') t of
360− (this, rest)
361− | T.null rest
362− , not (null inps) ->
363− -- line may span different input files, so do it
364− -- character by character
365− T.pack <$> manyTill anyChar newline
366− | otherwise -> do -- either end of inputs or newline in rest
367− setInput $ Sources ((fp, rest):inps)
368− char '\n' -- needed so parsec knows we won't match empty string
369− -- and so source pos is updated
370− return this
304+ pos <- getPosition
305+ case T.break (=='\n') inp of
306+ (this, T.uncons -> Just ('\n', rest)) -> do
307+ -- needed to persuade parsec that this won't match an empty string:
308+ anyChar
309+ setInput rest
310+ setPosition $ incSourceLine (setSourceColumn pos 1) 1
311+ return this
312+ _ -> mzero
371313
372314-- | Parse any line, include the final newline in the output
373−anyLineNewline :: Monad m => ParserT Sources st m Text
315+anyLineNewline :: Monad m => ParserT Text st m Text
374316anyLineNewline = (<> "\n") <$> anyLine
375317
376318-- | Parse indent by specified number of spaces (or equiv. tabs)
377−indentWith :: (Stream s m Char, UpdateSourcePos s Char)
319+indentWith :: Stream s m Char
378320 => HasReaderOptions st
379321 => Int -> ParserT s st m Text
380322indentWith num = do
@@ -459,13 +401,11 @@ notFollowedBy' p = try $ join $ do a <- try p
459401 return (return ())
460402-- (This version due to Andrew Pimlott on the Haskell mailing list.)
461403
462−oneOfStrings' :: (Stream s m Char, UpdateSourcePos s Char)
463− => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
404+oneOfStrings' :: Stream s m Char => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
464405oneOfStrings' f = fmap T.pack . oneOfStrings'' f . fmap T.unpack
465406
466407-- TODO: This should be re-implemented in a Text-aware way
467−oneOfStrings'' :: (Stream s m Char, UpdateSourcePos s Char)
468− => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
408+oneOfStrings'' :: Stream s m Char => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
469409oneOfStrings'' _ [] = Prelude.fail "no strings"
470410oneOfStrings'' matches strs = try $ do
471411 c <- anyChar
@@ -480,16 +420,14 @@ oneOfStrings'' matches strs = try $ do
480420-- | Parses one of a list of strings. If the list contains
481421-- two strings one of which is a prefix of the other, the longer
482422-- string will be matched if possible.
483−oneOfStrings :: (Stream s m Char, UpdateSourcePos s Char)
484− => [Text] -> ParserT s st m Text
423+oneOfStrings :: Stream s m Char => [Text] -> ParserT s st m Text
485424oneOfStrings = oneOfStrings' (==)
486425
487426-- | Parses one of a list of strings (tried in order), case insensitive.
488427
489428-- TODO: This will not be accurate with general Unicode (neither
490429-- Text.toLower nor Text.toCaseFold can be implemented with a map)
491−oneOfStringsCI :: (Stream s m Char, UpdateSourcePos s Char)
492− => [Text] -> ParserT s st m Text
430+oneOfStringsCI :: Stream s m Char => [Text] -> ParserT s st m Text
493431oneOfStringsCI = oneOfStrings' ciMatch
494432 where ciMatch x y = toLower' x == toLower' y
495433 -- this optimizes toLower by checking common ASCII case
@@ -500,13 +438,11 @@ oneOfStringsCI = oneOfStrings' ciMatch
500438 | otherwise = toLower c
501439
502440-- | Parses a space or tab.
503−spaceChar :: (Stream s m Char, UpdateSourcePos s Char)
504− => ParserT s st m Char
441+spaceChar :: Stream s m Char => ParserT s st m Char
505442spaceChar = satisfy $ \c -> c == ' ' || c == '\t'
506443
507444-- | Parses a nonspace, nonnewline character.
508−nonspaceChar :: (Stream s m Char, UpdateSourcePos s Char)
509− => ParserT s st m Char
445+nonspaceChar :: Stream s m Char => ParserT s st m Char
510446nonspaceChar = satisfy (not . isSpaceChar)
511447
512448isSpaceChar :: Char -> Bool
@@ -517,24 +453,21 @@ isSpaceChar '\r' = True
517453isSpaceChar _ = False
518454
519455-- | Skips zero or more spaces or tabs.
520−skipSpaces :: (Stream s m Char, UpdateSourcePos s Char)
521− => ParserT s st m ()
456+skipSpaces :: Stream s m Char => ParserT s st m ()
522457skipSpaces = skipMany spaceChar
523458
524459-- | Skips zero or more spaces or tabs, then reads a newline.
525−blankline :: (Stream s m Char, UpdateSourcePos s Char)
526− => ParserT s st m Char
460+blankline :: Stream s m Char => ParserT s st m Char
527461blankline = try $ skipSpaces >> newline
528462
529463-- | Parses one or more blank lines and returns a string of newlines.
530−blanklines :: (Stream s m Char, UpdateSourcePos s Char)
531− => ParserT s st m Text
464+blanklines :: Stream s m Char => ParserT s st m Text
532465blanklines = T.pack <$> many1 blankline
533466
534467-- | Gobble n spaces; if tabs are encountered, expand them
535468-- and gobble some or all of their spaces, leaving the rest.
536469gobbleSpaces :: (HasReaderOptions st, Monad m)
537− => Int -> ParserT Sources st m ()
470+ => Int -> ParserT Text st m ()
538471gobbleSpaces 0 = return ()
539472gobbleSpaces n
540473 | n < 0 = error "gobbleSpaces called with negative number"
@@ -542,26 +475,18 @@ gobbleSpaces n
542475 char ' ' <|> eatOneSpaceOfTab
543476 gobbleSpaces (n - 1)
544477
545−eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Sources st m Char
478+eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Text st m Char
546479eatOneSpaceOfTab = do
547− lookAhead (char '\t')
548− pos <- getPosition
480+ char '\t'
549481 tabstop <- getOption readerTabStop
550− -- replace the tab on the input stream with spaces
551− let numSpaces = tabstop - ((sourceColumn pos - 1) `mod` tabstop)
552482 inp <- getInput
553− setInput $
554− case inp of
555− Sources [] -> error "eatOneSpaceOfTab - empty Sources list"
556− Sources ((fp,t):rest) ->
557− -- drop the tab and add spaces
558− Sources ((fp, T.replicate numSpaces " " <> T.drop 1 t):rest)
559− char ' '
483+ setInput $ T.replicate (tabstop - 1) " " <> inp
484+ return ' '
560485
561486-- | Gobble up to n spaces; if tabs are encountered, expand them
562487-- and gobble some or all of their spaces, leaving the rest.
563488gobbleAtMostSpaces :: (HasReaderOptions st, Monad m)
564− => Int -> ParserT Sources st m Int
489+ => Int -> ParserT Text st m Int
565490gobbleAtMostSpaces 0 = return 0
566491gobbleAtMostSpaces n
567492 | n < 0 = error "gobbleAtMostSpaces called with negative number"
@@ -570,8 +495,7 @@ gobbleAtMostSpaces n
570495 (+ 1) <$> gobbleAtMostSpaces (n - 1)
571496
572497-- | Parses material enclosed between start and end parsers.
573−enclosed :: (Show end, Stream s m Char, UpdateSourcePos s Char)
574− => ParserT s st m t -- ^ start parser
498+enclosed :: (Show end, Stream s m Char) => ParserT s st m t -- ^ start parser
575499 -> ParserT s st m end -- ^ end parser
576500 -> ParserT s st m a -- ^ content parser (to be used repeatedly)
577501 -> ParserT s st m [a]
@@ -579,41 +503,39 @@ enclosed start end parser = try $
579503 start >> notFollowedBy space >> many1Till parser end
580504
581505-- | Parse string, case insensitive.
582−stringAnyCase :: (Stream s m Char, UpdateSourcePos s Char)
583− => Text -> ParserT s st m Text
506+stringAnyCase :: Stream s m Char => Text -> ParserT s st m Text
584507stringAnyCase = fmap T.pack . stringAnyCase' . T.unpack
585508
586−stringAnyCase' :: (Stream s m Char, UpdateSourcePos s Char)
587− => String -> ParserT s st m String
509+stringAnyCase' :: Stream s m Char => String -> ParserT s st m String
588510stringAnyCase' [] = string ""
589511stringAnyCase' (x:xs) = do
590512 firstChar <- char (toUpper x) <|> char (toLower x)
591513 rest <- stringAnyCase' xs
592514 return (firstChar:rest)
593515
594−-- TODO rewrite by just adding to Sources stream?
595516-- | Parse contents of 'str' using 'parser' and return result.
596−parseFromString :: Monad m
597− => ParserT Sources st m r
517+parseFromString :: (Stream s m Char, IsString s)
518+ => ParserT s st m r
598519 -> Text
599− -> ParserT Sources st m r
520+ -> ParserT s st m r
600521parseFromString parser str = do
601522 oldPos <- getPosition
602523 setPosition $ initialPos "chunk"
603524 oldInput <- getInput
604− setInput $ toSources str
525+ setInput $ fromString $ T.unpack str
605526 result <- parser
606527 spaces
528+ eof
607529 setInput oldInput
608530 setPosition oldPos
609531 return result
610532
611533-- | Like 'parseFromString' but specialized for 'ParserState'.
612534-- This resets 'stateLastStrPos', which is almost always what we want.
613−parseFromString' :: (Monad m, HasLastStrPosition u)
614− => ParserT Sources u m a
535+parseFromString' :: (Stream s m Char, IsString s, HasLastStrPosition u)
536+ => ParserT s u m a
615537 -> Text
616− -> ParserT Sources u m a
538+ -> ParserT s u m a
617539parseFromString' parser str = do
618540 oldLastStrPos <- getLastStrPos <$> getState
619541 updateState $ setLastStrPos Nothing
@@ -622,7 +544,7 @@ parseFromString' parser str = do
622544 return res
623545
624546-- | Parse raw line block up to and including blank lines.
625−lineClump :: Monad m => ParserT Sources st m Text
547+lineClump :: Monad m => ParserT Text st m Text
626548lineClump = blanklines
627549 <|> (T.unlines <$> many1 (notFollowedBy blankline >> anyLine))
628550
@@ -631,7 +553,7 @@ lineClump = blanklines
631553-- pairs of open and close, which must be different. For example,
632554-- @charsInBalanced '(' ')' anyChar@ will parse "(hello (there))"
633555-- and return "hello (there)".
634−charsInBalanced :: (Stream s m Char, UpdateSourcePos s Char) => Char -> Char -> ParserT s st m Char
556+charsInBalanced :: Stream s m Char => Char -> Char -> ParserT s st m Char
635557 -> ParserT s st m Text
636558charsInBalanced open close parser = try $ do
637559 char open
@@ -650,7 +572,7 @@ charsInBalanced open close parser = try $ do
650572-- Auxiliary functions for romanNumeral:
651573
652574-- | Parses a roman numeral (uppercase or lowercase), returns number.
653−romanNumeral :: (Stream s m Char, UpdateSourcePos s Char) => Bool -- ^ Uppercase if true
575+romanNumeral :: Stream s m Char => Bool -- ^ Uppercase if true
654576 -> ParserT s st m Int
655577romanNumeral upperCase = do
656578 let rchar uc = char $ if upperCase then uc else toLower uc
@@ -686,19 +608,20 @@ romanNumeral upperCase = do
686608
687609-- | Parses an email address; returns original and corresponding
688610-- escaped mailto: URI.
689−emailAddress :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
611+emailAddress :: Stream s m Char => ParserT s st m (Text, Text)
690612emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
691613 where toResult mbox dom = let full = fromEntities $ T.pack $ mbox ++ '@':dom
692614 in (full, escapeURI $ "mailto:" <> full)
693615 mailbox = intercalate "." <$> (emailWord `sepBy1'` dot)
694616 domain = intercalate "." <$> (subdomain `sepBy1'` dot)
695617 dot = char '.'
696− subdomain = many1 $ alphaNum <|> innerPunct (=='-')
618+ subdomain = many1 $ alphaNum <|> innerPunct
697619 -- this excludes some valid email addresses, since an
698620 -- email could contain e.g. '__', but gives better results
699621 -- for our purposes, when combined with markdown parsing:
700− innerPunct f = try (satisfy f
701− <* notFollowedBy (satisfy (not . isAlphaNum)))
622+ innerPunct = try (satisfy (\c -> isEmailPunct c || c == '@')
623+ <* notFollowedBy space
624+ <* notFollowedBy (satisfy isPunctuation))
702625 -- technically an email address could begin with a symbol,
703626 -- but allowing this creates too many problems.
704627 -- See e.g. https://github.com/jgm/pandoc/issues/2940
@@ -709,11 +632,11 @@ emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
709632 isEmailPunct c = T.any (== c) "!\"#$%&'*+-/=?^_{|}~;"
710633
711634
712−uriScheme :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Text
635+uriScheme :: Stream s m Char => ParserT s st m Text
713636uriScheme = oneOfStringsCI (Set.toList schemes)
714637
715638-- | Parses a URI. Returns pair of original and URI-escaped version.
716−uri :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
639+uri :: Stream s m Char => ParserT s st m (Text, Text)
717640uri = try $ do
718641 scheme <- uriScheme
719642 char ':'
@@ -756,7 +679,7 @@ uri = try $ do
756679 uriChunkBetween l r = try $ do chunk <- between (char l) (char r) uriChunk
757680 return (T.pack $ [l] ++ chunk ++ [r])
758681
759−mathInlineWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
682+mathInlineWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
760683mathInlineWith op cl = try $ do
761684 textStr op
762685 when (op == "$") $ notFollowedBy space
@@ -777,10 +700,10 @@ mathInlineWith op cl = try $ do
777700 notFollowedBy digit -- to prevent capture of $5
778701 return $ trimMath $ T.concat words'
779702 where
780− inBalancedBraces :: (Stream s m Char, UpdateSourcePos s Char) => Int -> Text -> ParserT s st m Text
703+ inBalancedBraces :: Stream s m Char => Int -> Text -> ParserT s st m Text
781704 inBalancedBraces n = fmap T.pack . inBalancedBraces' n . T.unpack
782705
783− inBalancedBraces' :: (Stream s m Char, UpdateSourcePos s Char) => Int -> String -> ParserT s st m String
706+ inBalancedBraces' :: Stream s m Char => Int -> String -> ParserT s st m String
784707 inBalancedBraces' 0 "" = do
785708 c <- anyChar
786709 if c == '{'
@@ -797,13 +720,13 @@ mathInlineWith op cl = try $ do
797720 '{' -> inBalancedBraces' (numOpen + 1) (c:xs)
798721 _ -> inBalancedBraces' numOpen (c:xs)
799722
800−mathDisplayWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
723+mathDisplayWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
801724mathDisplayWith op cl = try $ fmap T.pack $ do
802725 textStr op
803726 many1Till (satisfy (/= '\n') <|> (newline <* notFollowedBy' blankline))
804727 (try $ textStr cl)
805728
806−mathDisplay :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
729+mathDisplay :: (HasReaderOptions st, Stream s m Char)
807730 => ParserT s st m Text
808731mathDisplay =
809732 (guardEnabled Ext_tex_math_dollars >> mathDisplayWith "$$" "$$")
@@ -812,7 +735,7 @@ mathDisplay =
812735 <|> (guardEnabled Ext_tex_math_double_backslash >>
813736 mathDisplayWith "\\\\[" "\\\\]")
814737
815−mathInline :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
738+mathInline :: (HasReaderOptions st , Stream s m Char)
816739 => ParserT s st m Text
817740mathInline =
818741 (guardEnabled Ext_tex_math_dollars >> mathInlineWith "$" "$")
@@ -825,7 +748,7 @@ mathInline =
825748-- displacement (the difference between the source column at the end
826749-- and the source column at the beginning). Vertical displacement
827750-- (source row) is ignored.
828−withHorizDisplacement :: (Stream s m Char, UpdateSourcePos s Char)
751+withHorizDisplacement :: Stream s m Char
829752 => ParserT s st m a -- ^ Parser to apply
830753 -> ParserT s st m (a, Int) -- ^ (result, displacement)
831754withHorizDisplacement parser = do
@@ -837,37 +760,30 @@ withHorizDisplacement parser = do
837760-- | Applies a parser and returns the raw string that was parsed,
838761-- along with the value produced by the parser.
839762withRaw :: Monad m
840− => ParsecT Sources st m a
841− -> ParsecT Sources st m (a, Text)
763+ => ParsecT Text st m a
764+ -> ParsecT Text st m (a, Text)
842765withRaw parser = do
843− inps1 <- getInput
766+ pos1 <- getPosition
767+ inp <- getInput
844768 result <- parser
845− inps2 <- getInput
846− -- 'raw' is the difference between inps1 and inps2
847− return (result, sourcesDifference inps1 inps2)
848−
849−sourcesDifference :: Sources -> Sources -> Text
850−sourcesDifference (Sources is1) (Sources is2) = go is1 is2
851− where
852− go inps1 inps2 =
853− case (inps1, inps2) of
854− ([], _) -> mempty
855− (_, []) -> mconcat $ map snd inps1
856− ((p1,t1):rest1, (p2, t2):rest2)
857− | p1 == p2
858− , t1 == t2 -> go rest1 rest2
859− | p1 == p2
860− , t1 /= t2 -> fromMaybe mempty $ T.stripSuffix t2 t1
861− | otherwise -> t1 <> go rest1 inps2
769+ pos2 <- getPosition
770+ let (l1,c1) = (sourceLine pos1, sourceColumn pos1)
771+ let (l2,c2) = (sourceLine pos2, sourceColumn pos2)
772+ let inplines = take ((l2 - l1) + 1) $ T.lines inp
773+ let raw = case inplines of
774+ [] -> ""
775+ [l] -> T.take (c2 - c1) l
776+ ls -> T.unlines (init ls) <> T.take (c2 - 1) (last ls)
777+ return (result, raw)
862778
863779-- | Parses backslash, then applies character parser.
864−escaped :: (Stream s m Char, UpdateSourcePos s Char)
780+escaped :: Stream s m Char
865781 => ParserT s st m Char -- ^ Parser for character to escape
866782 -> ParserT s st m Char
867783escaped parser = try $ char '\\' >> parser
868784
869785-- | Parse character entity.
870−characterReference :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
786+characterReference :: Stream s m Char => ParserT s st m Char
871787characterReference = try $ do
872788 char '&'
873789 ent <- many1Till nonspaceChar (char ';')
@@ -880,19 +796,19 @@ characterReference = try $ do
880796 _ -> Prelude.fail "entity not found"
881797
882798-- | Parses an uppercase roman numeral and returns (UpperRoman, number).
883−upperRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
799+upperRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
884800upperRoman = do
885801 num <- romanNumeral True
886802 return (UpperRoman, num)
887803
888804-- | Parses a lowercase roman numeral and returns (LowerRoman, number).
889−lowerRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
805+lowerRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
890806lowerRoman = do
891807 num <- romanNumeral False
892808 return (LowerRoman, num)
893809
894810-- | Parses a decimal numeral and returns (Decimal, number).
895−decimal :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
811+decimal :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
896812decimal = do
897813 num <- many1 digit
898814 return (Decimal, fromMaybe 1 $ safeRead $ T.pack num)
@@ -901,7 +817,7 @@ decimal = do
901817-- returns (DefaultStyle, [next example number]). The next
902818-- example number is incremented in parser state, and the label
903819-- (if present) is added to the label table.
904−exampleNum :: (Stream s m Char, UpdateSourcePos s Char)
820+exampleNum :: Stream s m Char
905821 => ParserT s ParserState m (ListNumberStyle, Int)
906822exampleNum = do
907823 char '@'
@@ -920,37 +836,37 @@ exampleNum = do
920836 return (Example, num)
921837
922838-- | Parses a '#' returns (DefaultStyle, 1).
923−defaultNum :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
839+defaultNum :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
924840defaultNum = do
925841 char '#'
926842 return (DefaultStyle, 1)
927843
928844-- | Parses a lowercase letter and returns (LowerAlpha, number).
929−lowerAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
845+lowerAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
930846lowerAlpha = do
931847 ch <- satisfy isAsciiLower
932848 return (LowerAlpha, ord ch - ord 'a' + 1)
933849
934850-- | Parses an uppercase letter and returns (UpperAlpha, number).
935−upperAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
851+upperAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
936852upperAlpha = do
937853 ch <- satisfy isAsciiUpper
938854 return (UpperAlpha, ord ch - ord 'A' + 1)
939855
940856-- | Parses a roman numeral i or I
941−romanOne :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
857+romanOne :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
942858romanOne = (char 'i' >> return (LowerRoman, 1)) <|>
943859 (char 'I' >> return (UpperRoman, 1))
944860
945861-- | Parses an ordered list marker and returns list attributes.
946−anyOrderedListMarker :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s ParserState m ListAttributes
862+anyOrderedListMarker :: Stream s m Char => ParserT s ParserState m ListAttributes
947863anyOrderedListMarker = choice
948864 [delimParser numParser | delimParser <- [inPeriod, inOneParen, inTwoParens],
949865 numParser <- [decimal, exampleNum, defaultNum, romanOne,
950866 lowerAlpha, lowerRoman, upperAlpha, upperRoman]]
951867
952868-- | Parses a list number (num) followed by a period, returns list attributes.
953−inPeriod :: (Stream s m Char, UpdateSourcePos s Char)
869+inPeriod :: Stream s m Char
954870 => ParserT s st m (ListNumberStyle, Int)
955871 -> ParserT s st m ListAttributes
956872inPeriod num = try $ do
@@ -962,7 +878,7 @@ inPeriod num = try $ do
962878 return (start, style, delim)
963879
964880-- | Parses a list number (num) followed by a paren, returns list attributes.
965−inOneParen :: (Stream s m Char, UpdateSourcePos s Char)
881+inOneParen :: Stream s m Char
966882 => ParserT s st m (ListNumberStyle, Int)
967883 -> ParserT s st m ListAttributes
968884inOneParen num = try $ do
@@ -971,7 +887,7 @@ inOneParen num = try $ do
971887 return (start, style, OneParen)
972888
973889-- | Parses a list number (num) enclosed in parens, returns list attributes.
974−inTwoParens :: (Stream s m Char, UpdateSourcePos s Char)
890+inTwoParens :: Stream s m Char
975891 => ParserT s st m (ListNumberStyle, Int)
976892 -> ParserT s st m ListAttributes
977893inTwoParens num = try $ do
@@ -982,7 +898,7 @@ inTwoParens num = try $ do
982898
983899-- | Parses an ordered list marker with a given style and delimiter,
984900-- returns number.
985−orderedListMarker :: (Stream s m Char, UpdateSourcePos s Char)
901+orderedListMarker :: Stream s m Char
986902 => ListNumberStyle
987903 -> ListNumberDelim
988904 -> ParserT s ParserState m Int
@@ -1005,10 +921,10 @@ orderedListMarker style delim = do
1005921 return start
1006922
1007923-- | Parses a character reference and returns a Str element.
1008−charRef :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inline
924+charRef :: Stream s m Char => ParserT s st m Inline
1009925charRef = Str . T.singleton <$> characterReference
1010926
1011−lineBlockLine :: Monad m => ParserT Sources st m Text
927+lineBlockLine :: Monad m => ParserT Text st m Text
1012928lineBlockLine = try $ do
1013929 char '|'
1014930 char ' '
@@ -1018,11 +934,11 @@ lineBlockLine = try $ do
1018934 continuations <- many (try $ char ' ' >> anyLine)
1019935 return $ white <> T.unwords (line : continuations)
1020936
1021−blankLineBlockLine :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
937+blankLineBlockLine :: Stream s m Char => ParserT s st m Char
1022938blankLineBlockLine = try (char '|' >> blankline)
1023939
1024940-- | Parses an RST-style line block and returns a list of strings.
1025−lineBlockLines :: Monad m => ParserT Sources st m [Text]
941+lineBlockLines :: Monad m => ParserT Text st m [Text]
1026942lineBlockLines = try $ do
1027943 lines' <- many1 (lineBlockLine <|> (T.singleton <$> blankLineBlockLine))
1028944 skipMany blankline
@@ -1030,8 +946,7 @@ lineBlockLines = try $ do
1030946
1031947-- | Parse a table using 'headerParser', 'rowParser',
1032948-- 'lineParser', and 'footerParser'.
1033−tableWith :: (Stream s m Char, UpdateSourcePos s Char,
1034− HasReaderOptions st, Monad mf)
949+tableWith :: (Stream s m Char, HasReaderOptions st, Monad mf)
1035950 => ParserT s st m (mf [Blocks], [Alignment], [Int])
1036951 -> ([Int] -> ParserT s st m (mf [Blocks]))
1037952 -> ParserT s st m sep
@@ -1051,8 +966,7 @@ tableWith headerParser rowParser lineParser footerParser = try $ do
1051966
1052967type TableComponents mf = ([Alignment], [Double], mf [Row], mf [Row])
1053968
1054−tableWith' :: (Stream s m Char, UpdateSourcePos s Char,
1055− HasReaderOptions st, Monad mf)
969+tableWith' :: (Stream s m Char, HasReaderOptions st, Monad mf)
1056970 => ParserT s st m (mf [Blocks], [Alignment], [Int])
1057971 -> ([Int] -> ParserT s st m (mf [Blocks]))
1058972 -> ParserT s st m sep
@@ -1101,19 +1015,20 @@ widthsFromIndices numColumns' indices =
11011015-- (which may be grid), then the rows,
11021016-- which may be grid, separated by blank lines, and
11031017-- ending with a footer (dashed line followed by blank line).
1104−gridTableWith :: (Monad m, HasReaderOptions st, HasLastStrPosition st, Monad mf)
1105− => ParserT Sources st m (mf Blocks) -- ^ Block list parser
1018+gridTableWith :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
1019+ Monad mf, IsString s)
1020+ => ParserT s st m (mf Blocks) -- ^ Block list parser
11061021 -> Bool -- ^ Headerless table
1107− -> ParserT Sources st m (mf Blocks)
1022+ -> ParserT s st m (mf Blocks)
11081023gridTableWith blocks headless =
11091024 tableWith (gridTableHeader headless blocks) (gridTableRow blocks)
11101025 (gridTableSep '-') gridTableFooter
11111026
1112−gridTableWith' :: (Monad m, HasReaderOptions st, HasLastStrPosition st,
1113− Monad mf)
1114− => ParserT Sources st m (mf Blocks) -- ^ Block list parser
1027+gridTableWith' :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
1028+ Monad mf, IsString s)
1029+ => ParserT s st m (mf Blocks) -- ^ Block list parser
11151030 -> Bool -- ^ Headerless table
1116− -> ParserT Sources st m (TableComponents mf)
1031+ -> ParserT s st m (TableComponents mf)
11171032gridTableWith' blocks headless =
11181033 tableWith' (gridTableHeader headless blocks) (gridTableRow blocks)
11191034 (gridTableSep '-') gridTableFooter
@@ -1122,7 +1037,7 @@ gridTableSplitLine :: [Int] -> Text -> [Text]
11221037gridTableSplitLine indices line = map removeFinalBar $ tail $
11231038 splitTextByIndices (init indices) $ trimr line
11241039
1125−gridPart :: Monad m => Char -> ParserT Sources st m ((Int, Int), Alignment)
1040+gridPart :: Stream s m Char => Char -> ParserT s st m ((Int, Int), Alignment)
11261041gridPart ch = do
11271042 leftColon <- option False (True <$ char ':')
11281043 dashes <- many1 (char ch)
@@ -1137,7 +1052,7 @@ gridPart ch = do
11371052 (False, False) -> AlignDefault
11381053 return ((lengthDashes, lengthDashes + 1), alignment)
11391054
1140−gridDashedLines :: Monad m => Char -> ParserT Sources st m [((Int, Int), Alignment)]
1055+gridDashedLines :: Stream s m Char => Char -> ParserT s st m [((Int, Int), Alignment)]
11411056gridDashedLines ch = try $ char '+' >> many1 (gridPart ch) <* blankline
11421057
11431058removeFinalBar :: Text -> Text
@@ -1146,14 +1061,14 @@ removeFinalBar = T.dropWhileEnd go . T.dropWhileEnd (=='|')
11461061 go c = T.any (== c) " \t"
11471062
11481063-- | Separator between rows of grid table.
1149−gridTableSep :: Monad m => Char -> ParserT Sources st m Char
1064+gridTableSep :: Stream s m Char => Char -> ParserT s st m Char
11501065gridTableSep ch = try $ gridDashedLines ch >> return '\n'
11511066
11521067-- | Parse header for a grid table.
1153−gridTableHeader :: (Monad m, Monad mf, HasLastStrPosition st)
1068+gridTableHeader :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
11541069 => Bool -- ^ Headerless table
1155− -> ParserT Sources st m (mf Blocks)
1156− -> ParserT Sources st m (mf [Blocks], [Alignment], [Int])
1070+ -> ParserT s st m (mf Blocks)
1071+ -> ParserT s st m (mf [Blocks], [Alignment], [Int])
11571072gridTableHeader True _ = do
11581073 optional blanklines
11591074 dashes <- gridDashedLines '-'
@@ -1176,17 +1091,17 @@ gridTableHeader False blocks = try $ do
11761091 heads <- sequence <$> mapM (parseFromString' blocks . trim) rawHeads
11771092 return (heads, aligns, indices)
11781093
1179−gridTableRawLine :: (Stream s m Char, UpdateSourcePos s Char) => [Int] -> ParserT s st m [Text]
1094+gridTableRawLine :: Stream s m Char => [Int] -> ParserT s st m [Text]
11801095gridTableRawLine indices = do
11811096 char '|'
11821097 line <- many1Till anyChar newline
11831098 return (gridTableSplitLine indices $ T.pack line)
11841099
11851100-- | Parse row of grid table.
1186−gridTableRow :: (Monad m, Monad mf, HasLastStrPosition st)
1187− => ParserT Sources st m (mf Blocks)
1101+gridTableRow :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
1102+ => ParserT s st m (mf Blocks)
11881103 -> [Int]
1189− -> ParserT Sources st m (mf [Blocks])
1104+ -> ParserT s st m (mf [Blocks])
11901105gridTableRow blocks indices = do
11911106 colLines <- many1 (gridTableRawLine indices)
11921107 let cols = map ((<> "\n") . T.unlines . removeOneLeadingSpace) $
@@ -1207,38 +1122,34 @@ removeOneLeadingSpace xs =
12071122 Just (c, _) -> c == ' '
12081123
12091124-- | Parse footer for a grid table.
1210−gridTableFooter :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
1125+gridTableFooter :: Stream s m Char => ParserT s st m ()
12111126gridTableFooter = optional blanklines
12121127
12131128---
12141129
12151130-- | Removes the ParsecT layer from the monad transformer stack
1216−readWithM :: (Monad m, ToSources t)
1217− => ParserT Sources st m a -- ^ parser
1218− -> st -- ^ initial state
1219− -> t -- ^ input
1131+readWithM :: (Stream s m Char, ToText s)
1132+ => ParserT s st m a -- ^ parser
1133+ -> st -- ^ initial state
1134+ -> s -- ^ input
12201135 -> m (Either PandocError a)
12211136readWithM parser state input =
1222− mapLeft (PandocParsecError sources)
1223− <$> runParserT parser state (initialSourceName sources) sources
1224− where
1225− sources = toSources input
1137+ mapLeft (PandocParsecError $ toText input) `liftM` runParserT parser state "source" input
12261138
12271139-- | Parse a string with a given parser and state
1228−readWith :: ToSources t
1229− => Parser Sources st a
1140+readWith :: Parser Text st a
12301141 -> st
1231− -> t
1142+ -> Text
12321143 -> Either PandocError a
12331144readWith p t inp = runIdentity $ readWithM p t inp
12341145
12351146-- | Parse a string with @parser@ (for testing).
12361147testStringWith :: Show a
1237− => ParserT Sources ParserState Identity a
1148+ => ParserT Text ParserState Identity a
12381149 -> Text
12391150 -> IO ()
12401151testStringWith parser str = UTF8.putStrLn $ tshow $
1241− readWith parser defaultParserState (toSources str)
1152+ readWith parser defaultParserState str
12421153
12431154-- | Parsing options.
12441155data ParserState = ParserState
@@ -1258,7 +1169,7 @@ data ParserState = ParserState
12581169 stateInNote :: Bool, -- ^ True if parsing note contents
12591170 stateNoteNumber :: Int, -- ^ Last note number for citations
12601171 stateMeta :: Meta, -- ^ Document metadata
1261− stateMeta' :: Future ParserState Meta, -- ^ Document metadata
1172+ stateMeta' :: F Meta, -- ^ Document metadata
12621173 stateCitations :: M.Map Text Text, -- ^ RST-style citations
12631174 stateHeaderTable :: [HeaderType], -- ^ Ordered list of header types used
12641175 stateIdentifiers :: Set.Set Text, -- ^ Header identifiers used
@@ -1437,7 +1348,7 @@ data QuoteContext
14371348
14381349type NoteTable = [(Text, Text)]
14391350
1440−type NoteTable' = M.Map Text (SourcePos, Future ParserState Blocks)
1351+type NoteTable' = M.Map Text (SourcePos, F Blocks)
14411352-- used in markdown reader
14421353
14431354newtype Key = Key Text deriving (Show, Read, Eq, Ord)
@@ -1472,7 +1383,7 @@ registerHeader (ident,classes,kvs) header' = do
14721383 then do
14731384 let id' = uniqueIdent exts (B.toList header') ids
14741385 let id'' = if Ext_ascii_identifiers `extensionEnabled` exts
1475− then toAsciiText id'
1386+ then T.pack $ mapMaybe toAsciiChar $ T.unpack id'
14761387 else id'
14771388 updateState $ updateIdentifierList $ Set.insert id'
14781389 updateState $ updateIdentifierList $ Set.insert id''
@@ -1485,42 +1396,34 @@ registerHeader (ident,classes,kvs) header' = do
14851396 updateState $ updateIdentifierList $ Set.insert ident
14861397 return (ident,classes,kvs)
14871398
1488−smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st,
1489− HasQuoteContext st m,
1490− Stream s m Char, UpdateSourcePos s Char)
1399+smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
14911400 => ParserT s st m Inlines
14921401 -> ParserT s st m Inlines
14931402smartPunctuation inlineParser = do
14941403 guardEnabled Ext_smart
1495− choice [ quoted inlineParser, apostrophe, doubleCloseQuote, dash, ellipses ]
1404+ choice [ quoted inlineParser, apostrophe, dash, ellipses ]
14961405
1497−quoted :: (HasLastStrPosition st, HasQuoteContext st m,
1498− Stream s m Char, UpdateSourcePos s Char)
1406+apostrophe :: Stream s m Char => ParserT s st m Inlines
1407+apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\x2019")
1408+
1409+quoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
14991410 => ParserT s st m Inlines
15001411 -> ParserT s st m Inlines
15011412quoted inlineParser = doubleQuoted inlineParser <|> singleQuoted inlineParser
15021413
1503−singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m,
1504− Stream s m Char, UpdateSourcePos s Char)
1414+singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
15051415 => ParserT s st m Inlines
15061416 -> ParserT s st m Inlines
1507−singleQuoted inlineParser = do
1508− singleQuoteStart
1509− (B.singleQuoted . mconcat <$>
1510− try
1511− (withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)))
1512− <|> pure "\8217"
1513−
1514−doubleQuoted :: (HasQuoteContext st m, HasLastStrPosition st,
1515− Stream s m Char, UpdateSourcePos s Char)
1417+singleQuoted inlineParser = try $ B.singleQuoted . mconcat
1418+ <$ singleQuoteStart
1419+ <*> withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)
1420+
1421+doubleQuoted :: (HasQuoteContext st m, Stream s m Char)
15161422 => ParserT s st m Inlines
15171423 -> ParserT s st m Inlines
1518−doubleQuoted inlineParser = do
1519− doubleQuoteStart
1520− (B.doubleQuoted . mconcat <$>
1521− try
1522− (withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)))
1523− <|> pure (B.str "\8220")
1424+doubleQuoted inlineParser = try $ B.doubleQuoted . mconcat
1425+ <$ doubleQuoteStart
1426+ <*> withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)
15241427
15251428failIfInQuoteContext :: (HasQuoteContext st m, Stream s m t)
15261429 => QuoteContext
@@ -1529,14 +1432,13 @@ failIfInQuoteContext context = do
15291432 context' <- getQuoteContext
15301433 when (context' == context) $ Prelude.fail "already inside quotes"
15311434
1532−charOrRef :: (Stream s m Char, UpdateSourcePos s Char) => [Char] -> ParserT s st m Char
1435+charOrRef :: Stream s m Char => [Char] -> ParserT s st m Char
15331436charOrRef cs =
15341437 oneOf cs <|> try (do c <- characterReference
15351438 guard (c `elem` cs)
15361439 return c)
15371440
1538−singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m,
1539− Stream s m Char, UpdateSourcePos s Char)
1441+singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
15401442 => ParserT s st m ()
15411443singleQuoteStart = do
15421444 failIfInQuoteContext InSingleQuote
@@ -1544,39 +1446,30 @@ singleQuoteStart = do
15441446 guard =<< notAfterString
15451447 try $ do
15461448 charOrRef "'\8216\145"
1547− void $ lookAhead (satisfy (not . isSpaceChar))
1449+ notFollowedBy (oneOf [' ', '\t', '\n'])
15481450
1549−singleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
1451+singleQuoteEnd :: Stream s m Char
15501452 => ParserT s st m ()
15511453singleQuoteEnd = try $ do
15521454 charOrRef "'\8217\146"
15531455 notFollowedBy alphaNum
15541456
1555−doubleQuoteStart :: (HasLastStrPosition st,
1556− HasQuoteContext st m,
1557− Stream s m Char, UpdateSourcePos s Char)
1457+doubleQuoteStart :: (HasQuoteContext st m, Stream s m Char)
15581458 => ParserT s st m ()
15591459doubleQuoteStart = do
15601460 failIfInQuoteContext InDoubleQuote
1561− guard =<< notAfterString
15621461 try $ do charOrRef "\"\8220\147"
1563− void $ lookAhead (satisfy (not . isSpaceChar))
1462+ notFollowedBy (oneOf [' ', '\t', '\n'])
15641463
1565−doubleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
1464+doubleQuoteEnd :: Stream s m Char
15661465 => ParserT s st m ()
15671466doubleQuoteEnd = void (charOrRef "\"\8221\148")
15681467
1569−apostrophe :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1570−apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\8217")
1571−
1572−doubleCloseQuote :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1573−doubleCloseQuote = B.str "\8221" <$ char '"'
1574−
1575−ellipses :: (Stream s m Char, UpdateSourcePos s Char)
1468+ellipses :: Stream s m Char
15761469 => ParserT s st m Inlines
15771470ellipses = try (string "..." >> return (B.str "\8230"))
15781471
1579−dash :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
1472+dash :: (HasReaderOptions st, Stream s m Char)
15801473 => ParserT s st m Inlines
15811474dash = try $ do
15821475 oldDashes <- extensionEnabled Ext_old_dashes <$> getOption readerExtensions
@@ -1603,28 +1496,20 @@ nested p = do
16031496 updateState $ \st -> st{ stateMaxNestingLevel = nestlevel }
16041497 return res
16051498
1606−citeKey :: (Stream s m Char, UpdateSourcePos s Char, HasLastStrPosition st)
1607− => Bool -- ^ If True, allow expanded @{..} syntax.
1608− -> ParserT s st m (Bool, Text)
1609−citeKey allowBraced = try $ do
1499+citeKey :: (Stream s m Char, HasLastStrPosition st)
1500+ => ParserT s st m (Bool, Text)
1501+citeKey = try $ do
16101502 guard =<< notAfterString
16111503 suppress_author <- option False (True <$ char '-')
16121504 char '@'
1613− key <- simpleCiteIdentifier
1614− <|> if allowBraced
1615− then charsInBalanced '{' '}' (satisfy (not . isSpace))
1616− else mzero
1617− return (suppress_author, key)
1618−
1619−simpleCiteIdentifier :: (Stream s m Char, UpdateSourcePos s Char)
1620− => ParserT s st m Text
1621−simpleCiteIdentifier = do
16221505 firstChar <- alphaNum <|> char '_' <|> char '*' -- @* for wildcard in nocite
16231506 let regchar = satisfy (\c -> isAlphaNum c || c == '_')
16241507 let internal p = try $ p <* lookAhead regchar
16251508 rest <- many $ regchar <|> internal (oneOf ":.#$%&-+?<>~/") <|>
16261509 try (oneOf ":/" <* lookAhead (char '/'))
1627− return $ T.pack $ firstChar:rest
1510+ let key = firstChar:rest
1511+ return (suppress_author, T.pack key)
1512+
16281513
16291514token :: (Stream s m t)
16301515 => (t -> Text)
@@ -1644,15 +1529,12 @@ extractIdClass (ident, cls, kvs) = (ident', cls', kvs')
16441529 cls' = maybe cls T.words $ lookup "class" kvs
16451530 kvs' = filter (\(k,_) -> k /= "id" || k /= "class") kvs
16461531
1647−insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1648− => ParserT a st m b -- ^ parser to apply
1649− -> (Text -> a) -- ^ convert Text to stream type
1650− -> [FilePath] -- ^ search path (directories)
1651− -> FilePath -- ^ path of file to include
1652− -> Maybe Int -- ^ start line (negative counts from end)
1653− -> Maybe Int -- ^ end line (negative counts from end)
1654− -> ParserT a st m b
1655−insertIncludedFile parser toStream dirs f mbstartline mbendline = do
1532+insertIncludedFile' :: (PandocMonad m, HasIncludeFiles st)
1533+ => ParserT a st m (mf Blocks)
1534+ -> (Text -> a)
1535+ -> [FilePath] -> FilePath
1536+ -> ParserT a st m (mf Blocks)
1537+insertIncludedFile' blocks totoks dirs f = do
16561538 oldPos <- getPosition
16571539 oldInput <- getInput
16581540 containers <- getIncludeFiles <$> getState
@@ -1661,32 +1543,32 @@ insertIncludedFile parser toStream dirs f mbstartline mbendline = do
16611543 updateState $ addIncludeFile $ T.pack f
16621544 mbcontents <- readFileFromDirs dirs f
16631545 contents <- case mbcontents of
1664− Just s -> return $ exciseLines mbstartline mbendline s
1546+ Just s -> return s
16651547 Nothing -> do
16661548 report $ CouldNotLoadIncludeFile (T.pack f) oldPos
16671549 return ""
1668− setInput $ toStream contents
1669− setPosition $ newPos f (fromMaybe 1 mbstartline) 1
1670− result <- parser
1550+ setPosition $ newPos f 1 1
1551+ setInput $ totoks contents
1552+ bs <- blocks
16711553 setInput oldInput
16721554 setPosition oldPos
16731555 updateState dropLatestIncludeFile
1674− return result
1675−
1676−exciseLines :: Maybe Int -> Maybe Int -> Text -> Text
1677−exciseLines Nothing Nothing t = t
1678−exciseLines mbstartline mbendline t =
1679− T.unlines $ take (endline' - (startline' - 1))
1680− $ drop (startline' - 1) contentLines
1681− where
1682− contentLines = T.lines t
1683− numLines = length contentLines
1684− startline' = case mbstartline of
1685− Nothing -> 1
1686− Just x | x >= 0 -> x
1687− | otherwise -> numLines + x -- negative from end
1688− endline' = case mbendline of
1689− Nothing -> numLines
1690− Just x | x >= 0 -> x
1691− | otherwise -> numLines + x -- negative from end
1556+ return bs
16921557
1558+-- | Parse content of include file as blocks. Circular includes result in an
1559+-- @PandocParseError@.
1560+insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1561+ => ParserT [a] st m Blocks
1562+ -> (Text -> [a])
1563+ -> [FilePath] -> FilePath
1564+ -> ParserT [a] st m Blocks
1565+insertIncludedFile blocks totoks dirs f =
1566+ runIdentity <$> insertIncludedFile' (Identity <$> blocks) totoks dirs f
1567+
1568+-- | Parse content of include file as future blocks. Circular includes result in
1569+-- an @PandocParseError@.
1570+insertIncludedFileF :: (PandocMonad m, HasIncludeFiles st)
1571+ => ParserT Text st m (Future st Blocks)
1572+ -> [FilePath] -> FilePath
1573+ -> ParserT Text st m (Future st Blocks)
1574+insertIncludedFileF p = insertIncludedFile' p id
src/Text/Pandoc/Readers.hs+48−55
@@ -1,5 +1,4 @@
11{-# LANGUAGE MonoLocalBinds #-}
2−{-# LANGUAGE RankNTypes #-}
32{-# LANGUAGE ScopedTypeVariables #-}
43{-# LANGUAGE OverloadedStrings #-}
54{- |
@@ -66,14 +65,12 @@ import Data.Aeson
6665import qualified Data.ByteString.Lazy as BL
6766import Data.Text (Text)
6867import qualified Data.Text as T
69−import Text.Pandoc.Shared (tshow)
7068import Text.Pandoc.Class
7169import Text.Pandoc.Definition
7270import Text.Pandoc.Error
7371import Text.Pandoc.Extensions
7472import Text.Pandoc.Options
7573import Text.Pandoc.Readers.CommonMark
76−import Text.Pandoc.Readers.Markdown
7774import Text.Pandoc.Readers.Creole
7875import Text.Pandoc.Readers.DocBook
7976import Text.Pandoc.Readers.Docx
@@ -86,6 +83,7 @@ import Text.Pandoc.Readers.HTML (readHtml)
8683import Text.Pandoc.Readers.JATS (readJATS)
8784import Text.Pandoc.Readers.Jira (readJira)
8885import Text.Pandoc.Readers.LaTeX
86+import Text.Pandoc.Readers.Markdown
8987import Text.Pandoc.Readers.MediaWiki
9088import Text.Pandoc.Readers.Muse
9189import Text.Pandoc.Readers.Native
@@ -103,60 +101,59 @@ import Text.Pandoc.Readers.CSV
103101import Text.Pandoc.Readers.CslJson
104102import Text.Pandoc.Readers.BibTeX
105103import qualified Text.Pandoc.UTF8 as UTF8
106−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
104+import Text.Parsec.Error
107105
108−data Reader m = TextReader (forall a . ToSources a =>
109− ReaderOptions -> a -> m Pandoc)
106+data Reader m = TextReader (ReaderOptions -> Text -> m Pandoc)
110107 | ByteStringReader (ReaderOptions -> BL.ByteString -> m Pandoc)
111108
112109-- | Association list of formats and readers.
113110readers :: PandocMonad m => [(Text, Reader m)]
114−readers = [("native" , TextReader readNative)
115− ,("json" , TextReader readJSON)
116− ,("markdown" , TextReader readMarkdown)
117− ,("markdown_strict" , TextReader readMarkdown)
118− ,("markdown_phpextra" , TextReader readMarkdown)
119− ,("markdown_github" , TextReader readMarkdown)
120− ,("markdown_mmd", TextReader readMarkdown)
121− ,("commonmark" , TextReader readCommonMark)
122− ,("commonmark_x" , TextReader readCommonMark)
123− ,("creole" , TextReader readCreole)
124− ,("dokuwiki" , TextReader readDokuWiki)
125− ,("gfm" , TextReader readCommonMark)
126− ,("rst" , TextReader readRST)
127− ,("mediawiki" , TextReader readMediaWiki)
128− ,("vimwiki" , TextReader readVimwiki)
129− ,("docbook" , TextReader readDocBook)
130− ,("opml" , TextReader readOPML)
131− ,("org" , TextReader readOrg)
132− ,("textile" , TextReader readTextile) -- TODO : textile+lhs
133− ,("html" , TextReader readHtml)
134− ,("jats" , TextReader readJATS)
135− ,("jira" , TextReader readJira)
136− ,("latex" , TextReader readLaTeX)
137− ,("haddock" , TextReader readHaddock)
138− ,("twiki" , TextReader readTWiki)
139− ,("tikiwiki" , TextReader readTikiWiki)
140− ,("docx" , ByteStringReader readDocx)
141− ,("odt" , ByteStringReader readOdt)
142− ,("t2t" , TextReader readTxt2Tags)
143− ,("epub" , ByteStringReader readEPUB)
144− ,("muse" , TextReader readMuse)
145− ,("man" , TextReader readMan)
146− ,("fb2" , TextReader readFB2)
147− ,("ipynb" , TextReader readIpynb)
148− ,("csv" , TextReader readCSV)
149− ,("csljson" , TextReader readCslJson)
150− ,("bibtex" , TextReader readBibTeX)
151− ,("biblatex" , TextReader readBibLaTeX)
111+readers = [ ("native" , TextReader readNative)
112+ ,("json" , TextReader readJSON)
113+ ,("markdown" , TextReader readMarkdown)
114+ ,("markdown_strict" , TextReader readMarkdown)
115+ ,("markdown_phpextra" , TextReader readMarkdown)
116+ ,("markdown_github" , TextReader readMarkdown)
117+ ,("markdown_mmd", TextReader readMarkdown)
118+ ,("commonmark" , TextReader readCommonMark)
119+ ,("commonmark_x" , TextReader readCommonMark)
120+ ,("creole" , TextReader readCreole)
121+ ,("dokuwiki" , TextReader readDokuWiki)
122+ ,("gfm" , TextReader readCommonMark)
123+ ,("rst" , TextReader readRST)
124+ ,("mediawiki" , TextReader readMediaWiki)
125+ ,("vimwiki" , TextReader readVimwiki)
126+ ,("docbook" , TextReader readDocBook)
127+ ,("opml" , TextReader readOPML)
128+ ,("org" , TextReader readOrg)
129+ ,("textile" , TextReader readTextile) -- TODO : textile+lhs
130+ ,("html" , TextReader readHtml)
131+ ,("jats" , TextReader readJATS)
132+ ,("jira" , TextReader readJira)
133+ ,("latex" , TextReader readLaTeX)
134+ ,("haddock" , TextReader readHaddock)
135+ ,("twiki" , TextReader readTWiki)
136+ ,("tikiwiki" , TextReader readTikiWiki)
137+ ,("docx" , ByteStringReader readDocx)
138+ ,("odt" , ByteStringReader readOdt)
139+ ,("t2t" , TextReader readTxt2Tags)
140+ ,("epub" , ByteStringReader readEPUB)
141+ ,("muse" , TextReader readMuse)
142+ ,("man" , TextReader readMan)
143+ ,("fb2" , TextReader readFB2)
144+ ,("ipynb" , TextReader readIpynb)
145+ ,("csv" , TextReader readCSV)
146+ ,("csljson" , TextReader readCslJson)
147+ ,("bibtex" , TextReader readBibTeX)
148+ ,("biblatex" , TextReader readBibLaTeX)
152149 ]
153150
154151-- | Retrieve reader, extensions based on formatSpec (format+extensions).
155152getReader :: PandocMonad m => Text -> m (Reader m, Extensions)
156153getReader s =
157154 case parseFormatSpec s of
158− Left e -> throwError $ PandocAppError $
159− "Error parsing reader format " <> tshow s <> ": " <> tshow e
155+ Left e -> throwError $ PandocAppError
156+ $ T.intercalate "\n" [T.pack m | Message m <- errorMessages e]
160157 Right (readerName, extsToEnable, extsToDisable) ->
161158 case lookup readerName readers of
162159 Nothing -> throwError $ PandocUnknownReaderError
@@ -176,13 +173,9 @@ getReader s =
176173 return (r, exts)
177174
178175-- | Read pandoc document from JSON format.
179−readJSON :: (PandocMonad m, ToSources a)
180− => ReaderOptions
181− -> a
182− -> m Pandoc
183−readJSON _ s =
184− case eitherDecode' . BL.fromStrict . UTF8.fromText
185− . sourcesToText . toSources $ s of
176+readJSON :: PandocMonad m
177+ => ReaderOptions -> Text -> m Pandoc
178+readJSON _ t =
179+ case eitherDecode' . BL.fromStrict . UTF8.fromText $ t of
186180 Right doc -> return doc
187− Left e -> throwError $ PandocParseError ("JSON parse error: "
188− <> T.pack e)
181+ Left e -> throwError $ PandocParseError ("JSON parse error: " <> T.pack e)
src/Text/Pandoc/Readers/BibTeX.hs+8−14
@@ -23,47 +23,41 @@ where
2323import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
26+import Data.Text (Text)
2627import Citeproc (Lang(..), parseLang)
2728import Citeproc.Locale (getLocale)
2829import Text.Pandoc.Error (PandocError(..))
2930import Text.Pandoc.Class (PandocMonad, lookupEnv)
3031import Text.Pandoc.Citeproc.BibTeX as BibTeX
3132import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
32−import Text.Pandoc.Sources (ToSources(..))
3333import Control.Monad.Except (throwError)
3434
3535-- | Read BibTeX from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readBibTeX :: (PandocMonad m, ToSources a)
40− => ReaderOptions -> a -> m Pandoc
39+readBibTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
4140readBibTeX = readBibTeX' BibTeX.Bibtex
4241
4342-- | Read BibLaTeX from an input string and return a Pandoc document.
4443-- The document will have only metadata, with an empty body.
4544-- The metadata will contain a `references` field with the
4645-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
47−readBibLaTeX :: (PandocMonad m, ToSources a)
48− => ReaderOptions -> a -> m Pandoc
46+readBibLaTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
4947readBibLaTeX = readBibTeX' BibTeX.Biblatex
5048
51−readBibTeX' :: (PandocMonad m, ToSources a)
52− => Variant -> ReaderOptions -> a -> m Pandoc
49+readBibTeX' :: PandocMonad m => Variant -> ReaderOptions -> Text -> m Pandoc
5350readBibTeX' variant _opts t = do
54− mblangEnv <- lookupEnv "LANG"
55− let defaultLang = Lang "en" Nothing (Just "US") [] [] []
56− let lang = case mblangEnv of
57− Nothing -> defaultLang
58− Just l -> either (const defaultLang) id $ parseLang l
51+ lang <- maybe (Lang "en" (Just "US")) parseLang
52+ <$> lookupEnv "LANG"
5953 locale <- case getLocale lang of
6054 Left e ->
61− case getLocale (Lang "en" Nothing (Just "US") [] [] []) of
55+ case getLocale (Lang "en" (Just "US")) of
6256 Right l -> return l
6357 Left _ -> throwError $ PandocCiteprocError e
6458 Right l -> return l
6559 case BibTeX.readBibtexString variant locale (const True) t of
66− Left e -> throwError $ PandocParsecError (toSources t) e
60+ Left e -> throwError $ PandocParsecError t e
6761 Right refs -> return $ setMeta "references"
6862 (map referenceToMetaValue refs)
6963 . setMeta "nocite"
src/Text/Pandoc/Readers/CSV.hs+7−7
@@ -13,23 +13,23 @@
1313Conversion from CSV to a 'Pandoc' table.
1414-}
1515module Text.Pandoc.Readers.CSV ( readCSV ) where
16+import Data.Text (Text)
1617import qualified Data.Text as T
1718import Text.Pandoc.CSV (parseCSV, defaultCSVOptions)
1819import Text.Pandoc.Definition
1920import qualified Text.Pandoc.Builder as B
2021import Text.Pandoc.Class (PandocMonad)
22+import Text.Pandoc.Shared (crFilter)
2123import Text.Pandoc.Error
22−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2324import Text.Pandoc.Options (ReaderOptions)
2425import Control.Monad.Except (throwError)
2526
26−readCSV :: (PandocMonad m, ToSources a)
27+readCSV :: PandocMonad m
2728 => ReaderOptions -- ^ Reader options
28− -> a
29+ -> Text -- ^ Text to parse (assuming @'\n'@ line endings)
2930 -> m Pandoc
30−readCSV _opts s = do
31− let txt = sourcesToText $ toSources s
32− case parseCSV defaultCSVOptions txt of
31+readCSV _opts s =
32+ case parseCSV defaultCSVOptions (crFilter s) of
3333 Right (r:rs) -> return $ B.doc $ B.table capt
3434 (zip aligns widths)
3535 (TableHead nullAttr hdrs)
@@ -45,4 +45,4 @@ readCSV _opts s = do
4545 aligns = replicate numcols AlignDefault
4646 widths = replicate numcols ColWidthDefault
4747 Right [] -> return $ B.doc mempty
48− Left e -> throwError $ PandocParsecError (toSources [("",txt)]) e
48+ Left e -> throwError $ PandocParsecError s e
src/Text/Pandoc/Readers/CommonMark.hs+4−46
@@ -26,59 +26,19 @@ import Text.Pandoc.Definition
2626import Text.Pandoc.Builder as B
2727import Text.Pandoc.Options
2828import Text.Pandoc.Error
29−import Text.Pandoc.Readers.Metadata (yamlMetaBlock)
3029import Control.Monad.Except
3130import Data.Functor.Identity (runIdentity)
3231import Data.Typeable
33−import Text.Pandoc.Parsing (runParserT, getInput,
34− runF, defaultParserState, option, many1, anyChar,
35− Sources(..), ToSources(..), ParserT, Future,
36− sourceName)
37−import qualified Data.Text as T
3832
3933-- | Parse a CommonMark formatted string into a 'Pandoc' structure.
40−readCommonMark :: (PandocMonad m, ToSources a)
41− => ReaderOptions -> a -> m Pandoc
34+readCommonMark :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
4235readCommonMark opts s
43− | isEnabled Ext_yaml_metadata_block opts = do
44− let sources = toSources s
45− let toks = concatMap sourceToToks (unSources sources)
46− res <- runParserT (do meta <- yamlMetaBlock (metaValueParser opts)
47− rest <- getInput
48− return (meta, rest))
49− defaultParserState "YAML metadata" (toSources s)
50− case res of
51− Left _ -> readCommonMarkBody opts sources toks
52− Right (meta, rest) -> do
53− -- strip off metadata section and parse body
54− let body = concatMap sourceToToks (unSources rest)
55− Pandoc _ bs <- readCommonMarkBody opts sources body
56− return $ Pandoc (runF meta defaultParserState) bs
57− | otherwise = do
58− let sources = toSources s
59− let toks = concatMap sourceToToks (unSources sources)
60− readCommonMarkBody opts sources toks
61−
62−sourceToToks :: (SourcePos, Text) -> [Tok]
63−sourceToToks (pos, s) = tokenize (sourceName pos) s
64−
65−metaValueParser :: Monad m
66− => ReaderOptions -> ParserT Sources st m (Future st MetaValue)
67−metaValueParser opts = do
68− inp <- option "" $ T.pack <$> many1 anyChar
69− let toks = concatMap sourceToToks (unSources (toSources inp))
70− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
71− Left _ -> mzero
72− Right (Cm bls :: Cm () Blocks) -> return $ return $ B.toMetaValue bls
73−
74−readCommonMarkBody :: PandocMonad m => ReaderOptions -> Sources -> [Tok] -> m Pandoc
75−readCommonMarkBody opts s toks
7636 | isEnabled Ext_sourcepos opts =
77− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
37+ case runIdentity (commonmarkWith (specFor opts) "" s) of
7838 Left err -> throwError $ PandocParsecError s err
7939 Right (Cm bls :: Cm SourceRange Blocks) -> return $ B.doc bls
8040 | otherwise =
81− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
41+ case runIdentity (commonmarkWith (specFor opts) "" s) of
8242 Left err -> throwError $ PandocParsecError s err
8343 Right (Cm bls :: Cm () Blocks) -> return $ B.doc bls
8444
@@ -112,7 +72,5 @@ specFor opts = foldr ($) defaultSyntaxSpec exts
11272 | isEnabled Ext_implicit_header_references opts ] ++
11373 [ (footnoteSpec <>) | isEnabled Ext_footnotes opts ] ++
11474 [ (definitionListSpec <>) | isEnabled Ext_definition_lists opts ] ++
115− [ (taskListSpec <>) | isEnabled Ext_task_lists opts ] ++
116− [ (rebaseRelativePathsSpec <>)
117− | isEnabled Ext_rebase_relative_paths opts ]
75+ [ (taskListSpec <>) | isEnabled Ext_task_lists opts ]
11876
src/Text/Pandoc/Readers/Creole.hs+6−5
@@ -23,20 +23,21 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad (..))
2323import Text.Pandoc.Definition
2424import Text.Pandoc.Options
2525import Text.Pandoc.Parsing hiding (enclosed)
26+import Text.Pandoc.Shared (crFilter)
27+
2628
2729-- | Read creole from an input string and return a Pandoc document.
28−readCreole :: (PandocMonad m, ToSources a)
30+readCreole :: PandocMonad m
2931 => ReaderOptions
30− -> a
32+ -> Text
3133 -> m Pandoc
3234readCreole opts s = do
33− let sources = ensureFinalNewlines 2 (toSources s)
34− res <- readWithM parseCreole def{ stateOptions = opts } sources
35+ res <- readWithM parseCreole def{ stateOptions = opts } $ crFilter s <> "\n\n"
3536 case res of
3637 Left e -> throwError e
3738 Right d -> return d
3839
39−type CRLParser = ParserT Sources ParserState
40+type CRLParser = ParserT Text ParserState
4041
4142--
4243-- Utility functions
src/Text/Pandoc/Readers/CslJson.hs+4−5
@@ -24,22 +24,21 @@ import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
2626import qualified Text.Pandoc.UTF8 as UTF8
27+import Data.Text (Text)
2728import qualified Data.Text as T
2829import Text.Pandoc.Error (PandocError(..))
2930import Text.Pandoc.Class (PandocMonad)
3031import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
3132import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
3233import Control.Monad.Except (throwError)
33−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3434
3535-- | Read CSL JSON from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readCslJson :: (PandocMonad m, ToSources a)
40− => ReaderOptions -> a -> m Pandoc
41−readCslJson _opts x =
42− case cslJsonToReferences (UTF8.fromText $ sourcesToText $ toSources x) of
39+readCslJson :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
40+readCslJson _opts t =
41+ case cslJsonToReferences (UTF8.fromText t) of
4342 Left e -> throwError $ PandocParseError $ T.pack e
4443 Right refs -> return $ setMeta "references"
4544 (map referenceToMetaValue refs)
src/Text/Pandoc/Readers/DocBook.hs+15−30
@@ -18,7 +18,6 @@ import Data.Either (rights)
1818import Data.Foldable (asum)
1919import Data.Generics
2020import Data.List (intersperse,elemIndex)
21−import Data.List.NonEmpty (nonEmpty)
2221import Data.Maybe (fromMaybe,mapMaybe)
2322import Data.Text (Text)
2423import qualified Data.Text as T
@@ -30,8 +29,7 @@ import Text.Pandoc.Builder
3029import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
3130import Text.Pandoc.Options
3231import Text.Pandoc.Logging (LogMessage(..))
33−import Text.Pandoc.Shared (safeRead, extractSpaces)
34−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
32+import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
3533import Text.TeXMath (readMathML, writeTeX)
3634import Text.Pandoc.XML.Light
3735
@@ -134,7 +132,6 @@ List of all DocBook tags, with [x] indicating implemented,
134132[ ] corpcredit - A corporation or organization credited in a document
135133[ ] corpname - The name of a corporation
136134[ ] country - The name of a country
137−[x] danger - An admonition set off from the text indicating hazardous situation
138135[ ] database - The name of a database, or part of a database
139136[x] date - The date of publication or revision of a document
140137[ ] dedication - A wrapper for the dedication section of a book
@@ -541,15 +538,11 @@ instance Default DBState where
541538 , dbContent = [] }
542539
543540
544−readDocBook :: (PandocMonad m, ToSources a)
545− => ReaderOptions
546− -> a
547− -> m Pandoc
541+readDocBook :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
548542readDocBook _ inp = do
549− let sources = toSources inp
550543 tree <- either (throwError . PandocXMLError "") return $
551544 parseXMLContents
552− (TL.fromStrict . handleInstructions . sourcesToText $ sources)
545+ (TL.fromStrict . handleInstructions $ crFilter inp)
553546 (bs, st') <- flip runStateT (def{ dbContent = tree }) $ mapM parseBlock tree
554547 return $ Pandoc (dbMeta st') (toList . mconcat $ bs)
555548
@@ -601,24 +594,16 @@ addMetadataFromElement e = do
601594 Nothing -> return ()
602595 Just z -> addMetaField "author" z
603596 addMetaField "subtitle" e
604− addAuthor e
597+ addMetaField "author" e
605598 addMetaField "date" e
606599 addMetaField "release" e
607600 addMetaField "releaseinfo" e
608601 return mempty
609− where
610− addAuthor elt =
611− case filterChildren (named "author") elt of
612− [] -> return ()
613− [z] -> fromAuthor z >>= addMeta "author"
614− zs -> mapM fromAuthor zs >>= addMeta "author"
615− fromAuthor elt =
616− mconcat . intersperse space <$> mapM getInlines (elChildren elt)
617− addMetaField fieldname elt =
618− case filterChildren (named fieldname) elt of
619− [] -> return ()
620− [z] -> getInlines z >>= addMeta fieldname
621− zs -> mapM getInlines zs >>= addMeta fieldname
602+ where addMetaField fieldname elt =
603+ case filterChildren (named fieldname) elt of
604+ [] -> return ()
605+ [z] -> getInlines z >>= addMeta fieldname
606+ zs -> mapM getInlines zs >>= addMeta fieldname
622607
623608addMeta :: PandocMonad m => ToMetaValue a => Text -> a -> DB m ()
624609addMeta field val = modify (setMeta field val)
@@ -719,7 +704,7 @@ blockTags =
719704 ] ++ admonitionTags
720705
721706admonitionTags :: [Text]
722−admonitionTags = ["caution","danger","important","note","tip","warning"]
707+admonitionTags = ["important","caution","note","tip","warning"]
723708
724709-- Trim leading and trailing newline characters
725710trimNl :: Text -> Text
@@ -964,8 +949,9 @@ parseBlock (Elem e) =
964949 (x >= '0' && x <= '9')
965950 || x == '.') w
966951 if n > 0 then Just n else Nothing
967− let numrows = maybe 0 maximum $ nonEmpty
968− $ map length bodyrows
952+ let numrows = case bodyrows of
953+ [] -> 0
954+ xs -> maximum $ map length xs
969955 let aligns = case colspecs of
970956 [] -> replicate numrows AlignDefault
971957 cs -> map toAlignment cs
@@ -1089,7 +1075,7 @@ parseInline (Elem e) =
10891075 return $ spanWith (attrValue "id" e, [], []) mempty
10901076 "phrase" -> do
10911077 let ident = attrValue "id" e
1092− let classes = T.words $ attrValue "role" e
1078+ let classes = T.words $ attrValue "class" e
10931079 if ident /= "" || classes /= []
10941080 then innerInlines (spanWith (ident,classes,[]))
10951081 else innerInlines id
@@ -1253,8 +1239,7 @@ equation e constructor =
12531239 where
12541240 mathMLEquations :: [Text]
12551241 mathMLEquations = map writeTeX $ rights $ readMath
1256− (\x -> qName (elName x) == "math" &&
1257− qURI (elName x) == Just "http://www.w3.org/1998/Math/MathML")
1242+ (\x -> qName (elName x) == "math" && qPrefix (elName x) == Just "mml")
12581243 (readMathML . showElement)
12591244
12601245 latexEquations :: [Text]
src/Text/Pandoc/Readers/Docx.hs+42−59
@@ -61,14 +61,13 @@ module Text.Pandoc.Readers.Docx
6161import Codec.Archive.Zip
6262import Control.Monad.Reader
6363import Control.Monad.State.Strict
64−import Data.Bifunctor (bimap, first)
6564import qualified Data.ByteString.Lazy as B
6665import Data.Default (Default)
67−import Data.List (delete, intersect, foldl')
66+import Data.List (delete, intersect)
6867import Data.Char (isSpace)
6968import qualified Data.Map as M
7069import qualified Data.Text as T
71−import Data.Maybe (catMaybes, isJust, fromMaybe)
70+import Data.Maybe (isJust, fromMaybe)
7271import Data.Sequence (ViewL (..), viewl)
7372import qualified Data.Sequence as Seq
7473import qualified Data.Set as Set
@@ -86,7 +85,6 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
8685import qualified Text.Pandoc.Class.PandocMonad as P
8786import Text.Pandoc.Error
8887import Text.Pandoc.Logging
89−import Data.List.NonEmpty (nonEmpty)
9088
9189readDocx :: PandocMonad m
9290 => ReaderOptions
@@ -114,7 +112,6 @@ data DState = DState { docxAnchorMap :: M.Map T.Text T.Text
114112 -- restarting
115113 , docxListState :: M.Map (T.Text, T.Text) Integer
116114 , docxPrevPara :: Inlines
117− , docxTableCaptions :: [Blocks]
118115 }
119116
120117instance Default DState where
@@ -125,7 +122,6 @@ instance Default DState where
125122 , docxDropCap = mempty
126123 , docxListState = M.empty
127124 , docxPrevPara = mempty
128− , docxTableCaptions = []
129125 }
130126
131127data DEnv = DEnv { docxOptions :: ReaderOptions
@@ -494,32 +490,15 @@ singleParaToPlain blks
494490 singleton $ Plain ils
495491singleParaToPlain blks = blks
496492
497−cellToCell :: PandocMonad m => RowSpan -> Docx.Cell -> DocxContext m Pandoc.Cell
498−cellToCell rowSpan (Docx.Cell gridSpan _ bps) = do
493+cellToBlocks :: PandocMonad m => Docx.Cell -> DocxContext m Blocks
494+cellToBlocks (Docx.Cell bps) = do
499495 blks <- smushBlocks <$> mapM bodyPartToBlocks bps
500− let blks' = singleParaToPlain $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
501− return (cell AlignDefault rowSpan (ColSpan (fromIntegral gridSpan)) blks')
502−
503−rowsToRows :: PandocMonad m => [Docx.Row] -> DocxContext m [Pandoc.Row]
504−rowsToRows rows = do
505− let rowspans = (fmap . fmap) (first RowSpan) (Docx.rowsToRowspans rows)
506− cells <- traverse (traverse (uncurry cellToCell)) rowspans
507− return (fmap (Pandoc.Row nullAttr) cells)
508−
509−splitHeaderRows :: Bool -> [Docx.Row] -> ([Docx.Row], [Docx.Row])
510−splitHeaderRows hasFirstRowFormatting rs = bimap reverse reverse $ fst
511− $ if hasFirstRowFormatting
512− then foldl' f ((take 1 rs, []), True) (drop 1 rs)
513− else foldl' f (([], []), False) rs
514− where
515− f ((headerRows, bodyRows), previousRowWasHeader) r@(Docx.Row h cs)
516− | h == HasTblHeader || (previousRowWasHeader && any isContinuationCell cs)
517− = ((r : headerRows, bodyRows), True)
518− | otherwise
519− = ((headerRows, r : bodyRows), False)
520−
521− isContinuationCell (Docx.Cell _ vm _) = vm == Docx.Continue
496+ return $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
522497
498+rowToBlocksList :: PandocMonad m => Docx.Row -> DocxContext m [Blocks]
499+rowToBlocksList (Docx.Row cells) = do
500+ blksList <- mapM cellToBlocks cells
501+ return $ map singleParaToPlain blksList
523502
524503-- like trimInlines, but also take out linebreaks
525504trimSps :: Inlines -> Inlines
@@ -566,11 +545,6 @@ normalizeToClassName = T.map go . fromStyleName
566545 where go c | isSpace c = '-'
567546 | otherwise = c
568547
569−bodyPartToTableCaption :: PandocMonad m => BodyPart -> DocxContext m (Maybe Blocks)
570−bodyPartToTableCaption (TblCaption pPr parparts) =
571− Just <$> bodyPartToBlocks (Paragraph pPr parparts)
572−bodyPartToTableCaption _ = pure Nothing
573−
574548bodyPartToBlocks :: PandocMonad m => BodyPart -> DocxContext m Blocks
575549bodyPartToBlocks (Paragraph pPr parparts)
576550 | Just True <- pBidi pPr = do
@@ -662,43 +636,54 @@ bodyPartToBlocks (ListItem pPr _ _ _ parparts) =
662636 let pPr' = pPr {pStyle = constructBogusParStyleData "list-paragraph": pStyle pPr}
663637 in
664638 bodyPartToBlocks $ Paragraph pPr' parparts
665−bodyPartToBlocks (TblCaption _ _) =
666− return $ para mempty -- collected separately
667639bodyPartToBlocks (Tbl _ _ _ []) =
668640 return $ para mempty
669−bodyPartToBlocks (Tbl cap grid look parts) = do
670− captions <- gets docxTableCaptions
671− fullCaption <- case captions of
672− c : cs -> do
673− modify (\s -> s { docxTableCaptions = cs })
674− return c
675− [] -> return $ if T.null cap then mempty else plain (text cap)
676− let shortCaption = if T.null cap then Nothing else Just (toList (text cap))
677− cap' = caption shortCaption fullCaption
678− (hdr, rows) = splitHeaderRows (firstRowFormatting look) parts
641+bodyPartToBlocks (Tbl cap _ look parts@(r:rs)) = do
642+ let cap' = simpleCaption $ plain $ text cap
643+ (hdr, rows) = case firstRowFormatting look of
644+ True | null rs -> (Nothing, [r])
645+ | otherwise -> (Just r, rs)
646+ False -> (Nothing, r:rs)
647+
648+ cells <- mapM rowToBlocksList rows
679649
680650 let width = maybe 0 maximum $ nonEmpty $ map rowLength parts
651+ -- Data.List.NonEmpty is not available with ghc 7.10 so we roll out
652+ -- our own, see
653+ -- https://github.com/jgm/pandoc/pull/4361#issuecomment-365416155
654+ nonEmpty [] = Nothing
655+ nonEmpty l = Just l
681656 rowLength :: Docx.Row -> Int
682− rowLength (Docx.Row _ c) = sum (fmap (\(Docx.Cell gridSpan _ _) -> fromIntegral gridSpan) c)
657+ rowLength (Docx.Row c) = length c
683658
684− headerCells <- rowsToRows hdr
685− bodyCells <- rowsToRows rows
659+ let toRow = Pandoc.Row nullAttr . map simpleCell
660+ toHeaderRow l = [toRow l | not (null l)]
686661
687− -- Horizontal column alignment goes to the default at the moment. Getting
688− -- it might be difficult, since there doesn't seem to be a column entity
689− -- in docx.
662+ -- pad cells. New Text.Pandoc.Builder will do that for us,
663+ -- so this is for compatibility while we switch over.
664+ let cells' = map (\row -> toRow $ take width (row ++ repeat mempty)) cells
665+
666+ hdrCells <- case hdr of
667+ Just r' -> toHeaderRow <$> rowToBlocksList r'
668+ Nothing -> return []
669+
670+ -- The two following variables (horizontal column alignment and
671+ -- relative column widths) go to the default at the
672+ -- moment. Width information is in the TblGrid field of the Tbl,
673+ -- so should be possible. Alignment might be more difficult,
674+ -- since there doesn't seem to be a column entity in docx.
690675 let alignments = replicate width AlignDefault
691− totalWidth = sum grid
692− widths = (\w -> ColWidth (fromInteger w / fromInteger totalWidth)) <$> grid
676+ widths = replicate width ColWidthDefault
693677
694678 return $ table cap'
695679 (zip alignments widths)
696− (TableHead nullAttr headerCells)
697− [TableBody nullAttr 0 [] bodyCells]
680+ (TableHead nullAttr hdrCells)
681+ [TableBody nullAttr 0 [] cells']
698682 (TableFoot nullAttr [])
699683bodyPartToBlocks (OMathPara e) =
700684 return $ para $ displayMath (writeTeX e)
701685
686+
702687-- replace targets with generated anchors.
703688rewriteLink' :: PandocMonad m => Inline -> DocxContext m Inline
704689rewriteLink' l@(Link attr ils (T.uncons -> Just ('#',target), title)) = do
@@ -734,8 +719,6 @@ bodyToOutput :: PandocMonad m => Body -> DocxContext m (Meta, [Block])
734719bodyToOutput (Body bps) = do
735720 let (metabps, blkbps) = sepBodyParts bps
736721 meta <- bodyPartsToMeta metabps
737− captions <- catMaybes <$> mapM bodyPartToTableCaption blkbps
738− modify (\s -> s { docxTableCaptions = captions })
739722 blks <- smushBlocks <$> mapM bodyPartToBlocks blkbps
740723 blks' <- rewriteLinks $ blocksToDefinitions $ blocksToBullets $ toList blks
741724 blks'' <- removeOrphanAnchors blks'
src/Text/Pandoc/Readers/Docx/Combine.hs+5−5
@@ -61,7 +61,7 @@ import Data.List
6161import Data.Bifunctor
6262import Data.Sequence ( ViewL (..), ViewR (..), viewl, viewr, spanr, spanl
6363 , (><), (|>) )
64−import Text.Pandoc.Builder as B
64+import Text.Pandoc.Builder
6565
6666data Modifier a = Modifier (a -> a)
6767 | AttrModifier (Attr -> a -> a) Attr
@@ -116,12 +116,12 @@ ilModifierAndInnards ils = case viewl $ unMany ils of
116116
117117inlinesL :: Inlines -> (Inlines, Inlines)
118118inlinesL ils = case viewl $ unMany ils of
119− (s :< sq) -> (B.singleton s, Many sq)
119+ (s :< sq) -> (singleton s, Many sq)
120120 _ -> (mempty, ils)
121121
122122inlinesR :: Inlines -> (Inlines, Inlines)
123123inlinesR ils = case viewr $ unMany ils of
124− (sq :> s) -> (Many sq, B.singleton s)
124+ (sq :> s) -> (Many sq, singleton s)
125125 _ -> (ils, mempty)
126126
127127combineInlines :: Inlines -> Inlines -> Inlines
@@ -182,7 +182,7 @@ isAttrModifier _ = False
182182
183183smushInlines :: [Inlines] -> Inlines
184184smushInlines xs = combineInlines xs' mempty
185− where xs' = foldl' combineInlines mempty xs
185+ where xs' = foldl combineInlines mempty xs
186186
187187smushBlocks :: [Blocks] -> Blocks
188−smushBlocks xs = foldl' combineBlocks mempty xs
188+smushBlocks xs = foldl combineBlocks mempty xs
src/Text/Pandoc/Readers/Docx/Parse.hs+21−121
@@ -33,9 +33,7 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
3333 , ParStyle
3434 , CharStyle(cStyleData)
3535 , Row(..)
36− , TblHeader(..)
3736 , Cell(..)
38− , VMerge(..)
3937 , TrackedChange(..)
4038 , ChangeType(..)
4139 , ChangeInfo(..)
@@ -52,7 +50,6 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
5250 , pHeading
5351 , constructBogusParStyleData
5452 , leftBiasedMergeRunStyle
55− , rowsToRowspans
5653 ) where
5754import Text.Pandoc.Readers.Docx.Parse.Styles
5855import Codec.Archive.Zip
@@ -228,7 +225,6 @@ defaultParagraphStyle = ParagraphStyle { pStyle = []
228225data BodyPart = Paragraph ParagraphStyle [ParPart]
229226 | ListItem ParagraphStyle T.Text T.Text (Maybe Level) [ParPart]
230227 | Tbl T.Text TblGrid TblLook [Row]
231− | TblCaption ParagraphStyle [ParPart]
232228 | OMathPara [Exp]
233229 deriving Show
234230
@@ -240,61 +236,12 @@ newtype TblLook = TblLook {firstRowFormatting::Bool}
240236defaultTblLook :: TblLook
241237defaultTblLook = TblLook{firstRowFormatting = False}
242238
243−data Row = Row TblHeader [Cell] deriving Show
244−
245−data TblHeader = HasTblHeader | NoTblHeader deriving (Show, Eq)
239+newtype Row = Row [Cell]
240+ deriving Show
246241
247−data Cell = Cell GridSpan VMerge [BodyPart]
242+newtype Cell = Cell [BodyPart]
248243 deriving Show
249244
250−type GridSpan = Integer
251−
252−data VMerge = Continue
253− -- ^ This cell should be merged with the one above it
254− | Restart
255− -- ^ This cell should not be merged with the one above it
256− deriving (Show, Eq)
257−
258−rowsToRowspans :: [Row] -> [[(Int, Cell)]]
259−rowsToRowspans rows = let
260− removeMergedCells = fmap (filter (\(_, Cell _ vmerge _) -> vmerge == Restart))
261− in removeMergedCells (foldr f [] rows)
262− where
263− f :: Row -> [[(Int, Cell)]] -> [[(Int, Cell)]]
264− f (Row _ cells) acc = let
265− spans = g cells Nothing (listToMaybe acc)
266− in spans : acc
267−
268− g ::
269− -- | The current row
270− [Cell] ->
271− -- | Number of columns left below
272− Maybe Integer ->
273− -- | (rowspan so far, cell) for the row below this one
274− Maybe [(Int, Cell)] ->
275− -- | (rowspan so far, cell) for this row
276− [(Int, Cell)]
277− g cells _ Nothing = zip (repeat 1) cells
278− g cells columnsLeftBelow (Just rowBelow) =
279− case cells of
280− [] -> []
281− thisCell@(Cell thisGridSpan _ _) : restOfRow -> case rowBelow of
282− [] -> zip (repeat 1) cells
283− (spanSoFarBelow, Cell gridSpanBelow vmerge _) : _ ->
284− let spanSoFar = case vmerge of
285− Restart -> 1
286− Continue -> 1 + spanSoFarBelow
287− columnsToDrop = thisGridSpan + (gridSpanBelow - fromMaybe gridSpanBelow columnsLeftBelow)
288− (newColumnsLeftBelow, restOfRowBelow) = dropColumns columnsToDrop rowBelow
289− in (spanSoFar, thisCell) : g restOfRow (Just newColumnsLeftBelow) (Just restOfRowBelow)
290−
291− dropColumns :: Integer -> [(a, Cell)] -> (Integer, [(a, Cell)])
292− dropColumns n [] = (n, [])
293− dropColumns n cells@((_, Cell gridSpan _ _) : otherCells) =
294− if n < gridSpan
295− then (gridSpan - n, cells)
296− else dropColumns (n - gridSpan) otherCells
297−
298245leftBiasedMergeRunStyle :: RunStyle -> RunStyle -> RunStyle
299246leftBiasedMergeRunStyle a b = RunStyle
300247 { isBold = isBold a <|> isBold b
@@ -479,26 +426,20 @@ filePathToRelType path docXmlPath =
479426 then Just InDocument
480427 else Nothing
481428
482−relElemToRelationship :: FilePath -> DocumentLocation -> Element
483− -> Maybe Relationship
484−relElemToRelationship fp relType element | qName (elName element) == "Relationship" =
429+relElemToRelationship :: DocumentLocation -> Element -> Maybe Relationship
430+relElemToRelationship relType element | qName (elName element) == "Relationship" =
485431 do
486432 relId <- findAttr (QName "Id" Nothing Nothing) element
487433 target <- findAttr (QName "Target" Nothing Nothing) element
488− -- target may be relative (media/image1.jpeg) or absolute
489− -- (/word/media/image1.jpeg); we need to relativize it (see #7374)
490− let frontOfFp = T.pack $ takeWhile (/= '_') fp
491− let target' = fromMaybe target $
492− T.stripPrefix frontOfFp $ T.dropWhile (== '/') target
493− return $ Relationship relType relId target'
494−relElemToRelationship _ _ _ = Nothing
434+ return $ Relationship relType relId target
435+relElemToRelationship _ _ = Nothing
495436
496437filePathToRelationships :: Archive -> FilePath -> FilePath -> [Relationship]
497438filePathToRelationships ar docXmlPath fp
498439 | Just relType <- filePathToRelType fp docXmlPath
499440 , Just entry <- findEntryByPath fp ar
500441 , Just relElems <- parseXMLFromEntry entry =
501− mapMaybe (relElemToRelationship fp relType) $ elChildren relElems
442+ mapMaybe (relElemToRelationship relType) $ elChildren relElems
502443filePathToRelationships _ _ _ = []
503444
504445archiveToRelationships :: Archive -> FilePath -> [Relationship]
@@ -622,7 +563,7 @@ elemToTblGrid :: NameSpaces -> Element -> D TblGrid
622563elemToTblGrid ns element | isElem ns "w" "tblGrid" element =
623564 let cols = findChildrenByName ns "w" "gridCol" element
624565 in
625− mapD (\e -> maybeToD (findAttrByName ns "w" "w" e >>= stringToInteger))
566+ mapD (\e -> maybeToD (findAttrByName ns "w" "val" e >>= stringToInteger))
626567 cols
627568elemToTblGrid _ _ = throwError WrongElem
628569
@@ -646,31 +587,14 @@ elemToRow ns element | isElem ns "w" "tr" element =
646587 do
647588 let cellElems = findChildrenByName ns "w" "tc" element
648589 cells <- mapD (elemToCell ns) cellElems
649− let hasTblHeader = maybe NoTblHeader (const HasTblHeader)
650− (findChildByName ns "w" "trPr" element
651− >>= findChildByName ns "w" "tblHeader")
652− return $ Row hasTblHeader cells
590+ return $ Row cells
653591elemToRow _ _ = throwError WrongElem
654592
655593elemToCell :: NameSpaces -> Element -> D Cell
656594elemToCell ns element | isElem ns "w" "tc" element =
657595 do
658− let properties = findChildByName ns "w" "tcPr" element
659− let gridSpan = properties
660− >>= findChildByName ns "w" "gridSpan"
661− >>= findAttrByName ns "w" "val"
662− >>= stringToInteger
663− let vMerge = case properties >>= findChildByName ns "w" "vMerge" of
664− Nothing -> Restart
665− Just e ->
666− fromMaybe Continue $ do
667− s <- findAttrByName ns "w" "val" e
668− case s of
669− "continue" -> Just Continue
670− "restart" -> Just Restart
671− _ -> Nothing
672596 cellContents <- mapD (elemToBodyPart ns) (elChildren element)
673− return $ Cell (fromMaybe 1 gridSpan) vMerge cellContents
597+ return $ Cell cellContents
674598elemToCell _ _ = throwError WrongElem
675599
676600elemToParIndentation :: NameSpaces -> Element -> Maybe ParIndentation
@@ -702,9 +626,10 @@ pNumInfo = getParStyleField numInfo . pStyle
702626elemToBodyPart :: NameSpaces -> Element -> D BodyPart
703627elemToBodyPart ns element
704628 | isElem ns "w" "p" element
705− , (c:_) <- findChildrenByName ns "m" "oMathPara" element = do
706− expsLst <- eitherToD $ readOMML $ showElement c
707− return $ OMathPara expsLst
629+ , (c:_) <- findChildrenByName ns "m" "oMathPara" element =
630+ do
631+ expsLst <- eitherToD $ readOMML $ showElement c
632+ return $ OMathPara expsLst
708633elemToBodyPart ns element
709634 | isElem ns "w" "p" element
710635 , Just (numId, lvl) <- getNumInfo ns element = do
@@ -722,31 +647,13 @@ elemToBodyPart ns element
722647 Nothing | Just (numId, lvl) <- pNumInfo parstyle -> do
723648 levelInfo <- lookupLevel numId lvl <$> asks envNumbering
724649 return $ ListItem parstyle numId lvl levelInfo parparts
725− _ -> let
726− hasCaptionStyle = elem "Caption" (pStyleId <$> pStyle parstyle)
727−
728− hasSimpleTableField = fromMaybe False $ do
729− fldSimple <- findChildByName ns "w" "fldSimple" element
730− instr <- findAttrByName ns "w" "instr" fldSimple
731− pure ("Table" `elem` T.words instr)
732−
733− hasComplexTableField = fromMaybe False $ do
734− instrText <- findElementByName ns "w" "instrText" element
735− pure ("Table" `elem` T.words (strContent instrText))
736−
737− in if hasCaptionStyle && (hasSimpleTableField || hasComplexTableField)
738− then return $ TblCaption parstyle parparts
739− else return $ Paragraph parstyle parparts
740−
650+ _ -> return $ Paragraph parstyle parparts
741651elemToBodyPart ns element
742652 | isElem ns "w" "tbl" element = do
743− let tblProperties = findChildByName ns "w" "tblPr" element
744− caption = fromMaybe "" $ tblProperties
653+ let caption' = findChildByName ns "w" "tblPr" element
745654 >>= findChildByName ns "w" "tblCaption"
746655 >>= findAttrByName ns "w" "val"
747− description = fromMaybe "" $ tblProperties
748− >>= findChildByName ns "w" "tblDescription"
749− >>= findAttrByName ns "w" "val"
656+ caption = fromMaybe "" caption'
750657 grid' = case findChildByName ns "w" "tblGrid" element of
751658 Just g -> elemToTblGrid ns g
752659 Nothing -> return []
@@ -759,7 +666,7 @@ elemToBodyPart ns element
759666 grid <- grid'
760667 tblLook <- tblLook'
761668 rows <- mapD (elemToRow ns) (elChildren element)
762− return $ Tbl (caption <> description) grid tblLook rows
669+ return $ Tbl caption grid tblLook rows
763670elemToBodyPart _ _ = throwError WrongElem
764671
765672lookupRelationship :: DocumentLocation -> RelId -> [Relationship] -> Maybe Target
@@ -802,8 +709,7 @@ elemToParPart ns element
802709 case drawing of
803710 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp title alt bs $ elemToExtent drawingElem)
804711 Nothing -> throwError WrongElem
805−-- The two cases below are an attempt to deal with images in deprecated vml format.
806−-- Todo: check out title and attr for deprecated format.
712+-- The below is an attempt to deal with images in deprecated vml format.
807713elemToParPart ns element
808714 | isElem ns "w" "r" element
809715 , Just _ <- findChildByName ns "w" "pict" element =
@@ -811,15 +717,9 @@ elemToParPart ns element
811717 >>= findAttrByName ns "r" "id"
812718 in
813719 case drawing of
720+ -- Todo: check out title and attr for deprecated format.
814721 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
815722 Nothing -> throwError WrongElem
816−elemToParPart ns element
817− | isElem ns "w" "r" element
818− , Just objectElem <- findChildByName ns "w" "object" element
819− , Just shapeElem <- findChildByName ns "v" "shape" objectElem
820− , Just imagedataElem <- findChildByName ns "v" "imagedata" shapeElem
821− , Just drawingId <- findAttrByName ns "r" "id" imagedataElem
822− = expandDrawingId drawingId >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
823723-- Chart
824724elemToParPart ns element
825725 | isElem ns "w" "r" element
src/Text/Pandoc/Readers/Docx/Util.hs−7
@@ -19,7 +19,6 @@ module Text.Pandoc.Readers.Docx.Util (
1919 , elemToNameSpaces
2020 , findChildByName
2121 , findChildrenByName
22− , findElementByName
2322 , findAttrByName
2423 ) where
2524
@@ -57,12 +56,6 @@ findChildrenByName ns pref name el =
5756 let ns' = ns <> elemToNameSpaces el
5857 in findChildren (elemName ns' pref name) el
5958
60−-- | Like 'findChildrenByName', but searches descendants.
61−findElementByName :: NameSpaces -> Text -> Text -> Element -> Maybe Element
62−findElementByName ns pref name el =
63− let ns' = ns <> elemToNameSpaces el
64− in findElement (elemName ns' pref name) el
65−
6659findAttrByName :: NameSpaces -> Text -> Text -> Element -> Maybe Text
6760findAttrByName ns pref name el =
6861 let ns' = ns <> elemToNameSpaces el
src/Text/Pandoc/Readers/DokuWiki.hs+8−9
@@ -29,27 +29,26 @@ import Text.Pandoc.Definition
2929import Text.Pandoc.Error (PandocError (PandocParsecError))
3030import Text.Pandoc.Options
3131import Text.Pandoc.Parsing hiding (enclosed, nested)
32−import Text.Pandoc.Shared (trim, stringify, tshow)
32+import Text.Pandoc.Shared (crFilter, trim, stringify, tshow)
3333
3434-- | Read DokuWiki from an input string and return a Pandoc document.
35−readDokuWiki :: (PandocMonad m, ToSources a)
35+readDokuWiki :: PandocMonad m
3636 => ReaderOptions
37− -> a
37+ -> Text
3838 -> m Pandoc
3939readDokuWiki opts s = do
40− let sources = toSources s
41− res <- runParserT parseDokuWiki def {stateOptions = opts }
42− (initialSourceName sources) sources
40+ let input = crFilter s
41+ res <- runParserT parseDokuWiki def {stateOptions = opts } "source" input
4342 case res of
44− Left e -> throwError $ PandocParsecError sources e
43+ Left e -> throwError $ PandocParsecError input e
4544 Right d -> return d
4645
47−type DWParser = ParserT Sources ParserState
46+type DWParser = ParserT Text ParserState
4847
4948-- * Utility functions
5049
5150-- | Parse end-of-line, which can be either a newline or end-of-file.
52−eol :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
51+eol :: Stream s m Char => ParserT s st m ()
5352eol = void newline <|> eof
5453
5554nested :: PandocMonad m => DWParser m a -> DWParser m a
src/Text/Pandoc/Readers/FB2.hs+3−6
@@ -40,9 +40,9 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad, insertMedia, report)
4040import Text.Pandoc.Error
4141import Text.Pandoc.Logging
4242import Text.Pandoc.Options
43+import Text.Pandoc.Shared (crFilter)
4344import Text.Pandoc.XML.Light
4445import qualified Text.Pandoc.UTF8 as UTF8
45−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
4646
4747type FB2 m = StateT FB2State m
4848
@@ -63,12 +63,9 @@ instance HasMeta FB2State where
6363 setMeta field v s = s {fb2Meta = setMeta field v (fb2Meta s)}
6464 deleteMeta field s = s {fb2Meta = deleteMeta field (fb2Meta s)}
6565
66−readFB2 :: (PandocMonad m, ToSources a)
67− => ReaderOptions
68− -> a
69− -> m Pandoc
66+readFB2 :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
7067readFB2 _ inp =
71− case parseXMLElement $ TL.fromStrict $ sourcesToText $ toSources inp of
68+ case parseXMLElement $ TL.fromStrict $ crFilter inp of
7269 Left msg -> throwError $ PandocXMLError "" msg
7370 Right el -> do
7471 (bs, st) <- runStateT (parseRootElement el) def
src/Text/Pandoc/Readers/HTML.hs+28−48
@@ -32,7 +32,6 @@ import Data.Char (isAlphaNum, isLetter)
3232import Data.Default (Default (..), def)
3333import Data.Foldable (for_)
3434import Data.List.Split (splitWhen)
35−import Data.List (foldl')
3635import qualified Data.Map as M
3736import Data.Maybe (fromMaybe, isJust, isNothing)
3837import Data.Monoid (First (..))
@@ -62,21 +61,21 @@ import Text.Pandoc.Options (
6261 extensionEnabled)
6362import Text.Pandoc.Parsing hiding ((<|>))
6463import Text.Pandoc.Shared (
65− addMetaField, blocksToInlines', escapeURI, extractSpaces,
64+ addMetaField, blocksToInlines', crFilter, escapeURI, extractSpaces,
6665 htmlSpanLikeElements, renderTags', safeRead, tshow)
6766import Text.Pandoc.Walk
6867import Text.Parsec.Error
6968import Text.TeXMath (readMathML, writeTeX)
7069
7170-- | Convert HTML-formatted string to 'Pandoc' document.
72−readHtml :: (PandocMonad m, ToSources a)
71+readHtml :: PandocMonad m
7372 => ReaderOptions -- ^ Reader options
74− -> a -- ^ Input to parse
73+ -> Text -- ^ String to parse (assumes @'\n'@ line endings)
7574 -> m Pandoc
7675readHtml opts inp = do
7776 let tags = stripPrefixes $ canonicalizeTags $
7877 parseTagsOptions parseOptions{ optTagPosition = True }
79− (sourcesToText $ toSources inp)
78+ (crFilter inp)
8079 parseDoc = do
8180 blocks <- fixPlains False . mconcat <$> manyTill block eof
8281 meta <- stateMeta . parserState <$> getState
@@ -205,7 +204,6 @@ block = ((do
205204 | otherwise
206205 -> pDiv
207206 "section" -> pDiv
208− "header" -> pDiv
209207 "main" -> pDiv
210208 "figure" -> pFigure
211209 "iframe" -> pIframe
@@ -362,10 +360,7 @@ pDefListItem = try $ do
362360 terms <- many1 (try $ skipMany nonItem >> pInTags "dt" inline)
363361 defs <- many1 (try $ skipMany nonItem >> pInTags "dd" block)
364362 skipMany nonItem
365− let term = foldl' (\x y -> if null x
366− then trimInlines y
367− else x <> B.linebreak <> trimInlines y)
368− mempty terms
363+ let term = foldl1 (\x y -> x <> B.linebreak <> y) $ map trimInlines terms
369364 return (term, map (fixPlains True) defs)
370365
371366fixPlains :: Bool -> Blocks -> Blocks
@@ -405,7 +400,6 @@ pLineBlock = try $ do
405400isDivLike :: Text -> Bool
406401isDivLike "div" = True
407402isDivLike "section" = True
408−isDivLike "header" = True
409403isDivLike "main" = True
410404isDivLike _ = False
411405
@@ -501,13 +495,17 @@ pHeader = try $ do
501495 tagOpen (`elem` ["h1","h2","h3","h4","h5","h6"])
502496 (const True)
503497 let attr = toStringAttr attr'
498+ let bodyTitle = TagOpen tagtype attr' ~== TagOpen ("h1" :: Text)
499+ [("class","title")]
504500 level <- headerLevel tagtype
505501 contents <- trimInlines . mconcat <$> manyTill inline (pCloses tagtype <|> eof)
506502 let ident = fromMaybe "" $ lookup "id" attr
507503 let classes = maybe [] T.words $ lookup "class" attr
508504 let keyvals = [(k,v) | (k,v) <- attr, k /= "class", k /= "id"]
509505 attr'' <- registerHeader (ident, classes, keyvals) contents
510− return $ B.headerWith attr'' level contents
506+ return $ if bodyTitle
507+ then mempty -- skip a representation of the title in the body
508+ else B.headerWith attr'' level contents
511509
512510pHrule :: PandocMonad m => TagParser m Blocks
513511pHrule = do
@@ -557,18 +555,7 @@ pFigure = try $ do
557555pCodeBlock :: PandocMonad m => TagParser m Blocks
558556pCodeBlock = try $ do
559557 TagOpen _ attr' <- pSatisfy (matchTagOpen "pre" [])
560− -- if the `pre` has no attributes, try if it is followed by a `code`
561− -- element and use those attributes if possible.
562− attr <- case attr' of
563− _:_ -> pure (toAttr attr')
564− [] -> option nullAttr $ do
565− TagOpen _ codeAttr <- pSatisfy (matchTagOpen "code" [])
566− pure $ toAttr
567− [ (k, v') | (k, v) <- codeAttr
568− -- strip language from class
569− , let v' = if k == "class"
570− then fromMaybe v (T.stripPrefix "language-" v)
571− else v ]
558+ let attr = toAttr attr'
572559 contents <- manyTill pAny (pCloses "pre" <|> eof)
573560 let rawText = T.concat $ map tagToText contents
574561 -- drop leading newline if any
@@ -839,19 +826,17 @@ pInlinesInTags tagtype f = extractSpaces f <$> pInTags tagtype inline
839826
840827pTagText :: PandocMonad m => TagParser m Inlines
841828pTagText = try $ do
842− pos <- getPosition
843829 (TagText str) <- pSatisfy isTagText
844830 st <- getState
845831 qu <- ask
846832 parsed <- lift $ lift $
847− flip runReaderT qu $ runParserT (many pTagContents) st "text"
848− (Sources [(pos, str)])
833+ flip runReaderT qu $ runParserT (many pTagContents) st "text" str
849834 case parsed of
850835 Left _ -> throwError $ PandocParseError $
851836 "Could not parse `" <> str <> "'"
852837 Right result -> return $ mconcat result
853838
854−type InlinesParser m = HTMLParser m Sources
839+type InlinesParser m = HTMLParser m Text
855840
856841pTagContents :: PandocMonad m => InlinesParser m Inlines
857842pTagContents =
@@ -951,15 +936,13 @@ getTagName (TagClose t) = Just t
951936getTagName _ = Nothing
952937
953938isInlineTag :: Tag Text -> Bool
954−isInlineTag t = isCommentTag t || case t of
955− TagOpen "script" _ -> "math/tex" `T.isPrefixOf` fromAttrib "type" t
956− TagClose "script" -> True
957− TagOpen name _ -> isInlineTagName name
958− TagClose name -> isInlineTagName name
959− _ -> False
960− where isInlineTagName x =
961− x `Set.notMember` blockTags ||
962− T.take 1 x == "?" -- processing instr.
939+isInlineTag t =
940+ isCommentTag t ||
941+ case getTagName t of
942+ Nothing -> False
943+ Just "script" -> "math/tex" `T.isPrefixOf` fromAttrib "type" t
944+ Just x -> x `Set.notMember` blockTags ||
945+ T.take 1 x == "?" -- processing instr.
963946
964947isBlockTag :: Tag Text -> Bool
965948isBlockTag t = isBlockTagName || isTagComment t
@@ -983,14 +966,13 @@ isCommentTag = tagComment (const True)
983966-- | Matches a stretch of HTML in balanced tags.
984967htmlInBalanced :: Monad m
985968 => (Tag Text -> Bool)
986− -> ParserT Sources st m Text
969+ -> ParserT Text st m Text
987970htmlInBalanced f = try $ do
988971 lookAhead (char '<')
989− sources <- getInput
990− let ts = canonicalizeTags
991− $ parseTagsOptions parseOptions{ optTagWarning = True,
992− optTagPosition = True }
993− $ sourcesToText sources
972+ inp <- getInput
973+ let ts = canonicalizeTags $
974+ parseTagsOptions parseOptions{ optTagWarning = True,
975+ optTagPosition = True } inp
994976 case ts of
995977 (TagPosition sr sc : t@(TagOpen tn _) : rest) -> do
996978 guard $ f t
@@ -1032,17 +1014,15 @@ hasTagWarning _ = False
10321014-- | Matches a tag meeting a certain condition.
10331015htmlTag :: (HasReaderOptions st, Monad m)
10341016 => (Tag Text -> Bool)
1035− -> ParserT Sources st m (Tag Text, Text)
1017+ -> ParserT Text st m (Tag Text, Text)
10361018htmlTag f = try $ do
10371019 lookAhead (char '<')
10381020 startpos <- getPosition
1039− sources <- getInput
1040− let inp = sourcesToText sources
1021+ inp <- getInput
10411022 let ts = canonicalizeTags $ parseTagsOptions
10421023 parseOptions{ optTagWarning = False
10431024 , optTagPosition = True }
1044− (inp <> " ")
1045− -- add space to ensure that
1025+ (inp <> " ") -- add space to ensure that
10461026 -- we get a TagPosition after the tag
10471027 (next, ln, col) <- case ts of
10481028 (TagPosition{} : next : TagPosition ln col : _)
src/Text/Pandoc/Readers/HTML/Table.hs+15−35
@@ -17,8 +17,6 @@ module Text.Pandoc.Readers.HTML.Table (pTable) where
1717
1818import Control.Applicative ((<|>))
1919import Data.Maybe (fromMaybe)
20−import Data.Either (lefts, rights)
21−import Data.List.NonEmpty (nonEmpty)
2220import Data.Text (Text)
2321import Text.HTML.TagSoup
2422import Text.Pandoc.Builder (Blocks)
@@ -34,51 +32,34 @@ import Text.Pandoc.Shared (onlySimpleTableCells, safeRead)
3432import qualified Data.Text as T
3533import qualified Text.Pandoc.Builder as B
3634
37−-- | Parses a @<col>@ element, returning the column's width.
38−-- An Either value is used: Left i means a "relative length" with
39−-- integral value i (see https://www.w3.org/TR/html4/types.html#h-6.6);
40−-- Right w means a regular width. Defaults to @'Right ColWidthDefault'@
41−-- if the width is not set or cannot be determined.
42−pCol :: PandocMonad m => TagParser m (Either Int ColWidth)
35+-- | Parses a @<col>@ element, returning the column's width. Defaults to
36+-- @'ColWidthDefault'@ if the width is not set or cannot be determined.
37+pCol :: PandocMonad m => TagParser m ColWidth
4338pCol = try $ do
4439 TagOpen _ attribs' <- pSatisfy (matchTagOpen "col" [])
4540 let attribs = toStringAttr attribs'
4641 skipMany pBlank
4742 optional $ pSatisfy (matchTagClose "col")
4843 skipMany pBlank
49− return $ case lookup "width" attribs of
44+ let width = case lookup "width" attribs of
5045 Nothing -> case lookup "style" attribs of
5146 Just (T.stripPrefix "width:" -> Just xs) | T.any (== '%') xs ->
52− maybe (Right ColWidthDefault) (Right . ColWidth . (/ 100.0))
53− $ safeRead (T.filter
54− (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
55− _ -> Right ColWidthDefault
56− Just (T.unsnoc -> Just (xs, '*')) ->
57− maybe (Left 1) Left $ safeRead xs
47+ fromMaybe 0.0 $ safeRead (T.filter
48+ (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
49+ _ -> 0.0
5850 Just (T.unsnoc -> Just (xs, '%')) ->
59− maybe (Right ColWidthDefault)
60− (Right . ColWidth . (/ 100.0)) $ safeRead xs
61− _ -> Right ColWidthDefault
51+ fromMaybe 0.0 $ safeRead xs
52+ _ -> 0.0
53+ if width > 0.0
54+ then return $ ColWidth $ width / 100.0
55+ else return ColWidthDefault
6256
63−pColgroup :: PandocMonad m => TagParser m [Either Int ColWidth]
57+pColgroup :: PandocMonad m => TagParser m [ColWidth]
6458pColgroup = try $ do
6559 pSatisfy (matchTagOpen "colgroup" [])
6660 skipMany pBlank
6761 manyTill pCol (pCloses "colgroup" <|> eof) <* skipMany pBlank
6862
69−resolveRelativeLengths :: [Either Int ColWidth] -> [ColWidth]
70−resolveRelativeLengths ws =
71− let remaining = 1 - sum (map getColWidth $ rights ws)
72− relatives = sum $ lefts ws
73− relUnit = remaining / fromIntegral relatives
74− toColWidth (Right x) = x
75− toColWidth (Left i) = ColWidth (fromIntegral i * relUnit)
76− in map toColWidth ws
77−
78−getColWidth :: ColWidth -> Double
79−getColWidth ColWidthDefault = 0
80−getColWidth (ColWidth w) = w
81−
8263data CellType
8364 = HeaderCell
8465 | BodyCell
@@ -200,8 +181,7 @@ pTable :: PandocMonad m
200181pTable block = try $ do
201182 TagOpen _ attribs <- pSatisfy (matchTagOpen "table" []) <* skipMany pBlank
202183 caption <- option mempty $ pInTags "caption" block <* skipMany pBlank
203− widths <- resolveRelativeLengths <$>
204− ((mconcat <$> many1 pColgroup) <|> many pCol) <* skipMany pBlank
184+ widths <- ((mconcat <$> many1 pColgroup) <|> many pCol) <* skipMany pBlank
205185 thead <- pTableHead block <* skipMany pBlank
206186 topfoot <- optionMaybe (pTableFoot block) <* skipMany pBlank
207187 tbodies <- many (pTableBody block) <* skipMany pBlank
@@ -236,7 +216,7 @@ normalize widths head' bodies foot = do
236216 let rows = headRows head' <> concatMap bodyRows bodies <> footRows foot
237217 let cellWidth (Cell _ _ _ (ColSpan cs) _) = cs
238218 let rowLength = foldr (\cell acc -> cellWidth cell + acc) 0 . rowCells
239− let ncols = maybe 0 maximum $ nonEmpty $ map rowLength rows
219+ let ncols = maximum (map rowLength rows)
240220 let tblType = tableType (map rowCells rows)
241221 -- fail on empty table
242222 if null rows
src/Text/Pandoc/Readers/Haddock.hs+7−11
@@ -17,9 +17,8 @@ module Text.Pandoc.Readers.Haddock
1717
1818import Control.Monad.Except (throwError)
1919import Data.List (intersperse)
20−import Data.List.NonEmpty (nonEmpty)
2120import Data.Maybe (fromMaybe)
22−import Data.Text (unpack)
21+import Data.Text (Text, unpack)
2322import qualified Data.Text as T
2423import Documentation.Haddock.Parser
2524import Documentation.Haddock.Types as H
@@ -29,17 +28,15 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
2928import Text.Pandoc.Definition
3029import Text.Pandoc.Error
3130import Text.Pandoc.Options
32−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
33−import Text.Pandoc.Shared (splitTextBy, trim)
31+import Text.Pandoc.Shared (crFilter, splitTextBy, trim)
3432
3533
3634-- | Parse Haddock markup and return a 'Pandoc' document.
37−readHaddock :: (PandocMonad m, ToSources a)
35+readHaddock :: PandocMonad m
3836 => ReaderOptions
39− -> a
37+ -> Text
4038 -> m Pandoc
41−readHaddock opts s = case readHaddockEither opts
42− (unpack . sourcesToText . toSources $ s) of
39+readHaddock opts s = case readHaddockEither opts (unpack (crFilter s)) of
4340 Right result -> return result
4441 Left e -> throwError e
4542
@@ -95,7 +92,7 @@ docHToBlocks d' =
9592 then ([], map toCells bodyRows)
9693 else (toCells (head headerRows),
9794 map toCells (tail headerRows ++ bodyRows))
98− colspecs = replicate (maybe 0 maximum (nonEmpty (map length body)))
95+ colspecs = replicate (maximum (map length body))
9996 (AlignDefault, ColWidthDefault)
10097 in B.table B.emptyCaption
10198 colspecs
@@ -131,8 +128,7 @@ docHToInlines isCode d' =
131128 DocIdentifier s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
132129 _ -> mempty
133130 DocIdentifierUnchecked s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
134− DocModule s -> B.codeWith ("",["haskell","module"],[]) $
135− T.pack (modLinkName s)
131+ DocModule s -> B.codeWith ("",["haskell","module"],[]) $ T.pack s
136132 DocWarning _ -> mempty -- TODO
137133 DocEmphasis d -> B.emph (docHToInlines isCode d)
138134 DocMonospaced (DocString s) -> B.code $ T.pack s
src/Text/Pandoc/Readers/Ipynb.hs+3−5
@@ -39,12 +39,10 @@ import Data.Aeson as Aeson
3939import Control.Monad.Except (throwError)
4040import Text.Pandoc.Readers.Markdown (readMarkdown)
4141import qualified Text.Pandoc.UTF8 as UTF8
42−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
4342
44−readIpynb :: (PandocMonad m, ToSources a)
45− => ReaderOptions -> a -> m Pandoc
46−readIpynb opts x = do
47− let src = BL.fromStrict . TE.encodeUtf8 . sourcesToText $ toSources x
43+readIpynb :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
44+readIpynb opts t = do
45+ let src = BL.fromStrict (TE.encodeUtf8 t)
4846 case eitherDecode src of
4947 Right (notebook4 :: Notebook NbV4) -> notebookToPandoc opts notebook4
5048 Left _ ->
src/Text/Pandoc/Readers/JATS.hs+6−13
@@ -29,12 +29,11 @@ import Text.HTML.TagSoup.Entity (lookupEntity)
2929import Text.Pandoc.Builder
3030import Text.Pandoc.Class.PandocMonad (PandocMonad)
3131import Text.Pandoc.Options
32−import Text.Pandoc.Shared (safeRead, extractSpaces)
32+import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
3333import Text.TeXMath (readMathML, writeTeX)
3434import Text.Pandoc.XML.Light
3535import qualified Data.Set as S (fromList, member)
3636import Data.Set ((\\))
37−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3837
3938type JATS m = StateT JATSState m
4039
@@ -53,14 +52,10 @@ instance Default JATSState where
5352 , jatsContent = [] }
5453
5554
56−readJATS :: (PandocMonad m, ToSources a)
57− => ReaderOptions
58− -> a
59− -> m Pandoc
55+readJATS :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
6056readJATS _ inp = do
61− let sources = toSources inp
6257 tree <- either (throwError . PandocXMLError "") return $
63− parseXMLContents (TL.fromStrict . sourcesToText $ sources)
58+ parseXMLContents (TL.fromStrict $ crFilter inp)
6459 (bs, st') <- flip runStateT (def{ jatsContent = tree }) $ mapM parseBlock tree
6560 return $ Pandoc (jatsMeta st') (toList . mconcat $ bs)
6661
@@ -496,8 +491,7 @@ parseInline (Elem e) =
496491
497492 "disp-formula" -> formula displayMath
498493 "inline-formula" -> formula math
499− "math" | qURI (elName e) == Just "http://www.w3.org/1998/Math/MathML"
500− -> return . math $ mathML e
494+ "math" | qPrefix (elName e) == Just "mml" -> return . math $ mathML e
501495 "tex-math" -> return . math $ textContent e
502496
503497 "email" -> return $ link ("mailto:" <> textContent e) ""
@@ -520,9 +514,8 @@ parseInline (Elem e) =
520514 filterChildren isMathML whereToLook
521515 return . mconcat . take 1 . map constructor $ texMaths ++ mathMLs
522516
523− isMathML x = qName (elName x) == "math" &&
524− qURI (elName x) ==
525− Just "http://www.w3.org/1998/Math/MathML"
517+ isMathML x = qName (elName x) == "math" &&
518+ qPrefix (elName x) == Just "mml"
526519 removePrefix elname = elname { qPrefix = Nothing }
527520 codeWithLang = do
528521 let classes' = case attrValue "language" e of
src/Text/Pandoc/Readers/Jira.hs+9−13
@@ -20,20 +20,18 @@ import Text.Pandoc.Builder hiding (cell)
2020import Text.Pandoc.Error (PandocError (PandocParseError))
2121import Text.Pandoc.Options (ReaderOptions)
2222import Text.Pandoc.Shared (stringify)
23−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
23+
2424import qualified Text.Jira.Markup as Jira
2525
2626-- | Read Jira wiki markup.
27−readJira :: (PandocMonad m, ToSources a)
27+readJira :: PandocMonad m
2828 => ReaderOptions
29− -> a
29+ -> Text
3030 -> m Pandoc
31−readJira _opts inp = do
32− let sources = toSources inp
33− case parse (sourcesToText sources) of
34− Right d -> return $ jiraToPandoc d
35− Left e -> throwError . PandocParseError $
36− "Jira parse error" `append` pack (show e)
31+readJira _opts s = case parse s of
32+ Right d -> return $ jiraToPandoc d
33+ Left e -> throwError . PandocParseError $
34+ "Jira parse error" `append` pack (show e)
3735
3836jiraToPandoc :: Jira.Doc -> Pandoc
3937jiraToPandoc (Jira.Doc blks) = doc $ foldMap jiraToPandocBlocks blks
@@ -73,10 +71,10 @@ toPandocCodeBlocks langMay params txt =
7371 Nothing -> []
7472 in codeBlockWith ("", classes, map paramToPair params) txt
7573
76−-- | Create a pandoc @'Div'@ from a panel.
74+-- | Create a pandoc @'Div'@
7775toPandocDiv :: [Jira.Parameter] -> [Jira.Block] -> Blocks
7876toPandocDiv params =
79− divWith ("", ["panel"], map paramToPair params) . foldMap jiraToPandocBlocks
77+ divWith ("", [], map paramToPair params) . foldMap jiraToPandocBlocks
8078
8179paramToPair :: Jira.Parameter -> (Text, Text)
8280paramToPair (Jira.Parameter key value) = (key, value)
@@ -172,8 +170,6 @@ jiraLinkToPandoc linkType alias url =
172170 Jira.Email -> link ("mailto:" <> url') "" alias'
173171 Jira.Attachment -> linkWith ("", ["attachment"], []) url' "" alias'
174172 Jira.User -> linkWith ("", ["user-account"], []) url' "" alias'
175− Jira.SmartCard -> linkWith ("", ["smart-card"], []) url' "" alias'
176− Jira.SmartLink -> linkWith ("", ["smart-link"], []) url' "" alias'
177173
178174-- | Get unicode representation of a Jira icon.
179175iconUnicode :: Jira.Icon -> Text
src/Text/Pandoc/Readers/LaTeX.hs+34−22
@@ -1,3 +1,4 @@
1+{-# LANGUAGE CPP #-}
12{-# LANGUAGE OverloadedStrings #-}
23{-# LANGUAGE PatternGuards #-}
34{-# LANGUAGE ScopedTypeVariables #-}
@@ -24,7 +25,7 @@ module Text.Pandoc.Readers.LaTeX ( readLaTeX,
2425import Control.Applicative (many, optional, (<|>))
2526import Control.Monad
2627import Control.Monad.Except (throwError)
27−import Data.Char (isDigit, isLetter, isAlphaNum, toUpper, chr)
28+import Data.Char (isDigit, isLetter, toUpper, chr)
2829import Data.Default
2930import Data.List (intercalate)
3031import qualified Data.Map as M
@@ -32,9 +33,8 @@ import Data.Maybe (fromMaybe, maybeToList)
3233import qualified Data.Set as Set
3334import Data.Text (Text)
3435import qualified Data.Text as T
35−import Skylighting (defaultSyntaxMap)
3636import System.FilePath (addExtension, replaceExtension, takeExtension)
37−import Text.Collate.Lang (renderLang)
37+import Text.Pandoc.BCP47 (renderLang)
3838import Text.Pandoc.Builder as B
3939import Text.Pandoc.Class.PandocPure (PandocPure)
4040import Text.Pandoc.Class.PandocMonad (PandocMonad (..), getResourcePath,
@@ -69,7 +69,7 @@ import Text.Pandoc.Readers.LaTeX.Inline (acronymCommands, refCommands,
6969 listingsLanguage)
7070import Text.Pandoc.Shared
7171import Text.Pandoc.Walk
72−import Data.List.NonEmpty (nonEmpty)
72+import Safe
7373
7474-- for debugging:
7575-- import Text.Pandoc.Extensions (getDefaultExtensions)
@@ -77,17 +77,16 @@ import Data.List.NonEmpty (nonEmpty)
7777-- import Debug.Trace (traceShowId)
7878
7979-- | Parse LaTeX from string and return 'Pandoc' document.
80−readLaTeX :: (PandocMonad m, ToSources a)
80+readLaTeX :: PandocMonad m
8181 => ReaderOptions -- ^ Reader options
82− -> a -- ^ Input to parse
82+ -> Text -- ^ String to parse (assumes @'\n'@ line endings)
8383 -> m Pandoc
8484readLaTeX opts ltx = do
85− let sources = toSources ltx
8685 parsed <- runParserT parseLaTeX def{ sOptions = opts } "source"
87− (tokenizeSources sources)
86+ (tokenize "source" (crFilter ltx))
8887 case parsed of
8988 Right result -> return result
90− Left e -> throwError $ PandocParsecError sources e
89+ Left e -> throwError $ PandocParsecError ltx e
9190
9291parseLaTeX :: PandocMonad m => LP m Pandoc
9392parseLaTeX = do
@@ -98,7 +97,11 @@ parseLaTeX = do
9897 let doc' = doc bs
9998 let headerLevel (Header n _ _) = [n]
10099 headerLevel _ = []
101− let bottomLevel = maybe 1 minimum $ nonEmpty $ query headerLevel doc'
100+#if MIN_VERSION_safe(0,3,18)
101+ let bottomLevel = minimumBound 1 $ query headerLevel doc'
102+#else
103+ let bottomLevel = minimumDef 1 $ query headerLevel doc'
104+#endif
102105 let adjustHeaders m (Header n attr ils) = Header (n+m) attr ils
103106 adjustHeaders _ x = x
104107 let (Pandoc _ bs') =
@@ -133,11 +136,11 @@ resolveRefs _ x = x
133136
134137
135138rawLaTeXBlock :: (PandocMonad m, HasMacros s, HasReaderOptions s)
136− => ParserT Sources s m Text
139+ => ParserT Text s m Text
137140rawLaTeXBlock = do
138141 lookAhead (try (char '\\' >> letter))
139142 inp <- getInput
140− let toks = tokenizeSources inp
143+ let toks = tokenize "source" inp
141144 snd <$> (rawLaTeXParser toks False (macroDef (const mempty)) blocks
142145 <|> rawLaTeXParser toks True
143146 (do choice (map controlSeq
@@ -164,11 +167,11 @@ beginOrEndCommand = try $ do
164167 (txt <> untokenize rawargs)
165168
166169rawLaTeXInline :: (PandocMonad m, HasMacros s, HasReaderOptions s)
167− => ParserT Sources s m Text
170+ => ParserT Text s m Text
168171rawLaTeXInline = do
169172 lookAhead (try (char '\\' >> letter))
170173 inp <- getInput
171− let toks = tokenizeSources inp
174+ let toks = tokenize "source" inp
172175 raw <- snd <$>
173176 ( rawLaTeXParser toks True
174177 (mempty <$ (controlSeq "input" >> skipMany rawopt >> braced))
@@ -179,11 +182,11 @@ rawLaTeXInline = do
179182 finalbraces <- mconcat <$> many (try (string "{}")) -- see #5439
180183 return $ raw <> T.pack finalbraces
181184
182−inlineCommand :: PandocMonad m => ParserT Sources ParserState m Inlines
185+inlineCommand :: PandocMonad m => ParserT Text ParserState m Inlines
183186inlineCommand = do
184187 lookAhead (try (char '\\' >> letter))
185188 inp <- getInput
186− let toks = tokenizeSources inp
189+ let toks = tokenize "source" inp
187190 fst <$> rawLaTeXParser toks True (inlineEnvironment <|> inlineCommand')
188191 inlines
189192
@@ -232,6 +235,16 @@ mkImage options (T.unpack -> src) = do
232235 _ -> return src
233236 return $ imageWith attr (T.pack src') "" alt
234237
238+doxspace :: PandocMonad m => LP m Inlines
239+doxspace =
240+ (space <$ lookAhead (satisfyTok startsWithLetter)) <|> return mempty
241+ where startsWithLetter (Tok _ Word t) =
242+ case T.uncons t of
243+ Just (c, _) | isLetter c -> True
244+ _ -> False
245+ startsWithLetter _ = False
246+
247+
235248removeDoubleQuotes :: Text -> Text
236249removeDoubleQuotes t =
237250 Data.Maybe.fromMaybe t $ T.stripPrefix "\"" t >>= T.stripSuffix "\""
@@ -300,9 +313,7 @@ inlineCommand' :: PandocMonad m => LP m Inlines
300313inlineCommand' = try $ do
301314 Tok _ (CtrlSeq name) cmd <- anyControlSeq
302315 guard $ name /= "begin" && name /= "end" && name /= "and"
303− star <- if T.all isAlphaNum name
304− then option "" ("*" <$ symbol '*' <* sp)
305− else pure ""
316+ star <- option "" ("*" <$ symbol '*' <* sp)
306317 overlay <- option "" overlaySpecification
307318 let name' = name <> star <> overlay
308319 let names = ordNub [name', name] -- check non-starred as fallback
@@ -409,6 +420,8 @@ inlineCommands = M.unions
409420 -- LaTeX colors
410421 , ("textcolor", coloredInline "color")
411422 , ("colorbox", coloredInline "background-color")
423+ -- xspace
424+ , ("xspace", doxspace)
412425 -- etoolbox
413426 , ("ifstrequal", ifstrequal)
414427 , ("newtoggle", braced >>= newToggle)
@@ -632,7 +645,7 @@ opt = do
632645 parsed <- runParserT (mconcat <$> many inline) st "bracketed option" toks
633646 case parsed of
634647 Right result -> return result
635− Left e -> throwError $ PandocParsecError (toSources toks) e
648+ Left e -> throwError $ PandocParsecError (untokenize toks) e
636649
637650-- block elements:
638651
@@ -1162,8 +1175,7 @@ inputListing = do
11621175 let (ident,classes,kvs) = parseListingsOptions options
11631176 let classes' =
11641177 (case listingsLanguage options of
1165− Nothing -> (take 1 (languagesByExtension defaultSyntaxMap
1166− (T.pack $ takeExtension $ T.unpack f)) <>)
1178+ Nothing -> (take 1 (languagesByExtension (T.pack $ takeExtension $ T.unpack f)) <>)
11671179 Just _ -> id) classes
11681180 let firstline = fromMaybe 1 $ lookup "firstline" options >>= safeRead
11691181 let lastline = fromMaybe (length codeLines) $
src/Text/Pandoc/Readers/LaTeX/Citation.hs+1−1
@@ -120,7 +120,7 @@ simpleCiteArgs inline = try $ do
120120 runParserT (mconcat <$> many inline) st "bracketed option" toks
121121 case parsed of
122122 Right result -> return result
123− Left e -> throwError $ PandocParsecError (toSources toks) e
123+ Left e -> throwError $ PandocParsecError (untokenize toks) e
124124
125125
126126
src/Text/Pandoc/Readers/LaTeX/Lang.hs+119−122
@@ -23,7 +23,7 @@ import qualified Data.Map as M
2323import Data.Text (Text)
2424import qualified Data.Text as T
2525import Text.Pandoc.Shared (extractSpaces)
26−import Text.Collate.Lang (Lang(..), renderLang)
26+import Text.Pandoc.BCP47 (Lang(..), renderLang)
2727import Text.Pandoc.Class (PandocMonad(..), setTranslations)
2828import Text.Pandoc.Readers.LaTeX.Parsing
2929import Text.Pandoc.Parsing (updateState, option, getState, QuoteContext(..),
@@ -99,136 +99,133 @@ setDefaultLanguage = do
9999polyglossiaLangToBCP47 :: M.Map T.Text (T.Text -> Lang)
100100polyglossiaLangToBCP47 = M.fromList
101101 [ ("arabic", \o -> case T.filter (/=' ') o of
102− "locale=algeria" -> Lang "ar" Nothing (Just "DZ") [] [] []
103− "locale=mashriq" -> Lang "ar" Nothing (Just "SY") [] [] []
104− "locale=libya" -> Lang "ar" Nothing (Just "LY") [] [] []
105− "locale=morocco" -> Lang "ar" Nothing (Just "MA") [] [] []
106− "locale=mauritania" -> Lang "ar" Nothing (Just "MR") [] [] []
107− "locale=tunisia" -> Lang "ar" Nothing (Just "TN") [] [] []
108− _ -> Lang "ar" Nothing (Just "") [] [] [])
102+ "locale=algeria" -> Lang "ar" "" "DZ" []
103+ "locale=mashriq" -> Lang "ar" "" "SY" []
104+ "locale=libya" -> Lang "ar" "" "LY" []
105+ "locale=morocco" -> Lang "ar" "" "MA" []
106+ "locale=mauritania" -> Lang "ar" "" "MR" []
107+ "locale=tunisia" -> Lang "ar" "" "TN" []
108+ _ -> Lang "ar" "" "" [])
109109 , ("german", \o -> case T.filter (/=' ') o of
110− "spelling=old" -> Lang "de" Nothing (Just "DE") ["1901"] [] []
110+ "spelling=old" -> Lang "de" "" "DE" ["1901"]
111111 "variant=austrian,spelling=old"
112− -> Lang "de" Nothing (Just "AT") ["1901"] [] []
113− "variant=austrian" -> Lang "de" Nothing (Just "AT") [] [] []
112+ -> Lang "de" "" "AT" ["1901"]
113+ "variant=austrian" -> Lang "de" "" "AT" []
114114 "variant=swiss,spelling=old"
115− -> Lang "de" Nothing (Just "CH") ["1901"] [] []
116− "variant=swiss" -> Lang "de" Nothing (Just "CH") [] [] []
117− _ -> Lang "de" Nothing Nothing [] [] [])
118− , ("lsorbian", \_ -> Lang "dsb" Nothing Nothing [] [] [])
115+ -> Lang "de" "" "CH" ["1901"]
116+ "variant=swiss" -> Lang "de" "" "CH" []
117+ _ -> Lang "de" "" "" [])
118+ , ("lsorbian", \_ -> Lang "dsb" "" "" [])
119119 , ("greek", \o -> case T.filter (/=' ') o of
120− "variant=poly" -> Lang "el" Nothing (Just "polyton") [] [] []
121− "variant=ancient" -> Lang "grc" Nothing Nothing [] [] []
122− _ -> Lang "el" Nothing Nothing [] [] [])
120+ "variant=poly" -> Lang "el" "" "polyton" []
121+ "variant=ancient" -> Lang "grc" "" "" []
122+ _ -> Lang "el" "" "" [])
123123 , ("english", \o -> case T.filter (/=' ') o of
124− "variant=australian" -> Lang "en" Nothing (Just "AU") [] [] []
125− "variant=canadian" -> Lang "en" Nothing (Just "CA") [] [] []
126− "variant=british" -> Lang "en" Nothing (Just "GB") [] [] []
127− "variant=newzealand" -> Lang "en" Nothing (Just "NZ") [] [] []
128− "variant=american" -> Lang "en" Nothing (Just "US") [] [] []
129− _ -> Lang "en" Nothing (Just "") [] [] [])
130− , ("usorbian", \_ -> Lang "hsb" Nothing Nothing [] [] [])
124+ "variant=australian" -> Lang "en" "" "AU" []
125+ "variant=canadian" -> Lang "en" "" "CA" []
126+ "variant=british" -> Lang "en" "" "GB" []
127+ "variant=newzealand" -> Lang "en" "" "NZ" []
128+ "variant=american" -> Lang "en" "" "US" []
129+ _ -> Lang "en" "" "" [])
130+ , ("usorbian", \_ -> Lang "hsb" "" "" [])
131131 , ("latin", \o -> case T.filter (/=' ') o of
132− "variant=classic" -> Lang "la" Nothing Nothing ["x-classic"] [] []
133− _ -> Lang "la" Nothing Nothing [] [] [])
134− , ("slovenian", \_ -> Lang "sl" Nothing Nothing [] [] [])
135− , ("serbianc", \_ -> Lang "sr" (Just "Cyrl") Nothing [] [] [])
136− , ("pinyin", \_ -> Lang "zh" (Just "Latn") Nothing ["pinyin"] [] [])
137− , ("afrikaans", \_ -> simpleLang "af")
138− , ("amharic", \_ -> simpleLang "am")
139− , ("assamese", \_ -> simpleLang "as")
140− , ("asturian", \_ -> simpleLang "ast")
141− , ("bulgarian", \_ -> simpleLang "bg")
142− , ("bengali", \_ -> simpleLang "bn")
143− , ("tibetan", \_ -> simpleLang "bo")
144− , ("breton", \_ -> simpleLang "br")
145− , ("catalan", \_ -> simpleLang "ca")
146− , ("welsh", \_ -> simpleLang "cy")
147− , ("czech", \_ -> simpleLang "cs")
148− , ("coptic", \_ -> simpleLang "cop")
149− , ("danish", \_ -> simpleLang "da")
150− , ("divehi", \_ -> simpleLang "dv")
151− , ("esperanto", \_ -> simpleLang "eo")
152− , ("spanish", \_ -> simpleLang "es")
153− , ("estonian", \_ -> simpleLang "et")
154− , ("basque", \_ -> simpleLang "eu")
155− , ("farsi", \_ -> simpleLang "fa")
156− , ("finnish", \_ -> simpleLang "fi")
157− , ("french", \_ -> simpleLang "fr")
158− , ("friulan", \_ -> simpleLang "fur")
159− , ("irish", \_ -> simpleLang "ga")
160− , ("scottish", \_ -> simpleLang "gd")
161− , ("ethiopic", \_ -> simpleLang "gez")
162− , ("galician", \_ -> simpleLang "gl")
163− , ("hebrew", \_ -> simpleLang "he")
164− , ("hindi", \_ -> simpleLang "hi")
165− , ("croatian", \_ -> simpleLang "hr")
166− , ("magyar", \_ -> simpleLang "hu")
167− , ("armenian", \_ -> simpleLang "hy")
168− , ("interlingua", \_ -> simpleLang "ia")
169− , ("indonesian", \_ -> simpleLang "id")
170− , ("icelandic", \_ -> simpleLang "is")
171− , ("italian", \_ -> simpleLang "it")
172− , ("japanese", \_ -> simpleLang "jp")
173− , ("khmer", \_ -> simpleLang "km")
174− , ("kurmanji", \_ -> simpleLang "kmr")
175− , ("kannada", \_ -> simpleLang "kn")
176− , ("korean", \_ -> simpleLang "ko")
177− , ("lao", \_ -> simpleLang "lo")
178− , ("lithuanian", \_ -> simpleLang "lt")
179− , ("latvian", \_ -> simpleLang "lv")
180− , ("malayalam", \_ -> simpleLang "ml")
181− , ("mongolian", \_ -> simpleLang "mn")
182− , ("marathi", \_ -> simpleLang "mr")
183− , ("dutch", \_ -> simpleLang "nl")
184− , ("nynorsk", \_ -> simpleLang "nn")
185− , ("norsk", \_ -> simpleLang "no")
186− , ("nko", \_ -> simpleLang "nqo")
187− , ("occitan", \_ -> simpleLang "oc")
188− , ("panjabi", \_ -> simpleLang "pa")
189− , ("polish", \_ -> simpleLang "pl")
190− , ("piedmontese", \_ -> simpleLang "pms")
191− , ("portuguese", \_ -> simpleLang "pt")
192− , ("romansh", \_ -> simpleLang "rm")
193− , ("romanian", \_ -> simpleLang "ro")
194− , ("russian", \_ -> simpleLang "ru")
195− , ("sanskrit", \_ -> simpleLang "sa")
196− , ("samin", \_ -> simpleLang "se")
197− , ("slovak", \_ -> simpleLang "sk")
198− , ("albanian", \_ -> simpleLang "sq")
199− , ("serbian", \_ -> simpleLang "sr")
200− , ("swedish", \_ -> simpleLang "sv")
201− , ("syriac", \_ -> simpleLang "syr")
202− , ("tamil", \_ -> simpleLang "ta")
203− , ("telugu", \_ -> simpleLang "te")
204− , ("thai", \_ -> simpleLang "th")
205− , ("turkmen", \_ -> simpleLang "tk")
206− , ("turkish", \_ -> simpleLang "tr")
207− , ("ukrainian", \_ -> simpleLang "uk")
208− , ("urdu", \_ -> simpleLang "ur")
209− , ("vietnamese", \_ -> simpleLang "vi")
132+ "variant=classic" -> Lang "la" "" "" ["x-classic"]
133+ _ -> Lang "la" "" "" [])
134+ , ("slovenian", \_ -> Lang "sl" "" "" [])
135+ , ("serbianc", \_ -> Lang "sr" "cyrl" "" [])
136+ , ("pinyin", \_ -> Lang "zh" "Latn" "" ["pinyin"])
137+ , ("afrikaans", \_ -> Lang "af" "" "" [])
138+ , ("amharic", \_ -> Lang "am" "" "" [])
139+ , ("assamese", \_ -> Lang "as" "" "" [])
140+ , ("asturian", \_ -> Lang "ast" "" "" [])
141+ , ("bulgarian", \_ -> Lang "bg" "" "" [])
142+ , ("bengali", \_ -> Lang "bn" "" "" [])
143+ , ("tibetan", \_ -> Lang "bo" "" "" [])
144+ , ("breton", \_ -> Lang "br" "" "" [])
145+ , ("catalan", \_ -> Lang "ca" "" "" [])
146+ , ("welsh", \_ -> Lang "cy" "" "" [])
147+ , ("czech", \_ -> Lang "cs" "" "" [])
148+ , ("coptic", \_ -> Lang "cop" "" "" [])
149+ , ("danish", \_ -> Lang "da" "" "" [])
150+ , ("divehi", \_ -> Lang "dv" "" "" [])
151+ , ("esperanto", \_ -> Lang "eo" "" "" [])
152+ , ("spanish", \_ -> Lang "es" "" "" [])
153+ , ("estonian", \_ -> Lang "et" "" "" [])
154+ , ("basque", \_ -> Lang "eu" "" "" [])
155+ , ("farsi", \_ -> Lang "fa" "" "" [])
156+ , ("finnish", \_ -> Lang "fi" "" "" [])
157+ , ("french", \_ -> Lang "fr" "" "" [])
158+ , ("friulan", \_ -> Lang "fur" "" "" [])
159+ , ("irish", \_ -> Lang "ga" "" "" [])
160+ , ("scottish", \_ -> Lang "gd" "" "" [])
161+ , ("ethiopic", \_ -> Lang "gez" "" "" [])
162+ , ("galician", \_ -> Lang "gl" "" "" [])
163+ , ("hebrew", \_ -> Lang "he" "" "" [])
164+ , ("hindi", \_ -> Lang "hi" "" "" [])
165+ , ("croatian", \_ -> Lang "hr" "" "" [])
166+ , ("magyar", \_ -> Lang "hu" "" "" [])
167+ , ("armenian", \_ -> Lang "hy" "" "" [])
168+ , ("interlingua", \_ -> Lang "ia" "" "" [])
169+ , ("indonesian", \_ -> Lang "id" "" "" [])
170+ , ("icelandic", \_ -> Lang "is" "" "" [])
171+ , ("italian", \_ -> Lang "it" "" "" [])
172+ , ("japanese", \_ -> Lang "jp" "" "" [])
173+ , ("khmer", \_ -> Lang "km" "" "" [])
174+ , ("kurmanji", \_ -> Lang "kmr" "" "" [])
175+ , ("kannada", \_ -> Lang "kn" "" "" [])
176+ , ("korean", \_ -> Lang "ko" "" "" [])
177+ , ("lao", \_ -> Lang "lo" "" "" [])
178+ , ("lithuanian", \_ -> Lang "lt" "" "" [])
179+ , ("latvian", \_ -> Lang "lv" "" "" [])
180+ , ("malayalam", \_ -> Lang "ml" "" "" [])
181+ , ("mongolian", \_ -> Lang "mn" "" "" [])
182+ , ("marathi", \_ -> Lang "mr" "" "" [])
183+ , ("dutch", \_ -> Lang "nl" "" "" [])
184+ , ("nynorsk", \_ -> Lang "nn" "" "" [])
185+ , ("norsk", \_ -> Lang "no" "" "" [])
186+ , ("nko", \_ -> Lang "nqo" "" "" [])
187+ , ("occitan", \_ -> Lang "oc" "" "" [])
188+ , ("panjabi", \_ -> Lang "pa" "" "" [])
189+ , ("polish", \_ -> Lang "pl" "" "" [])
190+ , ("piedmontese", \_ -> Lang "pms" "" "" [])
191+ , ("portuguese", \_ -> Lang "pt" "" "" [])
192+ , ("romansh", \_ -> Lang "rm" "" "" [])
193+ , ("romanian", \_ -> Lang "ro" "" "" [])
194+ , ("russian", \_ -> Lang "ru" "" "" [])
195+ , ("sanskrit", \_ -> Lang "sa" "" "" [])
196+ , ("samin", \_ -> Lang "se" "" "" [])
197+ , ("slovak", \_ -> Lang "sk" "" "" [])
198+ , ("albanian", \_ -> Lang "sq" "" "" [])
199+ , ("serbian", \_ -> Lang "sr" "" "" [])
200+ , ("swedish", \_ -> Lang "sv" "" "" [])
201+ , ("syriac", \_ -> Lang "syr" "" "" [])
202+ , ("tamil", \_ -> Lang "ta" "" "" [])
203+ , ("telugu", \_ -> Lang "te" "" "" [])
204+ , ("thai", \_ -> Lang "th" "" "" [])
205+ , ("turkmen", \_ -> Lang "tk" "" "" [])
206+ , ("turkish", \_ -> Lang "tr" "" "" [])
207+ , ("ukrainian", \_ -> Lang "uk" "" "" [])
208+ , ("urdu", \_ -> Lang "ur" "" "" [])
209+ , ("vietnamese", \_ -> Lang "vi" "" "" [])
210210 ]
211211
212−simpleLang :: Text -> Lang
213−simpleLang l = Lang l Nothing Nothing [] [] []
214−
215212babelLangToBCP47 :: T.Text -> Maybe Lang
216213babelLangToBCP47 s =
217214 case s of
218− "austrian" -> Just $ Lang "de" Nothing (Just "AT") ["1901"] [] []
219− "naustrian" -> Just $ Lang "de" Nothing (Just "AT") [] [] []
220− "swissgerman" -> Just $ Lang "de" Nothing (Just "CH") ["1901"] [] []
221− "nswissgerman" -> Just $ Lang "de" Nothing (Just "CH") [] [] []
222− "german" -> Just $ Lang "de" Nothing (Just "DE") ["1901"] [] []
223− "ngerman" -> Just $ Lang "de" Nothing (Just "DE") [] [] []
224− "lowersorbian" -> Just $ Lang "dsb" Nothing Nothing [] [] []
225− "uppersorbian" -> Just $ Lang "hsb" Nothing Nothing [] [] []
226− "polutonikogreek" -> Just $ Lang "el" Nothing Nothing ["polyton"] [] []
227− "slovene" -> Just $ simpleLang "sl"
228− "australian" -> Just $ Lang "en" Nothing (Just "AU") [] [] []
229− "canadian" -> Just $ Lang "en" Nothing (Just "CA") [] [] []
230− "british" -> Just $ Lang "en" Nothing (Just "GB") [] [] []
231− "newzealand" -> Just $ Lang "en" Nothing (Just "NZ") [] [] []
232− "american" -> Just $ Lang "en" Nothing (Just "US") [] [] []
233− "classiclatin" -> Just $ Lang "la" Nothing Nothing ["x-classic"] [] []
215+ "austrian" -> Just $ Lang "de" "" "AT" ["1901"]
216+ "naustrian" -> Just $ Lang "de" "" "AT" []
217+ "swissgerman" -> Just $ Lang "de" "" "CH" ["1901"]
218+ "nswissgerman" -> Just $ Lang "de" "" "CH" []
219+ "german" -> Just $ Lang "de" "" "DE" ["1901"]
220+ "ngerman" -> Just $ Lang "de" "" "DE" []
221+ "lowersorbian" -> Just $ Lang "dsb" "" "" []
222+ "uppersorbian" -> Just $ Lang "hsb" "" "" []
223+ "polutonikogreek" -> Just $ Lang "el" "" "" ["polyton"]
224+ "slovene" -> Just $ Lang "sl" "" "" []
225+ "australian" -> Just $ Lang "en" "" "AU" []
226+ "canadian" -> Just $ Lang "en" "" "CA" []
227+ "british" -> Just $ Lang "en" "" "GB" []
228+ "newzealand" -> Just $ Lang "en" "" "NZ" []
229+ "american" -> Just $ Lang "en" "" "US" []
230+ "classiclatin" -> Just $ Lang "la" "" "" ["x-classic"]
234231 _ -> ($ "") <$> M.lookup s polyglossiaLangToBCP47
src/Text/Pandoc/Readers/LaTeX/Macro.hs+14−45
@@ -14,7 +14,6 @@ import Text.Pandoc.Parsing hiding (blankline, mathDisplay, mathInline,
1414import Control.Applicative ((<|>), optional)
1515import qualified Data.Map as M
1616import Data.Text (Text)
17−import qualified Data.Text as T
1817
1918macroDef :: (PandocMonad m, Monoid a) => (Text -> a) -> LP m a
2019macroDef constructor = do
@@ -23,11 +22,9 @@ macroDef constructor = do
2322 guardDisabled Ext_latex_macros)
2423 <|> return mempty
2524 where commandDef = do
26− nameMacroPairs <- newcommand <|> letmacro <|> defmacro <|> newif
25+ (name, macro') <- newcommand <|> letmacro <|> defmacro
2726 guardDisabled Ext_latex_macros <|>
28− mapM_ (\(name, macro') ->
29− updateState (\s -> s{ sMacros = M.insert name macro'
30− (sMacros s) })) nameMacroPairs
27+ updateState (\s -> s{ sMacros = M.insert name macro' (sMacros s) })
3128 environmentDef = do
3229 mbenv <- newenvironment
3330 case mbenv of
@@ -43,7 +40,7 @@ macroDef constructor = do
4340 -- @\newcommand{\envname}[n-args][default]{begin}@
4441 -- @\newcommand{\endenvname}@
4542
46−letmacro :: PandocMonad m => LP m [(Text, Macro)]
43+letmacro :: PandocMonad m => LP m (Text, Macro)
4744letmacro = do
4845 controlSeq "let"
4946 (name, contents) <- withVerbatimMode $ do
@@ -56,47 +53,18 @@ letmacro = do
5653 contents <- bracedOrToken
5754 return (name, contents)
5855 contents' <- doMacros' 0 contents
59− return [(name, Macro ExpandWhenDefined [] Nothing contents')]
56+ return (name, Macro ExpandWhenDefined [] Nothing contents')
6057
61−defmacro :: PandocMonad m => LP m [(Text, Macro)]
62−defmacro = do
58+defmacro :: PandocMonad m => LP m (Text, Macro)
59+defmacro = try $
6360 -- we use withVerbatimMode, because macros are to be expanded
6461 -- at point of use, not point of definition
65− controlSeq "def"
6662 withVerbatimMode $ do
63+ controlSeq "def"
6764 Tok _ (CtrlSeq name) _ <- anyControlSeq
6865 argspecs <- many (argspecArg <|> argspecPattern)
6966 contents <- bracedOrToken
70− return [(name, Macro ExpandWhenUsed argspecs Nothing contents)]
71−
72−-- \newif\iffoo' defines:
73−-- \iffoo to be \iffalse
74−-- \footrue to be a command that defines \iffoo to be \iftrue
75−-- \foofalse to be a command that defines \iffoo to be \iffalse
76−newif :: PandocMonad m => LP m [(Text, Macro)]
77−newif = do
78− controlSeq "newif"
79− withVerbatimMode $ do
80− Tok pos (CtrlSeq name) _ <- anyControlSeq
81− -- \def\iffoo\iffalse
82− -- \def\footrue{\def\iffoo\iftrue}
83− -- \def\foofalse{\def\iffoo\iffalse}
84− let base = T.drop 2 name
85− return [ (name, Macro ExpandWhenUsed [] Nothing
86− [Tok pos (CtrlSeq "iffalse") "\\iffalse"])
87− , (base <> "true",
88− Macro ExpandWhenUsed [] Nothing
89− [ Tok pos (CtrlSeq "def") "\\def"
90− , Tok pos (CtrlSeq name) ("\\" <> name)
91− , Tok pos (CtrlSeq "iftrue") "\\iftrue"
92− ])
93− , (base <> "false",
94− Macro ExpandWhenUsed [] Nothing
95− [ Tok pos (CtrlSeq "def") "\\def"
96− , Tok pos (CtrlSeq name) ("\\" <> name)
97− , Tok pos (CtrlSeq "iffalse") "\\iffalse"
98− ])
99− ]
67+ return (name, Macro ExpandWhenUsed argspecs Nothing contents)
10068
10169argspecArg :: PandocMonad m => LP m ArgSpec
10270argspecArg = do
@@ -109,9 +77,10 @@ argspecPattern =
10977 (toktype' == Symbol || toktype' == Word) &&
11078 (txt /= "{" && txt /= "\\" && txt /= "}")))
11179
112−newcommand :: PandocMonad m => LP m [(Text, Macro)]
80+newcommand :: PandocMonad m => LP m (Text, Macro)
11381newcommand = do
114− Tok pos (CtrlSeq mtype) _ <- controlSeq "newcommand" <|>
82+ pos <- getPosition
83+ Tok _ (CtrlSeq mtype) _ <- controlSeq "newcommand" <|>
11584 controlSeq "renewcommand" <|>
11685 controlSeq "providecommand" <|>
11786 controlSeq "DeclareMathOperator" <|>
@@ -143,9 +112,9 @@ newcommand = do
143112 Just macro
144113 | mtype == "newcommand" -> do
145114 report $ MacroAlreadyDefined txt pos
146− return [(name, macro)]
147− | mtype == "providecommand" -> return [(name, macro)]
148− _ -> return [(name, Macro ExpandWhenUsed argspecs optarg contents)]
115+ return (name, macro)
116+ | mtype == "providecommand" -> return (name, macro)
117+ _ -> return (name, Macro ExpandWhenUsed argspecs optarg contents)
149118
150119newenvironment :: PandocMonad m => LP m (Maybe (Text, Macro, Macro))
151120newenvironment = do
src/Text/Pandoc/Readers/LaTeX/Parsing.hs+7−48
@@ -27,7 +27,6 @@ module Text.Pandoc.Readers.LaTeX.Parsing
2727 , rawLaTeXParser
2828 , applyMacros
2929 , tokenize
30− , tokenizeSources
3130 , untokenize
3231 , untoken
3332 , totoks
@@ -113,6 +112,7 @@ import Text.Pandoc.Readers.LaTeX.Types (ExpansionPoint (..), Macro (..),
113112 ArgSpec (..), Tok (..), TokType (..))
114113import Text.Pandoc.Shared
115114import Text.Parsec.Pos
115+-- import Debug.Trace
116116
117117newtype DottedNum = DottedNum [Int]
118118 deriving (Show, Eq)
@@ -248,7 +248,7 @@ withVerbatimMode parser = do
248248
249249rawLaTeXParser :: (PandocMonad m, HasMacros s, HasReaderOptions s)
250250 => [Tok] -> Bool -> LP m a -> LP m a
251− -> ParserT Sources s m (a, Text)
251+ -> ParserT Text s m (a, Text)
252252rawLaTeXParser toks retokenize parser valParser = do
253253 pstate <- getState
254254 let lstate = def{ sOptions = extractReaderOptions pstate }
@@ -268,7 +268,7 @@ rawLaTeXParser toks retokenize parser valParser = do
268268 Left _ -> mzero
269269 Right ((val, raw), st) -> do
270270 updateState (updateMacros (sMacros st <>))
271− void $ count (T.length (untokenize toks')) anyChar
271+ _ <- takeP (T.length (untokenize toks'))
272272 let result = untokenize raw
273273 -- ensure we end with space if input did, see #4442
274274 let result' =
@@ -281,7 +281,7 @@ rawLaTeXParser toks retokenize parser valParser = do
281281 return (val, result')
282282
283283applyMacros :: (PandocMonad m, HasMacros s, HasReaderOptions s)
284− => Text -> ParserT Sources s m Text
284+ => Text -> ParserT Text s m Text
285285applyMacros s = (guardDisabled Ext_latex_macros >> return s) <|>
286286 do let retokenize = untokenize <$> many (satisfyTok (const True))
287287 pstate <- getState
@@ -301,11 +301,6 @@ QuickCheck property:
301301> let t = T.pack s in untokenize (tokenize "random" t) == t
302302-}
303303
304−tokenizeSources :: Sources -> [Tok]
305−tokenizeSources = concatMap tokenizeSource . unSources
306− where
307− tokenizeSource (pos, t) = totoks pos t
308−
309304tokenize :: SourceName -> Text -> [Tok]
310305tokenize sourcename = totoks (initialPos sourcename)
311306
@@ -463,7 +458,7 @@ satisfyTok f = do
463458doMacros :: PandocMonad m => LP m ()
464459doMacros = do
465460 st <- getState
466− unless (sVerbatimMode st) $
461+ unless (sVerbatimMode st || M.null (sMacros st)) $ do
467462 getInput >>= doMacros' 1 >>= setInput
468463
469464doMacros' :: PandocMonad m => Int -> [Tok] -> LP m [Tok]
@@ -525,7 +520,7 @@ doMacros' n inp =
525520 $ throwError $ PandocMacroLoop name
526521 macros <- sMacros <$> getState
527522 case M.lookup name macros of
528− Nothing -> trySpecialMacro name ts
523+ Nothing -> mzero
529524 Just (Macro expansionPoint argspecs optarg newtoks) -> do
530525 let getargs' = do
531526 args <-
@@ -553,41 +548,6 @@ doMacros' n inp =
553548 ExpandWhenUsed -> doMacros' (n' + 1) result
554549 ExpandWhenDefined -> return result
555550
556−-- | Certain macros do low-level tex manipulations that can't
557−-- be represented in our Macro type, so we handle them here.
558−trySpecialMacro :: PandocMonad m => Text -> [Tok] -> LP m [Tok]
559−trySpecialMacro "xspace" ts = do
560− ts' <- doMacros' 1 ts
561− case ts' of
562− Tok pos Word t : _
563− | startsWithAlphaNum t -> return $ Tok pos Spaces " " : ts'
564− _ -> return ts'
565−trySpecialMacro "iftrue" ts = handleIf True ts
566−trySpecialMacro "iffalse" ts = handleIf False ts
567−trySpecialMacro _ _ = mzero
568−
569−handleIf :: PandocMonad m => Bool -> [Tok] -> LP m [Tok]
570−handleIf b ts = do
571− res' <- lift $ runParserT (ifParser b) defaultLaTeXState "tokens" ts
572− case res' of
573− Left _ -> Prelude.fail "Could not parse conditional"
574− Right ts' -> return ts'
575−
576−ifParser :: PandocMonad m => Bool -> LP m [Tok]
577−ifParser b = do
578− ifToks <- many (notFollowedBy (controlSeq "else" <|> controlSeq "fi")
579− *> anyTok)
580− elseToks <- (controlSeq "else" >> manyTill anyTok (controlSeq "fi"))
581− <|> ([] <$ controlSeq "fi")
582− rest <- getInput
583− return $ (if b then ifToks else elseToks) ++ rest
584−
585−startsWithAlphaNum :: Text -> Bool
586−startsWithAlphaNum t =
587− case T.uncons t of
588− Just (c, _) | isAlphaNum c -> True
589− _ -> False
590−
591551setpos :: SourcePos -> Tok -> Tok
592552setpos spos (Tok _ tt txt) = Tok spos tt txt
593553
@@ -823,8 +783,7 @@ withRaw parser = do
823783
824784keyval :: PandocMonad m => LP m (Text, Text)
825785keyval = try $ do
826− key <- untokenize <$> many1 (notFollowedBy (symbol '=') >>
827− (symbol '-' <|> symbol '_' <|> satisfyTok isWordTok))
786+ Tok _ Word key <- satisfyTok isWordTok
828787 sp
829788 val <- option mempty $ do
830789 symbol '='
src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+66−140
@@ -1,4 +1,3 @@
1−{-# LANGUAGE ScopedTypeVariables #-}
21{-# LANGUAGE OverloadedStrings #-}
32module Text.Pandoc.Readers.LaTeX.SIunitx
43 ( siunitxCommands )
@@ -10,32 +9,27 @@ import Text.Pandoc.Class
109import Text.Pandoc.Parsing hiding (blankline, mathDisplay, mathInline,
1110 optional, space, spaces, withRaw, (<|>))
1211import Control.Applicative ((<|>))
13−import Control.Monad (void)
1412import qualified Data.Map as M
1513import Data.Char (isDigit)
1614import Data.Text (Text)
1715import qualified Data.Text as T
1816import Data.List (intersperse)
19−import qualified Data.Sequence as Seq
20−import Text.Pandoc.Walk (walk)
17+
2118
2219siunitxCommands :: PandocMonad m
2320 => LP m Inlines -> M.Map Text (LP m Inlines)
2421siunitxCommands tok = M.fromList
25− [ ("si", dosi tok)
22+ [ ("si", skipopts *> dosi tok)
2623 , ("SI", doSI tok)
2724 , ("SIrange", doSIrange True tok)
2825 , ("numrange", doSIrange False tok)
2926 , ("numlist", doSInumlist)
30− , ("SIlist", doSIlist tok)
3127 , ("num", doSInum)
3228 , ("ang", doSIang)
3329 ]
3430
3531dosi :: PandocMonad m => LP m Inlines -> LP m Inlines
36−dosi tok = do
37− options <- option [] keyvals
38− grouped (siUnit options tok) <|> siUnit options tok
32+dosi tok = grouped (siUnit tok) <|> siUnit tok
3933
4034-- converts e.g. \SI{1}[\$]{} to "$ 1" or \SI{1}{\euro} to "1 €"
4135doSI :: PandocMonad m => LP m Inlines -> LP m Inlines
@@ -70,50 +64,23 @@ doSInumlist = do
7064 mconcat (intersperse (str "," <> space) (init xs)) <>
7165 text ", & " <> last xs
7266
73−doSIlist :: PandocMonad m => LP m Inlines -> LP m Inlines
74−doSIlist tok = do
75− options <- option [] keyvals
76− nums <- map tonum . T.splitOn ";" . untokenize <$> braced
77− unit <- grouped (siUnit options tok) <|> siUnit options tok
78− let xs = map (<> (str "\xa0" <> unit)) nums
79− case xs of
80− [] -> return mempty
81− [x] -> return x
82− _ -> return $
83− mconcat (intersperse (str "," <> space) (init xs)) <>
84− text ", & " <> last xs
85−
8667parseNum :: Parser Text () Inlines
8768parseNum = (mconcat <$> many parseNumPart) <* eof
8869
89−minus :: Text
90−minus = "\x2212"
91−
92−hyphenToMinus :: Inline -> Inline
93−hyphenToMinus (Str t) = Str (T.replace "-" minus t)
94−hyphenToMinus x = x
95−
9670parseNumPart :: Parser Text () Inlines
9771parseNumPart =
9872 parseDecimalNum <|>
9973 parseComma <|>
10074 parsePlusMinus <|>
101− parsePM <|>
10275 parseI <|>
10376 parseExp <|>
10477 parseX <|>
10578 parseSpace
10679 where
107− parseDecimalNum, parsePlusMinus, parsePM,
108− parseComma, parseI, parseX,
109− parseExp, parseSpace :: Parser Text () Inlines
110− parseDecimalNum = try $ do
111− pref <- option mempty $ (mempty <$ char '+') <|> (minus <$ char '-')
112− basenum' <- many1 (satisfy (\c -> isDigit c || c == '.'))
113− let basenum = pref <> T.pack
114− (case basenum' of
115− '.':_ -> '0':basenum'
116− _ -> basenum')
80+ parseDecimalNum = do
81+ pref <- option mempty $ (mempty <$ char '+') <|> ("\x2212" <$ char '-')
82+ basenum <- (pref <>) . T.pack
83+ <$> many1 (satisfy (\c -> isDigit c || c == '.'))
11784 uncertainty <- option mempty $ T.pack <$> parseParens
11885 if T.null uncertainty
11986 then return $ str basenum
@@ -131,7 +98,6 @@ parseNumPart =
13198 | otherwise -> "." <> t
13299 parseComma = str "." <$ char ','
133100 parsePlusMinus = str "\xa0\xb1\xa0" <$ try (string "+-")
134− parsePM = str "\xa0\xb1\xa0" <$ try (string "\\pm")
135101 parseParens =
136102 char '(' *> many1 (satisfy (\c -> isDigit c || c == '.')) <* char ')'
137103 parseI = str "i" <$ char 'i'
@@ -144,14 +110,11 @@ doSIang :: PandocMonad m => LP m Inlines
144110doSIang = do
145111 skipopts
146112 ps <- T.splitOn ";" . untokenize <$> braced
147− let dropPlus t = case T.uncons t of
148− Just ('+',t') -> t'
149− _ -> t
150113 case ps ++ repeat "" of
151114 (d:m:s:_) -> return $
152− (if T.null d then mempty else str (dropPlus d) <> str "\xb0") <>
153− (if T.null m then mempty else str (dropPlus m) <> str "\x2032") <>
154− (if T.null s then mempty else str (dropPlus s) <> str "\x2033")
115+ (if T.null d then mempty else str d <> str "\xb0") <>
116+ (if T.null m then mempty else str m <> str "\x2032") <>
117+ (if T.null s then mempty else str s <> str "\x2033")
155118 _ -> return mempty
156119
157120-- converts e.g. \SIrange{100}{200}{\ms} to "100 ms--200 ms"
@@ -180,99 +143,40 @@ doSIrange includeUnits tok = do
180143emptyOr160 :: Inlines -> Inlines
181144emptyOr160 x = if x == mempty then x else str "\160"
182145
183−siUnit :: forall m. PandocMonad m => [(Text,Text)] -> LP m Inlines -> LP m Inlines
184−siUnit options tok = mconcat . intersperse (str "\xa0") <$> many1 siUnitPart
185− where
186− siUnitPart :: LP m Inlines
187− siUnitPart = try $ do
188− skipMany (void (symbol '.') <|> void (symbol '~') <|> spaces1)
189− x <- ((siPrefix <*> siBase)
190− <|> (do u <- siBase <|> tok
191− option u $ siSuffix <*> pure u))
192− option x (siInfix x)
193− siInfix :: Inlines -> LP m Inlines
194− siInfix u1 = try $
195− (do _ <- controlSeq "per"
196− u2 <- siUnitPart
197− let useSlash = lookup "per-mode" options == Just "symbol"
198− if useSlash
199− then return (u1 <> str "/" <> u2)
200− else return (u1 <> str "\xa0" <> negateExponent u2))
201− <|> (do _ <- symbol '/'
202− u2 <- siUnitPart
203− return (u1 <> str "/" <> u2))
204− siPrefix :: LP m (Inlines -> Inlines)
205− siPrefix =
206− (do _ <- controlSeq "square"
207− skipopts
208− return (<> superscript "2"))
209− <|> (do _ <- controlSeq "cubic"
210− skipopts
211− return (<> superscript "3"))
212− <|> (do _ <- controlSeq "raisetothe"
213− skipopts
214− n <- walk hyphenToMinus <$> tok
215− return (<> superscript n))
216− siSuffix :: LP m (Inlines -> Inlines)
217− siSuffix =
218− (do _ <- controlSeq "squared"
219− skipopts
220− return (<> superscript "2"))
221− <|> (do _ <- controlSeq "cubed"
222− skipopts
223− return (<> superscript "3"))
224− <|> (do _ <- controlSeq "tothe"
225− skipopts
226− n <- walk hyphenToMinus <$> tok
227− return (<> superscript n))
228− <|> (symbol '^' *> (do n <- walk hyphenToMinus <$> tok
229− return (<> superscript n)))
230− <|> (symbol '_' *> (do n <- walk hyphenToMinus <$> tok
231− return (<> subscript n)))
232− negateExponent :: Inlines -> Inlines
233− negateExponent ils =
234− case Seq.viewr (unMany ils) of
235− xs Seq.:> Superscript ss -> (Many xs) <>
236− superscript (str minus <> fromList ss)
237− _ -> ils <> superscript (str (minus <> "1"))
238− siBase :: LP m Inlines
239− siBase =
240− ((try
241− (do Tok _ (CtrlSeq name) _ <- anyControlSeq
242− case M.lookup name siUnitModifierMap of
243− Just il -> (il <>) <$> siBase
244− Nothing ->
245− case M.lookup name siUnitMap of
246− Just il -> pure il
247− Nothing -> fail "not a unit command"))
248− <|> (do Tok _ Word t <- satisfyTok isWordTok
249− return $ str t)
250− )
251−
252−siUnitModifierMap :: M.Map Text Inlines
253−siUnitModifierMap = M.fromList
254− [ ("atto", str "a")
255− , ("centi", str "c")
256− , ("deca", str "d")
257− , ("deci", str "d")
258− , ("deka", str "d")
259− , ("exa", str "E")
260− , ("femto", str "f")
261− , ("giga", str "G")
262− , ("hecto", str "h")
263− , ("kilo", str "k")
264− , ("mega", str "M")
265− , ("micro", str "μ")
266− , ("milli", str "m")
267− , ("nano", str "n")
268− , ("peta", str "P")
269− , ("pico", str "p")
270− , ("tera", str "T")
271− , ("yocto", str "y")
272− , ("yotta", str "Y")
273− , ("zepto", str "z")
274− , ("zetta", str "Z")
275− ]
146+siUnit :: PandocMonad m => LP m Inlines -> LP m Inlines
147+siUnit tok = try (do
148+ Tok _ (CtrlSeq name) _ <- anyControlSeq
149+ case name of
150+ "square" -> do
151+ unit <- siUnit tok
152+ return $ unit <> superscript "2"
153+ "cubic" -> do
154+ unit <- siUnit tok
155+ return $ unit <> superscript "3"
156+ "raisetothe" -> do
157+ n <- tok
158+ unit <- siUnit tok
159+ return $ unit <> superscript n
160+ _ ->
161+ case M.lookup name siUnitMap of
162+ Just il ->
163+ option il $
164+ choice
165+ [ (il <> superscript "2") <$ controlSeq "squared"
166+ , (il <> superscript "3") <$ controlSeq "cubed"
167+ , (\n -> il <> superscript n) <$> (controlSeq "tothe" *> tok)
168+ ]
169+ Nothing -> fail "not an siunit unit command")
170+ <|> (lookAhead anyControlSeq >> tok)
171+ <|> (do Tok _ Word t <- satisfyTok isWordTok
172+ return $ str t)
173+ <|> (symbol '^' *> (superscript <$> tok))
174+ <|> (symbol '_' *> (subscript <$> tok))
175+ <|> ("\xa0" <$ symbol '.')
176+ <|> ("\xa0" <$ symbol '~')
177+ <|> tok
178+ <|> (do Tok _ _ t <- anyTok
179+ return (str t))
276180
277181siUnitMap :: M.Map Text Inlines
278182siUnitMap = M.fromList
@@ -372,6 +276,7 @@ siUnitMap = M.fromList
372276 , ("arcsecond", str "″")
373277 , ("astronomicalunit", str "ua")
374278 , ("atomicmassunit", str "u")
279+ , ("atto", str "a")
375280 , ("bar", str "bar")
376281 , ("barn", str "b")
377282 , ("becquerel", str "Bq")
@@ -379,38 +284,51 @@ siUnitMap = M.fromList
379284 , ("bohr", emph (str "a") <> subscript (str "0"))
380285 , ("candela", str "cd")
381286 , ("celsius", str "°C")
287+ , ("centi", str "c")
382288 , ("clight", emph (str "c") <> subscript (str "0"))
383289 , ("coulomb", str "C")
384290 , ("dalton", str "Da")
385291 , ("day", str "d")
292+ , ("deca", str "d")
293+ , ("deci", str "d")
386294 , ("decibel", str "db")
387295 , ("degreeCelsius",str "°C")
388296 , ("degree", str "°")
297+ , ("deka", str "d")
389298 , ("electronmass", emph (str "m") <> subscript (str "e"))
390299 , ("electronvolt", str "eV")
391300 , ("elementarycharge", emph (str "e"))
301+ , ("exa", str "E")
392302 , ("farad", str "F")
303+ , ("femto", str "f")
304+ , ("giga", str "G")
393305 , ("gram", str "g")
394306 , ("gray", str "Gy")
395307 , ("hartree", emph (str "E") <> subscript (str "h"))
396308 , ("hectare", str "ha")
309+ , ("hecto", str "h")
397310 , ("henry", str "H")
398311 , ("hertz", str "Hz")
399312 , ("hour", str "h")
400313 , ("joule", str "J")
401314 , ("katal", str "kat")
402315 , ("kelvin", str "K")
316+ , ("kilo", str "k")
403317 , ("kilogram", str "kg")
404318 , ("knot", str "kn")
405319 , ("liter", str "L")
406320 , ("litre", str "l")
407321 , ("lumen", str "lm")
408322 , ("lux", str "lx")
323+ , ("mega", str "M")
409324 , ("meter", str "m")
410325 , ("metre", str "m")
326+ , ("micro", str "μ")
327+ , ("milli", str "m")
411328 , ("minute", str "min")
412329 , ("mmHg", str "mmHg")
413330 , ("mole", str "mol")
331+ , ("nano", str "n")
414332 , ("nauticalmile", str "M")
415333 , ("neper", str "Np")
416334 , ("newton", str "N")
@@ -418,17 +336,25 @@ siUnitMap = M.fromList
418336 , ("Pa", str "Pa")
419337 , ("pascal", str "Pa")
420338 , ("percent", str "%")
339+ , ("per", str "/")
340+ , ("peta", str "P")
341+ , ("pico", str "p")
421342 , ("planckbar", emph (str "\x210f"))
422343 , ("radian", str "rad")
423344 , ("second", str "s")
424345 , ("siemens", str "S")
425346 , ("sievert", str "Sv")
426347 , ("steradian", str "sr")
348+ , ("tera", str "T")
427349 , ("tesla", str "T")
428350 , ("tonne", str "t")
429351 , ("volt", str "V")
430352 , ("watt", str "W")
431353 , ("weber", str "Wb")
354+ , ("yocto", str "y")
355+ , ("yotta", str "Y")
356+ , ("zepto", str "z")
357+ , ("zetta", str "Z")
432358 ]
433359
434360
src/Text/Pandoc/Readers/LaTeX/Types.hs+1−14
@@ -1,4 +1,3 @@
1−{-# LANGUAGE FlexibleInstances #-}
21{- |
32 Module : Text.Pandoc.Readers.LaTeX.Types
43 Copyright : Copyright (C) 2017-2021 John MacFarlane
@@ -19,9 +18,7 @@ module Text.Pandoc.Readers.LaTeX.Types ( Tok(..)
1918 )
2019where
2120import Data.Text (Text)
22−import Text.Parsec.Pos (SourcePos, sourceName)
23−import Text.Pandoc.Sources
24−import Data.List (groupBy)
21+import Text.Parsec.Pos (SourcePos)
2522
2623data TokType = CtrlSeq Text | Spaces | Newline | Symbol | Word | Comment |
2724 Esc1 | Esc2 | Arg Int
@@ -30,16 +27,6 @@ data TokType = CtrlSeq Text | Spaces | Newline | Symbol | Word | Comment |
3027data Tok = Tok SourcePos TokType Text
3128 deriving (Eq, Ord, Show)
3229
33−instance ToSources [Tok] where
34− toSources = Sources
35− . map (\ts -> case ts of
36− Tok p _ _ : _ -> (p, mconcat $ map tokToText ts)
37− _ -> error "toSources [Tok] encountered empty group")
38− . groupBy (\(Tok p1 _ _) (Tok p2 _ _) -> sourceName p1 == sourceName p2)
39−
40−tokToText :: Tok -> Text
41−tokToText (Tok _ _ t) = t
42−
4330data ExpansionPoint = ExpandWhenDefined | ExpandWhenUsed
4431 deriving (Eq, Ord, Show)
4532
src/Text/Pandoc/Readers/Man.hs+11−18
@@ -20,7 +20,7 @@ import Control.Monad (liftM, mzero, guard, void)
2020import Control.Monad.Trans (lift)
2121import Control.Monad.Except (throwError)
2222import Data.Maybe (catMaybes, isJust)
23−import Data.List (intersperse)
23+import Data.List (intersperse, intercalate)
2424import qualified Data.Text as T
2525import Text.Pandoc.Builder as B
2626import Text.Pandoc.Class.PandocMonad (PandocMonad(..), report)
@@ -29,8 +29,9 @@ import Text.Pandoc.Logging (LogMessage(..))
2929import Text.Pandoc.Options
3030import Text.Pandoc.Parsing
3131import Text.Pandoc.Walk (query)
32−import Text.Pandoc.Shared (mapLeft)
32+import Text.Pandoc.Shared (crFilter, mapLeft)
3333import Text.Pandoc.Readers.Roff -- TODO explicit imports
34+import Text.Parsec hiding (tokenPrim)
3435import qualified Text.Parsec as Parsec
3536import Text.Parsec.Pos (updatePosString)
3637import qualified Data.Foldable as Foldable
@@ -49,20 +50,13 @@ type ManParser m = ParserT [RoffToken] ManState m
4950
5051
5152-- | Read man (troff) from an input string and return a Pandoc document.
52−readMan :: (PandocMonad m, ToSources a)
53− => ReaderOptions
54− -> a
55− -> m Pandoc
56−readMan opts s = do
57− let Sources inps = toSources s
58− tokenz <- mconcat <$> mapM (uncurry lexRoff) inps
53+readMan :: PandocMonad m => ReaderOptions -> T.Text -> m Pandoc
54+readMan opts txt = do
55+ tokenz <- lexRoff (initialPos "input") (crFilter txt)
5956 let state = def {readerOptions = opts} :: ManState
60− let fixError (PandocParsecError _ e) = PandocParsecError (Sources inps) e
61− fixError e = e
6257 eitherdoc <- readWithMTokens parseMan state
6358 (Foldable.toList . unRoffTokens $ tokenz)
64− either (throwError . fixError) return eitherdoc
65−
59+ either throwError return eitherdoc
6660
6761readWithMTokens :: PandocMonad m
6862 => ParserT [RoffToken] ManState m a -- ^ parser
@@ -70,10 +64,9 @@ readWithMTokens :: PandocMonad m
7064 -> [RoffToken] -- ^ input
7165 -> m (Either PandocError a)
7266readWithMTokens parser state input =
73− let leftF = PandocParsecError mempty
67+ let leftF = PandocParsecError . T.pack . intercalate "\n" $ show <$> input
7468 in mapLeft leftF `liftM` runParserT parser state "source" input
7569
76−
7770parseMan :: PandocMonad m => ManParser m Pandoc
7871parseMan = do
7972 bs <- many parseBlock <* eof
@@ -96,7 +89,7 @@ parseBlock = choice [ parseList
9689
9790parseTable :: PandocMonad m => ManParser m Blocks
9891parseTable = do
99− updateState $ \st -> st { tableCellsPlain = True }
92+ modifyState $ \st -> st { tableCellsPlain = True }
10093 let isTbl Tbl{} = True
10194 isTbl _ = False
10295 Tbl _opts rows pos <- msatisfy isTbl
@@ -142,7 +135,7 @@ parseTable = do
142135 case res' of
143136 Left _ -> Prelude.fail "Could not parse table cell"
144137 Right x -> do
145− updateState $ \s -> s{ tableCellsPlain = False }
138+ modifyState $ \s -> s{ tableCellsPlain = False }
146139 return x
147140 Right x -> return x
148141
@@ -229,7 +222,7 @@ parseTitle = do
229222 setMeta "section" (linePartsToInlines y)
230223 [x] -> setMeta "title" (linePartsToInlines x)
231224 [] -> id
232− updateState $ \st -> st{ metadata = adjustMeta $ metadata st }
225+ modifyState $ \st -> st{ metadata = adjustMeta $ metadata st }
233226 return mempty
234227
235228linePartsToInlines :: [LinePart] -> Inlines
src/Text/Pandoc/Readers/Markdown.hs+72−92
@@ -21,17 +21,15 @@ module Text.Pandoc.Readers.Markdown (
2121import Control.Monad
2222import Control.Monad.Except (throwError)
2323import Data.Char (isAlphaNum, isPunctuation, isSpace)
24−import Data.List (transpose, elemIndex, sortOn, foldl')
25−import Data.List.NonEmpty (NonEmpty(..))
24+import Data.List (transpose, elemIndex, sortOn)
2625import qualified Data.Map as M
2726import Data.Maybe
2827import qualified Data.Set as Set
2928import Data.Text (Text)
3029import qualified Data.Text as T
30+import qualified Data.Text.Lazy as TL
3131import qualified Data.ByteString.Lazy as BL
32−import System.FilePath (addExtension, takeExtension, takeDirectory)
33−import qualified System.FilePath.Windows as Windows
34−import qualified System.FilePath.Posix as Posix
32+import System.FilePath (addExtension, takeExtension)
3533import Text.HTML.TagSoup hiding (Row)
3634import Text.Pandoc.Builder (Blocks, Inlines)
3735import qualified Text.Pandoc.Builder as B
@@ -47,22 +45,20 @@ import Text.Pandoc.Readers.HTML (htmlInBalanced, htmlTag, isBlockTag,
4745 isCommentTag, isInlineTag, isTextTag)
4846import Text.Pandoc.Readers.LaTeX (applyMacros, rawLaTeXBlock, rawLaTeXInline)
4947import Text.Pandoc.Shared
48+import qualified Text.Pandoc.UTF8 as UTF8
5049import Text.Pandoc.XML (fromEntities)
51−import Text.Pandoc.Readers.Metadata (yamlBsToMeta, yamlBsToRefs, yamlMetaBlock)
52−-- import Debug.Trace (traceShowId)
50+import Text.Pandoc.Readers.Metadata (yamlBsToMeta, yamlBsToRefs)
5351
54−type MarkdownParser m = ParserT Sources ParserState m
55−
56−type F = Future ParserState
52+type MarkdownParser m = ParserT Text ParserState m
5753
5854-- | Read markdown from an input string and return a Pandoc document.
59−readMarkdown :: (PandocMonad m, ToSources a)
55+readMarkdown :: PandocMonad m
6056 => ReaderOptions -- ^ Reader options
61− -> a -- ^ Input
57+ -> Text -- ^ String to parse (assuming @'\n'@ line endings)
6258 -> m Pandoc
6359readMarkdown opts s = do
6460 parsed <- readWithM parseMarkdown def{ stateOptions = opts }
65− (ensureFinalNewlines 3 (toSources s))
61+ (crFilter s <> "\n\n")
6662 case parsed of
6763 Right result -> return result
6864 Left e -> throwError e
@@ -83,7 +79,7 @@ yamlToMeta opts mbfp bstr = do
8379 meta <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks) bstr
8480 setPosition oldPos
8581 return $ runF meta defaultParserState
86− parsed <- readWithM parser def{ stateOptions = opts } ("" :: Text)
82+ parsed <- readWithM parser def{ stateOptions = opts } ""
8783 case parsed of
8884 Right result -> return result
8985 Left e -> throwError e
@@ -99,12 +95,14 @@ yamlToRefs :: PandocMonad m
9995 -> m [MetaValue]
10096yamlToRefs idpred opts mbfp bstr = do
10197 let parser = do
98+ oldPos <- getPosition
10299 case mbfp of
103100 Nothing -> return ()
104101 Just fp -> setPosition $ initialPos fp
105102 refs <- yamlBsToRefs (fmap B.toMetaValue <$> parseBlocks) idpred bstr
103+ setPosition oldPos
106104 return $ runF refs defaultParserState
107− parsed <- readWithM parser def{ stateOptions = opts } ("" :: Text)
105+ parsed <- readWithM parser def{ stateOptions = opts } ""
108106 case parsed of
109107 Right result -> return result
110108 Left e -> throwError e
@@ -147,14 +145,14 @@ inList = do
147145 ctx <- stateParserContext <$> getState
148146 guard (ctx == ListItemState)
149147
150−spnl :: PandocMonad m => ParserT Sources st m ()
148+spnl :: PandocMonad m => ParserT Text st m ()
151149spnl = try $ do
152150 skipSpaces
153151 optional newline
154152 skipSpaces
155153 notFollowedBy (char '\n')
156154
157−spnl' :: PandocMonad m => ParserT Sources st m Text
155+spnl' :: PandocMonad m => ParserT Text st m Text
158156spnl' = try $ do
159157 xs <- many spaceChar
160158 ys <- option "" $ try $ (:) <$> newline
@@ -273,13 +271,25 @@ pandocTitleBlock = do
273271 $ nullMeta
274272 updateState $ \st -> st{ stateMeta' = stateMeta' st <> meta' }
275273
276−yamlMetaBlock' :: PandocMonad m => MarkdownParser m (F Blocks)
277−yamlMetaBlock' = do
274+yamlMetaBlock :: PandocMonad m => MarkdownParser m (F Blocks)
275+yamlMetaBlock = do
278276 guardEnabled Ext_yaml_metadata_block
279− newMetaF <- yamlMetaBlock (fmap B.toMetaValue <$> parseBlocks)
280− -- Since `<>` is left-biased, existing values are not touched:
281− updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }
282− return mempty
277+ try $ do
278+ string "---"
279+ blankline
280+ notFollowedBy blankline -- if --- is followed by a blank it's an HRULE
281+ rawYamlLines <- manyTill anyLine stopLine
282+ -- by including --- and ..., we allow yaml blocks with just comments:
283+ let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))
284+ optional blanklines
285+ newMetaF <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks)
286+ $ UTF8.fromTextLazy $ TL.fromStrict rawYaml
287+ -- Since `<>` is left-biased, existing values are not touched:
288+ updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }
289+ return mempty
290+
291+stopLine :: PandocMonad m => MarkdownParser m ()
292+stopLine = try $ (string "---" <|> string "...") >> blankline >> return ()
283293
284294mmdTitleBlock :: PandocMonad m => MarkdownParser m ()
285295mmdTitleBlock = do
@@ -299,7 +309,7 @@ kvPair allowEmpty = try $ do
299309 (try $ newline >> lookAhead (blankline <|> nonspaceChar))
300310 guard $ allowEmpty || not (T.null val)
301311 let key' = T.concat $ T.words $ T.toLower key
302− let val' = MetaInlines $ B.toList $ B.text val
312+ let val' = MetaBlocks $ B.toList $ B.plain $ B.text val
303313 return (key',val')
304314
305315parseMarkdown :: PandocMonad m => MarkdownParser m Pandoc
@@ -338,9 +348,7 @@ referenceKey = try $ do
338348 notFollowedBy' (() <$ reference)
339349 many1Char $ notFollowedBy space >> litChar
340350 let betweenAngles = try $ char '<' >> manyTillChar litChar (char '>')
341− rebase <- option False (True <$ guardEnabled Ext_rebase_relative_paths)
342− src <- (if rebase then rebasePath pos else id) <$>
343− (try betweenAngles <|> sourceURL)
351+ src <- try betweenAngles <|> sourceURL
344352 tit <- option "" referenceTitle
345353 attr <- option nullAttr $ try $
346354 do guardEnabled Ext_link_attributes
@@ -349,7 +357,7 @@ referenceKey = try $ do
349357 addKvs <- option [] $ guardEnabled Ext_mmd_link_attributes
350358 >> many (try $ spnl >> keyValAttr)
351359 blanklines
352− let attr' = extractIdClass $ foldl' (\x f -> f x) attr addKvs
360+ let attr' = extractIdClass $ foldl (\x f -> f x) attr addKvs
353361 target = (escapeURI $ trimr src, tit)
354362 st <- getState
355363 let oldkeys = stateKeys st
@@ -445,7 +453,7 @@ block :: PandocMonad m => MarkdownParser m (F Blocks)
445453block = do
446454 res <- choice [ mempty <$ blanklines
447455 , codeBlockFenced
448− , yamlMetaBlock'
456+ , yamlMetaBlock
449457 -- note: bulletList needs to be before header because of
450458 -- the possibility of empty list items: -
451459 , bulletList
@@ -571,7 +579,7 @@ registerImplicitHeader raw attr@(ident, _, _)
571579-- hrule block
572580--
573581
574−hrule :: PandocMonad m => ParserT Sources st m (F Blocks)
582+hrule :: PandocMonad m => ParserT Text st m (F Blocks)
575583hrule = try $ do
576584 skipSpaces
577585 start <- satisfy isHruleChar
@@ -591,7 +599,7 @@ indentedLine = indentSpaces >> anyLineNewline
591599blockDelimiter :: PandocMonad m
592600 => (Char -> Bool)
593601 -> Maybe Int
594− -> ParserT Sources ParserState m Int
602+ -> ParserT Text ParserState m Int
595603blockDelimiter f len = try $ do
596604 skipNonindentSpaces
597605 c <- lookAhead (satisfy f)
@@ -605,7 +613,7 @@ attributes = try $ do
605613 spnl
606614 attrs <- many (attribute <* spnl)
607615 char '}'
608− return $ foldl' (\x f -> f x) nullAttr attrs
616+ return $ foldl (\x f -> f x) nullAttr attrs
609617
610618attribute :: PandocMonad m => MarkdownParser m (Attr -> Attr)
611619attribute = identifierAttr <|> classAttr <|> keyValAttr <|> specialAttr
@@ -735,7 +743,7 @@ lhsCodeBlockBirdWith c = try $ do
735743 blanklines
736744 return $ T.intercalate "\n" lns'
737745
738−birdTrackLine :: PandocMonad m => Char -> ParserT Sources st m Text
746+birdTrackLine :: PandocMonad m => Char -> ParserT Text st m Text
739747birdTrackLine c = try $ do
740748 char c
741749 -- allow html tags on left margin:
@@ -1028,7 +1036,7 @@ para = try $ do
10281036 option (B.plain <$> result)
10291037 $ try $ do
10301038 newline
1031− (mempty <$ blanklines)
1039+ (blanklines >> return mempty)
10321040 <|> (guardDisabled Ext_blank_before_blockquote >> () <$ lookAhead blockQuote)
10331041 <|> (guardEnabled Ext_backtick_code_blocks >> () <$ lookAhead codeBlockFenced)
10341042 <|> (guardDisabled Ext_blank_before_header >> () <$ lookAhead header)
@@ -1173,7 +1181,7 @@ lineBlock = do
11731181-- and the length including trailing space.
11741182dashedLine :: PandocMonad m
11751183 => Char
1176− -> ParserT Sources st m (Int, Int)
1184+ -> ParserT Text st m (Int, Int)
11771185dashedLine ch = do
11781186 dashes <- many1 (char ch)
11791187 sp <- many spaceChar
@@ -1242,7 +1250,7 @@ rawTableLine :: PandocMonad m
12421250 -> MarkdownParser m [Text]
12431251rawTableLine indices = do
12441252 notFollowedBy' (blanklines' <|> tableFooter)
1245− line <- anyLine
1253+ line <- take1WhileP (/='\n') <* newline
12461254 return $ map trim $ tail $
12471255 splitTextByIndices (init indices) line
12481256
@@ -1356,7 +1364,7 @@ pipeTable = try $ do
13561364 lines' <- many pipeTableRow
13571365 let lines'' = map (take (length aligns) <$>) lines'
13581366 let maxlength = maximum $
1359− fmap (\x -> T.length . stringify $ runF x def) (heads' :| lines'')
1367+ map (\x -> T.length . stringify $ runF x def) (heads' : lines'')
13601368 numColumns <- getOption readerColumns
13611369 let widths = if maxlength > numColumns
13621370 then map (\len ->
@@ -1393,7 +1401,7 @@ pipeTableCell =
13931401 return $ B.plain <$> result)
13941402 <|> return mempty
13951403
1396−pipeTableHeaderPart :: PandocMonad m => ParserT Sources st m (Alignment, Int)
1404+pipeTableHeaderPart :: PandocMonad m => ParserT Text st m (Alignment, Int)
13971405pipeTableHeaderPart = try $ do
13981406 skipMany spaceChar
13991407 left <- optionMaybe (char ':')
@@ -1409,14 +1417,10 @@ pipeTableHeaderPart = try $ do
14091417 (Just _,Just _) -> AlignCenter, len)
14101418
14111419-- Succeed only if current line contains a pipe.
1412−scanForPipe :: PandocMonad m => ParserT Sources st m ()
1420+scanForPipe :: PandocMonad m => ParserT Text st m ()
14131421scanForPipe = do
1414− Sources inps <- getInput
1415− let ln = case inps of
1416− [] -> ""
1417− ((_,t):(_,t'):_) | T.null t -> t'
1418− ((_,t):_) -> t
1419− case T.break (\c -> c == '\n' || c == '|') ln of
1422+ inp <- getInput
1423+ case T.break (\c -> c == '\n' || c == '|') inp of
14201424 (_, T.uncons -> Just ('|', _)) -> return ()
14211425 _ -> mzero
14221426
@@ -1499,7 +1503,7 @@ inline = do
14991503 '$' -> math
15001504 '~' -> strikeout <|> subscript
15011505 '<' -> autoLink <|> spanHtml <|> rawHtmlInline <|> ltSign
1502− '\\' -> math <|> escapedNewline <|> escapedChar <|> rawLaTeXInline'
1506+ '\\' -> escapedNewline <|> escapedChar <|> rawLaTeXInline'
15031507 '@' -> cite <|> exampleRef
15041508 '"' -> smart
15051509 '\'' -> smart
@@ -1710,13 +1714,13 @@ whitespace = spaceChar >> return <$> (lb <|> regsp) <?> "whitespace"
17101714 where lb = spaceChar >> skipMany spaceChar >> option B.space (endline >> return B.linebreak)
17111715 regsp = skipMany spaceChar >> return B.space
17121716
1713−nonEndline :: PandocMonad m => ParserT Sources st m Char
1717+nonEndline :: PandocMonad m => ParserT Text st m Char
17141718nonEndline = satisfy (/='\n')
17151719
17161720str :: PandocMonad m => MarkdownParser m (F Inlines)
17171721str = do
17181722 result <- mconcat <$> many1
1719− ( T.pack <$> (many1 alphaNum)
1723+ ( take1WhileP isAlphaNum
17201724 <|> "." <$ try (char '.' <* notFollowedBy (char '.')) )
17211725 updateLastStrPos
17221726 (do guardEnabled Ext_smart
@@ -1840,12 +1844,9 @@ regLink :: PandocMonad m
18401844 -> MarkdownParser m (F Inlines)
18411845regLink constructor lab = try $ do
18421846 (src, tit) <- source
1843− rebase <- option False (True <$ guardEnabled Ext_rebase_relative_paths)
1844− pos <- getPosition
1845− let src' = if rebase then rebasePath pos src else src
18461847 attr <- option nullAttr $
18471848 guardEnabled Ext_link_attributes >> attributes
1848− return $ constructor attr src' tit <$> lab
1849+ return $ constructor attr src tit <$> lab
18491850
18501851-- a link like [this][ref] or [this][] or [this]
18511852referenceLink :: PandocMonad m
@@ -1885,8 +1886,7 @@ referenceLink constructor (lab, raw) = do
18851886 Just ((src, tit), _) -> constructor nullAttr src tit <$> lab
18861887 Nothing -> makeFallback
18871888 else makeFallback
1888− Just ((src,tit), attr) ->
1889− constructor attr src tit <$> lab
1889+ Just ((src,tit), attr) -> constructor attr src tit <$> lab
18901890
18911891dropBrackets :: Text -> Text
18921892dropBrackets = dropRB . dropLB
@@ -1919,33 +1919,15 @@ autoLink = try $ do
19191919 return $ return $ B.linkWith attr (src <> escapeURI extra) ""
19201920 (B.str $ orig <> extra)
19211921
1922−-- | Rebase a relative path, by adding the (relative) directory
1923−-- of the containing source position. Absolute links and URLs
1924−-- are untouched.
1925−rebasePath :: SourcePos -> Text -> Text
1926−rebasePath pos path = do
1927− let fp = sourceName pos
1928− isFragment = T.take 1 path == "#"
1929− path' = T.unpack path
1930− isAbsolutePath = Posix.isAbsolute path' || Windows.isAbsolute path'
1931− in if T.null path || isFragment || isAbsolutePath || isURI path
1932− then path
1933− else
1934− case takeDirectory fp of
1935− "" -> path
1936− "." -> path
1937− d -> T.pack d <> "/" <> path
1938−
19391922image :: PandocMonad m => MarkdownParser m (F Inlines)
19401923image = try $ do
19411924 char '!'
19421925 (lab,raw) <- reference
19431926 defaultExt <- getOption readerDefaultImageExtension
1944− let constructor attr' src =
1945− case takeExtension (T.unpack src) of
1946− "" -> B.imageWith attr' (T.pack $ addExtension (T.unpack src)
1947− $ T.unpack defaultExt)
1948− _ -> B.imageWith attr' src
1927+ let constructor attr' src = case takeExtension (T.unpack src) of
1928+ "" -> B.imageWith attr' (T.pack $ addExtension (T.unpack src)
1929+ $ T.unpack defaultExt)
1930+ _ -> B.imageWith attr' src
19491931 regLink constructor lab <|> referenceLink constructor (lab,raw)
19501932
19511933note :: PandocMonad m => MarkdownParser m (F Inlines)
@@ -1991,7 +1973,7 @@ rawLaTeXInline' = do
19911973 s <- rawLaTeXInline
19921974 return $ return $ B.rawInline "tex" s -- "tex" because it might be context
19931975
1994−rawConTeXtEnvironment :: PandocMonad m => ParserT Sources st m Text
1976+rawConTeXtEnvironment :: PandocMonad m => ParserT Text st m Text
19951977rawConTeXtEnvironment = try $ do
19961978 string "\\start"
19971979 completion <- inBrackets (letter <|> digit <|> spaceChar)
@@ -2000,7 +1982,7 @@ rawConTeXtEnvironment = try $ do
20001982 (try $ string "\\stop" >> textStr completion)
20011983 return $ "\\start" <> completion <> T.concat contents <> "\\stop" <> completion
20021984
2003−inBrackets :: PandocMonad m => ParserT Sources st m Char -> ParserT Sources st m Text
1985+inBrackets :: PandocMonad m => ParserT Text st m Char -> ParserT Text st m Text
20041986inBrackets parser = do
20051987 char '['
20061988 contents <- manyChar parser
@@ -2120,7 +2102,7 @@ cite = do
21202102
21212103textualCite :: PandocMonad m => MarkdownParser m (F Inlines)
21222104textualCite = try $ do
2123− (suppressAuthor, key) <- citeKey True
2105+ (suppressAuthor, key) <- citeKey
21242106 -- If this is a reference to an earlier example list item,
21252107 -- then don't parse it as a citation. If the example list
21262108 -- item comes later, we'll parse it here and figure out in
@@ -2200,7 +2182,7 @@ prefix = trimInlinesF . mconcat <$>
22002182 manyTill inline (char ']'
22012183 <|> lookAhead
22022184 (try $ do optional (try (char ';' >> spnl))
2203− citeKey True
2185+ citeKey
22042186 return ']'))
22052187
22062188citeList :: PandocMonad m => MarkdownParser m (F [Citation])
@@ -2209,7 +2191,7 @@ citeList = fmap sequence $ sepBy1 citation (try $ char ';' >> spnl)
22092191citation :: PandocMonad m => MarkdownParser m (F Citation)
22102192citation = try $ do
22112193 pref <- prefix
2212− (suppress_author, key) <- citeKey True
2194+ (suppress_author, key) <- citeKey
22132195 suff <- suffix
22142196 noteNum <- stateNoteNumber <$> getState
22152197 return $ do
@@ -2228,30 +2210,28 @@ citation = try $ do
22282210smart :: PandocMonad m => MarkdownParser m (F Inlines)
22292211smart = do
22302212 guardEnabled Ext_smart
2231− doubleQuoted <|> singleQuoted <|> (return <$> doubleCloseQuote) <|>
2232− (return <$> apostrophe) <|> (return <$> dash) <|> (return <$> ellipses)
2213+ doubleQuoted <|> singleQuoted <|>
2214+ choice (map (return <$>) [apostrophe, dash, ellipses])
22332215
22342216singleQuoted :: PandocMonad m => MarkdownParser m (F Inlines)
2235−singleQuoted = do
2217+singleQuoted = try $ do
22362218 singleQuoteStart
2237− (try (withQuoteContext InSingleQuote $
2219+ withQuoteContext InSingleQuote $
22382220 fmap B.singleQuoted . trimInlinesF . mconcat <$>
2239− many1Till inline singleQuoteEnd))
2240− <|> (return (return (B.str "\8217")))
2221+ many1Till inline singleQuoteEnd
22412222
22422223-- doubleQuoted will handle regular double-quoted sections, as well
22432224-- as dialogues with an open double-quote without a close double-quote
22442225-- in the same paragraph.
22452226doubleQuoted :: PandocMonad m => MarkdownParser m (F Inlines)
2246−doubleQuoted = do
2227+doubleQuoted = try $ do
22472228 doubleQuoteStart
2248− (try (withQuoteContext InDoubleQuote $
2229+ withQuoteContext InDoubleQuote $
22492230 fmap B.doubleQuoted . trimInlinesF . mconcat <$>
2250− many1Till inline doubleQuoteEnd))
2251− <|> (return (return (B.str "\8220")))
2231+ many1Till inline doubleQuoteEnd
22522232
22532233toRow :: [Blocks] -> Row
22542234toRow = Row nullAttr . map B.simpleCell
22552235
22562236toHeaderRow :: [Blocks] -> [Row]
2257−toHeaderRow l = [toRow l | not (null l) && not (all null l)]
2237+toHeaderRow l = [toRow l | not (null l)]
src/Text/Pandoc/Readers/MediaWiki.hs+7−16
@@ -36,18 +36,17 @@ import Text.Pandoc.Logging
3636import Text.Pandoc.Options
3737import Text.Pandoc.Parsing hiding (nested)
3838import Text.Pandoc.Readers.HTML (htmlTag, isBlockTag, isCommentTag)
39−import Text.Pandoc.Shared (safeRead, stringify, stripTrailingNewlines,
39+import Text.Pandoc.Shared (crFilter, safeRead, stringify, stripTrailingNewlines,
4040 trim, splitTextBy, tshow)
4141import Text.Pandoc.Walk (walk)
4242import Text.Pandoc.XML (fromEntities)
4343
4444-- | Read mediawiki from an input string and return a Pandoc document.
45−readMediaWiki :: (PandocMonad m, ToSources a)
46− => ReaderOptions
47− -> a
45+readMediaWiki :: PandocMonad m
46+ => ReaderOptions -- ^ Reader options
47+ -> Text -- ^ String to parse (assuming @'\n'@ line endings)
4848 -> m Pandoc
4949readMediaWiki opts s = do
50− let sources = toSources s
5150 parsed <- readWithM parseMediaWiki MWState{ mwOptions = opts
5251 , mwMaxNestingLevel = 4
5352 , mwNextLinkNumber = 1
@@ -56,7 +55,7 @@ readMediaWiki opts s = do
5655 , mwLogMessages = []
5756 , mwInTT = False
5857 }
59− sources
58+ (crFilter s <> "\n")
6059 case parsed of
6160 Right result -> return result
6261 Left e -> throwError e
@@ -70,7 +69,7 @@ data MWState = MWState { mwOptions :: ReaderOptions
7069 , mwInTT :: Bool
7170 }
7271
73−type MWParser m = ParserT Sources MWState m
72+type MWParser m = ParserT Text MWState m
7473
7574instance HasReaderOptions MWState where
7675 extractReaderOptions = mwOptions
@@ -113,14 +112,12 @@ newBlockTags = ["haskell","syntaxhighlight","source","gallery","references"]
113112isBlockTag' :: Tag Text -> Bool
114113isBlockTag' tag@(TagOpen t _) = (isBlockTag tag || t `elem` newBlockTags) &&
115114 t `notElem` eitherBlockOrInline
116−isBlockTag' (TagClose "ref") = True -- needed so 'special' doesn't parse it
117115isBlockTag' tag@(TagClose t) = (isBlockTag tag || t `elem` newBlockTags) &&
118116 t `notElem` eitherBlockOrInline
119117isBlockTag' tag = isBlockTag tag
120118
121119isInlineTag' :: Tag Text -> Bool
122120isInlineTag' (TagComment _) = True
123−isInlineTag' (TagClose "ref") = False -- see below inlineTag
124121isInlineTag' t = not (isBlockTag' t)
125122
126123eitherBlockOrInline :: [Text]
@@ -557,17 +554,11 @@ variable = try $ do
557554 contents <- manyTillChar anyChar (try $ string "}}}")
558555 return $ "{{{" <> contents <> "}}}"
559556
560−singleParaToPlain :: Blocks -> Blocks
561−singleParaToPlain bs =
562− case B.toList bs of
563− [Para ils] -> B.fromList [Plain ils]
564− _ -> bs
565−
566557inlineTag :: PandocMonad m => MWParser m Inlines
567558inlineTag = do
568559 (tag, _) <- lookAhead $ htmlTag isInlineTag'
569560 case tag of
570− TagOpen "ref" _ -> B.note . singleParaToPlain <$> blocksInTags "ref"
561+ TagOpen "ref" _ -> B.note . B.plain <$> inlinesInTags "ref"
571562 TagOpen "nowiki" _ -> try $ do
572563 (_,raw) <- htmlTag (~== tag)
573564 if T.any (== '/') raw
src/Text/Pandoc/Readers/Metadata.hs+17−44
@@ -14,7 +14,6 @@ Parse YAML/JSON metadata to 'Pandoc' 'Meta'.
1414module Text.Pandoc.Readers.Metadata (
1515 yamlBsToMeta,
1616 yamlBsToRefs,
17− yamlMetaBlock,
1817 yamlMap ) where
1918
2019import Control.Monad
@@ -31,13 +30,11 @@ import Text.Pandoc.Definition
3130import Text.Pandoc.Error
3231import Text.Pandoc.Parsing hiding (tableWith)
3332import Text.Pandoc.Shared
34−import qualified Data.Text.Lazy as TL
35−import qualified Text.Pandoc.UTF8 as UTF8
3633
37−yamlBsToMeta :: (PandocMonad m, HasLastStrPosition st)
38− => ParserT Sources st m (Future st MetaValue)
34+yamlBsToMeta :: PandocMonad m
35+ => ParserT Text ParserState m (F MetaValue)
3936 -> BL.ByteString
40− -> ParserT Sources st m (Future st Meta)
37+ -> ParserT Text ParserState m (F Meta)
4138yamlBsToMeta pMetaValue bstr = do
4239 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of
4340 Right (YAML.Doc (YAML.Mapping _ _ o):_)
@@ -45,9 +42,6 @@ yamlBsToMeta pMetaValue bstr = do
4542 Right [] -> return . return $ mempty
4643 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]
4744 -> return . return $ mempty
48− -- the following is what we get from a comment:
49− Right [YAML.Doc (YAML.Scalar _ (YAML.SUnknown _ ""))]
50− -> return . return $ mempty
5145 Right _ -> Prelude.fail "expected YAML object"
5246 Left (yamlpos, err')
5347 -> do pos <- getPosition
@@ -69,11 +63,11 @@ lookupYAML t (YAML.Mapping _ _ m) =
6963lookupYAML _ _ = Nothing
7064
7165-- Returns filtered list of references.
72−yamlBsToRefs :: (PandocMonad m, HasLastStrPosition st)
73− => ParserT Sources st m (Future st MetaValue)
66+yamlBsToRefs :: PandocMonad m
67+ => ParserT Text ParserState m (F MetaValue)
7468 -> (Text -> Bool) -- ^ Filter for id
7569 -> BL.ByteString
76− -> ParserT Sources st m (Future st [MetaValue])
70+ -> ParserT Text ParserState m (F [MetaValue])
7771yamlBsToRefs pMetaValue idpred bstr =
7872 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of
7973 Right (YAML.Doc o@YAML.Mapping{}:_)
@@ -101,12 +95,8 @@ yamlBsToRefs pMetaValue idpred bstr =
10195 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]
10296 -> return . return $ mempty
10397 Right _ -> Prelude.fail "expecting YAML object"
104− Left (yamlpos, err')
105− -> do pos <- getPosition
106− setPosition $ incSourceLine
107− (setSourceColumn pos (YE.posColumn yamlpos))
108− (YE.posLine yamlpos - 1)
109− Prelude.fail err'
98+ Left (_pos, err')
99+ -> Prelude.fail err'
110100
111101
112102nodeToKey :: YAML.Node YE.Pos -> Maybe Text
@@ -114,10 +104,10 @@ nodeToKey (YAML.Scalar _ (YAML.SStr t)) = Just t
114104nodeToKey (YAML.Scalar _ (YAML.SUnknown _ t)) = Just t
115105nodeToKey _ = Nothing
116106
117−normalizeMetaValue :: (PandocMonad m, HasLastStrPosition st)
118− => ParserT Sources st m (Future st MetaValue)
107+normalizeMetaValue :: PandocMonad m
108+ => ParserT Text ParserState m (F MetaValue)
119109 -> Text
120− -> ParserT Sources st m (Future st MetaValue)
110+ -> ParserT Text ParserState m (F MetaValue)
121111normalizeMetaValue pMetaValue x =
122112 -- Note: a standard quoted or unquoted YAML value will
123113 -- not end in a newline, but a "block" set off with
@@ -139,10 +129,10 @@ checkBoolean t
139129 | t == T.pack "false" || t == T.pack "False" || t == T.pack "FALSE" = Just False
140130 | otherwise = Nothing
141131
142−yamlToMetaValue :: (PandocMonad m, HasLastStrPosition st)
143− => ParserT Sources st m (Future st MetaValue)
132+yamlToMetaValue :: PandocMonad m
133+ => ParserT Text ParserState m (F MetaValue)
144134 -> YAML.Node YE.Pos
145− -> ParserT Sources st m (Future st MetaValue)
135+ -> ParserT Text ParserState m (F MetaValue)
146136yamlToMetaValue pMetaValue (YAML.Scalar _ x) =
147137 case x of
148138 YAML.SStr t -> normalizeMetaValue pMetaValue t
@@ -162,10 +152,10 @@ yamlToMetaValue pMetaValue (YAML.Mapping _ _ o) =
162152 fmap MetaMap <$> yamlMap pMetaValue o
163153yamlToMetaValue _ _ = return $ return $ MetaString ""
164154
165−yamlMap :: (PandocMonad m, HasLastStrPosition st)
166− => ParserT Sources st m (Future st MetaValue)
155+yamlMap :: PandocMonad m
156+ => ParserT Text ParserState m (F MetaValue)
167157 -> M.Map (YAML.Node YE.Pos) (YAML.Node YE.Pos)
168− -> ParserT Sources st m (Future st (M.Map Text MetaValue))
158+ -> ParserT Text ParserState m (F (M.Map Text MetaValue))
169159yamlMap pMetaValue o = do
170160 kvs <- forM (M.toList o) $ \(key, v) -> do
171161 k <- maybe (throwError $ PandocParseError
@@ -181,20 +171,3 @@ yamlMap pMetaValue o = do
181171 return $ do
182172 v' <- fv
183173 return (k, v')
184−
185−-- | Parse a YAML metadata block using the supplied 'MetaValue' parser.
186−yamlMetaBlock :: (HasLastStrPosition st, PandocMonad m)
187− => ParserT Sources st m (Future st MetaValue)
188− -> ParserT Sources st m (Future st Meta)
189−yamlMetaBlock parser = try $ do
190− string "---"
191− blankline
192− notFollowedBy blankline -- if --- is followed by a blank it's an HRULE
193− rawYamlLines <- manyTill anyLine stopLine
194− -- by including --- and ..., we allow yaml blocks with just comments:
195− let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))
196− optional blanklines
197− yamlBsToMeta parser $ UTF8.fromTextLazy $ TL.fromStrict rawYaml
198−
199−stopLine :: Monad m => ParserT Sources st m ()
200−stopLine = try $ (string "---" <|> string "...") >> blankline >> return ()
src/Text/Pandoc/Readers/Muse.hs+9−10
@@ -36,20 +36,19 @@ import Text.Pandoc.Definition
3636import Text.Pandoc.Error (PandocError (PandocParsecError))
3737import Text.Pandoc.Logging
3838import Text.Pandoc.Options
39−import Text.Pandoc.Parsing
40−import Text.Pandoc.Shared (trimr, tshow)
39+import Text.Pandoc.Parsing hiding (F)
40+import Text.Pandoc.Shared (crFilter, trimr, tshow)
4141
4242-- | Read Muse from an input string and return a Pandoc document.
43−readMuse :: (PandocMonad m, ToSources a)
43+readMuse :: PandocMonad m
4444 => ReaderOptions
45− -> a
45+ -> Text
4646 -> m Pandoc
4747readMuse opts s = do
48− let sources = toSources s
49− res <- flip runReaderT def $ runParserT parseMuse def{ museOptions = opts }
50− (initialSourceName sources) sources
48+ let input = crFilter s
49+ res <- flip runReaderT def $ runParserT parseMuse def{ museOptions = opts } "source" input
5150 case res of
52− Left e -> throwError $ PandocParsecError sources e
51+ Left e -> throwError $ PandocParsecError input e
5352 Right d -> return d
5453
5554type F = Future MuseState
@@ -83,7 +82,7 @@ instance Default MuseEnv where
8382 , museInPara = False
8483 }
8584
86−type MuseParser m = ParserT Sources MuseState (ReaderT MuseEnv m)
85+type MuseParser m = ParserT Text MuseState (ReaderT MuseEnv m)
8786
8887instance HasReaderOptions MuseState where
8988 extractReaderOptions = museOptions
@@ -156,7 +155,7 @@ firstColumn = getPosition >>= \pos -> guard (sourceColumn pos == 1)
156155-- * Parsers
157156
158157-- | Parse end-of-line, which can be either a newline or end-of-file.
159−eol :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
158+eol :: Stream s m Char => ParserT s st m ()
160159eol = void newline <|> eof
161160
162161getIndent :: PandocMonad m
src/Text/Pandoc/Readers/Native.hs+5−7
@@ -21,7 +21,6 @@ import Control.Monad.Except (throwError)
2121import Data.Text (Text)
2222import Text.Pandoc.Class.PandocMonad (PandocMonad)
2323import Text.Pandoc.Error
24−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2524
2625-- | Read native formatted text and return a Pandoc document.
2726-- The input may be a full pandoc document, a block list, a block,
@@ -33,15 +32,14 @@ import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3332--
3433-- > Pandoc nullMeta [Plain [Str "hi"]]
3534--
36−readNative :: (PandocMonad m, ToSources a)
35+readNative :: PandocMonad m
3736 => ReaderOptions
38− -> a
37+ -> Text -- ^ String to parse (assuming @'\n'@ line endings)
3938 -> m Pandoc
4039readNative _ s =
41− let t = sourcesToText . toSources $ s
42− in case maybe (Pandoc nullMeta <$> readBlocks t) Right (safeRead t) of
43− Right doc -> return doc
44− Left _ -> throwError $ PandocParseError "couldn't read native"
40+ case maybe (Pandoc nullMeta <$> readBlocks s) Right (safeRead s) of
41+ Right doc -> return doc
42+ Left _ -> throwError $ PandocParseError "couldn't read native"
4543
4644readBlocks :: Text -> Either PandocError [Block]
4745readBlocks s = maybe ((:[]) <$> readBlock s) Right (safeRead s)
src/Text/Pandoc/Readers/OPML.hs+4−9
@@ -24,8 +24,7 @@ import Text.Pandoc.Options
2424import Text.Pandoc.Error (PandocError(..))
2525import Text.Pandoc.Readers.HTML (readHtml)
2626import Text.Pandoc.Readers.Markdown (readMarkdown)
27−import Text.Pandoc.Shared (blocksToInlines')
28−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
27+import Text.Pandoc.Shared (crFilter, blocksToInlines')
2928import Text.Pandoc.XML.Light
3029import Control.Monad.Except (throwError)
3130
@@ -47,14 +46,10 @@ instance Default OPMLState where
4746 , opmlOptions = def
4847 }
4948
50−readOPML :: (PandocMonad m, ToSources a)
51− => ReaderOptions
52− -> a
53− -> m Pandoc
49+readOPML :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
5450readOPML opts inp = do
55− let sources = toSources inp
56− (bs, st') <-
57− runStateT (case parseXMLContents (TL.fromStrict . sourcesToText $ sources) of
51+ (bs, st') <- runStateT
52+ (case parseXMLContents (TL.fromStrict (crFilter inp)) of
5853 Left msg -> throwError $ PandocXMLError "" msg
5954 Right ns -> mapM parseBlock ns)
6055 def{ opmlOptions = opts }
src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2
@@ -22,7 +22,7 @@ module Text.Pandoc.Readers.Odt.Arrows.State where
2222import Control.Arrow
2323import qualified Control.Category as Cat
2424import Control.Monad
25−import Data.List (foldl')
25+
2626import Text.Pandoc.Readers.Odt.Arrows.Utils
2727import Text.Pandoc.Readers.Odt.Generic.Fallible
2828
@@ -122,7 +122,7 @@ iterateS a = ArrowState $ \(s,f) -> foldr a' (s,mzero) f
122122iterateSL :: (Foldable f, MonadPlus m)
123123 => ArrowState s x y
124124 -> ArrowState s (f x) (m y)
125−iterateSL a = ArrowState $ \(s,f) -> foldl' a' (s,mzero) f
125+iterateSL a = ArrowState $ \(s,f) -> foldl a' (s,mzero) f
126126 where a' (s',m) x = second (mplus m.return) $ runArrowState a (s',x)
127127
128128
src/Text/Pandoc/Readers/Odt/ContentReader.hs+7−12
@@ -1,5 +1,4 @@
11{-# LANGUAGE Arrows #-}
2−{-# LANGUAGE CPP #-}
32{-# LANGUAGE DeriveFoldable #-}
43{-# LANGUAGE GeneralizedNewtypeDeriving #-}
54{-# LANGUAGE PatternGuards #-}
@@ -34,7 +33,7 @@ import Data.List (find)
3433import qualified Data.Map as M
3534import qualified Data.Text as T
3635import Data.Maybe
37−import Data.Monoid (Alt (..))
36+import Data.Semigroup (First(..), Option(..))
3837
3938import Text.TeXMath (readMathML, writeTeX)
4039import qualified Text.Pandoc.XML.Light as XML
@@ -506,11 +505,13 @@ type InlineMatcher = ElementMatcher Inlines
506505
507506type BlockMatcher = ElementMatcher Blocks
508507
509−newtype FirstMatch a = FirstMatch (Alt Maybe a)
510− deriving (Foldable, Monoid, Semigroup)
508+
509+newtype FirstMatch a = FirstMatch (Option (First a))
510+ deriving (Foldable, Monoid, Semigroup)
511511
512512firstMatch :: a -> FirstMatch a
513−firstMatch = FirstMatch . Alt . Just
513+firstMatch = FirstMatch . Option . Just . First
514+
514515
515516--
516517matchingElement :: (Monoid e)
@@ -576,10 +577,7 @@ read_spaces = matchingElement NsText "s" (
576577read_line_break :: InlineMatcher
577578read_line_break = matchingElement NsText "line-break"
578579 $ returnV linebreak
579−--
580−read_tab :: InlineMatcher
581−read_tab = matchingElement NsText "tab"
582− $ returnV space
580+
583581--
584582read_span :: InlineMatcher
585583read_span = matchingElement NsText "span"
@@ -587,7 +585,6 @@ read_span = matchingElement NsText "span"
587585 $ matchChildContent [ read_span
588586 , read_spaces
589587 , read_line_break
590− , read_tab
591588 , read_link
592589 , read_note
593590 , read_citation
@@ -607,7 +604,6 @@ read_paragraph = matchingElement NsText "p"
607604 $ matchChildContent [ read_span
608605 , read_spaces
609606 , read_line_break
610− , read_tab
611607 , read_link
612608 , read_note
613609 , read_citation
@@ -634,7 +630,6 @@ read_header = matchingElement NsText "h"
634630 children <- ( matchChildContent [ read_span
635631 , read_spaces
636632 , read_line_break
637− , read_tab
638633 , read_link
639634 , read_note
640635 , read_citation
src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+2−2
@@ -64,12 +64,12 @@ import qualified Data.Map as M
6464import Data.Text (Text)
6565import Data.Default
6666import Data.Maybe
67−import Data.List (foldl')
6867
6968import qualified Text.Pandoc.XML.Light as XML
7069
7170import Text.Pandoc.Readers.Odt.Arrows.State
7271import Text.Pandoc.Readers.Odt.Arrows.Utils
72+
7373import Text.Pandoc.Readers.Odt.Generic.Namespaces
7474import Text.Pandoc.Readers.Odt.Generic.Utils
7575import Text.Pandoc.Readers.Odt.Generic.Fallible
@@ -293,7 +293,7 @@ readNSattributes = fromState $ \state -> maybe (state, failEmpty )
293293 => XMLConverterState nsID extraState
294294 -> Maybe (XMLConverterState nsID extraState)
295295 extractNSAttrs startState
296− = foldl' (\state d -> state >>= addNS d)
296+ = foldl (\state d -> state >>= addNS d)
297297 (Just startState)
298298 nsAttribs
299299 where nsAttribs = mapMaybe readNSattr (XML.elAttribs element)
src/Text/Pandoc/Readers/Odt/StyleReader.hs+7−8
@@ -44,7 +44,7 @@ import Control.Arrow
4444
4545import Data.Default
4646import qualified Data.Foldable as F
47−import Data.List (unfoldr, foldl')
47+import Data.List (unfoldr)
4848import qualified Data.Map as M
4949import Data.Maybe
5050import Data.Text (Text)
@@ -120,7 +120,7 @@ fontPitchReader = executeInSub NsOffice "font-face-decls" (
120120 &&&
121121 lookupDefaultingAttr NsStyle "font-pitch"
122122 ))
123− >>?^ ( M.fromList . foldl' accumLegalPitches [] )
123+ >>?^ ( M.fromList . foldl accumLegalPitches [] )
124124 ) `ifFailedDo` returnV (Right M.empty)
125125 where accumLegalPitches ls (Nothing,_) = ls
126126 accumLegalPitches ls (Just n,p) = (n,p):ls
@@ -563,13 +563,12 @@ readListLevelStyle levelType = readAttr NsText "level"
563563
564564--
565565chooseMostSpecificListLevelStyle :: S.Set ListLevelStyle -> Maybe ListLevelStyle
566−chooseMostSpecificListLevelStyle ls = F.foldr select Nothing ls
566+chooseMostSpecificListLevelStyle ls | ls == mempty = Nothing
567+ | otherwise = Just ( F.foldr1 select ls )
567568 where
568− select l Nothing = Just l
569− select ( ListLevelStyle t1 p1 s1 f1 b1 )
570− ( Just ( ListLevelStyle t2 p2 s2 f2 _ ))
571− = Just $ ListLevelStyle (select' t1 t2) (p1 <|> p2) (s1 <|> s2)
572− (selectLinf f1 f2) b1
569+ select ( ListLevelStyle t1 p1 s1 f1 b1 )
570+ ( ListLevelStyle t2 p2 s2 f2 _ )
571+ = ListLevelStyle (select' t1 t2) (p1 <|> p2) (s1 <|> s2) (selectLinf f1 f2) b1
573572 select' LltNumbered _ = LltNumbered
574573 select' _ LltNumbered = LltNumbered
575574 select' _ _ = LltBullet
src/Text/Pandoc/Readers/Org.hs+7−4
@@ -18,19 +18,22 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
1818import Text.Pandoc.Definition
1919import Text.Pandoc.Options
2020import Text.Pandoc.Parsing (reportLogMessages)
21−import Text.Pandoc.Sources (ToSources(..), ensureFinalNewlines)
21+import Text.Pandoc.Shared (crFilter)
22+
2223import Control.Monad.Except (throwError)
2324import Control.Monad.Reader (runReaderT)
2425
26+import Data.Text (Text)
27+
2528-- | Parse org-mode string and return a Pandoc document.
26−readOrg :: (PandocMonad m, ToSources a)
29+readOrg :: PandocMonad m
2730 => ReaderOptions -- ^ Reader options
28− -> a
31+ -> Text -- ^ String to parse (assuming @'\n'@ line endings)
2932 -> m Pandoc
3033readOrg opts s = do
3134 parsed <- flip runReaderT def $
3235 readWithM parseOrg (optionsToParserState opts)
33− (ensureFinalNewlines 2 (toSources s))
36+ (crFilter s <> "\n\n")
3437 case parsed of
3538 Right result -> return result
3639 Left e -> throwError e
src/Text/Pandoc/Readers/Org/Blocks.hs+4−7
@@ -39,12 +39,10 @@ import Data.Functor (($>))
3939import Data.List (foldl', intersperse)
4040import Data.Maybe (fromMaybe, isJust, isNothing)
4141import Data.Text (Text)
42−import Data.List.NonEmpty (nonEmpty)
43−import System.FilePath
42+
4443import qualified Data.Text as T
4544import qualified Text.Pandoc.Builder as B
4645import qualified Text.Pandoc.Walk as Walk
47−import Text.Pandoc.Sources (ToSources(..))
4846
4947--
5048-- parsing blocks
@@ -528,9 +526,7 @@ include = try $ do
528526 _ -> nullAttr
529527 return $ pure . B.codeBlockWith attr <$> parseRaw
530528 _ -> return $ return . B.fromList . blockFilter params <$> blockList
531− currentDir <- takeDirectory . sourceName <$> getPosition
532− insertIncludedFile blocksParser toSources
533− [currentDir] filename Nothing Nothing
529+ insertIncludedFileF blocksParser ["."] filename
534530 where
535531 includeTarget :: PandocMonad m => OrgParser m FilePath
536532 includeTarget = do
@@ -546,7 +542,8 @@ include = try $ do
546542 in case (minlvl >>= safeRead :: Maybe Int) of
547543 Nothing -> blks
548544 Just lvl -> let levels = Walk.query headerLevel blks
549− curMin = maybe 0 minimum $ nonEmpty levels
545+ -- CAVE: partial function in else
546+ curMin = if null levels then 0 else minimum levels
550547 in Walk.walk (shiftHeader (curMin - lvl)) blks
551548
552549 headerLevel :: Block -> [Int]
src/Text/Pandoc/Readers/Org/Inlines.hs+5−6
@@ -29,7 +29,6 @@ import Text.Pandoc.Definition
2929import Text.Pandoc.Options
3030import Text.Pandoc.Readers.LaTeX (inlineCommand, rawLaTeXInline)
3131import Text.TeXMath (DisplayType (..), readTeX, writePandoc)
32−import Text.Pandoc.Sources (ToSources(..))
3332import qualified Text.TeXMath.Readers.MathML.EntityMap as MathMLEntityMap
3433
3534import Control.Monad (guard, mplus, mzero, unless, void, when)
@@ -263,7 +262,7 @@ berkeleyCitationList = try $ do
263262 where
264263 citationListPart :: PandocMonad m => OrgParser m (F Inlines)
265264 citationListPart = fmap (trimInlinesF . mconcat) . try . many1 $ do
266− notFollowedBy' $ citeKey False
265+ notFollowedBy' citeKey
267266 notFollowedBy (oneOf ";]")
268267 inline
269268
@@ -278,7 +277,7 @@ berkeleyBareTag' = try $ void (string "cite")
278277
279278berkeleyTextualCite :: PandocMonad m => OrgParser m (F [Citation])
280279berkeleyTextualCite = try $ do
281− (suppressAuthor, key) <- citeKey False
280+ (suppressAuthor, key) <- citeKey
282281 returnF . return $ Citation
283282 { citationId = key
284283 , citationPrefix = mempty
@@ -351,7 +350,7 @@ citeList = sequence <$> sepEndBy1 citation (try $ char ';' *> skipSpaces)
351350citation :: PandocMonad m => OrgParser m (F Citation)
352351citation = try $ do
353352 pref <- prefix
354− (suppress_author, key) <- citeKey False
353+ (suppress_author, key) <- citeKey
355354 suff <- suffix
356355 return $ do
357356 x <- pref
@@ -368,7 +367,7 @@ citation = try $ do
368367 }
369368 where
370369 prefix = trimInlinesF . mconcat <$>
371− manyTill inline (char ']' <|> (']' <$ lookAhead (citeKey False)))
370+ manyTill inline (char ']' <|> (']' <$ lookAhead citeKey))
372371 suffix = try $ do
373372 hasSpace <- option False (notFollowedBy nonspaceChar >> return True)
374373 skipSpaces
@@ -803,7 +802,7 @@ inlineLaTeX = try $ do
803802 parseAsInlineLaTeX :: PandocMonad m
804803 => Text -> TeXExport -> OrgParser m (Maybe Inlines)
805804 parseAsInlineLaTeX cs = \case
806− TeXExport -> maybeRight <$> runParserT inlineCommand state "" (toSources cs)
805+ TeXExport -> maybeRight <$> runParserT inlineCommand state "" cs
807806 TeXIgnore -> return (Just mempty)
808807 TeXVerbatim -> return (Just $ B.str cs)
809808
src/Text/Pandoc/Readers/Org/Parsing.hs+5−5
@@ -63,7 +63,8 @@ module Text.Pandoc.Readers.Org.Parsing
6363 , ellipses
6464 , citeKey
6565 , gridTableWith
66− , insertIncludedFile
66+ , insertIncludedFileF
67+ -- * Re-exports from Text.Pandoc.Parsec
6768 , runParser
6869 , runParserT
6970 , getInput
@@ -99,22 +100,21 @@ module Text.Pandoc.Readers.Org.Parsing
99100 , getState
100101 , updateState
101102 , SourcePos
102− , sourceName
103103 , getPosition
104104 ) where
105105
106106import Data.Text (Text)
107107import Text.Pandoc.Readers.Org.ParserState
108108
109−import Text.Pandoc.Parsing hiding (anyLine, blanklines, newline,
110− parseFromString)
109+import Text.Pandoc.Parsing hiding (F, anyLine, blanklines, newline,
110+ parseFromString)
111111import qualified Text.Pandoc.Parsing as P
112112
113113import Control.Monad (guard)
114114import Control.Monad.Reader (ReaderT)
115115
116116-- | The parser used to read org files.
117−type OrgParser m = ParserT Sources OrgParserState (ReaderT OrgParserLocal m)
117+type OrgParser m = ParserT Text OrgParserState (ReaderT OrgParserLocal m)
118118
119119--
120120-- Adaptions and specializations of parsing utilities
src/Text/Pandoc/Readers/RST.hs+110−78
@@ -27,7 +27,8 @@ import Data.Text (Text)
2727import qualified Data.Text as T
2828import Text.Pandoc.Builder (Blocks, Inlines, fromList, setMeta, trimInlines)
2929import qualified Text.Pandoc.Builder as B
30−import Text.Pandoc.Class.PandocMonad (PandocMonad, fetchItem, getTimestamp)
30+import Text.Pandoc.Class.PandocMonad (PandocMonad, fetchItem,
31+ readFileFromDirs, getTimestamp)
3132import Text.Pandoc.CSV (CSVOptions (..), defaultCSVOptions, parseCSV)
3233import Text.Pandoc.Definition
3334import Text.Pandoc.Error
@@ -37,25 +38,25 @@ import Text.Pandoc.Options
3738import Text.Pandoc.Parsing
3839import Text.Pandoc.Shared
3940import qualified Text.Pandoc.UTF8 as UTF8
41+import Text.Printf (printf)
4042import Data.Time.Format
41−import System.FilePath (takeDirectory)
4243
4344-- TODO:
4445-- [ ] .. parsed-literal
4546
4647-- | Parse reStructuredText string and return Pandoc document.
47−readRST :: (PandocMonad m, ToSources a)
48+readRST :: PandocMonad m
4849 => ReaderOptions -- ^ Reader options
49− -> a
50+ -> Text -- ^ Text to parse (assuming @'\n'@ line endings)
5051 -> m Pandoc
5152readRST opts s = do
5253 parsed <- readWithM parseRST def{ stateOptions = opts }
53− (ensureFinalNewlines 2 (toSources s))
54+ (crFilter s <> "\n\n")
5455 case parsed of
5556 Right result -> return result
5657 Left e -> throwError e
5758
58−type RSTParser m = ParserT Sources ParserState m
59+type RSTParser m = ParserT Text ParserState m
5960
6061--
6162-- Constants and data structure definitions
@@ -150,19 +151,11 @@ parseRST = do
150151 startPos <- getPosition
151152 -- go through once just to get list of reference keys and notes
152153 -- docMinusKeys is the raw document with blanks where the keys were...
153− let chunk = referenceKey
154− <|> anchorDef
155− <|> noteBlock
156− <|> citationBlock
157− <|> (snd <$> withRaw comment)
158− <|> headerBlock
159− <|> lineClump
160− docMinusKeys <- Sources <$>
161− manyTill (do pos <- getPosition
162− t <- chunk
163− return (pos, t)) eof
164− -- UGLY: we collapse source position information.
165− -- TODO: fix the parser to use the F monad instead of two passes
154+ docMinusKeys <- T.concat <$>
155+ manyTill (referenceKey <|> anchorDef <|>
156+ noteBlock <|> citationBlock <|>
157+ (snd <$> withRaw comment) <|>
158+ headerBlock <|> lineClump) eof
166159 setInput docMinusKeys
167160 setPosition startPos
168161 st' <- getState
@@ -355,7 +348,7 @@ singleHeader' = try $ do
355348-- hrule block
356349--
357350
358−hrule :: Monad m => ParserT Sources st m Blocks
351+hrule :: Monad m => ParserT Text st m Blocks
359352hrule = try $ do
360353 chr <- oneOf underlineChars
361354 count 3 (char chr)
@@ -370,7 +363,7 @@ hrule = try $ do
370363
371364-- read a line indented by a given string
372365indentedLine :: (HasReaderOptions st, Monad m)
373− => Int -> ParserT Sources st m Text
366+ => Int -> ParserT Text st m Text
374367indentedLine indents = try $ do
375368 lookAhead spaceChar
376369 gobbleAtMostSpaces indents
@@ -379,7 +372,7 @@ indentedLine indents = try $ do
379372-- one or more indented lines, possibly separated by blank lines.
380373-- any amount of indentation will work.
381374indentedBlock :: (HasReaderOptions st, Monad m)
382− => ParserT Sources st m Text
375+ => ParserT Text st m Text
383376indentedBlock = try $ do
384377 indents <- length <$> lookAhead (many1 spaceChar)
385378 lns <- many1 $ try $ do b <- option "" blanklines
@@ -388,20 +381,20 @@ indentedBlock = try $ do
388381 optional blanklines
389382 return $ T.unlines lns
390383
391−quotedBlock :: Monad m => ParserT Sources st m Text
384+quotedBlock :: Monad m => ParserT Text st m Text
392385quotedBlock = try $ do
393386 quote <- lookAhead $ oneOf "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
394387 lns <- many1 $ lookAhead (char quote) >> anyLine
395388 optional blanklines
396389 return $ T.unlines lns
397390
398−codeBlockStart :: Monad m => ParserT Sources st m Char
391+codeBlockStart :: Monad m => ParserT Text st m Char
399392codeBlockStart = string "::" >> blankline >> blankline
400393
401−codeBlock :: Monad m => ParserT Sources ParserState m Blocks
394+codeBlock :: Monad m => ParserT Text ParserState m Blocks
402395codeBlock = try $ codeBlockStart >> codeBlockBody
403396
404−codeBlockBody :: Monad m => ParserT Sources ParserState m Blocks
397+codeBlockBody :: Monad m => ParserT Text ParserState m Blocks
405398codeBlockBody = do
406399 lang <- stateRstHighlight <$> getState
407400 try $ B.codeBlockWith ("", maybeToList lang, []) . stripTrailingNewlines <$>
@@ -417,14 +410,14 @@ lhsCodeBlock = try $ do
417410 return $ B.codeBlockWith ("", ["haskell","literate"], [])
418411 $ T.intercalate "\n" lns
419412
420−latexCodeBlock :: Monad m => ParserT Sources st m [Text]
413+latexCodeBlock :: Monad m => ParserT Text st m [Text]
421414latexCodeBlock = try $ do
422415 try (latexBlockLine "\\begin{code}")
423416 many1Till anyLine (try $ latexBlockLine "\\end{code}")
424417 where
425418 latexBlockLine s = skipMany spaceChar >> string s >> blankline
426419
427−birdCodeBlock :: Monad m => ParserT Sources st m [Text]
420+birdCodeBlock :: Monad m => ParserT Text st m [Text]
428421birdCodeBlock = filterSpace <$> many1 birdTrackLine
429422 where filterSpace lns =
430423 -- if (as is normal) there is always a space after >, drop it
@@ -432,7 +425,7 @@ birdCodeBlock = filterSpace <$> many1 birdTrackLine
432425 then map (T.drop 1) lns
433426 else lns
434427
435−birdTrackLine :: Monad m => ParserT Sources st m Text
428+birdTrackLine :: Monad m => ParserT Text st m Text
436429birdTrackLine = char '>' >> anyLine
437430
438431--
@@ -453,39 +446,64 @@ encoding
453446-}
454447
455448includeDirective :: PandocMonad m
456− => Text
457− -> [(Text, Text)]
458− -> Text
449+ => Text -> [(Text, Text)] -> Text
459450 -> RSTParser m Blocks
460451includeDirective top fields body = do
461− let f = T.unpack $ trim top
462− guard $ not $ null f
452+ let f = trim top
453+ guard $ not (T.null f)
463454 guard $ T.null (trim body)
464− let startLine = lookup "start-line" fields >>= safeRead
465− let endLine = lookup "end-line" fields >>= safeRead
466− let classes = maybe [] T.words (lookup "class" fields)
467− let ident = maybe "" trimr $ lookup "name" fields
468− let parser =
469− case lookup "code" fields of
470− Just lang ->
471− (codeblock ident classes fields (trimr lang) False
472− . sourcesToText) <$> getInput
473− Nothing ->
474− case lookup "literal" fields of
475− Just _ -> B.rawBlock "rst" . sourcesToText <$> getInput
476− Nothing -> parseBlocks
477− let selectLines =
478− (case trim <$> lookup "end-before" fields of
479− Just patt -> takeWhile (not . (patt `T.isInfixOf`))
480− Nothing -> id) .
481− (case trim <$> lookup "start-after" fields of
482− Just patt -> drop 1 .
483− dropWhile (not . (patt `T.isInfixOf`))
484− Nothing -> id)
485− let toStream t =
486− toSources [(f, T.unlines . selectLines . T.lines $ t)]
487− currentDir <- takeDirectory . sourceName <$> getPosition
488− insertIncludedFile parser toStream [currentDir] f startLine endLine
455+ -- options
456+ let (startLine :: Maybe Int) = lookup "start-line" fields >>= safeRead
457+ let (endLine :: Maybe Int) = lookup "end-line" fields >>= safeRead
458+ oldPos <- getPosition
459+ oldInput <- getInput
460+ containers <- stateContainers <$> getState
461+ when (f `elem` containers) $
462+ throwError $ PandocParseError $ "Include file loop at " <> tshow oldPos
463+ updateState $ \s -> s{ stateContainers = f : stateContainers s }
464+ mbContents <- readFileFromDirs ["."] $ T.unpack f
465+ contentLines <- case mbContents of
466+ Just s -> return $ T.lines s
467+ Nothing -> do
468+ logMessage $ CouldNotLoadIncludeFile f oldPos
469+ return []
470+ let numLines = length contentLines
471+ let startLine' = case startLine of
472+ Nothing -> 1
473+ Just x | x >= 0 -> x
474+ | otherwise -> numLines + x -- negative from end
475+ let endLine' = case endLine of
476+ Nothing -> numLines + 1
477+ Just x | x >= 0 -> x
478+ | otherwise -> numLines + x -- negative from end
479+ let contentLines' = drop (startLine' - 1)
480+ $ take (endLine' - 1) contentLines
481+ let contentLines'' = (case trim <$> lookup "end-before" fields of
482+ Just patt -> takeWhile (not . (patt `T.isInfixOf`))
483+ Nothing -> id) .
484+ (case trim <$> lookup "start-after" fields of
485+ Just patt -> drop 1 .
486+ dropWhile (not . (patt `T.isInfixOf`))
487+ Nothing -> id) $ contentLines'
488+ let contents' = T.unlines contentLines''
489+ case lookup "code" fields of
490+ Just lang -> do
491+ let classes = maybe [] T.words (lookup "class" fields)
492+ let ident = maybe "" trimr $ lookup "name" fields
493+ codeblock ident classes fields (trimr lang) contents' False
494+ Nothing -> case lookup "literal" fields of
495+ Just _ -> return $ B.rawBlock "rst" contents'
496+ Nothing -> do
497+ setPosition $ newPos (T.unpack f) 1 1
498+ setInput $ contents' <> "\n"
499+ bs <- optional blanklines >>
500+ (mconcat <$> many block)
501+ setInput oldInput
502+ setPosition oldPos
503+ updateState $ \s -> s{ stateContainers =
504+ tail $ stateContainers s }
505+ return bs
506+
489507
490508--
491509-- list blocks
@@ -508,7 +526,7 @@ definitionList :: PandocMonad m => RSTParser m Blocks
508526definitionList = B.definitionList <$> many1 definitionListItem
509527
510528-- parses bullet list start and returns its length (inc. following whitespace)
511−bulletListStart :: Monad m => ParserT Sources st m Int
529+bulletListStart :: Monad m => ParserT Text st m Int
512530bulletListStart = try $ do
513531 notFollowedBy' hrule -- because hrules start out just like lists
514532 marker <- oneOf bulletListMarkers
@@ -595,9 +613,8 @@ comment = try $ do
595613 string ".."
596614 skipMany1 spaceChar <|> (() <$ lookAhead newline)
597615 -- notFollowedBy' directiveLabel -- comment comes after directive so unnec.
598− _ <- anyLine
616+ manyTill anyChar blanklines
599617 optional indentedBlock
600− optional blanklines
601618 return mempty
602619
603620directiveLabel :: Monad m => RSTParser m Text
@@ -714,8 +731,8 @@ directive' = do
714731 "" -> stateRstHighlight def
715732 lang -> Just lang })
716733 x | x == "code" || x == "code-block" || x == "sourcecode" ->
717− return $ codeblock name classes (map (second trimr) fields)
718− (trim top) True body
734+ codeblock name classes (map (second trimr) fields)
735+ (trim top) body True
719736 "aafig" -> do
720737 let attribs = (name, ["aafig"], map (second trimr) fields)
721738 return $ B.codeBlockWith attribs $ stripTrailingNewlines body
@@ -1001,10 +1018,10 @@ toChunks = dropWhile T.null
10011018 then "\\begin{aligned}\n" <> s <> "\n\\end{aligned}"
10021019 else s
10031020
1004−codeblock :: Text -> [Text] -> [(Text, Text)] -> Text -> Bool -> Text
1005− -> Blocks
1006−codeblock ident classes fields lang rmTrailingNewlines body =
1007− B.codeBlockWith attribs $ stripTrailingNewlines' body
1021+codeblock :: Text -> [Text] -> [(Text, Text)] -> Text -> Text -> Bool
1022+ -> RSTParser m Blocks
1023+codeblock ident classes fields lang body rmTrailingNewlines =
1024+ return $ B.codeBlockWith attribs $ stripTrailingNewlines' body
10081025 where stripTrailingNewlines' = if rmTrailingNewlines
10091026 then stripTrailingNewlines
10101027 else id
@@ -1085,7 +1102,7 @@ quotedReferenceName = try $ do
10851102-- plus isolated (no two adjacent) internal hyphens, underscores,
10861103-- periods, colons and plus signs; no whitespace or other characters
10871104-- are allowed.
1088−simpleReferenceName :: Monad m => ParserT Sources st m Text
1105+simpleReferenceName :: Monad m => ParserT Text st m Text
10891106simpleReferenceName = do
10901107 x <- alphaNum
10911108 xs <- many $ alphaNum
@@ -1104,7 +1121,7 @@ referenceKey = do
11041121 -- return enough blanks to replace key
11051122 return $ T.replicate (sourceLine endPos - sourceLine startPos) "\n"
11061123
1107−targetURI :: Monad m => ParserT Sources st m Text
1124+targetURI :: Monad m => ParserT Text st m Text
11081125targetURI = do
11091126 skipSpaces
11101127 optional $ try $ newline >> notFollowedBy blankline
@@ -1142,10 +1159,8 @@ anonymousKey :: Monad m => RSTParser m ()
11421159anonymousKey = try $ do
11431160 oneOfStrings [".. __:", "__"]
11441161 src <- targetURI
1145− -- we need to ensure that the keys are ordered by occurrence in
1146− -- the document.
1147− numKeys <- M.size . stateKeys <$> getState
1148− let key = toKey $ "_" <> T.pack (show numKeys)
1162+ pos <- getPosition
1163+ let key = toKey $ "_" <> T.pack (printf "%09d" (sourceLine pos))
11491164 updateState $ \s -> s { stateKeys = M.insert key ((src,""), nullAttr) $
11501165 stateKeys s }
11511166
@@ -1234,13 +1249,13 @@ headerBlock = do
12341249-- Grid tables TODO:
12351250-- - column spans
12361251
1237−dashedLine :: Monad m => Char -> ParserT Sources st m (Int, Int)
1252+dashedLine :: Monad m => Char -> ParserT Text st m (Int, Int)
12381253dashedLine ch = do
12391254 dashes <- many1 (char ch)
12401255 sp <- many (char ' ')
12411256 return (length dashes, length $ dashes ++ sp)
12421257
1243−simpleDashedLines :: Monad m => Char -> ParserT Sources st m [(Int,Int)]
1258+simpleDashedLines :: Monad m => Char -> ParserT Text st m [(Int,Int)]
12441259simpleDashedLines ch = try $ many1 (dashedLine ch)
12451260
12461261-- Parse a table row separator
@@ -1366,7 +1381,7 @@ hyphens = do
13661381 -- don't want to treat endline after hyphen or dash as a space
13671382 return $ B.str result
13681383
1369−escapedChar :: Monad m => ParserT Sources st m Inlines
1384+escapedChar :: Monad m => ParserT Text st m Inlines
13701385escapedChar = do c <- escaped anyChar
13711386 return $ if c == ' ' || c == '\n' || c == '\r'
13721387 -- '\ ' is null in RST
@@ -1642,4 +1657,21 @@ note = try $ do
16421657 return $ B.note contents
16431658
16441659smart :: PandocMonad m => RSTParser m Inlines
1645−smart = smartPunctuation inline
1660+smart = do
1661+ guardEnabled Ext_smart
1662+ doubleQuoted <|> singleQuoted <|>
1663+ choice [apostrophe, dash, ellipses]
1664+
1665+singleQuoted :: PandocMonad m => RSTParser m Inlines
1666+singleQuoted = try $ do
1667+ singleQuoteStart
1668+ withQuoteContext InSingleQuote $
1669+ B.singleQuoted . trimInlines . mconcat <$>
1670+ many1Till inline singleQuoteEnd
1671+
1672+doubleQuoted :: PandocMonad m => RSTParser m Inlines
1673+doubleQuoted = try $ do
1674+ doubleQuoteStart
1675+ withQuoteContext InDoubleQuote $
1676+ B.doubleQuoted . trimInlines . mconcat <$>
1677+ many1Till inline doubleQuoteEnd
src/Text/Pandoc/Readers/Roff.hs+18−16
@@ -42,6 +42,7 @@ import Text.Pandoc.Logging (LogMessage(..))
4242import Text.Pandoc.Options
4343import Text.Pandoc.Parsing
4444import Text.Pandoc.Shared (safeRead)
45+import Text.Parsec hiding (tokenPrim)
4546import Text.Pandoc.RoffChar (characterCodes, combiningAccents)
4647import qualified Data.Sequence as Seq
4748import qualified Data.Foldable as Foldable
@@ -121,16 +122,16 @@ instance Default RoffState where
121122 , afterConditional = False
122123 }
123124
124−type RoffLexer m = ParserT Sources RoffState m
125+type RoffLexer m = ParserT T.Text RoffState m
125126
126127--
127128-- Lexer: T.Text -> RoffToken
128129--
129130
130−eofline :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s u m ()
131+eofline :: Stream s m Char => ParsecT s u m ()
131132eofline = void newline <|> eof <|> () <$ lookAhead (string "\\}")
132133
133−spacetab :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s u m Char
134+spacetab :: Stream s m Char => ParsecT s u m Char
134135spacetab = char ' ' <|> char '\t'
135136
136137characterCodeMap :: M.Map T.Text Char
@@ -302,7 +303,8 @@ expandString = try $ do
302303 char '*'
303304 cs <- escapeArg <|> countChar 1 anyChar
304305 s <- linePartsToText <$> resolveText cs pos
305− addToInput s
306+ getInput >>= setInput . (s <>)
307+ return ()
306308
307309-- Parses: '..'
308310quoteArg :: PandocMonad m => RoffLexer m T.Text
@@ -314,7 +316,7 @@ escFont = do
314316 font' <- if T.null font || font == "P"
315317 then prevFont <$> getState
316318 else return $ foldr processFontLetter defaultFontSpec $ T.unpack font
317− updateState $ \st -> st{ prevFont = currentFont st
319+ modifyState $ \st -> st{ prevFont = currentFont st
318320 , currentFont = font' }
319321 return [Font font']
320322 where
@@ -370,8 +372,8 @@ lexTable pos = do
370372 spaces
371373 opts <- try tableOptions <|> [] <$ optional (char ';')
372374 case lookup "tab" opts of
373− Just (T.uncons -> Just (c, _)) -> updateState $ \st -> st{ tableTabChar = c }
374− _ -> updateState $ \st -> st{ tableTabChar = '\t' }
375+ Just (T.uncons -> Just (c, _)) -> modifyState $ \st -> st{ tableTabChar = c }
376+ _ -> modifyState $ \st -> st{ tableTabChar = '\t' }
375377 spaces
376378 skipMany lexComment
377379 spaces
@@ -487,18 +489,18 @@ lexConditional mname = do
487489 ifPart <- do
488490 optional $ try $ char '\\' >> newline
489491 lexGroup
490− <|> do updateState $ \s -> s{ afterConditional = True }
492+ <|> do modifyState $ \s -> s{ afterConditional = True }
491493 t <- manToken
492− updateState $ \s -> s{ afterConditional = False }
494+ modifyState $ \s -> s{ afterConditional = False }
493495 return t
494496 case mbtest of
495497 Nothing -> do
496− setState st -- reset state, so we don't record macros in skipped section
498+ putState st -- reset state, so we don't record macros in skipped section
497499 report $ SkippedContent (T.cons '.' mname) pos
498500 return mempty
499501 Just True -> return ifPart
500502 Just False -> do
501− setState st
503+ putState st
502504 return mempty
503505
504506expression :: PandocMonad m => RoffLexer m (Maybe Bool)
@@ -513,7 +515,7 @@ expression = do
513515 _ -> Nothing
514516 where
515517 returnValue v = do
516− updateState $ \st -> st{ lastExpression = v }
518+ modifyState $ \st -> st{ lastExpression = v }
517519 return v
518520
519521lexGroup :: PandocMonad m => RoffLexer m RoffTokens
@@ -534,7 +536,7 @@ lexIncludeFile args = do
534536 result <- readFileFromDirs dirs $ T.unpack fp
535537 case result of
536538 Nothing -> report $ CouldNotLoadIncludeFile fp pos
537− Just s -> addToInput s
539+ Just s -> getInput >>= setInput . (s <>)
538540 return mempty
539541 [] -> return mempty
540542
@@ -562,13 +564,13 @@ lexStringDef args = do -- string definition
562564 (x:ys) -> do
563565 let ts = singleTok $ TextLine (intercalate [RoffStr " " ] ys)
564566 let stringName = linePartsToText x
565− updateState $ \st ->
567+ modifyState $ \st ->
566568 st{ customMacros = M.insert stringName ts (customMacros st) }
567569 return mempty
568570
569571lexMacroDef :: PandocMonad m => [Arg] -> RoffLexer m RoffTokens
570572lexMacroDef args = do -- macro definition
571− updateState $ \st -> st{ roffMode = CopyMode }
573+ modifyState $ \st -> st{ roffMode = CopyMode }
572574 (macroName, stopMacro) <-
573575 case args of
574576 (x : y : _) -> return (linePartsToText x, linePartsToText y)
@@ -582,7 +584,7 @@ lexMacroDef args = do -- macro definition
582584 _ <- lexArgs
583585 return ()
584586 ts <- mconcat <$> manyTill manToken stop
585− updateState $ \st ->
587+ modifyState $ \st ->
586588 st{ customMacros = M.insert macroName ts (customMacros st)
587589 , roffMode = NormalMode }
588590 return mempty
src/Text/Pandoc/Readers/TWiki.hs+27−7
@@ -28,22 +28,22 @@ import Text.Pandoc.Definition
2828import Text.Pandoc.Options
2929import Text.Pandoc.Parsing hiding (enclosed, nested)
3030import Text.Pandoc.Readers.HTML (htmlTag, isCommentTag)
31−import Text.Pandoc.Shared (tshow)
31+import Text.Pandoc.Shared (crFilter, tshow)
3232import Text.Pandoc.XML (fromEntities)
3333
3434-- | Read twiki from an input string and return a Pandoc document.
35−readTWiki :: (PandocMonad m, ToSources a)
35+readTWiki :: PandocMonad m
3636 => ReaderOptions
37− -> a
37+ -> Text
3838 -> m Pandoc
3939readTWiki opts s = do
40− let sources = ensureFinalNewlines 2 (toSources s)
41− res <- readWithM parseTWiki def{ stateOptions = opts } sources
40+ res <- readWithM parseTWiki def{ stateOptions = opts }
41+ (crFilter s <> "\n\n")
4242 case res of
4343 Left e -> throwError e
4444 Right d -> return d
4545
46−type TWParser = ParserT Sources ParserState
46+type TWParser = ParserT Text ParserState
4747
4848--
4949-- utility functions
@@ -469,7 +469,27 @@ symbol :: PandocMonad m => TWParser m B.Inlines
469469symbol = B.str <$> countChar 1 nonspaceChar
470470
471471smart :: PandocMonad m => TWParser m B.Inlines
472−smart = smartPunctuation inline
472+smart = do
473+ guardEnabled Ext_smart
474+ doubleQuoted <|> singleQuoted <|>
475+ choice [ apostrophe
476+ , dash
477+ , ellipses
478+ ]
479+
480+singleQuoted :: PandocMonad m => TWParser m B.Inlines
481+singleQuoted = try $ do
482+ singleQuoteStart
483+ withQuoteContext InSingleQuote
484+ (B.singleQuoted . B.trimInlines . mconcat <$> many1Till inline singleQuoteEnd)
485+
486+doubleQuoted :: PandocMonad m => TWParser m B.Inlines
487+doubleQuoted = try $ do
488+ doubleQuoteStart
489+ contents <- mconcat <$> many (try $ notFollowedBy doubleQuoteEnd >> inline)
490+ withQuoteContext InDoubleQuote (doubleQuoteEnd >>
491+ return (B.doubleQuoted $ B.trimInlines contents))
492+ <|> return (B.str "\8220" B.<> contents)
473493
474494link :: PandocMonad m => TWParser m B.Inlines
475495link = try $ do
src/Text/Pandoc/Readers/Textile.hs+88−94
@@ -38,8 +38,7 @@ module Text.Pandoc.Readers.Textile ( readTextile) where
3838import Control.Monad (guard, liftM)
3939import Control.Monad.Except (throwError)
4040import Data.Char (digitToInt, isUpper)
41−import Data.List (intersperse, transpose, foldl')
42−import Data.List.NonEmpty (NonEmpty(..), nonEmpty)
41+import Data.List (intersperse, transpose)
4342import Data.Text (Text)
4443import qualified Data.Text as T
4544import Text.HTML.TagSoup (Tag (..), fromAttrib)
@@ -53,34 +52,30 @@ import Text.Pandoc.Options
5352import Text.Pandoc.Parsing
5453import Text.Pandoc.Readers.HTML (htmlTag, isBlockTag, isInlineTag)
5554import Text.Pandoc.Readers.LaTeX (rawLaTeXBlock, rawLaTeXInline)
56−import Text.Pandoc.Shared (trim, tshow)
55+import Text.Pandoc.Shared (crFilter, trim, tshow)
5756
5857-- | Parse a Textile text and return a Pandoc document.
59−readTextile :: (PandocMonad m, ToSources a)
58+readTextile :: PandocMonad m
6059 => ReaderOptions -- ^ Reader options
61− -> a
60+ -> Text -- ^ String to parse (assuming @'\n'@ line endings)
6261 -> m Pandoc
6362readTextile opts s = do
64− let sources = ensureFinalNewlines 2 (toSources s)
65− parsed <- readWithM parseTextile def{ stateOptions = opts } sources
63+ parsed <- readWithM parseTextile def{ stateOptions = opts }
64+ (crFilter s <> "\n\n")
6665 case parsed of
6766 Right result -> return result
6867 Left e -> throwError e
6968
70−type TextileParser = ParserT Sources ParserState
7169
7270-- | Generate a Pandoc ADT from a textile document
73−parseTextile :: PandocMonad m => TextileParser m Pandoc
71+parseTextile :: PandocMonad m => ParserT Text ParserState m Pandoc
7472parseTextile = do
7573 many blankline
7674 startPos <- getPosition
7775 -- go through once just to get list of reference keys and notes
7876 -- docMinusKeys is the raw document with blanks where the keys/notes were...
79− let firstPassParser = do
80− pos <- getPosition
81− t <- noteBlock <|> lineClump
82− return (pos, t)
83− manyTill firstPassParser eof >>= setInput . Sources
77+ let firstPassParser = noteBlock <|> lineClump
78+ manyTill firstPassParser eof >>= setInput . T.concat
8479 setPosition startPos
8580 st' <- getState
8681 let reversedNotes = stateNotes st'
@@ -88,10 +83,10 @@ parseTextile = do
8883 -- now parse it for real...
8984 Pandoc nullMeta . B.toList <$> parseBlocks -- FIXME
9085
91−noteMarker :: PandocMonad m => TextileParser m Text
86+noteMarker :: PandocMonad m => ParserT Text ParserState m Text
9287noteMarker = skipMany spaceChar >> string "fn" >> T.pack <$> manyTill digit (char '.')
9388
94−noteBlock :: PandocMonad m => TextileParser m Text
89+noteBlock :: PandocMonad m => ParserT Text ParserState m Text
9590noteBlock = try $ do
9691 startPos <- getPosition
9792 ref <- noteMarker
@@ -106,11 +101,11 @@ noteBlock = try $ do
106101 return $ T.replicate (sourceLine endPos - sourceLine startPos) "\n"
107102
108103-- | Parse document blocks
109−parseBlocks :: PandocMonad m => TextileParser m Blocks
104+parseBlocks :: PandocMonad m => ParserT Text ParserState m Blocks
110105parseBlocks = mconcat <$> manyTill block eof
111106
112107-- | Block parsers list tried in definition order
113−blockParsers :: PandocMonad m => [TextileParser m Blocks]
108+blockParsers :: PandocMonad m => [ParserT Text ParserState m Blocks]
114109blockParsers = [ codeBlock
115110 , header
116111 , blockQuote
@@ -125,22 +120,22 @@ blockParsers = [ codeBlock
125120 ]
126121
127122-- | Any block in the order of definition of blockParsers
128−block :: PandocMonad m => TextileParser m Blocks
123+block :: PandocMonad m => ParserT Text ParserState m Blocks
129124block = do
130125 res <- choice blockParsers <?> "block"
131126 trace (T.take 60 $ tshow $ B.toList res)
132127 return res
133128
134−commentBlock :: PandocMonad m => TextileParser m Blocks
129+commentBlock :: PandocMonad m => ParserT Text ParserState m Blocks
135130commentBlock = try $ do
136131 string "###."
137132 manyTill anyLine blanklines
138133 return mempty
139134
140−codeBlock :: PandocMonad m => TextileParser m Blocks
135+codeBlock :: PandocMonad m => ParserT Text ParserState m Blocks
141136codeBlock = codeBlockTextile <|> codeBlockHtml
142137
143−codeBlockTextile :: PandocMonad m => TextileParser m Blocks
138+codeBlockTextile :: PandocMonad m => ParserT Text ParserState m Blocks
144139codeBlockTextile = try $ do
145140 string "bc." <|> string "pre."
146141 extended <- option False (True <$ char '.')
@@ -160,7 +155,7 @@ trimTrailingNewlines :: Text -> Text
160155trimTrailingNewlines = T.dropWhileEnd (=='\n')
161156
162157-- | Code Blocks in Textile are between <pre> and </pre>
163−codeBlockHtml :: PandocMonad m => TextileParser m Blocks
158+codeBlockHtml :: PandocMonad m => ParserT Text ParserState m Blocks
164159codeBlockHtml = try $ do
165160 (t@(TagOpen _ attrs),_) <- htmlTag (tagOpen (=="pre") (const True))
166161 result' <- T.pack <$> manyTill anyChar (htmlTag (tagClose (=="pre")))
@@ -178,7 +173,7 @@ codeBlockHtml = try $ do
178173 return $ B.codeBlockWith (ident,classes,kvs) result'''
179174
180175-- | Header of the form "hN. content" with N in 1..6
181−header :: PandocMonad m => TextileParser m Blocks
176+header :: PandocMonad m => ParserT Text ParserState m Blocks
182177header = try $ do
183178 char 'h'
184179 level <- digitToInt <$> oneOf "123456"
@@ -190,14 +185,14 @@ header = try $ do
190185 return $ B.headerWith attr' level name
191186
192187-- | Blockquote of the form "bq. content"
193−blockQuote :: PandocMonad m => TextileParser m Blocks
188+blockQuote :: PandocMonad m => ParserT Text ParserState m Blocks
194189blockQuote = try $ do
195190 string "bq" >> attributes >> char '.' >> whitespace
196191 B.blockQuote <$> para
197192
198193-- Horizontal rule
199194
200−hrule :: PandocMonad m => TextileParser m Blocks
195+hrule :: PandocMonad m => ParserT Text st m Blocks
201196hrule = try $ do
202197 skipSpaces
203198 start <- oneOf "-*"
@@ -212,39 +207,39 @@ hrule = try $ do
212207-- | Can be a bullet list or an ordered list. This implementation is
213208-- strict in the nesting, sublist must start at exactly "parent depth
214209-- plus one"
215−anyList :: PandocMonad m => TextileParser m Blocks
210+anyList :: PandocMonad m => ParserT Text ParserState m Blocks
216211anyList = try $ anyListAtDepth 1 <* blanklines
217212
218213-- | This allow one type of list to be nested into an other type,
219214-- provided correct nesting
220−anyListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
215+anyListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
221216anyListAtDepth depth = choice [ bulletListAtDepth depth,
222217 orderedListAtDepth depth,
223218 definitionList ]
224219
225220-- | Bullet List of given depth, depth being the number of leading '*'
226−bulletListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
221+bulletListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
227222bulletListAtDepth depth = try $ B.bulletList <$> many1 (bulletListItemAtDepth depth)
228223
229224-- | Bullet List Item of given depth, depth being the number of
230225-- leading '*'
231−bulletListItemAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
226+bulletListItemAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
232227bulletListItemAtDepth = genericListItemAtDepth '*'
233228
234229-- | Ordered List of given depth, depth being the number of
235230-- leading '#'
236−orderedListAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
231+orderedListAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
237232orderedListAtDepth depth = try $ do
238233 items <- many1 (orderedListItemAtDepth depth)
239234 return $ B.orderedList items
240235
241236-- | Ordered List Item of given depth, depth being the number of
242237-- leading '#'
243−orderedListItemAtDepth :: PandocMonad m => Int -> TextileParser m Blocks
238+orderedListItemAtDepth :: PandocMonad m => Int -> ParserT Text ParserState m Blocks
244239orderedListItemAtDepth = genericListItemAtDepth '#'
245240
246241-- | Common implementation of list items
247−genericListItemAtDepth :: PandocMonad m => Char -> Int -> TextileParser m Blocks
242+genericListItemAtDepth :: PandocMonad m => Char -> Int -> ParserT Text ParserState m Blocks
248243genericListItemAtDepth c depth = try $ do
249244 count depth (char c) >> attributes >> whitespace
250245 contents <- mconcat <$> many ((B.plain . mconcat <$> many1 inline) <|>
@@ -254,25 +249,25 @@ genericListItemAtDepth c depth = try $ do
254249 return $ contents <> sublist
255250
256251-- | A definition list is a set of consecutive definition items
257−definitionList :: PandocMonad m => TextileParser m Blocks
252+definitionList :: PandocMonad m => ParserT Text ParserState m Blocks
258253definitionList = try $ B.definitionList <$> many1 definitionListItem
259254
260255-- | List start character.
261−listStart :: PandocMonad m => TextileParser m ()
256+listStart :: PandocMonad m => ParserT Text ParserState m ()
262257listStart = genericListStart '*'
263258 <|> () <$ genericListStart '#'
264259 <|> () <$ definitionListStart
265260
266−genericListStart :: PandocMonad m => Char -> TextileParser m ()
261+genericListStart :: PandocMonad m => Char -> ParserT Text st m ()
267262genericListStart c = () <$ try (many1 (char c) >> whitespace)
268263
269−basicDLStart :: PandocMonad m => TextileParser m ()
264+basicDLStart :: PandocMonad m => ParserT Text ParserState m ()
270265basicDLStart = do
271266 char '-'
272267 whitespace
273268 notFollowedBy newline
274269
275−definitionListStart :: PandocMonad m => TextileParser m Inlines
270+definitionListStart :: PandocMonad m => ParserT Text ParserState m Inlines
276271definitionListStart = try $ do
277272 basicDLStart
278273 trimInlines . mconcat <$>
@@ -285,15 +280,15 @@ definitionListStart = try $ do
285280-- the term defined, then spaces and ":=". The definition follows, on
286281-- the same single line, or spaned on multiple line, after a line
287282-- break.
288−definitionListItem :: PandocMonad m => TextileParser m (Inlines, [Blocks])
283+definitionListItem :: PandocMonad m => ParserT Text ParserState m (Inlines, [Blocks])
289284definitionListItem = try $ do
290285 term <- mconcat . intersperse B.linebreak <$> many1 definitionListStart
291286 def' <- string ":=" *> optional whitespace *> (multilineDef <|> inlineDef)
292287 return (term, def')
293− where inlineDef :: PandocMonad m => TextileParser m [Blocks]
288+ where inlineDef :: PandocMonad m => ParserT Text ParserState m [Blocks]
294289 inlineDef = liftM (\d -> [B.plain d])
295290 $ optional whitespace >> (trimInlines . mconcat <$> many inline) <* newline
296− multilineDef :: PandocMonad m => TextileParser m [Blocks]
291+ multilineDef :: PandocMonad m => ParserT Text ParserState m [Blocks]
297292 multilineDef = try $ do
298293 optional whitespace >> newline
299294 s <- T.pack <$> many1Till anyChar (try (string "=:" >> newline))
@@ -304,7 +299,7 @@ definitionListItem = try $ do
304299-- raw content
305300
306301-- | A raw Html Block, optionally followed by blanklines
307−rawHtmlBlock :: PandocMonad m => TextileParser m Blocks
302+rawHtmlBlock :: PandocMonad m => ParserT Text ParserState m Blocks
308303rawHtmlBlock = try $ do
309304 skipMany spaceChar
310305 (_,b) <- htmlTag isBlockTag
@@ -312,14 +307,14 @@ rawHtmlBlock = try $ do
312307 return $ B.rawBlock "html" b
313308
314309-- | Raw block of LaTeX content
315−rawLaTeXBlock' :: PandocMonad m => TextileParser m Blocks
310+rawLaTeXBlock' :: PandocMonad m => ParserT Text ParserState m Blocks
316311rawLaTeXBlock' = do
317312 guardEnabled Ext_raw_tex
318313 B.rawBlock "latex" <$> (rawLaTeXBlock <* spaces)
319314
320315
321316-- | In textile, paragraphs are separated by blank lines.
322−para :: PandocMonad m => TextileParser m Blocks
317+para :: PandocMonad m => ParserT Text ParserState m Blocks
323318para = B.para . trimInlines . mconcat <$> many1 inline
324319
325320-- Tables
@@ -330,7 +325,7 @@ toAlignment '>' = AlignRight
330325toAlignment '=' = AlignCenter
331326toAlignment _ = AlignDefault
332327
333−cellAttributes :: PandocMonad m => TextileParser m (Bool, Alignment)
328+cellAttributes :: PandocMonad m => ParserT Text ParserState m (Bool, Alignment)
334329cellAttributes = try $ do
335330 isHeader <- option False (True <$ char '_')
336331 -- we just ignore colspan and rowspan markers:
@@ -343,7 +338,7 @@ cellAttributes = try $ do
343338 return (isHeader, alignment)
344339
345340-- | A table cell spans until a pipe |
346−tableCell :: PandocMonad m => TextileParser m ((Bool, Alignment), Blocks)
341+tableCell :: PandocMonad m => ParserT Text ParserState m ((Bool, Alignment), Blocks)
347342tableCell = try $ do
348343 char '|'
349344 (isHeader, alignment) <- option (False, AlignDefault) cellAttributes
@@ -354,7 +349,7 @@ tableCell = try $ do
354349 return ((isHeader, alignment), B.plain content)
355350
356351-- | A table row is made of many table cells
357−tableRow :: PandocMonad m => TextileParser m [((Bool, Alignment), Blocks)]
352+tableRow :: PandocMonad m => ParserT Text ParserState m [((Bool, Alignment), Blocks)]
358353tableRow = try $ do
359354 -- skip optional row attributes
360355 optional $ try $ do
@@ -364,7 +359,7 @@ tableRow = try $ do
364359 many1 tableCell <* char '|' <* blankline
365360
366361-- | A table with an optional header.
367−table :: PandocMonad m => TextileParser m Blocks
362+table :: PandocMonad m => ParserT Text ParserState m Blocks
368363table = try $ do
369364 -- ignore table attributes
370365 caption <- option mempty $ try $ do
@@ -380,9 +375,8 @@ table = try $ do
380375 (toprow:rest) | any (fst . fst) toprow ->
381376 (toprow, rest)
382377 _ -> (mempty, rawrows)
383− let nbOfCols = maximum $ fmap length (headers :| rows)
384− let aligns = map (maybe AlignDefault minimum . nonEmpty) $
385− transpose $ map (map (snd . fst)) (headers:rows)
378+ let nbOfCols = maximum $ map length (headers:rows)
379+ let aligns = map minimum $ transpose $ map (map (snd . fst)) (headers:rows)
386380 let toRow = Row nullAttr . map B.simpleCell
387381 toHeaderRow l = [toRow l | not (null l)]
388382 return $ B.table (B.simpleCaption $ B.plain caption)
@@ -392,7 +386,7 @@ table = try $ do
392386 (TableFoot nullAttr [])
393387
394388-- | Ignore markers for cols, thead, tfoot.
395−ignorableRow :: PandocMonad m => TextileParser m ()
389+ignorableRow :: PandocMonad m => ParserT Text ParserState m ()
396390ignorableRow = try $ do
397391 char '|'
398392 oneOf ":^-~"
@@ -401,7 +395,7 @@ ignorableRow = try $ do
401395 _ <- anyLine
402396 return ()
403397
404−explicitBlockStart :: PandocMonad m => Text -> TextileParser m ()
398+explicitBlockStart :: PandocMonad m => Text -> ParserT Text ParserState m ()
405399explicitBlockStart name = try $ do
406400 string (T.unpack name)
407401 attributes
@@ -413,8 +407,8 @@ explicitBlockStart name = try $ do
413407-- However, they can be used to set HTML/CSS attributes when needed.
414408maybeExplicitBlock :: PandocMonad m
415409 => Text -- ^ block tag name
416− -> TextileParser m Blocks -- ^ implicit block
417− -> TextileParser m Blocks
410+ -> ParserT Text ParserState m Blocks -- ^ implicit block
411+ -> ParserT Text ParserState m Blocks
418412maybeExplicitBlock name blk = try $ do
419413 optional $ explicitBlockStart name
420414 blk
@@ -427,11 +421,11 @@ maybeExplicitBlock name blk = try $ do
427421
428422
429423-- | Any inline element
430−inline :: PandocMonad m => TextileParser m Inlines
424+inline :: PandocMonad m => ParserT Text ParserState m Inlines
431425inline = choice inlineParsers <?> "inline"
432426
433427-- | Inline parsers tried in order
434−inlineParsers :: PandocMonad m => [TextileParser m Inlines]
428+inlineParsers :: PandocMonad m => [ParserT Text ParserState m Inlines]
435429inlineParsers = [ str
436430 , whitespace
437431 , endline
@@ -451,7 +445,7 @@ inlineParsers = [ str
451445 ]
452446
453447-- | Inline markups
454−inlineMarkup :: PandocMonad m => TextileParser m Inlines
448+inlineMarkup :: PandocMonad m => ParserT Text ParserState m Inlines
455449inlineMarkup = choice [ simpleInline (string "??") (B.cite [])
456450 , simpleInline (string "**") B.strong
457451 , simpleInline (string "__") B.emph
@@ -465,29 +459,29 @@ inlineMarkup = choice [ simpleInline (string "??") (B.cite [])
465459 ]
466460
467461-- | Trademark, registered, copyright
468−mark :: PandocMonad m => TextileParser m Inlines
462+mark :: PandocMonad m => ParserT Text st m Inlines
469463mark = try $ char '(' >> (try tm <|> try reg <|> copy)
470464
471−reg :: PandocMonad m => TextileParser m Inlines
465+reg :: PandocMonad m => ParserT Text st m Inlines
472466reg = do
473467 oneOf "Rr"
474468 char ')'
475469 return $ B.str "\174"
476470
477−tm :: PandocMonad m => TextileParser m Inlines
471+tm :: PandocMonad m => ParserT Text st m Inlines
478472tm = do
479473 oneOf "Tt"
480474 oneOf "Mm"
481475 char ')'
482476 return $ B.str "\8482"
483477
484−copy :: PandocMonad m => TextileParser m Inlines
478+copy :: PandocMonad m => ParserT Text st m Inlines
485479copy = do
486480 oneOf "Cc"
487481 char ')'
488482 return $ B.str "\169"
489483
490−note :: PandocMonad m => TextileParser m Inlines
484+note :: PandocMonad m => ParserT Text ParserState m Inlines
491485note = try $ do
492486 ref <- char '[' *> many1 digit <* char ']'
493487 notes <- stateNotes <$> getState
@@ -511,13 +505,13 @@ wordBoundaries :: [Char]
511505wordBoundaries = markupChars <> stringBreakers
512506
513507-- | Parse a hyphened sequence of words
514−hyphenedWords :: PandocMonad m => TextileParser m Text
508+hyphenedWords :: PandocMonad m => ParserT Text ParserState m Text
515509hyphenedWords = do
516510 x <- wordChunk
517511 xs <- many (try $ char '-' >> wordChunk)
518512 return $ T.intercalate "-" (x:xs)
519513
520−wordChunk :: PandocMonad m => TextileParser m Text
514+wordChunk :: PandocMonad m => ParserT Text ParserState m Text
521515wordChunk = try $ do
522516 hd <- noneOf wordBoundaries
523517 tl <- many ( noneOf wordBoundaries <|>
@@ -526,7 +520,7 @@ wordChunk = try $ do
526520 return $ T.pack $ hd:tl
527521
528522-- | Any string
529−str :: PandocMonad m => TextileParser m Inlines
523+str :: PandocMonad m => ParserT Text ParserState m Inlines
530524str = do
531525 baseStr <- hyphenedWords
532526 -- RedCloth compliance : if parsed word is uppercase and immediately
@@ -539,11 +533,11 @@ str = do
539533 return $ B.str fullStr
540534
541535-- | Some number of space chars
542−whitespace :: PandocMonad m => TextileParser m Inlines
536+whitespace :: PandocMonad m => ParserT Text st m Inlines
543537whitespace = many1 spaceChar >> return B.space <?> "whitespace"
544538
545539-- | In Textile, an isolated endline character is a line break
546−endline :: PandocMonad m => TextileParser m Inlines
540+endline :: PandocMonad m => ParserT Text ParserState m Inlines
547541endline = try $ do
548542 newline
549543 notFollowedBy blankline
@@ -551,18 +545,18 @@ endline = try $ do
551545 notFollowedBy rawHtmlBlock
552546 return B.linebreak
553547
554−rawHtmlInline :: PandocMonad m => TextileParser m Inlines
548+rawHtmlInline :: PandocMonad m => ParserT Text ParserState m Inlines
555549rawHtmlInline = B.rawInline "html" . snd <$> htmlTag isInlineTag
556550
557551-- | Raw LaTeX Inline
558−rawLaTeXInline' :: PandocMonad m => TextileParser m Inlines
552+rawLaTeXInline' :: PandocMonad m => ParserT Text ParserState m Inlines
559553rawLaTeXInline' = try $ do
560554 guardEnabled Ext_raw_tex
561555 B.rawInline "latex" <$> rawLaTeXInline
562556
563557-- | Textile standard link syntax is "label":target. But we
564558-- can also have ["label":target].
565−link :: PandocMonad m => TextileParser m Inlines
559+link :: PandocMonad m => ParserT Text ParserState m Inlines
566560link = try $ do
567561 bracketed <- (True <$ char '[') <|> return False
568562 char '"' *> notFollowedBy (oneOf " \t\n\r")
@@ -582,7 +576,7 @@ link = try $ do
582576 else B.spanWith attr $ B.link url "" name'
583577
584578-- | image embedding
585−image :: PandocMonad m => TextileParser m Inlines
579+image :: PandocMonad m => ParserT Text ParserState m Inlines
586580image = try $ do
587581 char '!' >> notFollowedBy space
588582 (ident, cls, kvs) <- attributes
@@ -594,51 +588,51 @@ image = try $ do
594588 char '!'
595589 return $ B.imageWith attr src alt (B.str alt)
596590
597−escapedInline :: PandocMonad m => TextileParser m Inlines
591+escapedInline :: PandocMonad m => ParserT Text ParserState m Inlines
598592escapedInline = escapedEqs <|> escapedTag
599593
600−escapedEqs :: PandocMonad m => TextileParser m Inlines
594+escapedEqs :: PandocMonad m => ParserT Text ParserState m Inlines
601595escapedEqs = B.str . T.pack <$>
602596 try (string "==" *> manyTill anyChar' (try $ string "=="))
603597
604598-- | literal text escaped btw <notextile> tags
605−escapedTag :: PandocMonad m => TextileParser m Inlines
599+escapedTag :: PandocMonad m => ParserT Text ParserState m Inlines
606600escapedTag = B.str . T.pack <$>
607601 try (string "<notextile>" *>
608602 manyTill anyChar' (try $ string "</notextile>"))
609603
610604-- | Any special symbol defined in wordBoundaries
611−symbol :: PandocMonad m => TextileParser m Inlines
605+symbol :: PandocMonad m => ParserT Text ParserState m Inlines
612606symbol = B.str . T.singleton <$> (notFollowedBy newline *>
613607 notFollowedBy rawHtmlBlock *>
614608 oneOf wordBoundaries)
615609
616610-- | Inline code
617−code :: PandocMonad m => TextileParser m Inlines
611+code :: PandocMonad m => ParserT Text ParserState m Inlines
618612code = code1 <|> code2
619613
620614-- any character except a newline before a blank line
621−anyChar' :: PandocMonad m => TextileParser m Char
615+anyChar' :: PandocMonad m => ParserT Text ParserState m Char
622616anyChar' =
623617 satisfy (/='\n') <|>
624618 try (char '\n' <* notFollowedBy blankline)
625619
626−code1 :: PandocMonad m => TextileParser m Inlines
620+code1 :: PandocMonad m => ParserT Text ParserState m Inlines
627621code1 = B.code . T.pack <$> surrounded (char '@') anyChar'
628622
629−code2 :: PandocMonad m => TextileParser m Inlines
623+code2 :: PandocMonad m => ParserT Text ParserState m Inlines
630624code2 = do
631625 htmlTag (tagOpen (=="tt") null)
632626 B.code . T.pack <$> manyTill anyChar' (try $ htmlTag $ tagClose (=="tt"))
633627
634628-- | Html / CSS attributes
635−attributes :: PandocMonad m => TextileParser m Attr
636−attributes = foldl' (flip ($)) ("",[],[]) <$>
629+attributes :: PandocMonad m => ParserT Text ParserState m Attr
630+attributes = foldl (flip ($)) ("",[],[]) <$>
637631 try (do special <- option id specialAttribute
638632 attrs <- many attribute
639633 return (special : attrs))
640634
641−specialAttribute :: PandocMonad m => TextileParser m (Attr -> Attr)
635+specialAttribute :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
642636specialAttribute = do
643637 alignStr <- ("center" <$ char '=') <|>
644638 ("justify" <$ try (string "<>")) <|>
@@ -647,11 +641,11 @@ specialAttribute = do
647641 notFollowedBy spaceChar
648642 return $ addStyle $ T.pack $ "text-align:" ++ alignStr
649643
650−attribute :: PandocMonad m => TextileParser m (Attr -> Attr)
644+attribute :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
651645attribute = try $
652646 (classIdAttr <|> styleAttr <|> langAttr) <* notFollowedBy spaceChar
653647
654−classIdAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
648+classIdAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
655649classIdAttr = try $ do -- (class class #id)
656650 char '('
657651 ws <- T.words `fmap` T.pack <$> manyTill anyChar' (char ')')
@@ -663,7 +657,7 @@ classIdAttr = try $ do -- (class class #id)
663657 classes'
664658 -> return $ \(_,_,keyvals) -> ("",classes',keyvals)
665659
666−styleAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
660+styleAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
667661styleAttr = do
668662 style <- try $ enclosed (char '{') (char '}') anyChar'
669663 return $ addStyle $ T.pack style
@@ -674,23 +668,23 @@ addStyle style (id',classes,keyvals) =
674668 where keyvals' = ("style", style') : [(k,v) | (k,v) <- keyvals, k /= "style"]
675669 style' = style <> ";" <> T.concat [v | ("style",v) <- keyvals]
676670
677−langAttr :: PandocMonad m => TextileParser m (Attr -> Attr)
671+langAttr :: PandocMonad m => ParserT Text ParserState m (Attr -> Attr)
678672langAttr = do
679673 lang <- try $ enclosed (char '[') (char ']') alphaNum
680674 return $ \(id',classes,keyvals) -> (id',classes,("lang",T.pack lang):keyvals)
681675
682676-- | Parses material surrounded by a parser.
683677surrounded :: (PandocMonad m, Show t)
684− => ParserT Sources st m t -- ^ surrounding parser
685− -> ParserT Sources st m a -- ^ content parser (to be used repeatedly)
686− -> ParserT Sources st m [a]
678+ => ParserT Text st m t -- ^ surrounding parser
679+ -> ParserT Text st m a -- ^ content parser (to be used repeatedly)
680+ -> ParserT Text st m [a]
687681surrounded border =
688682 enclosed (border *> notFollowedBy (oneOf " \t\n\r")) (try border)
689683
690684simpleInline :: PandocMonad m
691− => TextileParser m t -- ^ surrounding parser
685+ => ParserT Text ParserState m t -- ^ surrounding parser
692686 -> (Inlines -> Inlines) -- ^ Inline constructor
693− -> TextileParser m Inlines -- ^ content parser (to be used repeatedly)
687+ -> ParserT Text ParserState m Inlines -- ^ content parser (to be used repeatedly)
694688simpleInline border construct = try $ do
695689 notAfterString
696690 border *> notFollowedBy (oneOf " \t\n\r")
@@ -704,7 +698,7 @@ simpleInline border construct = try $ do
704698 then body
705699 else B.spanWith attr body
706700
707−groupedInlineMarkup :: PandocMonad m => TextileParser m Inlines
701+groupedInlineMarkup :: PandocMonad m => ParserT Text ParserState m Inlines
708702groupedInlineMarkup = try $ do
709703 char '['
710704 sp1 <- option mempty $ B.space <$ whitespace
@@ -713,5 +707,5 @@ groupedInlineMarkup = try $ do
713707 char ']'
714708 return $ sp1 <> result <> sp2
715709
716−eof' :: Monad m => ParserT Sources s m Char
710+eof' :: Monad m => ParserT Text s m Char
717711eof' = '\n' <$ eof
src/Text/Pandoc/Readers/TikiWiki.hs+6−6
@@ -30,23 +30,23 @@ import Text.Pandoc.Definition
3030import Text.Pandoc.Logging (Verbosity (..))
3131import Text.Pandoc.Options
3232import Text.Pandoc.Parsing hiding (enclosed, nested)
33−import Text.Pandoc.Shared (safeRead)
33+import Text.Pandoc.Shared (crFilter, safeRead)
3434import Text.Pandoc.XML (fromEntities)
3535import Text.Printf (printf)
3636
3737-- | Read TikiWiki from an input string and return a Pandoc document.
38−readTikiWiki :: (PandocMonad m, ToSources a)
38+readTikiWiki :: PandocMonad m
3939 => ReaderOptions
40− -> a
40+ -> Text
4141 -> m Pandoc
4242readTikiWiki opts s = do
43− let sources = ensureFinalNewlines 2 (toSources s)
44− res <- readWithM parseTikiWiki def{ stateOptions = opts } sources
43+ res <- readWithM parseTikiWiki def{ stateOptions = opts }
44+ (crFilter s <> "\n\n")
4545 case res of
4646 Left e -> throwError e
4747 Right d -> return d
4848
49−type TikiWikiParser = ParserT Sources ParserState
49+type TikiWikiParser = ParserT Text ParserState
5050
5151--
5252-- utility functions
src/Text/Pandoc/Readers/Txt2Tags.hs+21−47
@@ -1,5 +1,4 @@
11{-# LANGUAGE OverloadedStrings #-}
2−{-# LANGUAGE LambdaCase #-}
32{- |
43 Module : Text.Pandoc.Readers.Txt2Tags
54 Copyright : Copyright (C) 2014 Matthew Pickering
@@ -20,7 +19,6 @@ import Control.Monad.Except (catchError, throwError)
2019import Control.Monad.Reader (Reader, asks, runReader)
2120import Data.Default
2221import Data.List (intercalate, transpose)
23−import Data.List.NonEmpty (nonEmpty)
2422import Data.Maybe (fromMaybe)
2523import Data.Text (Text)
2624import qualified Data.Text as T
@@ -33,9 +31,9 @@ import Data.Time (defaultTimeLocale)
3331import Text.Pandoc.Definition
3432import Text.Pandoc.Options
3533import Text.Pandoc.Parsing hiding (space, spaces, uri)
36−import Text.Pandoc.Shared (compactify, compactifyDL, escapeURI)
34+import Text.Pandoc.Shared (compactify, compactifyDL, crFilter, escapeURI)
3735
38−type T2T = ParserT Sources ParserState (Reader T2TMeta)
36+type T2T = ParserT Text ParserState (Reader T2TMeta)
3937
4038-- | An object for the T2T macros meta information
4139-- the contents of each field is simply substituted verbatim into the file
@@ -55,28 +53,25 @@ getT2TMeta = do
5553 inps <- P.getInputFiles
5654 outp <- fromMaybe "" <$> P.getOutputFile
5755 curDate <- formatTime defaultTimeLocale "%F" <$> P.getZonedTime
58− curMtime <- catchError
59− (mapM P.getModificationTime inps >>=
60− (\case
61− Nothing ->
62− formatTime defaultTimeLocale "%T" <$> P.getZonedTime
63− Just ts -> return $
64− formatTime defaultTimeLocale "%T" $ maximum ts)
65− . nonEmpty)
66− (const (return ""))
67− return $ T2TMeta (T.pack curDate) (T.pack curMtime)
68− (intercalate ", " inps) outp
56+ let getModTime = fmap (formatTime defaultTimeLocale "%T") .
57+ P.getModificationTime
58+ curMtime <- case inps of
59+ [] -> formatTime defaultTimeLocale "%T" <$> P.getZonedTime
60+ _ -> catchError
61+ (maximum <$> mapM getModTime inps)
62+ (const (return ""))
63+ return $ T2TMeta (T.pack curDate) (T.pack curMtime) (intercalate ", " inps) outp
6964
7065-- | Read Txt2Tags from an input string returning a Pandoc document
71−readTxt2Tags :: (PandocMonad m, ToSources a)
66+readTxt2Tags :: PandocMonad m
7267 => ReaderOptions
73− -> a
68+ -> Text
7469 -> m Pandoc
7570readTxt2Tags opts s = do
76− let sources = ensureFinalNewlines 2 (toSources s)
7771 meta <- getT2TMeta
7872 let parsed = flip runReader meta $
79− readWithM parseT2T (def {stateOptions = opts}) sources
73+ readWithM parseT2T (def {stateOptions = opts}) $
74+ crFilter s <> "\n\n"
8075 case parsed of
8176 Right result -> return result
8277 Left e -> throwError e
@@ -266,9 +261,9 @@ table = try $ do
266261 rows <- many1 (many commentLine *> tableRow)
267262 let columns = transpose rows
268263 let ncolumns = length columns
269− let aligns = map (fromMaybe AlignDefault . foldr findAlign Nothing) columns
264+ let aligns = map (foldr1 findAlign . map fst) columns
270265 let rows' = map (map snd) rows
271− let size = maybe 0 maximum $ nonEmpty $ map length rows'
266+ let size = maximum (map length rows')
272267 let rowsPadded = map (pad size) rows'
273268 let headerPadded = if null tableHeader then mempty else pad size tableHeader
274269 let toRow = Row nullAttr . map B.simpleCell
@@ -283,11 +278,10 @@ pad :: (Monoid a) => Int -> [a] -> [a]
283278pad n xs = xs ++ replicate (n - length xs) mempty
284279
285280
286−findAlign :: (Alignment, a) -> Maybe Alignment -> Maybe Alignment
287−findAlign (x,_) (Just y)
288− | x == y = Just x
289− | otherwise = Just AlignDefault
290−findAlign (x,_) Nothing = Just x
281+findAlign :: Alignment -> Alignment -> Alignment
282+findAlign x y
283+ | x == y = x
284+ | otherwise = AlignDefault
291285
292286headerRow :: T2T [(Alignment, Blocks)]
293287headerRow = genericRow (string "||")
@@ -478,29 +472,9 @@ macro = try $ do
478472-- raw URLs in text are automatically linked
479473url :: T2T Inlines
480474url = try $ do
481− (rawUrl, escapedUrl) <- try uri <|> emailAddress'
475+ (rawUrl, escapedUrl) <- try uri <|> emailAddress
482476 return $ B.link rawUrl "" (B.str escapedUrl)
483477
484−emailAddress' :: T2T (Text, Text)
485−emailAddress' = do
486− (base, mailURI) <- emailAddress
487− query <- option "" emailQuery
488− return (base <> query, mailURI <> query)
489−
490−emailQuery :: T2T Text
491−emailQuery = do
492− char '?'
493− parts <- kv `sepBy1` (char '&')
494− return $ "?" <> T.intercalate "&" parts
495−
496−kv :: T2T Text
497−kv = do
498− k <- T.pack <$> many1 alphaNum
499− char '='
500− let vchar = alphaNum <|> try (oneOf "%._/~:,=$@&+-;*" <* lookAhead alphaNum)
501− v <- T.pack <$> many1 vchar
502− return (k <> "=" <> v)
503−
504478uri :: T2T (Text, Text)
505479uri = try $ do
506480 address <- t2tURI
src/Text/Pandoc/Readers/Vimwiki.hs+7−12
@@ -74,28 +74,23 @@ import Text.Pandoc.Parsing (ParserState, ParserT, blanklines, emailAddress,
7474 many1Till, orderedListMarker, readWithM,
7575 registerHeader, spaceChar, stateMeta,
7676 stateOptions, uri, manyTillChar, manyChar, textStr,
77− many1Char, countChar, many1TillChar,
78− alphaNum, anyChar, char, newline, noneOf, oneOf,
79− space, spaces, string)
80−import Text.Pandoc.Sources (ToSources(..), Sources)
81−import Text.Pandoc.Shared (splitTextBy, stringify, stripFirstAndLast,
77+ many1Char, countChar, many1TillChar)
78+import Text.Pandoc.Shared (crFilter, splitTextBy, stringify, stripFirstAndLast,
8279 isURI, tshow)
80+import Text.Parsec.Char (alphaNum, anyChar, char, newline, noneOf, oneOf, space,
81+ spaces, string)
8382import Text.Parsec.Combinator (between, choice, eof, lookAhead, many1,
8483 manyTill, notFollowedBy, option, skipMany1)
8584import Text.Parsec.Prim (getState, many, try, updateState, (<|>))
8685
87−readVimwiki :: (PandocMonad m, ToSources a)
88− => ReaderOptions
89− -> a
90− -> m Pandoc
86+readVimwiki :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
9187readVimwiki opts s = do
92− let sources = toSources s
93− res <- readWithM parseVimwiki def{ stateOptions = opts } sources
88+ res <- readWithM parseVimwiki def{ stateOptions = opts } $ crFilter s
9489 case res of
9590 Left e -> throwError e
9691 Right result -> return result
9792
98−type VwParser = ParserT Sources ParserState
93+type VwParser = ParserT Text ParserState
9994
10095
10196-- constants
src/Text/Pandoc/SelfContained.hs+7−4
@@ -16,6 +16,7 @@ the HTML using data URIs.
1616module Text.Pandoc.SelfContained ( makeDataURI, makeSelfContained ) where
1717import Codec.Compression.GZip as Gzip
1818import Control.Applicative ((<|>))
19+import Control.Monad.Except (throwError)
1920import Control.Monad.Trans (lift)
2021import Data.ByteString (ByteString)
2122import Data.ByteString.Base64
@@ -28,6 +29,7 @@ import System.FilePath (takeDirectory, takeExtension, (</>))
2829import Text.HTML.TagSoup
2930import Text.Pandoc.Class.PandocMonad (PandocMonad (..), fetchItem,
3031 getInputFiles, report, setInputFiles)
32+import Text.Pandoc.Error
3133import Text.Pandoc.Logging
3234import Text.Pandoc.MIME (MimeType)
3335import Text.Pandoc.Shared (isURI, renderTags', trim)
@@ -242,10 +244,11 @@ getData mimetype src
242244 let raw' = if ext `elem` [".gz", ".svgz"]
243245 then B.concat $ L.toChunks $ Gzip.decompress $ L.fromChunks [raw]
244246 else raw
245− let mime = case (mimetype, respMime) of
246− ("",Nothing) -> "application/octet-stream"
247− (x, Nothing) -> x
248− (_, Just x ) -> x
247+ mime <- case (mimetype, respMime) of
248+ ("",Nothing) -> throwError $ PandocSomeError
249+ $ "Could not determine mime type for `" <> src <> "'"
250+ (x, Nothing) -> return x
251+ (_, Just x ) -> return x
249252 result <- if "text/css" `T.isPrefixOf` mime
250253 then do
251254 oldInputs <- getInputFiles
src/Text/Pandoc/Shared.hs+56−29
@@ -25,7 +25,11 @@ module Text.Pandoc.Shared (
2525 ordNub,
2626 findM,
2727 -- * Text processing
28+ ToString (..),
29+ ToText (..),
2830 tshow,
31+ backslashEscapes,
32+ escapeStringUsing,
2933 elemText,
3034 notElemText,
3135 stripTrailingNewlines,
@@ -105,7 +109,7 @@ import qualified Data.Bifunctor as Bifunctor
105109import Data.Char (isAlpha, isLower, isSpace, isUpper, toLower, isAlphaNum,
106110 generalCategory, GeneralCategory(NonSpacingMark,
107111 SpacingCombiningMark, EnclosingMark, ConnectorPunctuation))
108−import Data.List (find, intercalate, intersperse, sortOn, foldl')
112+import Data.List (find, intercalate, intersperse, sortOn)
109113import qualified Data.Map as M
110114import Data.Maybe (mapMaybe, fromMaybe)
111115import Data.Monoid (Any (..))
@@ -123,7 +127,7 @@ import Text.HTML.TagSoup (RenderOptions (..), Tag (..), renderOptions,
123127import Text.Pandoc.Builder (Blocks, Inlines, ToMetaValue (..))
124128import qualified Text.Pandoc.Builder as B
125129import Data.Time
126−import Text.Pandoc.Asciify (toAsciiText)
130+import Text.Pandoc.Asciify (toAsciiChar)
127131import Text.Pandoc.Definition
128132import Text.Pandoc.Extensions (Extensions, Extension(..), extensionEnabled)
129133import Text.Pandoc.Generic (bottomUp)
@@ -153,20 +157,12 @@ splitTextBy isSep t
153157 in first : splitTextBy isSep (T.dropWhile isSep rest)
154158
155159splitTextByIndices :: [Int] -> T.Text -> [T.Text]
156−splitTextByIndices ns = splitTextByRelIndices (zipWith (-) ns (0:ns)) . T.unpack
160+splitTextByIndices ns = splitTextByRelIndices (zipWith (-) ns (0:ns))
157161 where
158− splitTextByRelIndices [] cs = [T.pack cs]
159− splitTextByRelIndices (x:xs) cs =
160− let (first, rest) = splitAt' x cs
161− in T.pack first : splitTextByRelIndices xs rest
162−
163−-- Note: don't replace this with T.splitAt, which is not sensitive
164−-- to character widths!
165−splitAt' :: Int -> [Char] -> ([Char],[Char])
166−splitAt' _ [] = ([],[])
167−splitAt' n xs | n <= 0 = ([],xs)
168−splitAt' n (x:xs) = (x:ys,zs)
169− where (ys,zs) = splitAt' (n - charWidth x) xs
162+ splitTextByRelIndices [] t = [t]
163+ splitTextByRelIndices (x:xs) t =
164+ let (first, rest) = T.splitAt x t
165+ in first : splitTextByRelIndices xs rest
170166
171167ordNub :: (Ord a) => [a] -> [a]
172168ordNub l = go Set.empty l
@@ -187,9 +183,38 @@ findM p = foldr go (pure Nothing)
187183-- Text processing
188184--
189185
186+class ToString a where
187+ toString :: a -> String
188+
189+instance ToString String where
190+ toString = id
191+
192+instance ToString T.Text where
193+ toString = T.unpack
194+
195+class ToText a where
196+ toText :: a -> T.Text
197+
198+instance ToText String where
199+ toText = T.pack
200+
201+instance ToText T.Text where
202+ toText = id
203+
190204tshow :: Show a => a -> T.Text
191205tshow = T.pack . show
192206
207+-- | Returns an association list of backslash escapes for the
208+-- designated characters.
209+backslashEscapes :: [Char] -- ^ list of special characters to escape
210+ -> [(Char, T.Text)]
211+backslashEscapes = map (\ch -> (ch, T.pack ['\\',ch]))
212+
213+-- | Escape a string of characters, using an association list of
214+-- characters and strings.
215+escapeStringUsing :: [(Char, T.Text)] -> T.Text -> T.Text
216+escapeStringUsing tbl = T.concatMap $ \c -> fromMaybe (T.singleton c) $ lookup c tbl
217+
193218-- | @True@ exactly when the @Char@ appears in the @Text@.
194219elemText :: Char -> T.Text -> Bool
195220elemText c = T.any (== c)
@@ -298,7 +323,6 @@ tabFilter tabStop = T.unlines . map go . T.lines
298323 (tabStop - (T.length s1 `mod` tabStop)) (T.pack " ")
299324 <> go (T.drop 1 s2)
300325
301−{-# DEPRECATED crFilter "readers filter crs automatically" #-}
302326-- | Strip out DOS line endings.
303327crFilter :: T.Text -> T.Text
304328crFilter = T.filter (/= '\r')
@@ -440,20 +464,22 @@ plainToPara :: Block -> Block
440464plainToPara (Plain ils) = Para ils
441465plainToPara x = x
442466
443−
444467-- | Like @compactify@, but acts on items of definition lists.
445468compactifyDL :: [(Inlines, [Blocks])] -> [(Inlines, [Blocks])]
446469compactifyDL items =
447− case reverse items of
448− ((t,ds):ys) ->
449− case reverse (map (reverse . B.toList) ds) of
450− ((Para x:xs) : zs) | not (any isPara xs) ->
451− reverse ys ++
452− [(t, reverse (map B.fromList zs) ++
453− [B.fromList (reverse (Plain x:xs))])]
454− _ -> items
455− _ -> items
456−
470+ let defs = concatMap snd items
471+ in case reverse (concatMap B.toList defs) of
472+ (Para x:xs)
473+ | not (any isPara xs) ->
474+ let (t,ds) = last items
475+ lastDef = B.toList $ last ds
476+ ds' = init ds ++
477+ if null lastDef
478+ then [B.fromList lastDef]
479+ else [B.fromList $ init lastDef ++ [Plain x]]
480+ in init items ++ [(t, ds')]
481+ | otherwise -> items
482+ _ -> items
457483
458484-- | Combine a list of lines by adding hard linebreaks.
459485combineLines :: [[Inline]] -> [Inline]
@@ -487,7 +513,7 @@ inlineListToIdentifier exts =
487513 | otherwise = T.dropWhile (not . isAlpha)
488514 filterAscii
489515 | extensionEnabled Ext_ascii_identifiers exts
490− = toAsciiText
516+ = T.pack . mapMaybe toAsciiChar . T.unpack
491517 | otherwise = id
492518 toIdent
493519 | extensionEnabled Ext_gfm_auto_identifiers exts =
@@ -816,7 +842,7 @@ mapLeft = Bifunctor.first
816842-- > collapseFilePath "parent/foo/.." == "parent"
817843-- > collapseFilePath "/parent/foo/../../bar" == "/bar"
818844collapseFilePath :: FilePath -> FilePath
819−collapseFilePath = Posix.joinPath . reverse . foldl' go [] . splitDirectories
845+collapseFilePath = Posix.joinPath . reverse . foldl go [] . splitDirectories
820846 where
821847 go rs "." = rs
822848 go r@(p:rs) ".." = case p of
@@ -853,6 +879,7 @@ filteredFilesFromArchive zf f =
853879 fileAndBinary :: Archive -> FilePath -> Maybe (FilePath, BL.ByteString)
854880 fileAndBinary a fp = findEntryByPath fp a >>= \e -> Just (fp, fromEntry e)
855881
882+
856883--
857884-- IANA URIs
858885--
src/Text/Pandoc/Sources.hsdeleted−195
@@ -1,195 +0,0 @@
1−{-# LANGUAGE FlexibleInstances #-}
2−{-# LANGUAGE FlexibleContexts #-}
3−{-# LANGUAGE MultiParamTypeClasses #-}
4−{-# LANGUAGE GeneralizedNewtypeDeriving #-}
5−{-# LANGUAGE OverloadedStrings #-}
6−{- |
7− Module : Text.Pandoc.Sources
8− Copyright : Copyright (C) 2021 John MacFarlane
9− License : GNU GPL, version 2 or above
10−
11− Maintainer : John MacFarlane <jgm@berkeley.edu>
12− Stability : alpha
13− Portability : portable
14−
15−Defines Sources object to be used as input to pandoc parsers and redefines Char
16−parsers so they get source position information from it.
17−-}
18−
19−module Text.Pandoc.Sources
20− ( Sources(..)
21− , ToSources(..)
22− , UpdateSourcePos(..)
23− , sourcesToText
24− , initialSourceName
25− , addToSources
26− , ensureFinalNewlines
27− , addToInput
28− , satisfy
29− , oneOf
30− , noneOf
31− , anyChar
32− , char
33− , string
34− , newline
35− , space
36− , spaces
37− , letter
38− , digit
39− , hexDigit
40− , alphaNum
41− )
42−where
43−import qualified Text.Parsec as P
44−import Text.Parsec (Stream(..), ParsecT)
45−import Text.Parsec.Pos as P
46−import Data.Text (Text)
47−import qualified Data.Text as T
48−import Data.Char (isSpace, isLetter, isAlphaNum, isDigit, isHexDigit)
49−import Data.String (IsString(..))
50−import qualified Data.List.NonEmpty as NonEmpty
51−
52−-- | A list of inputs labeled with source positions. It is assumed
53−-- that the 'Text's have @\n@ line endings.
54−newtype Sources = Sources { unSources :: [(SourcePos, Text)] }
55− deriving (Show, Semigroup, Monoid)
56−
57−instance Monad m => Stream Sources m Char where
58− uncons (Sources []) = return Nothing
59− uncons (Sources ((pos,t):rest)) =
60− case T.uncons t of
61− Nothing -> uncons (Sources rest)
62− Just (c,t') -> return $ Just (c, Sources ((pos,t'):rest))
63−
64−instance IsString Sources where
65− fromString s = Sources [(P.initialPos "", T.pack (filter (/='\r') s))]
66−
67−class ToSources a where
68− toSources :: a -> Sources
69−
70−instance ToSources Text where
71− toSources t = Sources [(P.initialPos "", T.filter (/='\r') t)]
72−
73−instance ToSources [(FilePath, Text)] where
74− toSources = Sources
75− . map (\(fp,t) ->
76− (P.initialPos fp, T.snoc (T.filter (/='\r') t) '\n'))
77−
78−instance ToSources Sources where
79− toSources = id
80−
81−sourcesToText :: Sources -> Text
82−sourcesToText (Sources xs) = mconcat $ map snd xs
83−
84−addToSources :: Monad m => SourcePos -> Text -> ParsecT Sources u m ()
85−addToSources pos t = do
86− curpos <- P.getPosition
87− Sources xs <- P.getInput
88− let xs' = case xs of
89− [] -> []
90− ((_,t'):rest) -> (curpos,t'):rest
91− P.setInput $ Sources ((pos, T.filter (/='\r') t):xs')
92−
93−ensureFinalNewlines :: Int -- ^ number of trailing newlines
94− -> Sources
95− -> Sources
96−ensureFinalNewlines n (Sources xs) =
97− case NonEmpty.nonEmpty xs of
98− Nothing -> Sources [(initialPos "", T.replicate n "\n")]
99− Just lst ->
100− case NonEmpty.last lst of
101− (spos, t) ->
102− case T.length (T.takeWhileEnd (=='\n') t) of
103− len | len >= n -> Sources xs
104− | otherwise -> Sources (NonEmpty.init lst ++
105− [(spos,
106− t <> T.replicate (n - len) "\n")])
107−
108−class UpdateSourcePos s c where
109− updateSourcePos :: SourcePos -> c -> s -> SourcePos
110−
111−instance UpdateSourcePos Text Char where
112− updateSourcePos pos c _ = updatePosChar pos c
113−
114−instance UpdateSourcePos Sources Char where
115− updateSourcePos pos c sources =
116− case sources of
117− Sources [] -> updatePosChar pos c
118− Sources ((_,t):(pos',_):_)
119− | T.null t -> pos'
120− Sources _ ->
121− case c of
122− '\n' -> incSourceLine (setSourceColumn pos 1) 1
123− '\t' -> incSourceColumn pos (4 - ((sourceColumn pos - 1) `mod` 4))
124− _ -> incSourceColumn pos 1
125−
126−-- | Get name of first source in 'Sources'.
127−initialSourceName :: Sources -> FilePath
128−initialSourceName (Sources []) = ""
129−initialSourceName (Sources ((pos,_):_)) = sourceName pos
130−
131−-- | Add some text to the beginning of the input sources.
132−-- This simplifies code that expands macros.
133−addToInput :: Monad m => Text -> ParsecT Sources u m ()
134−addToInput t = do
135− Sources xs <- P.getInput
136− case xs of
137− [] -> P.setInput $ Sources [(initialPos "",t)]
138− (pos,t'):rest -> P.setInput $ Sources ((pos, t <> t'):rest)
139−
140−-- We need to redefine the parsers in Text.Parsec.Char so that they
141−-- update source positions properly from the Sources stream.
142−
143−satisfy :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
144− => (Char -> Bool) -> ParsecT s u m Char
145−satisfy f = P.tokenPrim show updateSourcePos matcher
146− where
147− matcher c = if f c then Just c else Nothing
148−
149−oneOf :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
150− => [Char] -> ParsecT s u m Char
151−oneOf cs = satisfy (`elem` cs)
152−
153−noneOf :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
154− => [Char] -> ParsecT s u m Char
155−noneOf cs = satisfy (`notElem` cs)
156−
157−anyChar :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
158− => ParsecT s u m Char
159−anyChar = satisfy (const True)
160−
161−char :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
162− => Char -> ParsecT s u m Char
163−char c = satisfy (== c)
164−
165−string :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
166− => [Char] -> ParsecT s u m [Char]
167−string = mapM char
168−
169−newline :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
170− => ParsecT s u m Char
171−newline = satisfy (== '\n')
172−
173−space :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
174− => ParsecT s u m Char
175−space = satisfy isSpace
176−
177−spaces :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
178− => ParsecT s u m ()
179−spaces = P.skipMany space P.<?> "white space"
180−
181−letter :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
182− => ParsecT s u m Char
183−letter = satisfy isLetter
184−
185−alphaNum :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
186− => ParsecT s u m Char
187−alphaNum = satisfy isAlphaNum
188−
189−digit :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
190− => ParsecT s u m Char
191−digit = satisfy isDigit
192−
193−hexDigit :: (Monad m, Stream s m Char, UpdateSourcePos s Char)
194− => ParsecT s u m Char
195−hexDigit = satisfy isHexDigit
src/Text/Pandoc/Templates.hs−4
@@ -34,7 +34,6 @@ import Control.Monad.Except (catchError, throwError)
3434import Data.Text (Text)
3535import qualified Data.Text as T
3636import Text.Pandoc.Error
37−import System.IO.Error (isDoesNotExistError)
3837
3938-- | Wrap a Monad in this if you want partials to
4039-- be taken only from the default data files.
@@ -71,9 +70,6 @@ getTemplate tp = UTF8.toText <$>
7170 PandocResourceNotFound _ ->
7271 -- see #5987 on reason for takeFileName
7372 readDataFile ("templates" </> takeFileName tp)
74− PandocIOError _ ioe | isDoesNotExistError ioe ->
75− -- see #5987 on reason for takeFileName
76− readDataFile ("templates" </> takeFileName tp)
7773 _ -> throwError e))
7874
7975-- | Get default template for the specified writer.
src/Text/Pandoc/Writers.hs+3−3
@@ -81,7 +81,6 @@ import Data.Aeson
8181import qualified Data.ByteString.Lazy as BL
8282import Data.Text (Text)
8383import qualified Data.Text as T
84−import Text.Pandoc.Shared (tshow)
8584import Text.Pandoc.Class
8685import Text.Pandoc.Definition
8786import Text.Pandoc.Options
@@ -123,6 +122,7 @@ import Text.Pandoc.Writers.Texinfo
123122import Text.Pandoc.Writers.Textile
124123import Text.Pandoc.Writers.XWiki
125124import Text.Pandoc.Writers.ZimWiki
125+import Text.Parsec.Error
126126
127127data Writer m = TextWriter (WriterOptions -> Pandoc -> m Text)
128128 | ByteStringWriter (WriterOptions -> Pandoc -> m BL.ByteString)
@@ -196,8 +196,8 @@ writers = [
196196getWriter :: PandocMonad m => Text -> m (Writer m, Extensions)
197197getWriter s =
198198 case parseFormatSpec s of
199− Left e -> throwError $ PandocAppError $
200− "Error parsing writer format " <> tshow s <> ": " <> tshow e
199+ Left e -> throwError $ PandocAppError
200+ $ T.intercalate "\n" [T.pack m | Message m <- errorMessages e]
201201 Right (writerName, extsToEnable, extsToDisable) ->
202202 case lookup writerName writers of
203203 Nothing -> throwError $
src/Text/Pandoc/Writers/AsciiDoc.hs+4−10
@@ -22,7 +22,6 @@ module Text.Pandoc.Writers.AsciiDoc (writeAsciiDoc, writeAsciiDoctor) where
2222import Control.Monad.State.Strict
2323import Data.Char (isPunctuation, isSpace)
2424import Data.List (intercalate, intersperse)
25−import Data.List.NonEmpty (NonEmpty(..))
2625import Data.Maybe (fromMaybe, isJust)
2726import qualified Data.Set as Set
2827import qualified Data.Text as T
@@ -105,11 +104,8 @@ pandocToAsciiDoc opts (Pandoc meta blocks) = do
105104
106105-- | Escape special characters for AsciiDoc.
107106escapeString :: Text -> Text
108−escapeString t
109− | T.any (== '{') t = T.concatMap escChar t
110− | otherwise = t
111− where escChar '{' = "\\{"
112− escChar c = T.singleton c
107+escapeString = escapeStringUsing escs
108+ where escs = backslashEscapes "{"
113109
114110-- | Ordered list start parser for use in Para below.
115111olMarker :: Parser Text ParserState Char
@@ -278,7 +274,7 @@ blockToAsciiDoc opts block@(Table _ blkCapt specs thead tbody tfoot) = do
278274 let colwidth = if writerWrapText opts == WrapAuto
279275 then writerColumns opts
280276 else 100000
281− let maxwidth = maximum $ fmap offset (head' :| rows')
277+ let maxwidth = maximum $ map offset (head':rows')
282278 let body = if maxwidth > colwidth then vsep rows' else vcat rows'
283279 let border = separator <> text "==="
284280 return $
@@ -499,9 +495,7 @@ inlineToAsciiDoc opts (Quoted qt lst) = do
499495 | otherwise -> [Str "``"] ++ lst ++ [Str "''"]
500496inlineToAsciiDoc _ (Code _ str) = do
501497 isAsciidoctor <- gets asciidoctorVariant
502− let escChar '`' = "\\'"
503− escChar c = T.singleton c
504− let contents = literal (T.concatMap escChar str)
498+ let contents = literal (escapeStringUsing (backslashEscapes "`") str)
505499 return $
506500 if isAsciidoctor
507501 then text "`+" <> contents <> "+`"
src/Text/Pandoc/Writers/BibTeX.hs+1−1
@@ -43,7 +43,7 @@ writeBibTeX' :: PandocMonad m => Variant -> WriterOptions -> Pandoc -> m Text
4343writeBibTeX' variant opts (Pandoc meta _) = do
4444 let mblang = case lookupMetaString "lang" meta of
4545 "" -> Nothing
46− t -> either (const Nothing) Just $ parseLang t
46+ t -> Just $ parseLang t
4747 let refs = case lookupMeta "references" meta of
4848 Just (MetaList xs) -> mapMaybe metaValueToReference xs
4949 _ -> []
src/Text/Pandoc/Writers/ConTeXt.hs+38−44
@@ -20,7 +20,7 @@ import Data.Maybe (mapMaybe)
2020import Data.Text (Text)
2121import qualified Data.Text as T
2222import Network.URI (unEscapeString)
23−import Text.Collate.Lang (Lang(..))
23+import Text.Pandoc.BCP47
2424import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
2525import Text.Pandoc.Definition
2626import Text.Pandoc.ImageSize
@@ -177,12 +177,8 @@ blockToConTeXt (Para lst) = do
177177 contents <- inlineListToConTeXt lst
178178 return $ contents <> blankline
179179blockToConTeXt (LineBlock lns) = do
180− let emptyToBlankline doc = if isEmpty doc
181− then blankline
182− else doc
183− doclines <- mapM inlineListToConTeXt lns
184− let contextLines = vcat . map emptyToBlankline $ doclines
185− return $ "\\startlines" $$ contextLines $$ "\\stoplines" <> blankline
180+ doclines <- nowrap . vcat <$> mapM inlineListToConTeXt lns
181+ return $ "\\startlines" $$ doclines $$ "\\stoplines" <> blankline
186182blockToConTeXt (BlockQuote lst) = do
187183 contents <- blockListToConTeXt lst
188184 return $ "\\startblockquote" $$ nest 0 contents $$ "\\stopblockquote" <> blankline
@@ -232,7 +228,13 @@ blockToConTeXt (OrderedList (start, style', delim) lst) = do
232228 Period -> "stopper=."
233229 OneParen -> "stopper=)"
234230 TwoParens -> "left=(,stopper=)"
235− let specs2Items = filter (not . T.null) [start', delim']
231+ let width = maximum $ map T.length $ take (length contents)
232+ (orderedListMarkers (start, style', delim))
233+ let width' = (toEnum width + 1) / 2
234+ let width'' = if width' > (1.5 :: Double)
235+ then "width=" <> tshow width' <> "em"
236+ else ""
237+ let specs2Items = filter (not . T.null) [start', delim', width'']
236238 let specs2 = if null specs2Items
237239 then ""
238240 else "[" <> T.intercalate "," specs2Items <> "]"
@@ -246,8 +248,8 @@ blockToConTeXt (OrderedList (start, style', delim) lst) = do
246248 UpperAlpha -> 'A') :
247249 if isTightList lst then ",packed]" else "]"
248250 let specs = T.pack style'' <> specs2
249− return $ "\\startenumerate" <> literal specs $$ vcat contents $$
250− "\\stopenumerate" <> blankline
251+ return $ "\\startitemize" <> literal specs $$ vcat contents $$
252+ "\\stopitemize" <> blankline
251253blockToConTeXt (DefinitionList lst) =
252254 liftM vcat $ mapM defListItemToConTeXt lst
253255blockToConTeXt HorizontalRule = return $ "\\thinrule" <> blankline
@@ -430,13 +432,9 @@ inlineToConTeXt (Link _ txt (src, _)) = do
430432 put $ st {stNextRef = next + 1}
431433 let ref = "url" <> tshow next
432434 contents <- inlineListToConTeXt txt
433− let escChar '#' = "\\#"
434− escChar '%' = "\\%"
435− escChar c = T.singleton c
436− let escContextURL = T.concatMap escChar
437435 return $ "\\useURL"
438436 <> brackets (literal ref)
439− <> brackets (literal $ escContextURL src)
437+ <> brackets (literal $ escapeStringUsing [('#',"\\#"),('%',"\\%")] src)
440438 <> (if isAutolink
441439 then empty
442440 else brackets empty <> brackets contents)
@@ -479,7 +477,7 @@ inlineToConTeXt (Note contents) = do
479477 then literal "\\footnote{" <> nest 2 (chomp contents') <> char '}'
480478 else literal "\\startbuffer " <> nest 2 (chomp contents') <>
481479 literal "\\stopbuffer\\footnote{\\getbuffer}"
482−inlineToConTeXt (Span (ident,_,kvs) ils) = do
480+inlineToConTeXt (Span (_,_,kvs) ils) = do
483481 mblang <- fromBCP47 (lookup "lang" kvs)
484482 let wrapDir txt = case lookup "dir" kvs of
485483 Just "rtl" -> braces $ "\\righttoleft " <> txt
@@ -489,11 +487,7 @@ inlineToConTeXt (Span (ident,_,kvs) ils) = do
489487 Just lng -> braces ("\\language" <>
490488 brackets (literal lng) <> txt)
491489 Nothing -> txt
492− addReference =
493− if T.null ident
494− then id
495− else (("\\reference" <> brackets (literal ident) <> "{}") <>)
496− addReference . wrapLang . wrapDir <$> inlineListToConTeXt ils
490+ wrapLang . wrapDir <$> inlineListToConTeXt ils
497491
498492-- | Craft the section header, inserting the section reference, if supplied.
499493sectionHeader :: PandocMonad m
@@ -555,26 +549,26 @@ fromBCP47 mbs = fromBCP47' <$> toLang mbs
555549-- https://tools.ietf.org/html/bcp47#section-2.1
556550-- http://wiki.contextgarden.net/Language_Codes
557551fromBCP47' :: Maybe Lang -> Maybe Text
558−fromBCP47' (Just (Lang "ar" _ (Just "SY") _ _ _)) = Just "ar-sy"
559−fromBCP47' (Just (Lang "ar" _ (Just "IQ") _ _ _)) = Just "ar-iq"
560−fromBCP47' (Just (Lang "ar" _ (Just "JO") _ _ _)) = Just "ar-jo"
561−fromBCP47' (Just (Lang "ar" _ (Just "LB") _ _ _)) = Just "ar-lb"
562−fromBCP47' (Just (Lang "ar" _ (Just "DZ") _ _ _)) = Just "ar-dz"
563−fromBCP47' (Just (Lang "ar" _ (Just "MA") _ _ _)) = Just "ar-ma"
564−fromBCP47' (Just (Lang "de" _ _ ["1901"] _ _)) = Just "deo"
565−fromBCP47' (Just (Lang "de" _ (Just "DE") _ _ _)) = Just "de-de"
566−fromBCP47' (Just (Lang "de" _ (Just "AT") _ _ _)) = Just "de-at"
567−fromBCP47' (Just (Lang "de" _ (Just "CH") _ _ _)) = Just "de-ch"
568−fromBCP47' (Just (Lang "el" _ _ ["poly"] _ _)) = Just "agr"
569−fromBCP47' (Just (Lang "en" _ (Just "US") _ _ _)) = Just "en-us"
570−fromBCP47' (Just (Lang "en" _ (Just "GB") _ _ _)) = Just "en-gb"
571−fromBCP47' (Just (Lang "grc"_ _ _ _ _)) = Just "agr"
572−fromBCP47' (Just (Lang "el" _ _ _ _ _)) = Just "gr"
573−fromBCP47' (Just (Lang "eu" _ _ _ _ _)) = Just "ba"
574−fromBCP47' (Just (Lang "he" _ _ _ _ _)) = Just "il"
575−fromBCP47' (Just (Lang "jp" _ _ _ _ _)) = Just "ja"
576−fromBCP47' (Just (Lang "uk" _ _ _ _ _)) = Just "ua"
577−fromBCP47' (Just (Lang "vi" _ _ _ _ _)) = Just "vn"
578−fromBCP47' (Just (Lang "zh" _ _ _ _ _)) = Just "cn"
579−fromBCP47' (Just (Lang l _ _ _ _ _)) = Just l
580−fromBCP47' Nothing = Nothing
552+fromBCP47' (Just (Lang "ar" _ "SY" _) ) = Just "ar-sy"
553+fromBCP47' (Just (Lang "ar" _ "IQ" _) ) = Just "ar-iq"
554+fromBCP47' (Just (Lang "ar" _ "JO" _) ) = Just "ar-jo"
555+fromBCP47' (Just (Lang "ar" _ "LB" _) ) = Just "ar-lb"
556+fromBCP47' (Just (Lang "ar" _ "DZ" _) ) = Just "ar-dz"
557+fromBCP47' (Just (Lang "ar" _ "MA" _) ) = Just "ar-ma"
558+fromBCP47' (Just (Lang "de" _ _ ["1901"]) ) = Just "deo"
559+fromBCP47' (Just (Lang "de" _ "DE" _) ) = Just "de-de"
560+fromBCP47' (Just (Lang "de" _ "AT" _) ) = Just "de-at"
561+fromBCP47' (Just (Lang "de" _ "CH" _) ) = Just "de-ch"
562+fromBCP47' (Just (Lang "el" _ _ ["poly"]) ) = Just "agr"
563+fromBCP47' (Just (Lang "en" _ "US" _) ) = Just "en-us"
564+fromBCP47' (Just (Lang "en" _ "GB" _) ) = Just "en-gb"
565+fromBCP47' (Just (Lang "grc"_ _ _) ) = Just "agr"
566+fromBCP47' (Just (Lang "el" _ _ _) ) = Just "gr"
567+fromBCP47' (Just (Lang "eu" _ _ _) ) = Just "ba"
568+fromBCP47' (Just (Lang "he" _ _ _) ) = Just "il"
569+fromBCP47' (Just (Lang "jp" _ _ _) ) = Just "ja"
570+fromBCP47' (Just (Lang "uk" _ _ _) ) = Just "ua"
571+fromBCP47' (Just (Lang "vi" _ _ _) ) = Just "vn"
572+fromBCP47' (Just (Lang "zh" _ _ _) ) = Just "cn"
573+fromBCP47' (Just (Lang l _ _ _) ) = Just l
574+fromBCP47' Nothing = Nothing
src/Text/Pandoc/Writers/CslJson.hs+3−4
@@ -34,16 +34,15 @@ import Control.Monad.Identity
3434import Citeproc.Locale (getLocale)
3535import Citeproc.CslJson
3636import Text.Pandoc.Options (WriterOptions)
37−import Data.Maybe (mapMaybe, fromMaybe)
37+import Data.Maybe (mapMaybe)
3838import Data.Aeson.Encode.Pretty (Config (..), Indent (Spaces),
3939 NumberFormat (Generic),
4040 defConfig, encodePretty')
4141
4242writeCslJson :: PandocMonad m => WriterOptions -> Pandoc -> m Text
4343writeCslJson _opts (Pandoc meta _) = do
44− let lang = fromMaybe (Lang "en" Nothing (Just "US") [] [] [])
45− (lookupMeta "lang" meta >>= metaValueToText >>=
46− either (const Nothing) Just . parseLang)
44+ let lang = maybe (Lang "en" (Just "US")) parseLang
45+ (lookupMeta "lang" meta >>= metaValueToText)
4746 locale <- case getLocale lang of
4847 Left e -> throwError $ PandocCiteprocError e
4948 Right l -> return l
src/Text/Pandoc/Writers/Docbook.hs+7−50
@@ -168,7 +168,7 @@ blockToDocbook :: PandocMonad m => WriterOptions -> Block -> DB m (Doc Text)
168168blockToDocbook _ Null = return empty
169169-- Add ids to paragraphs in divs with ids - this is needed for
170170-- pandoc-citeproc to get link anchors in bibliographies:
171−blockToDocbook opts (Div (id',"section":_,_) (Header lvl (_,_,attrs) ils : xs)) = do
171+blockToDocbook opts (Div (id',"section":_,_) (Header lvl _ ils : xs)) = do
172172 version <- ask
173173 -- Docbook doesn't allow sections with no content, so insert some if needed
174174 let bs = if null xs
@@ -188,17 +188,15 @@ blockToDocbook opts (Div (id',"section":_,_) (Header lvl (_,_,attrs) ils : xs))
188188 -- standalone documents will include them in the template.
189189 then [("xmlns", "http://docbook.org/ns/docbook"),("xmlns:xlink", "http://www.w3.org/1999/xlink")]
190190 else []
191−
192− -- Populate miscAttr with Header.Attr.attributes, filtering out non-valid DocBook section attributes, id, and xml:id
193− miscAttr = filter (isSectionAttr version) attrs
194− attribs = nsAttr <> idAttr <> miscAttr
191+ attribs = nsAttr <> idAttr
195192 title' <- inlinesToDocbook opts ils
196193 contents <- blocksToDocbook opts bs
197194 return $ inTags True tag attribs $ inTagsSimple "title" title' $$ contents
198195blockToDocbook opts (Div (ident,classes,_) bs) = do
199196 version <- ask
200197 let identAttribs = [(idName version, ident) | not (T.null ident)]
201− admonitions = ["caution","danger","important","note","tip","warning"]
198+ admonitions = ["attention","caution","danger","error","hint",
199+ "important","note","tip","warning"]
202200 case classes of
203201 (l:_) | l `elem` admonitions -> do
204202 let (mTitleBs, bodyBs) =
@@ -252,18 +250,17 @@ blockToDocbook opts (LineBlock lns) =
252250 blockToDocbook opts $ linesToPara lns
253251blockToDocbook opts (BlockQuote blocks) =
254252 inTagsIndented "blockquote" <$> blocksToDocbook opts blocks
255−blockToDocbook opts (CodeBlock (_,classes,_) str) = return $
253+blockToDocbook _ (CodeBlock (_,classes,_) str) = return $
256254 literal ("<programlisting" <> lang <> ">") <> cr <>
257255 flush (literal (escapeStringForXML str) <> cr <> literal "</programlisting>")
258256 where lang = if null langs
259257 then ""
260258 else " language=\"" <> escapeStringForXML (head langs) <>
261259 "\""
262− syntaxMap = writerSyntaxMap opts
263− isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
260+ isLang l = T.toLower l `elem` map T.toLower languages
264261 langsFrom s = if isLang s
265262 then [s]
266− else (languagesByExtension syntaxMap) . T.toLower $ s
263+ else languagesByExtension . T.toLower $ s
267264 langs = concatMap langsFrom classes
268265blockToDocbook opts (BulletList lst) = do
269266 let attribs = [("spacing", "compact") | isTightList lst]
@@ -454,43 +451,3 @@ idAndRole (id',cls,_) = ident <> role
454451 where
455452 ident = [("id", id') | not (T.null id')]
456453 role = [("role", T.unwords cls) | not (null cls)]
457−
458−isSectionAttr :: DocBookVersion -> (Text, Text) -> Bool
459−isSectionAttr _ ("label",_) = True
460−isSectionAttr _ ("status",_) = True
461−isSectionAttr DocBook5 ("annotations",_) = True
462−isSectionAttr DocBook5 ("dir","ltr") = True
463−isSectionAttr DocBook5 ("dir","rtl") = True
464−isSectionAttr DocBook5 ("dir","lro") = True
465−isSectionAttr DocBook5 ("dir","rlo") = True
466−isSectionAttr _ ("remap",_) = True
467−isSectionAttr _ ("revisionflag","changed") = True
468−isSectionAttr _ ("revisionflag","added") = True
469−isSectionAttr _ ("revisionflag","deleted") = True
470−isSectionAttr _ ("revisionflag","off") = True
471−isSectionAttr _ ("role",_) = True
472−isSectionAttr DocBook5 ("version",_) = True
473−isSectionAttr DocBook5 ("xml:base",_) = True
474−isSectionAttr DocBook5 ("xml:lang",_) = True
475−isSectionAttr _ ("xreflabel",_) = True
476−isSectionAttr DocBook5 ("linkend",_) = True
477−isSectionAttr DocBook5 ("linkends",_) = True
478−isSectionAttr DocBook5 ("xlink:actuate",_) = True
479−isSectionAttr DocBook5 ("xlink:arcrole",_) = True
480−isSectionAttr DocBook5 ("xlink:from",_) = True
481−isSectionAttr DocBook5 ("xlink:href",_) = True
482−isSectionAttr DocBook5 ("xlink:label",_) = True
483−isSectionAttr DocBook5 ("xlink:role",_) = True
484−isSectionAttr DocBook5 ("xlink:show",_) = True
485−isSectionAttr DocBook5 ("xlink:title",_) = True
486−isSectionAttr DocBook5 ("xlink:to",_) = True
487−isSectionAttr DocBook5 ("xlink:type",_) = True
488−isSectionAttr DocBook4 ("arch",_) = True
489−isSectionAttr DocBook4 ("condition",_) = True
490−isSectionAttr DocBook4 ("conformance",_) = True
491−isSectionAttr DocBook4 ("lang",_) = True
492−isSectionAttr DocBook4 ("os",_) = True
493−isSectionAttr DocBook4 ("revision",_) = True
494−isSectionAttr DocBook4 ("security",_) = True
495−isSectionAttr DocBook4 ("vendor",_) = True
496−isSectionAttr _ (_,_) = False
src/Text/Pandoc/Writers/Docx.hs+244−60
@@ -22,6 +22,7 @@ import Control.Applicative ((<|>))
2222import Control.Monad.Except (catchError, throwError)
2323import Control.Monad.Reader
2424import Control.Monad.State.Strict
25+import qualified Data.ByteString as B
2526import qualified Data.ByteString.Lazy as BL
2627import Data.Char (isSpace, isLetter)
2728import Data.List (intercalate, isPrefixOf, isSuffixOf)
@@ -35,9 +36,9 @@ import qualified Data.Text.Lazy as TL
3536import Data.Time.Clock.POSIX
3637import Data.Digest.Pure.SHA (sha1, showDigest)
3738import Skylighting
38−import Text.Collate.Lang (renderLang)
39−import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang, translateTerm)
40−import qualified Text.Pandoc.Translations as Term
39+import System.Random (randomRs, mkStdGen)
40+import Text.Pandoc.BCP47 (getLang, renderLang)
41+import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
4142import qualified Text.Pandoc.Class.PandocMonad as P
4243import Data.Time
4344import Text.Pandoc.UTF8 (fromTextLazy)
@@ -47,25 +48,123 @@ import Text.Pandoc.Highlighting (highlight)
4748import Text.Pandoc.Error
4849import Text.Pandoc.ImageSize
4950import Text.Pandoc.Logging
50−import Text.Pandoc.MIME (extensionFromMimeType, getMimeType, getMimeTypeDef)
51+import Text.Pandoc.MIME (MimeType, extensionFromMimeType, getMimeType,
52+ getMimeTypeDef)
5153import Text.Pandoc.Options
5254import Text.Pandoc.Writers.Docx.StyleMap
53−import Text.Pandoc.Writers.Docx.Table
54−import Text.Pandoc.Writers.Docx.Types
5555import Text.Pandoc.Shared
5656import Text.Pandoc.Walk
57−import qualified Text.Pandoc.Writers.GridTable as Grid
5857import Text.Pandoc.Writers.Math
5958import Text.Pandoc.Writers.Shared
59+import Text.Printf (printf)
6060import Text.TeXMath
6161import Text.Pandoc.Writers.OOXML
6262import Text.Pandoc.XML.Light as XML
6363import Data.Generics (mkT, everywhere)
6464
65+data ListMarker = NoMarker
66+ | BulletMarker
67+ | NumberMarker ListNumberStyle ListNumberDelim Int
68+ deriving (Show, Read, Eq, Ord)
69+
70+listMarkerToId :: ListMarker -> Text
71+listMarkerToId NoMarker = "990"
72+listMarkerToId BulletMarker = "991"
73+listMarkerToId (NumberMarker sty delim n) = T.pack $
74+ '9' : '9' : styNum : delimNum : show n
75+ where styNum = case sty of
76+ DefaultStyle -> '2'
77+ Example -> '3'
78+ Decimal -> '4'
79+ LowerRoman -> '5'
80+ UpperRoman -> '6'
81+ LowerAlpha -> '7'
82+ UpperAlpha -> '8'
83+ delimNum = case delim of
84+ DefaultDelim -> '0'
85+ Period -> '1'
86+ OneParen -> '2'
87+ TwoParens -> '3'
88+
89+data EnvProps = EnvProps{ styleElement :: Maybe Element
90+ , otherElements :: [Element]
91+ }
92+
93+instance Semigroup EnvProps where
94+ EnvProps s es <> EnvProps s' es' = EnvProps (s <|> s') (es ++ es')
95+
96+instance Monoid EnvProps where
97+ mempty = EnvProps Nothing []
98+ mappend = (<>)
99+
65100squashProps :: EnvProps -> [Element]
66101squashProps (EnvProps Nothing es) = es
67102squashProps (EnvProps (Just e) es) = e : es
68103
104+data WriterEnv = WriterEnv{ envTextProperties :: EnvProps
105+ , envParaProperties :: EnvProps
106+ , envRTL :: Bool
107+ , envListLevel :: Int
108+ , envListNumId :: Int
109+ , envInDel :: Bool
110+ , envChangesAuthor :: Text
111+ , envChangesDate :: Text
112+ , envPrintWidth :: Integer
113+ }
114+
115+defaultWriterEnv :: WriterEnv
116+defaultWriterEnv = WriterEnv{ envTextProperties = mempty
117+ , envParaProperties = mempty
118+ , envRTL = False
119+ , envListLevel = -1
120+ , envListNumId = 1
121+ , envInDel = False
122+ , envChangesAuthor = "unknown"
123+ , envChangesDate = "1969-12-31T19:00:00Z"
124+ , envPrintWidth = 1
125+ }
126+
127+data WriterState = WriterState{
128+ stFootnotes :: [Element]
129+ , stComments :: [([(Text, Text)], [Inline])]
130+ , stSectionIds :: Set.Set Text
131+ , stExternalLinks :: M.Map Text Text
132+ , stImages :: M.Map FilePath (String, String, Maybe MimeType, B.ByteString)
133+ , stLists :: [ListMarker]
134+ , stInsId :: Int
135+ , stDelId :: Int
136+ , stStyleMaps :: StyleMaps
137+ , stFirstPara :: Bool
138+ , stInTable :: Bool
139+ , stInList :: Bool
140+ , stTocTitle :: [Inline]
141+ , stDynamicParaProps :: Set.Set ParaStyleName
142+ , stDynamicTextProps :: Set.Set CharStyleName
143+ , stCurId :: Int
144+ }
145+
146+defaultWriterState :: WriterState
147+defaultWriterState = WriterState{
148+ stFootnotes = defaultFootnotes
149+ , stComments = []
150+ , stSectionIds = Set.empty
151+ , stExternalLinks = M.empty
152+ , stImages = M.empty
153+ , stLists = [NoMarker]
154+ , stInsId = 1
155+ , stDelId = 1
156+ , stStyleMaps = StyleMaps M.empty M.empty
157+ , stFirstPara = False
158+ , stInTable = False
159+ , stInList = False
160+ , stTocTitle = [Str "Table of Contents"]
161+ , stDynamicParaProps = Set.empty
162+ , stDynamicTextProps = Set.empty
163+ , stCurId = 20
164+ }
165+
166+type WS m = ReaderT WriterEnv (StateT WriterState m)
167+
69168renumIdMap :: Int -> [Element] -> M.Map Text Text
70169renumIdMap _ [] = M.empty
71170renumIdMap n (e:es)
@@ -470,27 +569,12 @@ writeDocx opts doc = do
470569 -- adds references to footnotes or endnotes we don't have...
471570 -- we do, however, copy some settings over from reference
472571 let settingsPath = "word/settings.xml"
473− settingsList = [ "zoom"
474− , "embedSystemFonts"
475− , "doNotTrackMoves"
476− , "defaultTabStop"
477− , "drawingGridHorizontalSpacing"
478− , "drawingGridVerticalSpacing"
479− , "displayHorizontalDrawingGridEvery"
480− , "displayVerticalDrawingGridEvery"
481− , "characterSpacingControl"
482− , "savePreviewPicture"
483− , "mathPr"
484− , "themeFontLang"
485− , "decimalSymbol"
486− , "listSeparator"
487− , "autoHyphenation"
488− , "consecutiveHyphenLimit"
489− , "hyphenationZone"
490− , "doNotHyphenateCap"
491− , "evenAndOddHeaders"
492− , "proofState"
493− , "compat"
572+ settingsList = [ "w:autoHyphenation"
573+ , "w:consecutiveHyphenLimit"
574+ , "w:hyphenationZone"
575+ , "w:doNotHyphenateCap"
576+ , "w:evenAndOddHeaders"
577+ , "w:proofState"
494578 ]
495579 settingsEntry <- copyChildren refArchive distArchive settingsPath epochtime settingsList
496580
@@ -593,17 +677,16 @@ copyChildren :: (PandocMonad m)
593677copyChildren refArchive distArchive path timestamp elNames = do
594678 ref <- parseXml refArchive distArchive path
595679 dist <- parseXml distArchive distArchive path
596− let elsToCopy =
597− map cleanElem $ filterChildrenName (\e -> qName e `elem` elNames) ref
598− let elsToKeep =
599− [e | Elem e <- elContent dist, not (any (hasSameNameAs e) elsToCopy)]
600680 return $ toEntry path timestamp $ renderXml dist{
601− elContent = map Elem elsToKeep ++ map Elem elsToCopy
681+ elContent = elContent dist ++ copyContent ref
602682 }
603683 where
604− hasSameNameAs (Element {elName = n1}) (Element {elName = n2}) =
605− qName n1 == qName n2
606− cleanElem el@Element{elName=name} = el{elName=name{qURI=Nothing}}
684+ strName QName{qName=name, qPrefix=prefix}
685+ | Just p <- prefix = p <> ":" <> name
686+ | otherwise = name
687+ shouldCopy = (`elem` elNames) . strName
688+ cleanElem el@Element{elName=name} = Elem el{elName=name{qURI=Nothing}}
689+ copyContent = map cleanElem . filterChildrenName shouldCopy
607690
608691-- this is the lowest number used for a list numId
609692baseListId :: Int
@@ -612,7 +695,8 @@ baseListId = 1000
612695mkNumbering :: [ListMarker] -> [Element]
613696mkNumbering lists =
614697 elts ++ zipWith mkNum lists [baseListId..(baseListId + length lists - 1)]
615− where elts = map mkAbstractNum (ordNub lists)
698+ where elts = zipWith mkAbstractNum (ordNub lists) $
699+ randomRs (0x10000000, 0xFFFFFFFF) $ mkStdGen 1848
616700
617701maxListLevel :: Int
618702maxListLevel = 8
@@ -629,10 +713,10 @@ mkNum marker numid =
629713 $ mknode "w:startOverride" [("w:val",tshow start)] ())
630714 [0..maxListLevel]
631715
632−mkAbstractNum :: ListMarker -> Element
633−mkAbstractNum marker =
716+mkAbstractNum :: ListMarker -> Integer -> Element
717+mkAbstractNum marker nsid =
634718 mknode "w:abstractNum" [("w:abstractNumId",listMarkerToId marker)]
635− $ mknode "w:nsid" [("w:val", "A" <> listMarkerToId marker)] ()
719+ $ mknode "w:nsid" [("w:val", T.pack $ printf "%8x" nsid)] ()
636720 : mknode "w:multiLevelType" [("w:val","multilevel")] ()
637721 : map (mkLvl marker)
638722 [0..maxListLevel]
@@ -776,6 +860,12 @@ separateTables (x@Table{}:xs@(Table{}:_)) =
776860 x : RawBlock (Format "openxml") "<w:p />" : separateTables xs
777861separateTables (x:xs) = x : separateTables xs
778862
863+pStyleM :: (PandocMonad m) => ParaStyleName -> WS m XML.Element
864+pStyleM styleName = do
865+ pStyleMap <- gets (smParaStyle . stStyleMaps)
866+ let sty' = getStyleIdFromName styleName pStyleMap
867+ return $ mknode "w:pStyle" [("w:val", fromStyleId sty')] ()
868+
779869rStyleM :: (PandocMonad m) => CharStyleName -> WS m XML.Element
780870rStyleM styleName = do
781871 cStyleMap <- gets (smCharStyle . stStyleMaps)
@@ -855,28 +945,14 @@ blockToOpenXML' opts (Plain lst) = do
855945-- title beginning with fig: indicates that the image is a figure
856946blockToOpenXML' opts (Para [Image attr alt (src,T.stripPrefix "fig:" -> Just tit)]) = do
857947 setFirstPara
858− fignum <- gets stNextFigureNum
859− unless (null alt) $ modify $ \st -> st{ stNextFigureNum = fignum + 1 }
860− let figid = "fig" <> tshow fignum
861− figname <- translateTerm Term.Figure
862948 prop <- pStyleM $
863949 if null alt
864950 then "Figure"
865951 else "Captioned Figure"
866952 paraProps <- local (\env -> env { envParaProperties = EnvProps (Just prop) [] <> envParaProperties env }) (getParaProps False)
867953 contents <- inlinesToOpenXML opts [Image attr alt (src,tit)]
868− captionNode <- if null alt
869− then return []
870− else withParaPropM (pStyleM "Image Caption")
871− $ blockToOpenXML opts
872− (Para $ Span (figid,[],[])
873− [Str (figname <> "\160"),
874− RawInline (Format "openxml")
875− ("<w:fldSimple w:instr=\"SEQ Figure"
876− <> " \\* ARABIC \"><w:r><w:t>"
877− <> tshow fignum
878− <> "</w:t></w:r></w:fldSimple>"),
879− Str ":", Space] : alt)
954+ captionNode <- withParaPropM (pStyleM "Image Caption")
955+ $ blockToOpenXML opts (Para alt)
880956 return $
881957 Elem (mknode "w:p" [] (map Elem paraProps ++ contents))
882958 : captionNode
@@ -921,9 +997,78 @@ blockToOpenXML' _ HorizontalRule = do
921997 $ mknode "v:rect" [("style","width:0;height:1.5pt"),
922998 ("o:hralign","center"),
923999 ("o:hrstd","t"),("o:hr","t")] () ]
924−blockToOpenXML' opts (Table attr caption colspecs thead tbodies tfoot) =
925− tableToOpenXML (blocksToOpenXML opts)
926− (Grid.toTable attr caption colspecs thead tbodies tfoot)
1000+blockToOpenXML' opts (Table _ blkCapt specs thead tbody tfoot) = do
1001+ let (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
1002+ setFirstPara
1003+ modify $ \s -> s { stInTable = True }
1004+ let captionStr = stringify caption
1005+ caption' <- if null caption
1006+ then return []
1007+ else withParaPropM (pStyleM "Table Caption")
1008+ $ blockToOpenXML opts (Para caption)
1009+ let alignmentFor al = mknode "w:jc" [("w:val",alignmentToString al)] ()
1010+ -- Table cells require a <w:p> element, even an empty one!
1011+ -- Not in the spec but in Word 2007, 2010. See #4953. And
1012+ -- apparently the last element must be a <w:p>, see #6983.
1013+ let cellToOpenXML (al, cell) = do
1014+ es <- withParaProp (alignmentFor al) $ blocksToOpenXML opts cell
1015+ return $
1016+ case reverse (onlyElems es) of
1017+ b:e:_ | qName (elName b) == "bookmarkEnd"
1018+ , qName (elName e) == "p" -> es
1019+ e:_ | qName (elName e) == "p" -> es
1020+ _ -> es ++ [Elem $ mknode "w:p" [] ()]
1021+ headers' <- mapM cellToOpenXML $ zip aligns headers
1022+ rows' <- mapM (mapM cellToOpenXML . zip aligns) rows
1023+ compactStyle <- pStyleM "Compact"
1024+ let emptyCell' = [Elem $ mknode "w:p" [] [mknode "w:pPr" [] [compactStyle]]]
1025+ let mkcell contents = mknode "w:tc" []
1026+ $ if null contents
1027+ then emptyCell'
1028+ else contents
1029+ let mkrow cells =
1030+ mknode "w:tr" [] $
1031+ map mkcell cells
1032+ let textwidth = 7920 -- 5.5 in in twips, 1/20 pt
1033+ let fullrow = 5000 -- 100% specified in pct
1034+ let rowwidth = fullrow * sum widths
1035+ let mkgridcol w = mknode "w:gridCol"
1036+ [("w:w", tshow (floor (textwidth * w) :: Integer))] ()
1037+ let hasHeader = not $ all null headers
1038+ modify $ \s -> s { stInTable = False }
1039+ -- for compatibility with Word <= 2007, we include a val with a bitmask
1040+ -- 0×0020 Apply first row conditional formatting
1041+ -- 0×0040 Apply last row conditional formatting
1042+ -- 0×0080 Apply first column conditional formatting
1043+ -- 0×0100 Apply last column conditional formatting
1044+ -- 0×0200 Do not apply row banding conditional formatting
1045+ -- 0×0400 Do not apply column banding conditional formattin
1046+ let tblLookVal :: Int
1047+ tblLookVal = if hasHeader then 0x20 else 0
1048+ return $
1049+ caption' ++
1050+ [Elem $
1051+ mknode "w:tbl" []
1052+ ( mknode "w:tblPr" []
1053+ ( mknode "w:tblStyle" [("w:val","Table")] () :
1054+ mknode "w:tblW" [("w:type", "pct"), ("w:w", tshow rowwidth)] () :
1055+ mknode "w:tblLook" [("w:firstRow",if hasHeader then "1" else "0")
1056+ ,("w:lastRow","0")
1057+ ,("w:firstColumn","0")
1058+ ,("w:lastColumn","0")
1059+ ,("w:noHBand","0")
1060+ ,("w:noVBand","0")
1061+ ,("w:val", T.pack $ printf "%04x" tblLookVal)
1062+ ] () :
1063+ [ mknode "w:tblCaption" [("w:val", captionStr)] ()
1064+ | not (null caption) ] )
1065+ : mknode "w:tblGrid" []
1066+ (if all (==0) widths
1067+ then []
1068+ else map mkgridcol widths)
1069+ : [ mkrow headers' | hasHeader ] ++
1070+ map mkrow rows'
1071+ )]
9271072blockToOpenXML' opts el
9281073 | BulletList lst <- el = addOpenXMLList BulletMarker lst
9291074 | OrderedList (start, numstyle, numdelim) lst <- el
@@ -978,6 +1123,13 @@ listItemToOpenXML opts numid (first:rest) = do
9781123 modify $ \st -> st{ stInList = oldInList }
9791124 return $ first'' ++ rest''
9801125
1126+alignmentToString :: Alignment -> Text
1127+alignmentToString alignment = case alignment of
1128+ AlignLeft -> "left"
1129+ AlignRight -> "right"
1130+ AlignCenter -> "center"
1131+ AlignDefault -> "left"
1132+
9811133-- | Convert a list of inline elements to OpenXML.
9821134inlinesToOpenXML :: PandocMonad m => WriterOptions -> [Inline] -> WS m [Content]
9831135inlinesToOpenXML opts lst = concat `fmap` mapM (inlineToOpenXML opts) lst
@@ -988,6 +1140,10 @@ withNumId numid = local $ \env -> env{ envListNumId = numid }
9881140asList :: (PandocMonad m) => WS m a -> WS m a
9891141asList = local $ \env -> env{ envListLevel = envListLevel env + 1 }
9901142
1143+isStyle :: Element -> Bool
1144+isStyle e = isElem [] "w" "rStyle" e ||
1145+ isElem [] "w" "pStyle" e
1146+
9911147getTextProps :: (PandocMonad m) => WS m [Element]
9921148getTextProps = do
9931149 props <- asks envTextProperties
@@ -1016,6 +1172,16 @@ getParaProps displayMathPara = do
10161172 [] -> []
10171173 ps -> [mknode "w:pPr" [] ps]
10181174
1175+withParaProp :: PandocMonad m => Element -> WS m a -> WS m a
1176+withParaProp d p =
1177+ local (\env -> env {envParaProperties = ep <> envParaProperties env}) p
1178+ where ep = if isStyle d then EnvProps (Just d) [] else EnvProps Nothing [d]
1179+
1180+withParaPropM :: PandocMonad m => WS m Element -> WS m a -> WS m a
1181+withParaPropM md p = do
1182+ d <- md
1183+ withParaProp d p
1184+
10191185formattedString :: PandocMonad m => Text -> WS m [Element]
10201186formattedString str =
10211187 -- properly handle soft hyphens
@@ -1036,6 +1202,9 @@ formattedRun els = do
10361202 props <- getTextProps
10371203 return [ mknode "w:r" [] $ props ++ els ]
10381204
1205+setFirstPara :: PandocMonad m => WS m ()
1206+setFirstPara = modify $ \s -> s { stFirstPara = True }
1207+
10391208-- | Convert an inline element to OpenXML.
10401209inlineToOpenXML :: PandocMonad m => WriterOptions -> Inline -> WS m [Content]
10411210inlineToOpenXML opts il = withDirection $ inlineToOpenXML' opts il
@@ -1304,7 +1473,6 @@ inlineToOpenXML' opts (Image attr@(imgident, _, _) alt (src, title)) = do
13041473 Just Eps -> ".eps"
13051474 Just Svg -> ".svg"
13061475 Just Emf -> ".emf"
1307− Just Tiff -> ".tiff"
13081476 Nothing -> ""
13091477 imgpath = "media/" <> ident <> imgext
13101478 mbMimeType = mt <|> getMimeType (T.unpack imgpath)
@@ -1328,6 +1496,22 @@ inlineToOpenXML' opts (Image attr@(imgident, _, _) alt (src, title)) = do
13281496br :: Element
13291497br = mknode "w:r" [] [mknode "w:br" [] ()]
13301498
1499+-- Word will insert these footnotes into the settings.xml file
1500+-- (whether or not they're visible in the document). If they're in the
1501+-- file, but not in the footnotes.xml file, it will produce
1502+-- problems. So we want to make sure we insert them into our document.
1503+defaultFootnotes :: [Element]
1504+defaultFootnotes = [ mknode "w:footnote"
1505+ [("w:type", "separator"), ("w:id", "-1")]
1506+ [ mknode "w:p" []
1507+ [mknode "w:r" []
1508+ [ mknode "w:separator" [] ()]]]
1509+ , mknode "w:footnote"
1510+ [("w:type", "continuationSeparator"), ("w:id", "0")]
1511+ [ mknode "w:p" []
1512+ [ mknode "w:r" []
1513+ [ mknode "w:continuationSeparator" [] ()]]]]
1514+
13311515
13321516withDirection :: PandocMonad m => WS m a -> WS m a
13331517withDirection x = do
src/Text/Pandoc/Writers/Docx/Table.hsdeleted−227
@@ -1,227 +0,0 @@
1−{-# LANGUAGE LambdaCase #-}
2−{-# LANGUAGE OverloadedStrings #-}
3−{-# LANGUAGE ScopedTypeVariables #-}
4−{- |
5−Module : Text.Pandoc.Writers.Docx.Table
6−Copyright : Copyright (C) 2012-2021 John MacFarlane
7−License : GNU GPL, version 2 or above
8−Maintainer : John MacFarlane <jgm@berkeley.edu>
9−
10−Conversion of table blocks to docx.
11−-}
12−module Text.Pandoc.Writers.Docx.Table
13− ( tableToOpenXML
14− ) where
15−
16−import Control.Monad.State.Strict
17−import Data.Array
18−import Data.Text (Text)
19−import Text.Pandoc.Definition
20−import Text.Pandoc.Class.PandocMonad (PandocMonad, translateTerm)
21−import Text.Pandoc.Writers.Docx.Types
22−import Text.Pandoc.Shared
23−import Text.Printf (printf)
24−import Text.Pandoc.Writers.GridTable hiding (Table)
25−import Text.Pandoc.Writers.OOXML
26−import Text.Pandoc.XML.Light as XML hiding (Attr)
27−import qualified Data.Text as T
28−import qualified Text.Pandoc.Translations as Term
29−import qualified Text.Pandoc.Writers.GridTable as Grid
30−
31−tableToOpenXML :: PandocMonad m
32− => ([Block] -> WS m [Content])
33− -> Grid.Table
34− -> WS m [Content]
35−tableToOpenXML blocksToOpenXML gridTable = do
36− setFirstPara
37− let (Grid.Table (ident,_,_) caption colspecs _rowheads thead tbodies tfoot) =
38− gridTable
39− let (Caption _maybeShortCaption captionBlocks) = caption
40− tablenum <- gets stNextTableNum
41− unless (null captionBlocks) $
42− modify $ \st -> st{ stNextTableNum = tablenum + 1 }
43− let tableid = if T.null ident
44− then "table" <> tshow tablenum
45− else ident
46− tablename <- translateTerm Term.Table
47− let captionStr = stringify captionBlocks
48− let aligns = map fst $ elems colspecs
49− captionXml <- if null captionBlocks
50− then return []
51− else withParaPropM (pStyleM "Table Caption")
52− $ blocksToOpenXML
53− $ addLabel tableid tablename tablenum captionBlocks
54− -- We set "in table" after processing the caption, because we don't
55− -- want the "Table Caption" style to be overwritten with "Compact".
56− modify $ \s -> s { stInTable = True }
57− head' <- cellGridToOpenXML blocksToOpenXML HeadRow aligns thead
58− bodies <- mapM (cellGridToOpenXML blocksToOpenXML BodyRow aligns) tbodies
59− foot' <- cellGridToOpenXML blocksToOpenXML FootRow aligns tfoot
60−
61− let hasHeader = not . null . indices . partRowAttrs $ thead
62− let hasFooter = not . null . indices . partRowAttrs $ tfoot
63− -- for compatibility with Word <= 2007, we include a val with a bitmask
64− -- 0×0020 Apply first row conditional formatting
65− -- 0×0040 Apply last row conditional formatting
66− -- 0×0080 Apply first column conditional formatting
67− -- 0×0100 Apply last column conditional formatting
68− -- 0×0200 Do not apply row banding conditional formatting
69− -- 0×0400 Do not apply column banding conditional formattin
70− let tblLookVal = if hasHeader then (0x20 :: Int) else 0
71− let (gridCols, tblWattr) = tableLayout (elems colspecs)
72− let tbl = mknode "w:tbl" []
73− ( mknode "w:tblPr" []
74− ( mknode "w:tblStyle" [("w:val","Table")] () :
75− mknode "w:tblW" tblWattr () :
76− mknode "w:tblLook" [("w:firstRow",if hasHeader then "1" else "0")
77− ,("w:lastRow",if hasFooter then "1" else "0")
78− ,("w:firstColumn","0")
79− ,("w:lastColumn","0")
80− ,("w:noHBand","0")
81− ,("w:noVBand","0")
82− ,("w:val", T.pack $ printf "%04x" tblLookVal)
83− ] () :
84− [ mknode "w:tblCaption" [("w:val", captionStr)] ()
85− | not (T.null captionStr) ]
86− )
87− : mknode "w:tblGrid" [] gridCols
88− : head' ++ mconcat bodies ++ foot'
89− )
90− modify $ \s -> s { stInTable = False }
91− return $ captionXml ++ [Elem tbl]
92−
93−addLabel :: Text -> Text -> Int -> [Block] -> [Block]
94−addLabel tableid tablename tablenum bs =
95− case bs of
96− (Para ils : rest) -> Para (label : Space : ils) : rest
97− (Plain ils : rest) -> Plain (label : Space : ils) : rest
98− _ -> Para [label] : bs
99− where
100− label = Span (tableid,[],[])
101− [Str (tablename <> "\160"),
102− RawInline (Format "openxml")
103− ("<w:fldSimple w:instr=\"SEQ Table"
104− <> " \\* ARABIC \"><w:r><w:t>"
105− <> tshow tablenum
106− <> "</w:t></w:r></w:fldSimple>"),
107− Str ":"]
108−
109−-- | Parts of a table
110−data RowType = HeadRow | BodyRow | FootRow
111−
112−alignmentToString :: Alignment -> Text
113−alignmentToString = \case
114− AlignLeft -> "left"
115− AlignRight -> "right"
116− AlignCenter -> "center"
117− AlignDefault -> "left"
118−
119−tableLayout :: [ColSpec] -> ([Element], [(Text, Text)])
120−tableLayout specs =
121− let
122− textwidth = 7920 -- 5.5 in in twips (1 twip == 1/20 pt)
123− fullrow = 5000 -- 100% specified in pct (1 pct == 1/50th of a percent)
124− ncols = length specs
125− getWidth = \case
126− ColWidth n -> n
127− _ -> 0
128− widths = map (getWidth . snd) specs
129− rowwidth = round (fullrow * sum widths) :: Int
130− widthToTwips w = floor (textwidth * w) :: Int
131− mkGridCol w = mknode "w:gridCol" [("w:w", tshow (widthToTwips w))] ()
132− in if all (== 0) widths
133− then ( replicate ncols $ mkGridCol (1.0 / fromIntegral ncols)
134− , [ ("w:type", "auto"), ("w:w", "0")])
135− else ( map mkGridCol widths
136− , [ ("w:type", "pct"), ("w:w", tshow rowwidth) ])
137−
138−cellGridToOpenXML :: PandocMonad m
139− => ([Block] -> WS m [Content])
140− -> RowType
141− -> [Alignment]
142− -> Part
143− -> WS m [Element]
144−cellGridToOpenXML blocksToOpenXML rowType aligns part@(Part _ cellArray _) =
145− if null (elems cellArray)
146− then return mempty
147− else mapM (rowToOpenXML blocksToOpenXML) $
148− partToRows rowType aligns part
149−
150−data OOXMLCell
151− = OOXMLCell Attr Alignment RowSpan ColSpan [Block]
152− | OOXMLCellMerge ColSpan
153−
154−data OOXMLRow = OOXMLRow RowType Attr [OOXMLCell]
155−
156−partToRows :: RowType -> [Alignment] -> Part -> [OOXMLRow]
157−partToRows rowType aligns part =
158− let
159− toOOXMLCell :: Alignment -> RowIndex -> ColIndex -> GridCell -> [OOXMLCell]
160− toOOXMLCell columnAlign ridx cidx = \case
161− ContentCell attr align rowspan colspan blocks ->
162− -- Respect non-default, cell specific alignment.
163− let align' = case align of
164− AlignDefault -> columnAlign
165− _ -> align
166− in [OOXMLCell attr align' rowspan colspan blocks]
167− ContinuationCell idx'@(ridx',cidx') | ridx /= ridx', cidx == cidx' ->
168− case (partCellArray part)!idx' of
169− (ContentCell _ _ _ colspan _) -> [OOXMLCellMerge colspan]
170− x -> error $ "Content cell expected, got, " ++ show x ++
171− " at index " ++ show idx'
172− _ -> mempty
173− mkRow :: (RowIndex, Attr) -> OOXMLRow
174− mkRow (ridx, attr) = OOXMLRow rowType attr
175− . mconcat
176− . zipWith (\align -> uncurry $ toOOXMLCell align ridx)
177− aligns
178− . assocs
179− . rowArray ridx
180− $ partCellArray part
181− in map mkRow $ assocs (partRowAttrs part)
182−
183−rowToOpenXML :: PandocMonad m
184− => ([Block] -> WS m [Content])
185− -> OOXMLRow
186− -> WS m Element
187−rowToOpenXML blocksToOpenXML (OOXMLRow rowType _attr cells) = do
188− xmlcells <- mapM (ooxmlCellToOpenXML blocksToOpenXML) cells
189− let addTrPr = case rowType of
190− HeadRow -> (mknode "w:trPr" []
191− [mknode "w:tblHeader" [("w:val", "true")] ()] :)
192− BodyRow -> id
193− FootRow -> id
194− return $ mknode "w:tr" [] (addTrPr xmlcells)
195−
196−ooxmlCellToOpenXML :: PandocMonad m
197− => ([Block] -> WS m [Content])
198− -> OOXMLCell
199− -> WS m Element
200−ooxmlCellToOpenXML blocksToOpenXML = \case
201− OOXMLCellMerge (ColSpan colspan) -> do
202− return $ mknode "w:tc" []
203− [ mknode "w:tcPr" [] [ mknode "w:gridSpan" [("w:val", tshow colspan)] ()
204− , mknode "w:vMerge" [("w:val", "continue")] () ]
205− , mknode "w:p" [] [mknode "w:pPr" [] ()]]
206− OOXMLCell _attr align rowspan (ColSpan colspan) contents -> do
207− compactStyle <- pStyleM "Compact"
208− es <- withParaProp (alignmentFor align) $ blocksToOpenXML contents
209− -- Table cells require a <w:p> element, even an empty one!
210− -- Not in the spec but in Word 2007, 2010. See #4953. And
211− -- apparently the last element must be a <w:p>, see #6983.
212− return . mknode "w:tc" [] $
213− Elem
214− (mknode "w:tcPr" [] ([ mknode "w:gridSpan" [("w:val", tshow colspan)] ()
215− | colspan > 1] ++
216− [ mknode "w:vMerge" [("w:val", "restart")] ()
217− | rowspan > RowSpan 1 ])) :
218− if null contents
219− then [Elem $ mknode "w:p" [] [mknode "w:pPr" [] [compactStyle]]]
220− else case reverse (onlyElems es) of
221− b:e:_ | qName (elName b) == "bookmarkEnd" -- y tho?
222− , qName (elName e) == "p" -> es
223− e:_ | qName (elName e) == "p" -> es
224− _ -> es ++ [Elem $ mknode "w:p" [] ()]
225−
226−alignmentFor :: Alignment -> Element
227−alignmentFor al = mknode "w:jc" [("w:val",alignmentToString al)] ()
src/Text/Pandoc/Writers/Docx/Types.hsdeleted−185
@@ -1,185 +0,0 @@
1−{-# LANGUAGE OverloadedStrings #-}
2−{- |
3−Module : Text.Pandoc.Writers.Docx
4−Copyright : Copyright (C) 2012-2021 John MacFarlane
5−License : GNU GPL, version 2 or above
6−Maintainer : John MacFarlane <jgm@berkeley.edu>
7−
8−Conversion of table blocks to docx.
9−-}
10−module Text.Pandoc.Writers.Docx.Types
11− ( EnvProps (..)
12− , WriterEnv (..)
13− , defaultWriterEnv
14− , WriterState (..)
15− , defaultWriterState
16− , WS
17− , ListMarker (..)
18− , listMarkerToId
19− , pStyleM
20− , isStyle
21− , setFirstPara
22− , withParaProp
23− , withParaPropM
24− ) where
25−
26−import Control.Applicative ((<|>))
27−import Control.Monad.Reader
28−import Control.Monad.State.Strict
29−import Data.Text (Text)
30−import Text.Pandoc.Class.PandocMonad (PandocMonad)
31−import Text.Pandoc.Definition
32−import Text.Pandoc.MIME (MimeType)
33−import Text.Pandoc.Writers.Docx.StyleMap
34−import Text.Pandoc.Writers.OOXML
35−import Text.Pandoc.XML.Light as XML
36−import qualified Data.ByteString as B
37−import qualified Data.Map as M
38−import qualified Data.Set as Set
39−import qualified Data.Text as T
40−
41−data ListMarker = NoMarker
42− | BulletMarker
43− | NumberMarker ListNumberStyle ListNumberDelim Int
44− deriving (Show, Read, Eq, Ord)
45−
46−listMarkerToId :: ListMarker -> Text
47−listMarkerToId NoMarker = "990"
48−listMarkerToId BulletMarker = "991"
49−listMarkerToId (NumberMarker sty delim n) = T.pack $
50− '9' : '9' : styNum : delimNum : show n
51− where styNum = case sty of
52− DefaultStyle -> '2'
53− Example -> '3'
54− Decimal -> '4'
55− LowerRoman -> '5'
56− UpperRoman -> '6'
57− LowerAlpha -> '7'
58− UpperAlpha -> '8'
59− delimNum = case delim of
60− DefaultDelim -> '0'
61− Period -> '1'
62− OneParen -> '2'
63− TwoParens -> '3'
64−
65−
66−data EnvProps = EnvProps{ styleElement :: Maybe Element
67− , otherElements :: [Element]
68− }
69−
70−instance Semigroup EnvProps where
71− EnvProps s es <> EnvProps s' es' = EnvProps (s <|> s') (es ++ es')
72−
73−instance Monoid EnvProps where
74− mempty = EnvProps Nothing []
75− mappend = (<>)
76−
77−data WriterEnv = WriterEnv
78− { envTextProperties :: EnvProps
79− , envParaProperties :: EnvProps
80− , envRTL :: Bool
81− , envListLevel :: Int
82− , envListNumId :: Int
83− , envInDel :: Bool
84− , envChangesAuthor :: Text
85− , envChangesDate :: Text
86− , envPrintWidth :: Integer
87− }
88−
89−defaultWriterEnv :: WriterEnv
90−defaultWriterEnv = WriterEnv
91− { envTextProperties = mempty
92− , envParaProperties = mempty
93− , envRTL = False
94− , envListLevel = -1
95− , envListNumId = 1
96− , envInDel = False
97− , envChangesAuthor = "unknown"
98− , envChangesDate = "1969-12-31T19:00:00Z"
99− , envPrintWidth = 1
100− }
101−
102−
103−data WriterState = WriterState{
104− stFootnotes :: [Element]
105− , stComments :: [([(Text, Text)], [Inline])]
106− , stSectionIds :: Set.Set Text
107− , stExternalLinks :: M.Map Text Text
108− , stImages :: M.Map FilePath (String, String, Maybe MimeType, B.ByteString)
109− , stLists :: [ListMarker]
110− , stInsId :: Int
111− , stDelId :: Int
112− , stStyleMaps :: StyleMaps
113− , stFirstPara :: Bool
114− , stInTable :: Bool
115− , stInList :: Bool
116− , stTocTitle :: [Inline]
117− , stDynamicParaProps :: Set.Set ParaStyleName
118− , stDynamicTextProps :: Set.Set CharStyleName
119− , stCurId :: Int
120− , stNextFigureNum :: Int
121− , stNextTableNum :: Int
122− }
123−
124−defaultWriterState :: WriterState
125−defaultWriterState = WriterState{
126− stFootnotes = defaultFootnotes
127− , stComments = []
128− , stSectionIds = Set.empty
129− , stExternalLinks = M.empty
130− , stImages = M.empty
131− , stLists = [NoMarker]
132− , stInsId = 1
133− , stDelId = 1
134− , stStyleMaps = StyleMaps M.empty M.empty
135− , stFirstPara = False
136− , stInTable = False
137− , stInList = False
138− , stTocTitle = [Str "Table of Contents"]
139− , stDynamicParaProps = Set.empty
140− , stDynamicTextProps = Set.empty
141− , stCurId = 20
142− , stNextFigureNum = 1
143− , stNextTableNum = 1
144− }
145−
146−setFirstPara :: PandocMonad m => WS m ()
147−setFirstPara = modify $ \s -> s { stFirstPara = True }
148−
149−type WS m = ReaderT WriterEnv (StateT WriterState m)
150−
151−-- Word will insert these footnotes into the settings.xml file
152−-- (whether or not they're visible in the document). If they're in the
153−-- file, but not in the footnotes.xml file, it will produce
154−-- problems. So we want to make sure we insert them into our document.
155−defaultFootnotes :: [Element]
156−defaultFootnotes = [ mknode "w:footnote"
157− [("w:type", "separator"), ("w:id", "-1")]
158− [ mknode "w:p" []
159− [mknode "w:r" []
160− [ mknode "w:separator" [] ()]]]
161− , mknode "w:footnote"
162− [("w:type", "continuationSeparator"), ("w:id", "0")]
163− [ mknode "w:p" []
164− [ mknode "w:r" []
165− [ mknode "w:continuationSeparator" [] ()]]]]
166−
167−pStyleM :: (PandocMonad m) => ParaStyleName -> WS m XML.Element
168−pStyleM styleName = do
169− pStyleMap <- gets (smParaStyle . stStyleMaps)
170− let sty' = getStyleIdFromName styleName pStyleMap
171− return $ mknode "w:pStyle" [("w:val", fromStyleId sty')] ()
172−
173−withParaProp :: PandocMonad m => Element -> WS m a -> WS m a
174−withParaProp d p =
175− local (\env -> env {envParaProperties = ep <> envParaProperties env}) p
176− where ep = if isStyle d then EnvProps (Just d) [] else EnvProps Nothing [d]
177−
178−withParaPropM :: PandocMonad m => WS m Element -> WS m a -> WS m a
179−withParaPropM md p = do
180− d <- md
181− withParaProp d p
182−
183−isStyle :: Element -> Bool
184−isStyle e = isElem [] "w" "rStyle" e ||
185− isElem [] "w" "pStyle" e
src/Text/Pandoc/Writers/DokuWiki.hs+1−3
@@ -27,7 +27,6 @@ import Control.Monad.Reader (ReaderT, asks, local, runReaderT)
2727import Control.Monad.State.Strict (StateT, evalStateT)
2828import Data.Default (Default (..))
2929import Data.List (transpose)
30−import Data.List.NonEmpty (nonEmpty)
3130import Data.Text (Text)
3231import qualified Data.Text as T
3332import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
@@ -173,8 +172,7 @@ blockToDokuWiki opts (Table _ blkCapt specs thead tbody tfoot) = do
173172 then return []
174173 else zipWithM (tableItemToDokuWiki opts) aligns headers
175174 rows' <- mapM (zipWithM (tableItemToDokuWiki opts) aligns) rows
176− let widths = map (maybe 0 maximum . nonEmpty . map T.length)
177− $ transpose (headers':rows')
175+ let widths = map (maximum . map T.length) $ transpose (headers':rows')
178176 let padTo (width, al) s =
179177 case width - T.length s of
180178 x | x > 0 ->
src/Text/Pandoc/Writers/EPUB.hs+3−3
@@ -991,12 +991,12 @@ metadataElement version md currentTime =
991991 showDateTimeISO8601 currentTime | version == EPUB3 ]
992992 belongsToCollectionNodes =
993993 maybe []
994− (\belongsToCollection -> (unode "meta" ! [("property", "belongs-to-collection"), ("id", "epub-collection-1")] $ belongsToCollection )
994+ (\belongsToCollection -> (unode "meta" ! [("property", "belongs-to-collection"), ("id", "epub-id-1")] $ belongsToCollection )
995995 :
996− [unode "meta" ! [("refines", "#epub-collection-1"), ("property", "collection-type")] $ ("series" :: Text) ])
996+ [unode "meta" ! [("refines", "#epub-id-1"), ("property", "collection-type")] $ ("series" :: Text) ])
997997 (epubBelongsToCollection md)++
998998 maybe []
999− (\groupPosition -> [unode "meta" ! [("refines", "#epub-collection-1"), ("property", "group-position")] $ groupPosition ])
999+ (\groupPosition -> [unode "meta" ! [("refines", "#epub-id-1"), ("property", "group-position")] $ groupPosition ])
10001000 (epubGroupPosition md)
10011001 dcTag n s = unode ("dc:" <> n) s
10021002 dcTag' n s = [dcTag n s]
src/Text/Pandoc/Writers/GridTable.hsdeleted−157
@@ -1,157 +0,0 @@
1−{-# LANGUAGE FlexibleContexts #-}
2−{-# LANGUAGE GeneralizedNewtypeDeriving #-}
3−{-# LANGUAGE LambdaCase #-}
4−{-# LANGUAGE TupleSections #-}
5−
6−{- |
7−Module : Text.Pandoc.Writers.GridTable
8−Copyright : © 2020-2021 Albert Krewinkel
9−License : GNU GPL, version 2 or above
10−
11−Maintainer : Albert Krewinkel <albert@zeitkraut.de>
12−
13−Grid representation of pandoc tables.
14−-}
15−module Text.Pandoc.Writers.GridTable
16− ( Table (..)
17− , GridCell (..)
18− , RowIndex (..)
19− , ColIndex (..)
20− , CellIndex
21− , Part (..)
22− , toTable
23− , rowArray
24− ) where
25−
26−import Control.Monad (forM_)
27−import Control.Monad.ST
28−import Data.Array
29−import Data.Array.MArray
30−import Data.Array.ST
31−import Data.Maybe (listToMaybe)
32−import Data.STRef
33−import Text.Pandoc.Definition hiding (Table)
34−import qualified Text.Pandoc.Builder as B
35−
36−-- | A grid cell contains either a real table cell, or is the
37−-- continuation of a column or row-spanning cell. In the latter case,
38−-- the index of the continued cell is provided.
39−data GridCell
40− = ContentCell Attr Alignment RowSpan ColSpan [Block]
41− | ContinuationCell CellIndex
42− deriving (Show)
43−
44−-- | Row index in a table part.
45−newtype RowIndex = RowIndex Int deriving (Enum, Eq, Ix, Ord, Show)
46−-- | Column index in a table part.
47−newtype ColIndex = ColIndex Int deriving (Enum, Eq, Ix, Ord, Show)
48−
49−-- | Index to a cell in a table part.
50−type CellIndex = (RowIndex, ColIndex)
51−
52−-- | Cells are placed on a grid. Row attributes are stored in a separate
53−-- array.
54−data Part = Part
55− { partAttr :: Attr
56− , partCellArray :: Array (RowIndex,ColIndex) GridCell
57− , partRowAttrs :: Array RowIndex Attr
58− }
59−
60−data Table = Table
61− { tableAttr :: Attr
62− , tableCaption :: Caption
63− , tableColSpecs :: Array ColIndex ColSpec
64− , tableRowHeads :: RowHeadColumns
65− , tableHead :: Part
66− , tableBodies :: [Part]
67− , tableFoot :: Part
68− }
69−
70−toTable
71− :: B.Attr
72− -> B.Caption
73− -> [B.ColSpec]
74− -> B.TableHead
75− -> [B.TableBody]
76− -> B.TableFoot
77− -> Table
78−toTable attr caption colSpecs thead tbodies tfoot =
79− Table attr caption colSpecs' rowHeads thGrid tbGrids tfGrid
80− where
81− colSpecs' = listArray (ColIndex 1, ColIndex $ length colSpecs) colSpecs
82− rowHeads = case listToMaybe tbodies of
83− Nothing -> RowHeadColumns 0
84− Just (TableBody _attr rowHeadCols _headerRows _rows) -> rowHeadCols
85− thGrid = let (TableHead headAttr rows) = thead
86− in rowsToPart headAttr rows
87− tbGrids = map bodyToGrid tbodies
88− tfGrid = let (TableFoot footAttr rows) = tfoot
89− in rowsToPart footAttr rows
90− bodyToGrid (TableBody bodyAttr _rowHeadCols headRows rows) =
91− rowsToPart bodyAttr (headRows ++ rows)
92−
93−data BuilderCell
94− = FilledCell GridCell
95− | FreeCell
96−
97−fromBuilderCell :: BuilderCell -> GridCell
98−fromBuilderCell = \case
99− FilledCell c -> c
100− FreeCell -> error "Found an unassigned cell."
101−
102−rowsToPart :: Attr -> [B.Row] -> Part
103−rowsToPart attr = \case
104− [] -> Part
105− attr
106− (listArray ((RowIndex 1, ColIndex 1), (RowIndex 0, ColIndex 0)) [])
107− (listArray (RowIndex 1, RowIndex 0) [])
108− rows@(Row _attr firstRow:_) ->
109− let nrows = length rows
110− ncols = sum $ map (\(Cell _ _ _ (ColSpan cs) _) -> cs) firstRow
111− gbounds = ((RowIndex 1, ColIndex 1), (RowIndex nrows, ColIndex ncols))
112− mutableGrid :: ST s (STArray s CellIndex GridCell)
113− mutableGrid = do
114− grid <- newArray gbounds FreeCell
115− ridx <- newSTRef (RowIndex 1)
116− forM_ rows $ \(Row _attr cells) -> do
117− cidx <- newSTRef (ColIndex 1)
118− forM_ cells $ \(Cell cellAttr align rs cs blks) -> do
119− ridx' <- readSTRef ridx
120− let nextFreeInRow colindex@(ColIndex c) = do
121− readArray grid (ridx', colindex) >>= \case
122− FreeCell -> pure colindex
123− _ -> nextFreeInRow $ ColIndex (c + 1)
124− cidx' <- readSTRef cidx >>= nextFreeInRow
125− writeArray grid (ridx', cidx') . FilledCell $
126− ContentCell cellAttr align rs cs blks
127− forM_ (continuationIndices ridx' cidx' rs cs) $ \idx -> do
128− writeArray grid idx . FilledCell $
129− ContinuationCell (ridx', cidx')
130− -- go to new column
131− writeSTRef cidx cidx'
132− -- go to next row
133− modifySTRef ridx (incrRowIndex 1)
134− -- Swap BuilderCells with normal GridCells.
135− mapArray fromBuilderCell grid
136− in Part
137− { partCellArray = runSTArray mutableGrid
138− , partRowAttrs = listArray (RowIndex 1, RowIndex nrows) $
139− map (\(Row rowAttr _) -> rowAttr) rows
140− , partAttr = attr
141− }
142−
143−continuationIndices :: RowIndex -> ColIndex -> RowSpan -> ColSpan -> [CellIndex]
144−continuationIndices (RowIndex ridx) (ColIndex cidx) rowspan colspan =
145− let (RowSpan rs) = rowspan
146− (ColSpan cs) = colspan
147− in [ (RowIndex r, ColIndex c) | r <- [ridx..(ridx + rs - 1)]
148− , c <- [cidx..(cidx + cs - 1)]
149− , (r, c) /= (ridx, cidx)]
150−
151−rowArray :: RowIndex -> Array CellIndex GridCell -> Array ColIndex GridCell
152−rowArray ridx grid =
153− let ((_minRidx, minCidx), (_maxRidx, maxCidx)) = bounds grid
154− in ixmap (minCidx, maxCidx) (ridx,) grid
155−
156−incrRowIndex :: RowSpan -> RowIndex -> RowIndex
157−incrRowIndex (RowSpan n) (RowIndex r) = RowIndex $ r + n
src/Text/Pandoc/Writers/HTML.hs+34−107
@@ -26,18 +26,13 @@ module Text.Pandoc.Writers.HTML (
2626 writeSlideous,
2727 writeDZSlides,
2828 writeRevealJs,
29− tagWithAttributes,
30− inlineToHtml,
31− blockToHtml,
32− WriterState(..),
33− defaultWriterState
29+ tagWithAttributes
3430 ) where
35−
3631import Control.Monad.State.Strict
3732import Data.Char (ord)
38−import Data.List (intercalate, intersperse, partition, delete, (\\), foldl')
33+import Data.List (intercalate, intersperse, partition, delete, (\\))
3934import Data.List.NonEmpty (NonEmpty((:|)))
40−import Data.Maybe (fromMaybe, isJust, isNothing)
35+import Data.Maybe (fromMaybe, isJust, isNothing, mapMaybe)
4136import qualified Data.Set as Set
4237import Data.Text (Text)
4338import qualified Data.Text as T
@@ -349,52 +344,6 @@ pandocToHtml opts (Pandoc meta blocks) = do
349344 PlainMath -> defField "displaymath-css" True
350345 WebTeX _ -> defField "displaymath-css" True
351346 _ -> id) .
352− (if slideVariant == RevealJsSlides
353− then -- set boolean options explicitly, since
354− -- template can't distinguish False/undefined
355− defField "controls" True .
356− defField "controlsTutorial" True .
357− defField "controlsLayout" ("bottom-right" :: Text) .
358− defField "controlsBackArrows" ("faded" :: Text) .
359− defField "progress" True .
360− defField "slideNumber" False .
361− defField "showSlideNumber" ("all" :: Text) .
362− defField "hashOneBasedIndex" False .
363− defField "hash" False .
364− defField "respondToHashChanges" True .
365− defField "history" False .
366− defField "keyboard" True .
367− defField "overview" True .
368− defField "disableLayout" False .
369− defField "center" True .
370− defField "touch" True .
371− defField "loop" False .
372− defField "rtl" False .
373− defField "navigationMode" ("default" :: Text) .
374− defField "shuffle" False .
375− defField "fragments" True .
376− defField "fragmentInURL" True .
377− defField "embedded" False .
378− defField "help" True .
379− defField "pause" True .
380− defField "showNotes" False .
381− defField "autoPlayMedia" ("null" :: Text) .
382− defField "preloadIframes" ("null" :: Text) .
383− defField "autoSlide" ("0" :: Text) .
384− defField "autoSlideStoppable" True .
385− defField "autoSlideMethod" ("null" :: Text) .
386− defField "defaultTiming" ("null" :: Text) .
387− defField "mouseWheel" False .
388− defField "display" ("block" :: Text) .
389− defField "hideInactiveCursor" True .
390− defField "hideCursorTime" ("5000" :: Text) .
391− defField "previewLinks" False .
392− defField "transition" ("slide" :: Text) .
393− defField "transitionSpeed" ("default" :: Text) .
394− defField "backgroundTransition" ("fade" :: Text) .
395− defField "viewDistance" ("3" :: Text) .
396− defField "mobileViewDistance" ("2" :: Text)
397− else id) .
398347 defField "document-css" (isNothing mCss && slideVariant == NoSlides) .
399348 defField "quotes" (stQuotes st) .
400349 -- for backwards compatibility we populate toc
@@ -595,35 +544,30 @@ tagWithAttributes opts html5 selfClosing tagname attr =
595544
596545addAttrs :: PandocMonad m
597546 => WriterOptions -> Attr -> Html -> StateT WriterState m Html
598−addAttrs opts attr h = foldl' (!) h <$> attrsToHtml opts attr
547+addAttrs opts attr h = foldl (!) h <$> attrsToHtml opts attr
599548
600549toAttrs :: PandocMonad m
601550 => [(Text, Text)] -> StateT WriterState m [Attribute]
602551toAttrs kvs = do
603552 html5 <- gets stHtml5
604553 mbEpubVersion <- gets stEPUBVersion
605− reverse . snd <$> foldM (go html5 mbEpubVersion) (Set.empty, []) kvs
606− where
607− go html5 mbEpubVersion (keys, attrs) (k,v) = do
608− if k `Set.member` keys
609− then do
610− report $ DuplicateAttribute k v
611− return (keys, attrs)
612− else return (Set.insert k keys, addAttr html5 mbEpubVersion k v attrs)
613− addAttr html5 mbEpubVersion x y
614− | html5
615− = if x `Set.member` (html5Attributes <> rdfaAttributes)
616− || T.any (== ':') x -- e.g. epub: namespace
617− || "data-" `T.isPrefixOf` x
618− || "aria-" `T.isPrefixOf` x
619− then (customAttribute (textTag x) (toValue y) :)
620− else (customAttribute (textTag ("data-" <> x)) (toValue y) :)
621− | mbEpubVersion == Just EPUB2
622− , not (x `Set.member` (html4Attributes <> rdfaAttributes) ||
623− "xml:" `T.isPrefixOf` x)
624− = id
625− | otherwise
626− = (customAttribute (textTag x) (toValue y) :)
554+ return $ mapMaybe (\(x,y) ->
555+ if html5
556+ then
557+ if x `Set.member` (html5Attributes <> rdfaAttributes)
558+ || T.any (== ':') x -- e.g. epub: namespace
559+ || "data-" `T.isPrefixOf` x
560+ || "aria-" `T.isPrefixOf` x
561+ then Just $ customAttribute (textTag x) (toValue y)
562+ else Just $ customAttribute (textTag ("data-" <> x))
563+ (toValue y)
564+ else
565+ if mbEpubVersion == Just EPUB2 &&
566+ not (x `Set.member` (html4Attributes <> rdfaAttributes) ||
567+ "xml:" `T.isPrefixOf` x)
568+ then Nothing
569+ else Just $ customAttribute (textTag x) (toValue y))
570+ kvs
627571
628572attrsToHtml :: PandocMonad m
629573 => WriterOptions -> Attr -> StateT WriterState m [Attribute]
@@ -664,20 +608,17 @@ dimensionsToAttrList attr = consolidateStyles $ go Width ++ go Height
664608figure :: PandocMonad m
665609 => WriterOptions -> Attr -> [Inline] -> (Text, Text)
666610 -> StateT WriterState m Html
667−figure opts attr@(_, _, attrList) txt (s,tit) = do
611+figure opts attr txt (s,tit) = do
668612 html5 <- gets stHtml5
669613 -- Screen-readers will normally read the @alt@ text and the figure; we
670614 -- want to avoid them reading the same text twice. With HTML5 we can
671615 -- use aria-hidden for the caption; with HTML4, we use an empty
672616 -- alt-text instead.
673− -- When the alt text differs from the caption both should be read.
674617 let alt = if html5 then txt else [Str ""]
675618 let tocapt = if html5
676− then (H5.figcaption !) $
677− if isJust (lookup "alt" attrList)
678− then mempty
679− else H5.customAttribute (textTag "aria-hidden")
680− (toValue @Text "true")
619+ then H5.figcaption !
620+ H5.customAttribute (textTag "aria-hidden")
621+ (toValue @Text "true")
681622 else H.p ! A.class_ "caption"
682623 img <- inlineToHtml opts (Image attr alt (s,tit))
683624 capt <- if null txt
@@ -824,10 +765,9 @@ blockToHtml opts (Div attr@(ident, classes, kvs') bs) = do
824765 lookup "entry-spacing" kvs' >>=
825766 safeRead }
826767 let isCslBibEntry = "csl-entry" `elem` classes
827− let kvs = [(k,v) | (k,v) <- kvs'
828− , k /= "width" || "column" `notElem` classes] ++
829− [("style", "width:" <> w <> ";") | "column" `elem` classes
830− , ("width", w) <- kvs'] ++
768+ let kvs = [(k,v) | (k,v) <- kvs', k /= "width"] ++
769+ [("style", "width:" <> w <> ";") | "column" `elem` classes,
770+ ("width", w) <- kvs'] ++
831771 [("role", "doc-bibliography") | isCslBibBody && html5] ++
832772 [("role", "doc-biblioentry") | isCslBibEntry && html5]
833773 let speakerNotes = "notes" `elem` classes
@@ -937,7 +877,7 @@ blockToHtml opts (BlockQuote blocks) = do
937877 else do
938878 contents <- blockListToHtml opts blocks
939879 return $ H.blockquote $ nl opts >> contents >> nl opts
940−blockToHtml opts (Header level (ident,classes,kvs) lst) = do
880+blockToHtml opts (Header level attr@(_,classes,kvs) lst) = do
941881 contents <- inlineListToHtml opts lst
942882 let secnum = fromMaybe mempty $ lookup "number" kvs
943883 let contents' = if writerNumberSections opts && not (T.null secnum)
@@ -945,13 +885,7 @@ blockToHtml opts (Header level (ident,classes,kvs) lst) = do
945885 then (H.span ! A.class_ "header-section-number"
946886 $ toHtml secnum) >> strToHtml " " >> contents
947887 else contents
948− html5 <- gets stHtml5
949− let kvs' = if html5
950− then kvs
951− else [ (k, v) | (k, v) <- kvs
952− , k `elem` (["lang", "dir", "title", "style"
953− , "align"] ++ intrinsicEventsHTML4)]
954− addAttrs opts (ident,classes,kvs')
888+ addAttrs opts attr
955889 $ case level of
956890 1 -> H.h1 contents'
957891 2 -> H.h2 contents'
@@ -987,7 +921,7 @@ blockToHtml opts (OrderedList (startnum, numstyle, _) lst) = do
987921 numstyle']
988922 else [])
989923 l <- ordList opts contents
990− return $ foldl' (!) l attribs
924+ return $ foldl (!) l attribs
991925blockToHtml opts (DefinitionList lst) = do
992926 contents <- mapM (\(term, defs) ->
993927 do term' <- liftM H.dt $ inlineListToHtml opts term
@@ -1441,7 +1375,7 @@ inlineToHtml opts inline = do
14411375 return $ if T.null tit
14421376 then link'
14431377 else link' ! A.title (toValue tit)
1444− (Image attr@(_, _, attrList) txt (s, tit)) -> do
1378+ (Image attr txt (s,tit)) -> do
14451379 let alternate = stringify txt
14461380 slideVariant <- gets stSlideVariant
14471381 let isReveal = slideVariant == RevealJsSlides
@@ -1454,8 +1388,7 @@ inlineToHtml opts inline = do
14541388 [A.title $ toValue tit | not (T.null tit)] ++
14551389 attrs
14561390 imageTag = (if html5 then H5.img else H.img
1457− , [A.alt $ toValue alternate | not (null txt) &&
1458− isNothing (lookup "alt" attrList)] )
1391+ , [A.alt $ toValue alternate | not (null txt)] )
14591392 mediaTag tg fallbackTxt =
14601393 let linkTxt = if null txt
14611394 then fallbackTxt
@@ -1469,7 +1402,7 @@ inlineToHtml opts inline = do
14691402 Just "audio" -> mediaTag H5.audio "Audio"
14701403 Just _ -> (H5.embed, [])
14711404 _ -> imageTag
1472− return $ foldl' (!) tag $ attributes ++ specAttrs
1405+ return $ foldl (!) tag $ attributes ++ specAttrs
14731406 -- note: null title included, as in Markdown.pl
14741407 (Note contents) -> do
14751408 notes <- gets stNotes
@@ -1588,12 +1521,6 @@ allowsMathEnvironments MathML = True
15881521allowsMathEnvironments (WebTeX _) = True
15891522allowsMathEnvironments _ = False
15901523
1591−-- | List of intrinsic event attributes allowed on all elements in HTML4.
1592−intrinsicEventsHTML4 :: [Text]
1593−intrinsicEventsHTML4 =
1594− [ "onclick", "ondblclick", "onmousedown", "onmouseup", "onmouseover"
1595− , "onmouseout", "onmouseout", "onkeypress", "onkeydown", "onkeyup"]
1596−
15971524isRawHtml :: PandocMonad m => Format -> StateT WriterState m Bool
15981525isRawHtml f = do
15991526 html5 <- gets stHtml5
src/Text/Pandoc/Writers/Haddock.hs+2−13
@@ -15,7 +15,6 @@ Haddock: <http://www.haskell.org/haddock/doc/html/>
1515-}
1616module Text.Pandoc.Writers.Haddock (writeHaddock) where
1717import Control.Monad.State.Strict
18−import Data.Char (isAlphaNum)
1918import Data.Default
2019import Data.Text (Text)
2120import qualified Data.Text as T
@@ -72,18 +71,8 @@ notesToHaddock opts notes =
7271
7372-- | Escape special characters for Haddock.
7473escapeString :: Text -> Text
75−escapeString t
76− | T.all isAlphaNum t = t
77− | otherwise = T.concatMap escChar t
78− where
79− escChar '\\' = "\\\\"
80− escChar '/' = "\\/"
81− escChar '\'' = "\\'"
82− escChar '`' = "\\`"
83− escChar '"' = "\\\""
84− escChar '@' = "\\@"
85− escChar '<' = "\\<"
86− escChar c = T.singleton c
74+escapeString = escapeStringUsing haddockEscapes
75+ where haddockEscapes = backslashEscapes "\\/'`\"@<"
8776
8877-- | Convert Pandoc block element to haddock.
8978blockToHaddock :: PandocMonad m
src/Text/Pandoc/Writers/JATS.hs+44−67
@@ -1,7 +1,6 @@
1−{-# LANGUAGE LambdaCase #-}
2−{-# LANGUAGE OverloadedStrings #-}
31{-# LANGUAGE ScopedTypeVariables #-}
4−{-# LANGUAGE ViewPatterns #-}
2+{-# LANGUAGE OverloadedStrings #-}
3+{-# LANGUAGE ViewPatterns #-}
54{- |
65 Module : Text.Pandoc.Writers.JATS
76 Copyright : 2017-2021 John MacFarlane
@@ -21,13 +20,12 @@ module Text.Pandoc.Writers.JATS
2120 , writeJatsPublishing
2221 , writeJatsArticleAuthoring
2322 ) where
24−import Control.Applicative ((<|>))
2523import Control.Monad.Reader
2624import Control.Monad.State
2725import Data.Generics (everywhere, mkT)
2826import Data.List (partition)
2927import qualified Data.Map as M
30−import Data.Maybe (fromMaybe, listToMaybe)
28+import Data.Maybe (fromMaybe)
3129import Data.Time (toGregorian, Day, parseTimeM, defaultTimeLocale, formatTime)
3230import qualified Data.Text as T
3331import Data.Text (Text)
@@ -82,7 +80,7 @@ writeJats tagSet opts d = do
8280 let environment = JATSEnv
8381 { jatsTagSet = tagSet
8482 , jatsInlinesWriter = inlinesToJATS
85− , jatsBlockWriter = wrappedBlocksToJATS
83+ , jatsBlockWriter = blockToJATS
8684 , jatsReferences = refs
8785 }
8886 let initialState = JATSState { jatsNotes = [] }
@@ -164,9 +162,11 @@ wrappedBlocksToJATS needsWrap opts =
164162 wrappedBlockToJATS b = do
165163 inner <- blockToJATS opts b
166164 return $
167− if needsWrap b
165+ if needsWrap b || isBlockQuote b -- see #7041
168166 then inTags True "p" [("specific-use","wrapper")] inner
169167 else inner
168+ isBlockQuote (BlockQuote _) = True
169+ isBlockQuote _ = False
170170
171171-- | Auxiliary function to convert Plain block to Para.
172172plainToPara :: Block -> Block
@@ -225,36 +225,33 @@ imageMimeType src kvs =
225225 (T.drop 1 . T.dropWhile (/='/') <$> mbMT)
226226 in (maintype, subtype)
227227
228−languageFor :: WriterOptions -> [Text] -> Text
229−languageFor opts classes =
228+languageFor :: [Text] -> Text
229+languageFor classes =
230230 case langs of
231231 (l:_) -> escapeStringForXML l
232232 [] -> ""
233− where
234− syntaxMap = writerSyntaxMap opts
235− isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
233+ where isLang l = T.toLower l `elem` map T.toLower languages
236234 langsFrom s = if isLang s
237235 then [s]
238− else (languagesByExtension syntaxMap) . T.toLower $ s
236+ else languagesByExtension . T.toLower $ s
239237 langs = concatMap langsFrom classes
240238
241−codeAttr :: WriterOptions -> Attr -> (Text, [(Text, Text)])
242−codeAttr opts (ident,classes,kvs) = (lang, attr)
239+codeAttr :: Attr -> (Text, [(Text, Text)])
240+codeAttr (ident,classes,kvs) = (lang, attr)
243241 where
244− attr = [("id", escapeNCName ident) | not (T.null ident)] ++
242+ attr = [("id",ident) | not (T.null ident)] ++
245243 [("language",lang) | not (T.null lang)] ++
246244 [(k,v) | (k,v) <- kvs, k `elem` ["code-type",
247245 "code-version", "executable",
248246 "language-version", "orientation",
249247 "platforms", "position", "specific-use"]]
250− lang = languageFor opts classes
248+ lang = languageFor classes
251249
252250-- | Convert a Pandoc block element to JATS.
253251blockToJATS :: PandocMonad m => WriterOptions -> Block -> JATS m (Doc Text)
254252blockToJATS _ Null = return empty
255253blockToJATS opts (Div (id',"section":_,kvs) (Header _lvl _ ils : xs)) = do
256− let idAttr = [ ("id", writerIdentifierPrefix opts <> escapeNCName id')
257− | not (T.null id')]
254+ let idAttr = [("id", writerIdentifierPrefix opts <> id') | not (T.null id')]
258255 let otherAttrs = ["sec-type", "specific-use"]
259256 let attribs = idAttr ++ [(k,v) | (k,v) <- kvs, k `elem` otherAttrs]
260257 title' <- inlinesToJATS opts ils
@@ -263,7 +260,7 @@ blockToJATS opts (Div (id',"section":_,kvs) (Header _lvl _ ils : xs)) = do
263260 inTagsSimple "title" title' $$ contents
264261-- Bibliography reference:
265262blockToJATS opts (Div (ident,_,_) [Para lst]) | "ref-" `T.isPrefixOf` ident =
266− inTags True "ref" [("id", escapeNCName ident)] .
263+ inTags True "ref" [("id", ident)] .
267264 inTagsSimple "mixed-citation" <$>
268265 inlinesToJATS opts lst
269266blockToJATS opts (Div ("refs",_,_) xs) = do
@@ -274,14 +271,14 @@ blockToJATS opts (Div ("refs",_,_) xs) = do
274271 return $ inTagsIndented "ref-list" contents
275272blockToJATS opts (Div (ident,[cls],kvs) bs) | cls `elem` ["fig", "caption", "table-wrap"] = do
276273 contents <- blocksToJATS opts bs
277− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
274+ let attr = [("id", ident) | not (T.null ident)] ++
278275 [("xml:lang",l) | ("lang",l) <- kvs] ++
279276 [(k,v) | (k,v) <- kvs, k `elem` ["specific-use",
280277 "content-type", "orientation", "position"]]
281278 return $ inTags True cls attr contents
282279blockToJATS opts (Div (ident,_,kvs) bs) = do
283280 contents <- blocksToJATS opts bs
284− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
281+ let attr = [("id", ident) | not (T.null ident)] ++
285282 [("xml:lang",l) | ("lang",l) <- kvs] ++
286283 [(k,v) | (k,v) <- kvs, k `elem` ["specific-use",
287284 "content-type", "orientation", "position"]]
@@ -299,7 +296,7 @@ blockToJATS opts (Para [Image (ident,_,kvs) txt
299296 let capt = if null txt
300297 then empty
301298 else inTagsSimple "caption" $ inTagsSimple "p" alt
302− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
299+ let attr = [("id", ident) | not (T.null ident)] ++
303300 [(k,v) | (k,v) <- kvs, k `elem` ["fig-type", "orientation",
304301 "position", "specific-use"]]
305302 let graphicattr = [("mimetype",maintype),
@@ -310,7 +307,7 @@ blockToJATS opts (Para [Image (ident,_,kvs) txt
310307 capt $$ selfClosingTag "graphic" graphicattr
311308blockToJATS _ (Para [Image (ident,_,kvs) _ (src, tit)]) = do
312309 let (maintype, subtype) = imageMimeType src kvs
313− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
310+ let attr = [("id", ident) | not (T.null ident)] ++
314311 [("mimetype", maintype),
315312 ("mime-subtype", subtype),
316313 ("xlink:href", src)] ++
@@ -326,16 +323,13 @@ blockToJATS opts (LineBlock lns) =
326323 blockToJATS opts $ linesToPara lns
327324blockToJATS opts (BlockQuote blocks) = do
328325 tagSet <- asks jatsTagSet
329− let needsWrap = if tagSet == TagSetArticleAuthoring
330− then not . isPara
331− else \case
332− Header{} -> True
333− HorizontalRule -> True
334− _ -> False
335− inTagsIndented "disp-quote" <$> wrappedBlocksToJATS needsWrap opts blocks
336−blockToJATS opts (CodeBlock a str) = return $
326+ let blocksToJats' = if tagSet == TagSetArticleAuthoring
327+ then wrappedBlocksToJATS (not . isPara)
328+ else blocksToJATS
329+ inTagsIndented "disp-quote" <$> blocksToJats' opts blocks
330+blockToJATS _ (CodeBlock a str) = return $
337331 inTags False tag attr (flush (text (T.unpack $ escapeStringForXML str)))
338− where (lang, attr) = codeAttr opts a
332+ where (lang, attr) = codeAttr a
339333 tag = if T.null lang then "preformat" else "code"
340334blockToJATS _ (BulletList []) = return empty
341335blockToJATS opts (BulletList lst) =
@@ -415,9 +409,9 @@ inlineToJATS opts (Quoted SingleQuote lst) = do
415409inlineToJATS opts (Quoted DoubleQuote lst) = do
416410 contents <- inlinesToJATS opts lst
417411 return $ char '“' <> contents <> char '”'
418−inlineToJATS opts (Code a str) =
412+inlineToJATS _ (Code a str) =
419413 return $ inTags False tag attr $ literal (escapeStringForXML str)
420− where (lang, attr) = codeAttr opts a
414+ where (lang, attr) = codeAttr a
421415 tag = if T.null lang then "monospace" else "code"
422416inlineToJATS _ il@(RawInline f x)
423417 | f == "jats" = return $ literal x
@@ -440,8 +434,7 @@ inlineToJATS opts (Note contents) = do
440434 let notenum = case notes of
441435 (n, _):_ -> n + 1
442436 [] -> 1
443− thenote <- inTags True "fn" [("id", "fn" <> tshow notenum)]
444− . (inTagsSimple "label" (literal $ tshow notenum) <>)
437+ thenote <- inTags True "fn" [("id","fn" <> tshow notenum)]
445438 <$> wrappedBlocksToJATS (not . isPara) opts
446439 (walk demoteHeaderAndRefs contents)
447440 modify $ \st -> st{ jatsNotes = (notenum, thenote) : notes }
@@ -449,34 +442,18 @@ inlineToJATS opts (Note contents) = do
449442 ("rid", "fn" <> tshow notenum)]
450443 $ text (show notenum)
451444inlineToJATS opts (Cite _ lst) =
445+ -- TODO revisit this after examining the jats.csl pipeline
452446 inlinesToJATS opts lst
453−inlineToJATS opts (Span (ident,classes,kvs) ils) = do
447+inlineToJATS opts (Span ("",_,[]) ils) = inlinesToJATS opts ils
448+inlineToJATS opts (Span (ident,_,kvs) ils) = do
454449 contents <- inlinesToJATS opts ils
455− let commonAttr = [("id", escapeNCName ident) | not (T.null ident)] ++
456− [("xml:lang",l) | ("lang",l) <- kvs] ++
457− [(k,v) | (k,v) <- kvs, k `elem` ["alt", "specific-use"]]
458− -- A named-content element is a good fit for spans, but requires a
459− -- content-type attribute to be present. We use either the explicit
460− -- attribute or the first class as content type. If neither is
461− -- available, then we fall back to using a @styled-content@ element.
462− let (tag, specificAttr) =
463− case lookup "content-type" kvs <|> listToMaybe classes of
464− Just ct -> ( "named-content"
465− , ("content-type", ct) :
466− [(k, v) | (k, v) <- kvs
467− , k `elem` ["rid", "vocab", "vocab-identifier",
468− "vocab-term", "vocab-term-identifier"]])
469− -- Fall back to styled-content
470− Nothing -> ("styled-content"
471− , [(k, v) | (k,v) <- kvs
472− , k `elem` ["style", "style-type", "style-detail",
473− "toggle"]])
474− let attr = commonAttr ++ specificAttr
475− -- unwrap if wrapping element would have no attributes
476− return $
477− if null attr
478− then contents
479− else inTags False tag attr contents
450+ let attr = [("id",ident) | not (T.null ident)] ++
451+ [("xml:lang",l) | ("lang",l) <- kvs] ++
452+ [(k,v) | (k,v) <- kvs
453+ , k `elem` ["content-type", "rationale",
454+ "rid", "specific-use"]]
455+ return $ selfClosingTag "milestone-start" attr <> contents <>
456+ selfClosingTag "milestone-end" []
480457inlineToJATS _ (Math t str) = do
481458 let addPref (Xml.Attr q v)
482459 | Xml.qName q == "xmlns" = Xml.Attr q{ Xml.qName = "xmlns:mml" } v
@@ -511,9 +488,9 @@ inlineToJATS _ (Link _attr [Str t] (T.stripPrefix "mailto:" -> Just email, _))
511488 return $ inTagsSimple "email" $ literal (escapeStringForXML email)
512489inlineToJATS opts (Link (ident,_,kvs) txt (T.uncons -> Just ('#', src), _)) = do
513490 let attr = mconcat
514− [ [("id", escapeNCName ident) | not (T.null ident)]
491+ [ [("id", ident) | not (T.null ident)]
515492 , [("alt", stringify txt) | not (null txt)]
516− , [("rid", escapeNCName src)]
493+ , [("rid", src)]
517494 , [(k,v) | (k,v) <- kvs, k `elem` ["ref-type", "specific-use"]]
518495 , [("ref-type", "bibr") | "ref-" `T.isPrefixOf` src]
519496 ]
@@ -523,7 +500,7 @@ inlineToJATS opts (Link (ident,_,kvs) txt (T.uncons -> Just ('#', src), _)) = do
523500 contents <- inlinesToJATS opts txt
524501 return $ inTags False "xref" attr contents
525502inlineToJATS opts (Link (ident,_,kvs) txt (src, tit)) = do
526− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
503+ let attr = [("id", ident) | not (T.null ident)] ++
527504 [("ext-link-type", "uri"),
528505 ("xlink:href", src)] ++
529506 [("xlink:title", tit) | not (T.null tit)] ++
@@ -541,7 +518,7 @@ inlineToJATS _ (Image (ident,_,kvs) _ (src, tit)) = do
541518 let subtype = fromMaybe "" $
542519 lookup "mime-subtype" kvs `mplus`
543520 (T.drop 1 . T.dropWhile (/='/') <$> mbMT)
544− let attr = [("id", escapeNCName ident) | not (T.null ident)] ++
521+ let attr = [("id", ident) | not (T.null ident)] ++
545522 [("mimetype", maintype),
546523 ("mime-subtype", subtype),
547524 ("xlink:href", src)] ++
src/Text/Pandoc/Writers/JATS/References.hs+2−3
@@ -29,7 +29,7 @@ import Text.Pandoc.Builder (Inlines)
2929import Text.Pandoc.Options (WriterOptions)
3030import Text.Pandoc.Shared (tshow)
3131import Text.Pandoc.Writers.JATS.Types
32−import Text.Pandoc.XML (escapeNCName, escapeStringForXML, inTags)
32+import Text.Pandoc.XML (escapeStringForXML, inTags)
3333import qualified Data.Text as T
3434
3535referencesToJATS :: PandocMonad m
@@ -46,8 +46,7 @@ referenceToJATS :: PandocMonad m
4646referenceToJATS _opts ref = do
4747 let refType = referenceType ref
4848 let pubType = [("publication-type", refType) | not (T.null refType)]
49− let ident = escapeNCName $ "ref-" <> unItemId (referenceId ref)
50− let wrap = inTags True "ref" [("id", ident)]
49+ let wrap = inTags True "ref" [("id", "ref-" <> unItemId (referenceId ref))]
5150 . inTags True "element-citation" pubType
5251 return . wrap . vcat $
5352 [ authors
src/Text/Pandoc/Writers/JATS/Table.hs+5−25
@@ -24,7 +24,7 @@ import Text.Pandoc.Definition
2424import Text.Pandoc.Options (WriterOptions)
2525import Text.Pandoc.Shared (tshow)
2626import Text.Pandoc.Writers.JATS.Types
27−import Text.Pandoc.XML (escapeNCName, inTags, inTagsIndented, selfClosingTag)
27+import Text.Pandoc.XML (inTags, inTagsIndented, selfClosingTag)
2828import qualified Data.Text as T
2929import qualified Text.Pandoc.Writers.AnnotatedTable as Ann
3030
@@ -34,19 +34,13 @@ tableToJATS :: PandocMonad m
3434 -> JATS m (Doc Text)
3535tableToJATS opts (Ann.Table attr caption colspecs thead tbodies tfoot) = do
3636 let (Caption _maybeShortCaption captionBlocks) = caption
37− -- Only paragraphs are allowed in captions, all other blocks must be
38− -- wrapped in @<p>@ elements.
39− let needsWrapping = \case
40− Plain{} -> False
41− Para{} -> False
42− _ -> True
4337 tbl <- captionlessTable opts attr colspecs thead tbodies tfoot
4438 captionDoc <- if null captionBlocks
4539 then return empty
4640 else do
4741 blockToJATS <- asks jatsBlockWriter
48− inTagsIndented "caption" <$>
49− blockToJATS needsWrapping opts captionBlocks
42+ inTagsIndented "caption" . vcat <$>
43+ mapM (blockToJATS opts) captionBlocks
5044 return $ inTags True "table-wrap" [] $ captionDoc $$ tbl
5145
5246captionlessTable :: PandocMonad m
@@ -222,7 +216,7 @@ cellToJats opts celltype (Ann.Cell (colspec :| _) _colNum cell) =
222216
223217toAttribs :: Attr -> [Text] -> [(Text, Text)]
224218toAttribs (ident, _classes, kvs) knownAttribs =
225− (if T.null ident then id else (("id", escapeNCName ident) :)) $
219+ (if T.null ident then id else (("id", ident) :)) $
226220 filter ((`elem` knownAttribs) . fst) kvs
227221
228222tableCellToJats :: PandocMonad m
@@ -236,7 +230,7 @@ tableCellToJats opts ctype colAlign (Cell attr align rowspan colspan item) = do
236230 inlinesToJats <- asks jatsInlinesWriter
237231 let cellContents = \case
238232 [Plain inlines] -> inlinesToJats opts inlines
239− blocks -> blockToJats needsWrapInCell opts blocks
233+ blocks -> vcat <$> mapM (blockToJats opts) blocks
240234 let tag' = case ctype of
241235 BodyCell -> "td"
242236 HeaderCell -> "th"
@@ -252,17 +246,3 @@ tableCellToJats opts ctype colAlign (Cell attr align rowspan colspan item) = do
252246 . maybeCons (colspanAttrib colspan)
253247 $ toAttribs attr validAttribs
254248 inTags False tag' attribs <$> cellContents item
255−
256−-- | Whether the JATS produced from this block should be wrapped in a
257−-- @<p>@ element when put directly below a @<td>@ element.
258−needsWrapInCell :: Block -> Bool
259−needsWrapInCell = \case
260− Plain{} -> False -- should be unwrapped anyway
261− Para{} -> False
262− BulletList{} -> False
263− OrderedList{} -> False
264− DefinitionList{} -> False
265− HorizontalRule -> False
266− CodeBlock{} -> False
267− RawBlock{} -> False -- responsibility of the user
268− _ -> True
src/Text/Pandoc/Writers/JATS/Types.hs+3−12
@@ -37,20 +37,11 @@ newtype JATSState = JATSState
3737 { jatsNotes :: [(Int, Doc Text)]
3838 }
3939
40−-- | Environment containing all information relevant for rendering.
4140data JATSEnv m = JATSEnv
42− { jatsTagSet :: JATSTagSet -- ^ The tag set that's being ouput
43−
44− , jatsBlockWriter :: (Block -> Bool)
45− -> WriterOptions -> [Block] -> JATS m (Doc Text)
46− -- ^ Converts a block list to JATS, wrapping top-level blocks into a
47− -- @<p>@ element if the property evaluates to @True@.
48− -- See #7227.
49−
41+ { jatsTagSet :: JATSTagSet
5042 , jatsInlinesWriter :: WriterOptions -> [Inline] -> JATS m (Doc Text)
51− -- ^ Converts an inline list to JATS.
52−
53− , jatsReferences :: [Reference Inlines] -- ^ List of references
43+ , jatsBlockWriter :: WriterOptions -> Block -> JATS m (Doc Text)
44+ , jatsReferences :: [Reference Inlines]
5445 }
5546
5647-- | JATS writer type
src/Text/Pandoc/Writers/Jira.hs+18−41
@@ -39,17 +39,11 @@ writeJira :: PandocMonad m => WriterOptions -> Pandoc -> m Text
3939writeJira opts = runDefaultConverter (writerWrapText opts) (pandocToJira opts)
4040
4141-- | State to keep track of footnotes.
42−data ConverterState = ConverterState
43− { stNotes :: [Text] -- ^ Footnotes to be appended to the end of the text
44− , stInPanel :: Bool -- ^ whether we are in a @{panel}@ block
45− }
42+newtype ConverterState = ConverterState { stNotes :: [Text] }
4643
4744-- | Initial converter state.
4845startState :: ConverterState
49−startState = ConverterState
50− { stNotes = []
51− , stInPanel = False
52− }
46+startState = ConverterState { stNotes = [] }
5347
5448-- | Converter monad
5549type JiraConverter m = ReaderT WrapOption (StateT ConverterState m)
@@ -132,19 +126,13 @@ toJiraCode :: PandocMonad m
132126 -> Text
133127 -> JiraConverter m [Jira.Block]
134128toJiraCode (ident, classes, _attribs) code = do
135− return . addAnchor ident . singleton $
136− case find (\c -> T.toLower c `elem` knownLanguages) classes of
137− Nothing -> Jira.NoFormat mempty code
138− Just l -> Jira.Code (Jira.Language l) mempty code
139−
140−-- | Prepends an anchor with the given identifier.
141−addAnchor :: Text -> [Jira.Block] -> [Jira.Block]
142−addAnchor ident =
143− if T.null ident
144− then id
145− else \case
146− Jira.Para xs : bs -> (Jira.Para (Jira.Anchor ident : xs) : bs)
147− bs -> (Jira.Para (singleton (Jira.Anchor ident)) : bs)
129+ let lang = case find (\c -> T.toLower c `elem` knownLanguages) classes of
130+ Nothing -> Jira.Language "java"
131+ Just l -> Jira.Language l
132+ let addAnchor b = if T.null ident
133+ then b
134+ else [Jira.Para (singleton (Jira.Anchor ident))] <> b
135+ return . addAnchor . singleton $ Jira.Code lang mempty code
148136
149137-- | Creates a Jira definition list
150138toJiraDefinitionList :: PandocMonad m
@@ -161,16 +149,11 @@ toJiraDefinitionList defItems = do
161149toJiraPanel :: PandocMonad m
162150 => Attr -> [Block]
163151 -> JiraConverter m [Jira.Block]
164−toJiraPanel (ident, classes, attribs) blocks = do
165− inPanel <- gets stInPanel
166− if inPanel || ("panel" `notElem` classes && null attribs)
167− then addAnchor ident <$> toJiraBlocks blocks
168− else do
169− modify $ \st -> st{ stInPanel = True }
170− jiraBlocks <- toJiraBlocks blocks
171− modify $ \st -> st{ stInPanel = inPanel }
172− let params = map (uncurry Jira.Parameter) attribs
173− return $ singleton (Jira.Panel params $ addAnchor ident jiraBlocks)
152+toJiraPanel attr blocks = do
153+ jiraBlocks <- toJiraBlocks blocks
154+ return $ if attr == nullAttr
155+ then jiraBlocks
156+ else singleton (Jira.Panel [] jiraBlocks)
174157
175158-- | Creates a Jira header
176159toJiraHeader :: PandocMonad m
@@ -280,8 +263,6 @@ toJiraLink (_, classes, _) (url, _) alias = do
280263 | Just email <- T.stripPrefix "mailto:" url' = (Jira.Email, email)
281264 | "user-account" `elem` classes = (Jira.User, dropTilde url)
282265 | "attachment" `elem` classes = (Jira.Attachment, url)
283− | "smart-card" `elem` classes = (Jira.SmartCard, url)
284− | "smart-link" `elem` classes = (Jira.SmartLink, url)
285266 | otherwise = (Jira.External, url)
286267 dropTilde txt = case T.uncons txt of
287268 Just ('~', username) -> username
@@ -311,13 +292,9 @@ quotedToJira qtype xs = do
311292spanToJira :: PandocMonad m
312293 => Attr -> [Inline]
313294 -> JiraConverter m [Jira.Inline]
314−spanToJira (ident, _classes, attribs) inls =
315− let wrap = case lookup "color" attribs of
316− Nothing -> id
317− Just color -> singleton . Jira.ColorInline (Jira.ColorName color)
318− in wrap <$> case ident of
319− "" -> toJiraInlines inls
320− _ -> (Jira.Anchor ident :) <$> toJiraInlines inls
295+spanToJira (ident, _classes, _attribs) inls = case ident of
296+ "" -> toJiraInlines inls
297+ _ -> (Jira.Anchor ident :) <$> toJiraInlines inls
321298
322299registerNotes :: PandocMonad m => [Block] -> JiraConverter m [Jira.Inline]
323300registerNotes contents = do
@@ -333,7 +310,7 @@ registerNotes contents = do
333310knownLanguages :: [Text]
334311knownLanguages =
335312 [ "actionscript", "ada", "applescript", "bash", "c", "c#", "c++"
336− , "css", "erlang", "go", "groovy", "haskell", "html", "java", "javascript"
313+ , "css", "erlang", "go", "groovy", "haskell", "html", "javascript"
337314 , "json", "lua", "nyan", "objc", "perl", "php", "python", "r", "ruby"
338315 , "scala", "sql", "swift", "visualbasic", "xml", "yaml"
339316 ]
src/Text/Pandoc/Writers/LaTeX.hs+12−37
@@ -1,5 +1,4 @@
11{-# LANGUAGE OverloadedStrings #-}
2−{-# LANGUAGE LambdaCase #-}
32{-# LANGUAGE TupleSections #-}
43{-# LANGUAGE PatternGuards #-}
54{-# LANGUAGE ScopedTypeVariables #-}
@@ -29,7 +28,7 @@ import qualified Data.Text as T
2928import Network.URI (unEscapeString)
3029import Text.DocTemplates (FromContext(lookupContext), renderTemplate,
3130 Val(..), Context(..))
32−import Text.Collate.Lang (Lang (..), renderLang)
31+import Text.Pandoc.BCP47 (Lang (..), getLang, renderLang)
3332import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
3433import Text.Pandoc.Definition
3534import Text.Pandoc.Highlighting (formatLaTeXBlock, formatLaTeXInline, highlight,
@@ -146,11 +145,6 @@ pandocToLaTeX options (Pandoc meta blocks) = do
146145
147146 let dirs = query (extract "dir") blocks
148147
149− let nociteIds = query (\case
150− Cite cs _ -> map citationId cs
151− _ -> [])
152− $ lookupMetaInlines "nocite" meta
153−
154148 let context = defField "toc" (writerTableOfContents options) $
155149 defField "toc-depth" (tshow
156150 (writerTOCDepth options -
@@ -173,7 +167,6 @@ pandocToLaTeX options (Pandoc meta blocks) = do
173167 defField "has-chapters" (stHasChapters st) $
174168 defField "has-frontmatter" (documentClass `elem` frontmatterClasses) $
175169 defField "listings" (writerListings options || stLHS st) $
176− defField "zero-width-non-joiner" (stZwnj st) $
177170 defField "beamer" beamer $
178171 (if stHighlighting st
179172 then case writerHighlightStyle options of
@@ -184,11 +177,9 @@ pandocToLaTeX options (Pandoc meta blocks) = do
184177 else id) $
185178 (case writerCiteMethod options of
186179 Natbib -> defField "biblio-title" biblioTitle .
187− defField "natbib" True .
188− defField "nocite-ids" nociteIds
180+ defField "natbib" True
189181 Biblatex -> defField "biblio-title" biblioTitle .
190− defField "biblatex" True .
191− defField "nocite-ids" nociteIds
182+ defField "biblatex" True
192183 _ -> id) $
193184 defField "colorlinks" (any hasStringValue
194185 ["citecolor", "urlcolor", "linkcolor", "toccolor",
@@ -291,12 +282,7 @@ blockToLaTeX :: PandocMonad m
291282 => Block -- ^ Block to convert
292283 -> LW m (Doc Text)
293284blockToLaTeX Null = return empty
294−blockToLaTeX (Div attr@(identifier,"block":dclasses,_)
295− (Header _ _ ils : bs)) = do
296− let blockname
297− | "example" `elem` dclasses = "exampleblock"
298− | "alert" `elem` dclasses = "alertblock"
299− | otherwise = "block"
285+blockToLaTeX (Div attr@(identifier,"block":_,_) (Header _ _ ils : bs)) = do
300286 ref <- toLabel identifier
301287 let anchor = if T.null identifier
302288 then empty
@@ -304,8 +290,8 @@ blockToLaTeX (Div attr@(identifier,"block":dclasses,_)
304290 braces (literal ref) <> braces empty
305291 title' <- inlineListToLaTeX ils
306292 contents <- blockListToLaTeX bs
307− wrapDiv attr $ ("\\begin" <> braces blockname <> braces title' <> anchor) $$
308− contents $$ "\\end" <> braces blockname
293+ wrapDiv attr $ ("\\begin{block}" <> braces title' <> anchor) $$
294+ contents $$ "\\end{block}"
309295blockToLaTeX (Div (identifier,"slide":dclasses,dkvs)
310296 (Header _ (_,hclasses,hkvs) ils : bs)) = do
311297 -- note: [fragile] is required or verbatim breaks
@@ -831,19 +817,7 @@ inlineToLaTeX (Code (_,classes,kvs) str) = do
831817 let chr = case "!\"'()*,-./:;?@" \\ T.unpack str of
832818 (c:_) -> c
833819 [] -> '!'
834− let isEscapable '\\' = True
835− isEscapable '{' = True
836− isEscapable '}' = True
837− isEscapable '%' = True
838− isEscapable '~' = True
839− isEscapable '_' = True
840− isEscapable '&' = True
841− isEscapable '#' = True
842− isEscapable '^' = True
843− isEscapable _ = False
844− let escChar c | isEscapable c = T.pack ['\\',c]
845− | otherwise = T.singleton c
846− let str' = T.concatMap escChar str
820+ let str' = escapeStringUsing (backslashEscapes "\\{}%~_&#^") str
847821 -- we always put lstinline in a dummy 'passthrough' command
848822 -- (defined in the default template) so that we don't have
849823 -- to change the way we escape characters depending on whether
@@ -894,9 +868,8 @@ inlineToLaTeX (Quoted qt lst) = do
894868 then char '`' <> inner <> char '\''
895869 else char '\x2018' <> inner <> char '\x2019'
896870 where
897− isQuoted (Span _ (x:_)) = isQuoted x
898− isQuoted (Quoted _ _) = True
899− isQuoted _ = False
871+ isQuoted (Quoted _ _) = True
872+ isQuoted _ = False
900873inlineToLaTeX (Str str) = do
901874 setEmptyLine False
902875 liftM literal $ stringToLaTeX TextString str
@@ -919,7 +892,7 @@ inlineToLaTeX il@(RawInline f str) = do
919892inlineToLaTeX LineBreak = do
920893 emptyLine <- gets stEmptyLine
921894 setEmptyLine True
922− return $ (if emptyLine then "\\strut " else "") <> "\\\\" <> cr
895+ return $ (if emptyLine then "~" else "") <> "\\\\" <> cr
923896inlineToLaTeX SoftBreak = do
924897 wrapText <- gets (writerWrapText . stOptions)
925898 case wrapText of
@@ -1050,3 +1023,5 @@ extractInline _ _ = []
10501023-- Look up a key in an attribute and give a list of its values
10511024lookKey :: Text -> Attr -> [Text]
10521025lookKey key (_,_,kvs) = maybe [] T.words $ lookup key kvs
1026+
1027+
src/Text/Pandoc/Writers/LaTeX/Caption.hs+1
@@ -1,3 +1,4 @@
1+{-# LANGUAGE OverloadedStrings #-}
12{- |
23 Module : Text.Pandoc.Writers.LaTeX.Caption
34 Copyright : Copyright (C) 2006-2021 John MacFarlane
src/Text/Pandoc/Writers/LaTeX/Lang.hs+60−61
@@ -15,7 +15,7 @@ module Text.Pandoc.Writers.LaTeX.Lang
1515 toBabel
1616 ) where
1717import Data.Text (Text)
18−import Text.Collate.Lang (Lang(..))
18+import Text.Pandoc.BCP47 (Lang (..))
1919
2020
2121-- In environments \Arabic instead of \arabic is used
@@ -25,89 +25,88 @@ toPolyglossiaEnv l =
2525 ("arabic", o) -> ("Arabic", o)
2626 x -> x
2727
28−-- Takes a list of the constituents of a BCP47 language code and
28+-- Takes a list of the constituents of a BCP 47 language code and
2929-- converts it to a Polyglossia (language, options) tuple
3030-- http://mirrors.ctan.org/macros/latex/contrib/polyglossia/polyglossia.pdf
3131toPolyglossia :: Lang -> (Text, Text)
32−toPolyglossia (Lang "ar" _ (Just "DZ") _ _ _) = ("arabic", "locale=algeria")
33−toPolyglossia (Lang "ar" _ (Just "IQ") _ _ _) = ("arabic", "locale=mashriq")
34−toPolyglossia (Lang "ar" _ (Just "JO") _ _ _) = ("arabic", "locale=mashriq")
35−toPolyglossia (Lang "ar" _ (Just "LB") _ _ _) = ("arabic", "locale=mashriq")
36−toPolyglossia (Lang "ar" _ (Just "LY") _ _ _) = ("arabic", "locale=libya")
37−toPolyglossia (Lang "ar" _ (Just "MA") _ _ _) = ("arabic", "locale=morocco")
38−toPolyglossia (Lang "ar" _ (Just "MR") _ _ _) = ("arabic", "locale=mauritania")
39−toPolyglossia (Lang "ar" _ (Just "PS") _ _ _) = ("arabic", "locale=mashriq")
40−toPolyglossia (Lang "ar" _ (Just "SY") _ _ _) = ("arabic", "locale=mashriq")
41−toPolyglossia (Lang "ar" _ (Just "TN") _ _ _) = ("arabic", "locale=tunisia")
42−toPolyglossia (Lang "de" _ _ vars _ _)
43− | "1901" `elem` vars = ("german", "spelling=old")
44−toPolyglossia (Lang "de" _ (Just "AT") vars _ _)
45− | "1901" `elem` vars = ("german", "variant=austrian, spelling=old")
46−toPolyglossia (Lang "de" _ (Just "AT") _ _ _) = ("german", "variant=austrian")
47−toPolyglossia (Lang "de" _ (Just "CH") vars _ _)
48− | "1901" `elem` vars = ("german", "variant=swiss, spelling=old")
49−toPolyglossia (Lang "de" _ (Just "CH") _ _ _) = ("german", "variant=swiss")
50−toPolyglossia (Lang "de" _ _ _ _ _) = ("german", "")
51−toPolyglossia (Lang "dsb" _ _ _ _ _) = ("lsorbian", "")
52−toPolyglossia (Lang "el" _ _ vars _ _)
53− | "polyton" `elem` vars = ("greek", "variant=poly")
54−toPolyglossia (Lang "en" _ (Just "AU") _ _ _) = ("english", "variant=australian")
55−toPolyglossia (Lang "en" _ (Just "CA") _ _ _) = ("english", "variant=canadian")
56−toPolyglossia (Lang "en" _ (Just "GB") _ _ _) = ("english", "variant=british")
57−toPolyglossia (Lang "en" _ (Just "NZ") _ _ _) = ("english", "variant=newzealand")
58−toPolyglossia (Lang "en" _ (Just "UK") _ _ _) = ("english", "variant=british")
59−toPolyglossia (Lang "en" _ (Just "US") _ _ _) = ("english", "variant=american")
60−toPolyglossia (Lang "grc" _ _ _ _ _) = ("greek", "variant=ancient")
61−toPolyglossia (Lang "hsb" _ _ _ _ _) = ("usorbian", "")
62−toPolyglossia (Lang "la" _ _ vars _ _)
63− | "x-classic" `elem` vars = ("latin", "variant=classic")
64−toPolyglossia (Lang "pt" _ (Just "BR") _ _ _) = ("portuguese", "variant=brazilian")
65−toPolyglossia (Lang "sl" _ _ _ _ _) = ("slovenian", "")
66−toPolyglossia x = (commonFromBcp47 x, "")
32+toPolyglossia (Lang "ar" _ "DZ" _) = ("arabic", "locale=algeria")
33+toPolyglossia (Lang "ar" _ "IQ" _) = ("arabic", "locale=mashriq")
34+toPolyglossia (Lang "ar" _ "JO" _) = ("arabic", "locale=mashriq")
35+toPolyglossia (Lang "ar" _ "LB" _) = ("arabic", "locale=mashriq")
36+toPolyglossia (Lang "ar" _ "LY" _) = ("arabic", "locale=libya")
37+toPolyglossia (Lang "ar" _ "MA" _) = ("arabic", "locale=morocco")
38+toPolyglossia (Lang "ar" _ "MR" _) = ("arabic", "locale=mauritania")
39+toPolyglossia (Lang "ar" _ "PS" _) = ("arabic", "locale=mashriq")
40+toPolyglossia (Lang "ar" _ "SY" _) = ("arabic", "locale=mashriq")
41+toPolyglossia (Lang "ar" _ "TN" _) = ("arabic", "locale=tunisia")
42+toPolyglossia (Lang "de" _ _ vars)
43+ | "1901" `elem` vars = ("german", "spelling=old")
44+toPolyglossia (Lang "de" _ "AT" vars)
45+ | "1901" `elem` vars = ("german", "variant=austrian, spelling=old")
46+toPolyglossia (Lang "de" _ "AT" _) = ("german", "variant=austrian")
47+toPolyglossia (Lang "de" _ "CH" vars)
48+ | "1901" `elem` vars = ("german", "variant=swiss, spelling=old")
49+toPolyglossia (Lang "de" _ "CH" _) = ("german", "variant=swiss")
50+toPolyglossia (Lang "de" _ _ _) = ("german", "")
51+toPolyglossia (Lang "dsb" _ _ _) = ("lsorbian", "")
52+toPolyglossia (Lang "el" _ "polyton" _) = ("greek", "variant=poly")
53+toPolyglossia (Lang "en" _ "AU" _) = ("english", "variant=australian")
54+toPolyglossia (Lang "en" _ "CA" _) = ("english", "variant=canadian")
55+toPolyglossia (Lang "en" _ "GB" _) = ("english", "variant=british")
56+toPolyglossia (Lang "en" _ "NZ" _) = ("english", "variant=newzealand")
57+toPolyglossia (Lang "en" _ "UK" _) = ("english", "variant=british")
58+toPolyglossia (Lang "en" _ "US" _) = ("english", "variant=american")
59+toPolyglossia (Lang "grc" _ _ _) = ("greek", "variant=ancient")
60+toPolyglossia (Lang "hsb" _ _ _) = ("usorbian", "")
61+toPolyglossia (Lang "la" _ _ vars)
62+ | "x-classic" `elem` vars = ("latin", "variant=classic")
63+toPolyglossia (Lang "pt" _ "BR" _) = ("portuguese", "variant=brazilian")
64+toPolyglossia (Lang "sl" _ _ _) = ("slovenian", "")
65+toPolyglossia x = (commonFromBcp47 x, "")
6766
68−-- Takes a list of the constituents of a BCP47 language code and
67+-- Takes a list of the constituents of a BCP 47 language code and
6968-- converts it to a Babel language string.
7069-- http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf
7170-- List of supported languages (slightly outdated):
7271-- http://tug.ctan.org/language/hyph-utf8/doc/generic/hyph-utf8/hyphenation.pdf
7372toBabel :: Lang -> Text
74−toBabel (Lang "de" _ (Just "AT") vars _ _)
73+toBabel (Lang "de" _ "AT" vars)
7574 | "1901" `elem` vars = "austrian"
7675 | otherwise = "naustrian"
77−toBabel (Lang "de" _ (Just "CH") vars _ _)
76+toBabel (Lang "de" _ "CH" vars)
7877 | "1901" `elem` vars = "swissgerman"
7978 | otherwise = "nswissgerman"
80−toBabel (Lang "de" _ _ vars _ _)
79+toBabel (Lang "de" _ _ vars)
8180 | "1901" `elem` vars = "german"
8281 | otherwise = "ngerman"
83−toBabel (Lang "dsb" _ _ _ _ _) = "lowersorbian"
84−toBabel (Lang "el" _ _ vars _ _)
82+toBabel (Lang "dsb" _ _ _) = "lowersorbian"
83+toBabel (Lang "el" _ _ vars)
8584 | "polyton" `elem` vars = "polutonikogreek"
86−toBabel (Lang "en" _ (Just "AU") _ _ _) = "australian"
87−toBabel (Lang "en" _ (Just "CA") _ _ _) = "canadian"
88−toBabel (Lang "en" _ (Just "GB") _ _ _) = "british"
89−toBabel (Lang "en" _ (Just "NZ") _ _ _) = "newzealand"
90−toBabel (Lang "en" _ (Just "UK") _ _ _) = "british"
91−toBabel (Lang "en" _ (Just "US") _ _ _) = "american"
92−toBabel (Lang "fr" _ (Just "CA") _ _ _) = "canadien"
93−toBabel (Lang "fra" _ _ vars _ _)
85+toBabel (Lang "en" _ "AU" _) = "australian"
86+toBabel (Lang "en" _ "CA" _) = "canadian"
87+toBabel (Lang "en" _ "GB" _) = "british"
88+toBabel (Lang "en" _ "NZ" _) = "newzealand"
89+toBabel (Lang "en" _ "UK" _) = "british"
90+toBabel (Lang "en" _ "US" _) = "american"
91+toBabel (Lang "fr" _ "CA" _) = "canadien"
92+toBabel (Lang "fra" _ _ vars)
9493 | "aca" `elem` vars = "acadian"
95−toBabel (Lang "grc" _ _ _ _ _) = "polutonikogreek"
96−toBabel (Lang "hsb" _ _ _ _ _) = "uppersorbian"
97−toBabel (Lang "la" _ _ vars _ _)
94+toBabel (Lang "grc" _ _ _) = "polutonikogreek"
95+toBabel (Lang "hsb" _ _ _) = "uppersorbian"
96+toBabel (Lang "la" _ _ vars)
9897 | "x-classic" `elem` vars = "classiclatin"
99−toBabel (Lang "pt" _ (Just "BR") _ _ _) = "brazilian"
100−toBabel (Lang "sl" _ _ _ _ _) = "slovene"
98+toBabel (Lang "pt" _ "BR" _) = "brazilian"
99+toBabel (Lang "sl" _ _ _) = "slovene"
101100toBabel x = commonFromBcp47 x
102101
103−-- Takes a list of the constituents of a BCP47 language code
102+-- Takes a list of the constituents of a BCP 47 language code
104103-- and converts it to a string shared by Babel and Polyglossia.
105104-- https://tools.ietf.org/html/bcp47#section-2.1
106105commonFromBcp47 :: Lang -> Text
107−commonFromBcp47 (Lang "sr" (Just "Cyrl") _ _ _ _) = "serbianc"
108−commonFromBcp47 (Lang "zh" (Just "Latn") _ vars _ _)
109− | "pinyin" `elem` vars = "pinyin"
110−commonFromBcp47 (Lang l _ _ _ _ _) = fromIso l
106+commonFromBcp47 (Lang "sr" "Cyrl" _ _) = "serbianc"
107+commonFromBcp47 (Lang "zh" "Latn" _ vars)
108+ | "pinyin" `elem` vars = "pinyin"
109+commonFromBcp47 (Lang l _ _ _) = fromIso l
111110 where
112111 fromIso "af" = "afrikaans"
113112 fromIso "am" = "amharic"
src/Text/Pandoc/Writers/LaTeX/Table.hs+8−21
@@ -16,7 +16,6 @@ module Text.Pandoc.Writers.LaTeX.Table
1616 ) where
1717import Control.Monad.State.Strict
1818import Data.List (intersperse)
19−import qualified Data.List.NonEmpty as NonEmpty
2019import Data.List.NonEmpty (NonEmpty ((:|)))
2120import Data.Text (Text)
2221import qualified Data.Text as T
@@ -26,8 +25,7 @@ import Text.DocLayout
2625 ( Doc, braces, cr, empty, hcat, hsep, isEmpty, literal, nest
2726 , text, vcat, ($$) )
2827import Text.Pandoc.Shared (blocksToInlines, splitBy, tshow)
29−import Text.Pandoc.Walk (walk, query)
30−import Data.Monoid (Any(..))
28+import Text.Pandoc.Walk (walk)
3129import Text.Pandoc.Writers.LaTeX.Caption (getCaption)
3230import Text.Pandoc.Writers.LaTeX.Notes (notesToLaTeX)
3331import Text.Pandoc.Writers.LaTeX.Types
@@ -221,7 +219,9 @@ footRows (Ann.TableFoot _attr rows) = map headerRowCells rows
221219-- we need to go to some lengths to get line breaks working:
222220-- as LineBreak bs = \vtop{\hbox{\strut as}\hbox{\strut bs}}.
223221fixLineBreaks :: Block -> Block
224−fixLineBreaks = walk fixLineBreaks'
222+fixLineBreaks (Para ils) = Para $ fixLineBreaks' ils
223+fixLineBreaks (Plain ils) = Plain $ fixLineBreaks' ils
224+fixLineBreaks x = x
225225
226226fixLineBreaks' :: [Inline] -> [Inline]
227227fixLineBreaks' ils = case splitBy (== LineBreak) ils of
@@ -245,13 +245,8 @@ cellToLaTeX :: PandocMonad m
245245 -> Ann.Cell
246246 -> LW m (Doc Text)
247247cellToLaTeX blockListToLaTeX celltype annotatedCell = do
248− let (Ann.Cell specs _colnum cell) = annotatedCell
249− let hasWidths = snd (NonEmpty.head specs) /= ColWidthDefault
250− let specAlign = fst (NonEmpty.head specs)
251− let (Cell _attr align' rowspan colspan blocks) = cell
252− let align = case align' of
253− AlignDefault -> specAlign
254− _ -> align'
248+ let (Ann.Cell _specs _colnum cell) = annotatedCell
249+ let (Cell _attr align rowspan colspan blocks) = cell
255250 beamer <- gets stBeamer
256251 externalNotes <- gets stExternalNotes
257252 inMinipage <- gets stInMinipage
@@ -262,13 +257,8 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
262257 Para{} -> True
263258 Plain{} -> True
264259 _ -> False
265− let hasLineBreak LineBreak = Any True
266− hasLineBreak _ = Any False
267− let hasLineBreaks = getAny $ query hasLineBreak blocks
268260 result <-
269− if not hasWidths || (celltype /= HeaderCell
270− && all isPlainOrPara blocks
271− && not hasLineBreaks)
261+ if all isPlainOrPara blocks
272262 then
273263 blockListToLaTeX $ walk fixLineBreaks $ walk displayMathToInline blocks
274264 else do
@@ -281,9 +271,7 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
281271 let halign = literal $ alignCommand align
282272 return $ "\\begin{minipage}" <> valign <>
283273 braces "\\linewidth" <> halign <> cr <>
284− cellContents <>
285− (if hasLineBreaks then "\\strut" else mempty)
286− <> cr <>
274+ cellContents <> cr <>
287275 "\\end{minipage}"
288276 modify $ \st -> st{ stExternalNotes = externalNotes }
289277 when (rowspan /= RowSpan 1) $
@@ -304,4 +292,3 @@ cellToLaTeX blockListToLaTeX celltype annotatedCell = do
304292data CellType
305293 = HeaderCell
306294 | BodyCell
307− deriving Eq
src/Text/Pandoc/Writers/LaTeX/Types.hs+1−2
@@ -40,7 +40,7 @@ data WriterState =
4040 , stCsquotes :: Bool -- ^ true if document uses csquotes
4141 , stHighlighting :: Bool -- ^ true if document has highlighted code
4242 , stIncremental :: Bool -- ^ true if beamer lists should be
43− , stZwnj :: Bool -- ^ true if document has a ZWNJ character
43+ -- displayed bit by bit
4444 , stInternalLinks :: [Text] -- ^ list of internal link targets
4545 , stBeamer :: Bool -- ^ produce beamer
4646 , stEmptyLine :: Bool -- ^ true if no content on line
@@ -74,7 +74,6 @@ startingState options =
7474 , stCsquotes = False
7575 , stHighlighting = False
7676 , stIncremental = writerIncremental options
77− , stZwnj = False
7877 , stInternalLinks = []
7978 , stBeamer = False
8079 , stEmptyLine = True
src/Text/Pandoc/Writers/LaTeX/Util.hs+3−4
@@ -22,7 +22,6 @@ module Text.Pandoc.Writers.LaTeX.Util (
2222where
2323
2424import Control.Applicative ((<|>))
25−import Control.Monad (when)
2625import Text.Pandoc.Class (PandocMonad, toLang)
2726import Text.Pandoc.Options (WriterOptions(..), isEnabled)
2827import Text.Pandoc.Writers.LaTeX.Types (LW, WriterState(..))
@@ -31,7 +30,7 @@ import Text.Pandoc.Highlighting (toListingsLanguage)
3130import Text.DocLayout
3231import Text.Pandoc.Definition
3332import Text.Pandoc.ImageSize (showFl)
34−import Control.Monad.State.Strict (gets, modify)
33+import Control.Monad.State.Strict (gets)
3534import Data.Text (Text)
3635import qualified Data.Text as T
3736import Text.Pandoc.Extensions (Extension(Ext_smart))
@@ -50,8 +49,6 @@ data StringContext = TextString
5049stringToLaTeX :: PandocMonad m => StringContext -> Text -> LW m Text
5150stringToLaTeX context zs = do
5251 opts <- gets stOptions
53− when ('\x200c' `elemText` zs) $
54− modify (\s -> s { stZwnj = True })
5552 return $ T.pack $
5653 foldr (go opts context) mempty $ T.unpack $
5754 if writerPreferAscii opts
@@ -273,3 +270,5 @@ mbBraced :: Text -> Text
273270mbBraced x = if not (T.all isAlphaNum x)
274271 then "{" <> x <> "}"
275272 else x
273+
274+
src/Text/Pandoc/Writers/Man.hs+2−2
@@ -16,7 +16,6 @@ Conversion of 'Pandoc' documents to roff man page format.
1616module Text.Pandoc.Writers.Man ( writeMan ) where
1717import Control.Monad.State.Strict
1818import Data.List (intersperse)
19−import Data.List.NonEmpty (nonEmpty)
2019import Data.Maybe (fromMaybe)
2120import Data.Text (Text)
2221import qualified Data.Text as T
@@ -176,7 +175,8 @@ blockToMan opts (BulletList items) = do
176175 return (vcat contents)
177176blockToMan opts (OrderedList attribs items) = do
178177 let markers = take (length items) $ orderedListMarkers attribs
179− let indent = 1 + maybe 0 maximum (nonEmpty (map T.length markers))
178+ let indent = 1 +
179+ maximum (map T.length markers)
180180 contents <- mapM (\(num, item) -> orderedListItemToMan opts num indent item) $
181181 zip markers items
182182 return (vcat contents)
src/Text/Pandoc/Writers/Markdown.hs+56−54
@@ -24,9 +24,8 @@ import Control.Monad.Reader
2424import Control.Monad.State.Strict
2525import Data.Default
2626import Data.List (intersperse, sortOn, transpose)
27−import Data.List.NonEmpty (nonEmpty, NonEmpty(..))
2827import qualified Data.Map as M
29−import Data.Maybe (fromMaybe, mapMaybe, isNothing)
28+import Data.Maybe (fromMaybe, mapMaybe)
3029import qualified Data.Set as Set
3130import Data.Text (Text)
3231import qualified Data.Text as T
@@ -43,7 +42,7 @@ import Text.Pandoc.Templates (renderTemplate)
4342import Text.DocTemplates (Val(..), Context(..), FromContext(..))
4443import Text.Pandoc.Walk
4544import Text.Pandoc.Writers.HTML (writeHtml5String)
46−import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown, linkAttributes, attrsToMarkdown)
45+import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown)
4746import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(..),
4847 WriterState(..),
4948 WriterEnv(..),
@@ -67,16 +66,7 @@ writePlain opts document =
6766-- | Convert Pandoc to Commonmark.
6867writeCommonMark :: PandocMonad m => WriterOptions -> Pandoc -> m Text
6968writeCommonMark opts document =
70− evalMD (pandocToMarkdown opts' document) def{ envVariant = Commonmark } def
71− where
72− opts' = opts{ writerExtensions =
73− -- These extensions can't be enabled or disabled
74− -- for commonmark because they're part of the core;
75− -- we set them here so that escapeText will behave
76− -- properly.
77− enableExtension Ext_all_symbols_escapable $
78− enableExtension Ext_intraword_underscores $
79− writerExtensions opts }
69+ evalMD (pandocToMarkdown opts document) def{ envVariant = Commonmark } def
8070
8171pandocTitleBlock :: Doc Text -> [Doc Text] -> Doc Text -> Doc Text
8272pandocTitleBlock tit auths dat =
@@ -141,17 +131,18 @@ valToYaml (SimpleVal x)
141131 | otherwise =
142132 if hasNewlines x
143133 then hang 0 ("|" <> cr) x
144− else if isNothing $ foldM needsDoubleQuotes True x
134+ else if fst $ foldr needsDoubleQuotes (False, True) x
145135 then "\"" <> fmap escapeInDoubleQuotes x <> "\""
146136 else x
147137 where
148− needsDoubleQuotes isFirst t
138+ needsDoubleQuotes t (positive, isFirst)
149139 = if T.any isBadAnywhere t ||
150140 (isFirst && T.any isYamlPunct (T.take 1 t))
151− then Nothing
152− else Just False
141+ then (True, False)
142+ else (positive, False)
153143 isBadAnywhere '#' = True
154144 isBadAnywhere ':' = True
145+ isBadAnywhere '`' = False
155146 isBadAnywhere _ = False
156147 hasNewlines NewLine = True
157148 hasNewlines BlankLines{} = True
@@ -256,11 +247,32 @@ noteToMarkdown opts num blocks = do
256247 then hang (writerTabStop opts) (marker <> spacer) contents
257248 else marker <> spacer <> contents
258249
259−-- | (Code) blocks with a single class and no attributes can just use it
260−-- standalone, no need to bother with curly braces.
261−classOrAttrsToMarkdown :: Attr -> Doc Text
262−classOrAttrsToMarkdown ("",[cls],[]) = literal cls
263−classOrAttrsToMarkdown attrs = attrsToMarkdown attrs
250+attrsToMarkdown :: Attr -> Doc Text
251+attrsToMarkdown attribs = braces $ hsep [attribId, attribClasses, attribKeys]
252+ where attribId = case attribs of
253+ ("",_,_) -> empty
254+ (i,_,_) -> "#" <> escAttr i
255+ attribClasses = case attribs of
256+ (_,[],_) -> empty
257+ (_,cs,_) -> hsep $
258+ map (escAttr . ("."<>))
259+ cs
260+ attribKeys = case attribs of
261+ (_,_,[]) -> empty
262+ (_,_,ks) -> hsep $
263+ map (\(k,v) -> escAttr k
264+ <> "=\"" <>
265+ escAttr v <> "\"") ks
266+ escAttr = mconcat . map escAttrChar . T.unpack
267+ escAttrChar '"' = literal "\\\""
268+ escAttrChar '\\' = literal "\\\\"
269+ escAttrChar c = literal $ T.singleton c
270+
271+linkAttributes :: WriterOptions -> Attr -> Doc Text
272+linkAttributes opts attr =
273+ if isEnabled Ext_link_attributes opts && attr /= nullAttr
274+ then attrsToMarkdown attr
275+ else empty
264276
265277-- | Ordered list start parser for use in Para below.
266278olMarker :: Parser Text ParserState ()
@@ -321,12 +333,9 @@ blockToMarkdown' opts (Div attrs ils) = do
321333 case () of
322334 _ | isEnabled Ext_fenced_divs opts &&
323335 attrs /= nullAttr ->
324− let attrsToMd = if variant == Commonmark
325− then attrsToMarkdown
326− else classOrAttrsToMarkdown
327− in nowrap (literal ":::" <+> attrsToMd attrs) $$
328− chomp contents $$
329− literal ":::" <> blankline
336+ nowrap (literal ":::" <+> attrsToMarkdown attrs) $$
337+ chomp contents $$
338+ literal ":::" <> blankline
330339 | isEnabled Ext_native_divs opts ||
331340 (isEnabled Ext_raw_html opts &&
332341 (variant == Commonmark ||
@@ -368,7 +377,7 @@ blockToMarkdown' opts (Plain inlines) = do
368377-- title beginning with fig: indicates figure
369378blockToMarkdown' opts (Para [Image attr alt (src,tgt@(T.stripPrefix "fig:" -> Just tit))])
370379 | isEnabled Ext_raw_html opts &&
371− not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) &&
380+ not (isEnabled Ext_link_attributes opts) &&
372381 attr /= nullAttr = -- use raw HTML
373382 (<> blankline) . literal . T.strip <$>
374383 writeHtml5String opts{ writerTemplate = Nothing }
@@ -433,8 +442,7 @@ blockToMarkdown' opts (Header level attr inlines) = do
433442 && id' == autoId -> empty
434443 (id',_,_) | isEnabled Ext_mmd_header_identifiers opts ->
435444 space <> brackets (literal id')
436− _ | isEnabled Ext_header_attributes opts ||
437− isEnabled Ext_attributes opts ->
445+ _ | isEnabled Ext_header_attributes opts ->
438446 space <> attrsToMarkdown attr
439447 | otherwise -> empty
440448 contents <- inlineListToMarkdown opts $
@@ -484,21 +492,19 @@ blockToMarkdown' opts (CodeBlock attribs str) = do
484492 | isEnabled Ext_fenced_code_blocks opts ->
485493 tildes <> attrs <> cr <> literal str <> cr <> tildes <> blankline
486494 _ -> nest (writerTabStop opts) (literal str) <> blankline
487− where
488− endlineLen c = maybe 3 ((+1) . maximum) $ nonEmpty $
489− [T.length ln
490− | ln <- map trim (T.lines str)
491− , T.pack [c,c,c] `T.isPrefixOf` ln
492− , T.all (== c) ln]
493− endline c = literal $ T.replicate (endlineLen c) $ T.singleton c
494− backticks = endline '`'
495− tildes = endline '~'
496− attrs = if isEnabled Ext_fenced_code_attributes opts ||
497− isEnabled Ext_attributes opts
498− then nowrap $ " " <> classOrAttrsToMarkdown attribs
499− else case attribs of
500− (_,cls:_,_) -> " " <> literal cls
501− _ -> empty
495+ where endline c = literal $ case [T.length ln
496+ | ln <- map trim (T.lines str)
497+ , T.pack [c,c,c] `T.isPrefixOf` ln
498+ , T.all (== c) ln] of
499+ [] -> T.replicate 3 $ T.singleton c
500+ xs -> T.replicate (maximum xs + 1) $ T.singleton c
501+ backticks = endline '`'
502+ tildes = endline '~'
503+ attrs = if isEnabled Ext_fenced_code_attributes opts
504+ then nowrap $ " " <> attrsToMarkdown attribs
505+ else case attribs of
506+ (_,cls:_,_) -> " " <> literal cls
507+ _ -> empty
502508blockToMarkdown' opts (BlockQuote blocks) = do
503509 variant <- asks envVariant
504510 -- if we're writing literate haskell, put a space before the bird tracks
@@ -511,7 +517,7 @@ blockToMarkdown' opts (BlockQuote blocks) = do
511517 return $ prefixed leader contents <> blankline
512518blockToMarkdown' opts t@(Table _ blkCapt specs thead tbody tfoot) = do
513519 let (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
514− let numcols = maximum (length aligns :| length widths :
520+ let numcols = maximum (length aligns : length widths :
515521 map length (headers:rows))
516522 caption' <- inlineListToMarkdown opts caption
517523 let caption''
@@ -613,8 +619,7 @@ pipeTable headless aligns rawHeaders rawRows = do
613619 blockFor AlignCenter x y = cblock (x + 2) (sp <> y <> sp) <> lblock 0 empty
614620 blockFor AlignRight x y = rblock (x + 2) (y <> sp) <> lblock 0 empty
615621 blockFor _ x y = lblock (x + 2) (sp <> y) <> lblock 0 empty
616− let widths = map (max 3 . maybe 3 maximum . nonEmpty . map offset) $
617− transpose (rawHeaders : rawRows)
622+ let widths = map (max 3 . maximum . map offset) $ transpose (rawHeaders : rawRows)
618623 let torow cs = nowrap $ literal "|" <>
619624 hcat (intersperse (literal "|") $
620625 zipWith3 blockFor aligns widths (map chomp cs))
@@ -648,11 +653,11 @@ pandocTable opts multiline headless aligns widths rawHeaders rawRows = do
648653 -- Number of characters per column necessary to output every cell
649654 -- without requiring a line break.
650655 -- The @+2@ is needed for specifying the alignment.
651− let numChars = (+ 2) . maybe 0 maximum . nonEmpty . map offset
656+ let numChars = (+ 2) . maximum . map offset
652657 -- Number of characters per column necessary to output every cell
653658 -- without requiring a line break *inside a word*.
654659 -- The @+2@ is needed for specifying the alignment.
655− let minNumChars = (+ 2) . maybe 0 maximum . nonEmpty . map minOffset
660+ let minNumChars = (+ 2) . maximum . map minOffset
656661 let columns = transpose (rawHeaders : rawRows)
657662 -- minimal column width without wrapping a single word
658663 let relWidth w col =
@@ -791,9 +796,6 @@ blockListToMarkdown opts blocks = do
791796 b1 : commentSep : fixBlocks (b2:bs)
792797 fixBlocks (Plain ils : bs@(RawBlock{}:_)) =
793798 Plain ils : fixBlocks bs
794− fixBlocks (Plain ils : bs@(Div{}:_))
795− | isEnabled Ext_fenced_divs opts =
796− Para ils : fixBlocks bs
797799 fixBlocks (Plain ils : bs) | inlist =
798800 Plain ils : fixBlocks bs
799801 fixBlocks (Plain ils : bs) =
src/Text/Pandoc/Writers/Markdown/Inline.hs+21−36
@@ -11,15 +11,12 @@
1111 Portability : portable
1212-}
1313module Text.Pandoc.Writers.Markdown.Inline (
14− inlineListToMarkdown,
15− linkAttributes,
16− attrsToMarkdown
14+ inlineListToMarkdown
1715 ) where
1816import Control.Monad.Reader
1917import Control.Monad.State.Strict
2018import Data.Char (isAlphaNum, isDigit)
2119import Data.List (find, intersperse)
22−import Data.List.NonEmpty (nonEmpty)
2320import qualified Data.Map as M
2421import Data.Maybe (fromMaybe)
2522import Data.Text (Text)
@@ -45,11 +42,7 @@ import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(..),
4542-- | Escape special characters for Markdown.
4643escapeText :: WriterOptions -> Text -> Text
4744escapeText opts = T.pack . go . T.unpack
48− where
49− startsWithSpace (' ':_) = True
50− startsWithSpace ('\t':_) = True
51− startsWithSpace [] = True
52− startsWithSpace _ = False
45+ where
5346 go [] = []
5447 go (c:cs) =
5548 case c of
@@ -62,13 +55,10 @@ escapeText opts = T.pack . go . T.unpack
6255 '@' | isEnabled Ext_citations opts ->
6356 case cs of
6457 (d:_)
65− | isAlphaNum d || d == '_' || d == '{'
58+ | isAlphaNum d || d == '_'
6659 -> '\\':'@':go cs
6760 _ -> '@':go cs
68− '#' | isEnabled Ext_space_in_atx_header opts
69− , startsWithSpace cs
70− -> '\\':'#':go cs
71− _ | c `elem` ['\\','`','*','_','[',']'] ->
61+ _ | c `elem` ['\\','`','*','_','[',']','#'] ->
7262 '\\':c:go cs
7363 '|' | isEnabled Ext_pipe_tables opts -> '\\':'|':go cs
7464 '^' | isEnabled Ext_superscript opts -> '\\':'^':go cs
@@ -90,8 +80,6 @@ escapeText opts = T.pack . go . T.unpack
9080 | isEnabled Ext_intraword_underscores opts
9181 , isAlphaNum c
9282 , isAlphaNum x -> c : '_' : x : go xs
93− '#':xs -> c : '#' : go xs
94− '>':xs -> c : '>' : go xs
9583 _ -> c : go cs
9684
9785attrsToMarkdown :: Attr -> Doc Text
@@ -117,7 +105,7 @@ attrsToMarkdown attribs = braces $ hsep [attribId, attribClasses, attribKeys]
117105
118106linkAttributes :: WriterOptions -> Attr -> Doc Text
119107linkAttributes opts attr =
120− if (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) && attr /= nullAttr
108+ if isEnabled Ext_link_attributes opts && attr /= nullAttr
121109 then attrsToMarkdown attr
122110 else empty
123111
@@ -394,15 +382,15 @@ inlineToMarkdown opts (Quoted DoubleQuote lst) = do
394382 then "&ldquo;" <> contents <> "&rdquo;"
395383 else "“" <> contents <> "”"
396384inlineToMarkdown opts (Code attr str) = do
397− let tickGroups = filter (T.any (== '`')) $ T.group str
398− let longest = maybe 0 maximum $ nonEmpty $ map T.length tickGroups
399− let marker = T.replicate (longest + 1) "`"
400− let spacer = if longest == 0 then "" else " "
401− let attrsEnabled = isEnabled Ext_inline_code_attributes opts ||
402− isEnabled Ext_attributes opts
403− let attrs = if attrsEnabled && attr /= nullAttr
404− then attrsToMarkdown attr
405− else empty
385+ let tickGroups = filter (T.any (== '`')) $ T.group str
386+ let longest = if null tickGroups
387+ then 0
388+ else maximum $ map T.length tickGroups
389+ let marker = T.replicate (longest + 1) "`"
390+ let spacer = if longest == 0 then "" else " "
391+ let attrs = if isEnabled Ext_inline_code_attributes opts && attr /= nullAttr
392+ then attrsToMarkdown attr
393+ else empty
406394 variant <- asks envVariant
407395 case variant of
408396 PlainText -> return $ literal str
@@ -450,7 +438,9 @@ inlineToMarkdown opts (Math DisplayMath str) =
450438 (texMathToInlines DisplayMath str >>= inlineListToMarkdown opts)
451439inlineToMarkdown opts il@(RawInline f str) = do
452440 let tickGroups = filter (T.any (== '`')) $ T.group str
453− let numticks = 1 + maybe 0 maximum (nonEmpty (map T.length tickGroups))
441+ let numticks = if null tickGroups
442+ then 1
443+ else 1 + maximum (map T.length tickGroups)
454444 variant <- asks envVariant
455445 let Format fmt = f
456446 let rawAttribInline = return $
@@ -504,16 +494,11 @@ inlineToMarkdown opts (Cite (c:cs) lst)
504494 rest <- mapM convertOne cs
505495 let inbr = suffs <+> joincits rest
506496 br = if isEmpty inbr then empty else char '[' <> inbr <> char ']'
507− return $ literal ("@" <> maybeInBraces (citationId c)) <+> br
497+ return $ literal ("@" <> citationId c) <+> br
508498 else do
509499 cits <- mapM convertOne (c:cs)
510500 return $ literal "[" <> joincits cits <> literal "]"
511501 where
512− maybeInBraces key =
513− case readWith (citeKey False >> spaces >> eof)
514− defaultParserState ("@" <> key) of
515− Left _ -> "{" <> key <> "}"
516− Right _ -> key
517502 joincits = hcat . intersperse (literal "; ") . filter (not . isEmpty)
518503 convertOne Citation { citationId = k
519504 , citationPrefix = pinlines
@@ -522,7 +507,7 @@ inlineToMarkdown opts (Cite (c:cs) lst)
522507 = do
523508 pdoc <- inlineListToMarkdown opts pinlines
524509 sdoc <- inlineListToMarkdown opts sinlines
525− let k' = literal (modekey m <> "@" <> maybeInBraces k)
510+ let k' = literal (modekey m <> "@" <> k)
526511 r = case sinlines of
527512 Str (T.uncons -> Just (y,_)):_ | y `elem` (",;]@" :: String) -> k' <> sdoc
528513 _ -> k' <+> sdoc
@@ -561,7 +546,7 @@ inlineToMarkdown opts lnk@(Link attr txt (src, tit)) = do
561546 else "[" <> reftext <> "]"
562547 in return $ first <> second
563548 | isEnabled Ext_raw_html opts
564− , not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts)
549+ , not (isEnabled Ext_link_attributes opts)
565550 , attr /= nullAttr -> -- use raw HTML to render attributes
566551 literal . T.strip <$>
567552 writeHtml5String opts{ writerTemplate = Nothing }
@@ -571,7 +556,7 @@ inlineToMarkdown opts lnk@(Link attr txt (src, tit)) = do
571556 linkAttributes opts attr
572557inlineToMarkdown opts img@(Image attr alternate (source, tit))
573558 | isEnabled Ext_raw_html opts &&
574− not (isEnabled Ext_link_attributes opts || isEnabled Ext_attributes opts) &&
559+ not (isEnabled Ext_link_attributes opts) &&
575560 attr /= nullAttr = -- use raw HTML
576561 literal . T.strip <$>
577562 writeHtml5String opts{ writerTemplate = Nothing } (Pandoc nullMeta [Plain [img]])
src/Text/Pandoc/Writers/Ms.hs+8−24
@@ -23,7 +23,6 @@ module Text.Pandoc.Writers.Ms ( writeMs ) where
2323import Control.Monad.State.Strict
2424import Data.Char (isLower, isUpper, ord)
2525import Data.List (intercalate, intersperse)
26−import Data.List.NonEmpty (nonEmpty)
2726import qualified Data.Map as Map
2827import Data.Maybe (catMaybes)
2928import Data.Text (Text)
@@ -245,17 +244,13 @@ blockToMs opts (Table _ blkCapt specs thead tbody tfoot) =
245244 aligncode AlignDefault = "l"
246245 in do
247246 caption' <- inlineListToMs' opts caption
248− let isSimple = all (== 0) widths
249− let totalWidth = 70
247+ let iwidths = if all (== 0) widths
248+ then repeat ""
249+ else map (T.pack . printf "w(%0.1fn)" . (70 *)) widths
250250 -- 78n default width - 8n indent = 70n
251251 let coldescriptions = literal $ T.unwords
252− (zipWith (\align width -> aligncode align <>
253− if width == 0
254− then ""
255− else T.pack $
256− printf "w(%0.1fn)"
257− (totalWidth * width))
258− alignments widths) <> "."
252+ (zipWith (\align width -> aligncode align <> width)
253+ alignments iwidths) <> "."
259254 colheadings <- mapM (blockListToMs opts) headers
260255 let makeRow cols = literal "T{" $$
261256 vcat (intersperse (literal "T}\tT{") cols) $$
@@ -264,25 +259,13 @@ blockToMs opts (Table _ blkCapt specs thead tbody tfoot) =
264259 then empty
265260 else makeRow colheadings $$ char '_'
266261 body <- mapM (\row -> do
267− cols <- mapM (\(cell, w) ->
268− (if isSimple
269− then id
270− else (literal (".nr LL " <>
271− T.pack (printf "%0.1fn"
272− (w * totalWidth))) $$)) <$>
273− blockListToMs opts cell) (zip row widths)
262+ cols <- mapM (blockListToMs opts) row
274263 return $ makeRow cols) rows
275264 setFirstPara
276265 return $ literal ".PP" $$ caption' $$
277266 literal ".na" $$ -- we don't want justification in table cells
278− (if isSimple
279− then ""
280− else ".nr LLold \\n[LL]") $$
281267 literal ".TS" $$ literal "delim(@@) tab(\t);" $$ coldescriptions $$
282268 colheadings' $$ vcat body $$ literal ".TE" $$
283− (if isSimple
284− then ""
285− else ".nr LL \\n[LLold]") $$
286269 literal ".ad"
287270
288271blockToMs opts (BulletList items) = do
@@ -291,7 +274,8 @@ blockToMs opts (BulletList items) = do
291274 return (vcat contents)
292275blockToMs opts (OrderedList attribs items) = do
293276 let markers = take (length items) $ orderedListMarkers attribs
294− let indent = 2 + maybe 0 maximum (nonEmpty (map T.length markers))
277+ let indent = 2 +
278+ maximum (map T.length markers)
295279 contents <- mapM (\(num, item) -> orderedListItemToMs opts num indent item) $
296280 zip markers items
297281 setFirstPara
src/Text/Pandoc/Writers/Muse.hs+3−6
@@ -31,7 +31,6 @@ import Control.Monad.State.Strict
3131import Data.Char (isAlphaNum, isAsciiLower, isAsciiUpper, isDigit, isSpace)
3232import Data.Default
3333import Data.List (intersperse, transpose)
34−import Data.List.NonEmpty (nonEmpty, NonEmpty(..))
3534import qualified Data.Set as Set
3635import qualified Data.Text as T
3736import Data.Text (Text)
@@ -159,8 +158,7 @@ simpleTable caption headers rows = do
159158 caption' <- inlineListToMuse caption
160159 headers' <- mapM blockListToMuse headers
161160 rows' <- mapM (mapM blockListToMuse) rows
162− let widthsInChars = maybe 0 maximum . nonEmpty . map offset <$>
163− transpose (headers' : rows')
161+ let widthsInChars = maximum . map offset <$> transpose (headers' : rows')
164162 let hpipeBlocks sep blocks = hcat $ intersperse sep' blocks
165163 where sep' = lblock (T.length sep) $ literal sep
166164 let makeRow sep = hpipeBlocks sep . zipWith lblock widthsInChars
@@ -240,7 +238,7 @@ blockToMuse (DefinitionList items) = do
240238 label' <- local (\env -> env { envOneLine = True, envAfterSpace = True }) $ inlineListToMuse' label
241239 let ind = offset' label' -- using Text.DocLayout.offset results in round trip failures
242240 hang ind (nowrap label') . vcat <$> mapM descriptionToMuse defs
243− where offset' d = maximum (0 :| map T.length
241+ where offset' d = maximum (0: map T.length
244242 (T.lines $ render Nothing d))
245243 descriptionToMuse :: PandocMonad m
246244 => [Block]
@@ -271,8 +269,7 @@ blockToMuse (Table _ blkCapt specs thead tbody tfoot) =
271269 (caption, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot
272270 blocksToDoc opts blocks =
273271 local (\env -> env { envOptions = opts }) $ blockListToMuse blocks
274− numcols = maximum
275− (length aligns :| length widths : map length (headers:rows))
272+ numcols = maximum (length aligns : length widths : map length (headers:rows))
276273 isSimple = onlySimpleTableCells (headers : rows) && all (== 0) widths
277274blockToMuse (Div _ bs) = flatBlockListToMuse bs
278275blockToMuse Null = return empty
src/Text/Pandoc/Writers/ODT.hs+3−4
@@ -16,7 +16,6 @@ import Codec.Archive.Zip
1616import Control.Monad.Except (catchError, throwError)
1717import Control.Monad.State.Strict
1818import qualified Data.ByteString.Lazy as B
19−import Data.Maybe (fromMaybe)
2019import Data.Generics (everywhere', mkT)
2120import Data.List (isPrefixOf)
2221import qualified Data.Map as Map
@@ -24,7 +23,7 @@ import qualified Data.Text as T
2423import qualified Data.Text.Lazy as TL
2524import Data.Time
2625import System.FilePath (takeDirectory, takeExtension, (<.>))
27−import Text.Collate.Lang (Lang (..), renderLang)
26+import Text.Pandoc.BCP47 (Lang (..), getLang, renderLang)
2827import Text.Pandoc.Class.PandocMonad (PandocMonad, report, toLang)
2928import qualified Text.Pandoc.Class.PandocMonad as P
3029import Text.Pandoc.Definition
@@ -36,7 +35,7 @@ import Text.Pandoc.Options (WrapOption (..), WriterOptions (..))
3635import Text.DocLayout
3736import Text.Pandoc.Shared (stringify, pandocVersion, tshow)
3837import Text.Pandoc.Writers.Shared (lookupMetaString, lookupMetaBlocks,
39− fixDisplayMath, getLang)
38+ fixDisplayMath)
4039import Text.Pandoc.UTF8 (fromStringLazy, fromTextLazy, toTextLazy)
4140import Text.Pandoc.Walk
4241import Text.Pandoc.Writers.OpenDocument (writeOpenDocument)
@@ -195,7 +194,7 @@ addLang lang = everywhere' (mkT updateLangAttr)
195194 where updateLangAttr (Attr n@(QName "language" _ (Just "fo")) _)
196195 = Attr n (langLanguage lang)
197196 updateLangAttr (Attr n@(QName "country" _ (Just "fo")) _)
198− = Attr n (fromMaybe "" $ langRegion lang)
197+ = Attr n (langRegion lang)
199198 updateLangAttr x = x
200199
201200-- | transform both Image and Math elements
src/Text/Pandoc/Writers/OpenDocument.hs+54−50
@@ -1,5 +1,4 @@
11{-# LANGUAGE FlexibleContexts #-}
2−{-# LANGUAGE LambdaCase #-}
32{-# LANGUAGE OverloadedStrings #-}
43{-# LANGUAGE PatternGuards #-}
54{-# LANGUAGE ViewPatterns #-}
@@ -26,7 +25,7 @@ import Data.Ord (comparing)
2625import qualified Data.Set as Set
2726import Data.Text (Text)
2827import qualified Data.Text as T
29−import Text.Collate.Lang (Lang (..), parseLang)
28+import Text.Pandoc.BCP47 (Lang (..), parseBCP47)
3029import Text.Pandoc.Class.PandocMonad (PandocMonad, report, translateTerm,
3130 setTranslations, toLang)
3231import Text.Pandoc.Definition
@@ -237,7 +236,7 @@ handleSpaces s = case T.uncons s of
237236-- | Convert Pandoc document to string in OpenDocument format.
238237writeOpenDocument :: PandocMonad m => WriterOptions -> Pandoc -> m Text
239238writeOpenDocument opts (Pandoc meta blocks) = do
240− let defLang = Lang "en" (Just "US") Nothing [] [] []
239+ let defLang = Lang "en" "US" "" []
241240 lang <- case lookupMetaString "lang" meta of
242241 "" -> pure defLang
243242 s -> fromMaybe defLang <$> toLang (Just s)
@@ -373,32 +372,36 @@ blocksToOpenDocument o b = vcat <$> mapM (blockToOpenDocument o) b
373372
374373-- | Convert a Pandoc block element to OpenDocument.
375374blockToOpenDocument :: PandocMonad m => WriterOptions -> Block -> OD m (Doc Text)
376−blockToOpenDocument o = \case
377− Plain b -> if null b
378− then return empty
379− else inParagraphTags =<< inlinesToOpenDocument o b
380− Para [Image attr c (s,T.stripPrefix "fig:" -> Just t)] -> figure attr c s t
381− Para b -> if null b &&
382− not (isEnabled Ext_empty_paragraphs o)
383− then return empty
384− else inParagraphTags =<< inlinesToOpenDocument o b
385− LineBlock b -> blockToOpenDocument o $ linesToPara b
386− Div attr xs -> mkDiv attr xs
387− Header i (ident,_,_) b -> do
388− setFirstPara
389− inHeaderTags i ident =<< inlinesToOpenDocument o b
390− BlockQuote b -> setFirstPara >> mkBlockQuote b
391− DefinitionList b -> setFirstPara >> defList b
392− BulletList b -> setFirstPara >> bulletListToOpenDocument o b
393− OrderedList a b -> setFirstPara >> orderedList a b
394− CodeBlock _ s -> setFirstPara >> preformatted s
395− Table a bc s th tb tf -> setFirstPara >> table (Ann.toTable a bc s th tb tf)
396− HorizontalRule -> setFirstPara >> return (selfClosingTag "text:p"
397− [ ("text:style-name", "Horizontal_20_Line") ])
398− b@(RawBlock f s) -> if f == Format "opendocument"
399− then return $ text $ T.unpack s
400− else empty <$ report (BlockNotRendered b)
401− Null -> return empty
375+blockToOpenDocument o bs
376+ | Plain b <- bs = if null b
377+ then return empty
378+ else inParagraphTags =<< inlinesToOpenDocument o b
379+ | Para [Image attr c (s,T.stripPrefix "fig:" -> Just t)] <- bs
380+ = figure attr c s t
381+ | Para b <- bs = if null b &&
382+ not (isEnabled Ext_empty_paragraphs o)
383+ then return empty
384+ else inParagraphTags =<< inlinesToOpenDocument o b
385+ | LineBlock b <- bs = blockToOpenDocument o $ linesToPara b
386+ | Div attr xs <- bs = mkDiv attr xs
387+ | Header i (ident,_,_) b
388+ <- bs = setFirstPara >> (inHeaderTags i ident
389+ =<< inlinesToOpenDocument o b)
390+ | BlockQuote b <- bs = setFirstPara >> mkBlockQuote b
391+ | DefinitionList b <- bs = setFirstPara >> defList b
392+ | BulletList b <- bs = setFirstPara >> bulletListToOpenDocument o b
393+ | OrderedList a b <- bs = setFirstPara >> orderedList a b
394+ | CodeBlock _ s <- bs = setFirstPara >> preformatted s
395+ | Table a bc s th tb tf <- bs = setFirstPara >> table (Ann.toTable a bc s th tb tf)
396+ | HorizontalRule <- bs = setFirstPara >> return (selfClosingTag "text:p"
397+ [ ("text:style-name", "Horizontal_20_Line") ])
398+ | RawBlock f s <- bs = if f == Format "opendocument"
399+ then return $ text $ T.unpack s
400+ else do
401+ report $ BlockNotRendered bs
402+ return empty
403+ | Null <- bs = return empty
404+ | otherwise = return empty
402405 where
403406 defList b = do setInDefinitionList True
404407 r <- vcat <$> mapM (deflistItemToOpenDocument o) b
@@ -871,33 +874,34 @@ data TextStyle = Italic
871874textStyleAttr :: Map.Map Text Text
872875 -> TextStyle
873876 -> Map.Map Text Text
874−textStyleAttr m = \case
875− Italic -> Map.insert "fo:font-style" "italic" .
876− Map.insert "style:font-style-asian" "italic" .
877− Map.insert "style:font-style-complex" "italic" $ m
878− Bold -> Map.insert "fo:font-weight" "bold" .
879− Map.insert "style:font-weight-asian" "bold" .
880− Map.insert "style:font-weight-complex" "bold" $ m
881− Under -> Map.insert "style:text-underline-style" "solid" .
882− Map.insert "style:text-underline-width" "auto" .
883− Map.insert "style:text-underline-color" "font-color" $ m
884− Strike -> Map.insert "style:text-line-through-style" "solid" m
885− Sub -> Map.insert "style:text-position" "sub 58%" m
886− Sup -> Map.insert "style:text-position" "super 58%" m
887− SmallC -> Map.insert "fo:font-variant" "small-caps" m
888− Pre -> Map.insert "style:font-name" "Courier New" .
889− Map.insert "style:font-name-asian" "Courier New" .
890− Map.insert "style:font-name-complex" "Courier New" $ m
891− Language lang ->
892− Map.insert "fo:language" (langLanguage lang) .
893− maybe id (Map.insert "fo:country") (langRegion lang) $ m
877+textStyleAttr m s
878+ | Italic <- s = Map.insert "fo:font-style" "italic" .
879+ Map.insert "style:font-style-asian" "italic" .
880+ Map.insert "style:font-style-complex" "italic" $ m
881+ | Bold <- s = Map.insert "fo:font-weight" "bold" .
882+ Map.insert "style:font-weight-asian" "bold" .
883+ Map.insert "style:font-weight-complex" "bold" $ m
884+ | Under <- s = Map.insert "style:text-underline-style" "solid" .
885+ Map.insert "style:text-underline-width" "auto" .
886+ Map.insert "style:text-underline-color" "font-color" $ m
887+ | Strike <- s = Map.insert "style:text-line-through-style" "solid" m
888+ | Sub <- s = Map.insert "style:text-position" "sub 58%" m
889+ | Sup <- s = Map.insert "style:text-position" "super 58%" m
890+ | SmallC <- s = Map.insert "fo:font-variant" "small-caps" m
891+ | Pre <- s = Map.insert "style:font-name" "Courier New" .
892+ Map.insert "style:font-name-asian" "Courier New" .
893+ Map.insert "style:font-name-complex" "Courier New" $ m
894+ | Language lang <- s
895+ = Map.insert "fo:language" (langLanguage lang) .
896+ Map.insert "fo:country" (langRegion lang) $ m
897+ | otherwise = m
894898
895899withLangFromAttr :: PandocMonad m => Attr -> OD m a -> OD m a
896900withLangFromAttr (_,_,kvs) action =
897901 case lookup "lang" kvs of
898902 Nothing -> action
899903 Just l ->
900− case parseLang l of
904+ case parseBCP47 l of
901905 Right lang -> withTextStyle (Language lang) action
902906 Left _ -> do
903907 report $ InvalidLang l
src/Text/Pandoc/Writers/Org.hs+14−24
@@ -17,9 +17,8 @@ Org-Mode: <http://orgmode.org>
1717-}
1818module Text.Pandoc.Writers.Org (writeOrg) where
1919import Control.Monad.State.Strict
20−import Data.Char (isAlphaNum, isDigit)
20+import Data.Char (isAlphaNum)
2121import Data.List (intersect, intersperse, partition, transpose)
22−import Data.List.NonEmpty (nonEmpty)
2322import Data.Text (Text)
2423import qualified Data.Text as T
2524import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
@@ -84,15 +83,12 @@ noteToOrg num note = do
8483
8584-- | Escape special characters for Org.
8685escapeString :: Text -> Text
87−escapeString t
88− | T.all (\c -> c < '\x2013' || c > '\x2026') t = t
89− | otherwise = T.concatMap escChar t
90− where
91− escChar '\x2013' = "--"
92− escChar '\x2014' = "---"
93− escChar '\x2019' = "'"
94− escChar '\x2026' = "..."
95− escChar c = T.singleton c
86+escapeString = escapeStringUsing
87+ [ ('\x2014',"---")
88+ , ('\x2013',"--")
89+ , ('\x2019',"'")
90+ , ('\x2026',"...")
91+ ]
9692
9793isRawFormat :: Format -> Bool
9894isRawFormat f =
@@ -167,7 +163,7 @@ blockToOrg (Table _ blkCapt specs thead tbody tfoot) = do
167163 else "#+caption: " <> caption''
168164 headers' <- mapM blockListToOrg headers
169165 rawRows <- mapM (mapM blockListToOrg) rows
170− let numChars = maybe 0 maximum . nonEmpty . map offset
166+ let numChars = maximum . map offset
171167 -- FIXME: width is not being used.
172168 let widthsInChars =
173169 map numChars $ transpose (headers' : rawRows)
@@ -202,7 +198,7 @@ blockToOrg (OrderedList (start, _, delim) items) = do
202198 x -> x
203199 let markers = take (length items) $ orderedListMarkers
204200 (start, Decimal, delim')
205− let maxMarkerLength = maybe 0 maximum . nonEmpty $ map T.length markers
201+ let maxMarkerLength = maximum $ map T.length markers
206202 let markers' = map (\m -> let s = maxMarkerLength - T.length m
207203 in m <> T.replicate s " ") markers
208204 contents <- zipWithM orderedListItemToOrg markers' items
@@ -351,20 +347,16 @@ inlineListToOrg :: PandocMonad m
351347 => [Inline]
352348 -> Org m (Doc Text)
353349inlineListToOrg lst = hcat <$> mapM inlineToOrg (fixMarkers lst)
354− where -- Prevent note refs and list markers from wrapping, see #4171
355− -- and #7132.
356− fixMarkers [] = []
350+ where fixMarkers [] = [] -- prevent note refs and list markers from wrapping, see #4171
357351 fixMarkers (Space : x : rest) | shouldFix x =
358352 Str " " : x : fixMarkers rest
359353 fixMarkers (SoftBreak : x : rest) | shouldFix x =
360354 Str " " : x : fixMarkers rest
361355 fixMarkers (x : rest) = x : fixMarkers rest
362356
363− shouldFix Note{} = True -- Prevent footnotes
357+ shouldFix Note{} = True -- Prevent footnotes
364358 shouldFix (Str "-") = True -- Prevent bullet list items
365− shouldFix (Str x) -- Prevent ordered list items
366− | Just (cs, c) <- T.unsnoc x = T.all isDigit cs &&
367− (c == '.' || c == ')')
359+ -- TODO: prevent ordered list items
368360 shouldFix _ = False
369361
370362-- | Convert Pandoc inline element to Org.
@@ -404,11 +396,9 @@ inlineToOrg (Str str) = return . literal $ escapeString str
404396inlineToOrg (Math t str) = do
405397 modify $ \st -> st{ stHasMath = True }
406398 return $ if t == InlineMath
407− then "\\(" <> literal str <> "\\)"
408− else "\\[" <> literal str <> "\\]"
399+ then "$" <> literal str <> "$"
400+ else "$$" <> literal str <> "$$"
409401inlineToOrg il@(RawInline f str)
410− | elem f ["tex", "latex"] && T.isPrefixOf "\\begin" str =
411− return $ cr <> literal str <> cr
412402 | isRawFormat f = return $ literal str
413403 | otherwise = do
414404 report $ InlineNotRendered il
src/Text/Pandoc/Writers/Powerpoint.hs+1−1
@@ -39,5 +39,5 @@ writePowerpoint opts (Pandoc meta blks) = do
3939 let blks' = walk fixDisplayMath blks
4040 let (pres, logMsgs) = documentToPresentation opts (Pandoc meta blks')
4141 mapM_ report logMsgs
42− archv <- presentationToArchive opts meta pres
42+ archv <- presentationToArchive opts pres
4343 return $ fromArchive archv
src/Text/Pandoc/Writers/Powerpoint/Output.hs+31−43
@@ -38,21 +38,18 @@ import qualified Text.Pandoc.Class.PandocMonad as P
3838import Text.Pandoc.Options
3939import Text.Pandoc.MIME
4040import qualified Data.ByteString.Lazy as BL
41−import Text.Pandoc.Writers.Shared (metaToContext)
4241import Text.Pandoc.Writers.OOXML
4342import qualified Data.Map as M
4443import Data.Maybe (mapMaybe, listToMaybe, fromMaybe, maybeToList, catMaybes, isJust)
4544import Text.Pandoc.ImageSize
4645import Control.Applicative ((<|>))
4746import System.FilePath.Glob
48−import Text.DocTemplates (FromContext(lookupContext), Context)
49−import Text.DocLayout (literal)
47+import Text.DocTemplates (FromContext(lookupContext))
5048import Text.TeXMath
5149import Text.Pandoc.Writers.Math (convertMath)
5250import Text.Pandoc.Writers.Powerpoint.Presentation
53−import Text.Pandoc.Shared (tshow, stringify)
51+import Text.Pandoc.Shared (tshow)
5452import Skylighting (fromColor)
55−import Data.List.NonEmpty (nonEmpty)
5653
5754-- |The 'EMU' type is used to specify sizes in English Metric Units.
5855type EMU = Integer
@@ -99,7 +96,6 @@ data WriterEnv = WriterEnv { envRefArchive :: Archive
9996 , envDistArchive :: Archive
10097 , envUTCTime :: UTCTime
10198 , envOpts :: WriterOptions
102− , envContext :: Context Text
10399 , envPresentationSize :: (Integer, Integer)
104100 , envSlideHasHeader :: Bool
105101 , envInList :: Bool
@@ -125,7 +121,6 @@ instance Default WriterEnv where
125121 , envDistArchive = emptyArchive
126122 , envUTCTime = posixSecondsToUTCTime 0
127123 , envOpts = def
128− , envContext = mempty
129124 , envPresentationSize = (720, 540)
130125 , envSlideHasHeader = False
131126 , envInList = False
@@ -172,7 +167,7 @@ runP env st p = evalStateT (runReaderT p env) st
172167
173168monospaceFont :: Monad m => P m T.Text
174169monospaceFont = do
175− vars <- asks envContext
170+ vars <- writerVariables <$> asks envOpts
176171 case lookupContext "monofont" vars of
177172 Just s -> return s
178173 Nothing -> return "Courier"
@@ -308,9 +303,8 @@ makeSpeakerNotesMap (Presentation _ slides) =
308303 then Nothing
309304 else Just n
310305
311−presentationToArchive :: PandocMonad m
312− => WriterOptions -> Meta -> Presentation -> m Archive
313−presentationToArchive opts meta pres = do
306+presentationToArchive :: PandocMonad m => WriterOptions -> Presentation -> m Archive
307+presentationToArchive opts pres = do
314308 distArchive <- toArchive . BL.fromStrict <$>
315309 P.readDefaultDataFile "reference.pptx"
316310 refArchive <- case writerReferenceDoc opts of
@@ -326,18 +320,10 @@ presentationToArchive opts meta pres = do
326320 PandocSomeError
327321 "Could not determine presentation size"
328322
329− -- note, we need writerTemplate to be Just _ or metaToContext does
330− -- nothing
331− context <- metaToContext opts{ writerTemplate =
332− writerTemplate opts <|> Just mempty }
333− (return . literal . stringify)
334− (return . literal . stringify) meta
335−
336323 let env = def { envRefArchive = refArchive
337324 , envDistArchive = distArchive
338325 , envUTCTime = utctime
339326 , envOpts = opts
340− , envContext = context
341327 , envPresentationSize = presSize
342328 , envSlideIdMap = makeSlideIdMap pres
343329 , envSpeakerNotesIdMap = makeSpeakerNotesMap pres
@@ -488,16 +474,15 @@ registerLink link = do
488474 linkReg <- gets stLinkIds
489475 mediaReg <- gets stMediaIds
490476 hasSpeakerNotes <- curSlideHasSpeakerNotes
491− let maxLinkId = case M.lookup curSlideId linkReg >>= nonEmpty . M.keys of
492− Just xs -> maximum xs
493− Nothing
494− | hasSpeakerNotes -> 2
495− | otherwise -> 1
496− maxMediaId = case M.lookup curSlideId mediaReg >>= nonEmpty of
497− Just mInfos -> maximum $ fmap mInfoLocalId mInfos
498− Nothing
499− | hasSpeakerNotes -> 2
500− | otherwise -> 1
477+ let maxLinkId = case M.lookup curSlideId linkReg of
478+ Just mp -> case M.keys mp of
479+ [] -> if hasSpeakerNotes then 2 else 1
480+ ks -> maximum ks
481+ Nothing -> if hasSpeakerNotes then 2 else 1
482+ maxMediaId = case M.lookup curSlideId mediaReg of
483+ Just [] -> if hasSpeakerNotes then 2 else 1
484+ Just mInfos -> maximum $ map mInfoLocalId mInfos
485+ Nothing -> if hasSpeakerNotes then 2 else 1
501486 maxId = max maxLinkId maxMediaId
502487 slideLinks = case M.lookup curSlideId linkReg of
503488 Just mp -> M.insert (maxId + 1) link mp
@@ -512,19 +497,20 @@ registerMedia fp caption = do
512497 mediaReg <- gets stMediaIds
513498 globalIds <- gets stMediaGlobalIds
514499 hasSpeakerNotes <- curSlideHasSpeakerNotes
515− let maxLinkId = case M.lookup curSlideId linkReg >>= nonEmpty . M.keys of
516− Just ks -> maximum ks
517− Nothing
518− | hasSpeakerNotes -> 2
519− | otherwise -> 1
520− maxMediaId = case M.lookup curSlideId mediaReg >>= nonEmpty of
521− Just mInfos -> maximum $ fmap mInfoLocalId mInfos
522− Nothing
523− | hasSpeakerNotes -> 2
524− | otherwise -> 1
500+ let maxLinkId = case M.lookup curSlideId linkReg of
501+ Just mp -> case M.keys mp of
502+ [] -> if hasSpeakerNotes then 2 else 1
503+ ks -> maximum ks
504+ Nothing -> if hasSpeakerNotes then 2 else 1
505+ maxMediaId = case M.lookup curSlideId mediaReg of
506+ Just [] -> if hasSpeakerNotes then 2 else 1
507+ Just mInfos -> maximum $ map mInfoLocalId mInfos
508+ Nothing -> if hasSpeakerNotes then 2 else 1
525509 maxLocalId = max maxLinkId maxMediaId
526510
527− maxGlobalId = maybe 0 maximum $ nonEmpty $ M.elems globalIds
511+ maxGlobalId = case M.elems globalIds of
512+ [] -> 0
513+ ids -> maximum ids
528514
529515 (imgBytes, mbMt) <- P.fetchItem $ T.pack fp
530516 let imgExt = (mbMt >>= extensionFromMimeType >>= (\x -> return $ "." <> x))
@@ -537,7 +523,6 @@ registerMedia fp caption = do
537523 Just Eps -> Just ".eps"
538524 Just Svg -> Just ".svg"
539525 Just Emf -> Just ".emf"
540− Just Tiff -> Just ".tiff"
541526 Nothing -> Nothing
542527
543528 let newGlobalId = fromMaybe (maxGlobalId + 1) (M.lookup fp globalIds)
@@ -1442,8 +1427,11 @@ presentationToRels pres@(Presentation _ slides) = do
14421427 -- all relWithoutSlide rels (unless they're 1)
14431428 -- 3. If we have a notesmaster slide, we make space for that as well.
14441429
1445− let minRelNotOne = maybe 0 minimum $ nonEmpty
1446− $ filter (1 <) $ map relId relsWeKeep
1430+ let minRelNotOne = case filter (1<) $ map relId relsWeKeep of
1431+ [] -> 0 -- doesn't matter in this case, since
1432+ -- there will be nothing to map the
1433+ -- function over
1434+ l -> minimum l
14471435
14481436 modifyRelNum :: Int -> Int
14491437 modifyRelNum 1 = 1
src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+3−2
@@ -45,7 +45,6 @@ module Text.Pandoc.Writers.Powerpoint.Presentation ( documentToPresentation
4545import Control.Monad.Reader
4646import Control.Monad.State
4747import Data.List (intercalate)
48−import Data.List.NonEmpty (nonEmpty)
4948import Data.Default
5049import Text.Pandoc.Definition
5150import Text.Pandoc.ImageSize
@@ -364,7 +363,9 @@ inlineToParElems (Note blks) = do
364363 then return []
365364 else do
366365 notes <- gets stNoteIds
367− let maxNoteId = maybe 0 maximum $ nonEmpty $ M.keys notes
366+ let maxNoteId = case M.keys notes of
367+ [] -> 0
368+ lst -> maximum lst
368369 curNoteId = maxNoteId + 1
369370 modify $ \st -> st { stNoteIds = M.insert curNoteId blks notes }
370371 local (\env -> env{envRunProps = (envRunProps env){rLink = Just $ InternalTarget endNotesSlideId}}) $
src/Text/Pandoc/Writers/RST.hs+10−14
@@ -16,8 +16,7 @@ reStructuredText: <http://docutils.sourceforge.net/rst.html>
1616module Text.Pandoc.Writers.RST ( writeRST, flatten ) where
1717import Control.Monad.State.Strict
1818import Data.Char (isSpace)
19−import Data.List (transpose, intersperse, foldl')
20−import qualified Data.List.NonEmpty as NE
19+import Data.List (transpose, intersperse)
2120import Data.Maybe (fromMaybe)
2221import qualified Data.Text as T
2322import Data.Text (Text)
@@ -336,7 +335,7 @@ blockToRST (OrderedList (start, style', delim) items) = do
336335 then replicate (length items) "#."
337336 else take (length items) $ orderedListMarkers
338337 (start, style', delim)
339− let maxMarkerLength = maybe 0 maximum $ NE.nonEmpty $ map T.length markers
338+ let maxMarkerLength = maximum $ map T.length markers
340339 let markers' = map (\m -> let s = maxMarkerLength - T.length m
341340 in m <> T.replicate s " ") markers
342341 contents <- zipWithM orderedListItemToRST markers' items
@@ -510,7 +509,7 @@ flatten outer
510509 | null contents = [outer]
511510 | otherwise = combineAll contents
512511 where contents = dropInlineParent outer
513− combineAll = foldl' combine []
512+ combineAll = foldl combine []
514513
515514 combine :: [Inline] -> Inline -> [Inline]
516515 combine f i =
@@ -538,15 +537,11 @@ flatten outer
538537 collapse f i = appendToLast f $ dropInlineParent i
539538
540539 appendToLast :: [Inline] -> [Inline] -> [Inline]
541− appendToLast flattened toAppend =
542− case NE.nonEmpty flattened of
543− Nothing -> [setInlineChildren outer toAppend]
544− Just xs ->
545− if isOuter lastFlat
546− then NE.init xs <> [appendTo lastFlat toAppend]
547− else flattened <> [setInlineChildren outer toAppend]
548− where
549− lastFlat = NE.last xs
540+ appendToLast [] toAppend = [setInlineChildren outer toAppend]
541+ appendToLast flattened toAppend
542+ | isOuter lastFlat = init flattened <> [appendTo lastFlat toAppend]
543+ | otherwise = flattened <> [setInlineChildren outer toAppend]
544+ where lastFlat = last flattened
550545 appendTo o i = mapNested (<> i) o
551546 isOuter i = emptyParent i == emptyParent outer
552547 emptyParent i = setInlineChildren i []
@@ -766,7 +761,8 @@ simpleTable opts blocksToDoc headers rows = do
766761 then return []
767762 else fixEmpties <$> mapM (blocksToDoc opts) headers
768763 rowDocs <- mapM (fmap fixEmpties . mapM (blocksToDoc opts)) rows
769− let numChars = maybe 0 maximum . NE.nonEmpty . map offset
764+ let numChars [] = 0
765+ numChars xs = maximum . map offset $ xs
770766 let colWidths = map numChars $ transpose (headerDocs : rowDocs)
771767 let toRow = mconcat . intersperse (lblock 1 " ") . zipWith lblock colWidths
772768 let hline = nowrap $ hsep (map (\n -> literal (T.replicate n "=")) colWidths)
src/Text/Pandoc/Writers/RTF.hs+10−16
@@ -16,7 +16,7 @@ module Text.Pandoc.Writers.RTF ( writeRTF
1616import Control.Monad.Except (catchError, throwError)
1717import Control.Monad
1818import qualified Data.ByteString as B
19−import Data.Char (chr, isDigit, ord, isAlphaNum)
19+import Data.Char (chr, isDigit, ord)
2020import qualified Data.Map as M
2121import Data.Text (Text)
2222import qualified Data.Text as T
@@ -137,21 +137,15 @@ handleUnicode = T.concatMap $ \c ->
137137
138138-- | Escape special characters.
139139escapeSpecial :: Text -> Text
140−escapeSpecial t
141− | T.all isAlphaNum t = t
142− | otherwise = T.concatMap escChar t
143− where
144− escChar '\t' = "\\tab "
145− escChar '\8216' = "\\u8216'"
146− escChar '\8217' = "\\u8217'"
147− escChar '\8220' = "\\u8220\""
148− escChar '\8221' = "\\u8221\""
149− escChar '\8211' = "\\u8211-"
150− escChar '\8212' = "\\u8212-"
151− escChar '{' = "\\{"
152− escChar '}' = "\\}"
153− escChar '\\' = "\\\\"
154− escChar c = T.singleton c
140+escapeSpecial = escapeStringUsing $
141+ [ ('\t',"\\tab ")
142+ , ('\8216',"\\u8216'")
143+ , ('\8217',"\\u8217'")
144+ , ('\8220',"\\u8220\"")
145+ , ('\8221',"\\u8221\"")
146+ , ('\8211',"\\u8211-")
147+ , ('\8212',"\\u8212-")
148+ ] <> backslashEscapes "{\\}"
155149
156150-- | Escape strings as needed for rich text format.
157151stringToRTF :: Text -> Text
src/Text/Pandoc/Writers/Shared.hs+3−18
@@ -20,7 +20,6 @@ module Text.Pandoc.Writers.Shared (
2020 , setField
2121 , resetField
2222 , defField
23− , getLang
2423 , tagWithAttrs
2524 , isDisplayMath
2625 , fixDisplayMath
@@ -45,7 +44,6 @@ import Control.Monad (zipWithM)
4544import Data.Aeson (ToJSON (..), encode)
4645import Data.Char (chr, ord, isSpace)
4746import Data.List (groupBy, intersperse, transpose, foldl')
48−import Data.List.NonEmpty (NonEmpty(..), nonEmpty)
4947import Data.Text.Conversions (FromText(..))
5048import qualified Data.Map as M
5149import qualified Data.Text as T
@@ -148,19 +146,6 @@ defField field val (Context m) =
148146 where
149147 f _newval oldval = oldval
150148
151−-- | Get the contents of the `lang` metadata field or variable.
152−getLang :: WriterOptions -> Meta -> Maybe T.Text
153−getLang opts meta =
154− case lookupContext "lang" (writerVariables opts) of
155− Just s -> Just s
156− _ ->
157− case lookupMeta "lang" meta of
158− Just (MetaBlocks [Para [Str s]]) -> Just s
159− Just (MetaBlocks [Plain [Str s]]) -> Just s
160− Just (MetaInlines [Str s]) -> Just s
161− Just (MetaString s) -> Just s
162− _ -> Nothing
163−
164149-- | Produce an HTML tag with the given pandoc attributes.
165150tagWithAttrs :: HasChars a => T.Text -> Attr -> Doc a
166151tagWithAttrs tag (ident,classes,kvs) = hsep
@@ -239,7 +224,7 @@ gridTable :: (Monad m, HasChars a)
239224 -> m (Doc a)
240225gridTable opts blocksToDoc headless aligns widths headers rows = do
241226 -- the number of columns will be used in case of even widths
242− let numcols = maximum (length aligns :| length widths :
227+ let numcols = maximum (length aligns : length widths :
243228 map length (headers:rows))
244229 let officialWidthsInChars widths' = map (
245230 (\x -> if x < 1 then 1 else x) .
@@ -268,7 +253,8 @@ gridTable opts blocksToDoc headless aligns widths headers rows = do
268253 let handleFullWidths widths' = do
269254 rawHeaders' <- mapM (blocksToDoc opts) headers
270255 rawRows' <- mapM (mapM (blocksToDoc opts)) rows
271− let numChars = maybe 0 maximum . nonEmpty . map offset
256+ let numChars [] = 0
257+ numChars xs = maximum . map offset $ xs
272258 let minWidthsInChars =
273259 map numChars $ transpose (rawHeaders' : rawRows')
274260 let widthsInChars' = zipWith max
@@ -394,7 +380,6 @@ toSuperscript '2' = Just '\x00B2'
394380toSuperscript '3' = Just '\x00B3'
395381toSuperscript '+' = Just '\x207A'
396382toSuperscript '-' = Just '\x207B'
397−toSuperscript '\x2212' = Just '\x207B' -- unicode minus
398383toSuperscript '=' = Just '\x207C'
399384toSuperscript '(' = Just '\x207D'
400385toSuperscript ')' = Just '\x207E'
src/Text/Pandoc/Writers/TEI.hs+3−4
@@ -146,17 +146,16 @@ blockToTEI opts (LineBlock lns) =
146146 blockToTEI opts $ linesToPara lns
147147blockToTEI opts (BlockQuote blocks) =
148148 inTagsIndented "quote" <$> blocksToTEI opts blocks
149−blockToTEI opts (CodeBlock (_,classes,_) str) =
149+blockToTEI _ (CodeBlock (_,classes,_) str) =
150150 return $ literal ("<ab type='codeblock " <> lang <> "'>") <> cr <>
151151 flush (literal (escapeStringForXML str) <> cr <> text "</ab>")
152152 where lang = if null langs
153153 then ""
154154 else escapeStringForXML (head langs)
155− syntaxMap = writerSyntaxMap opts
156− isLang l = T.toLower l `elem` map T.toLower (languages syntaxMap)
155+ isLang l = T.toLower l `elem` map T.toLower languages
157156 langsFrom s = if isLang s
158157 then [s]
159− else (languagesByExtension syntaxMap) . T.toLower $ s
158+ else languagesByExtension . T.toLower $ s
160159 langs = concatMap langsFrom classes
161160blockToTEI opts (BulletList lst) = do
162161 let attribs = [("type", "unordered")]
src/Text/Pandoc/Writers/Texinfo.hs+15−22
@@ -14,9 +14,8 @@ Conversion of 'Pandoc' format into Texinfo.
1414module Text.Pandoc.Writers.Texinfo ( writeTexinfo ) where
1515import Control.Monad.Except (throwError)
1616import Control.Monad.State.Strict
17−import Data.Char (chr, ord, isAlphaNum)
18−import Data.List (maximumBy, transpose, foldl')
19−import Data.List.NonEmpty (nonEmpty)
17+import Data.Char (chr, ord)
18+import Data.List (maximumBy, transpose)
2019import Data.Ord (comparing)
2120import qualified Data.Set as Set
2221import Data.Text (Text)
@@ -85,18 +84,16 @@ pandocToTexinfo options (Pandoc meta blocks) = do
8584
8685-- | Escape things as needed for Texinfo.
8786stringToTexinfo :: Text -> Text
88−stringToTexinfo t
89− | T.all isAlphaNum t = t
90− | otherwise = T.concatMap escChar t
91− where escChar '{' = "@{"
92− escChar '}' = "@}"
93− escChar '@' = "@@"
94− escChar '\160' = "@ "
95− escChar '\x2014' = "---"
96− escChar '\x2013' = "--"
97− escChar '\x2026' = "@dots{}"
98− escChar '\x2019' = "'"
99− escChar c = T.singleton c
87+stringToTexinfo = escapeStringUsing texinfoEscapes
88+ where texinfoEscapes = [ ('{', "@{")
89+ , ('}', "@}")
90+ , ('@', "@@")
91+ , ('\160', "@ ")
92+ , ('\x2014', "---")
93+ , ('\x2013', "--")
94+ , ('\x2026', "@dots{}")
95+ , ('\x2019', "'")
96+ ]
10097
10198escapeCommas :: PandocMonad m => TI m (Doc Text) -> TI m (Doc Text)
10299escapeCommas parser = do
@@ -241,13 +238,9 @@ blockToTexinfo (Table _ blkCapt specs thead tbody tfoot) = do
241238 colDescriptors <-
242239 if all (== 0) widths
243240 then do -- use longest entry instead of column widths
244− cols <- mapM (mapM (fmap (T.unpack . render Nothing . hcat) .
245− mapM blockToTexinfo)) $
241+ cols <- mapM (mapM (liftM (T.unpack . render Nothing . hcat) . mapM blockToTexinfo)) $
246242 transpose $ heads : rows
247− return $ concatMap
248− ((\x -> "{"++x++"} ") .
249− maybe "" (maximumBy (comparing length)) . nonEmpty)
250− cols
243+ return $ concatMap ((\x -> "{"++x++"} ") . maximumBy (comparing length)) cols
251244 else return $ "@columnfractions " ++ concatMap (printf "%.2f ") widths
252245 let tableBody = text ("@multitable " ++ colDescriptors) $$
253246 headers $$
@@ -278,7 +271,7 @@ tableAnyRowToTexinfo :: PandocMonad m
278271 -> [[Block]]
279272 -> TI m (Doc Text)
280273tableAnyRowToTexinfo itemtype aligns cols =
281− (literal itemtype $$) . foldl' (\row item -> row $$
274+ (literal itemtype $$) . foldl (\row item -> row $$
282275 (if isEmpty row then empty else text " @tab ") <> item) empty <$> zipWithM alignedBlock aligns cols
283276
284277alignedBlock :: PandocMonad m
src/Text/Pandoc/Writers/ZimWiki.hs+2−4
@@ -20,7 +20,6 @@ import Control.Monad (zipWithM)
2020import Control.Monad.State.Strict (StateT, evalStateT, gets, modify)
2121import Data.Default (Default (..))
2222import Data.List (transpose)
23−import Data.List.NonEmpty (nonEmpty)
2423import qualified Data.Map as Map
2524import Text.DocLayout (render, literal)
2625import Data.Maybe (fromMaybe)
@@ -116,7 +115,7 @@ blockToZimWiki opts b@(RawBlock f str)
116115blockToZimWiki _ HorizontalRule = return "\n----\n"
117116
118117blockToZimWiki opts (Header level _ inlines) = do
119− contents <- inlineListToZimWiki opts inlines
118+ contents <- inlineListToZimWiki opts $ removeFormatting inlines -- emphasis, links etc. not allowed in headers
120119 let eqs = T.replicate ( 7 - level ) "="
121120 return $ eqs <> " " <> contents <> " " <> eqs <> "\n"
122121
@@ -144,8 +143,7 @@ blockToZimWiki opts (Table _ blkCapt specs thead tbody tfoot) = do
144143 then zipWithM (tableItemToZimWiki opts) aligns (head rows)
145144 else mapM (inlineListToZimWiki opts . removeFormatting)headers -- emphasis, links etc. are not allowed in table headers
146145 rows' <- mapM (zipWithM (tableItemToZimWiki opts) aligns) rows
147− let widths = map (maybe 0 maximum . nonEmpty . map T.length) $
148− transpose (headers':rows')
146+ let widths = map (maximum . map T.length) $ transpose (headers':rows')
149147 let padTo (width, al) s =
150148 case width - T.length s of
151149 x | x > 0 ->
src/Text/Pandoc/XML.hs+2−28
@@ -13,7 +13,6 @@ Functions for escaping and formatting XML.
1313-}
1414module Text.Pandoc.XML ( escapeCharForXML,
1515 escapeStringForXML,
16− escapeNCName,
1716 inTags,
1817 selfClosingTag,
1918 inTagsSimple,
@@ -25,7 +24,7 @@ module Text.Pandoc.XML ( escapeCharForXML,
2524 html5Attributes,
2625 rdfaAttributes ) where
2726
28−import Data.Char (isAscii, isSpace, ord, isLetter, isDigit)
27+import Data.Char (isAscii, isSpace, ord)
2928import Data.Text (Text)
3029import qualified Data.Text as T
3130import Text.HTML.TagSoup.Entity (lookupEntity, htmlEntities)
@@ -120,33 +119,8 @@ html5EntityMap = foldr go mempty htmlEntities
120119 where ent' = T.takeWhile (/=';') (T.pack ent)
121120 _ -> entmap
122121
123−-- | Converts a string into an NCName, i.e., an XML name without colons.
124−-- Disallowed characters are escaped using @ux%x@, where @%x@ is the
125−-- hexadecimal unicode identifier of the escaped character.
126−escapeNCName :: Text -> Text
127−escapeNCName t = case T.uncons t of
128− Nothing -> T.empty
129− Just (c, cs) -> escapeStartChar c <> T.concatMap escapeNCNameChar cs
130− where
131− escapeStartChar :: Char -> Text
132− escapeStartChar c = if isLetter c || c == '_'
133− then T.singleton c
134− else escapeChar c
135122
136− escapeNCNameChar :: Char -> Text
137− escapeNCNameChar c = if isNCNameChar c
138− then T.singleton c
139− else escapeChar c
140−
141− isNCNameChar :: Char -> Bool
142− isNCNameChar c = isLetter c || c `elem` ("_-.·" :: String) || isDigit c
143− || '\x0300' <= c && c <= '\x036f'
144− || '\x203f' <= c && c <= '\x2040'
145−
146− escapeChar :: Char -> Text
147− escapeChar = T.pack . printf "U%04X" . ord
148−
149−-- | Unescapes XML entities
123+-- Unescapes XML entities
150124fromEntities :: Text -> Text
151125fromEntities t
152126 = let (x, y) = T.break (== '&') t
src/Text/Pandoc/XML/Light/Proc.hs+1
@@ -1,4 +1,5 @@
11{-# LANGUAGE FlexibleInstances #-}
2+{-# LANGUAGE OverloadedStrings #-}
23{- |
34 Module : Text.Pandoc.XML.Light.Proc
45 Copyright : Copyright (C) 2007 Galois, Inc., 2021 John MacFarlane
src/Text/Pandoc/XML/Light/Types.hs+4−4
@@ -53,7 +53,7 @@ type Line = Integer
5353data Content = Elem Element
5454 | Text CData
5555 | CRef Text
56− deriving (Show, Typeable, Data, Ord, Eq)
56+ deriving (Show, Typeable, Data)
5757
5858-- | XML elements
5959data Element = Element {
@@ -61,7 +61,7 @@ data Element = Element {
6161 elAttribs :: [Attr],
6262 elContent :: [Content],
6363 elLine :: Maybe Line
64− } deriving (Show, Typeable, Data, Ord, Eq)
64+ } deriving (Show, Typeable, Data)
6565
6666-- | XML attributes
6767data Attr = Attr {
@@ -74,13 +74,13 @@ data CData = CData {
7474 cdVerbatim :: CDataKind,
7575 cdData :: Text,
7676 cdLine :: Maybe Line
77− } deriving (Show, Typeable, Data, Ord, Eq)
77+ } deriving (Show, Typeable, Data)
7878
7979data CDataKind
8080 = CDataText -- ^ Ordinary character data; pretty printer escapes &, < etc.
8181 | CDataVerbatim -- ^ Unescaped character data; pretty printer embeds it in <![CDATA[..
8282 | CDataRaw -- ^ As-is character data; pretty printer passes it along without any escaping or CDATA wrap-up.
83− deriving ( Eq, Ord, Show, Typeable, Data )
83+ deriving ( Eq, Show, Typeable, Data )
8484
8585-- | XML qualified names
8686data QName = QName {
stack.yaml+6−11
@@ -2,21 +2,16 @@ flags:
22 pandoc:
33 trypandoc: false
44 embed_data_files: true
5− QuickCheck:
6− old-random: false
75packages:
86- '.'
97extra-deps:
10−- commonmark-0.2.1
11−- commonmark-pandoc-0.2.1.1
12−- skylighting-core-0.10.5.2
13−- skylighting-0.10.5.2
14−- git: https://github.com/jgm/doctemplates
15− commit: 428c26d5303cf7a2b1051fe1ffd9aafe9ba71c81
16−- git: https://github.com/jgm/citeproc
17− commit: 4ce0501cd6f9c86eee087afcf878c256e49b9615
8+- hslua-1.3.0
9+- hslua-module-path-0.1.0
10+#- citeproc:
11+# git: https://github.com/jgm/citeproc.git
12+# commit: feb3b7580c6738eec3b23921f7c1739cfba611aa
1813ghc-options:
1914 "$locals": -fhide-source-paths -Wno-missing-home-modules
20−resolver: lts-18.0
15+resolver: lts-17.5
2116nix:
2217 packages: [zlib]
test/Tests/Command.hs+14−7
@@ -1,3 +1,4 @@
1+{-# LANGUAGE TupleSections #-}
12{- |
23 Module : Tests.Command
34 Copyright : © 2006-2021 John MacFarlane
@@ -13,13 +14,14 @@ module Tests.Command (runTest, tests)
1314where
1415
1516import Data.Algorithm.Diff
16−import System.Environment (getExecutablePath)
17+import System.Environment.Executable (getExecutablePath)
1718import qualified Data.ByteString as BS
1819import qualified Data.Text as T
1920import Data.List (isSuffixOf)
2021import System.Directory
22+import qualified System.Environment as Env
2123import System.Exit
22−import System.FilePath ((</>))
24+import System.FilePath (takeDirectory, (</>))
2325import System.IO (hPutStr, stderr)
2426import System.IO.Unsafe (unsafePerformIO)
2527import System.Process
@@ -36,7 +38,15 @@ execTest :: String -- ^ Path to test executable
3638 -> String -- ^ Input text
3739 -> IO (ExitCode, String) -- ^ Exit code and actual output
3840execTest testExePath cmd inp = do
39− env' <- setupEnvironment testExePath
41+ mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
42+ mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
43+ let env' = ("PATH",takeDirectory testExePath) :
44+ ("TMP",".") :
45+ ("LANG","en_US.UTF-8") :
46+ ("HOME", "./") :
47+ ("pandoc_datadir", "..") :
48+ maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
49+ maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
4050 let pr = (shell (pandocToEmulate True cmd)){ env = Just env' }
4151 (ec, out', err') <- readCreateProcessWithExitCode pr inp
4252 -- filter \r so the tests will work on Windows machines
@@ -130,7 +140,4 @@ extractCommandTest testExePath fp = unsafePerformIO $ do
130140 def{ readerExtensions = pandocExtensions } contents)
131141 let codeblocks = map extractCode $ filter isCodeBlock blocks
132142 let cases = zipWith (runCommandTest testExePath fp) [1..] codeblocks
133− return $ testGroup fp
134− $ if null cases
135− then [testCase "!!" $ assertFailure "No command tests defined"]
136− else cases
143+ return $ testGroup fp cases
test/Tests/Helpers.hs−23
@@ -1,5 +1,4 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TupleSections #-}
32{-# LANGUAGE FlexibleInstances #-}
43{- |
54 Module : Tests.Helpers
@@ -14,7 +13,6 @@ Utility functions for the test suite.
1413-}
1514module Tests.Helpers ( test
1615 , TestResult(..)
17− , setupEnvironment
1816 , showDiff
1917 , (=?>)
2018 , purely
@@ -27,8 +25,6 @@ import Data.Algorithm.Diff
2725import qualified Data.Map as M
2826import Data.Text (Text, unpack)
2927import System.Exit
30−import System.FilePath (takeDirectory)
31−import qualified System.Environment as Env
3228import Test.Tasty
3329import Test.Tasty.HUnit
3430import Text.Pandoc.Builder (Blocks, Inlines, doc, plain)
@@ -61,25 +57,6 @@ test fn name (input, expected) =
6157 dashes "" = replicate 72 '-'
6258 dashes x = replicate (72 - length x - 5) '-' ++ " " ++ x ++ " ---"
6359
64−-- | Set up environment for pandoc command tests.
65−setupEnvironment :: FilePath -> IO [(String, String)]
66−setupEnvironment testExePath = do
67− mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
68− mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
69− mpdd <- Env.lookupEnv "pandoc_datadir"
70− -- Note that Cabal sets the pandoc_datadir environment variable
71− -- to point to the source directory, since otherwise getDataFilename
72− -- will look in the data directory into which pandoc will be installed
73− -- (but has not yet been). So when we spawn a new process with
74− -- pandoc, we need to make sure this environment variable is set.
75− return $ ("PATH",takeDirectory testExePath) :
76− ("TMP",".") :
77− ("LANG","en_US.UTF-8") :
78− ("HOME", "./") :
79− maybe [] ((:[]) . ("pandoc_datadir",)) mpdd ++
80− maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
81− maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
82−
8360data TestResult = TestPassed
8461 | TestError ExitCode
8562 | TestFailed String FilePath [Diff String]
test/Tests/Old.hs+12−3
@@ -1,3 +1,4 @@
1+{-# LANGUAGE TupleSections #-}
12{- |
23 Module : Tests.Old
34 Copyright : © 2006-2021 John MacFarlane
@@ -14,7 +15,8 @@ module Tests.Old (tests) where
1415import Data.Algorithm.Diff
1516import System.Exit
1617import System.FilePath ((<.>), (</>))
17−import System.Environment (getExecutablePath)
18+import qualified System.Environment as Env
19+import System.Environment.Executable (getExecutablePath)
1820import Text.Pandoc.Process (pipeProcess)
1921import Test.Tasty (TestTree, testGroup)
2022import Test.Tasty.Golden.Advanced (goldenTest)
@@ -317,7 +319,14 @@ testWithNormalize normalizer pandocPath testname opts inp norm =
317319 (compareValues norm options) updateGolden
318320 where getExpected = normalizer <$> readFile' norm
319321 getActual = do
320− env <- setupEnvironment pandocPath
322+ mldpath <- Env.lookupEnv "LD_LIBRARY_PATH"
323+ mdyldpath <- Env.lookupEnv "DYLD_LIBRARY_PATH"
324+ let env = ("TMP",".") :
325+ ("LANG","en_US.UTF-8") :
326+ ("HOME", "./") :
327+ maybe [] ((:[]) . ("LD_LIBRARY_PATH",)) mldpath ++
328+ maybe [] ((:[]) . ("DYLD_LIBRARY_PATH",)) mdyldpath
329+
321330 (ec, out) <- pipeProcess (Just env) pandocPath
322331 ("--emulate":options) mempty
323332 if ec == ExitSuccess
@@ -326,7 +335,7 @@ testWithNormalize normalizer pandocPath testname opts inp norm =
326335 -- filter \r so the tests will work on Windows machines
327336 else fail $ "Pandoc failed with error code " ++ show ec
328337 updateGolden = UTF8.writeFile norm . T.pack
329− options = ["--quiet"] ++ [inp] ++ opts
338+ options = ["--data-dir=../data","--quiet"] ++ [inp] ++ opts
330339
331340compareValues :: FilePath -> [String] -> String -> String -> IO (Maybe String)
332341compareValues norm options expected actual = do
test/Tests/Readers/Docx.hs+5−25
@@ -24,7 +24,7 @@ import Test.Tasty.HUnit
2424import Tests.Helpers
2525import Text.Pandoc
2626import qualified Text.Pandoc.Class as P
27−import qualified Text.Pandoc.MediaBag as MB
27+import Text.Pandoc.MediaBag (MediaBag, lookupMedia, mediaDirectory)
2828import Text.Pandoc.UTF8 as UTF8
2929
3030-- We define a wrapper around pandoc that doesn't normalize in the
@@ -91,11 +91,11 @@ getMedia :: FilePath -> FilePath -> IO (Maybe B.ByteString)
9191getMedia archivePath mediaPath = fmap fromEntry . findEntryByPath
9292 ("word/" ++ mediaPath) . toArchive <$> B.readFile archivePath
9393
94−compareMediaPathIO :: FilePath -> MB.MediaBag -> FilePath -> IO Bool
94+compareMediaPathIO :: FilePath -> MediaBag -> FilePath -> IO Bool
9595compareMediaPathIO mediaPath mediaBag docxPath = do
9696 docxMedia <- getMedia docxPath mediaPath
97− let mbBS = case MB.lookupMedia mediaPath mediaBag of
98− Just item -> MB.mediaContents item
97+ let mbBS = case lookupMedia mediaPath mediaBag of
98+ Just (_, bs) -> bs
9999 Nothing -> error ("couldn't find " ++
100100 mediaPath ++
101101 " in media bag")
@@ -110,7 +110,7 @@ compareMediaBagIO docxFile = do
110110 mb <- runIOorExplode $ readDocx defopts df >> P.getMediaBag
111111 bools <- mapM
112112 (\(fp, _, _) -> compareMediaPathIO fp mb docxFile)
113− (MB.mediaDirectory mb)
113+ (mediaDirectory mb)
114114 return $ and bools
115115
116116testMediaBagIO :: String -> FilePath -> IO TestTree
@@ -155,10 +155,6 @@ tests = [ testGroup "document"
155155 "VML image"
156156 "docx/image_vml.docx"
157157 "docx/image_vml.native"
158− , testCompare
159− "VML image as object"
160− "docx/image_vml_as_object.docx"
161− "docx/image_vml_as_object.native"
162158 , testCompare
163159 "inline image in links"
164160 "docx/inline_images.docx"
@@ -317,30 +313,14 @@ tests = [ testGroup "document"
317313 "tables with lists in cells"
318314 "docx/table_with_list_cell.docx"
319315 "docx/table_with_list_cell.native"
320− , testCompare
321− "a table with a header which contains rowspans greater than 1"
322− "docx/table_header_rowspan.docx"
323− "docx/table_header_rowspan.native"
324316 , testCompare
325317 "tables with one row"
326318 "docx/table_one_row.docx"
327319 "docx/table_one_row.native"
328− , testCompare
329− "tables with just one row, which is a header"
330− "docx/table_one_header_row.docx"
331− "docx/table_one_header_row.native"
332320 , testCompare
333321 "tables with variable width"
334322 "docx/table_variable_width.docx"
335323 "docx/table_variable_width.native"
336− , testCompare
337− "tables with captions which contain a Table field"
338− "docx/table_captions_with_field.docx"
339− "docx/table_captions_with_field.native"
340− , testCompare
341− "tables with captions which don't contain a Table field"
342− "docx/table_captions_no_field.docx"
343− "docx/table_captions_no_field.native"
344324 , testCompare
345325 "code block"
346326 "docx/codeblock.docx"
test/Tests/Readers/EPUB.hs+1−1
@@ -47,7 +47,7 @@ featuresBag = [("img/check.gif","image/gif",1340)
4747
4848-- with additional meta tag for cover in EPUB2 format
4949epub3CoverBag :: [(String, String, Int)]
50−epub3CoverBag = [("wasteland-cover.jpg","image/jpeg", 16586)]
50+epub3CoverBag = [("wasteland-cover.jpg","image/jpeg",103477)]
5151
5252epub3NoCoverBag :: [(String, String, Int)]
5353epub3NoCoverBag = [("img/check.gif","image/gif",1340)
test/Tests/Readers/HTML.hs+5−20
@@ -99,26 +99,11 @@ tests = [ testGroup "base tag"
9999 plain (codeWith ("",["sample"],[]) "Answer is 42")
100100 ]
101101 , testGroup "var"
102− [ test html "inline var block" $
103− "<var>result</var>" =?>
104− plain (codeWith ("",["variable"],[]) "result")
105− ]
106− , testGroup "header"
107− [ test htmlNativeDivs "<header> is parsed as a div" $
108− "<header id=\"title\">Title</header>" =?>
109− divWith ("title", mempty, mempty) (plain "Title")
110− ]
111− , testGroup "code block"
112− [ test html "attributes in pre > code element" $
113− "<pre><code id=\"a\" class=\"python\">\nprint('hi')\n</code></pre>"
114− =?>
115− codeBlockWith ("a", ["python"], []) "print('hi')"
116−
117− , test html "attributes in pre take precendence" $
118− "<pre id=\"c\"><code id=\"d\">\nprint('hi mom!')\n</code></pre>"
119− =?>
120− codeBlockWith ("c", [], []) "print('hi mom!')"
121− ]
102+ [
103+ test html "inline var block" $
104+ "<var>result</var>" =?>
105+ plain (codeWith ("",["variable"],[]) "result")
106+ ]
122107 , askOption $ \(QuickCheckTests numtests) ->
123108 testProperty "Round trip" $
124109 withMaxSuccess (if QuickCheckTests numtests == defaultValue
test/Tests/Readers/Jira.hs−14
@@ -96,12 +96,6 @@ tests =
9696 simpleTable [para "Name"] [[para "Test"]]
9797 ]
9898
99− , testGroup "panel"
100− [ "simple panel" =:
101− "{panel}\nInterviewer: Jane Doe{panel}\n" =?>
102− divWith ("", ["panel"], []) (para "Interviewer: Jane Doe")
103− ]
104−
10599 , testGroup "inlines"
106100 [ "emphasis" =:
107101 "*quid pro quo*" =?>
@@ -167,14 +161,6 @@ tests =
167161 , "user with description" =:
168162 "[John Doe|~johndoe]" =?>
169163 para (linkWith ("", ["user-account"], []) "~johndoe" "" "John Doe")
170−
171− , "'smart' link" =:
172− "[x|http://example.com|smart-link]" =?>
173− para (linkWith ("", ["smart-link"], []) "http://example.com" "" "x")
174−
175− , "'smart' card" =:
176− "[x|http://example.com|smart-card]" =?>
177− para (linkWith ("", ["smart-card"], []) "http://example.com" "" "x")
178164 ]
179165
180166 , "image" =:
test/Tests/Readers/Markdown.hs+3−3
@@ -358,7 +358,7 @@ tests = [ testGroup "inline code"
358358 para (text "The value of the " <> math "x" <> text "\8217s and the systems\8217 condition.")
359359 , test markdownSmart "unclosed double quote"
360360 ("**this should \"be bold**"
361− =?> para (strong "this should \8220be bold"))
361+ =?> para (strong "this should \"be bold"))
362362 ]
363363 , testGroup "footnotes"
364364 [ "indent followed by newline and flush-left text" =:
@@ -374,8 +374,8 @@ tests = [ testGroup "inline code"
374374 , testGroup "lhs"
375375 [ test (purely $ readMarkdown def{ readerExtensions = enableExtension
376376 Ext_literate_haskell pandocExtensions })
377− "inverse bird tracks and html"
378− $ ("> a\n\n< b\n\n<div>\n" :: Text)
377+ "inverse bird tracks and html" $
378+ "> a\n\n< b\n\n<div>\n"
379379 =?> codeBlockWith ("",["haskell","literate"],[]) "a"
380380 <>
381381 codeBlockWith ("",["haskell"],[]) "b"
test/Tests/Readers/Odt.hs−1
@@ -176,7 +176,6 @@ namesOfTestsComparingToNative = [ "blockquote"
176176 , "referenceToText"
177177 , "simpleTable"
178178 , "simpleTableWithCaption"
179− , "tab"
180179-- , "table"
181180 , "textMixedStyles"
182181 , "tableWithContents"
test/Tests/Writers/ConTeXt.hs+113−122
@@ -39,125 +39,116 @@ infix 4 =:
3939(=:) = test context
4040
4141tests :: [TestTree]
42−tests =
43− [ testGroup "inline code"
44− [ "with '}'" =: code "}" =?> "\\mono{\\}}"
45− , "without '}'" =: code "]" =?> "\\type{]}"
46− , "span with ID" =:
47− spanWith ("city", [], []) "Berlin" =?>
48− "\\reference[city]{}Berlin"
49− , testProperty "code property" $ \s -> null s || '\n' `elem` s ||
50− if '{' `elem` s || '}' `elem` s
51− then context' (code $ pack s) == "\\mono{" ++
52− context' (str $ pack s) ++ "}"
53− else context' (code $ pack s) == "\\type{" ++ s ++ "}"
54− ]
55− , testGroup "headers"
56− [ "level 1" =:
57− headerWith ("my-header",[],[]) 1 "My header" =?>
58− "\\section[title={My header},reference={my-header}]"
59− , test contextDiv "section-divs" $
60− ( headerWith ("header1", [], []) 1 (text "Header1")
61− <> headerWith ("header2", [], []) 2 (text "Header2")
62− <> headerWith ("header3", [], []) 3 (text "Header3")
63− <> headerWith ("header4", [], []) 4 (text "Header4")
64− <> headerWith ("header5", [], []) 5 (text "Header5")
65− <> headerWith ("header6", [], []) 6 (text "Header6"))
66− =?>
67− unlines
68− [ "\\startsection[title={Header1},reference={header1}]\n"
69− , "\\startsubsection[title={Header2},reference={header2}]\n"
70− , "\\startsubsubsection[title={Header3},reference={header3}]\n"
71− , "\\startsubsubsubsection[title={Header4},reference={header4}]\n"
72− , "\\startsubsubsubsubsection[title={Header5},reference={header5}]\n"
73− , "\\startsubsubsubsubsubsection[title={Header6},reference={header6}]\n"
74− , "\\stopsubsubsubsubsubsection\n"
75− , "\\stopsubsubsubsubsection\n"
76− , "\\stopsubsubsubsection\n"
77− , "\\stopsubsubsection\n"
78− , "\\stopsubsection\n"
79− , "\\stopsection" ]
80− ]
81− , testGroup "bullet lists"
82− [ "nested" =:
83− bulletList [
84− plain (text "top")
85− <> bulletList [
86− plain (text "next")
87− <> bulletList [plain (text "bot")]
88− ]
89− ] =?> unlines
90− [ "\\startitemize[packed]"
91− , "\\item"
92− , " top"
93− , " \\startitemize[packed]"
94− , " \\item"
95− , " next"
96− , " \\startitemize[packed]"
97− , " \\item"
98− , " bot"
99− , " \\stopitemize"
100− , " \\stopitemize"
101− , "\\stopitemize" ]
102− ]
103− , testGroup "natural tables"
104− [ test contextNtb "table with header and caption" $
105− let capt = text "Table 1"
106− aligns = [ (AlignRight, ColWidthDefault)
107− , (AlignLeft, ColWidthDefault)
108− , (AlignCenter, ColWidthDefault)
109− , (AlignDefault, ColWidthDefault) ]
110− headers = [plain $ text "Right",
111− plain $ text "Left",
112− plain $ text "Center",
113− plain $ text "Default"]
114− rows = [[plain $ text "1.1",
115− plain $ text "1.2",
116− plain $ text "1.3",
117− plain $ text "1.4"]
118− ,[plain $ text "2.1",
119− plain $ text "2.2",
120− plain $ text "2.3",
121− plain $ text "2.4"]
122− ,[plain $ text "3.1",
123− plain $ text "3.2",
124− plain $ text "3.3",
125− plain $ text "3.4"]]
126− toRow = Row nullAttr . map simpleCell
127− in table (simpleCaption $ plain capt)
128− aligns
129− (TableHead nullAttr [toRow headers])
130− [TableBody nullAttr 0 [] $ map toRow rows]
131− (TableFoot nullAttr [])
132− =?> unlines [ "\\startplacetable[title={Table 1}]"
133− , "\\startTABLE"
134− , "\\startTABLEhead"
135− , "\\NC[align=left] Right"
136− , "\\NC[align=right] Left"
137− , "\\NC[align=middle] Center"
138− , "\\NC Default"
139− , "\\NC\\NR"
140− , "\\stopTABLEhead"
141− , "\\startTABLEbody"
142− , "\\NC[align=left] 1.1"
143− , "\\NC[align=right] 1.2"
144− , "\\NC[align=middle] 1.3"
145− , "\\NC 1.4"
146− , "\\NC\\NR"
147− , "\\NC[align=left] 2.1"
148− , "\\NC[align=right] 2.2"
149− , "\\NC[align=middle] 2.3"
150− , "\\NC 2.4"
151− , "\\NC\\NR"
152− , "\\stopTABLEbody"
153− , "\\startTABLEfoot"
154− , "\\NC[align=left] 3.1"
155− , "\\NC[align=right] 3.2"
156− , "\\NC[align=middle] 3.3"
157− , "\\NC 3.4"
158− , "\\NC\\NR"
159− , "\\stopTABLEfoot"
160− , "\\stopTABLE"
161− , "\\stopplacetable" ]
162− ]
163− ]
42+tests = [ testGroup "inline code"
43+ [ "with '}'" =: code "}" =?> "\\mono{\\}}"
44+ , "without '}'" =: code "]" =?> "\\type{]}"
45+ , testProperty "code property" $ \s -> null s || '\n' `elem` s ||
46+ if '{' `elem` s || '}' `elem` s
47+ then context' (code $ pack s) == "\\mono{" ++
48+ context' (str $ pack s) ++ "}"
49+ else context' (code $ pack s) == "\\type{" ++ s ++ "}"
50+ ]
51+ , testGroup "headers"
52+ [ "level 1" =:
53+ headerWith ("my-header",[],[]) 1 "My header" =?> "\\section[title={My header},reference={my-header}]"
54+ , test contextDiv "section-divs" $
55+ ( headerWith ("header1", [], []) 1 (text "Header1")
56+ <> headerWith ("header2", [], []) 2 (text "Header2")
57+ <> headerWith ("header3", [], []) 3 (text "Header3")
58+ <> headerWith ("header4", [], []) 4 (text "Header4")
59+ <> headerWith ("header5", [], []) 5 (text "Header5")
60+ <> headerWith ("header6", [], []) 6 (text "Header6"))
61+ =?>
62+ unlines [ "\\startsection[title={Header1},reference={header1}]\n"
63+ , "\\startsubsection[title={Header2},reference={header2}]\n"
64+ , "\\startsubsubsection[title={Header3},reference={header3}]\n"
65+ , "\\startsubsubsubsection[title={Header4},reference={header4}]\n"
66+ , "\\startsubsubsubsubsection[title={Header5},reference={header5}]\n"
67+ , "\\startsubsubsubsubsubsection[title={Header6},reference={header6}]\n"
68+ , "\\stopsubsubsubsubsubsection\n"
69+ , "\\stopsubsubsubsubsection\n"
70+ , "\\stopsubsubsubsection\n"
71+ , "\\stopsubsubsection\n"
72+ , "\\stopsubsection\n"
73+ , "\\stopsection" ]
74+ ]
75+ , testGroup "bullet lists"
76+ [ "nested" =:
77+ bulletList [
78+ plain (text "top")
79+ <> bulletList [
80+ plain (text "next")
81+ <> bulletList [plain (text "bot")]
82+ ]
83+ ] =?> unlines
84+ [ "\\startitemize[packed]"
85+ , "\\item"
86+ , " top"
87+ , " \\startitemize[packed]"
88+ , " \\item"
89+ , " next"
90+ , " \\startitemize[packed]"
91+ , " \\item"
92+ , " bot"
93+ , " \\stopitemize"
94+ , " \\stopitemize"
95+ , "\\stopitemize" ]
96+ ]
97+ , testGroup "natural tables"
98+ [ test contextNtb "table with header and caption" $
99+ let capt = text "Table 1"
100+ aligns = [(AlignRight, ColWidthDefault), (AlignLeft, ColWidthDefault), (AlignCenter, ColWidthDefault), (AlignDefault, ColWidthDefault)]
101+ headers = [plain $ text "Right",
102+ plain $ text "Left",
103+ plain $ text "Center",
104+ plain $ text "Default"]
105+ rows = [[plain $ text "1.1",
106+ plain $ text "1.2",
107+ plain $ text "1.3",
108+ plain $ text "1.4"]
109+ ,[plain $ text "2.1",
110+ plain $ text "2.2",
111+ plain $ text "2.3",
112+ plain $ text "2.4"]
113+ ,[plain $ text "3.1",
114+ plain $ text "3.2",
115+ plain $ text "3.3",
116+ plain $ text "3.4"]]
117+ toRow = Row nullAttr . map simpleCell
118+ in table (simpleCaption $ plain capt)
119+ aligns
120+ (TableHead nullAttr [toRow headers])
121+ [TableBody nullAttr 0 [] $ map toRow rows]
122+ (TableFoot nullAttr [])
123+ =?> unlines [ "\\startplacetable[title={Table 1}]"
124+ , "\\startTABLE"
125+ , "\\startTABLEhead"
126+ , "\\NC[align=left] Right"
127+ , "\\NC[align=right] Left"
128+ , "\\NC[align=middle] Center"
129+ , "\\NC Default"
130+ , "\\NC\\NR"
131+ , "\\stopTABLEhead"
132+ , "\\startTABLEbody"
133+ , "\\NC[align=left] 1.1"
134+ , "\\NC[align=right] 1.2"
135+ , "\\NC[align=middle] 1.3"
136+ , "\\NC 1.4"
137+ , "\\NC\\NR"
138+ , "\\NC[align=left] 2.1"
139+ , "\\NC[align=right] 2.2"
140+ , "\\NC[align=middle] 2.3"
141+ , "\\NC 2.4"
142+ , "\\NC\\NR"
143+ , "\\stopTABLEbody"
144+ , "\\startTABLEfoot"
145+ , "\\NC[align=left] 3.1"
146+ , "\\NC[align=right] 3.2"
147+ , "\\NC[align=middle] 3.3"
148+ , "\\NC 3.4"
149+ , "\\NC\\NR"
150+ , "\\stopTABLEfoot"
151+ , "\\stopTABLE"
152+ , "\\stopplacetable" ]
153+ ]
154+ ]
test/Tests/Writers/Docbook.hs+6−43
@@ -11,14 +11,9 @@ import Text.Pandoc.Builder
1111docbook :: (ToPandoc a) => a -> String
1212docbook = docbookWithOpts def{ writerWrapText = WrapNone }
1313
14−docbook5 :: (ToPandoc a) => a -> String
15−docbook5 = docbook5WithOpts def{ writerWrapText = WrapNone }
16−
1714docbookWithOpts :: ToPandoc a => WriterOptions -> a -> String
1815docbookWithOpts opts = unpack . purely (writeDocbook4 opts) . toPandoc
1916
20−docbook5WithOpts :: ToPandoc a => WriterOptions -> a -> String
21−docbook5WithOpts opts = unpack . purely (writeDocbook5 opts) . toPandoc
2217{-
2318 "my test" =: X =?> Y
2419
@@ -83,32 +78,32 @@ tests = [ testGroup "line blocks"
8378 , "</warning>"
8479 ]
8580 , "admonition-with-title" =:
86− divWith ("foo", ["note"], []) (
81+ divWith ("foo", ["attention"], []) (
8782 divWith ("foo", ["title"], [])
8883 (plain (text "This is title")) <>
8984 para "This is a test"
9085 )
9186 =?> unlines
92− [ "<note id=\"foo\">"
87+ [ "<attention id=\"foo\">"
9388 , " <title>This is title</title>"
9489 , " <para>"
9590 , " This is a test"
9691 , " </para>"
97− , "</note>"
92+ , "</attention>"
9893 ]
9994 , "admonition-with-title-in-para" =:
100− divWith ("foo", ["note"], []) (
95+ divWith ("foo", ["attention"], []) (
10196 divWith ("foo", ["title"], [])
10297 (para "This is title") <>
10398 para "This is a test"
10499 )
105100 =?> unlines
106− [ "<note id=\"foo\">"
101+ [ "<attention id=\"foo\">"
107102 , " <title>This is title</title>"
108103 , " <para>"
109104 , " This is a test"
110105 , " </para>"
111− , "</note>"
106+ , "</attention>"
112107 ]
113108 , "single-child" =:
114109 divWith ("foo", [], []) (para "This is a test")
@@ -371,36 +366,4 @@ tests = [ testGroup "line blocks"
371366 ]
372367 ]
373368 ]
374− , testGroup "section attributes" $
375− let
376− headers = headerWith ("myid1",[],[("role","internal"),("xml:id","anotherid"),("dir","rtl")]) 1 "header1"
377− <> headerWith ("myid2",[],[("invalidname","value"),("arch","linux"),("dir","invaliddir")]) 1 "header2"
378− in
379− [ test docbook5 "sections with attributes (db5)" $
380− headers =?>
381− unlines [ "<section xmlns=\"http://docbook.org/ns/docbook\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:id=\"myid1\" role=\"internal\" dir=\"rtl\">"
382− , " <title>header1</title>"
383− , " <para>"
384− , " </para>"
385− , "</section>"
386− , "<section xmlns=\"http://docbook.org/ns/docbook\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:id=\"myid2\">"
387− , " <title>header2</title>"
388− , " <para>"
389− , " </para>"
390− , "</section>"
391− ]
392− , test docbook "sections with attributes (db4)" $
393− headers =?>
394− unlines [ "<sect1 id=\"myid1\" role=\"internal\">"
395− , " <title>header1</title>"
396− , " <para>"
397− , " </para>"
398− , "</sect1>"
399− , "<sect1 id=\"myid2\" arch=\"linux\">"
400− , " <title>header2</title>"
401− , " <para>"
402− , " </para>"
403− , "</sect1>"
404− ]
405− ]
406369 ]
test/Tests/Writers/Docx.hs−5
@@ -111,11 +111,6 @@ tests = [ testGroup "inlines"
111111 def
112112 "docx/tables.native"
113113 "docx/golden/tables.docx"
114− , docxTest
115− "tables without explicit column widths"
116− def
117− "docx/tables-default-widths.native"
118− "docx/golden/tables-default-widths.docx"
119114 , docxTest
120115 "tables with lists in cells"
121116 def
test/Tests/Writers/HTML.hs+52−57
@@ -34,60 +34,55 @@ infix 4 =:
3434(=:) = test html
3535
3636tests :: [TestTree]
37−tests =
38− [ testGroup "inline code"
39− [ "basic" =: code "@&" =?> "<code>@&amp;</code>"
40− , "haskell" =: codeWith ("",["haskell"],[]) ">>="
41− =?> "<code class=\"sourceCode haskell\"><span class=\"op\">&gt;&gt;=</span></code>"
42− , "nolanguage" =: codeWith ("",["nolanguage"],[]) ">>="
43− =?> "<code class=\"nolanguage\">&gt;&gt;=</code>"
44− ]
45− , testGroup "images"
46− [ "alt with formatting" =:
47− image "/url" "title" ("my " <> emph "image")
48− =?> "<img src=\"/url\" title=\"title\" alt=\"my image\" />"
49− ]
50− , testGroup "blocks"
51− [ "definition list with empty <dt>" =:
52− definitionList [(mempty, [para $ text "foo bar"])]
53− =?> "<dl><dt></dt><dd><p>foo bar</p></dd></dl>"
54− , "heading with disallowed attributes" =:
55− headerWith ("", [], [("invalid","1"), ("lang", "en")]) 1 "test"
56− =?>
57− "<h1 lang=\"en\">test</h1>"
58− ]
59− , testGroup "quotes"
60− [ "quote with cite attribute (without q-tags)" =:
61− doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
62− =?> "“<span cite=\"http://example.org\">examples</span>”"
63− , tQ "quote with cite attribute (with q-tags)" $
64− doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
65− =?> "<q cite=\"http://example.org\">examples</q>"
66− ]
67− , testGroup "sample"
68− [ "sample should be rendered correctly" =:
69− plain (codeWith ("",["sample"],[]) "Answer is 42") =?>
70− "<samp>Answer is 42</samp>"
71− ]
72− , testGroup "variable"
73− [ "variable should be rendered correctly" =:
74− plain (codeWith ("",["variable"],[]) "result") =?>
75− "<var>result</var>"
76− ]
77− , testGroup "sample with style"
78− [ "samp should wrap highlighted code" =:
79− codeWith ("",["sample","haskell"],[]) ">>="
80− =?> ("<samp><code class=\"sourceCode haskell\">" ++
81− "<span class=\"op\">&gt;&gt;=</span></code></samp>")
82− ]
83− , testGroup "variable with style"
84− [ "var should wrap highlighted code" =:
85− codeWith ("",["haskell","variable"],[]) ">>="
86− =?> ("<var><code class=\"sourceCode haskell\">" ++
87− "<span class=\"op\">&gt;&gt;=</span></code></var>")
88− ]
89− ]
90− where
91− tQ :: (ToString a, ToPandoc a)
92− => String -> (a, String) -> TestTree
93− tQ = test htmlQTags
37+tests = [ testGroup "inline code"
38+ [ "basic" =: code "@&" =?> "<code>@&amp;</code>"
39+ , "haskell" =: codeWith ("",["haskell"],[]) ">>="
40+ =?> "<code class=\"sourceCode haskell\"><span class=\"op\">&gt;&gt;=</span></code>"
41+ , "nolanguage" =: codeWith ("",["nolanguage"],[]) ">>="
42+ =?> "<code class=\"nolanguage\">&gt;&gt;=</code>"
43+ ]
44+ , testGroup "images"
45+ [ "alt with formatting" =:
46+ image "/url" "title" ("my " <> emph "image")
47+ =?> "<img src=\"/url\" title=\"title\" alt=\"my image\" />"
48+ ]
49+ , testGroup "blocks"
50+ [ "definition list with empty <dt>" =:
51+ definitionList [(mempty, [para $ text "foo bar"])]
52+ =?> "<dl><dt></dt><dd><p>foo bar</p></dd></dl>"
53+ ]
54+ , testGroup "quotes"
55+ [ "quote with cite attribute (without q-tags)" =:
56+ doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
57+ =?> "“<span cite=\"http://example.org\">examples</span>”"
58+ , tQ "quote with cite attribute (with q-tags)" $
59+ doubleQuoted (spanWith ("", [], [("cite", "http://example.org")]) (str "examples"))
60+ =?> "<q cite=\"http://example.org\">examples</q>"
61+ ]
62+ , testGroup "sample"
63+ [ "sample should be rendered correctly" =:
64+ plain (codeWith ("",["sample"],[]) "Answer is 42") =?>
65+ "<samp>Answer is 42</samp>"
66+ ]
67+ , testGroup "variable"
68+ [ "variable should be rendered correctly" =:
69+ plain (codeWith ("",["variable"],[]) "result") =?>
70+ "<var>result</var>"
71+ ]
72+ , testGroup "sample with style"
73+ [ "samp should wrap highlighted code" =:
74+ codeWith ("",["sample","haskell"],[]) ">>="
75+ =?> ("<samp><code class=\"sourceCode haskell\">" ++
76+ "<span class=\"op\">&gt;&gt;=</span></code></samp>")
77+ ]
78+ , testGroup "variable with style"
79+ [ "var should wrap highlighted code" =:
80+ codeWith ("",["haskell","variable"],[]) ">>="
81+ =?> ("<var><code class=\"sourceCode haskell\">" ++
82+ "<span class=\"op\">&gt;&gt;=</span></code></var>")
83+ ]
84+ ]
85+ where
86+ tQ :: (ToString a, ToPandoc a)
87+ => String -> (a, String) -> TestTree
88+ tQ = test htmlQTags
test/Tests/Writers/JATS.hs+91−135
@@ -1,21 +1,21 @@
11{-# LANGUAGE OverloadedStrings #-}
22module Tests.Writers.JATS (tests) where
33
4−import Data.Text (Text)
4+import Data.Text (unpack)
55import Test.Tasty
66import Tests.Helpers
77import Text.Pandoc
88import Text.Pandoc.Arbitrary ()
99import Text.Pandoc.Builder
10−import qualified Data.Text as T
1110
12−jats :: (ToPandoc a) => a -> Text
13−jats = purely (writeJATS def{ writerWrapText = WrapNone })
14− . toPandoc
11+jats :: (ToPandoc a) => a -> String
12+jats = unpack
13+ . purely (writeJATS def{ writerWrapText = WrapNone })
14+ . toPandoc
1515
16−jatsArticleAuthoring :: (ToPandoc a) => a -> Text
17−jatsArticleAuthoring =
18− purely (writeJatsArticleAuthoring def{ writerWrapText = WrapNone })
16+jatsArticleAuthoring :: (ToPandoc a) => a -> String
17+jatsArticleAuthoring = unpack
18+ . purely (writeJatsArticleAuthoring def{ writerWrapText = WrapNone })
1919 . toPandoc
2020
2121{-
@@ -32,133 +32,89 @@ which is in turn shorthand for
3232
3333infix 4 =:
3434(=:) :: (ToString a, ToPandoc a)
35− => String -> (a, Text) -> TestTree
35+ => String -> (a, String) -> TestTree
3636(=:) = test jats
3737
3838tests :: [TestTree]
39−tests =
40− [ testGroup "inline code"
41− [ "basic" =: code "@&" =?> "<p><monospace>@&amp;</monospace></p>"
42− , "lang" =: codeWith ("", ["c"], []) "@&" =?> "<p><code language=\"c\">@&amp;</code></p>"
43− ]
44− , testGroup "block code"
45− [ "basic" =: codeBlock "@&" =?> "<preformat>@&amp;</preformat>"
46− , "lang" =: codeBlockWith ("", ["c"], []) "@&" =?> "<code language=\"c\">@&amp;</code>"
47− ]
48− , testGroup "images"
49− [ "basic" =:
50− image "/url" "title" mempty
51− =?> "<graphic mimetype=\"image\" mime-subtype=\"\" xlink:href=\"/url\" xlink:title=\"title\" />"
52− ]
53− , testGroup "inlines"
54− [ "Emphasis" =: emph "emphasized"
55− =?> "<p><italic>emphasized</italic></p>"
56−
57− , test jatsArticleAuthoring "footnote in articleauthoring tag set"
58− ("test" <> note (para "footnote") =?>
59− unlines [ "<p>test<fn>"
60− , " <p>footnote</p>"
61− , "</fn></p>"
62− ])
63− ]
64− , "bullet list" =: bulletList [ plain $ text "first"
65− , plain $ text "second"
66− , plain $ text "third"
67− ]
68− =?> "<list list-type=\"bullet\">\n\
69− \ <list-item>\n\
70− \ <p>first</p>\n\
71− \ </list-item>\n\
72− \ <list-item>\n\
73− \ <p>second</p>\n\
74− \ </list-item>\n\
75− \ <list-item>\n\
76− \ <p>third</p>\n\
77− \ </list-item>\n\
78− \</list>"
79− , testGroup "definition lists"
80− [ "with internal link" =: definitionList [(link "#go" "" (str "testing"),
81− [plain (text "hi there")])] =?>
82− "<def-list>\n\
83− \ <def-item>\n\
84− \ <term><xref alt=\"testing\" rid=\"go\">testing</xref></term>\n\
85− \ <def>\n\
86− \ <p>hi there</p>\n\
87− \ </def>\n\
88− \ </def-item>\n\
89− \</def-list>"
90− ]
91− , testGroup "math"
92− [ "escape |" =: para (math "\\sigma|_{\\{x\\}}") =?>
93− "<p><inline-formula><alternatives>\n\
94− \<tex-math><![CDATA[\\sigma|_{\\{x\\}}]]></tex-math>\n\
95− \<mml:math display=\"inline\" xmlns:mml=\"http://www.w3.org/1998/Math/MathML\"><mml:mrow><mml:mi>σ</mml:mi><mml:msub><mml:mo stretchy=\"false\" form=\"prefix\">|</mml:mo><mml:mrow><mml:mo stretchy=\"false\" form=\"prefix\">{</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy=\"false\" form=\"postfix\">}</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></alternatives></inline-formula></p>"
96− ]
97− , testGroup "headers"
98− [ "unnumbered header" =:
99− headerWith ("foo",["unnumbered"],[]) 1
100− (text "Header 1" <> note (plain $ text "note")) =?>
101− "<sec id=\"foo\">\n\
102− \ <title>Header 1<xref ref-type=\"fn\" rid=\"fn1\">1</xref></title>\n\
103− \</sec>"
104− , "unnumbered sub header" =:
105− headerWith ("foo",["unnumbered"],[]) 1
106− (text "Header")
107− <> headerWith ("foo",["unnumbered"],[]) 2
108− (text "Sub-Header") =?>
109− "<sec id=\"foo\">\n\
110− \ <title>Header</title>\n\
111− \ <sec id=\"foo\">\n\
112− \ <title>Sub-Header</title>\n\
113− \ </sec>\n\
114− \</sec>"
115− , "containing image" =:
116− header 1 (image "imgs/foo.jpg" "" (text "Alt text")) =?>
117− "<sec>\n\
118− \ <title><inline-graphic mimetype=\"image\" mime-subtype=\"jpeg\" xlink:href=\"imgs/foo.jpg\" /></title>\n\
119− \</sec>"
120− ]
121−
122− , testGroup "ids"
123− [ "non-ASCII in header ID" =:
124− headerWith ("smørbrød",[],[]) 1 (text "smørbrød") =?>
125− T.unlines [ "<sec id=\"smørbrød\">"
126− , " <title>smørbrød</title>"
127− , "</sec>"
128− ]
129−
130− , "disallowed symbol in header id" =:
131− headerWith ("i/o",[],[]) 1 (text "I/O") =?>
132− T.unlines [ "<sec id=\"iU002Fo\">"
133− , " <title>I/O</title>"
134− , "</sec>"
135− ]
136−
137− , "disallowed symbols in internal link target" =:
138− link "#foo:bar" "" "baz" =?>
139− "<p><xref alt=\"baz\" rid=\"fooU003Abar\">baz</xref></p>"
140−
141− , "code id starting with a number" =:
142− codeWith ("7y",[],[]) "print 5" =?>
143− "<p><monospace id=\"U0037y\">print 5</monospace></p>"
144− ]
145−
146− , testGroup "spans"
147− [ "unwrapped if no attributes given" =:
148− spanWith nullAttr "text in span" =?>
149− "<p>text in span</p>"
150−
151− , "converted to named-content element if class given" =:
152− spanWith ("a", ["genus-species"], [("alt", "aa")]) "C. elegans" =?>
153− ("<p><named-content id=\"a\" alt=\"aa\" content-type=\"genus-species\">"
154− <> "C. elegans</named-content></p>")
155−
156− , "unwrapped if styled-content element would have no attributes" =:
157− spanWith ("", [], [("hidden", "true")]) "text in span" =?>
158− "<p>text in span</p>"
159−
160− , "use content-type attribute if present" =:
161− spanWith ("", [], [("content-type", "species")]) "E. coli" =?>
162− "<p><named-content content-type=\"species\">E. coli</named-content></p>"
163− ]
164− ]
39+tests = [ testGroup "inline code"
40+ [ "basic" =: code "@&" =?> "<p><monospace>@&amp;</monospace></p>"
41+ , "lang" =: codeWith ("", ["c"], []) "@&" =?> "<p><code language=\"c\">@&amp;</code></p>"
42+ ]
43+ , testGroup "block code"
44+ [ "basic" =: codeBlock "@&" =?> "<preformat>@&amp;</preformat>"
45+ , "lang" =: codeBlockWith ("", ["c"], []) "@&" =?> "<code language=\"c\">@&amp;</code>"
46+ ]
47+ , testGroup "images"
48+ [ "basic" =:
49+ image "/url" "title" mempty
50+ =?> "<graphic mimetype=\"image\" mime-subtype=\"\" xlink:href=\"/url\" xlink:title=\"title\" />"
51+ ]
52+ , testGroup "inlines"
53+ [ "Emphasis" =: emph "emphasized"
54+ =?> "<p><italic>emphasized</italic></p>"
55+
56+ , test jatsArticleAuthoring "footnote in articleauthoring tag set"
57+ ("test" <> note (para "footnote") =?>
58+ unlines [ "<p>test<fn>"
59+ , " <p>footnote</p>"
60+ , "</fn></p>"
61+ ])
62+ ]
63+ , "bullet list" =: bulletList [ plain $ text "first"
64+ , plain $ text "second"
65+ , plain $ text "third"
66+ ]
67+ =?> "<list list-type=\"bullet\">\n\
68+ \ <list-item>\n\
69+ \ <p>first</p>\n\
70+ \ </list-item>\n\
71+ \ <list-item>\n\
72+ \ <p>second</p>\n\
73+ \ </list-item>\n\
74+ \ <list-item>\n\
75+ \ <p>third</p>\n\
76+ \ </list-item>\n\
77+ \</list>"
78+ , testGroup "definition lists"
79+ [ "with internal link" =: definitionList [(link "#go" "" (str "testing"),
80+ [plain (text "hi there")])] =?>
81+ "<def-list>\n\
82+ \ <def-item>\n\
83+ \ <term><xref alt=\"testing\" rid=\"go\">testing</xref></term>\n\
84+ \ <def>\n\
85+ \ <p>hi there</p>\n\
86+ \ </def>\n\
87+ \ </def-item>\n\
88+ \</def-list>"
89+ ]
90+ , testGroup "math"
91+ [ "escape |" =: para (math "\\sigma|_{\\{x\\}}") =?>
92+ "<p><inline-formula><alternatives>\n\
93+ \<tex-math><![CDATA[\\sigma|_{\\{x\\}}]]></tex-math>\n\
94+ \<mml:math display=\"inline\" xmlns:mml=\"http://www.w3.org/1998/Math/MathML\"><mml:mrow><mml:mi>σ</mml:mi><mml:msub><mml:mo stretchy=\"false\" form=\"prefix\">|</mml:mo><mml:mrow><mml:mo stretchy=\"false\" form=\"prefix\">{</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy=\"false\" form=\"postfix\">}</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></alternatives></inline-formula></p>"
95+ ]
96+ , testGroup "headers"
97+ [ "unnumbered header" =:
98+ headerWith ("foo",["unnumbered"],[]) 1
99+ (text "Header 1" <> note (plain $ text "note")) =?>
100+ "<sec id=\"foo\">\n\
101+ \ <title>Header 1<xref ref-type=\"fn\" rid=\"fn1\">1</xref></title>\n\
102+ \</sec>"
103+ , "unnumbered sub header" =:
104+ headerWith ("foo",["unnumbered"],[]) 1
105+ (text "Header")
106+ <> headerWith ("foo",["unnumbered"],[]) 2
107+ (text "Sub-Header") =?>
108+ "<sec id=\"foo\">\n\
109+ \ <title>Header</title>\n\
110+ \ <sec id=\"foo\">\n\
111+ \ <title>Sub-Header</title>\n\
112+ \ </sec>\n\
113+ \</sec>"
114+ , "containing image" =:
115+ header 1 (image "imgs/foo.jpg" "" (text "Alt text")) =?>
116+ "<sec>\n\
117+ \ <title><inline-graphic mimetype=\"image\" mime-subtype=\"jpeg\" xlink:href=\"imgs/foo.jpg\" /></title>\n\
118+ \</sec>"
119+ ]
120+ ]
test/Tests/Writers/Jira.hs−52
@@ -61,64 +61,12 @@ tests =
6161 , "user link with user as description" =:
6262 linkWith ("", ["user-account"], []) "~johndoe" "" "~johndoe" =?>
6363 "[~johndoe]"
64−
65− , "'smart' link" =:
66− para (linkWith ("", ["smart-link"], []) "http://example.com" "" "x") =?>
67− "[x|http://example.com|smart-link]"
68−
69− , "'smart' card" =:
70− para (linkWith ("", ["smart-card"], []) "http://example.org" "" "x") =?>
71− "[x|http://example.org|smart-card]"
7264 ]
7365
7466 , testGroup "spans"
7567 [ "id is used as anchor" =:
7668 spanWith ("unicorn", [], []) (str "Unicorn") =?>
7769 "{anchor:unicorn}Unicorn"
78−
79− , "use `color` attribute" =:
80− spanWith ("",[],[("color","red")]) "ruby" =?>
81− "{color:red}ruby{color}"
82− ]
83−
84− , testGroup "code"
85− [ "code block with known language" =:
86− codeBlockWith ("", ["java"], []) "Book book = new Book(\"Algebra\")" =?>
87− "{code:java}\nBook book = new Book(\"Algebra\")\n{code}"
88−
89− , "code block without language" =:
90− codeBlockWith ("", [], []) "preformatted\n text.\n" =?>
91− "{noformat}\npreformatted\n text.\n{noformat}"
92− ]
93− ]
94−
95− , testGroup "blocks"
96− [ testGroup "div"
97− [ "empty attributes" =:
98− divWith nullAttr (para "interesting text") =?>
99− "interesting text"
100−
101− , "just identifier" =:
102− divWith ("a", [], []) (para "interesting text") =?>
103− "{anchor:a}interesting text"
104−
105− , "with class 'panel'" =:
106− divWith ("", ["panel"], []) (para "Contents!") =?>
107− "{panel}\nContents\\!\n{panel}\n"
108−
109− , "panel with id" =:
110− divWith ("b", ["panel"], []) (para "text") =?>
111− "{panel}\n{anchor:b}text\n{panel}\n"
112−
113− , "title attribute" =:
114− divWith ("", [], [("title", "Gimme!")]) (para "Contents!") =?>
115− "{panel:title=Gimme!}\nContents\\!\n{panel}\n"
116−
117− , "nested panels" =:
118− let panelAttr = ("", ["panel"], [])
119− in divWith panelAttr (para "hi" <>
120− divWith panelAttr (para "wassup?")) =?>
121− "{panel}\nhi\n\nwassup?\n{panel}\n"
12270 ]
12371 ]
12472 ]
test/Tests/Writers/OOXML.hs+1−3
@@ -55,9 +55,7 @@ testArchive :: (WriterOptions -> Pandoc -> PandocIO BL.ByteString)
5555 -> IO Archive
5656testArchive writerFn opts fp = do
5757 txt <- T.readFile fp
58− bs <- runIOorExplode $ do
59− setTranslations "en-US"
60− readNative def txt >>= writerFn opts
58+ bs <- runIOorExplode $ readNative def txt >>= writerFn opts
6159 return $ toArchive bs
6260
6361compareFileList :: FilePath -> Archive -> Archive -> Maybe String
test/command/1279.md+1−1
@@ -9,7 +9,7 @@ title: My Article
99[^1]: Dept. of This and That
1010^D
1111---
12−author: John Doe[^1]
12+author: "John Doe[^1]"
1313date: 2014
1414title: My Article
1515---
test/command/2103.mddeleted−8
@@ -1,8 +0,0 @@
1−```
2−pandoc -t latex
3−| A happy pandoc user said "fix this bug please
4−| or I'll go crazy!"
5−^D
6−A happy pandoc user said ``fix this bug please\\
7−or I'll go crazy!''
8−```
test/command/2874.md+2−2
@@ -3,12 +3,12 @@
33<a></a>
44<br/>
55^D
6−{}\strut \\
6+{}~\\
77```
88
99```
1010% pandoc -f html -t latex
1111<a name="foo"></a><br/>
1212^D
13−\protect\hypertarget{foo}{}{}\strut \\
13+\protect\hypertarget{foo}{}{}~\\
1414```
test/command/3752.mddeleted−60
@@ -1,60 +0,0 @@
1−```
2−% pandoc command/chap1/text.md command/chap2/text.md -f markdown+rebase_relative_paths --verbose -t docx -o - | pandoc -f docx -t plain
3−^D
4−[INFO] Loaded command/chap1/spider.png from command/chap1/spider.png
5−[INFO] Loaded command/chap2/spider.png from command/chap2/spider.png
6−[INFO] Loaded command/chap1/../../lalune.jpg from command/chap1/../../lalune.jpg
7−Chapter one
8−
9−A spider: [spider]
10−
11−Another spider: [another spider]
12−
13−The moon: [moon]
14−
15−Link to spider picture.
16−
17−URL left alone: manual.
18−
19−Absolute path left alone: absolute.
20−
21−Link to fragment: chapter two.
22−
23−Empty path: empty.
24−
25−Chapter two
26−
27−A spider: [spider]
28−```
29−
30−```
31−% pandoc command/chap1/text.md command/chap2/text.md -f markdown+rebase_relative_paths -t html
32−^D
33−<h1 id="chapter-one">Chapter one</h1>
34−<p>A spider: <img src="command/chap1/spider.png" alt="spider" /></p>
35−<p>Another spider: <img src="command/chap2/spider.png" alt="another spider" /></p>
36−<p>The moon: <img src="command/chap1/../../lalune.jpg" alt="moon" /></p>
37−<p>Link to <a href="command/chap1/spider.png">spider picture</a>.</p>
38−<p>URL left alone: <a href="https://pandoc.org/MANUAL.html">manual</a>.</p>
39−<p>Absolute path left alone: <a href="/foo/bar/baz.png">absolute</a>.</p>
40−<p>Link to fragment: <a href="#chapter-two">chapter two</a>.</p>
41−<p>Empty path: <a href="">empty</a>.</p>
42−<h1 id="chapter-two">Chapter two</h1>
43−<p>A spider: <img src="command/chap2/spider.png" alt="spider" /></p>
44−```
45−
46−```
47−% pandoc command/chap1/text.md command/chap2/text.md -f commonmark+rebase_relative_paths -t html
48−^D
49−<h1>Chapter one</h1>
50−<p>A spider: <img src="command/chap1/spider.png" alt="spider" /></p>
51−<p>Another spider: <img src="command/chap2/spider.png" alt="another spider" /></p>
52−<p>The moon: <img src="command/chap1/../../lalune.jpg" alt="moon" /></p>
53−<p>Link to <a href="command/chap1/spider.png">spider picture</a>.</p>
54−<p>URL left alone: <a href="https://pandoc.org/MANUAL.html">manual</a>.</p>
55−<p>Absolute path left alone: <a href="/foo/bar/baz.png">absolute</a>.</p>
56−<p>Link to fragment: <a href="#chapter-two">chapter two</a>.</p>
57−<p>Empty path: <a href="">empty</a>.</p>
58−<h1>Chapter two</h1>
59−<p>A spider: <img src="command/chap2/spider.png" alt="spider" /></p>
60−```
test/command/3792.md+1−1
@@ -6,7 +6,7 @@ and properly escaped.
66ok
77^D
88---
9−title: \<this> \*that\*
9+title: \<this\> \*that\*
1010---
1111
1212ok
test/command/4063.mddeleted−29
@@ -1,29 +0,0 @@
1−```
2−% pandoc -f html -t native
3−<table>
4−<colgroup>
5− <col width="30%" />
6− <col width="*" />
7−</colgroup>
8−<tr>
9− <td>1</td>
10− <td>2</td>
11−</tr>
12−</table>
13−^D
14−[Table ("",[],[]) (Caption Nothing
15− [])
16− [(AlignDefault,ColWidth 0.3)
17− ,(AlignDefault,ColWidth 0.7)]
18− (TableHead ("",[],[])
19− [])
20− [(TableBody ("",[],[]) (RowHeadColumns 0)
21− []
22− [Row ("",[],[])
23− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
24− [Plain [Str "1"]]
25− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
26− [Plain [Str "2"]]]])]
27− (TableFoot ("",[],[])
28− [])]
29−```
test/command/4164.md+1−1
@@ -26,6 +26,6 @@ Here is inline html:
2626
2727Here is inline html:
2828
29−\<div> \<balise> bla bla \</div>
29+\<div\> \<balise\> bla bla \</div\>
3030
3131```
test/command/4442.md+1−1
@@ -5,5 +5,5 @@
55^D
66\newcommand{\myFruit}{Mango\xspace}
77
8−Mango is the king of fruits.
8+Mango\xspace is the king of fruits.
99```
test/command/4465.mddeleted−15
@@ -1,15 +0,0 @@
1−```
2−% pandoc -f html -t markdown
3−<ol>
4− <li>An ordered list can contain block-level elements ind html, it means that divs are also allowed.</li>
5− <li>Let's see the problem! <div class="example">This is an example.</div></li>
6−</ol>
7−^D
8−1. An ordered list can contain block-level elements ind html, it means
9− that divs are also allowed.
10−2. Let\'s see the problem!
11−
12− ::: example
13− This is an example.
14− :::
15−```
test/command/5304.md+1−1
@@ -11,7 +11,7 @@
1111...
1212```
1313^D
14−``` markdown
14+``` {.markdown}
1515`«sträng»`
1616
1717`` «sträng» ``
test/command/5367.md+2−6
@@ -24,15 +24,11 @@ hello\footnote{doc footnote}
2424 >{\centering\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{0.17}}@{}}
2525\caption[Sample table.]{Sample table.\footnote{caption footnote}}\tabularnewline
2626\toprule
27−\begin{minipage}[b]{\linewidth}\centering
28−Fruit\footnote{header footnote}
29−\end{minipage} \\
27+Fruit\footnote{header footnote} \\
3028\midrule
3129\endfirsthead
3230\toprule
33−\begin{minipage}[b]{\linewidth}\centering
34−Fruit{}
35−\end{minipage} \\
31+Fruit{} \\
3632\midrule
3733\endhead
3834Bans\footnote{table cell footnote} \\
test/command/5519.md+1−1
@@ -6,7 +6,7 @@
66 ```
77``````
88^D
9−```` attr
9+```` {.attr}
1010 ```
1111 code
1212 ```
test/command/6026.mddeleted−19
@@ -1,19 +0,0 @@
1−```
2−% pandoc -t native
3−@{https://openreview.net/forum?id=HkwoSDPgg}
4−
5−@https://openreview.net/forum?id=HkwoSDPgg
6−^D
7−[Para [Cite [Citation {citationId = "https://openreview.net/forum?id=HkwoSDPgg", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 1, citationHash = 0}] [Str "@https://openreview.net/forum?id=HkwoSDPgg"]]
8−,Para [Cite [Citation {citationId = "https://openreview.net/forum?id", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "@https://openreview.net/forum?id"],Str "=HkwoSDPgg"]]
9−```
10−```
11−% pandoc -t markdown
12−@{https://openreview.net/forum?id=HkwoSDPgg}
13−
14−@https://openreview.net/forum?id=HkwoSDPgg
15−^D
16−@{https://openreview.net/forum?id=HkwoSDPgg}
17−
18−@https://openreview.net/forum?id=HkwoSDPgg
19−```
test/command/6541.mddeleted−27
@@ -1,27 +0,0 @@
1−```
2−% pandoc -f docbook -t markdown -s
3−<?xml version="1.0"?>
4−<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
5− "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
6−<book>
7−
8−<bookinfo>
9−<title>Title</title>
10−<author>
11−<firstname>Firstname</firstname><surname>Lastname</surname>
12−</author>
13−<releaseinfo>1.17</releaseinfo>
14−</bookinfo>
15−
16−<para>Text.</para>
17−
18−</book>
19−^D
20−---
21−author: Firstname Lastname
22−releaseinfo: 1.17
23−title: Title
24−---
25−
26−Text.
27−```
test/command/6620.md−3
@@ -7,12 +7,9 @@
77\SI{0.135(21)}{\m}
88
99\SI{12.3(60)}{\m}
10−
11−\SI{10.0 \pm 3.3}{\ms}
1210^D
1311<p>23 ± 2 m</p>
1412<p>125 ± 12 m</p>
1513<p>0.135 ± 0.021 m</p>
1614<p>12.3 ± 6 m</p>
17−<p>10.0 ± 3.3 ms</p>
1815```
test/command/6658.mddeleted−75
@@ -1,75 +0,0 @@
1−```
2−pandoc -f latex -t html
3−\num{12345,67890}
4−
5−\num{1+-2i}
6−
7−\num{.3e45}
8−
9−\num{1.654 x 2.34 x 3.430}
10−
11−\si{kg.m.s^{-1}}
12−
13−\si{\kilogram\metre\per\second}
14−
15−\si[per-mode=symbol]{\kilogram\metre\per\second}
16−
17−\si[per-mode=symbol]{\kilogram\metre\per\ampere\per\second}
18−
19−\numlist{10;20;30}
20−
21−\SIlist{0.13;0.67;0.80}{\milli\metre}
22−
23−\numrange{10}{20}
24−
25−\SIrange{0.13}{0.67}{\milli\metre}
26−
27−\ang{10}
28−
29−\ang{1;2;3}
30−
31−\ang{;;1}
32−
33−\ang{+10;;}
34−
35−\ang{-0;1;}
36−
37−\si{kg.m/s^2}
38−
39−\si{g_{polymer}~mol_{cat}.s^{-1}}
40−
41−\si{\kilo\gram\metre\per\square\second}
42−
43−\si{\gram\per\cubic\centi\metre}
44−
45−\si{\square\volt\cubic\lumen\per\farad}
46−
47−\si{\metre\squared\per\gray\cubic\lux}
48−
49−\si{\henry\second}
50−^D
51−<p>12345.67890</p>
52−<p>1 ± 2i</p>
53−<p>0.3 × 10<sup>45</sup></p>
54−<p>1.654 × 2.34 × 3.430</p>
55−<p>kg m s<sup>−1</sup></p>
56−<p>kg m s<sup>−1</sup></p>
57−<p>kg m/s</p>
58−<p>kg m/A/s</p>
59−<p>10, 20, &amp; 30</p>
60−<p>0.13 mm, 0.67 mm, &amp; 0.80 mm</p>
61−<p>10–20</p>
62−<p>0.13 mm–0.67 mm</p>
63−<p>10°</p>
64−<p>1°2′3″</p>
65−<p>1″</p>
66−<p>10°</p>
67−<p>-0°1′</p>
68−<p>kg m/s<sup>2</sup></p>
69−<p>g<sub>polymer</sub> mol<sub>cat</sub> s<sup>−1</sup></p>
70−<p>kg m s<sup>−2</sup></p>
71−<p>g cm<sup>−3</sup></p>
72−<p>V<sup>2</sup> lm<sup>3</sup> F<sup>−1</sup></p>
73−<p>m<sup>2</sup> Gy<sup>−1</sup> lx<sup>3</sup></p>
74−<p>H s</p>
75−```
test/command/6709.mddeleted−11
@@ -1,11 +0,0 @@
1−Tabs must be expanded even if --file-scope is used
2−````
3−% pandoc -t native --file-scope --tab-stop=2
4−```
5−if true; then
6− echo "yup"
7−fi
8−```
9−^D
10−[CodeBlock ("",[],[]) "if true; then\n echo \"yup\"\nfi"]
11−````
test/command/6890.md+2−2
@@ -23,12 +23,12 @@ references:
2323
2424@fruchtel-sozialer-2013a
2525
26−Some text.[^1]
26+Some text [^1].
2727
2828[^1]: @fruchtel-sozialer-2013a
2929^D
3030[Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 1, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]
31−,Para [Str "Some",Space,Str "text.",Note [Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]]]
31+,Para [Str "Some",Space,Str "text",Str ".",Note [Para [Cite [Citation {citationId = "fruchtel-sozialer-2013a", citationPrefix = [], citationSuffix = [], citationMode = AuthorInText, citationNoteNum = 2, citationHash = 0}] [Str "Fr\252chtel,",Space,Str "Budde,",Space,Str "and",Space,Str "Cyprian",Space,Str "(2013)"]]],Str ""]
3232,Div ("refs",["references","csl-bib-body","hanging-indent"],[])
3333 [Div ("ref-fruchtel-sozialer-2013a",["csl-entry"],[])
3434 [Para [Str "Fr\252chtel,",Space,Str "Frank,",Space,Str "Wolfgang",Space,Str "Budde,",Space,Str "and",Space,Str "Gudrun",Space,Str "Cyprian.",Space,Str "2013.",Space,Emph [Str "Sozialer",Space,Str "Raum",Space,Str "und",Space,Str "Soziale",Space,Str "Arbeit",Space,Str "Fieldbook:",Space,Str "Methoden",Space,Str "und",Space,Str "Techniken"],Str ".",Space,Str "3rd",Space,Str "ed.",Space,Str "Wiesbaden,",Space,Str "Germany:",Space,Str "Springer",Space,Str "VS."]]]]
test/command/6925.md+2−2
@@ -20,13 +20,13 @@ a
2020\end{thm2}
2121\end{document}
2222^D
23−::: thm
23+::: {.thm}
2424**Theorem 1**. *a*
2525
2626![image](1.png)
2727:::
2828
29−::: thm2
29+::: {.thm2}
3030**Theorem 1**. a
3131
3232![image](1.png)
test/command/6958.mddeleted−10
@@ -1,10 +0,0 @@
1−Add thin space between single and double quotes.
2−```
3−% pandoc -t latex+smart
4−---
5−lang: en-GB
6−---
7−'["On the Outside"]{}: Constructing Cycling Citizenship.'
8−^D
9−`\,{``On the Outside''}: Constructing Cycling Citizenship.'
10−```
test/command/7041.mddeleted−23
@@ -1,23 +0,0 @@
1−```
2−% pandoc -f html -t jats
3−<table>
4− <tr><td><blockquote>Fly, you fools!</blockquote></td></tr>
5−</table>
6−^D
7−<table-wrap>
8− <table>
9− <colgroup>
10− <col width="100%" />
11− </colgroup>
12− <tbody>
13− <tr>
14− <td><p specific-use="wrapper">
15− <disp-quote>
16− <p>Fly, you fools!</p>
17− </disp-quote>
18− </p></td>
19− </tr>
20− </tbody>
21− </table>
22−</table-wrap>
23−```
test/command/7099.md+1−1
@@ -11,7 +11,7 @@
1111<iframe src="h:invalid@url"></iframe>
1212^D
1313[INFO] Fetching h:invalid@url...
14−[WARNING] Could not fetch resource h:invalid@url: Could not fetch h:invalid@url
14+[WARNING] Could not fetch resource 'h:invalid@url': Could not fetch h:invalid@url
1515 InvalidUrlException "h:invalid@url" "Invalid scheme"
1616[INFO] Skipped '<iframe src="h:invalid@url"></iframe>' at input line 1 column 29
1717[]
test/command/7132.mddeleted−10
@@ -1,10 +0,0 @@
1−```
2−% pandoc -f markdown -t org --columns=72
3−- This line has exactly the wrong number of characters before the number 5.
4−- Long line ending with a number (this time it is in parentheses and a 23)
5−^D
6−- This line has exactly the wrong number of characters before the
7− number 5.
8−- Long line ending with a number (this time it is in parentheses and
9− a 23)
10−```
test/command/7134.mddeleted−16
@@ -1,16 +0,0 @@
1−```
2−% pandoc -f rst -t native
3−This is a paragraph.
4−
5− This is a block quote.
6−
7−..
8−
9− This should be a second block quote.
10−^D
11−[Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "paragraph."]
12−,BlockQuote
13− [Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "block",Space,Str "quote."]]
14−,BlockQuote
15− [Para [Str "This",Space,Str "should",Space,Str "be",Space,Str "a",Space,Str "second",Space,Str "block",Space,Str "quote."]]]
16−```
test/command/7145.mddeleted−12
@@ -1,12 +0,0 @@
1−```
2−% pandoc -f mediawiki -t native
3−Maecenas at sapien tempor, pretium turpis ut, imperdiet augue.<ref>This is a multiline
4−
5−reference
6−<i>with</i>
7−empty
8−
9−linebreaks</ref> Nulla ut massa eget ex venenatis lobortis id in eros.
10−^D
11−[Para [Str "Maecenas",Space,Str "at",Space,Str "sapien",Space,Str "tempor,",Space,Str "pretium",Space,Str "turpis",Space,Str "ut,",Space,Str "imperdiet",Space,Str "augue.",Note [Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "multiline"],Para [Str "reference",SoftBreak,RawInline (Format "html") "<i>",Str "with",RawInline (Format "html") "</i>",SoftBreak,Str "empty"],Para [Str "linebreaks"]],Space,Str "Nulla",Space,Str "ut",Space,Str "massa",Space,Str "eget",Space,Str "ex",Space,Str "venenatis",Space,Str "lobortis",Space,Str "id",Space,Str "in",Space,Str "eros."]]
12−```
test/command/7155.mddeleted−15
@@ -1,15 +0,0 @@
1−```
2−% pandoc -f markdown+tex_math_single_backslash -t native
3−\(x\) \[x\]
4−\\(x\\) \\[x\\]
5−^D
6−[Para [Math InlineMath "x",Space,Math DisplayMath "x",SoftBreak,Str "\\(x\\)",Space,Str "\\[x\\]"]]
7−```
8−
9−```
10−% pandoc -f markdown+tex_math_double_backslash -t native
11−\(x\) \[x\]
12−\\(x\\) \\[x\\]
13−^D
14−[Para [Str "(x)",Space,Str "[x]",SoftBreak,Math InlineMath "x",Space,Math DisplayMath "x"]]
15−```
test/command/7173.mddeleted−137
@@ -1,137 +0,0 @@
1−```
2−% pandoc -f docbook -t latex
3−<?xml version="1.0" encoding="UTF-8"?>
4−<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
5−<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
6−<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
7−<article xmlns="http://docbook.org/ns/docbook"
8− xmlns:xlink="http://www.w3.org/1999/xlink"
9− version="5.1">
10− <title>Untitled Document</title>
11− <para>Word</para>
12− <informalequation>
13− <alt role='tex'>1+2</alt>
14− <m:math xmlns:m="http://www.w3.org/1998/Math/MathML">
15− <m:mrow>
16− <m:mrow><m:mn>1</m:mn><m:mo>+</m:mo><m:mn>2</m:mn>
17− </m:mrow>
18− </m:mrow>
19− </m:math>
20− </informalequation>
21−</article>
22−^D
23−Word
24−
25−\[1 + 2\]
26−```
27−
28−```
29−% pandoc -f docbook -t latex
30−<?xml version="1.0" encoding="UTF-8"?>
31−<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
32−<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
33−<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
34−<article xmlns="http://docbook.org/ns/docbook"
35− xmlns:xlink="http://www.w3.org/1999/xlink"
36− version="5.1">
37− <title>Untitled Document</title>
38− <para>Word</para>
39− <informalequation>
40− <alt role='tex'>1+2</alt>
41− <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML">
42− <mml:mrow>
43− <mml:mrow><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mn>2</mml:mn>
44− </mml:mrow>
45− </mml:mrow>
46− </mml:math>
47− </informalequation>
48−</article>
49−^D
50−Word
51−
52−\[1 + 2\]
53−```
54−
55−```
56−% pandoc -f docbook -t latex
57−<?xml version="1.0" encoding="UTF-8"?>
58−<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
59−<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
60−<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
61−<article xmlns="http://docbook.org/ns/docbook"
62− xmlns:xlink="http://www.w3.org/1999/xlink"
63− version="5.1">
64− <title>Untitled Document</title>
65− <para>Word</para>
66− <informalequation>
67− <alt role='tex'>1+2</alt>
68− <math xmlns="http://www.w3.org/1998/Math/MathML">
69− <mrow>
70− <mrow><mn>1</mn><mo>+</mo><mn>2</mn>
71− </mrow>
72− </mrow>
73− </math>
74− </informalequation>
75−</article>
76−^D
77−Word
78−
79−\[1 + 2\]
80−```
81−
82−```
83−% pandoc -f docbook -t latex
84−<?xml version="1.0" encoding="UTF-8"?>
85−<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
86−<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
87−<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
88−<article xmlns="http://docbook.org/ns/docbook"
89− xmlns:xlink="http://www.w3.org/1999/xlink"
90− xmlns:m="http://www.w3.org/1998/Math/MathML"
91− version="5.1">
92− <title>Untitled Document</title>
93− <para>Word</para>
94− <informalequation>
95− <alt role='tex'>1+2</alt>
96− <m:math>
97− <m:mrow>
98− <m:mrow><m:mn>1</m:mn><m:mo>+</m:mo><m:mn>2</m:mn>
99− </m:mrow>
100− </m:mrow>
101− </m:math>
102− </informalequation>
103−</article>
104−^D
105−Word
106−
107−\[1 + 2\]
108−```
109−
110−```
111−% pandoc -f docbook -t latex
112−<?xml version="1.0" encoding="UTF-8"?>
113−<!DOCTYPE article PUBLIC "-//W3C//DTD MathML 2.0//EN" "http://www.w3.org/Math/DTD/mathml2/mathml2.dtd">
114−<?xml-model href="http://www.oxygenxml.com/docbook/xml/5.0/rng/dbmathml.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
115−<?xml-model href="http://docbook.org/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
116−<article xmlns="http://docbook.org/ns/docbook"
117− xmlns:xlink="http://www.w3.org/1999/xlink"
118− xmlns:mml="http://www.w3.org/1998/Math/MathML"
119− version="5.1">
120− <title>Untitled Document</title>
121− <para>Word</para>
122− <informalequation>
123− <alt role='tex'>1+2</alt>
124− <mml:math>
125− <mml:mrow>
126− <mml:mrow><mml:mn>1</mml:mn><mml:mo>+</mml:mo><mml:mn>2</mml:mn>
127− </mml:mrow>
128− </mml:mrow>
129− </mml:math>
130− </informalequation>
131−</article>
132−^D
133−Word
134−
135−\[1 + 2\]
136−```
137−
test/command/7208.mddeleted−6
@@ -1,6 +0,0 @@
1−```
2−% pandoc -t gfm
3−\<hi\>
4−^D
5−\<hi>
6−```
test/command/7214.mddeleted−28
@@ -1,28 +0,0 @@
1−```
2−% pandoc
3−+------------+----------+------------------+
4−|日本語 | の文字列 | words in english |
5−+------------+----------+------------------+
6−|abc defghij | def | xyz |
7−+------------+----------+------------------+
8−^D
9−<table style="width:60%;">
10−<colgroup>
11−<col style="width: 18%" />
12−<col style="width: 15%" />
13−<col style="width: 26%" />
14−</colgroup>
15−<tbody>
16−<tr class="odd">
17−<td>日本語</td>
18−<td>の文字列</td>
19−<td>words in english</td>
20−</tr>
21−<tr class="even">
22−<td>abc defghij</td>
23−<td>def</td>
24−<td>xyz</td>
25−</tr>
26−</tbody>
27−</table>
28−```
test/command/7216.mddeleted−19
@@ -1,19 +0,0 @@
1−```
2−pandoc -t latex
3−"This is some text in quotes. Another paragraph by the same speaker follows. The first paragraph should have no close quote.
4−
5−"The second paragraph should have open and close quotes."
6−
7−| "Open quote on this line,
8−| Close quote on the next line."
9−| "Quotes on the same line."
10−^D
11−``This is some text in quotes. Another paragraph by the same speaker
12−follows. The first paragraph should have no close quote.
13−
14−``The second paragraph should have open and close quotes.''
15−
16−``Open quote on this line,\\
17−Close quote on the next line.''\\
18−``Quotes on the same line.''
19−```
test/command/7266.mddeleted−10
@@ -1,10 +0,0 @@
1−```
2−% pandoc -f biblatex -t biblatex -s
3−@article{id,
4− annote = "annotation"
5−}
6−^D
7−@article{id,
8− annote = {annotation}
9−}
10−```
test/command/7272.mddeleted−27
@@ -1,27 +0,0 @@
1−```
2−% pandoc -t latex -f html
3−<table>
4− <tbody>
5− <tr>
6− <td>
7− <span>
8− text
9− <br />
10− text2
11− </span>
12− </td>
13− </tr>
14− </tbody>
15−</table>
16−^D
17−\begin{longtable}[]{@{}
18− >{\raggedright\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{1.00}}@{}}
19−\toprule
20−\endhead
21−\begin{minipage}[t]{\linewidth}\raggedright
22−{ text\\
23−text2 }\strut
24−\end{minipage} \\
25−\bottomrule
26−\end{longtable}
27−```
test/command/7278.mddeleted−38
@@ -1,38 +0,0 @@
1−```
2−% pandoc -t beamer
3−# Slide
4−
5−Some blocks:
6−
7−## example block title {.example}
8−
9−text in block
10−
11−## alert block title {.alert}
12−
13−text in block
14−
15−## block title
16−
17−text in block
18−^D
19−\begin{frame}{Slide}
20−\protect\hypertarget{slide}{}
21−Some blocks:
22−
23−\begin{exampleblock}{example block title}
24−\protect\hypertarget{example-block-title}{}
25−text in block
26−\end{exampleblock}
27−
28−\begin{alertblock}{alert block title}
29−\protect\hypertarget{alert-block-title}{}
30−text in block
31−\end{alertblock}
32−
33−\begin{block}{block title}
34−\protect\hypertarget{block-title}{}
35−text in block
36−\end{block}
37−\end{frame}
38−```
test/command/7282.mddeleted−7
@@ -1,7 +0,0 @@
1−Don't crash on unmatched closing tag.
2−```
3−% pandoc -f html -t native
4−</script>
5−^D
6−[]
7−```
test/command/7288.mddeleted−40
@@ -1,40 +0,0 @@
1−```
2−% pandoc -f rst -t ms
3−.. list-table::
4− :widths: 50 50
5− :header-rows: 1
6−
7− * - Left
8− - Right
9− * - Long text that should be easy to break up into multiple lines
10− - Another long text that should be easy to break up into multiple lines
11−
12− Bar
13−^D
14−.PP
15−.na
16−.nr LLold \n[LL]
17−.TS
18−delim(@@) tab( );
19−lw(35.0n) lw(35.0n).
20−T{
21−Left
22−T} T{
23−Right
24−T}
25−_
26−T{
27−.nr LL 35.0n
28−.LP
29−Long text that should be easy to break up into multiple lines
30−T} T{
31−.nr LL 35.0n
32−.PP
33−Another long text that should be easy to break up into multiple lines
34−.PP
35−Bar
36−T}
37−.TE
38−.nr LL \n[LLold]
39−.ad
40−```
test/command/7299.mddeleted−23
@@ -1,23 +0,0 @@
1−```
2−% pandoc -f latex -t plain
3−$1-{\ensuremath{r}\xspace}$
4−^D
5−1 − r
6−```
7−
8−```
9−% pandoc -f latex -t plain
10−\newcommand{\foo}{Foo\xspace}
11−
12−$\text{\foo bar}$
13−^D
14−Foo bar
15−```
16−
17−```
18−% pandoc -f latex -t plain
19−a\xspace b
20−^D
21−a b
22−```
23−
test/command/7321.mddeleted−24
@@ -1,24 +0,0 @@
1−```
2−% pandoc -t plain --citeproc --wrap=none
3−---
4−references:
5−- id: fenner2012a
6− title: One-click science marketing
7− author:
8− - family: Fenner
9− given: Martin
10− container-title: Nature Materials
11− volume: 11
12− issue: 4
13− publisher: Nature Publishing Group
14− type: article-journal
15− issued:
16− year: 2012
17−---
18−
19−[@fenner2012a, $a^2+b^2=c^2$]
20−^D
21−(Fenner 2012, a² + b² = c²)
22−
23−Fenner, Martin. 2012. “One-Click Science Marketing.” Nature Materials 11 (4).
24−```
test/command/7323.mddeleted−29
@@ -1,29 +0,0 @@
1−```
2−% pandoc --citeproc -t plain
3−---
4−references:
5−- id: smith
6− author: John Smith
7− issued: 2019
8− title: Insects
9− type: book
10−...
11−
12−@smith [chap. 6]
13−
14−@smith [chapter 6]
15−
16−@smith [Chap. 6]
17−
18−@smith [Chapter 6]
19−^D
20−John Smith (2019, chap. 6)
21−
22−John Smith (2019, chap. 6)
23−
24−John Smith (2019, chap. 6)
25−
26−John Smith (2019, chap. 6)
27−
28−John Smith. 2019. Insects.
29−```
test/command/7324.mddeleted−25
@@ -1,25 +0,0 @@
1−```
2−% pandoc --citeproc -t plain
3−---
4−references:
5−- id: smith
6− author: John Smith
7− issued: 2019
8− title: Insects
9− type: book
10−...
11−
12−@smith [, among others]
13−
14−@smith [ among others]
15−
16−@smith [among others]
17−^D
18−John Smith (2019, among others)
19−
20−John Smith (2019 among others)
21−
22−John Smith (2019, among others)
23−
24−John Smith. 2019. Insects.
25−```
test/command/7339.mddeleted−11
@@ -1,11 +0,0 @@
1−```
2−% pandoc -f gfm -s -t native
3−---
4−title: Test
5−---
6−
7−Hi
8−^D
9−Pandoc (Meta {unMeta = fromList [("title",MetaInlines [Str "Test"])]})
10−[Para [Str "Hi"]]
11−```
test/command/7340.mddeleted−6
@@ -1,6 +0,0 @@
1−```
2−% pandoc -f latex -t native
3−\(*\)
4−^D
5−[Para [Math InlineMath "*"]]
6−```
test/command/7376.mddeleted−16
@@ -1,16 +0,0 @@
1−```
2−% pandoc --citeproc -t plain
3−---
4−references:
5−- id: item1
6− type: book
7− author:
8− - family: Doe
9− given: Jane
10− issued: 2020
11− title: The title
12−...
13−^D
14−
15−
16−```
test/command/7394.mddeleted−85
@@ -1,85 +0,0 @@
1−```
2−% pandoc -f markdown -t plain --citeproc
3−---
4−csl: command/chicago-fullnote-bibliography.csl
5−references:
6−- author:
7− - family: Wandt
8− given: Manfred
9− edition: 6
10− id: wandt2014ges-sv
11− issued: 2014
12− publisher: Franz Vahlen
13− publisher-place: München
14− title: Gesetzliche schuldverhältnisse
15− title-short: Gesetzl SV
16− type: book
17−- author:
18− - family: Smith
19− given: Zenda
20− edition: 6
21− id: smith2015
22− issued: 2015
23− publisher: Macmillan
24− publisher-place: New York
25− title: A verb and a noun
26− type: book
27−---
28−
29−Hi^[@wandt2014ges-sv.].
30−
31−Hi^[[@wandt2014ges-sv].].
32−
33−Hi^[[See also @wandt2014ges-sv].].
34−
35−Hi^[See also @wandt2014ges-sv.].
36−
37−Hi^[@wandt2014ges-sv [@smith2015].].
38−
39−Hi^[[@wandt2014ges-sv; @smith2015].].
40−
41−Hi [@wandt2014ges-sv].
42−
43−Hi [see also @wandt2014ges-sv].
44−
45−^D
46−Hi[1].
47−
48−Hi[2].
49−
50−Hi[3].
51−
52−Hi[4].
53−
54−Hi[5].
55−
56−Hi[6].
57−
58−Hi.[7]
59−
60−Hi.[8]
61−
62−Smith, Zenda. A Verb and a Noun. 6th ed. New York: Macmillan, 2015.
63−
64−Wandt, Manfred. Gesetzliche Schuldverhältnisse. 6th ed. München: Franz
65−Vahlen, 2014.
66−
67−[1] Manfred Wandt, Gesetzliche Schuldverhältnisse, 6th ed. (München:
68−Franz Vahlen, 2014).
69−
70−[2] Wandt.
71−
72−[3] See also Wandt.
73−
74−[4] See also Wandt.
75−
76−[5] Wandt, Zenda Smith, A Verb and a Noun, 6th ed. (New York: Macmillan,
77−2015).
78−
79−[6] Wandt, Gesetzl SV; Smith, A Verb and a Noun.
80−
81−[7] Wandt, Gesetzl SV.
82−
83−[8] See also Wandt.
84−```
85−
test/command/7397.mddeleted−14
@@ -1,14 +0,0 @@
1−````
2−% pandoc -t markdown
3−~~~~ { .haskell startFrom="100"}
4−qsort [] = []
5−qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++
6− qsort (filter (>= x) xs)
7−~~~~
8−^D
9−``` {.haskell startFrom="100"}
10−qsort [] = []
11−qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++
12− qsort (filter (>= x) xs)
13−```
14−````
test/command/7400.mddeleted−9
@@ -1,9 +0,0 @@
1−```
2−% pandoc -t native -s
3−---
4−# Comment only
5−...
6−^D
7−Pandoc (Meta {unMeta = fromList []})
8−[]
9−```
test/command/7416.mddeleted−19
@@ -1,19 +0,0 @@
1−```
2−% pandoc -f markdown -t html
3−![caption](../media/rId25.jpg "title"){alt="alt"}
4−
5−^D
6−<figure>
7−<img src="../media/rId25.jpg" title="title" alt="alt" /><figcaption>caption</figcaption>
8−</figure>
9−```
10−
11−```
12−% pandoc -f markdown -t html
13−![caption](../media/rId25.jpg "title")
14−
15−^D
16−<figure>
17−<img src="../media/rId25.jpg" title="title" alt="caption" /><figcaption aria-hidden="true">caption</figcaption>
18−</figure>
19−```
test/command/averroes.bibdeleted−95
@@ -1,95 +0,0 @@
1−@string{anch-ie = {Angew.~Chem. Int.~Ed.}}
2−@string{cup = {Cambridge University Press}}
3−@string{dtv = {Deutscher Taschenbuch-Verlag}}
4−@string{hup = {Harvard University Press}}
5−@string{jams = {J.~Amer. Math. Soc.}}
6−@string{jchph = {J.~Chem. Phys.}}
7−@string{jomch = {J.~Organomet. Chem.}}
8−@string{pup = {Princeton University Press}}
9−
10−@book{averroes/bland,
11− author = {Averroes},
12− title = {The Epistle on the Possibility of Conjunction with the Active
13− Intellect by {Ibn Rushd} with the Commentary of {Moses Narboni}},
14− date = 1982,
15− editor = {Bland, Kalman P.},
16− translator = {Bland, Kalman P.},
17− series = {Moreshet: Studies in {Jewish} History, Literature and Thought},
18− number = 7,
19− publisher = {Jewish Theological Seminary of America},
20− location = {New York},
21− keywords = {primary},
22− langid = {english},
23− langidopts = {variant=american},
24− indextitle = {Epistle on the Possibility of Conjunction, The},
25− shorttitle = {Possibility of Conjunction},
26− annotation = {A \texttt{book} entry with a \texttt{series} and a
27− \texttt{number}. Note the concatenation of the \texttt{editor}
28− and \texttt{translator} fields as well as the
29− \texttt{indextitle} field},
30−}
31−
32−@book{averroes/hannes,
33− author = {Averroes},
34− title = {Des Averro{\"e}s Abhandlung: \mkbibquote{{\"U}ber die
35− M{\"o}glichkeit der Conjunktion} oder \mkbibquote{{\"U}ber den
36− materiellen Intellekt}},
37− date = 1892,
38− editor = {Hannes, Ludwig},
39− translator = {Hannes, Ludwig},
40− annotator = {Hannes, Ludwig},
41− publisher = {C.~A. Kaemmerer},
42− location = {Halle an der Saale},
43− keywords = {primary},
44− langid = {german},
45− sorttitle = {Uber die Moglichkeit der Conjunktion},
46− indexsorttitle= {Uber die Moglichkeit der Conjunktion},
47− indextitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
48− shorttitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
49− annotation = {An annotated edition. Note the concatenation of the
50− \texttt{editor}, \texttt{translator}, and \texttt{annotator}
51− fields. Also note the \texttt{shorttitle},
52− \texttt{indextitle}, \texttt{sorttitle}, and
53− \texttt{indexsorttitle} fields},
54−}
55−
56−@book{averroes/hercz,
57− author = {Averroes},
58− title = {Drei Abhandlungen {\"u}ber die Conjunction des separaten
59− Intellects mit dem Menschen},
60− date = 1869,
61− editor = {Hercz, J.},
62− translator = {Hercz, J.},
63− publisher = {S.~Hermann},
64− location = {Berlin},
65− keywords = {primary},
66− langid = {german},
67− indexsorttitle= {Drei Abhandlungen uber die Conjunction},
68− indextitle = {Drei Abhandlungen {\"u}ber die Conjunction},
69− subtitle = {Von Averroes (Vater und Sohn), aus dem Arabischen
70− {\"u}bersetzt von Samuel Ibn Tibbon},
71− shorttitle = {Drei Abhandlungen},
72− annotation = {A \texttt{book} entry. Note the concatenation of the
73− \texttt{editor} and \texttt{translator} fields as well as the
74− \texttt{indextitle} and \texttt{indexsorttitle} fields},
75−}
76−
77−@book{cicero,
78− author = {Cicero, Marcus Tullius},
79− title = {De natura deorum. {\"U}ber das Wesen der G{\"o}tter},
80− date = 1995,
81− editor = {Blank-Sangmeister, Ursula},
82− translator = {Blank-Sangmeister, Ursula},
83− afterword = {Thraede, Klaus},
84− language = {langlatin and langgerman},
85− publisher = {Reclam},
86− location = {Stuttgart},
87− langid = {german},
88− indextitle = {De natura deorum},
89− shorttitle = {De natura deorum},
90− annotation = {A bilingual edition of Cicero's \emph{De natura deorum}, with
91− a German translation. Note the format of the \texttt{language}
92− field in the database file, the concatenation of the
93− \texttt{editor} and \texttt{translator} fields, and the
94− \texttt{afterword} field},
95−}
test/command/biblatex-basic.md+3−5
@@ -1,5 +1,3 @@
1−```
2−% pandoc -f biblatex -t markdown -s
31@Book{item1,
42author="John Doe",
53title="First Book",
@@ -29,7 +27,7 @@ year="2007"
2927
3028^D
3129---
32−nocite: "[@*]"
30+nocite: '[@*]'
3331references:
3432- author:
3533 - family: Doe
@@ -46,9 +44,9 @@ references:
4644 container-title: Journal of Generic Studies
4745 id: item2
4846 issued: 2006
49− page: 33-34
47+ page: '33-34'
5048 title: Article
51− type: article-journal
49+ type: 'article-journal'
5250 volume: 6
5351- author:
5452 - family: Doe
test/command/biblatex-examples.bibnew file+1674
@@ -0,0 +1,1674 @@
1+@string{anch-ie = {Angew.~Chem. Int.~Ed.}}
2+@string{cup = {Cambridge University Press}}
3+@string{dtv = {Deutscher Taschenbuch-Verlag}}
4+@string{hup = {Harvard University Press}}
5+@string{jams = {J.~Amer. Math. Soc.}}
6+@string{jchph = {J.~Chem. Phys.}}
7+@string{jomch = {J.~Organomet. Chem.}}
8+@string{pup = {Princeton University Press}}
9+
10+@incollection{westfahl:space,
11+ author = {Westfahl, Gary},
12+ title = {The True Frontier},
13+ subtitle = {Confronting and Avoiding the Realities of Space in {American}
14+ Science Fiction Films},
15+ pages = {55-65},
16+ crossref = {westfahl:frontier},
17+ langid = {english},
18+ langidopts = {variant=american},
19+ indextitle = {True Frontier, The},
20+ annotation = {A cross-referenced article from a \texttt{collection}. This is
21+ an \texttt{incollection} entry with a \texttt{crossref}
22+ field. Note the \texttt{subtitle} and \texttt{indextitle}
23+ fields},
24+}
25+
26+@set{set,
27+ entryset = {herrmann,aksin,yoon},
28+ annotation = {A \texttt{set} with three members. The \texttt{crossref} field
29+ in the \texttt{@set} entry and the \texttt{entryset} field in
30+ each set member entry is needed only when using BibTeX as the
31+ backend},
32+}
33+
34+@set{stdmodel,
35+ entryset = {glashow,weinberg,salam},
36+ annotation = {A \texttt{set} with three members discussing the standard
37+ model of particle physics. The \texttt{crossref} field
38+ in the \texttt{@set} entry and the \texttt{entryset} field in
39+ each set member entry is needed only when using BibTeX as the
40+ backend},
41+}
42+
43+@article{aksin,
44+ author = {Aks{\i}n, {\"O}zge and T{\"u}rkmen, Hayati and Artok, Levent
45+ and {\c{C}}etinkaya, Bekir and Ni, Chaoying and
46+ B{\"u}y{\"u}kg{\"u}ng{\"o}r, Orhan and {\"O}zkal, Erhan},
47+ title = {Effect of immobilization on catalytic characteristics of
48+ saturated {Pd-N}-heterocyclic carbenes in {Mizoroki-Heck}
49+ reactions},
50+ journaltitle = jomch,
51+ date = 2006,
52+ volume = 691,
53+ number = 13,
54+ pages = {3027-3036},
55+ indextitle = {Effect of immobilization on catalytic characteristics},
56+}
57+
58+@article{angenendt,
59+ author = {Angenendt, Arnold},
60+ title = {In Honore Salvatoris~-- Vom Sinn und Unsinn der
61+ Patrozinienkunde},
62+ journaltitle = {Revue d'Histoire Eccl{\'e}siastique},
63+ date = 2002,
64+ volume = 97,
65+ pages = {431--456, 791--823},
66+ langid = {german},
67+ indextitle = {In Honore Salvatoris},
68+ shorttitle = {In Honore Salvatoris},
69+ annotation = {A German article in a French journal. Apart from that, a
70+ typical \texttt{article} entry. Note the \texttt{indextitle}
71+ field},
72+}
73+
74+@article{baez/article,
75+ author = {Baez, John C. and Lauda, Aaron D.},
76+ title = {Higher-Dimensional Algebra {V}: 2-Groups},
77+ journaltitle = {Theory and Applications of Categories},
78+ date = 2004,
79+ volume = 12,
80+ pages = {423-491},
81+ version = 3,
82+ eprint = {math/0307200v3},
83+ eprinttype = {arxiv},
84+ langid = {english},
85+ langidopts = {variant=american},
86+ annotation = {An \texttt{article} with \texttt{eprint} and
87+ \texttt{eprinttype} fields. Note that the arXiv reference is
88+ transformed into a clickable link if \texttt{hyperref} support
89+ has been enabled. Compare \texttt{baez\slash online}, which
90+ is the same item given as an \texttt{online} entry},
91+}
92+
93+@article{bertram,
94+ author = {Bertram, Aaron and Wentworth, Richard},
95+ title = {Gromov invariants for holomorphic maps on {Riemann} surfaces},
96+ journaltitle = jams,
97+ date = 1996,
98+ volume = 9,
99+ number = 2,
100+ pages = {529-571},
101+ langid = {english},
102+ langidopts = {variant=american},
103+ shorttitle = {Gromov invariants},
104+ annotation = {An \texttt{article} entry with a \texttt{volume} and a
105+ \texttt{number} field},
106+}
107+
108+@article{doody,
109+ author = {Doody, Terrence},
110+ title = {Hemingway's Style and {Jake's} Narration},
111+ year = 1974,
112+ volume = 4,
113+ number = 3,
114+ pages = {212-225},
115+ langid = {english},
116+ langidopts = {variant=american},
117+ related = {matuz:doody},
118+ relatedstring= {\autocap{e}xcerpt in},
119+ journal = {The Journal of Narrative Technique},
120+ annotation = {An \texttt{article} entry cited as an excerpt from a
121+ \texttt{collection} entry. Note the format of the
122+ \texttt{related} and \texttt{relatedstring} fields},
123+}
124+
125+@collection{matuz:doody,
126+ editor = {Matuz, Roger},
127+ title = {Contemporary Literary Criticism},
128+ year = 1990,
129+ volume = 61,
130+ publisher = {Gale},
131+ location = {Detroit},
132+ pages = {204-208},
133+ langid = {english},
134+ langidopts = {variant=american},
135+ annotation = {A \texttt{collection} entry providing the excerpt information
136+ for the \texttt{doody} entry. Note the format of the
137+ \texttt{pages} field},
138+}
139+
140+@article{gillies,
141+ author = {Gillies, Alexander},
142+ title = {Herder and the Preparation of {Goethe's} Idea of World
143+ Literature},
144+ journaltitle = {Publications of the English Goethe Society},
145+ date = 1933,
146+ series = {newseries},
147+ volume = 9,
148+ pages = {46-67},
149+ langid = {english},
150+ langidopts = {variant=british},
151+ annotation = {An \texttt{article} entry with a \texttt{series} and a
152+ \texttt{volume} field. Note that format of the \texttt{series}
153+ field in the database file},
154+}
155+
156+@article{glashow,
157+ author = {Glashow, Sheldon},
158+ title = {Partial Symmetries of Weak Interactions},
159+ journaltitle = {Nucl.~Phys.},
160+ date = 1961,
161+ volume = 22,
162+ pages = {579-588},
163+}
164+
165+@article{herrmann,
166+ author = {Herrmann, Wolfgang A. and {\"O}fele, Karl and Schneider,
167+ Sabine K. and Herdtweck, Eberhardt and Hoffmann, Stephan D.},
168+ title = {A carbocyclic carbene as an efficient catalyst ligand for {C--C}
169+ coupling reactions},
170+ journaltitle = anch-ie,
171+ date = 2006,
172+ volume = 45,
173+ number = 23,
174+ pages = {3859-3862},
175+ indextitle = {Carbocyclic carbene as an efficient catalyst, A},
176+}
177+
178+@article{kastenholz,
179+ author = {Kastenholz, M. A. and H{\"u}nenberger, Philippe H.},
180+ title = {Computation of methodology\hyphen independent ionic solvation
181+ free energies from molecular simulations},
182+ journaltitle = jchph,
183+ date = 2006,
184+ subtitle = {{I}. {The} electrostatic potential in molecular liquids},
185+ volume = 124,
186+ eid = 124106,
187+ doi = {10.1063/1.2172593},
188+ langid = {english},
189+ langidopts = {variant=american},
190+ indextitle = {Computation of ionic solvation free energies},
191+ annotation = {An \texttt{article} entry with an \texttt{eid} and a
192+ \texttt{doi} field. Note that the \textsc{doi} is transformed
193+ into a clickable link if \texttt{hyperref} support has been
194+ enabled},
195+ abstract = {The computation of ionic solvation free energies from
196+ atomistic simulations is a surprisingly difficult problem that
197+ has found no satisfactory solution for more than 15 years. The
198+ reason is that the charging free energies evaluated from such
199+ simulations are affected by very large errors. One of these is
200+ related to the choice of a specific convention for summing up
201+ the contributions of solvent charges to the electrostatic
202+ potential in the ionic cavity, namely, on the basis of point
203+ charges within entire solvent molecules (M scheme) or on the
204+ basis of individual point charges (P scheme). The use of an
205+ inappropriate convention may lead to a charge-independent
206+ offset in the calculated potential, which depends on the
207+ details of the summation scheme, on the quadrupole-moment
208+ trace of the solvent molecule, and on the approximate form
209+ used to represent electrostatic interactions in the
210+ system. However, whether the M or P scheme (if any) represents
211+ the appropriate convention is still a matter of on-going
212+ debate. The goal of the present article is to settle this
213+ long-standing controversy by carefully analyzing (both
214+ analytically and numerically) the properties of the
215+ electrostatic potential in molecular liquids (and inside
216+ cavities within them).},
217+}
218+
219+@article{murray,
220+ author = {Hostetler, Michael J. and Wingate, Julia E. and Zhong,
221+ Chuan-Jian and Harris, Jay E. and Vachet, Richard W. and
222+ Clark, Michael R. and Londono, J. David and Green, Stephen
223+ J. and Stokes, Jennifer J. and Wignall, George D. and Glish,
224+ Gary L. and Porter, Marc D. and Evans, Neal D. and Murray,
225+ Royce W.},
226+ title = {Alkanethiolate gold cluster molecules with core diameters from
227+ 1.5 to 5.2~{nm}},
228+ journaltitle = {Langmuir},
229+ date = 1998,
230+ subtitle = {Core and monolayer properties as a function of core size},
231+ volume = 14,
232+ number = 1,
233+ pages = {17-30},
234+ langid = {english},
235+ langidopts = {variant=american},
236+ indextitle = {Alkanethiolate gold cluster molecules},
237+ shorttitle = {Alkanethiolate gold cluster molecules},
238+ annotation = {An \texttt{article} entry with \arabic{author} authors. By
239+ default, long author and editor lists are automatically
240+ truncated. This is configurable},
241+}
242+
243+@article{reese,
244+ author = {Reese, Trevor R.},
245+ title = {Georgia in {Anglo-Spanish} Diplomacy, 1736--1739},
246+ journaltitle = {William and Mary Quarterly},
247+ date = 1958,
248+ series = 3,
249+ volume = 15,
250+ pages = {168-190},
251+ langid = {english},
252+ langidopts = {variant=american},
253+ annotation = {An \texttt{article} entry with a \texttt{series} and a
254+ \texttt{volume} field. Note the format of the series. If the
255+ value of the \texttt{series} field is an integer, this number
256+ is printed as an ordinal and the string \enquote*{series} is
257+ appended automatically},
258+}
259+
260+@article{sarfraz,
261+ author = {M. Sarfraz and M. F. A. Razzak},
262+ title = {Technical section: {An} algorithm for automatic capturing of
263+ the font outlines},
264+ year = 2002,
265+ volume = 26,
266+ number = 5,
267+ pages = {795-804},
268+ issn = {0097-8493},
269+ journal = {Computers and Graphics},
270+ annotation = {An \texttt{article} entry with an \texttt{issn} field},
271+}
272+
273+@article{shore,
274+ author = {Shore, Bradd},
275+ title = {Twice-Born, Once Conceived},
276+ journaltitle = {American Anthropologist},
277+ date = {1991-03},
278+ subtitle = {Meaning Construction and Cultural Cognition},
279+ series = {newseries},
280+ volume = 93,
281+ number = 1,
282+ pages = {9-27},
283+ annotation = {An \texttt{article} entry with \texttt{series},
284+ \texttt{volume}, and \texttt{number} fields. Note the format
285+ of the \texttt{series} which is a localization key},
286+}
287+
288+@article{sigfridsson,
289+ author = {Sigfridsson, Emma and Ryde, Ulf},
290+ title = {Comparison of methods for deriving atomic charges from the
291+ electrostatic potential and moments},
292+ journaltitle = {Journal of Computational Chemistry},
293+ date = 1998,
294+ volume = 19,
295+ number = 4,
296+ pages = {377-395},
297+ doi = {10.1002/(SICI)1096-987X(199803)19:4<377::AID-JCC1>3.0.CO;2-P},
298+ langid = {english},
299+ langidopts = {variant=american},
300+ indextitle = {Methods for deriving atomic charges},
301+ annotation = {An \texttt{article} entry with \texttt{volume},
302+ \texttt{number}, and \texttt{doi} fields. Note that the
303+ \textsc{doi} is transformed into a clickable link if
304+ \texttt{hyperref} support has been enabled},
305+ abstract = {Four methods for deriving partial atomic charges from the
306+ quantum chemical electrostatic potential (CHELP, CHELPG,
307+ Merz-Kollman, and RESP) have been compared and critically
308+ evaluated. It is shown that charges strongly depend on how and
309+ where the potential points are selected. Two alternative
310+ methods are suggested to avoid the arbitrariness in the
311+ point-selection schemes and van der Waals exclusion radii:
312+ CHELP-BOW, which also estimates the charges from the
313+ electrostatic potential, but with potential points that are
314+ Boltzmann-weighted after their occurrence in actual
315+ simulations using the energy function of the program in which
316+ the charges will be used, and CHELMO, which estimates the
317+ charges directly from the electrostatic multipole
318+ moments. Different criteria for the quality of the charges are
319+ discussed.},
320+}
321+
322+@article{spiegelberg,
323+ author = {Spiegelberg, Herbert},
324+ title = {\mkbibquote{Intention} und \mkbibquote{Intentionalit{\"a}t} in
325+ der Scholastik, bei Brentano und Husserl},
326+ journaltitle = {Studia Philosophica},
327+ date = 1969,
328+ volume = 29,
329+ pages = {189-216},
330+ langid = {german},
331+ sorttitle = {Intention und Intentionalitat in der Scholastik, bei Brentano
332+ und Husserl},
333+ indexsorttitle= {Intention und Intentionalitat in der Scholastik, bei Brentano
334+ und Husserl},
335+ shorttitle = {Intention und Intentionalit{\"a}t},
336+ annotation = {An \texttt{article} entry. Note the \texttt{sorttitle} and
337+ \texttt{indexsorttitle} fields and the markup of the quotes in
338+ the database file},
339+}
340+
341+@article{springer,
342+ author = {Springer, Otto},
343+ title = {Mediaeval Pilgrim Routes from {Scandinavia} to {Rome}},
344+ journaltitle = {Mediaeval Studies},
345+ date = 1950,
346+ volume = 12,
347+ pages = {92-122},
348+ langid = {english},
349+ langidopts = {variant=british},
350+ shorttitle = {Mediaeval Pilgrim Routes},
351+ annotation = {A plain \texttt{article} entry},
352+}
353+
354+@article{weinberg,
355+ author = {Weinberg, Steven},
356+ title = {A Model of Leptons},
357+ journaltitle = {Phys.~Rev.~Lett.},
358+ date = 1967,
359+ volume = 19,
360+ pages = {1264-1266},
361+}
362+
363+@article{yoon,
364+ author = {Yoon, Myeong S. and Ryu, Dowook and Kim, Jeongryul and Ahn,
365+ Kyo Han},
366+ title = {Palladium pincer complexes with reduced bond angle strain:
367+ efficient catalysts for the {Heck} reaction},
368+ journaltitle = {Organometallics},
369+ date = 2006,
370+ volume = 25,
371+ number = 10,
372+ pages = {2409-2411},
373+ indextitle = {Palladium pincer complexes},
374+}
375+
376+@book{aristotle:anima,
377+ author = {Aristotle},
378+ title = {De Anima},
379+ date = 1907,
380+ editor = {Hicks, Robert Drew},
381+ publisher = cup,
382+ location = {Cambridge},
383+ keywords = {primary},
384+ langid = {english},
385+ langidopts = {variant=british},
386+ annotation = {A \texttt{book} entry with an \texttt{author} and an
387+ \texttt{editor}},
388+}
389+
390+@book{aristotle:physics,
391+ author = {Aristotle},
392+ title = {Physics},
393+ date = 1929,
394+ translator = {Wicksteed, P. H. and Cornford, F. M.},
395+ publisher = {G. P. Putnam},
396+ location = {New York},
397+ keywords = {primary},
398+ langid = {english},
399+ langidopts = {variant=american},
400+ shorttitle = {Physics},
401+ annotation = {A \texttt{book} entry with a \texttt{translator} field},
402+}
403+
404+@book{aristotle:poetics,
405+ author = {Aristotle},
406+ title = {Poetics},
407+ date = 1968,
408+ editor = {Lucas, D. W.},
409+ series = {Clarendon {Aristotle}},
410+ publisher = {Clarendon Press},
411+ location = {Oxford},
412+ keywords = {primary},
413+ langid = {english},
414+ langidopts = {variant=british},
415+ shorttitle = {Poetics},
416+ annotation = {A \texttt{book} entry with an \texttt{author} and an
417+ \texttt{editor} as well as a \texttt{series} field},
418+}
419+
420+@book{aristotle:rhetoric,
421+ author = {Aristotle},
422+ title = {The Rhetoric of {Aristotle} with a commentary by the late {Edward
423+ Meredith Cope}},
424+ date = 1877,
425+ editor = {Cope, Edward Meredith},
426+ commentator = {Cope, Edward Meredith},
427+ volumes = 3,
428+ publisher = cup,
429+ keywords = {primary},
430+ langid = {english},
431+ langidopts = {variant=british},
432+ sorttitle = {Rhetoric of Aristotle},
433+ indextitle = {Rhetoric of {Aristotle}, The},
434+ shorttitle = {Rhetoric},
435+ annotation = {A commented edition. Note the concatenation of the
436+ \texttt{editor} and \texttt{commentator} fields as well as the
437+ \texttt{volumes}, \texttt{sorttitle}, and \texttt{indextitle}
438+ fields},
439+}
440+
441+@book{augustine,
442+ author = {Augustine, Robert L.},
443+ title = {Heterogeneous catalysis for the synthetic chemist},
444+ date = 1995,
445+ publisher = {Marcel Dekker},
446+ location = {New York},
447+ langid = {english},
448+ langidopts = {variant=american},
449+ shorttitle = {Heterogeneous catalysis},
450+ annotation = {A plain \texttt{book} entry},
451+}
452+
453+@book{averroes/bland,
454+ author = {Averroes},
455+ title = {The Epistle on the Possibility of Conjunction with the Active
456+ Intellect by {Ibn Rushd} with the Commentary of {Moses Narboni}},
457+ date = 1982,
458+ editor = {Bland, Kalman P.},
459+ translator = {Bland, Kalman P.},
460+ series = {Moreshet: Studies in {Jewish} History, Literature and Thought},
461+ number = 7,
462+ publisher = {Jewish Theological Seminary of America},
463+ location = {New York},
464+ keywords = {primary},
465+ langid = {english},
466+ langidopts = {variant=american},
467+ indextitle = {Epistle on the Possibility of Conjunction, The},
468+ shorttitle = {Possibility of Conjunction},
469+ annotation = {A \texttt{book} entry with a \texttt{series} and a
470+ \texttt{number}. Note the concatenation of the \texttt{editor}
471+ and \texttt{translator} fields as well as the
472+ \texttt{indextitle} field},
473+}
474+
475+@book{averroes/hannes,
476+ author = {Averroes},
477+ title = {Des Averro{\"e}s Abhandlung: \mkbibquote{{\"U}ber die
478+ M{\"o}glichkeit der Conjunktion} oder \mkbibquote{{\"U}ber den
479+ materiellen Intellekt}},
480+ date = 1892,
481+ editor = {Hannes, Ludwig},
482+ translator = {Hannes, Ludwig},
483+ annotator = {Hannes, Ludwig},
484+ publisher = {C.~A. Kaemmerer},
485+ location = {Halle an der Saale},
486+ keywords = {primary},
487+ langid = {german},
488+ sorttitle = {Uber die Moglichkeit der Conjunktion},
489+ indexsorttitle= {Uber die Moglichkeit der Conjunktion},
490+ indextitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
491+ shorttitle = {{\"U}ber die M{\"o}glichkeit der Conjunktion},
492+ annotation = {An annotated edition. Note the concatenation of the
493+ \texttt{editor}, \texttt{translator}, and \texttt{annotator}
494+ fields. Also note the \texttt{shorttitle},
495+ \texttt{indextitle}, \texttt{sorttitle}, and
496+ \texttt{indexsorttitle} fields},
497+}
498+
499+@book{averroes/hercz,
500+ author = {Averroes},
501+ title = {Drei Abhandlungen {\"u}ber die Conjunction des separaten
502+ Intellects mit dem Menschen},
503+ date = 1869,
504+ editor = {Hercz, J.},
505+ translator = {Hercz, J.},
506+ publisher = {S.~Hermann},
507+ location = {Berlin},
508+ keywords = {primary},
509+ langid = {german},
510+ indexsorttitle= {Drei Abhandlungen uber die Conjunction},
511+ indextitle = {Drei Abhandlungen {\"u}ber die Conjunction},
512+ subtitle = {Von Averroes (Vater und Sohn), aus dem Arabischen
513+ {\"u}bersetzt von Samuel Ibn Tibbon},
514+ shorttitle = {Drei Abhandlungen},
515+ annotation = {A \texttt{book} entry. Note the concatenation of the
516+ \texttt{editor} and \texttt{translator} fields as well as the
517+ \texttt{indextitle} and \texttt{indexsorttitle} fields},
518+}
519+
520+@book{cicero,
521+ author = {Cicero, Marcus Tullius},
522+ title = {De natura deorum. {\"U}ber das Wesen der G{\"o}tter},
523+ date = 1995,
524+ editor = {Blank-Sangmeister, Ursula},
525+ translator = {Blank-Sangmeister, Ursula},
526+ afterword = {Thraede, Klaus},
527+ language = {langlatin and langgerman},
528+ publisher = {Reclam},
529+ location = {Stuttgart},
530+ langid = {german},
531+ indextitle = {De natura deorum},
532+ shorttitle = {De natura deorum},
533+ annotation = {A bilingual edition of Cicero's \emph{De natura deorum}, with
534+ a German translation. Note the format of the \texttt{language}
535+ field in the database file, the concatenation of the
536+ \texttt{editor} and \texttt{translator} fields, and the
537+ \texttt{afterword} field},
538+}
539+
540+@book{coleridge,
541+ author = {Coleridge, Samuel Taylor},
542+ title = {Biographia literaria, or {Biographical} sketches of my literary
543+ life and opinions},
544+ date = 1983,
545+ editor = {Coburn, Kathleen and Engell, James and Bate, W. Jackson},
546+ maintitle = {The collected works of {Samuel Taylor Coleridge}},
547+ volume = 7,
548+ part = 2,
549+ series = {Bollingen Series},
550+ number = 75,
551+ publisher = {Routledge {and} Kegan Paul},
552+ location = {London},
553+ langid = {english},
554+ langidopts = {variant=british},
555+ indextitle = {Biographia literaria},
556+ shorttitle = {Biographia literaria},
557+ annotation = {One (partial) volume of a multivolume book. This is a
558+ \texttt{book} entry with a \texttt{volume} and a \texttt{part}
559+ field which explicitly refers to the second (physical) part of
560+ the seventh (logical) volume. Also note the \texttt{series}
561+ and \texttt{number} fields},
562+}
563+
564+@book{companion,
565+ author = {Goossens, Michel and Mittelbach, Frank and Samarin, Alexander},
566+ title = {The {LaTeX} Companion},
567+ date = 1994,
568+ edition = 1,
569+ publisher = {Addison-Wesley},
570+ location = {Reading, Mass.},
571+ pagetotal = 528,
572+ langid = {english},
573+ langidopts = {variant=american},
574+ sorttitle = {LaTeX Companion},
575+ indextitle = {LaTeX Companion, The},
576+ shorttitle = {LaTeX Companion},
577+ annotation = {A book with three authors. Note the formatting of the author
578+ list. By default, only the first name is reversed in the
579+ bibliography},
580+}
581+
582+@book{cotton,
583+ author = {Cotton, Frank Albert and Wilkinson, Geoffrey and Murillio,
584+ Carlos A. and Bochmann, Manfred},
585+ title = {Advanced inorganic chemistry},
586+ date = 1999,
587+ edition = 6,
588+ publisher = {Wiley},
589+ location = {Chichester},
590+ langid = {english},
591+ langidopts = {variant=british},
592+ annotation = {A \texttt{book} entry with \arabic{author} authors and an
593+ \texttt{edition} field. By default, long \texttt{author} and
594+ \texttt{editor} lists are automatically truncated. This is
595+ configurable},
596+}
597+
598+@book{gerhardt,
599+ author = {Gerhardt, Michael J.},
600+ title = {The Federal Appointments Process},
601+ date = 2000,
602+ publisher = {Duke University Press},
603+ location = {Durham and London},
604+ langid = {english},
605+ langidopts = {variant=american},
606+ sorttitle = {Federal Appointments Process},
607+ indextitle = {Federal Appointments Process, The},
608+ subtitle = {A Constitutional and Historical Analysis},
609+ shorttitle = {Federal Appointments Process},
610+ annotation = {This is a \texttt{book} entry. Note the format of the
611+ \texttt{location} field as well as the \texttt{sorttitle} and
612+ \texttt{indextitle} fields},
613+}
614+
615+@book{gonzalez,
616+ author = {Gonzalez, Ray},
617+ title = {The Ghost of {John Wayne} and Other Stories},
618+ date = 2001,
619+ publisher = {The University of Arizona Press},
620+ location = {Tucson},
621+ isbn = {0-816-52066-6},
622+ langid = {english},
623+ langidopts = {variant=american},
624+ sorttitle = {Ghost of John Wayne and Other Stories},
625+ indextitle = {Ghost of {John Wayne} and Other Stories, The},
626+ shorttitle = {Ghost of {John Wayne}},
627+ annotation = {A collection of short stories. This is a \texttt{book} entry.
628+ Note the \texttt{sorttitle} and \texttt{indextitle} fields in
629+ the database file. There's also an \texttt{isbn} field},
630+}
631+
632+@book{hammond,
633+ author = {Hammond, Christopher},
634+ title = {The basics of crystallography and diffraction},
635+ date = 1997,
636+ publisher = {International Union of Crystallography and Oxford University
637+ Press},
638+ location = {Oxford},
639+ langid = {english},
640+ langidopts = {variant=british},
641+ sorttitle = {Basics of crystallography and diffraction},
642+ indextitle = {Basics of crystallography and diffraction, The},
643+ shorttitle = {Crystallography and diffraction},
644+ annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
645+ \texttt{indextitle} fields as well as the format of the
646+ \texttt{publisher} field},
647+}
648+
649+@book{iliad,
650+ author = {Homer},
651+ title = {Die Ilias},
652+ date = 2004,
653+ translator = {Schadewaldt, Wolfgang},
654+ introduction = {Latacz, Joachim},
655+ edition = 3,
656+ publisher = {Artemis \& Winkler},
657+ location = {D{\"u}sseldorf and Z{\"u}rich},
658+ langid = {german},
659+ sorttitle = {Ilias},
660+ indextitle = {Ilias, Die},
661+ shorttitle = {Ilias},
662+ annotation = {A German translation of the \emph{Iliad}. Note the
663+ \texttt{translator} and \texttt{introduction} fields and the
664+ format of the \texttt{location} field in the database
665+ file. Also note the \texttt{sorttitle} and \texttt{indextitle}
666+ fields},
667+}
668+
669+@book{knuth:ct,
670+ author = {Knuth, Donald E.},
671+ title = {Computers \& Typesetting},
672+ date = {1984/1986},
673+ volumes = 5,
674+ publisher = {Addison-Wesley},
675+ location = {Reading, Mass.},
676+ langid = {english},
677+ langidopts = {variant=american},
678+ sortyear = {1984-0},
679+ sorttitle = {Computers & Typesetting},
680+ indexsorttitle= {Computers & Typesetting},
681+ annotation = {A five-volume book cited as a whole. This is a \texttt{book}
682+ entry, note the \texttt{volumes} field},
683+}
684+
685+@book{knuth:ct:a,
686+ author = {Knuth, Donald E.},
687+ title = {The {\TeX} book},
688+ date = 1984,
689+ maintitle = {Computers \& Typesetting},
690+ volume = {A},
691+ publisher = {Addison-Wesley},
692+ location = {Reading, Mass.},
693+ langid = {english},
694+ langidopts = {variant=american},
695+ sortyear = {1984-1},
696+ sorttitle = {Computers & Typesetting A},
697+ indexsorttitle= {The TeXbook},
698+ indextitle = {\protect\TeX book, The},
699+ shorttitle = {\TeX book},
700+ annotation = {The first volume of a five-volume book. Note the
701+ \texttt{sorttitle} and \texttt{sortyear} fields. We want this
702+ volume to be listed after the entry referring to the entire
703+ five-volume set. Also note the \texttt{indextitle} and
704+ \texttt{indexsorttitle} fields. Indexing packages that don't
705+ generate robust index entries require some control sequences
706+ to be protected from expansion},
707+}
708+
709+@book{knuth:ct:b,
710+ author = {Knuth, Donald E.},
711+ title = {\TeX: The Program},
712+ date = 1986,
713+ maintitle = {Computers \& Typesetting},
714+ volume = {B},
715+ publisher = {Addison-Wesley},
716+ location = {Reading, Mass.},
717+ langid = {english},
718+ langidopts = {variant=american},
719+ sortyear = {1986-1},
720+ sorttitle = {Computers & Typesetting B},
721+ indexsorttitle= {TeX: The Program},
722+ shorttitle = {\TeX},
723+ annotation = {The second volume of a five-volume book. Note the
724+ \texttt{sorttitle} and \texttt{sortyear} fields. Also note the
725+ \texttt{indexsorttitle} field},
726+}
727+
728+@book{knuth:ct:c,
729+ author = {Knuth, Donald E.},
730+ title = {The {METAFONTbook}},
731+ date = 1986,
732+ maintitle = {Computers \& Typesetting},
733+ volume = {C},
734+ publisher = {Addison-Wesley},
735+ location = {Reading, Mass.},
736+ langid = {english},
737+ langidopts = {variant=american},
738+ sortyear = {1986-2},
739+ sorttitle = {Computers & Typesetting C},
740+ indextitle = {METAFONTbook, The},
741+ shorttitle = {METAFONTbook},
742+ annotation = {The third volume of a five-volume book. Note the
743+ \texttt{sorttitle} and \texttt{sortyear} fields as well as the
744+ \texttt{indextitle} field},
745+}
746+
747+@book{knuth:ct:d,
748+ author = {Knuth, Donald E.},
749+ title = {METAFONT: The Program},
750+ date = 1986,
751+ maintitle = {Computers \& Typesetting},
752+ volume = {D},
753+ publisher = {Addison-Wesley},
754+ location = {Reading, Mass.},
755+ langid = {english},
756+ langidopts = {variant=american},
757+ sortyear = {1986-3},
758+ sorttitle = {Computers & Typesetting D},
759+ shorttitle = {METAFONT},
760+ annotation = {The fourth volume of a five-volume book. Note the
761+ \texttt{sorttitle} and \texttt{sortyear} fields},
762+}
763+
764+@book{knuth:ct:e,
765+ author = {Knuth, Donald E.},
766+ title = {{Computer Modern} Typefaces},
767+ date = 1986,
768+ maintitle = {Computers \& Typesetting},
769+ volume = {E},
770+ publisher = {Addison-Wesley},
771+ location = {Reading, Mass.},
772+ langid = {english},
773+ langidopts = {variant=american},
774+ sortyear = {1986-4},
775+ sorttitle = {Computers & Typesetting E},
776+ annotation = {The fifth volume of a five-volume book. Note the
777+ \texttt{sorttitle} and \texttt{sortyear} fields},
778+}
779+
780+@book{knuth:ct:related,
781+ author = {Knuth, Donald E.},
782+ title = {Computers \& Typesetting},
783+ date = {1984/1986},
784+ volumes = 5,
785+ publisher = {Addison-Wesley},
786+ location = {Reading, Mass.},
787+ langid = {english},
788+ langidopts = {variant=american},
789+ sortyear = {1984-0},
790+ sorttitle = {Computers & Typesetting},
791+ indexsorttitle= {Computers & Typesetting},
792+ related = {knuth:ct:a,knuth:ct:b,knuth:ct:c,knuth:ct:d,knuth:ct:e},
793+ relatedtype = {multivolume},
794+ annotation = {A five-volume book cited as a whole and related to its
795+ individual volumes. Note the \texttt{related} and
796+ \texttt{relatedtype} fields},
797+}
798+
799+@book{kullback,
800+ author = {Kullback, Solomon},
801+ title = {Information Theory and Statistics},
802+ year = 1959,
803+ publisher = {John Wiley \& Sons},
804+ location = {New York},
805+ langid = {english},
806+ langidopts = {variant=american},
807+}
808+
809+@book{kullback:reprint,
810+ author = {Kullback, Solomon},
811+ title = {Information Theory and Statistics},
812+ year = 1997,
813+ publisher = {Dover Publications},
814+ location = {New York},
815+ origyear = 1959,
816+ origpublisher= {John Wiley \& Sons},
817+ langid = {english},
818+ langidopts = {variant=american},
819+ annotation = {A reprint of the \texttt{kullback} entry. Note the format of
820+ \texttt{origyear} and \texttt{origpublisher}. These fields are
821+ not used by the standard bibliography styles},
822+}
823+
824+@book{kullback:related,
825+ author = {Kullback, Solomon},
826+ title = {Information Theory and Statistics},
827+ year = 1997,
828+ publisher = {Dover Publications},
829+ location = {New York},
830+ langid = {english},
831+ langidopts = {variant=american},
832+ related = {kullback},
833+ relatedtype = {origpubin},
834+ annotation = {A reprint of the \texttt{kullback} entry. Note the format of
835+ the \texttt{related} and \texttt{relatedtype} fields},
836+}
837+
838+@book{malinowski,
839+ author = {Malinowski, Bronis{\l}aw},
840+ title = {Argonauts of the {Western Pacific}},
841+ date = 1972,
842+ edition = 8,
843+ publisher = {Routledge {and} Kegan Paul},
844+ location = {London},
845+ langid = {english},
846+ langidopts = {variant=british},
847+ subtitle = {An account of native enterprise and adventure in the
848+ Archipelagoes of {Melanesian New Guinea}},
849+ shorttitle = {Argonauts},
850+ annotation = {This is a \texttt{book} entry. Note the format of the
851+ \texttt{publisher} and \texttt{edition} fields as well as the
852+ \texttt{subtitle} field},
853+}
854+
855+@book{maron,
856+ author = {Maron, Monika},
857+ title = {Animal Triste},
858+ date = 2000,
859+ translator = {Brigitte Goldstein},
860+ origlanguage = {german},
861+ publisher = {University of Nebraska Press},
862+ location = {Lincoln},
863+ langid = {english},
864+ langidopts = {variant=american},
865+ shorttitle = {Animal Triste},
866+ annotation = {An English translation of a German novel with a French title.
867+ In other words: a \texttt{book} entry with a
868+ \texttt{translator} field. Note the \texttt{origlanguage}
869+ field which is concatenated with the \texttt{translator}},
870+}
871+
872+@book{massa,
873+ author = {Werner Massa},
874+ title = {Crystal structure determination},
875+ date = 2004,
876+ edition = 2,
877+ publisher = {Spinger},
878+ location = {Berlin},
879+ langid = {english},
880+ langidopts = {variant=british},
881+ annotation = {A \texttt{book} entry with an \texttt{edition} field},
882+}
883+
884+@article{moore,
885+ author = {Moore, Gordon E.},
886+ title = {Cramming more components onto integrated circuits},
887+ journaltitle = {Electronics},
888+ year = 1965,
889+ volume = 38,
890+ number = 8,
891+ pages = {114-117},
892+ langid = {english},
893+ langidopts = {variant=american},
894+}
895+
896+@article{moore:related,
897+ author = {Moore, Gordon E.},
898+ title = {Cramming more components onto integrated circuits},
899+ journaltitle = {Proceedings of the {IEEE}},
900+ year = 1998,
901+ volume = 86,
902+ number = 1,
903+ pages = {82-85},
904+ langid = {english},
905+ langidopts = {variant=american},
906+ related = {moore},
907+ relatedtype = {reprintfrom},
908+ annotation = {A reprint of Moore's law. Note the \texttt{related} and
909+ \texttt{relatedtype} fields},
910+}
911+
912+@book{nietzsche:ksa,
913+ author = {Nietzsche, Friedrich},
914+ title = {S{\"a}mtliche Werke},
915+ date = 1988,
916+ editor = {Colli, Giorgio and Montinari, Mazzino},
917+ edition = 2,
918+ volumes = 15,
919+ publisher = dtv # { and Walter de Gruyter},
920+ location = {M{\"u}nchen and Berlin and New York},
921+ langid = {german},
922+ sortyear = {1988-0},
923+ sorttitle = {Werke-00-000},
924+ indexsorttitle= {Samtliche Werke},
925+ subtitle = {Kritische Studienausgabe},
926+ annotation = {The critical edition of Nietzsche's works. This is a
927+ \texttt{book} entry referring to a 15-volume work as a
928+ whole. Note the \texttt{volumes} field and the format of the
929+ \texttt{publisher} and \texttt{location} fields in the
930+ database file. Also note the \texttt{sorttitle} and
931+ \texttt{sortyear} fields which are used to fine-tune the
932+ sorting order of the bibliography. We want this item listed
933+ first in the bibliography},
934+}
935+
936+@book{nietzsche:ksa1,
937+ author = {Nietzsche, Friedrich},
938+ title = {Die Geburt der Trag{\"o}die. Unzeitgem{\"a}{\ss}e
939+ Betrachtungen I--IV. Nachgelassene Schriften 1870--1973},
940+ date = 1988,
941+ editor = {Colli, Giorgio and Montinari, Mazzino},
942+ maintitle = {S{\"a}mtliche Werke},
943+ mainsubtitle = {Kritische Studienausgabe},
944+ volume = 1,
945+ edition = 2,
946+ publisher = dtv # { and Walter de Gruyter},
947+ location = {M{\"u}nchen and Berlin and New York},
948+ langid = {german},
949+ sortyear = {1988-1},
950+ sorttitle = {Werke-01-000},
951+ indexsorttitle= {Samtliche Werke I},
952+ bookauthor = {Nietzsche, Friedrich},
953+ indextitle = {S{\"a}mtliche Werke I},
954+ shorttitle = {S{\"a}mtliche Werke I},
955+ annotation = {A single volume from the critical edition of Nietzsche's
956+ works. This \texttt{book} entry explicitly refers to the first
957+ volume only. Note the \texttt{title} and \texttt{maintitle}
958+ fields. Also note the \texttt{sorttitle} and \texttt{sortyear}
959+ fields. We want this entry to be listed after the entry
960+ referring to the entire edition},
961+}
962+
963+@book{nussbaum,
964+ author = {Nussbaum, Martha},
965+ title = {Aristotle's \mkbibquote{De Motu Animalium}},
966+ date = 1978,
967+ publisher = pup,
968+ location = {Princeton},
969+ keywords = {secondary},
970+ langid = {english},
971+ langidopts = {variant=american},
972+ sorttitle = {Aristotle's De Motu Animalium},
973+ indexsorttitle= {Aristotle's De Motu Animalium},
974+ annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
975+ \texttt{indexsorttitle} fields and the markup of the quotes in
976+ the database file},
977+}
978+
979+@book{piccato,
980+ author = {Piccato, Pablo},
981+ title = {City of Suspects},
982+ date = 2001,
983+ publisher = {Duke University Press},
984+ location = {Durham and London},
985+ langid = {english},
986+ langidopts = {variant=american},
987+ subtitle = {Crime in {Mexico City}, 1900--1931},
988+ shorttitle = {City of Suspects},
989+ annotation = {This is a \texttt{book} entry. Note the format of the
990+ \texttt{location} field in the database file},
991+}
992+
993+@book{vangennep,
994+ author = {van Gennep, Arnold},
995+ title = {Les rites de passage},
996+ date = 1909,
997+ publisher = {Nourry},
998+ location = {Paris},
999+ options = {useprefix},
1000+ langid = {french},
1001+ sorttitle = {Rites de passage},
1002+ indextitle = {Rites de passage, Les},
1003+ shorttitle = {Rites de passage},
1004+ annotation = {A \texttt{book} entry. Note the format of the printed name and
1005+ compare the \texttt{useprefix} option in the \texttt{options}
1006+ field as well as \texttt{brandt} and \texttt{geer}},
1007+}
1008+
1009+@book{vangennep:trans,
1010+ author = {van Gennep, Arnold},
1011+ title = {The Rites of Passage},
1012+ year = 1960,
1013+ translator = {Vizedom, Monika B. and Caffee, Gabrielle L.},
1014+ language = {english},
1015+ origlanguage = {french},
1016+ publisher = {University of Chicago Press},
1017+ options = {useprefix},
1018+ indextitle = {Rites of Passage, The},
1019+ sorttitle = {Rites of Passage},
1020+ shorttitle = {Rites of Passage},
1021+ langid = {english},
1022+ langidopts = {variant=american},
1023+ annotation = {A translation of the \texttt{vangennep} entry. Note the
1024+ \texttt{translator} and \texttt{origlanguage} fields. Compare
1025+ with the \texttt{vangennep:related} entry.},
1026+}
1027+
1028+@book{vangennep:related,
1029+ author = {van Gennep, Arnold},
1030+ title = {Les rites de passage},
1031+ date = 1909,
1032+ publisher = {Nourry},
1033+ location = {Paris},
1034+ options = {useprefix},
1035+ langid = {french},
1036+ related = {vizedom:related},
1037+ relatedtype = {bytranslator},
1038+ sorttitle = {Rites de passage},
1039+ indextitle = {Rites de passage, Les},
1040+ shorttitle = {Rites de passage},
1041+ annotation = {A variant of the \texttt{vangennep} entry related to its
1042+ translation. Note the format of the \texttt{related} and
1043+ \texttt{relatedtype} fields},
1044+}
1045+
1046+@book{vizedom:related,
1047+ title = {The Rites of Passage},
1048+ year = 1960,
1049+ translator = {Vizedom, Monika B. and Caffee, Gabrielle L.},
1050+ language = {english},
1051+ publisher = {University of Chicago Press},
1052+ langid = {english},
1053+ langidopts = {variant=american},
1054+ options = {usetranslator},
1055+ related = {vangennep},
1056+ relatedtype = {translationof},
1057+ indextitle = {Rites of Passage, The},
1058+ sorttitle = {Rites of Passage},
1059+ shorttitle = {Rites of Passage},
1060+ annotation = {A translated work from \texttt{vangennep}. Note the format of
1061+ the \texttt{related} and \texttt{relatedtype} fields},
1062+}
1063+
1064+@book{vazques-de-parga,
1065+ author = {V{\'a}zques{ de }Parga, Luis and Lacarra, Jos{\'e} Mar{\'i}a
1066+ and Ur{\'i}a R{\'i}u, Juan},
1067+ title = {Las Peregrinaciones a Santiago de Compostela},
1068+ date = 1993,
1069+ volumes = 3,
1070+ note = {Ed. facs. de la realizada en 1948--49},
1071+ publisher = {Iberdrola},
1072+ location = {Pamplona},
1073+ langid = {spanish},
1074+ sorttitle = {Peregrinaciones a Santiago de Compostela},
1075+ indextitle = {Peregrinaciones a Santiago de Compostela, Las},
1076+ shorttitle = {Peregrinaciones},
1077+ annotation = {A multivolume book cited as a whole. This is a \texttt{book}
1078+ entry with \texttt{volumes}, \texttt{note},
1079+ \texttt{sorttitle}, and \texttt{indextitle} fields},
1080+}
1081+
1082+@book{wilde,
1083+ author = {Wilde, Oscar},
1084+ title = {The Importance of Being Earnest: A Trivial Comedy for Serious
1085+ People},
1086+ year = 1899,
1087+ series = {English and {American} drama of the Nineteenth Century},
1088+ publisher = {Leonard Smithers {and} Company},
1089+ eprint = {4HIWAAAAYAAJ},
1090+ eprinttype = {googlebooks},
1091+ annotation = {A \texttt{book} with \texttt{eprint} and \texttt{eprinttype}
1092+ fields.},
1093+}
1094+
1095+@book{worman,
1096+ author = {Worman, Nancy},
1097+ title = {The Cast of Character},
1098+ date = 2002,
1099+ publisher = {University of Texas Press},
1100+ location = {Austin},
1101+ langid = {english},
1102+ langidopts = {variant=american},
1103+ sorttitle = {Cast of Character},
1104+ indextitle = {Cast of Character, The},
1105+ subtitle = {Style in {Greek} Literature},
1106+ shorttitle = {Cast of Character},
1107+ annotation = {A \texttt{book} entry. Note the \texttt{sorttitle} and
1108+ \texttt{indextitle} fields},
1109+}
1110+
1111+@collection{britannica,
1112+ editor = {Preece, Warren E.},
1113+ title = {The {New Encyclop{\ae}dia Britannica}},
1114+ date = 2003,
1115+ edition = 15,
1116+ volumes = 32,
1117+ publisher = {Encyclop{\ae}dia Britannica},
1118+ location = {Chicago, Ill.},
1119+ options = {useeditor=false},
1120+ label = {EB},
1121+ langid = {english},
1122+ langidopts = {variant=british},
1123+ sorttitle = {Encyclop{\ae}dia Britannica},
1124+ indextitle = {{Encyclop{\ae}dia Britannica}, The {New}},
1125+ shorttitle = {{Encyclop{\ae}dia Britannica}},
1126+ annotation = {This is a \texttt{collection} entry for an encyclopedia. Note
1127+ the \texttt{useeditor} option in the \texttt{options} field as
1128+ well as the \texttt{sorttitle} field. We want this entry to be
1129+ cited and alphabetized by title even though there is an
1130+ editor. In addition to that, we want the title to be
1131+ alphabetized under \enquote*{E} rather than \enquote*{T}. Also
1132+ note the \texttt{label} field which is provided for
1133+ author-year citation styles},
1134+}
1135+
1136+@collection{gaonkar,
1137+ editor = {Gaonkar, Dilip Parameshwar},
1138+ title = {Alternative Modernities},
1139+ date = 2001,
1140+ publisher = {Duke University Press},
1141+ location = {Durham and London},
1142+ isbn = {0-822-32714-7},
1143+ langid = {english},
1144+ langidopts = {variant=american},
1145+ annotation = {This is a \texttt{collection} entry. Note the format of the
1146+ \texttt{location} field in the database file as well as the
1147+ \texttt{isbn} field},
1148+}
1149+
1150+@InCollection{gaonkar:in,
1151+ author = {Gaonkar, Dilip Parameshwar},
1152+ editor = {Gaonkar, Dilip Parameshwar},
1153+ title = {On Alternative Modernities},
1154+ date = 2001,
1155+ booktitle = {Alternative Modernities},
1156+ publisher = {Duke University Press},
1157+ location = {Durham and London},
1158+ isbn = {0-822-32714-7},
1159+ pages = {1-23},
1160+}
1161+
1162+@collection{jaffe,
1163+ editor = {Jaff{\'e}, Philipp},
1164+ title = {Regesta Pontificum Romanorum ab condita ecclesia ad annum post
1165+ Christum natum \textsc{mcxcviii}},
1166+ date = {1885/1888},
1167+ editora = {Loewenfeld, Samuel and Kaltenbrunner, Ferdinand and Ewald,
1168+ Paul},
1169+ edition = 2,
1170+ volumes = 2,
1171+ location = {Leipzig},
1172+ langid = {latin},
1173+ editoratype = {redactor},
1174+ indextitle = {Regesta Pontificum Romanorum},
1175+ shorttitle = {Regesta Pontificum Romanorum},
1176+ annotation = {A \texttt{collection} entry with \texttt{edition} and
1177+ \texttt{volumes} fields. Note the \texttt{editora} and
1178+ \texttt{editoratype} fields},
1179+}
1180+
1181+@collection{westfahl:frontier,
1182+ editor = {Westfahl, Gary},
1183+ title = {Space and Beyond},
1184+ date = 2000,
1185+ subtitle = {The Frontier Theme in Science Fiction},
1186+ publisher = {Greenwood},
1187+ location = {Westport, Conn. and London},
1188+ langid = {english},
1189+ langidopts = {variant=american},
1190+ booktitle = {Space and Beyond},
1191+ booksubtitle = {The Frontier Theme in Science Fiction},
1192+ annotation = {This is a \texttt{collection} entry. Note the format of the
1193+ \texttt{location} field as well as the \texttt{subtitle} and
1194+ \texttt{booksubtitle} fields},
1195+}
1196+
1197+@inbook{kant:kpv,
1198+ title = {Kritik der praktischen Vernunft},
1199+ date = 1968,
1200+ author = {Kant, Immanuel},
1201+ booktitle = {Kritik der praktischen Vernunft. Kritik der Urtheilskraft},
1202+ bookauthor = {Kant, Immanuel},
1203+ maintitle = {Kants Werke. Akademie Textausgabe},
1204+ volume = 5,
1205+ publisher = {Walter de Gruyter},
1206+ location = {Berlin},
1207+ pages = {1-163},
1208+ shorthand = {KpV},
1209+ langid = {german},
1210+ shorttitle = {Kritik der praktischen Vernunft},
1211+ annotation = {An edition of Kant's \emph{Collected Works}, volume five. This
1212+ is an \texttt{inbook} entry which explicitly refers to the
1213+ \emph{Critique of Practical Reason} only, not to the entire
1214+ fifth volume. Note the \texttt{author} and \texttt{bookauthor}
1215+ fields in the database file. By default, the
1216+ \texttt{bookauthor} is omitted if the values of the
1217+ \texttt{author} and \texttt{bookauthor} fields are identical},
1218+}
1219+
1220+@inbook{kant:ku,
1221+ title = {Kritik der Urtheilskraft},
1222+ date = 1968,
1223+ author = {Kant, Immanuel},
1224+ booktitle = {Kritik der praktischen Vernunft. Kritik der Urtheilskraft},
1225+ bookauthor = {Kant, Immanuel},
1226+ maintitle = {Kants Werke. Akademie Textausgabe},
1227+ volume = 5,
1228+ publisher = {Walter de Gruyter},
1229+ location = {Berlin},
1230+ pages = {165-485},
1231+ shorthand = {KU},
1232+ langid = {german},
1233+ annotation = {An edition of Kant's \emph{Collected Works}, volume five. This
1234+ is an \texttt{inbook} entry which explicitly refers to the
1235+ \emph{Critique of Judgment} only, not to the entire fifth
1236+ volume},
1237+}
1238+
1239+@inbook{nietzsche:historie,
1240+ title = {Unzeitgem{\"a}sse Betrachtungen. Zweites St{\"u}ck},
1241+ date = 1988,
1242+ author = {Nietzsche, Friedrich},
1243+ booktitle = {Die Geburt der Trag{\"o}die. Unzeitgem{\"a}{\ss}e
1244+ Betrachtungen I--IV. Nachgelassene Schriften 1870--1973},
1245+ bookauthor = {Nietzsche, Friedrich},
1246+ editor = {Colli, Giorgio and Montinari, Mazzino},
1247+ subtitle = {Vom Nutzen und Nachtheil der Historie f{\"u}r das Leben},
1248+ maintitle = {S{\"a}mtliche Werke},
1249+ mainsubtitle = {Kritische Studienausgabe},
1250+ volume = 1,
1251+ publisher = dtv # { and Walter de Gruyter},
1252+ location = {M{\"u}nchen and Berlin and New York},
1253+ pages = {243-334},
1254+ langid = {german},
1255+ sortyear = {1988-2},
1256+ sorttitle = {Werke-01-243},
1257+ indexsorttitle= {Vom Nutzen und Nachtheil der Historie fur das Leben},
1258+ indextitle = {Vom Nutzen und Nachtheil der Historie f{\"u}r das Leben},
1259+ shorttitle = {Vom Nutzen und Nachtheil der Historie},
1260+ annotation = {A single essay from the critical edition of Nietzsche's works.
1261+ This \texttt{inbook} entry explicitly refers to an essay found
1262+ in the first volume. Note the \texttt{title},
1263+ \texttt{booktitle}, and \texttt{maintitle} fields. Also note
1264+ the \texttt{sorttitle} and \texttt{sortyear} fields. We want
1265+ this entry to be listed after the entry referring to the
1266+ entire first volume},
1267+}
1268+
1269+@incollection{brandt,
1270+ author = {von Brandt, Ahasver and Erich Hoffmann},
1271+ editor = {Ferdinand Seibt},
1272+ title = {Die nordischen L{\"a}nder von der Mitte des 11.~Jahrhunderts
1273+ bis 1448},
1274+ date = 1987,
1275+ booktitle = {Europa im Hoch- und Sp{\"a}tmittelalter},
1276+ series = {Handbuch der europ{\"a}ischen Geschichte},
1277+ number = 2,
1278+ publisher = {Klett-Cotta},
1279+ location = {Stuttgart},
1280+ pages = {884-917},
1281+ options = {useprefix=false},
1282+ langid = {german},
1283+ indexsorttitle= {Nordischen Lander von der Mitte des 11. Jahrhunderts bis
1284+ 1448},
1285+ indextitle = {Nordischen L{\"a}nder von der Mitte des 11.~Jahrhunderts bis
1286+ 1448, Die},
1287+ shorttitle = {Die nordischen L{\"a}nder},
1288+ annotation = {An \texttt{incollection} entry with a \texttt{series} and a
1289+ \texttt{number}. Note the format of the printed name and
1290+ compare the \texttt{useprefix} option in the \texttt{options}
1291+ field as well as \texttt{vangennep}. Also note the
1292+ \texttt{indextitle, and \texttt{indexsorttitle} fields}},
1293+}
1294+
1295+@incollection{hyman,
1296+ author = {Arthur Hyman},
1297+ editor = {O'Meara, Dominic J.},
1298+ title = {Aristotle's Theory of the Intellect and its Interpretation by
1299+ {Averroes}},
1300+ date = 1981,
1301+ booktitle = {Studies in {Aristotle}},
1302+ series = {Studies in Philosophy and the History of Philosophy},
1303+ number = 9,
1304+ publisher = {The Catholic University of America Press},
1305+ location = {Washington, D.C.},
1306+ pages = {161-191},
1307+ keywords = {secondary},
1308+ langid = {english},
1309+ langidopts = {variant=american},
1310+ indextitle = {Aristotle's Theory of the Intellect},
1311+ shorttitle = {Aristotle's Theory of the Intellect},
1312+ annotation = {An \texttt{incollection} entry with a \texttt{series} and
1313+ \texttt{number} field},
1314+}
1315+
1316+@incollection{pines,
1317+ author = {Pines, Shlomo},
1318+ editor = {Twersky, Isadore},
1319+ title = {The Limitations of Human Knowledge According to {Al-Farabi}, {ibn
1320+ Bajja}, and {Maimonides}},
1321+ date = 1979,
1322+ booktitle = {Studies in Medieval {Jewish} History and Literature},
1323+ publisher = hup,
1324+ location = {Cambridge, Mass.},
1325+ pages = {82-109},
1326+ keywords = {secondary},
1327+ langid = {english},
1328+ langidopts = {variant=american},
1329+ indextitle = {Limitations of Human Knowledge According to {Al-Farabi}, {ibn
1330+ Bajja}, and {Maimonides}, The},
1331+ shorttitle = {Limitations of Human Knowledge},
1332+ annotation = {A typical \texttt{incollection} entry. Note the
1333+ \texttt{indextitle} field},
1334+}
1335+
1336+@inproceedings{moraux,
1337+ author = {Moraux, Paul},
1338+ editor = {Lloyd, G. E. R. and Owen, G. E. L.},
1339+ title = {Le \emph{De Anima} dans la tradition gr{\`e}cque},
1340+ date = 1979,
1341+ booktitle = {Aristotle on Mind and the Senses},
1342+ subtitle = {Quelques aspects de l'interpretation du trait{\'e}, de
1343+ Theophraste {\`a} Themistius},
1344+ booktitleaddon= {Proceedings of the Seventh Symposium Aristotelicum},
1345+ eventdate = 1975,
1346+ publisher = cup,
1347+ location = {Cambridge},
1348+ pages = {281-324},
1349+ keywords = {secondary},
1350+ langid = {french},
1351+ indexsorttitle= {De Anima dans la tradition grecque},
1352+ indextitle = {\emph{De Anima} dans la tradition gr{\`e}cque, Le},
1353+ shorttitle = {\emph{De Anima} dans la tradition gr{\`e}cque},
1354+ annotation = {This is a typical \texttt{inproceedings} entry. Note the
1355+ \texttt{booksubtitle}, \texttt{shorttitle},
1356+ \texttt{indextitle}, and \texttt{indexsorttitle} fields. Also
1357+ note the \texttt{eventdate} field.},
1358+}
1359+
1360+@inproceedings{salam,
1361+ author = {Salam, Abdus},
1362+ editor = {Svartholm, Nils},
1363+ title = {Weak and Electromagnetic Interactions},
1364+ date = 1968,
1365+ booktitle = {Elementary particle theory},
1366+ booksubtitle = {Relativistic groups and analyticity},
1367+ booktitleaddon= {Proceedings of the {Eighth Nobel Symposium}},
1368+ eventdate = {1968-05-19/1968-05-25},
1369+ venue = {Aspen{\"a}sgarden, Lerum},
1370+ publisher = {Almquist \& Wiksell},
1371+ location = {Stockholm},
1372+ pages = {367-377},
1373+}
1374+
1375+@manual{cms,
1376+ title = {The {Chicago} Manual of Style},
1377+ date = 2003,
1378+ subtitle = {The Essential Guide for Writers, Editors, and Publishers},
1379+ edition = 15,
1380+ publisher = {University of Chicago Press},
1381+ location = {Chicago, Ill.},
1382+ isbn = {0-226-10403-6},
1383+ label = {CMS},
1384+ langid = {english},
1385+ langidopts = {variant=american},
1386+ sorttitle = {Chicago Manual of Style},
1387+ indextitle = {Chicago Manual of Style, The},
1388+ shorttitle = {Chicago Manual of Style},
1389+ annotation = {This is a \texttt{manual} entry without an \texttt{author} or
1390+ \texttt{editor}. Note the \texttt{label} field in the database
1391+ file which is provided for author-year citation styles. Also
1392+ note the \texttt{sorttitle} and \texttt{indextitle} fields. By
1393+ default, all entries without an \texttt{author} or
1394+ \texttt{editor} are alphabetized by \texttt{title} but we want
1395+ this entry to be alphabetized under \enquote*{C} rather than
1396+ \enquote*{T}. There's also an \texttt{isbn} field},
1397+}
1398+
1399+@online{baez/online,
1400+ author = {Baez, John C. and Lauda, Aaron D.},
1401+ title = {Higher-Dimensional Algebra {V}: 2-Groups},
1402+ date = {2004-10-27},
1403+ version = 3,
1404+ langid = {english},
1405+ langidopts = {variant=american},
1406+ eprinttype = {arxiv},
1407+ eprint = {math/0307200v3},
1408+ annotation = {An \texttt{online} reference from arXiv. Note the
1409+ \texttt{eprint} and \texttt{eprinttype} fields. Compare
1410+ \texttt{baez\slash article} which is the same item given as an
1411+ \texttt{article} entry with eprint information},
1412+}
1413+
1414+@online{ctan,
1415+ title = {CTAN},
1416+ date = 2006,
1417+ url = {http://www.ctan.org},
1418+ subtitle = {The {Comprehensive TeX Archive Network}},
1419+ urldate = {2006-10-01},
1420+ label = {CTAN},
1421+ langid = {english},
1422+ langidopts = {variant=american},
1423+ annotation = {This is an \texttt{online} entry. The \textsc{url}, which is
1424+ given in the \texttt{url} field, is transformed into a
1425+ clickable link if \texttt{hyperref} support has been
1426+ enabled. Note the format of the \texttt{urldate} field
1427+ (\texttt{yyyy-mm-dd}) in the database file. Also note the
1428+ \texttt{label} field which may be used as a fallback by
1429+ citation styles which need an \texttt{author} and\slash or a
1430+ \texttt{year}},
1431+}
1432+
1433+@online{itzhaki,
1434+ author = {Itzhaki, Nissan},
1435+ title = {Some remarks on {'t Hooft's} {S}-matrix for black holes},
1436+ date = {1996-03-11},
1437+ version = 1,
1438+ langid = {english},
1439+ langidopts = {variant=american},
1440+ eprinttype = {arxiv},
1441+ eprint = {hep-th/9603067},
1442+ annotation = {An \texttt{online} reference from arXiv. Note the
1443+ \texttt{eprint} and \texttt{eprinttype} fields. Also note that
1444+ the arXiv reference is transformed into a clickable link if
1445+ \texttt{hyperref} support has been enabled},
1446+ abstract = {We discuss the limitations of 't Hooft's proposal for the
1447+ black hole S-matrix. We find that the validity of the S-matrix
1448+ implies violation of the semi-classical approximation at
1449+ scales large compared to the Planck scale. We also show that
1450+ the effect of the centrifugal barrier on the S-matrix is
1451+ crucial even for large transverse distances.},
1452+}
1453+
1454+@online{markey,
1455+ author = {Markey, Nicolas},
1456+ title = {Tame the {BeaST}},
1457+ date = {2005-10-16},
1458+ url = {http://mirror.ctan.org/info/bibtex/tamethebeast/ttb_en.pdf},
1459+ subtitle = {The {B} to {X} of {BibTeX}},
1460+ version = {1.3},
1461+ urldate = {2006-10-01},
1462+ langid = {english},
1463+ langidopts = {variant=american},
1464+ sorttitle = {Tame the Beast},
1465+ annotation = {An \texttt{online} entry for a tutorial. Note the format of
1466+ the \texttt{date} field (\texttt{yyyy-mm-dd}) in the database
1467+ file.},
1468+}
1469+
1470+@online{wassenberg,
1471+ author = {Wassenberg, Jan and Sanders, Peter},
1472+ title = {Faster Radix Sort via Virtual Memory and Write-Combining},
1473+ date = {2010-08-17},
1474+ version = 1,
1475+ langid = {english},
1476+ langidopts = {variant=american},
1477+ eprinttype = {arxiv},
1478+ eprintclass = {cs.DS},
1479+ eprint = {1008.2849v1},
1480+ annotation = {A recent \texttt{online} reference from arXiv using the new
1481+ (April 2007 onward) identifier format. Note the
1482+ \texttt{eprint}, \texttt{eprinttype}, and \texttt{eprintclass}
1483+ fields. Also note that the arXiv reference is transformed into
1484+ a clickable link if \texttt{hyperref} support has been
1485+ enabled},
1486+ abstract = {Sorting algorithms are the deciding factor for the performance
1487+ of common operations such as removal of duplicates or database
1488+ sort-merge joins. This work focuses on 32-bit integer keys,
1489+ optionally paired with a 32-bit value. We present a fast radix
1490+ sorting algorithm that builds upon a microarchitecture-aware
1491+ variant of counting sort},
1492+}
1493+
1494+@patent{almendro,
1495+ author = {Almendro, Jos{\'e} L. and Mart{\'i}n, Jacinto and S{\'a}nchez,
1496+ Alberto and Nozal, Fernando},
1497+ title = {Elektromagnetisches Signalhorn},
1498+ number = {EU-29702195U},
1499+ date = 1998,
1500+ location = {countryfr and countryuk and countryde},
1501+ langid = {german},
1502+ annotation = {This is a \texttt{patent} entry with a \texttt{location}
1503+ field. The number is given in the \texttt{number} field. Note
1504+ the format of the \texttt{location} field in the database
1505+ file. Compare \texttt{laufenberg}, \texttt{sorace}, and
1506+ \texttt{kowalik}},
1507+}
1508+
1509+@patent{kowalik,
1510+ author = {Kowalik, F. and Isard, M.},
1511+ title = {Estimateur d'un d{\'e}faut de fonctionnement d'un modulateur
1512+ en quadrature et {\'e}tage de modulation l'utilisant},
1513+ number = 9500261,
1514+ date = {1995-01-11},
1515+ type = {patreqfr},
1516+ langid = {french},
1517+ indextitle = {Estimateur d'un d{\'e}faut de fonctionnement},
1518+ annotation = {This is a \texttt{patent} entry for a French patent request
1519+ with a full date. The number is given in the \texttt{number}
1520+ field. Note the format of the \texttt{type} and \texttt{date}
1521+ fields in the database file. Compare \texttt{almendro},
1522+ \texttt{laufenberg}, and \texttt{sorace}},
1523+}
1524+
1525+@patent{laufenberg,
1526+ author = {Laufenberg, Xaver and Eynius, Dominique and Suelzle, Helmut
1527+ and Usbeck, Stephan and Spaeth, Matthias and Neuser-Hoffmann,
1528+ Miriam and Myrzik, Christian and Schmid, Manfred and Nietfeld,
1529+ Franz and Thiel, Alexander and Braun, Harald and Ebner,
1530+ Norbert},
1531+ title = {Elektrische Einrichtung und Betriebsverfahren},
1532+ number = 1700367,
1533+ date = {2006-09-13},
1534+ holder = {{Robert Bosch GmbH} and {Daimler Chrysler AG} and {Bayerische
1535+ Motoren Werke AG}},
1536+ type = {patenteu},
1537+ langid = {german},
1538+ annotation = {This is a \texttt{patent} entry with a \texttt{holder} field.
1539+ Note the format of the \texttt{type} and \texttt{location}
1540+ fields in the database file. Compare \texttt{almendro},
1541+ \texttt{sorace}, and \texttt{kowalik}},
1542+ abstract = {The invention relates to an electric device comprising a
1543+ generator, in particular for use in the vehicle electric
1544+ system of a motor vehicle and a controller for controlling the
1545+ generator voltage. The device is equipped with a control zone,
1546+ in which the voltage is controlled and zones, in which the
1547+ torque is controlled. The invention also relates to methods
1548+ for operating a device of this type.},
1549+ file = {http://v3.espacenet.com/textdoc?IDX=EP1700367},
1550+}
1551+
1552+@patent{sorace,
1553+ author = {Sorace, Ronald E. and Reinhardt, Victor S. and Vaughn, Steven
1554+ A.},
1555+ title = {High-Speed Digital-to-{RF} Converter},
1556+ number = 5668842,
1557+ date = {1997-09-16},
1558+ holder = {{Hughes Aircraft Company}},
1559+ type = {patentus},
1560+ langid = {english},
1561+ langidopts = {variant=american},
1562+ annotation = {This is a \texttt{patent} entry with a \texttt{holder} field.
1563+ Note the format of the \texttt{type} and \texttt{date} fields
1564+ in the database file. Compare \texttt{almendro},
1565+ \texttt{laufenberg}, and \texttt{kowalik}},
1566+}
1567+
1568+@periodical{jcg,
1569+ title = {Computers and Graphics},
1570+ year = 2011,
1571+ issuetitle = {Semantic {3D} Media and Content},
1572+ volume = 35,
1573+ number = 4,
1574+ issn = {0097-8493},
1575+ annotation = {This is a \texttt{periodical} entry with an \texttt{issn}
1576+ field.},
1577+}
1578+
1579+@report{chiu,
1580+ author = {Chiu, Willy W. and Chow, We Min},
1581+ title = {A Hybrid Hierarchical Model of a {Multiple Virtual Storage}
1582+ ({MVS}) Operating System},
1583+ type = {resreport},
1584+ institution = {IBM},
1585+ date = 1978,
1586+ number = {RC-6947},
1587+ langid = {english},
1588+ langidopts = {variant=american},
1589+ sorttitle = {Hybrid Hierarchical Model of a Multiple Virtual Storage (MVS)
1590+ Operating System},
1591+ indextitle = {Hybrid Hierarchical Model, A},
1592+ annotation = {This is a \texttt{report} entry for a research report. Note
1593+ the format of the \texttt{type} field in the database file
1594+ which uses a localization key. The number of the report is
1595+ given in the \texttt{number} field. Also note the
1596+ \texttt{sorttitle} and \texttt{indextitle} fields},
1597+}
1598+
1599+@report{padhye,
1600+ author = {Padhye, Jitendra and Firoiu, Victor and Towsley, Don},
1601+ title = {A Stochastic Model of {TCP Reno} Congestion Avoidance and
1602+ Control},
1603+ type = {techreport},
1604+ institution = {University of Massachusetts},
1605+ date = 1999,
1606+ number = {99-02},
1607+ location = {Amherst, Mass.},
1608+ langid = {english},
1609+ langidopts = {variant=american},
1610+ sorttitle = {A Stochastic Model of TCP Reno Congestion Avoidance and
1611+ Control},
1612+ indextitle = {Stochastic Model of {TCP Reno} Congestion Avoidance and Control,
1613+ A},
1614+ annotation = {This is a \texttt{report} entry for a technical report. Note
1615+ the format of the \texttt{type} field in the database file
1616+ which uses a localization key. The number of the report is
1617+ given in the \texttt{number} field. Also note the
1618+ \texttt{sorttitle} and \texttt{indextitle} fields},
1619+ abstract = {The steady state performance of a bulk transfer TCP flow
1620+ (i.e. a flow with a large amount of data to send, such as FTP
1621+ transfers) may be characterized by three quantities. The first
1622+ is the send rate, which is the amount of data sent by the
1623+ sender in unit time. The second is the throughput, which is
1624+ the amount of data received by the receiver in unit time. Note
1625+ that the throughput will always be less than or equal to the
1626+ send rate due to losses. Finally, the number of non-duplicate
1627+ packets received by the receiver in unit time gives us the
1628+ goodput of the connection. The goodput is always less than or
1629+ equal to the throughput, since the receiver may receive two
1630+ copies of the same packet due to retransmissions by the
1631+ sender. In a previous paper, we presented a simple model for
1632+ predicting the steady state send rate of a bulk transfer TCP
1633+ flow as a function of loss rate and round trip time. In this
1634+ paper, we extend that work in two ways. First, we analyze the
1635+ performance of bulk transfer TCP flows using more precise,
1636+ stochastic analysis. Second, we build upon the previous
1637+ analysis to provide both an approximate formula as well as a
1638+ more accurate stochastic model for the steady state throughput
1639+ of a bulk transfer TCP flow.},
1640+ file = {ftp://gaia.cs.umass.edu/pub/Padhey99-markov.ps},
1641+}
1642+
1643+@thesis{geer,
1644+ author = {de Geer, Ingrid},
1645+ title = {Earl, Saint, Bishop, Skald~-- and Music},
1646+ type = {phdthesis},
1647+ institution = {Uppsala Universitet},
1648+ date = 1985,
1649+ subtitle = {The {Orkney Earldom} of the Twelfth Century. {A} Musicological
1650+ Study},
1651+ location = {Uppsala},
1652+ options = {useprefix=false},
1653+ langid = {english},
1654+ langidopts = {variant=british},
1655+ annotation = {This is a typical \texttt{thesis} entry for a PhD thesis. Note
1656+ the \texttt{type} field in the database file which uses a
1657+ localization key. Also note the format of the printed name and
1658+ compare the \texttt{useprefix} option in the \texttt{options}
1659+ field as well as \texttt{vangennep}},
1660+}
1661+
1662+@thesis{loh,
1663+ author = {Loh, Nin C.},
1664+ title = {High-Resolution Micromachined Interferometric Accelerometer},
1665+ type = {mathesis},
1666+ institution = {Massachusetts Institute of Technology},
1667+ date = 1992,
1668+ location = {Cambridge, Mass.},
1669+ langid = {english},
1670+ langidopts = {variant=american},
1671+ annotation = {This is a typical \texttt{thesis} entry for an MA thesis. Note
1672+ the \texttt{type} field in the database file which uses a
1673+ localization key},
1674+}
test/command/biblatex-inproceedings.md+1−1
@@ -63,7 +63,7 @@ references:
6363 publisher-place: Cambridge
6464 title: "Le *De Anima* dans la tradition grècque: Quelques aspects de
6565 l'interpretation du traité, de Theophraste à Themistius"
66− title-short: "*De Anima* dans la tradition grècque"
66+ title-short: *De Anima* dans la tradition grècque
6767 type: paper-conference
6868- author:
6969 - family: Salam
test/command/biblatex-jaffe.md+2−2
@@ -72,8 +72,8 @@ references:
7272 language: la
7373 number-of-volumes: 2
7474 publisher-place: Leipzig
75− title: Regesta Pontificum Romanorum ab condita ecclesia ad annum post
76− Christum natum [mcxcviii]{.smallcaps}
75+ title: "Regesta Pontificum Romanorum ab condita ecclesia ad annum post
76+ Christum natum [mcxcviii]{.smallcaps}"
7777 title-short: Regesta Pontificum Romanorum
7878 type: book
7979---
test/command/biblatex-moraux.md+1−1
@@ -86,7 +86,7 @@ references:
8686 publisher-place: Cambridge
8787 title: "Le *De Anima* dans la tradition grècque: Quelques aspects de
8888 l'interpretation du traité, de Theophraste à Themistius"
89− title-short: "*De Anima* dans la tradition grècque"
89+ title-short: *De Anima* dans la tradition grècque
9090 type: paper-conference
9191---
9292
test/command/biblatex-nussbaum.mdnew file+63
@@ -0,0 +1,63 @@
1+```
2+% pandoc -f biblatex -t markdown -s
3+@comment{
4+
5+Adapted from biblatex-example.bib
6+
7+
8+Formatted with pandoc and chicago-author-date.csl, 2013-10-23:
9+
10+(Nussbaum 1978)
11+
12+Nussbaum, Martha. 1978. *Aristotle’s “De Motu Animalium”*. Princeton:
13+Princeton University Press.
14+
15+
16+Formatted with pandoc and apa.csl, 2013-10-23:
17+
18+(Nussbaum, 1978)
19+
20+Nussbaum, M. (1978). *Aristotle’s “De Motu Animalium”*. Princeton:
21+Princeton University Press.
22+
23+
24+}
25+
26+@string{ pup = {Princeton University Press} }
27+
28+@Book{nussbaum,
29+ author = {Nussbaum, Martha},
30+ title = {Aristotle's \mkbibquote{De Motu Animalium}},
31+ date = 1978,
32+ publisher = pup,
33+ location = {Princeton},
34+ keywords = {secondary},
35+ hyphenation = {american},
36+ sorttitle = {Aristotle's De Motu Animalium},
37+ indexsorttitle= {Aristotle's De Motu Animalium},
38+ annotation = {A book entry. Note the sorttitle and
39+ indexsorttitle fields and the markup of the quotes in
40+ the database file},
41+}
42+
43+^D
44+---
45+nocite: "[@*]"
46+references:
47+- annote: A book entry. Note the sorttitle and indexsorttitle fields and
48+ the markup of the quotes in the database file
49+ author:
50+ - family: Nussbaum
51+ given: Martha
52+ id: nussbaum
53+ issued: 1978
54+ keyword: secondary
55+ language: en-US
56+ publisher: Princeton University Press
57+ publisher-place: Princeton
58+ title: "Aristotle's \"De Motu Animalium\""
59+ type: book
60+---
61+
62+
63+```
test/command/biblatex-quotes.md+1−1
@@ -36,7 +36,7 @@ references:
3636 language: en-US
3737 publisher: Princeton University Press
3838 publisher-place: Princeton
39− title: Aristotle's "De Motu Animalium"
39+ title: "Aristotle's \"De Motu Animalium\""
4040 type: book
4141---
4242
test/command/biblatex-sigfridsson.md+1−1
@@ -90,7 +90,7 @@ references:
9090 - family: Ryde
9191 given: Ulf
9292 container-title: Journal of Computational Chemistry
93− doi: "10.1002/(SICI)1096-987X(199803)19:4\\<377::AID-JCC1>3.0.CO;2-P"
93+ doi: "10.1002/(SICI)1096-987X(199803)19:4\\<377::AID-JCC1\\>3.0.CO;2-P"
9494 id: sigfridsson
9595 issue: 4
9696 issued: 1998
test/command/biblatex-spiegelberg.md+2−2
@@ -63,8 +63,8 @@ references:
6363 issued: 1969
6464 language: de-DE
6565 page: 189-216
66− title: "\"Intention\" und \"Intentionalität\" in der Scholastik, bei
67− Brentano und Husserl"
66+ title: "Intention" und "Intentionalität" in der Scholastik, bei
67+ Brentano und Husserl
6868 title-short: Intention und Intentionalität
6969 type: article-journal
7070 volume: 29
test/command/biblatex-test-case-conversion.md+2−2
@@ -63,9 +63,9 @@ references:
6363 id: item1
6464 issued: 2013
6565 language: en-US
66− title: A title, in English, with a Proper Name and an ACRONYM and a
66+ title: "A title, in English, with a Proper Name and an ACRONYM and a
6767 [camelCase]{.nocase} word and some units, 400 [nm]{.nocase},
68− 3 [cm]{.nocase}, and a quote, *Alea [iacta est]{.nocase}*
68+ 3 [cm]{.nocase}, and a quote, *Alea [iacta est]{.nocase}*"
6969 type: article-journal
7070---
7171
test/command/biblatex-textnormal.md+1−1
@@ -9,7 +9,7 @@
99nocite: "[@*]"
1010references:
1111- id: item1
12− title: The title [of this book]{.nodecor}
12+ title: "The title [of this book]{.nodecor}"
1313 type: book
1414---
1515
test/command/chap1/spider.pngdeleted

Binary file not shown.

test/command/chap1/text.mddeleted−17
@@ -1,17 +0,0 @@
1−# Chapter one
2−
3−A spider: ![spider](spider.png)
4−
5−Another spider: ![another spider][refspider]
6−
7−The moon: ![moon](../../lalune.jpg)
8−
9−Link to [spider picture](spider.png).
10−
11−URL left alone: [manual](https://pandoc.org/MANUAL.html).
12−
13−Absolute path left alone: [absolute](/foo/bar/baz.png).
14−
15−Link to fragment: [chapter two](#chapter-two).
16−
17−Empty path: [empty]().
test/command/chap2/spider.pngdeleted

Binary file not shown.

test/command/chap2/text.mddeleted−5
@@ -1,5 +0,0 @@
1−# Chapter two
2−
3−A spider: ![spider](spider.png)
4−
5−[refspider]: spider.png
test/command/defaults-inheritance-2.md+1−1
@@ -1,5 +1,5 @@
11```
2−% pandoc -d command/defaults6 2>&1
2+% pandoc -d command/defaults6
33^D
44Error: Circular defaults file reference in 'command/defaults7.yaml'
55```
test/command/duplicate_attributes.mddeleted−7
@@ -1,7 +0,0 @@
1−```
2−% pandoc
3−[span]{.foobar style="color:blue" class="zip" style="color:red"}
4−^D
5−[WARNING] Ignoring duplicate attribute style="color:red".
6−<p><span class="foobar zip" style="color:blue">span</span></p>
7−```
test/command/mmd-metadata.mddeleted−20
@@ -1,20 +0,0 @@
1−```
2−% pandoc -f markdown_mmd -t markdown -s
3−Title: Blah blah blah
4−Author: Doo de Doo
5−Base Header Level: 1
6−Bibliography: Pubs.bib
7−Lang: en-GB
8−
9−body
10−^D
11−---
12−author: Doo de Doo
13−baseheaderlevel: 1
14−bibliography: Pubs.bib
15−lang: en-GB
16−title: Blah blah blah
17−---
18−
19−body
20−```
test/command/newif.mddeleted−55
@@ -1,55 +0,0 @@
1−```
2−% pandoc -f latex -t plain
3−\iftrue
4−should print
5−\iftrue
6−should print
7−\else
8−should not print
9−\fi
10−\else
11−should not print
12−\fi
13−
14−\iffalse
15−should not print
16−\else
17−\iftrue
18−should print
19−\else
20−should not print
21−\fi
22−\fi
23−
24−\newif\ifepub
25−
26−\ifepub
27−should not print
28−\fi
29−
30−\epubtrue
31−
32−\ifepub
33−should print
34−\else
35−should not print
36−\fi
37−
38−\epubfalse
39−
40−\ifepub
41−should not print
42−\else
43−should print
44−\fi
45−^D
46−should print
47−
48−should print
49−
50−should print
51−
52−should print
53−
54−should print
55−```
test/command/pandoc-citeproc-119.md+1−1
@@ -1,7 +1,7 @@
11```
22% pandoc --citeproc -t markdown-citations
33---
4−bibliography: 'command/averroes.bib'
4+bibliography: 'command/biblatex-examples.bib'
55csl: command/apa.csl
66---
77
test/command/pandoc-citeproc-320a.md+3−3
@@ -49,7 +49,7 @@ references:
4949Foo [@item1; @item2; @item3; @item4; @item5; @item6; @item7; @item8].
5050^D
5151Foo (al-ʾUdhrī, n.d.; al-ʿUdhrī, n.d.; al-\'Udhrī, n.d.; al-'Udhrī,
52−n.d.a, n.d.b; Uch, n.d.; Uebel, n.d.; Zzz, n.d.).
52+n.d.; al-'Udhrī, n.d.; Uch, n.d.; Uebel, n.d.; Zzz, n.d.).
5353
5454::: {#refs .references .csl-bib-body .hanging-indent}
5555::: {#ref-item6 .csl-entry}
@@ -69,11 +69,11 @@ Uch, Ann. n.d.
6969:::
7070
7171::: {#ref-item4 .csl-entry}
72−'Udhrī, Jamīl al-. n.d.a.
72+'Udhrī, Jamīl al-. n.d.
7373:::
7474
7575::: {#ref-item5 .csl-entry}
76−---------. n.d.b.
76+'Udhrī, Jamīl al-. n.d.
7777:::
7878
7979::: {#ref-item7 .csl-entry}
test/command/pandoc-citeproc-322.md+1−1
@@ -19,7 +19,7 @@ references:
1919 type: 'article-journal'
2020---
2121
22−Foo [@timmory__justice_1950].
22+Foo[@timmory__justice_1950].
2323^D
2424Foo.[^1]
2525
test/command/pandoc-citeproc-356.md+7−2
@@ -15,6 +15,11 @@ references:
1515
1616[@bar]
1717^D
18−[WARNING] Citeproc: citation bar not found
19−(**bar?**)
18+(Alice 2042)
19+
20+::: {#refs .references .csl-bib-body .hanging-indent}
21+::: {#ref-foo .csl-entry}
22+Alice. 2042.
23+:::
24+:::
2025```
test/command/pandoc-citeproc-65.md+1−1
@@ -14,7 +14,7 @@ references:
1414 id: 'stotz:1996handbuch'
1515 issued:
1616 literal: 1996_2004
17− language: de
17+ language: German
1818 number-of-volumes: 5
1919 publisher: Beck
2020 publisher-place: Munich
test/command/pandoc-citeproc-70.md+2−2
@@ -22,7 +22,7 @@ references:
2222 issued:
2323 date-parts:
2424 - - 2009
25− language: fr
25+ language: French
2626 page: '155--174'
2727 publisher: 'Fédération Internationale des Instituts d''Études
2828 Médiévales'
@@ -42,7 +42,7 @@ references:
4242 issued:
4343 date-parts:
4444 - - 1955
45− language: la
45+ language: Latin
4646 note: 'ArticleType: research-article / Full publication date: 1955 /
4747 Copyright © 1955 Fordham University'
4848 page: '163--193'
test/command/pandoc-citeproc-chicago-author-date.md+7−7
@@ -54,14 +54,14 @@ References {#references .unnumbered}
5454
5555([**nonexistent?**](#ref-nonexistent))
5656
57−Doe ([2005](#ref-item1)) says blah.
57+[Doe](#ref-item1) ([2005](#ref-item1)) says blah.
5858
59−Doe ([2005, 30](#ref-item1)) says blah.
59+[Doe](#ref-item1) ([2005, 30](#ref-item1)) says blah.
6060
61−Doe ([2005, 30](#ref-item1), with suffix) says blah.
61+[Doe](#ref-item1) ([2005, 30](#ref-item1), with suffix) says blah.
6262
63−Doe ([2005](#ref-item1); [2006, 30](#ref-item2); see also [Doe and Roe
64−2007](#ref-пункт3)) says blah.
63+[Doe](#ref-item1) ([2005](#ref-item1); [2006, 30](#ref-item2); see also
64+[Doe and Roe 2007](#ref-пункт3)) says blah.
6565
6666In a note.[^1]
6767
@@ -98,8 +98,8 @@ edited by Sam Smith. Oxford: Oxford University Press.
9898:::
9999:::
100100
101−[^1]: Doe and Roe ([2007, 12](#ref-пункт3)) and a citation without
102− locators ([Doe and Roe 2007](#ref-пункт3)).
101+[^1]: [Doe and Roe](#ref-пункт3) ([2007, 12](#ref-пункт3)) and a
102+ citation without locators ([Doe and Roe 2007](#ref-пункт3)).
103103
104104[^2]: Some citations (see [Doe 2005, chap. 3](#ref-item1);
105105 [2006](#ref-item2); [Doe and Roe 2007](#ref-пункт3)).
test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+7−6
@@ -55,13 +55,13 @@ References {#references .unnumbered}
5555
5656[^2]
5757
58−John Doe[^3] says blah.
58+[John Doe](#ref-item1)[^3] says blah.
5959
60−Doe[^4] says blah.
60+[Doe](#ref-item1)[^4] says blah.
6161
62−Doe[^5] says blah.
62+[Doe](#ref-item1)[^5] says blah.
6363
64−Doe[^6] says blah.
64+[Doe](#ref-item1)[^6] says blah.
6565
6666In a note.[^7]
6767
@@ -112,8 +112,9 @@ Sam Smith. Oxford: Oxford University Press, 2007.
112112 Roe, "Why Water Is Wet," in *Third Book*, ed. Sam Smith (Oxford:
113113 Oxford University Press, 2007)](#ref-пункт3).
114114
115−[^7]: Doe and Roe, ["Why Water Is Wet," 12](#ref-пункт3) and a citation
116− without locators ([Doe and Roe, "Why Water Is Wet"](#ref-пункт3)).
115+[^7]: [Doe and Roe](#ref-пункт3), ["Why Water Is Wet," 12](#ref-пункт3)
116+ and a citation without locators ([Doe and Roe, "Why Water Is
117+ Wet"](#ref-пункт3)).
117118
118119[^8]: See [Doe, *First Book*, chap. 3](#ref-item1); also [Doe and Roe,
119120 "Why Water Is Wet," 34--35](#ref-пункт3).
test/command/svg.md+1−1
@@ -2,7 +2,7 @@
22% pandoc -f latex -t icml
33\includegraphics{command/corrupt.svg}
44^D
5−[WARNING] Could not determine image size for command/corrupt.svg: could not determine image type
5+[WARNING] Could not determine image size for 'command/corrupt.svg': could not determine image type
66<ParagraphStyleRange AppliedParagraphStyle="ParagraphStyle/Paragraph">
77 <CharacterStyleRange AppliedCharacterStyle="$ID/NormalCharacterStyle">
88 <Rectangle Self="uec" StrokeWeight="0" ItemTransform="1 0 0 1 150 -100">
test/command/toc.md+2−2
@@ -31,7 +31,7 @@
3131
3232## b
3333
34−::: interior
34+::: {.interior}
3535# C
3636
3737## cc
@@ -39,7 +39,7 @@
3939# D
4040:::
4141
42−::: blue
42+::: {.blue}
4343# E
4444
4545## e
test/command/unicode-collation.mddeleted−130
@@ -1,130 +0,0 @@
1−```
2−% pandoc --citeproc -t plain
3−---
4−lang: en-US
5−csl: command/apa.csl
6−references:
7−- id: a1
8− type: book
9− author:
10− - family: Ubina
11− given: A. John
12− issued: 1985
13−- id: a2
14− type: book
15− author:
16− - family: Über
17− given: Aglaia
18− issued: 1996
19−- id: a3
20− type: book
21− author:
22− - family: Oñate
23− given: José
24− issued: 1985
25−- id: a4
26− type: book
27− author:
28− - family: Onush
29− given: Frank
30− issued: 2002
31−- id: a5
32− type: book
33− author:
34− - family: O'Neil
35− given: Timothy
36− issued: 2010
37−---
38−
39−[@a1;@a2;@a3;@a4;@a5]
40−^D
41−(O’Neil, 2010; Oñate, 1985; Onush, 2002; Über, 1996; Ubina, 1985)
42−
43−O’Neil, T. (2010).
44−
45−Oñate, J. (1985).
46−
47−Onush, F. (2002).
48−
49−Über, A. (1996).
50−
51−Ubina, A. J. (1985).
52−```
53−
54−```
55−% pandoc --citeproc -t plain
56−---
57−lang: es
58−csl: command/apa.csl
59−references:
60−- id: a1
61− type: book
62− author:
63− - family: Ubina
64− given: A. John
65− issued: 1985
66−- id: a2
67− type: book
68− author:
69− - family: Über
70− given: Aglaia
71− issued: 1996
72−- id: a3
73− type: book
74− author:
75− - family: Oñate
76− given: José
77− issued: 1985
78−- id: a4
79− type: book
80− author:
81− - family: Onush
82− given: Frank
83− issued: 2002
84−- id: a5
85− type: book
86− author:
87− - family: O'Neil
88− given: Timothy
89− issued: 2010
90−---
91−
92−[@a1;@a2;@a3;@a4;@a5]
93−^D
94−(O’Neil, 2010; Onush, 2002; Oñate, 1985; Über, 1996; Ubina, 1985)
95−
96−O’Neil, T. (2010).
97−
98−Onush, F. (2002).
99−
100−Oñate, J. (1985).
101−
102−Über, A. (1996).
103−
104−Ubina, A. J. (1985).
105−```
106−
107−```
108−% pandoc -C -t plain
109−---
110−nocite: '@*'
111−lang: fr-FR-u-kb-true
112−references:
113−- id: cote
114− author: cote
115−- id: côte
116− author: côte
117−- id: coté
118− author: coté
119−- id: côté
120− author: côté
121−...
122−^D
123−cote. s. d.
124−
125−côte. s. d.
126−
127−coté. s. d.
128−
129−côté. s. d.
130−```
test/docbook-reader.docbook+1−3
@@ -6,9 +6,7 @@
66<!ENTITY let "LET" >
77<!ENTITY case "CASE" >
88]>
9−<article xmlns="http://docbook.org/ns/docbook"
10− xmlns:xlink="http://www.w3.org/1999/xlink"
11− xmlns:mml="http://www.w3.org/1998/Math/MathML">
9+<article>
1210 <articleinfo>
1311 <title>Pandoc Test Suite</title>
1412 <authorgroup>
test/docx/0_level_headers.native+2−2
@@ -1,6 +1,6 @@
11[Table ("",[],[]) (Caption Nothing
22 [])
3− [(AlignDefault,ColWidth 1.0)]
3+ [(AlignDefault,ColWidthDefault)]
44 (TableHead ("",[],[])
55 [])
66 [(TableBody ("",[],[]) (RowHeadColumns 0)
@@ -49,4 +49,4 @@
4949,Para [Strong [Str "Table",Space,Str "Page"]]
5050,Para [Strong [Str "No",Space,Str "table",Space,Str "of",Space,Str "figures",Space,Str "entries",Space,Str "found."]]
5151,Header 1 ("introduction",[],[]) [Str "Introduction"]
52−,Para [Str "Nothing",Space,Str "to",Space,Str "introduce,",Space,Str "yet."]]
52+,Para [Str "Nothing",Space,Str "to",Space,Str "introduce,",Space,Str "yet."]]
test/docx/golden/block_quotes.docx

Binary file not shown.

test/docx/golden/codeblock.docx

Binary file not shown.

test/docx/golden/comments.docx

Binary file not shown.

test/docx/golden/custom_style_no_reference.docx

Binary file not shown.

test/docx/golden/custom_style_preserve.docx

Binary file not shown.

test/docx/golden/custom_style_reference.docx

Binary file not shown.

test/docx/golden/definition_list.docx

Binary file not shown.

test/docx/golden/document-properties-short-desc.docx

Binary file not shown.

test/docx/golden/document-properties.docx

Binary file not shown.

test/docx/golden/headers.docx

Binary file not shown.

test/docx/golden/image.docx

Binary file not shown.

test/docx/golden/inline_code.docx

Binary file not shown.

test/docx/golden/inline_formatting.docx

Binary file not shown.

test/docx/golden/inline_images.docx

Binary file not shown.

test/docx/golden/link_in_notes.docx

Binary file not shown.

test/docx/golden/links.docx

Binary file not shown.

test/docx/golden/lists.docx

Binary file not shown.

test/docx/golden/lists_continuing.docx

Binary file not shown.

test/docx/golden/lists_multiple_initial.docx

Binary file not shown.

test/docx/golden/lists_restarting.docx

Binary file not shown.

test/docx/golden/nested_anchors_in_header.docx

Binary file not shown.

test/docx/golden/notes.docx

Binary file not shown.

test/docx/golden/raw-blocks.docx

Binary file not shown.

test/docx/golden/raw-bookmarks.docx

Binary file not shown.

test/docx/golden/table_one_row.docx

Binary file not shown.

test/docx/golden/table_with_list_cell.docx

Binary file not shown.

test/docx/golden/tables-default-widths.docxdeleted

Binary file not shown.

test/docx/golden/tables.docx

Binary file not shown.

test/docx/golden/track_changes_deletion.docx

Binary file not shown.

test/docx/golden/track_changes_insertion.docx

Binary file not shown.

test/docx/golden/track_changes_move.docx

Binary file not shown.

test/docx/golden/track_changes_scrubbed_metadata.docx

Binary file not shown.

test/docx/golden/unicode.docx

Binary file not shown.

test/docx/golden/verbatim_subsuper.docx

Binary file not shown.

test/docx/image_vml_as_object.docxdeleted

Binary file not shown.

test/docx/image_vml_as_object.nativedeleted−2
@@ -1,2 +0,0 @@
1−[Para [Str "Test",Space,Str "with",Space,Str "object",Space,Str "as",Space,Str "image:"]
2−,Para [Image ("",[],[]) [] ("media/image1.emf","")]]
test/docx/sdt_elements.native+10−9
@@ -1,19 +1,20 @@
11[Table ("",[],[]) (Caption Nothing
22 [])
3− [(AlignDefault,ColWidth 0.22069570301081556)
4− ,(AlignDefault,ColWidth 0.22069570301081556)
5− ,(AlignDefault,ColWidth 0.5586085939783689)]
3+ [(AlignDefault,ColWidthDefault)
4+ ,(AlignDefault,ColWidthDefault)
5+ ,(AlignDefault,ColWidthDefault)]
66 (TableHead ("",[],[])
7− [Row ("",[],[])
7+ [])
8+ [(TableBody ("",[],[]) (RowHeadColumns 0)
9+ []
10+ [Row ("",[],[])
811 [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
912 [Plain [Strong [Str "col1Header"]]]
1013 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
1114 [Plain [Strong [Str "col2Header"]]]
1215 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
13− [Plain [Strong [Str "col3Header"]]]]])
14− [(TableBody ("",[],[]) (RowHeadColumns 0)
15− []
16− [Row ("",[],[])
16+ [Plain [Strong [Str "col3Header"]]]]
17+ ,Row ("",[],[])
1718 [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
1819 [Plain [Str "col1",Space,Str "content"]]
1920 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
@@ -21,4 +22,4 @@
2122 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
2223 [Plain [Str "col3",Space,Str "content"]]]])]
2324 (TableFoot ("",[],[])
24− [])]
25+ [])]
test/docx/table_captions_no_field.docxdeleted

Binary file not shown.

test/docx/table_captions_no_field.nativedeleted−34
@@ -1,34 +0,0 @@
1−[Para [Str "See",Space,Str "Table",Space,Str "5.1."]
2−,Para [Str "Table",Space,Str "5.1"]
3−,Table ("",[],[]) (Caption Nothing
4− [])
5− [(AlignDefault,ColWidth 0.7605739372523825)
6− ,(AlignDefault,ColWidth 0.11971303137380876)
7− ,(AlignDefault,ColWidth 0.11971303137380876)]
8− (TableHead ("",[],[])
9−[Row ("",[],[])
10− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
11− []
12− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
13− [Plain [Str "Count"]]
14− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
15− [Plain [Str "%"]]]])
16− [(TableBody ("",[],[]) (RowHeadColumns 0)
17− []
18− [Row ("",[],[])
19− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
20− [Plain [Str "First",Space,Str "option"]]
21− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
22− [Plain [Str "242"]]
23− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
24− [Plain [Str "45"]]]
25−,Row ("",[],[])
26− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
27− [Plain [Str "Second",Space,Str "option"]]
28− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
29− [Plain [Str "99"]]
30− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
31− [Plain [Str "18"]]]])]
32− (TableFoot ("",[],[])
33− [])
34−,Header 2 ("section", [], []) []]
test/docx/table_captions_with_field.docxdeleted

Binary file not shown.

test/docx/table_captions_with_field.nativedeleted−54
@@ -1,54 +0,0 @@
1−[Para [Str "See",Space,Str "Table",Space,Str "1."]
2−,Para []
3−,Table ("",[],[]) (Caption Nothing
4− [Para [Str "Table",Space,Str "1"]])
5− [(AlignDefault,ColWidth 0.7605739372523825)
6− ,(AlignDefault,ColWidth 0.11971303137380876)
7− ,(AlignDefault,ColWidth 0.11971303137380876)]
8− (TableHead ("",[],[])
9−[Row ("",[],[])
10− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
11− []
12− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
13− [Plain [Str "Count"]]
14− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
15− [Plain [Str "%"]]]])
16− [(TableBody ("",[],[]) (RowHeadColumns 0)
17− []
18− [Row ("",[],[])
19− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
20− [Plain [Str "First",Space,Str "option"]]
21− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
22− [Plain [Str "242"]]
23− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
24− [Plain [Str "45"]]]
25−,Row ("",[],[])
26− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
27− [Plain [Str "Second",Space,Str "option"]]
28− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
29− [Plain [Str "99"]]
30− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
31− [Plain [Str "18"]]]])]
32− (TableFoot ("",[],[])
33− [])
34−,Header 2 ("section", [], []) []
35−,Table ("",[],[]) (Caption Nothing
36− [Para [Str "Table",Space,Str "2"]])
37− [(AlignDefault,ColWidth 0.3332963620230701)
38− ,(AlignDefault,ColWidth 0.3332963620230701)
39− ,(AlignDefault,ColWidth 0.3334072759538598)]
40− (TableHead ("",[],[])
41− [Row ("",[],[])
42− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
43− [Plain [Str "One"]]
44− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
45− [Plain [Str "Two"]]
46− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
47− [Plain [Str "Three"]]]])
48− [(TableBody ("",[],[]) (RowHeadColumns 0)
49− []
50− [])]
51− (TableFoot ("",[],[])
52− [])
53−,Para []
54−,Para [Str "See",Space,Str "Table",Space,Str "2."]]
test/docx/table_header_rowspan.docxdeleted

Binary file not shown.

test/docx/table_header_rowspan.nativedeleted−189
@@ -1,189 +0,0 @@
1−[Table ("",[],[]) (Caption Nothing
2− [])
3− [(AlignDefault,ColWidth 0.30701754385964913)
4− ,(AlignDefault,ColWidth 0.1364522417153996)
5− ,(AlignDefault,ColWidth 0.10009746588693957)
6− ,(AlignDefault,ColWidth 9.707602339181287e-2)
7− ,(AlignDefault,ColWidth 7.719298245614035e-2)
8− ,(AlignDefault,ColWidth 7.085769980506823e-2)
9− ,(AlignDefault,ColWidth 7.09551656920078e-2)
10− ,(AlignDefault,ColWidth 0.14035087719298245)]
11− (TableHead ("",[],[])
12−[Row ("",[],[])
13− [Cell ("",[],[]) AlignDefault (RowSpan 2) (ColSpan 1)
14− [Plain [Str "A"]]
15− ,Cell ("",[],[]) AlignDefault (RowSpan 2) (ColSpan 1)
16− [Plain [Strong [Str "B"]]]
17− ,Cell ("",[],[]) AlignDefault (RowSpan 2) (ColSpan 1)
18− [Plain [Strong [Str "C"]]]
19− ,Cell ("",[],[]) AlignDefault (RowSpan 2) (ColSpan 1)
20− [Plain [Strong [Str "D"]]]
21− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 3)
22− [Plain [Str "E"]]
23− ,Cell ("",[],[]) AlignDefault (RowSpan 2) (ColSpan 1)
24− [Plain [Str "F"]]]
25−,Row ("",[],[])
26− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
27− [Plain [Strong [Str "G"]]]
28− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
29− [Plain [Strong [Str "H"]]]
30− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
31− [Plain [Strong [Str "I"]]]]])
32− [(TableBody ("",[],[]) (RowHeadColumns 0)
33− []
34− [Row ("",[],[])
35− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
36− [Plain [Str "1"]]
37− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
38− [Plain [Str "2"]]
39− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
40− [Plain [Str "3"]]
41− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
42− [Plain [Str "4"]]
43− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
44− [Plain [Str "5"]]
45− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
46− [Plain [Str "6"]]
47− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
48− [Plain [Str "7"]]
49− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
50− [Plain [Str "8"]]]
51− ,Row ("",[],[])
52− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
53− [Plain [Str "1"]]
54− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
55− [Plain [Str "2"]]
56− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
57− [Plain [Str "3"]]
58− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
59− [Plain [Str "4"]]
60− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
61− [Plain [Str "5"]]
62− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
63− [Plain [Str "6"]]
64− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
65− [Plain [Str "7"]]
66− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
67− [Plain [Str "8"]]]
68− ,Row ("",[],[])
69− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
70− [Plain [Str "1"]]
71− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
72− [Plain [Str "2"]]
73− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
74− [Plain [Str "3"]]
75− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
76− [Plain [Str "4"]]
77− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
78− [Plain [Str "5"]]
79− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
80− [Plain [Str "6"]]
81− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
82− [Plain [Str "7"]]
83− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
84− [Plain [Str "8"]]]
85− ,Row ("",[],[])
86− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
87− [Plain [Str "1"]]
88− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
89− [Plain [Str "2"]]
90− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
91− [Plain [Str "3"]]
92− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
93− [Plain [Str "4"]]
94− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
95− [Plain [Str "5"]]
96− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
97− [Plain [Str "6"]]
98− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
99− [Plain [Str "7"]]
100− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
101− [Plain [Str "8"]]]
102− ,Row ("",[],[])
103− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
104− [Plain [Str "1"]]
105− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
106− [Plain [Str "2"]]
107− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
108− [Plain [Str "3"]]
109− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
110− [Plain [Str "4"]]
111− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
112− [Plain [Str "5"]]
113− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
114− [Plain [Str "6"]]
115− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
116− [Plain [Str "7"]]
117− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
118− [Plain [Str "8"]]]
119− ,Row ("",[],[])
120− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
121− [Plain [Str "1"]]
122− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
123− [Plain [Str "2"]]
124− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
125− [Plain [Str "3"]]
126− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
127− [Plain [Str "4"]]
128− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
129− [Plain [Str "5"]]
130− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
131− [Plain [Str "6"]]
132− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
133− [Plain [Str "7"]]
134− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
135− [Plain [Str "8"]]]
136− ,Row ("",[],[])
137− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
138− [Plain [Str "1"]]
139− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
140− [Plain [Str "2"]]
141− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
142− [Plain [Str "3"]]
143− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
144− [Plain [Str "4"]]
145− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
146− [Plain [Str "5"]]
147− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
148− [Plain [Str "6"]]
149− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
150− [Plain [Str "7"]]
151− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
152− [Plain [Str "8"]]]
153− ,Row ("",[],[])
154− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
155− [Plain [Str "1"]]
156− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
157− [Plain [Str "2"]]
158− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
159− [Plain [Str "3"]]
160− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
161− [Plain [Str "4"]]
162− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
163− [Plain [Str "5"]]
164− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
165− [Plain [Str "6"]]
166− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
167− [Plain [Str "7"]]
168− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
169− [Plain [Str "8"]]]
170− ,Row ("",[],[])
171− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
172− [Plain [Str "1"]]
173− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
174− [Plain [Str "2"]]
175− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
176− [Plain [Str "3"]]
177− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
178− [Plain [Str "4"]]
179− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
180− [Plain [Str "5"]]
181− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
182− [Plain [Str "6"]]
183− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
184− [Plain [Str "7"]]
185− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
186− [Plain [Str "8"]]]
187− ])]
188− (TableFoot ("",[],[])
189− [])]
test/docx/table_one_header_row.docxdeleted

Binary file not shown.

test/docx/table_one_header_row.nativedeleted−18
@@ -1,18 +0,0 @@
1−[Table ("",[],[]) (Caption Nothing
2− [])
3− [(AlignDefault,ColWidth 0.33302433371958284)
4− ,(AlignDefault,ColWidth 0.3332560834298957)
5− ,(AlignDefault,ColWidth 0.33371958285052145)]
6− (TableHead ("",[],[])
7− [Row ("",[],[])
8− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
9− [Plain [Str "One"]]
10− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
11− [Plain [Str "Row"]]
12− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
13− [Plain [Str "Table"]]]])
14− [(TableBody ("",[],[]) (RowHeadColumns 0)
15− []
16− [])]
17− (TableFoot ("",[],[])
18− [])]
test/docx/table_one_row.docx

Binary file not shown.

test/docx/table_one_row.native+4−4
@@ -1,8 +1,8 @@
11[Table ("",[],[]) (Caption Nothing
22 [])
3− [(AlignDefault,ColWidth 0.3333333333333333)
4− ,(AlignDefault,ColWidth 0.3333333333333333)
5− ,(AlignDefault,ColWidth 0.3333333333333333)]
3+ [(AlignDefault,ColWidthDefault)
4+ ,(AlignDefault,ColWidthDefault)
5+ ,(AlignDefault,ColWidthDefault)]
66 (TableHead ("",[],[])
77 [])
88 [(TableBody ("",[],[]) (RowHeadColumns 0)
@@ -15,4 +15,4 @@
1515 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
1616 [Plain [Str "Table"]]]])]
1717 (TableFoot ("",[],[])
18− [])]
18+ [])]
test/docx/table_variable_width.native+17−12
@@ -1,11 +1,10 @@
11[Table ("",[],[]) (Caption Nothing
22 [])
3− [(AlignDefault,ColWidth 2.0096205237840725e-2)
4− ,(AlignDefault,ColWidth 1.9882415820416888e-2)
5− ,(AlignDefault,ColWidth 0.22202030999465527)
6− ,(AlignDefault,ColWidth 0.4761090326028862)
7− ,(AlignDefault,ColWidth 1.0689470871191876e-4)
8− ,(AlignDefault,ColWidth 0.26178514163548905)]
3+ [(AlignDefault,ColWidthDefault)
4+ ,(AlignDefault,ColWidthDefault)
5+ ,(AlignDefault,ColWidthDefault)
6+ ,(AlignDefault,ColWidthDefault)
7+ ,(AlignDefault,ColWidthDefault)]
98 (TableHead ("",[],[])
109 [Row ("",[],[])
1110 [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
@@ -14,27 +13,33 @@
1413 []
1514 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
1615 [Plain [Str "h3"]]
17− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 2)
16+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
1817 [Plain [Str "h4"]]
1918 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
2019 [Plain [Str "h5"]]]])
2120 [(TableBody ("",[],[]) (RowHeadColumns 0)
2221 []
2322 [Row ("",[],[])
24− [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 3)
23+ [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
2524 [Plain [Str "c11"]]
2625 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
2726 []
28− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 2)
27+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
28+ []
29+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
30+ []
31+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
2932 []]
3033 ,Row ("",[],[])
3134 [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
3235 []
33− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 2)
36+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
3437 [Plain [Str "c22"]]
3538 ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
3639 [Plain [Str "c23"]]
37− ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 2)
40+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
41+ []
42+ ,Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
3843 []]])]
3944 (TableFoot ("",[],[])
40− [])]
45+ [])]
test/docx/table_with_list_cell.native+3−3
@@ -1,7 +1,7 @@
11[Table ("",[],[]) (Caption Nothing
22 [])
3− [(AlignDefault,ColWidth 0.5)
4− ,(AlignDefault,ColWidth 0.5)]
3+ [(AlignDefault,ColWidthDefault)
4+ ,(AlignDefault,ColWidthDefault)]
55 (TableHead ("",[],[])
66 [Row ("",[],[])
77 [Cell ("",[],[]) AlignDefault (RowSpan 1) (ColSpan 1)
@@ -22,4 +22,4 @@
2222 ,[Para [Str "A"]]
2323 ,[Para [Str "Numbered",Space,Str "list."]]]]]])]
2424 (TableFoot ("",[],[])
25− [])]
25+ [])]
test/docx/tables-default-widths.nativedeleted−92

This diff is too big to show here.

test/docx/tables.native+9−9

This diff is too big to show here.

test/epub/features.native+2−2

This diff is too big to show here.

test/epub/wasteland.epub

Binary file not shown.

test/fb2/images.fb2+1−1

This diff is too big to show here.

test/fb2/test.jpg

Binary file not shown.

test/html-reader.native+1−2

This diff is too big to show here.

test/jira-reader.native+1−1

This diff is too big to show here.

test/lhs-test.html+1−2

This diff is too big to show here.

test/lhs-test.html+lhs+1−2

This diff is too big to show here.

test/odt/native/tab.nativedeleted−1

This diff is too big to show here.

test/odt/odt/tab.odtdeleted

Binary file not shown.

test/pipe-tables.native+10−2

This diff is too big to show here.

test/pptx/code-custom_templated.pptx

Binary file not shown.

test/pptx/code_templated.pptx

Binary file not shown.

test/pptx/document-properties-short-desc_templated.pptx

Binary file not shown.

test/pptx/document-properties_templated.pptx

Binary file not shown.

test/pptx/endnotes_templated.pptx

Binary file not shown.

test/pptx/endnotes_toc_templated.pptx

Binary file not shown.

test/pptx/images_templated.pptx

Binary file not shown.

test/pptx/inline_formatting_templated.pptx

Binary file not shown.

test/pptx/lists_templated.pptx

Binary file not shown.

test/pptx/raw_ooxml_templated.pptx

Binary file not shown.

test/pptx/reference_depth.pptx

Binary file not shown.

test/pptx/remove_empty_slides_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_slide_level_1_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_toc_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_after_metadata_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterheader_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterseps_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_templated.pptx

Binary file not shown.

test/pptx/start_numbering_at_templated.pptx

Binary file not shown.

test/pptx/tables_templated.pptx

Binary file not shown.

test/pptx/two_column_templated.pptx

Binary file not shown.

test/tables.latex+3−27

This diff is too big to show here.

test/tables.ms−30

This diff is too big to show here.

test/tables/nordics.latex+4−22

This diff is too big to show here.

test/tables/planets.latex+4−4

This diff is too big to show here.

test/tables/students.latex+2−10

This diff is too big to show here.

test/test-pandoc.hs+2−1

This diff is too big to show here.

test/writer.context+38−42

This diff is too big to show here.

test/writer.html4+1−2

This diff is too big to show here.

test/writer.html5+1−2

This diff is too big to show here.

test/writer.jats_archiving+47−37

This diff is too big to show here.

test/writer.jats_articleauthoring+49−41

This diff is too big to show here.

test/writer.jats_publishing+47−37

This diff is too big to show here.

test/writer.jira+35−23

This diff is too big to show here.

test/writer.ms+1−1

This diff is too big to show here.

test/writer.opml+1−1

This diff is too big to show here.

test/writer.org+7−7

This diff is too big to show here.

test/writer.zimwiki+3−3

This diff is too big to show here.

test/writers-lang-and-dir.context−4

This diff is too big to show here.

tools/build-arm.sh+5−17

This diff is too big to show here.