git.iko.soy/typeable/pandoc
on ce606cb help sign in
b8e0b46
swap
0 ahead, 746 behind
602 files changed+12415−20202
since splitsnapshots
unifiedsplit
.circleci/config.yml+122−64 .editorconfig−8 .github/ISSUE_TEMPLATE+11 .github/ISSUE_TEMPLATE/bug_report.md−25 .github/ISSUE_TEMPLATE/config.yml−9 .github/ISSUE_TEMPLATE/feature_request.md−21 .github/workflows/ci.yml+8−92 .github/workflows/lint.yml−3 .github/workflows/nightly.yml+2−9 .github/workflows/release-candidate.yml+1 .gitignore−7 .hlint.yaml+5−9 .travis.yml+54 AUTHORS.md−31 CONTRIBUTING.md+25−97 COPYING.md+1−1 COPYRIGHT+26−17 INSTALL.md+15−18 MANUAL.txt+114−379 Makefile+19−47 README.md+7−9 README.template+6−6 RELEASE-CHECKLIST+2−3 SECURITY.md−12 benchmark/benchmark-pandoc.hs+84−65 benchmark/weigh-pandoc.hs+50 cabal.project+7−9 changelog.md+2−1741 data/default.csl+2−12 data/docx/word/settings.xml+16−6 data/docx/word/styles.xml−15 data/jats.csl+203 data/make-reference-files.hs+27 data/odt/styles.xml+1−1 data/templates/.github/ISSUE_TEMPLATE.md+3 data/templates/.github/PULL_REQUEST_TEMPLATE.md+4 data/templates/README.markdown+49 data/templates/article.jats_publishing+4−6 data/templates/default.biblatex−10 data/templates/default.bibtex−10 data/templates/default.context+5−7 data/templates/default.dzslides+1−1 data/templates/default.epub2+1−1 data/templates/default.epub3+1−1 data/templates/default.html4−3 data/templates/default.html5−3 data/templates/default.jats_articleauthoring+1−3 data/templates/default.latex+40−71 data/templates/default.revealjs+125−79 data/templates/default.s5−3 data/templates/default.slideous−7 data/templates/default.slidy−5 data/templates/styles.html+1−2 data/translations/zh-Hans.yaml−22 data/translations/zh-Hant.yaml−22 data/vimwiki.css+82 default.nix−13 doc/epub.md+2−9 doc/faqs.md−128 doc/filters.md+7−13 doc/jats.md+7−31 doc/libraries.md−63 doc/lua-filters.md+7−193 doc/org.md+2−16 doc/using-the-pandoc-api.md+12−25 linux/make_artifacts.sh+7−19 man/pandoc.1+101−417 man/pandoc.1.after+1−1 pandoc.cabal+171−123 app/pandoc.hs → pandoc.hs+3−1 release.nix−4 shell.nix−106 src/Text/Pandoc.hs+1−1 src/Text/Pandoc/App.hs+46−69 src/Text/Pandoc/App/CommandLineOptions.hs+51−64 src/Text/Pandoc/App/FormatHeuristics.hs+8−19 src/Text/Pandoc/App/Opt.hs+51−261 src/Text/Pandoc/App/OutputSettings.hs+3−3 src/Text/Pandoc/Asciify.hs+391−14 src/Text/Pandoc/BCP47.hs+112 src/Text/Pandoc/CSS.hs+17−22 src/Text/Pandoc/CSV.hs+3−2 src/Text/Pandoc/Citeproc.hs+133−213 src/Text/Pandoc/Citeproc/BibTeX.hs+58−299 src/Text/Pandoc/Citeproc/Data.hs+6−7 src/Text/Pandoc/Citeproc/Locator.hs+4−20 src/Text/Pandoc/Citeproc/MetaValue.hs+3−5 src/Text/Pandoc/Class/CommonState.hs+1−1 src/Text/Pandoc/Class/IO.hs+25−24 src/Text/Pandoc/Class/PandocMonad.hs+36−43 src/Text/Pandoc/Data.hs+1−1 src/Text/Pandoc/Error.hs+49−102 src/Text/Pandoc/Extensions.hs+4−32 src/Text/Pandoc/Filter.hs+1−1 src/Text/Pandoc/Filter/JSON.hs+1−1 src/Text/Pandoc/Filter/Lua.hs+1−1 src/Text/Pandoc/Filter/Path.hs+1−1 src/Text/Pandoc/Highlighting.hs+6−6 src/Text/Pandoc/Image.hs+1−1 src/Text/Pandoc/ImageSize.hs+341−55 src/Text/Pandoc/Logging.hs+10−33 src/Text/Pandoc/Lua.hs+1−1 src/Text/Pandoc/Lua/ErrorConversion.hs+1−1 src/Text/Pandoc/Lua/Filter.hs+3−6 src/Text/Pandoc/Lua/Global.hs+1−1 src/Text/Pandoc/Lua/Init.hs+5−15 src/Text/Pandoc/Lua/Marshaling.hs+2−2 src/Text/Pandoc/Lua/Marshaling/AST.hs+81−79 src/Text/Pandoc/Lua/Marshaling/AnyValue.hs+1−1 src/Text/Pandoc/Lua/Marshaling/CommonState.hs+2−2 src/Text/Pandoc/Lua/Marshaling/Context.hs+2−3 src/Text/Pandoc/Lua/Marshaling/List.hs+2−2 src/Text/Pandoc/Lua/Marshaling/MediaBag.hs+2−2 src/Text/Pandoc/Lua/Marshaling/PandocError.hs+1−1 src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs+2−2 src/Text/Pandoc/Lua/Marshaling/SimpleTable.hs+1−1 src/Text/Pandoc/Lua/Marshaling/Version.hs+1−1 src/Text/Pandoc/Lua/Module/MediaBag.hs+17−18 src/Text/Pandoc/Lua/Module/Pandoc.hs+23−29 src/Text/Pandoc/Lua/Module/System.hs+1−1 src/Text/Pandoc/Lua/Module/Types.hs+1−1 src/Text/Pandoc/Lua/Module/Utils.hs+2−2 src/Text/Pandoc/Lua/Packages.hs+27−10 src/Text/Pandoc/Lua/PandocLua.hs+13−20 src/Text/Pandoc/Lua/Util.hs+2−2 src/Text/Pandoc/Lua/Walk.hs+2−2 src/Text/Pandoc/MIME.hs+3−26 src/Text/Pandoc/MediaBag.hs+15−46 src/Text/Pandoc/Options.hs+18−19 src/Text/Pandoc/PDF.hs+16−34 src/Text/Pandoc/Parsing.hs+264−405 src/Text/Pandoc/Process.hs+1−1 src/Text/Pandoc/Readers.hs+49−56 src/Text/Pandoc/Readers/BibTeX.hs+9−15 src/Text/Pandoc/Readers/CSV.hs+9−9 src/Text/Pandoc/Readers/CommonMark.hs+9−61 src/Text/Pandoc/Readers/Creole.hs+6−5 src/Text/Pandoc/Readers/CslJson.hs+5−6 src/Text/Pandoc/Readers/DocBook.hs+82−95 src/Text/Pandoc/Readers/Docx.hs+42−59 src/Text/Pandoc/Readers/Docx/Combine.hs+6−6 src/Text/Pandoc/Readers/Docx/Parse.hs+120−214 src/Text/Pandoc/Readers/Docx/Parse/Styles.hs+25−32 src/Text/Pandoc/Readers/Docx/Util.hs+26−28 src/Text/Pandoc/Readers/DokuWiki.hs+9−10 src/Text/Pandoc/Readers/EPUB.hs+36−44 src/Text/Pandoc/Readers/FB2.hs+51−57 src/Text/Pandoc/Readers/HTML.hs+216−222 src/Text/Pandoc/Readers/HTML/Parsing.hs+19−62 src/Text/Pandoc/Readers/HTML/Table.hs+70−240 src/Text/Pandoc/Readers/HTML/TagCategories.hs+1−1 src/Text/Pandoc/Readers/HTML/Types.hs+1−1 src/Text/Pandoc/Readers/Haddock.hs+7−11 src/Text/Pandoc/Readers/Ipynb.hs+4−6 src/Text/Pandoc/Readers/JATS.hs+51−37 src/Text/Pandoc/Readers/Jira.hs+10−14 src/Text/Pandoc/Readers/LaTeX.hs+1416−264 src/Text/Pandoc/Readers/LaTeX/Citation.hs−210 src/Text/Pandoc/Readers/LaTeX/Inline.hs−397 src/Text/Pandoc/Readers/LaTeX/Lang.hs+121−200 src/Text/Pandoc/Readers/LaTeX/Macro.hs−184 src/Text/Pandoc/Readers/LaTeX/Math.hs−221 src/Text/Pandoc/Readers/LaTeX/Parsing.hs+54−182 src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+71−152 src/Text/Pandoc/Readers/LaTeX/Table.hs−378 src/Text/Pandoc/Readers/LaTeX/Types.hs+2−15 src/Text/Pandoc/Readers/Man.hs+11−18 src/Text/Pandoc/Readers/Markdown.hs+239−287 src/Text/Pandoc/Readers/MediaWiki.hs+14−21 src/Text/Pandoc/Readers/Metadata.hs+18−45 src/Text/Pandoc/Readers/Muse.hs+9−10 src/Text/Pandoc/Readers/Native.hs+6−8 src/Text/Pandoc/Readers/OPML.hs+36−22 src/Text/Pandoc/Readers/Odt.hs+25−28 src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2 src/Text/Pandoc/Readers/Odt/ContentReader.hs+13−19 src/Text/Pandoc/Readers/Odt/Generic/Namespaces.hs+1−2 src/Text/Pandoc/Readers/Odt/Generic/Utils.hs+24−9 src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+14−13 src/Text/Pandoc/Readers/Odt/Namespaces.hs+5−6 src/Text/Pandoc/Readers/Odt/StyleReader.hs+17−21 src/Text/Pandoc/Readers/Org.hs+8−5 src/Text/Pandoc/Readers/Org/BlockStarts.hs+1−1 src/Text/Pandoc/Readers/Org/Blocks.hs+21−59 src/Text/Pandoc/Readers/Org/DocumentTree.hs+1−1 src/Text/Pandoc/Readers/Org/ExportSettings.hs+1−1 src/Text/Pandoc/Readers/Org/Inlines.hs+13−13 src/Text/Pandoc/Readers/Org/Meta.hs+5−11 src/Text/Pandoc/Readers/Org/ParserState.hs+1−1 src/Text/Pandoc/Readers/Org/Parsing.hs+6−6 src/Text/Pandoc/Readers/Org/Shared.hs+1−1 src/Text/Pandoc/Readers/RST.hs+116−85 src/Text/Pandoc/Readers/Roff.hs+18−16 src/Text/Pandoc/Readers/TWiki.hs+27−7 src/Text/Pandoc/Readers/Textile.hs+90−96 src/Text/Pandoc/Readers/TikiWiki.hs+6−6 src/Text/Pandoc/Readers/Txt2Tags.hs+22−48 src/Text/Pandoc/Readers/Vimwiki.hs+7−12 src/Text/Pandoc/RoffChar.hs+1−1 src/Text/Pandoc/SelfContained.hs+8−5 src/Text/Pandoc/Shared.hs+101−52 src/Text/Pandoc/Slides.hs+1−1 src/Text/Pandoc/Sources.hs−195 src/Text/Pandoc/Templates.hs+1−5 src/Text/Pandoc/Translations.hs+1−1 src/Text/Pandoc/UTF8.hs+24−21 src/Text/Pandoc/UUID.hs+1−1 src/Text/Pandoc/Writers.hs+4−9 src/Text/Pandoc/Writers/AnnotatedTable.hs−23 src/Text/Pandoc/Writers/AsciiDoc.hs+12−43 src/Text/Pandoc/Writers/BibTeX.hs−61 src/Text/Pandoc/Writers/CommonMark.hs+1−1 src/Text/Pandoc/Writers/ConTeXt.hs+39−45 src/Text/Pandoc/Writers/CslJson.hs+9−10 src/Text/Pandoc/Writers/Custom.hs+1−1 src/Text/Pandoc/Writers/Docbook.hs+33−114 src/Text/Pandoc/Writers/Docx.hs+395−238 src/Text/Pandoc/Writers/Docx/StyleMap.hs+1−1 src/Text/Pandoc/Writers/Docx/Table.hs−227 src/Text/Pandoc/Writers/Docx/Types.hs−185 src/Text/Pandoc/Writers/DokuWiki.hs+2−4 src/Text/Pandoc/Writers/EPUB.hs+212−269 src/Text/Pandoc/Writers/FB2.hs+77−61 src/Text/Pandoc/Writers/GridTable.hs−157 src/Text/Pandoc/Writers/HTML.hs+59−136 src/Text/Pandoc/Writers/Haddock.hs+2−13 src/Text/Pandoc/Writers/ICML.hs+1−6 src/Text/Pandoc/Writers/Ipynb.hs+1−1 src/Text/Pandoc/Writers/JATS.hs+56−99 src/Text/Pandoc/Writers/JATS/References.hs−164 src/Text/Pandoc/Writers/JATS/Table.hs+6−26 src/Text/Pandoc/Writers/JATS/Types.hs+4−15 src/Text/Pandoc/Writers/Jira.hs+17−42 src/Text/Pandoc/Writers/LaTeX.hs+835−96 src/Text/Pandoc/Writers/LaTeX/Caption.hs−47 src/Text/Pandoc/Writers/LaTeX/Citation.hs−181 src/Text/Pandoc/Writers/LaTeX/Lang.hs−192 src/Text/Pandoc/Writers/LaTeX/Notes.hs−34 src/Text/Pandoc/Writers/LaTeX/Table.hs−307 src/Text/Pandoc/Writers/LaTeX/Types.hs−83 src/Text/Pandoc/Writers/LaTeX/Util.hs−275 src/Text/Pandoc/Writers/Man.hs+3−3 src/Text/Pandoc/Writers/Markdown.hs+657−85 src/Text/Pandoc/Writers/Markdown/Inline.hs−616 src/Text/Pandoc/Writers/Markdown/Types.hs−81 src/Text/Pandoc/Writers/MediaWiki.hs+1−1 src/Text/Pandoc/Writers/Ms.hs+10−28 src/Text/Pandoc/Writers/Muse.hs+3−6 src/Text/Pandoc/Writers/Native.hs+1−1 src/Text/Pandoc/Writers/ODT.hs+22−28 src/Text/Pandoc/Writers/OOXML.hs+22−22 src/Text/Pandoc/Writers/OPML.hs+1−1 src/Text/Pandoc/Writers/OpenDocument.hs+72−123 src/Text/Pandoc/Writers/Org.hs+19−39 src/Text/Pandoc/Writers/Powerpoint.hs+1−1 src/Text/Pandoc/Writers/Powerpoint/Output.hs+148−172 src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+3−2 src/Text/Pandoc/Writers/RST.hs+22−38 src/Text/Pandoc/Writers/RTF.hs+11−17 src/Text/Pandoc/Writers/Roff.hs+1−1 src/Text/Pandoc/Writers/Shared.hs+6−20 src/Text/Pandoc/Writers/TEI.hs+4−5 src/Text/Pandoc/Writers/Texinfo.hs+16−23 src/Text/Pandoc/Writers/Textile.hs+1−1 src/Text/Pandoc/Writers/ZimWiki.hs+3−5 src/Text/Pandoc/XML.hs+20−42 src/Text/Pandoc/XML/Light.hs−89 src/Text/Pandoc/XML/Light/Output.hs−234 src/Text/Pandoc/XML/Light/Proc.hs−139 src/Text/Pandoc/XML/Light/Types.hs−193 stack.yaml+22−11 test/Tests/Command.hs+39−34 test/Tests/Helpers.hs+36−25 test/Tests/Lua.hs+3−1 test/Tests/Lua/Module.hs+1−3 test/Tests/Old.hs+27−13 test/Tests/Readers/Creole.hs+3−1 test/Tests/Readers/Docx.hs+7−25 test/Tests/Readers/DokuWiki.hs+2 test/Tests/Readers/EPUB.hs+4−2 test/Tests/Readers/FB2.hs+2 test/Tests/Readers/HTML.hs+8−21 test/Tests/Readers/JATS.hs+2−1 test/Tests/Readers/Jira.hs+2−22 test/Tests/Readers/LaTeX.hs+21−2 test/Tests/Readers/Man.hs+2−1 test/Tests/Readers/Markdown.hs+6−4 test/Tests/Readers/Muse.hs+2 test/Tests/Readers/Odt.hs+3−2 test/Tests/Readers/Org.hs+1−1 test/Tests/Readers/Org/Block.hs+3−1 test/Tests/Readers/Org/Block/CodeBlock.hs+6−4 test/Tests/Readers/Org/Block/Figure.hs+3−1 test/Tests/Readers/Org/Block/Header.hs+3−1 test/Tests/Readers/Org/Block/List.hs+3−14 test/Tests/Readers/Org/Block/Table.hs+3−1 test/Tests/Readers/Org/Directive.hs+3−1 test/Tests/Readers/Org/Inline.hs+5−3 test/Tests/Readers/Org/Inline/Citation.hs+3−41 test/Tests/Readers/Org/Inline/Note.hs+3−1 test/Tests/Readers/Org/Inline/Smart.hs+3−1 test/Tests/Readers/Org/Meta.hs+5−15 test/Tests/Readers/Org/Shared.hs+3−1 test/Tests/Readers/RST.hs+3−1 test/Tests/Readers/Txt2Tags.hs+3−1 test/Tests/Shared.hs+3−1 test/Tests/Writers/AnnotatedTable.hs+1 test/Tests/Writers/AsciiDoc.hs+2 test/Tests/Writers/ConTeXt.hs+115−122 test/Tests/Writers/Docbook.hs+2−103 test/Tests/Writers/Docx.hs+2−15 test/Tests/Writers/FB2.hs+2 test/Tests/Writers/HTML.hs+54−57 test/Tests/Writers/JATS.hs+93−135 test/Tests/Writers/Jira.hs+1−60 test/Tests/Writers/LaTeX.hs+2 test/Tests/Writers/Markdown.hs+2 test/Tests/Writers/Ms.hs+2 test/Tests/Writers/Muse.hs+1 test/Tests/Writers/Native.hs+2 test/Tests/Writers/OOXML.hs+4−5 test/Tests/Writers/Org.hs+13−46 test/Tests/Writers/Plain.hs+2 test/Tests/Writers/Powerpoint.hs+2 test/Tests/Writers/RST.hs+2 test/Tests/Writers/TEI.hs+2 test/command/1279.md+1−1 test/command/1881.md+3−3 test/command/2103.md−8 test/command/2378.md+3−3 test/command/2874.md+2−2 test/command/3516.md+5−1 test/command/3752.md−60 test/command/3792.md+1−1 test/command/4063.md−29 test/command/4164.md+1−1 test/command/4420.md+11 test/command/4442.md+1−1 test/command/4465.md−15 test/command/5304.md+1−1 test/command/5321.md+2−2 test/command/5367.md+3−7 test/command/5519.md+1−1 test/command/5541-nesting.md−1 test/command/5686.md+1−1 test/command/5708.md+5−1 test/command/5986.md+2−3 test/command/6026.md−19 test/command/6388.md−16 test/command/6541.md−27 test/command/6620.md−3 test/command/6658.md−75 test/command/6709.md−11 test/command/6774.md−63 test/command/6890.md+2−2 test/command/6925.md−34 test/command/6948.md−31 test/command/6951.md−18 test/command/6958.md−10 test/command/6993.md−21 test/command/7003.md−37 test/command/7006.md−20 test/command/7009.md−8 test/command/7016.md−48 test/command/7041.md−23 test/command/7042.md−146 test/command/7058.md−6 test/command/7064.md−32 test/command/7067.md−90 test/command/7080.md−8 test/command/7092.md−8 test/command/7099.md−18 test/command/7112.md−15 test/command/7129.md−27 test/command/7132.md−10 test/command/7134.md−16 test/command/7145.md−12 test/command/7155.md−15 test/command/7173.md−137 test/command/7208.md−6 test/command/7214.md−28 test/command/7216.md−19 test/command/7266.md−10 test/command/7272.md−27 test/command/7278.md−38 test/command/7282.md−7 test/command/7288.md−40 test/command/7299.md−23 test/command/7321.md−24 test/command/7323.md−29 test/command/7324.md−25 test/command/7339.md−11 test/command/7340.md−6 test/command/7376.md−16 test/command/7394.md−85 test/command/7397.md−14 test/command/7400.md−9 test/command/7416.md−19 test/command/averroes.bib−95 test/command/biblatex-baez-article.md+1−1 test/command/biblatex-baez-online.md+1−1 test/command/biblatex-basic.md+3−5 test/command/biblatex-cotton.md+3−3 test/command/biblatex-examples.bib+1674 test/command/biblatex-inproceedings.md+1−1 test/command/biblatex-itzhaki.md+1−1 test/command/biblatex-jaffe.md+2−2 test/command/biblatex-moraux.md+1−1 test/command/biblatex-murray.md+2−3 test/command/biblatex-nussbaum.md+63 test/command/biblatex-quotes.md+1−1 test/command/biblatex-sigfridsson.md+1−1 test/command/biblatex-spiegelberg.md+2−2 test/command/biblatex-test-case-conversion.md+2−2 test/command/biblatex-textnormal.md+1−1 test/command/biblatex-wassenberg.md+1−1 test/command/biblatex-wilde.md+1−1 test/command/chap1/spider.png test/command/chap1/text.md−17 test/command/chap2/spider.png test/command/chap2/text.md−5 test/command/custom-attributes.html+16 test/command/defaults-inheritance-1.md−6 test/command/defaults-inheritance-2.md−5 test/command/defaults-inheritance-3.md−6 test/command/defaults3.yaml−4 test/command/defaults4.yaml−3 test/command/defaults5.yaml−2 test/command/defaults6.yaml−2 test/command/defaults7.yaml−2 test/command/defaults8.yaml−2 test/command/defaults9.yaml−1 test/command/duplicate_attributes.md−7 test/command/empty-inline-code.txt+6 test/command/latex-center.md−13 test/command/mmd-metadata.md−20 test/command/nested-table-to-asciidoc-6942.md−82 test/command/newif.md−55 test/command/pandoc-citeproc-119.md+1−1 test/command/pandoc-citeproc-320a.md+3−3 test/command/pandoc-citeproc-322.md+1−1 test/command/pandoc-citeproc-356.md+7−2 test/command/pandoc-citeproc-53.md+2−4 test/command/pandoc-citeproc-65.md+1−1 test/command/pandoc-citeproc-70.md+2−2 test/command/pandoc-citeproc-chicago-author-date.md+7−7 test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+18−15 test/command/svg.md+15−17 test/command/toc.md+2−2 test/command/unicode-collation.md−130 test/docbook-reader.docbook+2−9 test/docx/0_level_headers.native+2−2 test/docx/golden/block_quotes.docx test/docx/golden/codeblock.docx test/docx/golden/comments.docx test/docx/golden/custom_style_no_reference.docx test/docx/golden/custom_style_preserve.docx test/docx/golden/custom_style_reference.docx test/docx/golden/definition_list.docx test/docx/golden/document-properties-short-desc.docx test/docx/golden/document-properties.docx test/docx/golden/headers.docx test/docx/golden/image.docx test/docx/golden/inline_code.docx test/docx/golden/inline_formatting.docx test/docx/golden/inline_images.docx test/docx/golden/link_in_notes.docx test/docx/golden/links.docx test/docx/golden/lists.docx test/docx/golden/lists_continuing.docx test/docx/golden/lists_multiple_initial.docx test/docx/golden/lists_restarting.docx test/docx/golden/nested_anchors_in_header.docx test/docx/golden/notes.docx test/docx/golden/raw-blocks.docx test/docx/golden/raw-bookmarks.docx test/docx/golden/table_one_row.docx test/docx/golden/table_with_list_cell.docx test/docx/golden/tables-default-widths.docx test/docx/golden/tables.docx test/docx/golden/track_changes_deletion.docx test/docx/golden/track_changes_insertion.docx test/docx/golden/track_changes_move.docx test/docx/golden/track_changes_scrubbed_metadata.docx test/docx/golden/unicode.docx test/docx/golden/verbatim_subsuper.docx test/docx/image_vml_as_object.docx test/docx/image_vml_as_object.native−2 test/docx/raw-blocks.native−6 test/docx/raw-bookmarks.native−3 test/docx/sdt_elements.native+10−9 test/docx/table_captions_no_field.docx test/docx/table_captions_no_field.native−34 test/docx/table_captions_with_field.docx test/docx/table_captions_with_field.native−54 test/docx/table_header_rowspan.docx test/docx/table_header_rowspan.native−189 test/docx/table_one_header_row.docx test/docx/table_one_header_row.native−18 test/docx/table_one_row.docx test/docx/table_one_row.native+4−4 test/docx/table_variable_width.native+17−12 test/docx/table_with_list_cell.native+3−3 test/docx/tables-default-widths.native−92 test/docx/tables.native+9−9 test/epub/features.native+2−2 test/epub/wasteland.epub test/epub/wasteland.native+4−4 test/fb2/images.fb2+1−1 test/fb2/test.jpg test/grofftest.sh+26 test/html-reader.html−76 test/html-reader.native+37−148 test/jats-reader.native+1−1 test/jats-reader.xml−1 test/jira-reader.native+1−1 test/latex-reader.native+13−26 test/lhs-test.html+1−3 test/lhs-test.html+lhs+1−3 test/lhs-test.latex+3−3 test/lhs-test.latex+lhs+3−3 test/lua/module/pandoc-path.lua−44 test/markdown-reader-more.native+33−5 test/odt/native/tab.native−1 test/odt/odt/tab.odt test/pipe-tables.native+10−2 test/pptx/code-custom.pptx test/pptx/code-custom_templated.pptx test/pptx/code.pptx test/pptx/code_templated.pptx test/pptx/document-properties-short-desc.pptx test/pptx/document-properties-short-desc_templated.pptx test/pptx/document-properties.pptx test/pptx/document-properties_templated.pptx test/pptx/endnotes.pptx test/pptx/endnotes_templated.pptx test/pptx/endnotes_toc.pptx test/pptx/endnotes_toc_templated.pptx test/pptx/images.pptx test/pptx/images_templated.pptx test/pptx/inline_formatting.pptx test/pptx/inline_formatting_templated.pptx test/pptx/lists.pptx test/pptx/lists_templated.pptx test/pptx/raw_ooxml.native+1−1 test/pptx/raw_ooxml.pptx test/pptx/raw_ooxml_templated.pptx test/pptx/reference_depth.pptx test/pptx/remove_empty_slides.pptx test/pptx/remove_empty_slides_templated.pptx test/pptx/slide_breaks.pptx test/pptx/slide_breaks_slide_level_1.pptx test/pptx/slide_breaks_slide_level_1_templated.pptx test/pptx/slide_breaks_templated.pptx test/pptx/slide_breaks_toc.pptx test/pptx/slide_breaks_toc_templated.pptx test/pptx/speaker_notes.pptx test/pptx/speaker_notes_after_metadata.pptx test/pptx/speaker_notes_after_metadata_templated.pptx test/pptx/speaker_notes_afterheader.pptx test/pptx/speaker_notes_afterheader_templated.pptx test/pptx/speaker_notes_afterseps.pptx test/pptx/speaker_notes_afterseps_templated.pptx test/pptx/speaker_notes_templated.pptx test/pptx/start_numbering_at.pptx test/pptx/start_numbering_at_templated.pptx test/pptx/tables.pptx test/pptx/tables_templated.pptx test/pptx/two_column.pptx test/pptx/two_column_templated.pptx test/rst-reader.native+21−3 test/tables-rstsubset.native+9−1 test/tables.latex+35−50 test/tables.ms−44 test/tables/nordics.latex−42 test/tables/planets.latex−30 test/tables/students.latex−31 test/test-pandoc.hs+8−18 test/txt2tags.native+1−1 test/txt2tags.t2t+1−1 test/writer.context+40−42 test/writer.html4+1−2 test/writer.html5+1−2 test/writer.icml−2 test/writer.jats_archiving+8−10 test/writer.jats_articleauthoring+1−1 test/writer.jats_publishing+8−10 test/writer.jira+35−23 test/writer.latex+3−3 test/writer.ms+1−1 test/writer.opml+1−1 test/writer.org+7−7 test/writer.zimwiki+3−3 test/writers-lang-and-dir.context+2−4 test/writers-lang-and-dir.latex+6−6 tools/build-arm.sh−110 tools/changelog-helper.sh+4−11 tools/parseTimings.pl−33 trypandoc/index.html+1−1 trypandoc/trypandoc.hs+8−3 windows/Makefile+1−1 windows/pandoc.wxs+2−2
.circleci/config.yml+122−64
@@ -1,72 +1,130 @@
11version: "2.1"
2−jobs:
3− build:
4− macos:
5− xcode: 11.1.0
2+
3+env: &env
64 environment:
7− HOMEBREW_NO_AUTO_UPDATE: 1
5+ LC_ALL: "C.UTF-8"
6+ docker:
7+ - image: fpco/stack-build:lts-13
8+
9+#-----------------------------------------------------------------------------
10+# Common utility stuff, not to be modified usually
11+#-----------------------------------------------------------------------------
12+
13+preinstall: &preinstall
14+ run: |
15+ echo 'export PATH=/opt/cabal/bin:/opt/ghc/bin:$PATH' >> $BASH_ENV
16+ source $BASH_ENV
17+ apt-get update
18+
19+restore: &restore
20+ # Needs to happen after installing ca-certificates
21+ restore_cache:
22+ key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
23+
24+save: &save
25+ save_cache:
26+ key: v1-{{ .Environment.CIRCLE_JOB }}-{{ .Branch }}
27+ paths:
28+ - ~/.cabal
29+ - ~/.ghc
30+ - ~/.local
31+ - ~/.stack
32+
33+commands:
34+ cabal_build:
35+ parameters:
36+ ghcversion:
37+ type: string
838 steps:
9− - checkout
10− - restore_cache:
11− # See https://circleci.com/docs/2.0/caching/
12− name: Restore Cached Dependencies
13− keys:
14− - stack-dependencies-{{ checksum "stack.yaml" }}
15− - run:
16− name: Install stack
17− command: stack --version || (curl -sSL https://get.haskellstack.org/ | sh -s - -f)
18− - run:
19− name: Resolve/Update Dependencies
20− command: stack --no-terminal setup
21− - run:
22− name: Run tests
23− command: stack --no-terminal test
24− - run:
25− name: Install executable
26− command: stack --no-terminal install
27− - run:
28− name: Create artifacts
29− command: |
30− export VERSION=$(grep '^[Vv]ersion:' pandoc.cabal | awk '{print $$2;}')
31− export ARTIFACTS=macos-release-candidate
32− export RESOURCES=${ARTIFACTS}/Resources
33− export ROOT=${ARTIFACTS}/pandoc
34− export DEST=${ROOT}/usr/local
35− export ME=$(whoami)
36− export BASE=pandoc-$VERSION
37− mkdir -p ${ARTIFACTS}
38− mkdir -p ${RESOURCES}
39− mkdir -p ${DEST}/bin
40− mkdir -p ${DEST}/share/man/man1
41− cp ~/.local/bin/pandoc ${DEST}/bin/
42− strip ${DEST}/bin/pandoc
43− cp man/pandoc.1 ${DEST}/share/man/man1/pandoc.1
44− ~/.local/bin/pandoc -s COPYING.md -Vpagetitle=License -o ${RESOURCES}/license.html
45− chown -R $ME:staff ${ROOT}
46− sed -e "s/PANDOCVERSION/${VERSION}/" macos/distribution.xml.in > ${ARTIFACTS}/distribution.xml
47− cp macos/Makefile ${ARTIFACTS}/
48− echo ${VERSION} > ${ARTIFACTS}/version.txt
49− zip -r macos-release-candidate.zip ${ARTIFACTS}
39+ - checkout
40+ - *preinstall
41+ - *restore
42+ - run:
43+ name: install cabal
44+ command: apt-get install -y cabal-install-2.4
45+ - run:
46+ name: install ghc
47+ command: |
48+ apt-get install -y ghc-<< parameters.ghcversion >>
49+ - run:
50+ name: build and test project
51+ command: |
52+ set -e
53+ cabal v2-update
54+ TMPDIR=`mktemp -d -t circleci.XXXXXXX`
55+ cabal v2-sdist --output-dir=${TMPDIR}
56+ cd ${TMPDIR}
57+ tar xvzf *.tar.gz
58+ cd *
59+ cabal v2-build -w ghc-<< parameters.ghcversion >> -j2 --disable-optimization --dependencies-only --enable-tests --enable-benchmarks
60+ cabal v2-build -w ghc-<< parameters.ghcversion >> -j2 --disable-optimization --enable-tests --enable-benchmarks 2>build.log
61+ cat build.log
62+ # fail if we had warnings in local build
63+ # this is to work around the fact that cabal v2 doesn't allow
64+ # us to use -Werror for just local build:
65+ # https://github.com/haskell/cabal/issues/4247
66+ ! grep -q "[Ww]arning:" build.log
67+ cabal v2-test -w ghc-<< parameters.ghcversion >> -j2 --disable-optimization --enable-tests
68+ cabal v2-haddock -w ghc-<< parameters.ghcversion >>
69+ cabal check
70+ - *save
5071
51− - save_cache:
52− name: Cache Dependencies
53− key: stack-dependencies-{{ checksum "stack.yaml" }}
54− paths:
55− - "/root/.stack"
56− - ".stack-work"
57− - "/usr/local/bin/stack"
72+#-----------------------------------------------------------------------------
73+# Build matrix
74+#-----------------------------------------------------------------------------
5875
59− - store_artifacts:
60− # https://circleci.com/docs/2.0/artifacts/
61− path: macos-release-candidate.zip
62− destination: macos-release-candidate.zip
76+jobs:
77+ cabal-ghc-8_8_1:
78+ <<: *env
79+ steps:
80+ - cabal_build:
81+ ghcversion: 8.8.1
82+
83+ cabal-ghc-8_6_5:
84+ <<: *env
85+ steps:
86+ - cabal_build:
87+ ghcversion: 8.6.5
88+
89+ cabal-ghc-8_4_4:
90+ <<: *env
91+ steps:
92+ - cabal_build:
93+ ghcversion: 8.4.4
94+
95+ cabal-ghc-8_2_2:
96+ <<: *env
97+ steps:
98+ - cabal_build:
99+ ghcversion: 8.2.2
100+
101+ cabal-ghc-8_0_2:
102+ <<: *env
103+ steps:
104+ - cabal_build:
105+ ghcversion: 8.0.2
106+
107+ stack-ghc-8_6:
108+ <<: *env
109+ steps:
110+ - checkout
111+ - *preinstall
112+ - *restore
113+ - run: |
114+ set -e
115+ apt-get install -y ghc-8.6.5
116+ stack update
117+ stack test --system-ghc --fast --ghc-options="-Werror" .
118+ - *save
63119
64120workflows:
65− version: 2
66− release:
121+ version: "2.1"
122+ build:
67123 jobs:
68− - build:
69− filters:
70− branches:
71− only:
72− - /rc\/.*/
124+ - cabal-ghc-8_6_5
125+ - cabal-ghc-8_4_4
126+ - cabal-ghc-8_2_2
127+ - cabal-ghc-8_0_2
128+ - stack-ghc-8_6
129+ # - cabal-ghc-8_8_1
130+
.editorconfig−8
@@ -8,14 +8,6 @@ indent_size = 2
88insert_final_newline = true
99trim_trailing_whitespace = true
1010
11−[*.{zip,docx}]
12−charset =
13−end_of_line =
14−indent_style =
15−indent_size =
16−insert_final_newline =
17−trim_trailing_whitespace =
18−
1911[*.{markdown,md}]
2012trim_trailing_whitespace = false
2113
.github/ISSUE_TEMPLATEnew file+11
@@ -0,0 +1,11 @@
1+Asking a question?
2+Please use the pandoc-discuss mailing list instead:
3+https://groups.google.com/forum/#!forum/pandoc-discuss
4+This tracker is for bug reports and enhancement requests.
5+
6+Reporting a bug?
7+Please include your pandoc version and an example that allows us
8+to reproduce the problem (complete input, exact command line
9+used, expected and actual output). The smaller the example, the
10+better.
11+
.github/ISSUE_TEMPLATE/bug_report.mddeleted−25
@@ -1,25 +0,0 @@
1−---
2−name: Bug report
3−about: Create a report to help us improve
4−title: ''
5−labels: 'bug'
6−assignees: ''
7−
8−---
9−
10−<!--
11−Thank you for reporting an issue! Before you continue, please make sure that you have
12−
13−- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
14−- searched the pandoc-discuss mailing list for solutions: https://groups.google.com/forum/#!forum/pandoc-discuss
15−- reproduced your issue on the latest release of pandoc, or on https://pandoc.org/try
16−
17−Note that this bug tracker is for reporting bugs, not asking questions. For questions, use the pandoc-discuss mailing list.
18−-->
19−
20−**Explain the problem.**
21−Include the exact command line you used and all inputs necessary to reproduce the issue. Please create as minimal an example as possible, to help the maintainers isolate the problem. Explain the output you received and how it differs from what you expected.
22−
23−**Pandoc version?**
24−What version of pandoc are you using, on what OS?
25−
.github/ISSUE_TEMPLATE/config.ymldeleted−9
@@ -1,9 +0,0 @@
1−blank_issues_enabled: false
2−contact_links:
3− - name: Ask a question
4− url: https://groups.google.com/forum/#!forum/pandoc-discuss
5− about: "Discussion forum for pandoc (may be used as an email list)"
6− - name: Contributing Guidelines
7− url: https://pandoc.org/CONTRIBUTING.html
8− about: "Guidelines for submitting bug reports and code improvements"
9−
.github/ISSUE_TEMPLATE/feature_request.mddeleted−21
@@ -1,21 +0,0 @@
1−---
2−name: Feature request
3−about: Suggest an idea for this project
4−title: ''
5−labels: 'enhancement'
6−assignees: ''
7−
8−---
9−
10−<!--
11−Thank you for suggesting a feature! Before you continue, please make sure that you have
12−
13−- read the contributing guidelines: https://pandoc.org/CONTRIBUTING.html
14−- searched the issue tracker for similar issues (including closed issues): https://github.com/jgm/pandoc/issues
15−- searched the pandoc-discuss mailing list for relevant discussions: https://groups.google.com/forum/#!forum/pandoc-discuss
16−-->
17−
18−**Describe your proposed improvement and the problem it solves.**
19−
20−**Describe alternatives you've considered.**
21−
.github/workflows/ci.yml+8−92
@@ -45,50 +45,23 @@ jobs:
4545 versions:
4646 - ghc: '8.0.2'
4747 cabal: '2.4'
48− prefix: 'v2-'
49− cabalopts: ''
50− testopts: ''
5148 - ghc: '8.2.2'
5249 cabal: '2.4'
53− prefix: 'v2-'
54− cabalopts: ''
55− testopts: ''
5650 - ghc: '8.4.4'
5751 cabal: '2.4'
58− prefix: 'v2-'
59− cabalopts: '-f-embed_data_files'
60− testopts: ''
6152 - ghc: '8.6.5'
6253 cabal: '3.2'
63− prefix: ''
64− cabalopts: ''
65− testopts: '--test-option=--hide-successes'
6654 - ghc: '8.8.4'
6755 cabal: '3.2'
68− prefix: ''
69− cabalopts: ''
70− testopts: '--test-option=--hide-successes'
71− - ghc: '8.10.4'
56+ - ghc: '8.10.2'
7257 cabal: '3.2'
73− prefix: ''
74− cabalopts: '-ftrypandoc'
75− testopts: '--test-option=--hide-successes'
76− - ghc: '9.0.1'
77− cabal: '3.4'
78− prefix: ''
79− cabalopts: ''
80− testopts: '--test-option=--hide-successes'
8158 steps:
8259 - uses: actions/checkout@v2
8360
84− # needed by memory
85− - name: Install numa
86− run: sudo apt-get install libnuma-dev
87−
8861 # need to install older cabal/ghc versions from ppa repository
8962
9063 - name: Install recent cabal/ghc
91− uses: haskell/actions/setup@v1
64+ uses: actions/setup-haskell@v1.1.3
9265 with:
9366 ghc-version: ${{ matrix.versions.ghc }}
9467 cabal-version: ${{ matrix.versions.cabal }}
@@ -115,15 +88,14 @@ jobs:
11588
11689 - name: Install dependencies
11790 run: |
118− cabal ${{ matrix.versions.prefix }}update
119− cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --dependencies-only --enable-tests --disable-optimization
120−
91+ cabal v2-update
92+ cabal v2-build --dependencies-only --enable-tests --disable-optimization
12193 - name: Build and test
12294 run: |
123− cabal ${{ matrix.versions.prefix }}build ${{ matrix.versions.cabalopts }} --enable-tests --disable-optimization 2>&1 | tee build.log
95+ cabal v2-build --enable-tests --disable-optimization 2>&1 | tee build.log
12496 # fail if warnings in local build
12597 ! grep -q ": *[Ww]arning:" build.log || exit 1
126− cabal ${{ matrix.versions.prefix }}test ${{ matrix.versions.cabalopts }} --disable-optimization ${{ matrix.versions.testopts }}
98+ cabal v2-test --disable-optimization
12799
128100 windows:
129101
@@ -133,7 +105,7 @@ jobs:
133105 - uses: actions/checkout@v2
134106
135107 # We should have ghc 8.10, cabal, stack pre-installed
136− # - uses: haskell/actions/setup@v1
108+ # - uses: actions/setup-haskell@v1.1.3
137109 # with:
138110 # enable-stack: true
139111 # stack-version: 'latest'
@@ -182,7 +154,7 @@ jobs:
182154 - uses: actions/checkout@v2
183155
184156 - name: Install recent cabal/ghc
185− uses: haskell/actions/setup@v1
157+ uses: actions/setup-haskell@v1.1.3
186158 with:
187159 ghc-version: ${{ matrix.versions.ghc }}
188160 cabal-version: ${{ matrix.versions.cabal }}
@@ -217,59 +189,3 @@ jobs:
217189 # fail if warnings in local build
218190 ! grep -q ": *[Ww]arning:" build.log || exit 1
219191 cabal v2-test --disable-optimization
220−
221− benchmark:
222−
223− runs-on: ubuntu-18.04
224− strategy:
225− fail-fast: true
226− matrix:
227− versions:
228− - ghc: '8.8.4'
229− cabal: '3.2'
230− - ghc: '8.10.2'
231− cabal: '3.2'
232− steps:
233− - uses: actions/checkout@v2
234−
235− # need to install older cabal/ghc versions from ppa repository
236−
237− - name: Install recent cabal/ghc
238− uses: haskell/actions/setup@v1
239− with:
240− ghc-version: ${{ matrix.versions.ghc }}
241− cabal-version: ${{ matrix.versions.cabal }}
242−
243− # declare/restore cached things
244− # caching doesn't work for scheduled runs yet
245− # https://github.com/actions/cache/issues/63
246−
247− - name: Cache cabal global package db
248− id: cabal-global
249− uses: actions/cache@v2
250− with:
251− path: |
252− ~/.cabal
253− key: benchmark-${{ runner.os }}-${{ matrix.versions.ghc }}-${{ matrix.versions.cabal }}-cabal-global-${{ hashFiles('cabal.project') }}
254−
255− - name: Cache cabal work
256− id: cabal-local
257− uses: actions/cache@v2
258− with:
259− path: |
260− dist-newstyle
261− key: benchmark-${{ runner.os }}-${{ matrix.versions.ghc }}-${{ matrix.versions.cabal }}-cabal-local
262−
263− - name: Install dependencies
264− run: |
265− v2=$([ "${{ matrix.versions.cabal }}" = "2.2" ] && printf 'new' || printf 'v2')
266− cabal $v2-update
267− cabal $v2-build --dependencies-only --enable-optimization=1 --enable-benchmarks --disable-tests
268−
269− - name: Build and test
270− run: |
271− v2=$([ "${{ matrix.versions.cabal }}" = "2.2" ] && printf 'new' || printf 'v2')
272− cabal $v2-build --enable-optimization=1 --enable-benchmarks --disable-tests 2>&1 | tee build.log
273− # fail if warnings in local build
274− ! grep -q ": *[Ww]arning:" build.log || exit 1
275− cabal $v2-bench --enable-optimization=1 --benchmark-options='--timeout=6 +RTS -T -RTS'
.github/workflows/lint.yml−3
@@ -25,9 +25,6 @@ jobs:
2525 - name: Checkout
2626 uses: actions/checkout@v2
2727
28− - name: Check cabal file for missing test files
29− run: make check-cabal
30−
3128 - name: Download and run hlint
3229 run: |
3330 curl -sSL "${hlint_script}" | sh -s .
.github/workflows/nightly.yml+2−9
@@ -10,22 +10,15 @@ jobs:
1010 runs-on: ubuntu-18.04
1111 steps:
1212 - uses: actions/checkout@v2
13−
14− - name: Install recent cabal/ghc
15− uses: haskell/actions/setup@v1
16− with:
17− ghc-version: '8.10.3'
18− cabal-version: '3.2'
19−
2013 - name: Install dependencies
2114 run: |
2215 ghc --version
2316 cabal --version
2417 cabal v2-update
25− cabal v2-build --dependencies-only
18+ cabal v2-build --dependencies-only all
2619 - name: Build
2720 run: |
28− cabal v2-install
21+ cabal v2-install all
2922 strip $HOME/.cabal/bin/pandoc
3023 - name: Install artifact
3124 run: |
.github/workflows/release-candidate.yml+1
@@ -32,6 +32,7 @@ jobs:
3232 - osarch: "windows-x86_64"
3333 arch: "x64"
3434 stack_opts: "-j1"
35+
3536 steps:
3637 - uses: actions/checkout@v1
3738 - name: Install dependencies
.gitignore−7
@@ -1,6 +1,4 @@
11*~
2−.DS_Store
3−*.orig
42deb/.vagrant
53dist/*
64MANUAL.*
@@ -16,10 +14,6 @@ man/man?/*.html
1614*.o
1715*.hi
1816*.pyc
19−.hie/
20−.policeman-evidence/
21−.projectile/
22−.vscode/
2317/COPYING.rtf
2418/COPYRIGHT.txt
2519/cabal-dev/
@@ -32,4 +26,3 @@ data/reference.odt
3226/stack.yaml.lock
3327cabal.project.local
3428/dist-newstyle/
35−pandoc-2*/
.hlint.yaml+5−9
@@ -9,13 +9,11 @@
99# Ignore some builtin hints
1010#
1111- ignore: {name: "Avoid lambda"}
12−- ignore: {name: "Use bimap"}
1312- ignore: {name: "Eta reduce"}
1413- ignore: {name: "Evaluate"}
1514- ignore: {name: "Reduce duplication"} # TODO: could be more fine-grained
1615- ignore: {name: "Use &&&"}
1716- ignore: {name: "Use String"}
18−- ignore: {name: "Use camelCase"}
1917- ignore: {name: "Use fmap"} # specific for GHC 7.8 compat
2018- ignore: {name: "Use isDigit"}
2119
@@ -40,10 +38,13 @@
4038 name: "Redundant return"
4139 within: Text.Pandoc.Citeproc.BibTeX
4240
43−# With recent hlint this rule has given false positives on TH
41+# TODO: check
4442- ignore:
4543 name: "Redundant bracket"
46−
44+ within:
45+ - Text.Pandoc.Citeproc
46+ - Text.Pandoc.Citeproc.BibTeX
47+ - Text.Pandoc.Citeproc.MetaValue
4748
4849- ignore:
4950 name: "Use <$>"
@@ -59,7 +60,6 @@
5960 - Text.Pandoc.Citeproc
6061 - Text.Pandoc.Extensions
6162 - Text.Pandoc.Lua.Marshaling.Version
62− - Text.Pandoc.Lua.Module.Pandoc
6363 - Text.Pandoc.Lua.Module.Utils
6464 - Text.Pandoc.Readers.Odt.ContentReader
6565 - Text.Pandoc.Readers.Odt.Namespaces
@@ -104,9 +104,5 @@
104104 - Text.Pandoc.Readers.Markdown
105105 - Text.Pandoc.Readers.RST
106106
107−# fromRight is only in base >= 4.10
108−- ignore:
109− name: "Use fromRight"
110−
111107# Define some custom infix operators
112108# - fixity: infixr 3 ~^#^~
.travis.ymlnew file+54
@@ -0,0 +1,54 @@
1+os: osx
2+language: generic
3+cache:
4+ timeout: 1000
5+ directories:
6+ - "$HOME/.stack/"
7+ - ".stack-work/"
8+branches:
9+ only:
10+ - /rc\/.*/
11+install:
12+- set -e
13+- ulimit -n 4096
14+- brew install haskell-stack
15+- which stack
16+- stack --version
17+- stack setup
18+- stack exec -- ghc --version
19+- |
20+ export VERSION=$(grep '^[Vv]ersion:' pandoc.cabal | awk '{print $$2;}')
21+ export BASEDIR=$(pwd)
22+ export ARTIFACTS=${BASEDIR}/macos-release-candidate
23+ export RESOURCES=${ARTIFACTS}/Resources
24+ export ROOT=${ARTIFACTS}/pandoc
25+ export DEST=${ROOT}/usr/local
26+ export ME=$(whoami)
27+ export BASE=pandoc-$VERSION
28+ mkdir -p ${ARTIFACTS}
29+ mkdir -p ${RESOURCES}
30+ mkdir -p ${DEST}/bin
31+ mkdir -p ${DEST}/share/man/man1
32+ stack build --dependencies-only pandoc
33+ stack build pandoc
34+ for f in $(find .stack-work/install -name 'pandoc' -perm +001 -type f); do cp $f ${DEST}/bin/; done
35+ strip ${DEST}/bin/pandoc
36+ cp man/pandoc.1 ${DEST}/share/man/man1/pandoc.1
37+ ${DEST}/bin/pandoc -t html5 -s COPYING.md -Vpagetitle=License \
38+ -o ${RESOURCES}/license.html
39+ chown -R $ME:staff ${ROOT}
40+ sed -e "s/PANDOCVERSION/${VERSION}/" macos/distribution.xml.in > ${ARTIFACTS}/distribution.xml
41+ cp macos/Makefile ${ARTIFACTS}/
42+ echo ${VERSION} > ${ARTIFACTS}/version.txt
43+deploy:
44+ on:
45+ all_branches: true
46+ provider: s3
47+ bucket: travis-jgm-pandoc
48+ region: us-west-1
49+ edge: true
50+ local_dir: macos-release-candidate
51+ access_key_id:
52+ secure: ohEst8EP8Z3r/bPMlYgPATo3B/ye48Z0aGp4W1pEhDZjqufLUVTFBIoicvPApCkTrbKMNTdG+KmasrO/leHpju0uFKYQHDtJmHZe3avIOIdbGnOqn97+mmLZBN5skiJWNtrxnN/TuAM+wlePz1dQpohPt6J3S70ct8DsOdZOaGI=
53+ secret_access_key:
54+ secure: qZ2RN4Bx4WRcVz9H8ae4S6a3AA85kcca0RdMimIREdGSx4ewMYa8lSdcKDvRbPwXRy8H3SeviqL2tTZw3+ej1jccAHowMj8Zvde5EJwX0+qO4Sr0zdWjnsvMeUmx9pO1oL3L/09KTWiMTT+d4XNpE8eqiY08qN8oMj0EPeR/AS4=
AUTHORS.md−31
@@ -1,13 +1,11 @@
11# Contributors
22
33- Anabra
4−- Aner Lucero
54- Arata Mizuki
65- Arfon Smith
76- Aaron Wolen
87- Albert Krewinkel
98- Alex Ivkin
10−- Alex Toldaiev
119- Alex Vong
1210- Alexander Kondratskiy
1311- Alexander Krotov
@@ -20,7 +18,6 @@
2018- Andreas Lööw
2119- Andrew Dunning
2220- Andy Morris
23−- Anti-Distinctlyminty
2421- Antoine Latter
2522- Antonio Terceiro
2623- Arata Mizuki
@@ -36,7 +33,6 @@
3633- Benoit Schweblin
3734- Bjorn Buckwalter
3835- Blake Eryx
39−- Bodigrim
4036- Bradley Kuhn
4137- Brent Yorgey
4238- Brian Leung
@@ -47,7 +43,6 @@
4743- Carlos Sosa
4844- Cécile Chemin
4945- Cédric Couralet
50−- Charanjit Singh
5146- Chris Black
5247- Christian Conkle
5348- Christian Despres
@@ -65,23 +60,19 @@
6560- Daniele D'Orazio
6661- David A Roberts
6762- David Lazar
68−- David Martschenko
6963- David Röthlisberger
7064- Denis Laxalde
7165- Denis Maier
7266- Derek Chen-Becker
7367- Diego Balseiro
74−- Dimitri Sabadie
7568- Dmitry Pogodin
7669- Dmitry Volodin
7770- Douglas Calvert
7871- Eigil Rischel
7972- Emanuel Evans
8073- Emerson Harkin
81−- Emily Bourke
8274- Emily Eisenberg
8375- Eric Kow
84−- Erik Rask
8576- Eric Seidel
8677- Eric Schrijver
8778- Ethan Riley
@@ -91,7 +82,6 @@
9182- Florian Beeres
9283- Florian Eitel
9384- Florian Klink
94−- Florian Kohrt
9585- Francesco Occhipinti
9686- François Gannaz
9787- Frederik Elwert
@@ -114,7 +104,6 @@
114104- Hubert Plociniczak
115105- Ian Max Andolina
116106- Igor Khorlo
117−- Igor Pashev
118107- Ilya V. Portnov
119108- Ivan Trubach
120109- Ivo Clarysse
@@ -128,7 +117,6 @@
128117- Jamie F. Olson
129118- Jan Larres
130119- Jan Schulz
131−- Jan Tojnar
132120- Jan-Otto Kröpke
133121- Jason Ronallo
134122- Jeff Arnold
@@ -136,7 +124,6 @@
136124- Jens Getreu
137125- Jens Petersen
138126- Jeroen de Haas
139−- Jerry Sky
140127- Jesse Rosenthal
141128- Joe Hermaszewski
142129- Joe Hillenbrand
@@ -152,7 +139,6 @@
152139- Josef Svenningsson
153140- Joseph C. Sible
154141- Julien Cretel
155−- Julien Dutant
156142- Juliette Fourcot
157143- Juliusz Gonera
158144- Justin Bogner
@@ -170,8 +156,6 @@
170156- Leif Metcalf
171157- Leonard Rosenthol
172158- Lila
173−- Loïc Grobol
174−- Lorenzo
175159- Lucas Escot
176160- Luke Plant
177161- Marc Schreiber
@@ -205,12 +189,10 @@
205189- Nathan Gass
206190- Neil Mayhew
207191- Nick Bart
208−- Nick Berendsen
209192- Nick Fleisher
210193- Nicolas Kaiser
211194- Nikolay Yakimov
212195- Nils Carlson
213−- Nixon Enraght-Moony
214196- Nokome Bentley
215197- Ole Martin Ruud
216198- Oliver Matthews
@@ -235,7 +217,6 @@
235217- Raymond Ehlers
236218- Recai Oktaş
237219- Roland Hieber
238−- Roman Beránek
239220- RyanGlScott
240221- Salim B
241222- Sascha Wilde
@@ -246,14 +227,12 @@
246227- Shahbaz Youssefi
247228- Shaun Attfield
248229- Shim Myeongseob
249−- Shin Sang-jae
250230- Sidarth Kapur
251231- Sidharth Kapur
252232- Simon Hengel
253233- Stefan Dresselhaus
254234- Sumit Sahrawat
255235- Tarik Graba
256−- Tatiana Porras
257236- Thenaesh Elango
258237- Thomas Hodgson
259238- Thomas Weißschuh
@@ -287,12 +266,10 @@
287266- a-vrma
288267- andrebauer
289268- argent0
290−- badumont
291269- blmage
292270- bucklereed
293271- bumper314
294272- chinapedia
295−- cholonam
296273- csforste
297274- dbecher-ito
298275- d-dorazio
@@ -303,11 +280,8 @@
303280- kaizshang91
304281- lux-lth
305282- lwolfsonkin
306−- mbrackeantidot
307283- nkalvi
308284- niszet
309−- nuew
310−- obcat
311285- oltolm
312286- oquechy
313287- quasicomputational
@@ -318,13 +292,8 @@
318292- rski
319293- shreevatsa.public
320294- takahashim
321−- taotieren
322−- tecosaur
323−- TEC
324295- tgkokk
325296- the-solipsist
326−- timo-a
327297- thsutton
328298- vijayphoenix
329299- wiefling
330−- wuffi
CONTRIBUTING.md+25−97
@@ -1,64 +1,6 @@
11Contributing to pandoc
22======================
33
4−Welcome to pandoc! Very soon after its beginnings in 2006, pandoc
5−has been influenced, improved, and modified, by users, devs, and
6−newcomers alike. The project thrives on its active community. It is
7−great to have you here.
8−
9−How can I help?
10−---------------
11−
12−There are many ways in which you can support pandoc. Here are a few
13−ideas:
14−
15− * Participate in online discussions. The [pandoc-discuss] mailing
16− list is a good place for this. This gives valuable input and
17− makes it easier to improve the program.
18−
19− * Help with questions. Every request that is answered by the wider
20− community frees time for programming contributors. This will
21− speed up development of new features and issue fixes. Don't
22− underestimate your knowledge, please share it!
23−
24− Good places to help are the [pandoc-discuss] mailing list, Q/A
25− sites like StackOverflow, community forums (e.g.
26− [RStudio][RStudio Community], [Zettlr][Zettlr Forum]), and, for
27− technical questions, the GitHub [issue tracker].
28−
29− * Write or improve documentation. If you ran into a problem which
30− took more time to figure out than expected, please consider to
31− save other users from the same experience. People writing the
32− documentation tend to lack an outside view, so please help
33− provide one. Good documentation is both difficult and extremely
34− important.
35−
36− The official docs are not the only place for documentation.
37− Pandoc also has a [Wiki][pandoc wiki]. Private blogs can serve
38− as documentation just as the official manual can.
39−
40− * Contribute code. No matter whether it's a small fix in a format
41− template or a huge lump of Haskell code: help is welcome. It's
42− usually a good idea to talk about the plans early, as this can
43− prevent unnecessary work. See below for more information.
44−
45− * Last but not least: consider funding the development and
46− maintenance of pandoc financially. You can find sponsor buttons
47− on the [pandoc website] and the [GitHub repository][GitHub
48− repo].
49−
50−A rich ecosystem of libraries, editors, filters, and templates has
51−developed around pandoc; conversely, pandoc builds and depends on a
52−large number of libraries. Contributing to any of these projects is
53−another way that can help to ensure stability, and to keep pushing
54−the boundaries of what is possible with pandoc.
55−
56−[RStudio Community]: https://community.rstudio.com/
57−[Zettlr Forum]: https://forum.zettlr.com/
58−[pandoc wiki]: https://github.com/jgm/pandoc/wiki
59−[pandoc website]: https://pandoc.org
60−[GitHub repo]: https://github.com/jgm/pandoc
61−
624Have a question?
635----------------
646
@@ -163,12 +105,7 @@ Explain the rationale for the feature you're requesting. Why would this
163105feature be useful? Consider also any possible drawbacks, including backwards
164106compatibility, new library dependencies, and performance issues.
165107
166−Features are very rarely "implement and forget", as all code must be
167−maintained. This is especially relevant for large or complex
168−contributions. It is helpful to be sympathetic to that fact, and to
169−communicate future plans and availability clearly.
170−
171−Any potential new feature is best discussed on [pandoc-discuss]
108+It is best to discuss a potential new feature on [pandoc-discuss]
172109before opening an issue.
173110
174111Patches and pull requests
@@ -244,16 +181,9 @@ Or with stack:
244181
245182 stack test --test-arguments='-p markdown'
246183
247−It is often helpful to add `-j4` (run tests in parallel) and
248−`--hide-successes` (don't clutter output with successes) to the test
249−arguments as well. Collecting all options in a `cabal.project.local`
250−file in the project's root directory can help to keep `cabal`
251−commands short. E.g.:
252−
253− flags: +embed_data_files
254− tests: True
255− test-show-details: direct
256− test-options: -j4 --hide-successes
184+It is often helpful to add `-j4` (run tests in parallel)
185+and `--hide-successes` (don't clutter output with successes)
186+to the test arguments as well.
257187
258188If you add a new feature to pandoc, please add tests as well, following
259189the pattern of the existing tests. The test suite code is in
@@ -295,11 +225,7 @@ A good way to ensure no new warnings are introduced is to use a Git
295225before creating a commit:
296226
297227 #!/bin/sh
298− git diff --diff-filter=MA --cached --name-only | grep '\.hs$' | \
299− xargs hlint --hint .hlint.yaml
300−
301−(If you are using GNU `xargs`, add the `-r` option immediately
302−after `xargs`.)
228+ git diff --cached --name-only | grep '\.hs$' | xargs hlint
303229
304230Saving this to `.git/hooks/pre-commit`, and making the script
305231executable, will prevent accidental introduction of potentially
@@ -318,6 +244,9 @@ With stack:
318244
319245 stack bench
320246
247+You can also build `weigh-pandoc` (`stack build pandoc:weigh-pandoc`)
248+to get some statistics on memory usage. (Eventually this should
249+be incorporated into the benchmark suite.)
321250
322251Using the REPL
323252--------------
@@ -329,10 +258,10 @@ a ghci REPL for working with pandoc. With [stack], use
329258We recommend using the following `.ghci` file (which can be
330259placed in the source directory):
331260
332− :set -fobject-code
333− :set -XTypeSynonymInstances
334− :set -XScopedTypeVariables
335− :set -XOverloadedStrings
261+ :set -fobject-code
262+ :set -XTypeSynonymInstances
263+ :set -XScopedTypeVariables
264+ :set -XOverloadedStrings
336265
337266Profiling
338267---------
@@ -376,7 +305,7 @@ The code
376305--------
377306
378307Pandoc has a publicly accessible git repository on
379−GitHub: <https://github.com/jgm/pandoc>. To get a local copy of the source:
308+GitHub: <http://github.com/jgm/pandoc>. To get a local copy of the source:
380309
381310 git clone https://github.com/jgm/pandoc.git
382311
@@ -386,7 +315,7 @@ the pandoc library is in `src/`, the source for the tests is in
386315
387316The modules `Text.Pandoc.Definition`, `Text.Pandoc.Builder`, and
388317`Text.Pandoc.Generic` are in a separate library `pandoc-types`. The code can
389−be found in <https://github.com/jgm/pandoc-types>.
318+be found in <http://github.com/jgm/pandoc-types>.
390319
391320To build pandoc, you will need a working installation of the
392321[Haskell platform].
@@ -406,11 +335,11 @@ The library is structured as follows:
406335 tree automatically.
407336 - `Text.Pandoc.Readers.*` are the readers, and `Text.Pandoc.Writers.*`
408337 are the writers.
409− - `Text.Pandoc.Citeproc.*` contain the code for citation handling,
410− including an interface to the [citeproc] library.
338+ - `Text.Pandoc.Biblio` is a utility module for formatting citations
339+ using citeproc-hs.
411340 - `Text.Pandoc.Data` is used to embed data files when the `embed_data_files`
412− cabal flag is used.
413− - `Text.Pandoc.Emoji` is a thin wrapper around [emojis].
341+ cabal flag is used. It is generated from `src/Text/Pandoc/Data.hsb` using
342+ the preprocessor [hsb2hs].
414343 - `Text.Pandoc.Highlighting` contains the interface to the
415344 skylighting library, which is used for code syntax highlighting.
416345 - `Text.Pandoc.ImageSize` is a utility module containing functions for
@@ -421,6 +350,7 @@ The library is structured as follows:
421350 - `Text.Pandoc.PDF` contains functions for producing a PDF from a
422351 LaTeX source.
423352 - `Text.Pandoc.Parsing` contains parsing functions used in multiple readers.
353+ - `Text.Pandoc.Pretty` is a pretty-printing library specialized to
424354 the needs of pandoc.
425355 - `Text.Pandoc.SelfContained` contains functions for making an HTML
426356 file "self-contained," by importing remotely linked images, CSS,
@@ -448,16 +378,14 @@ you may want to consider submitting a pull request to the
448378[closed issues]: https://github.com/jgm/pandoc/issues?q=is%3Aissue+is%3Aclosed
449379[latest released version]: https://github.com/jgm/pandoc/releases/latest
450380[Nightly builds]: https://github.com/jgm/pandoc/actions?query=workflow%3ANightly
451−[pandoc-discuss]: https://groups.google.com/group/pandoc-discuss
381+[pandoc-discuss]: http://groups.google.com/group/pandoc-discuss
452382[issue tracker]: https://github.com/jgm/pandoc/issues
453−[User's Guide]: https://pandoc.org/MANUAL.html
454−[FAQs]: https://pandoc.org/faqs.html
455−[EditorConfig]: https://editorconfig.org/
456−[Haskell platform]: https://www.haskell.org/platform/
383+[User's Guide]: http://pandoc.org/MANUAL.html
384+[FAQs]: http://pandoc.org/faqs.html
385+[EditorConfig]: http://editorconfig.org/
386+[Haskell platform]: http://www.haskell.org/platform/
457387[hlint]: https://hackage.haskell.org/package/hlint
458−[citeproc]: https://hackage.haskell.org/package/citeproc
459−[emojis]: https://hackage.haskell.org/package/emojis
460−[hsb2hs]: https://hackage.haskell.org/package/hsb2hs
388+[hsb2hs]: http://hackage.haskell.org/package/hsb2hs
461389[pre-commit hook]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
462390[GitHub labels]: https://github.com/jgm/pandoc/labels
463391[good first issue]:https://github.com/jgm/pandoc/labels/good%20first%20issue
COPYING.md+1−1
@@ -357,5 +357,5 @@ into proprietary programs. If your program is a subroutine library,
357357you may consider it more useful to permit linking proprietary
358358applications with the library. If this is what you want to do, use the
359359[GNU Lesser General Public
360−License](https://www.gnu.org/licenses/lgpl.html) instead of this
360+License](http://www.gnu.org/licenses/lgpl.html) instead of this
361361License.
COPYRIGHT+26−17
@@ -1,5 +1,5 @@
11Pandoc
2−Copyright (C) 2006-2021 John MacFarlane <jgm at berkeley dot edu>
2+Copyright (C) 2006-2020 John MacFarlane <jgm at berkeley dot edu>
33
44With the exceptions noted below, this code is released under the [GPL],
55version 2 or later:
@@ -22,11 +22,11 @@ The GNU General Public License is available in the file COPYING.md in
2222the source distribution. On Debian systems, the complete text of the
2323GPL can be found in `/usr/share/common-licenses/GPL`.
2424
25−[GPL]: https://www.gnu.org/copyleft/gpl.html
25+[GPL]: http://www.gnu.org/copyleft/gpl.html
2626
2727Pandoc's complete source code is available from the [Pandoc home page].
2828
29−[Pandoc home page]: https://pandoc.org
29+[Pandoc home page]: http://pandoc.org
3030
3131Pandoc includes some code with different copyrights, or subject to different
3232licenses. The copyright and license statements for these sources are included
@@ -37,7 +37,7 @@ The modules in the `pandoc-types` repository (Text.Pandoc.Definition,
3737Text.Pandoc.Builder, Text.Pandoc.Generics, Text.Pandoc.JSON,
3838Text.Pandoc.Walk) are licensed under the BSD 3-clause license:
3939
40−Copyright (c) 2006-2021, John MacFarlane
40+Copyright (c) 2006-2020, John MacFarlane
4141
4242All rights reserved.
4343
@@ -73,7 +73,7 @@ Pandoc's templates (in `data/templates`) are dual-licensed as either
7373GPL (v2 or higher, same as pandoc) or (at your option) the BSD
74743-clause license.
7575
76−Copyright (c) 2014--2021, John MacFarlane
76+Copyright (c) 2014--2020, John MacFarlane
7777
7878----------------------------------------------------------------------
7979src/Text/Pandoc/Writers/Muse.hs
@@ -83,19 +83,19 @@ Released under the GNU General Public License version 2 or later.
8383
8484----------------------------------------------------------------------
8585src/Text/Pandoc/Writers/Texinfo.hs
86−Copyright (C) 2008-2021 John MacFarlane and Peter Wang
86+Copyright (C) 2008-2020 John MacFarlane and Peter Wang
8787
8888Released under the GNU General Public License version 2 or later.
8989
9090----------------------------------------------------------------------
9191src/Text/Pandoc/Writers/OpenDocument.hs
92−Copyright (C) 2008-2021 Andrea Rossato and John MacFarlane
92+Copyright (C) 2008-2020 Andrea Rossato and John MacFarlane
9393
9494Released under the GNU General Public License version 2 or later.
9595
9696----------------------------------------------------------------------
9797src/Text/Pandoc/Writers/Org.hs
98−Copyright (C) 2010-2021 Puneeth Chaganti, John MacFarlane, and
98+Copyright (C) 2010-2020 Puneeth Chaganti, John MacFarlane, and
9999 Albert Krewinkel
100100
101101Released under the GNU General Public License version 2 or later.
@@ -115,7 +115,7 @@ Released under the GNU General Public License version 2 or later.
115115
116116----------------------------------------------------------------------
117117src/Text/Pandoc/Readers/Textile.hs
118−Copyright (C) 2010-2021 Paul Rivier and John MacFarlane
118+Copyright (C) 2010-2020 Paul Rivier and John MacFarlane
119119
120120Released under the GNU General Public License version 2 or later.
121121
@@ -133,7 +133,7 @@ Released under the GNU General Public License version 2 or later.
133133
134134----------------------------------------------------------------------
135135src/Text/Pandoc/Readers/EPUB.hs
136−Copyright (C) 2014-2021 Matthew Pickering and John MacFarlane
136+Copyright (C) 2014-2020 Matthew Pickering and John MacFarlane
137137
138138Released under the GNU General Public License version 2 or later.
139139
@@ -141,7 +141,7 @@ Released under the GNU General Public License version 2 or later.
141141src/Text/Pandoc/Readers/Org.hs
142142src/Text/Pandoc/Readers/Org/*
143143test/Tests/Readers/Org/*
144−Copyright (C) 2014-2021 Albert Krewinkel
144+Copyright (C) 2014-2020 Albert Krewinkel
145145
146146Released under the GNU General Public License version 2 or later.
147147
@@ -149,7 +149,7 @@ Released under the GNU General Public License version 2 or later.
149149src/Text/Pandoc/Lua.hs
150150src/Text/Pandoc/Lua/*
151151test/lua/*
152−Copyright (C) 2017--2021 Albert Krewinkel and John MacFarlane
152+Copyright (C) 2017--2020 Albert Krewinkel and John MacFarlane
153153
154154Released under the GNU General Public License version 2 or later.
155155
@@ -157,7 +157,7 @@ Released under the GNU General Public License version 2 or later.
157157src/Text/Pandoc/Readers/Jira.hs
158158src/Text/Pandoc/Writers/Jira.hs
159159test/Tests/Readers/Jira.hs
160−Copyright (C) 2019--2021 Albert Krewinkel
160+Copyright (C) 2019--2020 Albert Krewinkel
161161
162162Released under the GNU General Public License version 2 or later.
163163
@@ -169,21 +169,21 @@ Released under the GNU General Public License version 2 or later.
169169
170170----------------------------------------------------------------------
171171data/pandoc.lua
172−Copyright (C) 2017-2021 Albert Krewinkel
172+Copyright (C) 2017-2020 Albert Krewinkel
173173
174174Released under the GNU General Public License version 2 or later.
175175
176176----------------------------------------------------------------------
177177The dzslides template contains JavaScript and CSS from Paul Rouget's
178178dzslides template.
179−https://github.com/paulrouget/dzslides
179+http://github.com/paulrouget/dzslides
180180
181181Released under the Do What the Fuck You Want To Public License.
182182
183183------------------------------------------------------------------------
184−Pandoc embeds a Lua interpreter (via hslua).
184+Pandoc embeds a lua interpreter (via hslua).
185185
186−Copyright © 1994–2020 Lua.org, PUC-Rio.
186+Copyright © 1994–2019 Lua.org, PUC-Rio.
187187
188188Permission is hereby granted, free of charge, to any person obtaining
189189a copy of this software and associated documentation files (the
@@ -203,3 +203,12 @@ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
203203LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
204204OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
205205WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
206+
207+------------------------------------------------------------------------
208+The template pandoc.jats is Copyright 2013--2015 Martin Fenner,
209+released under GPL version 2 or later.
210+
211+The file data/jats.csl is derived from a csl file by Martin Fenner,
212+revised by Martin Paul Eve and then John MacFarlane.
213+"This work is licensed under a Creative Commons Attribution-ShareAlike 3.0
214+License. Originally by Martin Fenner."
INSTALL.md+15−18
@@ -34,10 +34,6 @@ without SVG support), [Python] (to use Pandoc filters), and
3434
3535 choco install rsvg-convert python miktex
3636
37−Using multiple installation methods can result in two separate
38−installations of pandoc; it is recommended to properly uninstall
39−pandoc before switching to an alternative installation method.
40−
4137By default, Pandoc creates PDFs using LaTeX.
4238We recommend installing it via [MiKTeX].
4339
@@ -49,7 +45,7 @@ by downloading [this script][uninstaller]
4945and running it with `perl uninstall-pandoc.pl`.
5046
5147Alternatively, you can install pandoc using
52−[Homebrew](https://brew.sh):
48+[Homebrew](http://brew.sh):
5349
5450 brew install pandoc
5551
@@ -120,7 +116,7 @@ package repositories. For example, on Debian/Ubuntu,
120116you can install it with `apt-get install haskell-platform`.
121117
122118For PDF output, you'll need LaTeX. We recommend installing
123−[TeX Live](https://www.tug.org/texlive/) via your package
119+[TeX Live](http://www.tug.org/texlive/) via your package
124120manager. (On Debian/Ubuntu, `apt-get install texlive`.)
125121
126122## Chrome OS
@@ -242,7 +238,7 @@ The easiest way to build pandoc from source is to use [stack][stack]:
242238
243239 pandoc --help
244240
245− [Not sure where `$CABALDIR` is?](https://wiki.haskell.org/Cabal-Install#The_cabal-install_configuration_file)
241+ [Not sure where `$CABALDIR` is?](http://www.haskell.org/haskellwiki/Cabal-Install#The_cabal-install_configuration_file)
246242
2472435. By default `pandoc` uses the "i;unicode-casemap" method
248244 to sort bibliography entries (RFC 5051). If you would like to
@@ -387,29 +383,30 @@ To run just the markdown benchmarks:
387383
388384
389385[Arch]: https://www.archlinux.org/packages/community/x86_64/pandoc/
390−[Cabal User's Guide]: https://cabal.readthedocs.io/
386+[Cabal User's Guide]: http://www.haskell.org/cabal/release/latest/doc/users-guide/builders.html#setup-configure-paths
391387[Debian]: https://packages.debian.org/pandoc
392388[Fedora]: https://apps.fedoraproject.org/packages/pandoc
393−[FreeBSD ports]: https://www.freshports.org/textproc/hs-pandoc/
394−[GHC]: https://www.haskell.org/ghc/
395−[Haskell platform]: https://hackage.haskell.org/platform/
396−[MacPorts]: https://trac.macports.org/browser/trunk/dports/textproc/pandoc/Portfile
389+[FreeBSD ports]: http://www.freshports.org/textproc/hs-pandoc/
390+[GHC]: http://www.haskell.org/ghc/
391+[GPL]: http://www.gnu.org/copyleft/gpl.html
392+[Haskell platform]: http://hackage.haskell.org/platform/
393+[MacPorts]: http://trac.macports.org/browser/trunk/dports/textproc/pandoc/Portfile
397394[MacTeX]: https://tug.org/mactex/
398−[BasicTeX]: https://www.tug.org/mactex/morepackages.html
395+[BasicTeX]: http://www.tug.org/mactex/morepackages.html
399396[LaTeX]: https://www.latex-project.org
400−[MiKTeX]: https://miktex.org/
397+[MiKTeX]: http://miktex.org/
401398[librsvg]: https://wiki.gnome.org/Projects/LibRsvg
402399[Python]: https://www.python.org
403−[NetBSD]: https://pkgsrc.se/wip/pandoc
400+[NetBSD]: http://pkgsrc.se/wip/pandoc
404401[NixOS]: https://nixos.org/nixos/packages.html
405402[Slackware]: https://www.slackbuilds.org/result/?search=pandoc&sv=
406403[Ubuntu]: https://packages.ubuntu.com/pandoc
407404[download page]: https://github.com/jgm/pandoc/releases/latest
408−[gentoo]: https://packages.gentoo.org/package/app-text/pandoc
405+[gentoo]: http://packages.gentoo.org/package/app-text/pandoc
409406[haskell repository]: https://wiki.archlinux.org/index.php/Haskell_Package_Guidelines#.5Bhaskell.5D
410407[openSUSE]: https://software.opensuse.org/package/pandoc
411−[source tarball]: https://hackage.haskell.org/package/pandoc
408+[source tarball]: http://hackage.haskell.org/package/pandoc
412409[stack]: https://docs.haskellstack.org/en/stable/install_and_upgrade.html
413−[cabal-install]: https://hackage.haskell.org/package/cabal-install
410+[cabal-install]: http://hackage.haskell.org/package/cabal-install
414411[Void]: https://voidlinux.org/
415412[uninstaller]: https://raw.githubusercontent.com/jgm/pandoc/master/macos/uninstall-pandoc.pl
MANUAL.txt+114−379
@@ -1,7 +1,7 @@
11---
22title: Pandoc User's Guide
33author: John MacFarlane
4−date: June 20, 2021
4+date: November 19, 2020
55---
66
77# Synopsis
@@ -136,7 +136,7 @@ to output the generated LaTeX. You can then test it with `pdflatex test.tex`.
136136When using LaTeX, the following packages need to be available
137137(they are included with all recent versions of [TeX Live]):
138138[`amsfonts`], [`amsmath`], [`lm`], [`unicode-math`],
139−[`iftex`], [`listings`] (if the
139+[`ifxetex`], [`ifluatex`], [`listings`] (if the
140140`--listings` option is used), [`fancyvrb`], [`longtable`],
141141[`booktabs`], [`graphicx`] (if the document
142142contains images), [`hyperref`], [`xcolor`],
@@ -180,7 +180,8 @@ footnotes in tables).
180180[`graphicx`]: https://ctan.org/pkg/graphicx
181181[`grffile`]: https://ctan.org/pkg/grffile
182182[`hyperref`]: https://ctan.org/pkg/hyperref
183−[`iftex`]: https://ctan.org/pkg/iftex
183+[`ifluatex`]: https://ctan.org/pkg/ifluatex
184+[`ifxetex`]: https://ctan.org/pkg/ifxetex
184185[`listings`]: https://ctan.org/pkg/listings
185186[`lm`]: https://ctan.org/pkg/lm
186187[`longtable`]: https://ctan.org/pkg/longtable
@@ -280,8 +281,6 @@ header when requesting a document from a URL:
280281 ::: {#output-formats}
281282 - `asciidoc` ([AsciiDoc]) or `asciidoctor` ([AsciiDoctor])
282283 - `beamer` ([LaTeX beamer][`beamer`] slide show)
283− - `bibtex` ([BibTeX] bibliography)
284− - `biblatex` ([BibLaTeX] bibliography)
285284 - `commonmark` ([CommonMark] Markdown)
286285 - `commonmark_x` ([CommonMark] Markdown with extensions)
287286 - `context` ([ConTeXt])
@@ -361,15 +360,15 @@ header when requesting a document from a URL:
361360 will be used. On \*nix and macOS systems this will be the `pandoc`
362361 subdirectory of the XDG data directory (by default,
363362 `$HOME/.local/share`, overridable by setting the `XDG_DATA_HOME`
364− environment variable). If that directory does not exist and
365− `$HOME/.pandoc` exists, it will be used (for backwards compatibility).
366− On Windows the default user data directory is
363+ environment variable). If that directory does not exist,
364+ `$HOME/.pandoc` will be used (for backwards compatibility).
365+ In Windows the default user data directory is
367366 `C:\Users\USERNAME\AppData\Roaming\pandoc`.
368367 You can find the default user data directory on your system by
369368 looking at the output of `pandoc --version`.
370− Data files placed in this directory (for example, `reference.odt`,
371− `reference.docx`, `epub.css`, `templates`) will override
372− pandoc's normal defaults.
369+ A `reference.odt`, `reference.docx`, `epub.css`, `templates`,
370+ `slidy`, `slideous`, or `s5` directory
371+ placed in this directory will override pandoc's normal defaults.
373372
374373`-d` *FILE*, `--defaults=`*FILE*
375374
@@ -486,7 +485,7 @@ header when requesting a document from a URL:
486485[Emacs Org mode]: https://orgmode.org
487486[AsciiDoc]: https://www.methods.co.nz/asciidoc/
488487[AsciiDoctor]: https://asciidoctor.org/
489−[DZSlides]: https://paulrouget.com/dzslides/
488+[DZSlides]: http://paulrouget.com/dzslides/
490489[Word docx]: https://en.wikipedia.org/wiki/Office_Open_XML
491490[PDF]: https://www.adobe.com/pdf/
492491[reveal.js]: https://revealjs.com/
@@ -589,8 +588,8 @@ header when requesting a document from a URL:
589588
590589 3. `$PATH` (executable only)
591590
592− Filters, Lua-filters, and citeproc processing are applied in
593− the order specified on the command line.
591+ Filters and Lua-filters are applied in the order specified
592+ on the command line.
594593
595594`-L` *SCRIPT*, `--lua-filter=`*SCRIPT*
596595
@@ -604,17 +603,26 @@ header when requesting a document from a URL:
604603 The `pandoc` Lua module provides helper functions for element
605604 creation. It is always loaded into the script's Lua environment.
606605
607− See the [Lua filters documentation] for further details.
606+ The following is an example Lua script for macro-expansion:
608607
609− In order of preference, pandoc will look for Lua filters in
608+ function expand_hello_world(inline)
609+ if inline.c == '{{helloworld}}' then
610+ return pandoc.Emph{ pandoc.Str "Hello, World" }
611+ else
612+ return inline
613+ end
614+ end
610615
611− 1. a specified full or relative path
616+ return {{Str = expand_hello_world}}
612617
613− 2. `$DATADIR/filters` where `$DATADIR` is the user data
614− directory (see `--data-dir`, above).
618+ In order of preference, pandoc will look for Lua filters in
615619
616− Filters, Lua filters, and citeproc processing are applied in
617− the order specified on the command line.
620+ 1. a specified full or relative path (executable or
621+ non-executable)
622+
623+ 2. `$DATADIR/filters` (executable or non-executable)
624+ where `$DATADIR` is the user data directory (see
625+ `--data-dir`, above).
618626
619627`-M` *KEY*[`=`*VAL*], `--metadata=`*KEY*[`:`*VAL*]
620628
@@ -675,12 +683,12 @@ header when requesting a document from a URL:
675683: Extract images and other media contained in or linked from
676684 the source document to the path *DIR*, creating it if
677685 necessary, and adjust the images references in the document
678− so they point to the extracted files. Media are downloaded,
679− read from the file system, or extracted from a binary
680− container (e.g. docx), as needed. The original file paths
681− are used if they are relative paths not containing `..`.
682− Otherwise filenames are constructed from the SHA1 hash of
683− the contents.
686+ so they point to the extracted files. If the source format is
687+ a binary container (docx, epub, or odt), the media is
688+ extracted from the container and the original
689+ filenames are used. Otherwise the media is read from the
690+ file system or downloaded, and new filenames are constructed
691+ based on SHA1 hashes of the contents.
684692
685693`--abbreviations=`*FILE*
686694
@@ -691,15 +699,14 @@ header when requesting a document from a URL:
691699 system default, use
692700 `pandoc --print-default-data-file=abbreviations`. The only
693701 use pandoc makes of this list is in the Markdown reader.
694− Strings found in this list will be followed by a nonbreaking
695− space, and the period will not produce sentence-ending space
696− in formats like LaTeX. The strings may not contain spaces.
702+ Strings ending in a period that are found in this list will
703+ be followed by a nonbreaking space, so that the period will
704+ not produce sentence-ending space in formats like LaTeX.
697705
698706[`pandocfilters`]: https://github.com/jgm/pandocfilters
699707[PHP]: https://github.com/vinai/pandocfilters-php
700708[perl]: https://metacpan.org/pod/Pandoc::Filter
701709[JavaScript/node.js]: https://github.com/mvhenderson/pandoc-filter-node
702−[Lua filters documentation]: https://pandoc.org/lua-filters.html
703710
704711## General writer options {.options}
705712
@@ -891,11 +898,11 @@ header when requesting a document from a URL:
891898 will not be searched. For example:
892899 `--resource-path=.:test` will search the working directory
893900 and the `test` subdirectory, in that order.
894− This option can be used repeatedly. Search path components
895− that come later on the command line will be searched before
896− those that come earlier, so
897− `--resource-path foo:bar --resource-path baz:bim` is
898− equivalent to `--resource-path baz:bim:foo:bar`.
901+
902+ `--resource-path` only has an effect if (a) the output
903+ format embeds images (for example, `docx`, `pdf`, or `html`
904+ with `--self-contained`) or (b) it is used together with
905+ `--extract-media`.
899906
900907`--request-header=`*NAME*`:`*VAL*
901908
@@ -993,8 +1000,8 @@ header when requesting a document from a URL:
9931000
9941001`-N`, `--number-sections`
9951002
996−: Number section headings in LaTeX, ConTeXt, HTML, Docx, ms, or EPUB
997− output. By default, sections are not numbered. Sections with class
1003+: Number section headings in LaTeX, ConTeXt, HTML, Docx, or EPUB output.
1004+ By default, sections are not numbered. Sections with class
9981005 `unnumbered` will never be numbered, even if `--number-sections`
9991006 is specified.
10001007
@@ -1327,10 +1334,7 @@ header when requesting a document from a URL:
13271334: Set the `bibliography` field in the document's metadata to *FILE*,
13281335 overriding any value set in the metadata. If you supply
13291336 this argument multiple times, each *FILE* will be added to
1330− bibliography. If *FILE* is a URL, it will be fetched
1331− via HTTP. If *FILE* is not found relative to the
1332− working directory, it will be sought in the resource path
1333− (see `--resource-path`).
1337+ bibliography.
13341338
13351339`--csl=`*FILE*
13361340
@@ -1339,8 +1343,8 @@ header when requesting a document from a URL:
13391343 `--metadata csl=FILE`.) If *FILE* is a URL, it will be
13401344 fetched via HTTP. If *FILE* is not found relative to the
13411345 working directory, it will be sought in the resource path
1342− (see `--resource-path`) and finally in the `csl`
1343− subdirectory of the pandoc user data directory.
1346+ and finally in the `csl` subdirectory of the
1347+ pandoc user data directory.
13441348
13451349`--citation-abbreviations=`*FILE*
13461350
@@ -1349,8 +1353,8 @@ header when requesting a document from a URL:
13491353 `--metadata citation-abbreviations=FILE`.)
13501354 If *FILE* is a URL, it will be fetched via HTTP. If *FILE* is not
13511355 found relative to the working directory, it will be sought
1352− in the resource path (see `--resource-path`) and finally in
1353− the `csl` subdirectory of the pandoc user data directory.
1356+ in the resource path and finally in the `csl` subdirectory
1357+ of the pandoc user data directory.
13541358
13551359`--natbib`
13561360
@@ -1415,12 +1419,13 @@ results only for basic math, usually you will want to use
14151419`--gladtex`
14161420
14171421: Enclose TeX math in `<eq>` tags in HTML output. The resulting HTML
1418− can then be processed by [GladTeX] to produce SVG images of the typeset
1419− formulas and an HTML file with these images embedded.
1422+ can then be processed by [GladTeX] to produce images of the typeset
1423+ formulas and an HTML file with links to these images.
1424+ So, the procedure is:
14201425
14211426 pandoc -s --gladtex input.md -o myfile.htex
1422− gladtex -d image_dir myfile.htex
1423− # produces myfile.html and images in image_dir
1427+ gladtex -d myfile-images myfile.htex
1428+ # produces myfile.html and images in myfile-images
14241429
14251430[MathML]: https://www.w3.org/Math/
14261431[MathJax]: https://www.mathjax.org
@@ -1468,7 +1473,6 @@ Nonzero exit codes have the following meanings:
14681473 24 PandocCiteprocError
14691474 31 PandocEpubSubdirectoryError
14701475 43 PandocPDFError
1471− 44 PandocXMLError
14721476 47 PandocPDFProgramNotFoundError
14731477 61 PandocHttpError
14741478 62 PandocShouldNeverHappenError
@@ -1481,7 +1485,6 @@ Nonzero exit codes have the following meanings:
14811485 91 PandocMacroLoop
14821486 92 PandocUTF8DecodingError
14831487 93 PandocIpynbDecodingError
1484− 94 PandocUnsupportedCharsetError
14851488 97 PandocCouldNotFindDataFileError
14861489 99 PandocResourceNotFound
14871490----- ------------------------------------
@@ -1506,16 +1509,6 @@ input-files:
15061509- content.md
15071510# or you may use input-file: with a single value
15081511
1509−# Include options from the specified defaults files.
1510−# The files will be searched for first in the working directory
1511−# and then in the defaults subdirectory of the user data directory.
1512−# The files are included in the same order in which they appear in
1513−# the list. Options specified in this defaults file always have
1514−# priority over the included ones.
1515−defaults:
1516−- defsA
1517−- defsB
1518−
15191512template: letter
15201513standalone: true
15211514self-contained: false
@@ -1573,11 +1566,8 @@ data-dir:
15731566verbosity: INFO
15741567log-file: log.json
15751568
1576−# citeproc, natbib, or biblatex. This only affects LaTeX
1577−# output. If you want to use citeproc to format citations,
1578−# you should also set 'citeproc: true' (see above).
1569+# citeproc, natbib, or biblatex
15791570cite-method: citeproc
1580−
15811571# part, chapter, section, or default:
15821572top-level-division: chapter
15831573abbreviations:
@@ -1676,34 +1666,6 @@ one line:
16761666verbosity: INFO
16771667```
16781668
1679−In fields that expect a file path (or list of file paths), the
1680−following syntax may be used to interpolate environment variables:
1681−
1682−``` yaml
1683−csl: ${HOME}/mycsldir/special.csl
1684−```
1685−
1686−`${USERDATA}` may also be used; this will always resolve to the
1687−user data directory that is current when the defaults file is
1688−parsed, regardless of the setting of the environment
1689−variable `USERDATA`.
1690−
1691−`${.}` will resolve to the directory containing the default
1692−file itself. This allows you to refer to resources contained
1693−in that directory:
1694−
1695−``` yaml
1696−epub-cover-image: ${.}/cover.jpg
1697−epub-metadata: ${.}/meta.xml
1698−resource-path:
1699−- . # the working directory from which pandoc is run
1700−- ${.}/images # the images subdirectory of the directory
1701− # containing this defaults file
1702−```
1703−
1704−This environment variable interpolation syntax *only* works in
1705−fields that expect file paths.
1706−
17071669Default files can be placed in the `defaults` subdirectory of
17081670the user data directory and used from any directory. For
17091671example, one could create a file specifying defaults for writing
@@ -1931,8 +1893,7 @@ ${ styles.html() }
19311893```
19321894
19331895(If a partial is not found in the directory of the
1934−template and the template path is given as a relative
1935−path, it will also be sought in the `templates`
1896+template, it will also be sought in the `templates`
19361897subdirectory of the user data directory.)
19371898
19381899Partials may optionally be applied to variables using
@@ -2314,8 +2275,8 @@ math equations flush left using [YAML metadata](#layout) or with
23142275
23152276These affect HTML output when [producing slide shows with pandoc].
23162277
2317−`institute`
2318−: author affiliations: can be a list when there are multiple authors
2278+All [reveal.js configuration options] are available as variables.
2279+To turn off boolean flags that default to true in reveal.js, use `0`.
23192280
23202281`revealjs-url`
23212282: base URL for reveal.js documents (defaults to
@@ -2335,9 +2296,6 @@ These affect HTML output when [producing slide shows with pandoc].
23352296: additional attributes for the title slide of reveal.js slide shows.
23362297 See [background in reveal.js and beamer] for an example.
23372298
2338−All [reveal.js configuration options] are available as variables.
2339−To turn off boolean flags that default to true in reveal.js, use `0`.
2340−
23412299[reveal.js configuration options]: https://revealjs.com/config/
23422300
23432301### Variables for Beamer slides
@@ -3069,43 +3027,6 @@ This extension can be enabled/disabled for the following formats:
30693027output formats
30703028: `odt`, `opendocument`
30713029
3072−#### Extension: `xrefs_name` ####
3073−
3074−Links to headings, figures and tables inside the document are
3075−substituted with cross-references that will use the name or caption
3076−of the referenced item. The original link text is replaced once
3077−the generated document is refreshed. This extension can be combined
3078−with `xrefs_number` in which case numbers will appear before the
3079−name.
3080−
3081−Text in cross-references is only made consistent with the referenced
3082−item once the document has been refreshed.
3083−
3084−This extension can be enabled/disabled for the following formats:
3085−
3086−output formats
3087−: `odt`, `opendocument`
3088−
3089−#### Extension: `xrefs_number` ####
3090−
3091−Links to headings, figures and tables inside the document are
3092−substituted with cross-references that will use the number
3093−of the referenced item. The original link text is discarded.
3094−This extension can be combined with `xrefs_name` in which case
3095−the name or caption numbers will appear after the number.
3096−
3097−For the `xrefs_number` to be useful heading numbers must be enabled
3098−in the generated document, also table and figure captions must be enabled
3099−using for example the `native_numbering` extension.
3100−
3101−Numbers in cross-references are only visible in the final document once
3102−it has been refreshed.
3103−
3104−This extension can be enabled/disabled for the following formats:
3105−
3106−output formats
3107−: `odt`, `opendocument`
3108−
31093030#### Extension: `styles` #### {#ext-styles}
31103031
31113032When converting from docx, read all docx styles as divs (for
@@ -3135,13 +3056,6 @@ output format.
31353056Some aspects of [Pandoc's Markdown citation syntax](#citations)
31363057are also accepted in `org` input.
31373058
3138−#### Extension: `element_citations` ####
3139−
3140−In the `jats` output formats, this causes reference items to
3141−be replaced with `<element-citation>` elements. These
3142−elements are not influenced by CSL styles, but all information
3143−on the item is included in tags.
3144−
31453059#### Extension: `ntb` ####
31463060
31473061In the `context` output format this enables the use of [Natural Tables
@@ -3517,9 +3431,6 @@ line must begin with a space.
35173431 | 200 Main St.
35183432 | Berkeley, CA 94718
35193433
3520−Inline formatting (such as emphasis) is allowed in the content,
3521−but not block-level formatting (such as block quotes or lists).
3522−
35233434This syntax is borrowed from [reStructuredText].
35243435
35253436## Lists
@@ -3754,7 +3665,7 @@ definition:
37543665Note that space between items in a definition list is required.
37553666(A variant that loosens this requirement, but disallows "lazy"
37563667hard wrapping, can be activated with `compact_definition_lists`: see
3757−[Non-default extensions], below.)
3668+[Non-pandoc extensions], below.)
37583669
37593670[^3]: I have been influenced by the suggestions of [David
37603671 Wheeler](https://justatheory.com/2009/02/modest-markdown-proposal/).
@@ -4179,12 +4090,10 @@ option is used. All of the metadata will appear in a single block
41794090at the beginning of the document.
41804091
41814092Note that [YAML] escaping rules must be followed. Thus, for example,
4182−if a title contains a colon, it must be quoted, and if it contains a
4183−backslash escape, then it must be ensured that it is not treated as a
4184−[YAML escape sequence]. The pipe character (`|`) can be used to begin
4185−an indented block that will be interpreted literally, without need for
4186−escaping. This form is necessary when the field contains blank lines
4187−or block-level formatting:
4093+if a title contains a colon, it must be quoted. The pipe character
4094+(`|`) can be used to begin an indented block that will be interpreted
4095+literally, without need for escaping. This form is necessary
4096+when the field contains blank lines or block-level formatting:
41884097
41894098 ---
41904099 title: 'This is the title: it contains a colon'
@@ -4198,12 +4107,6 @@ or block-level formatting:
41984107 It consists of two paragraphs.
41994108 ...
42004109
4201−The literal block after the `|` must be indented relative to the
4202−line containing the `|`. If it is not, the YAML will be invalid
4203−and pandoc will not interpret it as metadata. For an overview
4204−of the complex rules governing YAML, see the [Wikipedia entry on
4205−YAML syntax].
4206−
42074110Template variables will be set automatically from the metadata. Thus, for
42084111example, in writing HTML, the variable `abstract` will be set to the HTML
42094112equivalent of the Markdown in the `abstract` field:
@@ -4250,24 +4153,6 @@ will be interpreted as markdown. For example:
42504153 \renewcommand{\section}[1]{\clearpage\oldsection{#1}}
42514154 ```
42524155
4253−Note: the `yaml_metadata_block` extension works with
4254−`commonmark` as well as `markdown` (and it is enabled by default
4255−in `gfm` and `commonmark_x`). However, in these formats the
4256−following restrictions apply:
4257−
4258−- The YAML metadata block must occur at the beginning of the
4259− document (and there can be only one). If multiple files are
4260− given as arguments to pandoc, only the first can be a YAML
4261− metadata block.
4262−
4263−- The leaf nodes of the YAML structure are parsed in isolation from
4264− each other and from the rest of the document. So, for
4265− example, you can't use a reference link in these contexts
4266− if the link definition is somewhere else in the document.
4267−
4268−[YAML escape sequence]: https://yaml.org/spec/1.2/spec.html#id2776092
4269−[Wikipedia entry on YAML syntax]: https://en.m.wikipedia.org/wiki/YAML#Syntax
4270−
42714156## Backslash escapes
42724157
42734158#### Extension: `all_symbols_escapable` ####
@@ -4425,8 +4310,7 @@ you need to enclose text in literal `$` characters, backslash-escape
44254310them and they won't be treated as math delimiters.
44264311
44274312For display math, use `$$` delimiters. (In this case, the delimiters
4428−may be separated from the formula by whitespace. However, there can be
4429−no blank lines betwen the opening and closing `$$` delimiters.)
4313+may be separated from the formula by whitespace.)
44304314
44314315TeX math will be printed in all output formats. How it is rendered
44324316depends on the output format:
@@ -4544,8 +4428,8 @@ into
45444428
45454429whereas `Markdown.pl` will preserve it as is.
45464430
4547−There is one exception to this rule: text between `<script>`,
4548−`<style>`, and `<textarea>` tags is not interpreted as Markdown.
4431+There is one exception to this rule: text between `<script>` and
4432+`<style>` tags is not interpreted as Markdown.
45494433
45504434This departure from standard Markdown should make it easier to mix
45514435Markdown with HTML block elements. For example, one can surround
@@ -4960,68 +4844,37 @@ Inline and regular footnotes may be mixed freely.
49604844
49614845#### Extension: `citations` ####
49624846
4963−To cite a bibliographic item with an identifier foo, use the
4964−syntax `@foo`. Normal citations should be included in square
4965−brackets, with semicolons separating distinct items:
4966−
4967− Blah blah [@doe99; @smith2000; @smith2004].
4968−
4969−How this is rendered depends on the citation style. In an
4970−author-date style, it might render as
4971−
4972− Blah blah (Doe 1999, Smith 2000, 2004).
4973−
4974−In a footnote style, it might render as
4975−
4976− Blah blah.[^1]
4977−
4978− [^1]: John Doe, "Frogs," *Journal of Amphibians* 44 (1999);
4979− Susan Smith, "Flies," *Journal of Insects* (2000);
4980− Susan Smith, "Bees," *Journal of Insects* (2004).
4981−
4982−See the [CSL user documentation] for more information about CSL
4983−styles and how they affect rendering.
4984−
4985−Unless a citation key start with a letter, digit, or `_`,
4986−and contains only alphanumerics and internal punctuation
4987−characters (`:.#$%&-+?<>~/`), it must be surrounded
4988−by curly braces, which are not considered part of the key.
4989−In `@Foo_bar.baz.`, the key is `Foo_bar.baz`. The final
4990−period is not *internal* punctuation, so it is not included in
4991−the key. In `@{Foo_bar.baz.}`, the key is `Foo_bar.baz.`, including
4992−the final period. The curly braces are recommended if you use URLs as
4993−keys: `[@{https://example.com/bib?name=foobar&date=2000}, p. 33]`.
4994−
4995−Citation items may optionally include a prefix, a locator, and
4996−a suffix. In
4997−
4998− Blah blah [see @doe99, pp. 33-35 and *passim*; @smith04, chap. 1].
4999−
5000−The first item (`doe99`) has prefix `see `, locator `pp. 33-35`,
5001−and suffix `and *passim*`. The second item (`smith04`) has
5002−locator `chap. 1` and no prefix or suffix.
5003−
5004−Pandoc uses some heuristics to separate the locator from the
5005−rest of the subject. It is sensitive to the locator terms
5006−defined in the [CSL locale files]. Either abbreviated or
5007−unabbreviated forms are accepted. In the `en-US` locale, locator
5008−terms can be written in either singular or plural forms, as
5009−`book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
5010−`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`,
5011−`fol.`/`fols.`; `number`, `no.`/`nos.`; `line`, `l.`/`ll.`;
5012−`note`, `n.`/`nn.`; `opus`, `op.`/`opp.`; `page`, `p.`/`pp.`;
5013−`paragraph`, `para.`/`paras.`; `part`, `pt.`/`pts.`; `section`,
5014−`sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`; `verse`,
5015−`v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
4847+Markdown citations go inside square brackets and are separated
4848+by semicolons. Each citation must have a key, composed of '@' +
4849+the citation identifier from the database, and may optionally
4850+have a prefix, a locator, and a suffix. The citation key must
4851+begin with a letter, digit, or `_`, and may contain
4852+alphanumerics, `_`, and internal punctuation characters
4853+(`:.#$%&-+?<>~/`). Here are some examples:
4854+
4855+ Blah blah [see @doe99, pp. 33-35; also @smith04, chap. 1].
4856+
4857+ Blah blah [@doe99, pp. 33-35, 38-39 and *passim*].
4858+
4859+ Blah blah [@smith04; @doe99].
4860+
4861+`pandoc` detects locator terms in the [CSL locale files].
4862+Either abbreviated or unabbreviated forms are accepted. In the `en-US`
4863+locale, locator terms can be written in either singular or plural forms,
4864+as `book`, `bk.`/`bks.`; `chapter`, `chap.`/`chaps.`; `column`,
4865+`col.`/`cols.`; `figure`, `fig.`/`figs.`; `folio`, `fol.`/`fols.`;
4866+`number`, `no.`/`nos.`; `line`, `l.`/`ll.`; `note`, `n.`/`nn.`; `opus`,
4867+`op.`/`opp.`; `page`, `p.`/`pp.`; `paragraph`, `para.`/`paras.`; `part`,
4868+`pt.`/`pts.`; `section`, `sec.`/`secs.`; `sub verbo`, `s.v.`/`s.vv.`;
4869+`verse`, `v.`/`vv.`; `volume`, `vol.`/`vols.`; `¶`/`¶¶`; `§`/`§§`. If no
50164870locator term is used, "page" is assumed.
50174871
5018−In complex cases, you can force something to be treated as
5019−a locator by enclosing it in curly braces or prevent parsing
5020−the suffix as locator by prepending curly braces:
4872+`pandoc` will use heuristics to distinguish the locator
4873+from the suffix. In complex cases, the locator can be enclosed
4874+in curly braces:
50214875
50224876 [@smith{ii, A, D-Z}, with a suffix]
50234877 [@smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5024− [@smith{}, 99 years later]
50254878
50264879A minus sign (`-`) before the `@` will suppress mention of
50274880the author in the citation. This can be useful when the
@@ -5029,26 +4882,13 @@ author is already mentioned in the text:
50294882
50304883 Smith says blah [-@smith04].
50314884
5032−You can also write an author-in-text citation, by omitting the
5033−square brackets:
4885+You can also write an in-text citation, as follows:
50344886
50354887 @smith04 says blah.
50364888
50374889 @smith04 [p. 33] says blah.
50384890
5039−This will cause the author's name to be rendered, followed by
5040−the bibliographical details. Use this form when you want to
5041−make the citation the subject of a sentence.
5042−
5043−When you are using a note style, it is usually better to let
5044−citeproc create the footnotes from citations rather than writing
5045−an explicit note. If you do write an explicit note that
5046−contains a citation, note that normal citations will be put in
5047−parentheses, while author-in-text citations will not. For
5048−this reason, it is sometimes preferable to use the
5049−author-in-text style inside notes when using a note style.
50504891
5051−[CSL user documentation]: https://citationstyles.org/authors/
50524892[CSL]: https://docs.citationstyles.org/en/stable/specification.html
50534893[CSL markup specs]: https://docs.citationstyles.org/en/1.0/release-notes.html#rich-text-markup-within-fields
50544894[Chicago Manual of Style]: https://chicagomanualofstyle.org
@@ -5057,44 +4897,13 @@ author-in-text style inside notes when using a note style.
50574897[finding and editing styles]: https://citationstyles.org/authors/
50584898[CSL locale files]: https://github.com/citation-style-language/locales
50594899
5060−## Non-default extensions
4900+## Non-pandoc extensions
50614901
50624902The following Markdown syntax extensions are not enabled by default
50634903in pandoc, but may be enabled by adding `+EXTENSION` to the format
50644904name, where `EXTENSION` is the name of the extension. Thus, for
50654905example, `markdown+hard_line_breaks` is Markdown with hard line breaks.
50664906
5067−#### Extension: `rebase_relative_paths` ####
5068−
5069−Rewrite relative paths for Markdown links and images, depending
5070−on the path of the file containing the link or image link. For
5071−each link or image, pandoc will compute the directory of the
5072−containing file, relative to the working directory, and prepend
5073−the resulting path to the link or image path.
5074−
5075−The use of this extension is best understood by example.
5076−Suppose you have a a subdirectory for each chapter of a book,
5077−`chap1`, `chap2`, `chap3`. Each contains a file `text.md` and a
5078−number of images used in the chapter. You would like to have
5079−`![image](spider.jpg)` in `chap1/text.md` refer to
5080−`chap1/spider.jpg` and `![image](spider.jpg)` in `chap2/text.md`
5081−refer to `chap2/spider.jpg`. To do this, use
5082−
5083− pandoc chap*/*.md -f markdown+rebase_relative_paths
5084−
5085−Without this extension, you would have to use
5086−`![image](chap1/spider.jpg)` in `chap1/text.md` and
5087−`![image](chap2/spider.jpg)` in `chap2/text.md`. Links with
5088−relative paths will be rewritten in the same way as images.
5089−
5090−Absolute paths and URLs are not changed. Neither are empty
5091−paths or paths consisting entirely of a fragment, e.g., `#foo`.
5092−
5093−Note that relative paths in reference links and images will
5094−be rewritten relative to the file containing the link
5095−reference definition, not the file containing the reference link
5096−or image itself, if these differ.
5097−
50984907#### Extension: `attributes` ####
50994908
51004909Allows attributes to be attached to any inline or block-level
@@ -5281,13 +5090,6 @@ for regular emphasis, add extra blank space around headings.
52815090
52825091 [Project Gutenberg]: https://www.gutenberg.org
52835092
5284−#### Extension: `sourcepos` ####
5285−
5286−Include source position attributes when parsing `commonmark`.
5287−For elements that accept attributes, a `data-pos` attribute
5288−is added; other elements are placed in a surrounding
5289−Div or Span elemnet with a `data-pos` attribute.
5290−
52915093## Markdown variants
52925094
52935095In addition to pandoc's extended Markdown, the following Markdown
@@ -5402,21 +5204,17 @@ array of YAML-encoded references, for example:
54025204 language: en-GB
54035205 ...
54045206
5405−If both an external bibliography and inline (YAML metadata)
5406−references are provided, both will be used. In case of
5407−conflicting `id`s, the inline references will take precedence.
5408−
54095207Note that `pandoc` can be used to produce such a YAML metadata
54105208section from a BibTeX, BibLaTeX, or CSL JSON bibliography:
54115209
54125210 pandoc chem.bib -s -f biblatex -t markdown
54135211 pandoc chem.json -s -f csljson -t markdown
54145212
5415−Indeed, `pandoc` can convert between any of these
5416−citation formats:
5213+`pandoc` can also be used to produce CSL JSON bibliography
5214+from BibTeX, BibLaTeX, or markdown YAML:
54175215
54185216 pandoc chem.bib -s -f biblatex -t csljson
5419− pandoc chem.yaml -s -f markdown -t biblatex
5217+ pandoc chem.yaml -s -f markdown -t csljson
54205218
54215219Running pandoc on a bibliography file with the `--citeproc`
54225220option will create a formatted bibliography in the format
@@ -5515,34 +5313,6 @@ example:
55155313 }
55165314 }
55175315
5518−
5519−## Citations in note styles
5520−
5521−Pandoc's citation processing is designed to allow you to
5522−move between author-date, numerical, and note styles without
5523−modifying the markdown source. When you're using a note
5524−style, avoid inserting footnotes manually. Instead, insert
5525−citations just as you would in an author-date style---for
5526−example,
5527−
5528− Blah blah [@foo, p. 33].
5529−
5530−The footnote will be created automatically. Pandoc will take
5531−care of removing the space and moving the note before or
5532−after the period, depending on the setting of
5533−`notes-after-punctuation`, as described below in [Other relevant
5534−metadata fields].
5535−
5536−In some cases you may need to put a citation inside a regular
5537−footnote. Normal citations in footnotes (such as `[@foo, p.
5538−33]`) will be rendered in parentheses. In-text citations (such
5539−as `@foo [p. 33]`) will be rendered without parentheses. (A
5540−comma will be added if appropriate.) Thus:
5541−
5542− [^1]: Some studies [@foo; @bar, p. 33] show that
5543− frubulicious zoosnaps are quantical. For a survey
5544− of the literature, see @baz [chap. 1].
5545−
55465316## Raw content in a style
55475317
55485318To include raw content in a prefix, suffix, delimiter, or term,
@@ -5628,23 +5398,10 @@ A few other metadata fields affect bibliography formatting:
56285398
56295399`lang`
56305400: The `lang` field will affect how the style is localized,
5631− for example in the translation of labels, the use
5632− of quotation marks, and the way items are sorted.
5633− (For backwards compatibility, `locale` may be used instead
5634− of `lang`, but this use is deprecated.)
5635−
5636− A BCP 47 language tag is expected: for example, `en`,
5637− `de`, `en-US`, `fr-CA`, `ug-Cyrl`. The unicode extension
5638− syntax (after `-u-`) may be used to specify options for
5639− collation (sorting) more precisely. Here are some examples:
5640−
5641− - `zh-u-co-pinyin` -- Chinese with the Pinyin collation.
5642− - `es-u-co-trad` -- Spanish with the traditional collation
5643− (with `Ch` sorting after `C`).
5644− - `fr-u-kb` -- French with "backwards" accent sorting
5645− (with `coté` sorting after `côte`).
5646− - `en-US-u-kf-upper` -- English with uppercase letters sorting
5647− before lower (default is lower before upper).
5401+ for example in the translation of labels and the use
5402+ of quotation marks. (For backwards compatibility,
5403+ `locale` may be used instead of `lang`, but this use
5404+ is deprecated.)
56485405
56495406`notes-after-punctuation`
56505407: If true (the default), pandoc will put footnote citations
@@ -5656,6 +5413,12 @@ A few other metadata fields affect bibliography formatting:
56565413 punctuation.
56575414
56585415
5416+
5417+
5418+
5419+
5420+
5421+
56595422# Slide shows
56605423
56615424You can use pandoc to produce an HTML + JavaScript slide presentation
@@ -5748,11 +5511,7 @@ rules:
57485511 * A heading at the slide level always starts a new slide.
57495512
57505513 * Headings *below* the slide level in the hierarchy create
5751− headings *within* a slide. (In beamer, a "block" will be
5752− created. If the heading has the class `example`, an
5753− `exampleblock` environment will be used; if it has the class
5754− `alert`, an `alertblock` will be used; otherwise a regular
5755− `block` will be used.)
5514+ headings *within* a slide.
57565515
57575516 * Headings *above* the slide level in the hierarchy create
57585517 "title slides," which just contain the section title
@@ -6085,15 +5844,6 @@ The following fields are recognized:
60855844
60865845`rights`
60875846 ~ A string value.
6088−
6089−`belongs-to-collection`
6090− ~ A string value. identifies the name of a collection to which
6091− the EPUB Publication belongs.
6092−
6093−`group-position`
6094− ~ The `group-position` field indicates the numeric position in which
6095− the EPUB Publication belongs relative to other works belonging to
6096− the same `belongs-to-collection` field.
60975847
60985848`cover-image`
60995849 ~ A string value (path to cover image).
@@ -6330,7 +6080,7 @@ Then edit `my.theme` and use it like this:
63306080If you are not satisfied with the built-in highlighting, or you
63316081want highlight a language that isn't supported, you can use the
63326082`--syntax-definition` option to load a [KDE-style XML syntax definition
6333−file](https://docs.kde.org/stable5/en/kate/katepart/highlight.html).
6083+file](https://docs.kde.org/stable5/en/applications/katepart/highlight.html).
63346084Before writing your own, have a look at KDE's [repository of syntax
63356085definitions](https://github.com/KDE/syntax-highlighting/tree/master/data/syntax).
63366086
@@ -6461,21 +6211,6 @@ subdirectory of your user data directory (see [Templates]).
64616211
64626212[Lua]: https://www.lua.org
64636213
6464−# Reproducible builds
6465−
6466−Some of the document formats pandoc targets (such as EPUB,
6467−docx, and ODT) include build timestamps in the generated document.
6468−That means that the files generated on successive builds will
6469−differ, even if the source does not. To avoid this, set the
6470−`SOURCE_DATE_EPOCH` environment variable, and the timestamp will
6471−be taken from it instead of the current time.
6472−`SOURCE_DATE_EPOCH` should contain an integer unix timestamp
6473−(specifying the number of second since midnight UTC January 1, 1970).
6474−
6475−Some document formats also include a unique identifier. For
6476−EPUB, this can be set explicitly by setting the `identifier`
6477−metadata field (see [EPUB Metadata], above).
6478−
64796214# A note on security
64806215
64816216If you use pandoc to convert user-contributed content in a web
@@ -6504,13 +6239,13 @@ application, here are some things to keep in mind:
650462394. The HTML generated by pandoc is not guaranteed to be safe.
65056240 If `raw_html` is enabled for the Markdown input, users can
65066241 inject arbitrary HTML. Even if `raw_html` is disabled,
6507− users can include dangerous content in URLs and attributes.
6508− To be safe, you should run all the generated HTML through
6509− an HTML sanitizer.
6242+ users can include dangerous content in attributes for
6243+ headings, spans, and code blocks. To be safe, you should
6244+ run all the generated HTML through an HTML sanitizer.
65106245
65116246# Authors
65126247
6513−Copyright 2006--2021 John MacFarlane (jgm@berkeley.edu). Released
6248+Copyright 2006--2020 John MacFarlane (jgm@berkeley.edu). Released
65146249under the [GPL], version 2 or greater. This software carries no
65156250warranty of any kind. (See COPYRIGHT for full copyright and
65166251warranty notices.) For a full list of contributors, see the file
Makefile+19−47
@@ -2,31 +2,17 @@ version?=$(shell grep '^[Vv]ersion:' pandoc.cabal | awk '{print $$2;}')
22pandoc=$(shell find dist -name pandoc -type f -exec ls -t {} \; | head -1)
33SOURCEFILES?=$(shell git ls-tree -r master --name-only | grep "\.hs$$")
44BRANCH?=master
5−ARCH=$(shell uname -m)
6−DOCKERIMAGE=registry.gitlab.b-data.ch/ghc/ghc4pandoc:8.10.4
7−COMMIT=$(shell git rev-parse --short HEAD)
8−TIMESTAMP=$(shell date "+%Y%m%d_%H%M")
9−LATESTBENCH=$(word 1,$(shell ls -t bench_*.csv 2>/dev/null))
10−BASELINE?=$(LATESTBENCH)
11−ifeq ($(BASELINE),)
12−BASELINECMD=
13−else
14−BASELINECMD=--baseline $(BASELINE)
15−endif
16−GHCOPTS=-fdiagnostics-color=always -j4 +RTS -A8m -RTS
5+RESOLVER?=lts-13
6+GHCOPTS=-fdiagnostics-color=always
177WEBSITE=../../web/pandoc.org
188REVISION?=1
19−# For gauge:
20−# BENCHARGS?=--small --ci=0.90 --match=pattern $(PATTERN)
21−# For tasty-bench:
22−BENCHARGS?=--csv bench_$(TIMESTAMP).csv $(BASELINECMD) --timeout=6 +RTS -T -RTS $(if $(PATTERN),--pattern "$(PATTERN)",)
239
2410quick:
25− stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='$(GHCOPTS)' --test-arguments='-j4 --hide-successes $(TESTARGS)'
11+ stack install --ghc-options='$(GHCOPTS)' --install-ghc --flag 'pandoc:embed_data_files' --fast --test --ghc-options='-j +RTS -A64m -RTS' --test-arguments='-j4 --hide-successes $(TESTARGS)'
2612
2713quick-cabal:
2814 cabal new-configure . --ghc-options '$(GHCOPTS)' --disable-optimization --enable-tests
29− cabal new-build -j4 . --disable-optimization
15+ cabal new-build . --disable-optimization
3016 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
3117
3218full-cabal:
@@ -35,7 +21,7 @@ full-cabal:
3521 cabal new-run test-pandoc --disable-optimization -- --hide-successes $(TESTARGS)
3622
3723full:
38− stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 $(GHCOPTS)'
24+ stack install --flag 'pandoc:embed_data_files' --flag 'pandoc:trypandoc' --bench --no-run-benchmarks --test --test-arguments='-j4 --hide-successes' --ghc-options '-Wall -Werror -fno-warn-unused-do-bind -O0 -j4 $(GHCOPTS)'
3925
4026ghci:
4127 stack ghci --flag 'pandoc:embed_data_files'
@@ -51,14 +37,11 @@ test:
5137ghcid:
5238 ghcid -c "stack repl --flag 'pandoc:embed_data_files'"
5339
54−ghcid-test:
55− ghcid -c "stack repl --ghc-options=-XNoImplicitPrelude --flag 'pandoc:embed_data_files' --ghci-options=-fobject-code pandoc:lib pandoc:test-pandoc"
56−
5740bench:
58− stack bench \
59− --ghc-options '$(GHCOPTS)' \
60− --benchmark-arguments='$(BENCHARGS)' 2>&1 | \
61− tee "bench_latest.txt"
41+ stack bench --benchmark-arguments='$(BENCHARGS)' --ghc-options '$(GHCOPTS)'
42+
43+weigh:
44+ stack build --ghc-options '$(GHCOPTS)' pandoc:weigh-pandoc && stack exec weigh-pandoc
6245
6346reformat:
6447 for f in $(SOURCEFILES); do echo $$f; stylish-haskell -i $$f ; done
@@ -82,22 +65,21 @@ dist: man/pandoc.1
8265 cd pandoc-${version}
8366 stack setup && stack test && cd .. && rm -rf "pandoc-${version}"
8467
85−check: checkdocs check-cabal
86−
87−checkdocs:
88− ! grep -q -n -e "\t" MANUAL.txt changelog.md
68+checkdocs: README.md
69+ ! grep -n -e "\t" MANUAL.txt changelog
8970
9071debpkg: man/pandoc.1
9172 docker run -v `pwd`:/mnt \
9273 -v `pwd`/linux/artifacts:/artifacts \
93− --user $(id -u):$(id -g) \
9474 -e REVISION=$(REVISION) \
9575 -w /mnt \
96− --memory=0 \
97− --rm \
98− $(DOCKERIMAGE) \
99− bash \
100− /mnt/linux/make_artifacts.sh 2>&1 > docker.log
76+ utdemir/ghc-musl:v12-libgmp-ghc8101 bash \
77+ /mnt/linux/make_artifacts.sh
78+
79+macospkg:
80+ rm -rf macos-release-candidate
81+ aws s3 sync s3://travis-jgm-pandoc macos-release-candidate
82+ make -C macos-release-candidate
10183
10284man/pandoc.1: MANUAL.txt man/pandoc.1.before man/pandoc.1.after
10385 pandoc $< -f markdown -t man -s \
@@ -135,14 +117,4 @@ update-website:
135117clean:
136118 stack clean
137119
138−check-cabal: git-files.txt sdist-files.txt
139− echo "Checking to see if all committed test/data files are in sdist."
140− diff -u $^
141−
142−sdist-files.txt: .FORCE
143− cabal sdist --list-only | sed 's/\.\///' | grep '^\(test\|data\)\/' | sort > $@
144−
145−git-files.txt: .FORCE
146− git ls-tree -r --name-only HEAD | grep '^\(test\|data\)\/' | sort > $@
147−
148−.PHONY: .FORCE deps quick full haddock install clean test bench changes_github dist prof download_stats reformat lint weigh doc/lua-filters.md pandoc-templates trypandoc update-website debpkg checkdocs ghcid ghci fix_spacing hlint check check-cabal
120+.PHONY: deps quick full haddock install clean test bench changes_github macospkg dist prof download_stats reformat lint weigh doc/lua-filters.md pandoc-templates trypandoc update-website debpkg macospkg checkdocs ghcid ghci fix_spacing hlint
README.md+7−9
@@ -8,10 +8,10 @@ pandoc --lua-filter tools/update-readme.lua README.template -o README.md
88[![github
99release](https://img.shields.io/github/release/jgm/pandoc.svg?label=current+release)](https://github.com/jgm/pandoc/releases)
1010[![hackage
11−release](https://img.shields.io/hackage/v/pandoc.svg?label=hackage)](https://hackage.haskell.org/package/pandoc)
12−[![homebrew](https://img.shields.io/homebrew/v/pandoc.svg)](https://formulae.brew.sh/formula/pandoc)
11+release](https://img.shields.io/hackage/v/pandoc.svg?label=hackage)](http://hackage.haskell.org/package/pandoc)
12+[![homebrew](https://img.shields.io/homebrew/v/pandoc.svg)](http://brewformulas.org/Pandoc)
1313[![stackage LTS
14−package](https://stackage.org/package/pandoc/badge/lts)](https://www.stackage.org/lts/package/pandoc-types)
14+package](http://stackage.org/package/pandoc/badge/lts)](http://stackage.org/lts/package/pandoc)
1515[![CI
1616tests](https://github.com/jgm/pandoc/workflows/CI%20tests/badge.svg)](https://github.com/jgm/pandoc/actions)
1717[![license](https://img.shields.io/badge/license-GPLv2+-lightgray.svg)](https://www.gnu.org/licenses/gpl.html)
@@ -20,7 +20,7 @@ groups](https://img.shields.io/badge/pandoc-discuss-red.svg?style=social)](https
2020
2121## The universal markup converter
2222
23−Pandoc is a [Haskell](https://haskell.org) library for converting from
23+Pandoc is a [Haskell](http://haskell.org) library for converting from
2424one markup format to another, and a command-line tool that uses this
2525library. It can convert *from*
2626
@@ -95,8 +95,6 @@ It can convert *to*
9595- `asciidoc` ([AsciiDoc](https://www.methods.co.nz/asciidoc/)) or
9696 `asciidoctor` ([AsciiDoctor](https://asciidoctor.org/))
9797- `beamer` ([LaTeX beamer](https://ctan.org/pkg/beamer) slide show)
98−- `bibtex` ([BibTeX](https://ctan.org/pkg/bibtex) bibliography)
99−- `biblatex` ([BibLaTeX](https://ctan.org/pkg/biblatex) bibliography)
10098- `commonmark` ([CommonMark](https://commonmark.org) Markdown)
10199- `commonmark_x` ([CommonMark](https://commonmark.org) Markdown with
102100 extensions)
@@ -175,7 +173,7 @@ It can convert *to*
175173 and JavaScript slide show)
176174- `slidy` ([Slidy](https://www.w3.org/Talks/Tools/Slidy2/) HTML and
177175 JavaScript slide show)
178−- `dzslides` ([DZSlides](https://paulrouget.com/dzslides/) HTML5 +
176+- `dzslides` ([DZSlides](http://paulrouget.com/dzslides/) HTML5 +
179177 JavaScript slide show),
180178- `revealjs` ([reveal.js](https://revealjs.com/) HTML5 + JavaScript
181179 slide show)
@@ -238,7 +236,7 @@ opening a new issue.
238236
239237## License
240238
241−© 2006-2021 John MacFarlane (jgm@berkeley.edu). Released under the
242−[GPL](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html "GNU General Public License"),
239+© 2006-2020 John MacFarlane (jgm@berkeley.edu). Released under the
240+[GPL](http://www.gnu.org/copyleft/gpl.html "GNU General Public License"),
243241version 2 or greater. This software carries no warranty of any kind.
244242(See COPYRIGHT for full copyright and warranty notices.)
README.template+6−6
@@ -7,9 +7,9 @@ Pandoc
77======
88
99[![github release](https://img.shields.io/github/release/jgm/pandoc.svg?label=current+release)](https://github.com/jgm/pandoc/releases)
10−[![hackage release](https://img.shields.io/hackage/v/pandoc.svg?label=hackage)](https://hackage.haskell.org/package/pandoc)
11−[![homebrew](https://img.shields.io/homebrew/v/pandoc.svg)](https://formulae.brew.sh/formula/pandoc)
12−[![stackage LTS package](https://stackage.org/package/pandoc/badge/lts)](https://www.stackage.org/lts/package/pandoc-types)
10+[![hackage release](https://img.shields.io/hackage/v/pandoc.svg?label=hackage)](http://hackage.haskell.org/package/pandoc)
11+[![homebrew](https://img.shields.io/homebrew/v/pandoc.svg)](http://brewformulas.org/Pandoc)
12+[![stackage LTS package](http://stackage.org/package/pandoc/badge/lts)](http://stackage.org/lts/package/pandoc)
1313[![CI tests](https://github.com/jgm/pandoc/workflows/CI%20tests/badge.svg)](https://github.com/jgm/pandoc/actions)
1414[![license](https://img.shields.io/badge/license-GPLv2+-lightgray.svg)](https://www.gnu.org/licenses/gpl.html)
1515[![pandoc-discuss on google groups](https://img.shields.io/badge/pandoc-discuss-red.svg?style=social)](https://groups.google.com/forum/#!forum/pandoc-discuss)
@@ -81,9 +81,9 @@ new issue.
8181License
8282-------
8383
84−© 2006-2021 John MacFarlane (jgm@berkeley.edu). Released under the
84+© 2006-2020 John MacFarlane (jgm@berkeley.edu). Released under the
8585[GPL], version 2 or greater. This software carries no warranty of
8686any kind. (See COPYRIGHT for full copyright and warranty notices.)
8787
88−[GPL]: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html "GNU General Public License"
89−[Haskell]: https://haskell.org
88+[GPL]: http://www.gnu.org/copyleft/gpl.html "GNU General Public License"
89+[Haskell]: http://haskell.org
RELEASE-CHECKLIST+2−3
@@ -1,11 +1,10 @@
1−[ ] use 'policeman' to check API changes and version update
21[ ] make README.md and commit if needed
32[ ] make man/pandoc.1 and commit if needed
43[ ] Finalize changelog
4+ tools/changelog-helper.sh
55[ ] push release candidate branch rc/VERSION and (if it builds
66 successfully) download artifacts
7−[ ] run tools/build-arm.sh to create and download arm64 linux package
8−[ ] download macos artifacts from CircleCI
7+[ ] make macospkg to fetch macos package (built on travis)
98[ ] Use 'make' in macos and windows artifacts to sign code
109[ ] make update-website
1110[ ] make trypandoc
SECURITY.mddeleted−12
@@ -1,12 +0,0 @@
1−# Security Policy
2−
3−## Supported Versions
4−
5−Only the most recent version of pandoc is supported with security updates.
6−
7−## Reporting a Vulnerability
8−
9−To report a vulnerability, email the maintainer, jgm@berkeley.edu.
10−But first please read the section of the manual entitled
11−[A note on security](https://pandoc.org/MANUAL.html#a-note-on-security),
12−which describes some security guarantees pandoc does NOT make.
benchmark/benchmark-pandoc.hs+84−65
@@ -1,6 +1,7 @@
1+{-# LANGUAGE NoImplicitPrelude #-}
12{-# LANGUAGE OverloadedStrings #-}
23{-
3−Copyright (C) 2012-2021 John MacFarlane <jgm@berkeley.edu>
4+Copyright (C) 2012-2019 John MacFarlane <jgm@berkeley.edu>
45
56This program is free software; you can redistribute it and/or modify
67it under the terms of the GNU General Public License as published by
@@ -16,88 +17,106 @@ You should have received a copy of the GNU General Public License
1617along with this program; if not, write to the Free Software
1718Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
1819-}
20+import Prelude
1921import Text.Pandoc
2022import Text.Pandoc.MIME
21−import Control.DeepSeq (force)
22−import Control.Monad.Except (throwError)
23+import Control.Monad.Except (throwError, liftIO)
2324import qualified Text.Pandoc.UTF8 as UTF8
2425import qualified Data.ByteString as B
2526import qualified Data.Text as T
26−import Test.Tasty.Bench
27−-- import Gauge
27+import Criterion.Main
28+import Criterion.Types (Config(..))
29+import Data.List (intersect)
30+import Data.Maybe (mapMaybe, catMaybes)
31+import System.Environment (getArgs)
2832import qualified Data.ByteString.Lazy as BL
29−import Data.Maybe (mapMaybe)
30−import Data.List (sortOn)
33+
34+data Input = InputText {unInputText :: T.Text}
35+ | InputBS {unInputBS :: BL.ByteString}
3136
3237readerBench :: Pandoc
3338 -> T.Text
34− -> Maybe Benchmark
35−readerBench _ name
36− | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
37−readerBench doc name = either (const Nothing) Just $
38− runPure $ do
39− (rdr, rexts) <- getReader name
40− (wtr, wexts) <- getWriter name
41− case (rdr, wtr) of
42− (TextReader r, TextWriter w) -> do
43− inp <- w def{ writerWrapText = WrapAuto
44− , writerExtensions = wexts } doc
45− return $ bench (T.unpack name) $
46− nf (either (error . show) id . runPure . r def) inp
47− (ByteStringReader r, ByteStringWriter w) -> do
48− tmpl <- Just <$> compileDefaultTemplate name
49− inp <- w def{ writerWrapText = WrapAuto
50− , writerExtensions = wexts
51− , writerTemplate = tmpl } doc
52− return $ bench (T.unpack name) $
53− nf (either (error . show) id .
54− runPure . r def{readerExtensions = rexts}) inp
55− _ -> throwError $ PandocSomeError $ "text/bytestring format mismatch: "
56− <> name
39+ -> IO (Maybe Benchmark)
40+readerBench doc name = do
41+ let (rdr, rexts) = either (error . show) id . runPure $ getReader name
42+ res <- runIO $ do
43+ (wtr, wexts) <- getWriter name
44+ case (rdr, wtr) of
45+ (TextReader r, TextWriter w) -> do
46+ setResourcePath ["./test"]
47+ inp <- w def{ writerWrapText = WrapAuto
48+ , writerExtensions = wexts } doc
49+ return (r def{ readerExtensions = rexts } . unInputText, InputText inp)
50+ (ByteStringReader r, ByteStringWriter w) -> do
51+ setResourcePath ["./test"]
52+ tmpl <- Just <$> compileDefaultTemplate name
53+ inp <- w def{ writerWrapText = WrapAuto
54+ , writerExtensions = wexts
55+ , writerTemplate = tmpl } doc
56+ liftIO $ BL.writeFile "/tmp/test.odt" inp
57+ return (r def{ readerExtensions = rexts } . unInputBS, InputBS inp)
58+ _ -> throwError $ PandocSomeError $ "text/bytestring format mismatch: "
59+ <> name
60+ return $ case res of
61+ Right (readerFun, inp) ->
62+ Just $ bench (T.unpack $ name <> " reader")
63+ $ nf (\i -> either (error . show) id $ runPure (readerFun i))
64+ inp
65+ Left _ -> Nothing
5766
5867getImages :: IO [(FilePath, MimeType, BL.ByteString)]
5968getImages = do
60− ll <- B.readFile "test/lalune.jpg"
61− mv <- B.readFile "test/movie.jpg"
62− return [("lalune.jpg", "image/jpg", BL.fromStrict ll)
63− ,("movie.jpg", "image/jpg", BL.fromStrict mv)]
69+ ll <- BL.readFile "test/lalune.jpg"
70+ mv <- BL.readFile "test/movie.jpg"
71+ return [("lalune.jpg", "image/jpg", ll)
72+ ,("movie.jpg", "image/jpg", mv)]
6473
65−writerBench :: [(FilePath, MimeType, BL.ByteString)]
66− -> Pandoc
74+writerBench :: Pandoc
6775 -> T.Text
6876 -> Maybe Benchmark
69−writerBench _ _ name
70− | name `elem` ["bibtex", "biblatex", "csljson"] = Nothing
71−writerBench imgs doc name = either (const Nothing) Just $
72− runPure $ do
73− (wtr, wexts) <- getWriter name
74− case wtr of
75− TextWriter writerFun ->
76− return $ bench (T.unpack name)
77− $ nf (\d -> either (error . show) id $
78− runPure $ do
79− mapM_ (\(fp,mt,bs) -> insertMedia fp (Just mt) bs) imgs
80− writerFun def{ writerExtensions = wexts} d)
81− doc
82− ByteStringWriter writerFun ->
83− return $ bench (T.unpack name)
77+writerBench doc name =
78+ case res of
79+ Right writerFun ->
80+ Just $ env getImages $ \imgs ->
81+ bench (T.unpack $ name <> " writer")
8482 $ nf (\d -> either (error . show) id $
85− runPure $ do
86− mapM_ (\(fp,mt,bs) -> insertMedia fp (Just mt) bs) imgs
87− writerFun def{ writerExtensions = wexts} d)
88− doc
83+ runPure (do mapM_
84+ (\(fp, mt, bs) ->
85+ insertMedia fp (Just mt) bs)
86+ imgs
87+ writerFun d)) doc
88+ Left _ -> Nothing
89+ where res = runPure $ do
90+ (wtr, wexts) <- getWriter name
91+ case wtr of
92+ TextWriter w ->
93+ return $ w def{ writerExtensions = wexts }
94+ _ -> throwError $ PandocSomeError
95+ $ "could not get text writer for " <> name
8996
9097main :: IO ()
9198main = do
99+ args <- filter (\x -> T.take 1 x /= "-") . fmap T.pack <$> getArgs
100+ print args
101+ let matchReader (n, _) =
102+ null args || ("reader" `elem` args && n `elem` args)
103+ matchWriter (n, TextWriter _) =
104+ null args || ("writer" `elem` args && n `elem` args)
105+ matchWriter _ = False
106+ allWriters = map fst (writers :: [(T.Text, Writer PandocPure)])
107+ matchedReaders = map fst (filter matchReader readers
108+ :: [(T.Text, Reader PandocPure)])
109+ matchedWriters = map fst (filter matchWriter writers
110+ :: [(T.Text, Writer PandocPure)])
92111 inp <- UTF8.toText <$> B.readFile "test/testsuite.txt"
93112 let opts = def
94− let doc = either (error . show) force $ runPure $ readMarkdown opts inp
95− defaultMain
96− [ env getImages $ \imgs ->
97− bgroup "writers" $ mapMaybe (writerBench imgs doc . fst)
98− (sortOn fst
99− writers :: [(T.Text, Writer PandocPure)])
100− , bgroup "readers" $ mapMaybe (readerBench doc . fst)
101− (sortOn fst
102− readers :: [(T.Text, Reader PandocPure)])
103− ]
113+ let doc = either (error . show) id $ runPure $ readMarkdown opts inp
114+ readerBs <- fmap catMaybes
115+ $ mapM (readerBench doc)
116+ $ filter (/="haddock")
117+ (matchedReaders `intersect` allWriters)
118+ -- we need the corresponding writer to generate
119+ -- input for the reader
120+ let writerBs = mapMaybe (writerBench doc) matchedWriters
121+ defaultMainWith defaultConfig{ timeLimit = 6.0 }
122+ (writerBs ++ readerBs)
benchmark/weigh-pandoc.hsnew file+50
@@ -0,0 +1,50 @@
1+{-# LANGUAGE NoImplicitPrelude #-}
2+{-# LANGUAGE OverloadedStrings #-}
3+{- |
4+ Module : Main
5+ Copyright : © 2016-2019 John MacFarlane <jgm@berkeley.edu>
6+ License : GNU GPL, version 2 or above
7+
8+ Maintainer : John MacFarlane <jgm@berkeley.edu>
9+ Stability : alpha
10+ Portability : portable
11+
12+Benchmarks to determine resource use of readers and writers.
13+-}
14+import Prelude
15+import Weigh
16+import Text.Pandoc
17+import Data.Text (Text, unpack)
18+
19+main :: IO ()
20+main = do
21+ doc <- read <$> readFile "test/testsuite.native"
22+ mainWith $ do
23+ func "Pandoc document" id doc
24+ mapM_
25+ (\(n,r) -> weighReader doc n (either (error . show) id . runPure . r def{readerExtensions = pandocExtensions}))
26+ [("markdown", readMarkdown)
27+ ,("html", readHtml)
28+ ,("docbook", readDocBook)
29+ ,("latex", readLaTeX)
30+ ,("commonmark", readCommonMark)
31+ ]
32+ mapM_
33+ (\(n,w) -> weighWriter doc n (either (error . show) id . runPure . w def))
34+ [("markdown", writeMarkdown)
35+ ,("html", writeHtml5String)
36+ ,("docbook", writeDocbook5)
37+ ,("latex", writeLaTeX)
38+ ,("commonmark", writeCommonMark)
39+ ]
40+
41+weighWriter :: Pandoc -> String -> (Pandoc -> Text) -> Weigh ()
42+weighWriter doc name writer = func (name ++ " writer") writer doc
43+
44+weighReader :: Pandoc -> Text -> (Text -> Pandoc) -> Weigh ()
45+weighReader doc name reader =
46+ case lookup name writers of
47+ Just (TextWriter writer) ->
48+ let inp = either (error . show) id $ runPure $ writer def{ writerWrapText = WrapAuto} doc
49+ in func (unpack $ name <> " reader") reader inp
50+ _ -> return () -- no writer for reader
cabal.project+7−9
@@ -1,13 +1,11 @@
11packages: pandoc.cabal
2−tests: True
3−flags: +embed_data_files
42
5−source-repository-package
6− type: git
7− location: https://github.com/jgm/doctemplates
8− tag: 428c26d5303cf7a2b1051fe1ffd9aafe9ba71c81
3+package pandoc
4+ flags: +embed_data_files -trypandoc
5+ ghc-options: -j +RTS -A64m -RTS
96
107source-repository-package
11− type: git
12− location: https://github.com/jgm/citeproc
13− tag: 4ce0501cd6f9c86eee087afcf878c256e49b9615
8+ type: git
9+ location: https://github.com/jgm/citeproc
10+ tag: 42b1d154b02435229acbe98ae0f17d01b757ee93
11+
changelog.md+2−1741
@@ -1,1741 +1,5 @@
11# Revision history for pandoc
22
3−## pandoc 2.14.0.3 (2021-06-22)
4−
5− * Text.Pandoc.MediaBag `insertMediaBag`: ensure we get a sane mediaPath
6− for URLs (#7391). In earlier 2.14.x versions, we'd get
7− incorrect paths for resources downloaded from URLs when the
8− media are extracted (including in PDF production).
9− * Text.Pandoc.Parsing: improve `emailAddress` (#7398).
10− Previously the parser would accept characters in domains
11− that are illegal in domains, and this sometimes caused it
12− to gobble bits of the following text.
13− * txt2tags reader: modify the email address parser so
14− it still includes form parameters, even after the change to
15− `emailAddress` in Text.Pandoc.Parsing.
16− * Text.Pandoc.Readers.Metadata: Fix regression with comment-only YAML
17− metadata blocks (#7400).
18− * reveal.js writer and template: better handling of options. Previously
19− it was impossible to specify false values for options that default to
20− true (e.g. `center`); setting the option to false just caused the portion
21− of the template setting the option to be omitted. Now we prepopulate
22− all the variables with their default values, including them all
23− unconditionally and allowing them to be overridden.
24− * Markdown writer: Fix regression in code blocks with attributes (#7397).
25− Code blocks with a single class but nonempty attributes
26− were having attributes drop as a result of #7242.
27− * LaTeX writer:
28− + Add strut at end of minipage if it contains line breaks.
29− Without them, the last line is not as tall as it should be in
30− some cases.
31− + Always use a minipage for cells with line breaks, when
32− width information is available (#7393). Otherwise the way we treat them
33− can lead to content that overflows a cell.
34− + Use `\strut` instead of `~` before `\\` in empty line.
35− * Use lts-18.0 stack resolver.
36− * Require skylighting 0.10.5.2 (adding support for Swift).
37− * Require commonmark 0.2.1.
38− * Rephrase section on unsafe HTML in manual.
39− * Create SECURITY.md
40−
41−## pandoc 2.14.0.2 (2021-06-13)
42−
43− * Fix MediaBag regressions (#7345). iIn the 2.14 release `--extract-media`
44− stopped working as before; there could be mismatches between the
45− paths in the rendered document and the extracted media.
46− This patch makes several changes that restore the earlier behavior
47− (while keeping the same API). The `mediaPath` in 2.14 was always
48− constructed from the SHA1 hash of the media contents. Now, we
49− preserve the original path unless it's an absolute path or contains
50− `..` segments (in that case we use a path based on the SHA1 hash of
51− the contents).
52−
53− In Text.Pandoc.MediaBag, `mediaDirectory` and `mediaItems` now use the
54− `mediaPath`, rather than the mediabag key, for the first component of the
55− tuple. This makes more sense, I think, and fits with the documentation of
56− these functions; eventually, though, we should rework the API so that
57− `mediaItems` returns both the keys and the MediaItems.
58−
59− In Text.Pandoc.Class.IO, rewriting of source paths in `extractMedia` has
60− been fixed.
61−
62− In Text.Pandoc.Class.PandocMonad, `fillMediaBag` has been modified so that
63− it doesn't modify image paths (that was part of the problem in #7345).
64−
65− We now do path normalization (e.g. `\` separators on Windows) in
66− writing the media.
67−
68− * Text.Pandoc.PDF:
69−
70− + Text.Pandoc.PDF: Fix regression in 2.14 for generation of PDFs with
71− SVGs (#7344).
72− + Only print relevant part of environment on `--verbose`. Since
73− `--verbose` output might be put in an issue, we want to avoid
74− spilling out secrets in environment variables.
75−
76− * Markdown reader: fix pipe table regression in 2.11.4 (#7343).
77− Previously pipe tables with empty headers (that is, a header
78− line with all empty cells) would be rendered as headerless
79− tables. This broke in 2.11.4. The fix here is to produce an
80− AST with an empty table head when a pipe table has all empty
81− header cells.
82−
83− * LaTeX reader: don't allow optional `*` on symbol control sequences
84− (#7340). Generally we allow optional starred variants of LaTeX commands
85− (since many allow them, and if we don't accept these explicitly,
86− ignoring the star usually gives acceptable results). But we
87− don't want to do this for `\(*\)` and similar cases.
88−
89− * Docx reader: handle absolute URIs in Relationship Target (#7374).
90−
91− * Docx writer: fix handling of empty table headers (Albert Krewinkel,
92− #7369). A table header which does not contain any cells is now treated as
93− an empty header.
94−
95− * LaTeX writer: Fix regression in table header position (#7347).
96− In recent versions the table headers were no longer bottom-aligned
97− (if more than one line). This patch fixes that by using minipages
98− for table headers in non-simple tables.
99−
100− * CommonMark writer:
101−
102− + Do not use simple class for fenced-divs (Jan Tojnar, amends #7242.)
103− + Do not throw away attributes when `Ext_attributes` is enabled.
104− `Ext_attributes` covers at least the following:
105− `Ext_fenced_code_attributes`, `Ext_header_attributes`,
106− `Ext_inline_code_attributes`, `Ext_link_attributes`.
107−
108− * Markdown writer:
109−
110− + Allow `pipe_tables` to be disabled for commonmark formats
111− (`commonmark_x`, `gfm`) (#7375).
112− + Re-use functions from Text.Pandoc.Markdown.Inline (Jan Tojnar).
113−
114− * DocBook writer: Remove non-existent admonitions (Jan Tojnar).
115− `attention`, `error` and `hint` are reStructuredText specific.
116−
117− * HTML writer: Don't omit width attribute on div (#7342).
118−
119− * Text.Pandoc.MIME, `extensionFromMimeType`: add a few special cases.
120− When we do a reverse lookup in the MIME table, we just get the
121− last match, so when the same mime type is associated with several
122− different extensions, we sometimes got weird results, e.g. `.vs`
123− for `text/plain`. These special cases help us get the most standard
124− extensions for mime types like `text/plain`.
125−
126− * Lua utils: fix handling of table headers in `from_simple_table` (Albert
127− Krewinkel, #7369). Passing an empty list of header cells now results
128− in an empty table header.
129−
130− * Text.Pandoc.Citeproc:
131−
132− + Avoid duplicate classes and attributes on references div.
133− + Fix regression in citeproc processing (#7376). If inline
134− references are used (in the metadata `references` field), we
135− should still only include in the bibliography items that are
136− actually cited (unless `nocite` is used).
137−
138− * Require citeproc 0.4.0.1. This fixes a bug which led to doubled
139− "et al." in some (rare) circumstances.
140−
141− * MANUAL.txt:
142−
143− + Mention GladTeX for EPUB export (Sebastian Humenda).
144− This updates the manual and the web site about the GladTeX usage.
145− + More details and a useful link for YAML syntax.
146−
147− * CONTRIBUTING.md: update modules overview (Albert Krewinkel).
148−
149− * using-the-pandoc-api.md: switch from String to Text (Albert Krewinkel).
150−
151−
152−## pandoc 2.14.0.1 (2021-06-01)
153−
154− * Commonmark reader: Fix regression in 2.14 with YAML metdata block parsing,
155− which could cause the document body to be omitted after metadata (#7339).
156−
157− * HTML reader: fix column width regression in 2.14 (#7334).
158− Column widths specified with a style attribute were off by a factor of 100.
159−
160− * Markdown reader: in `rebasePaths`, check for both Windows and Posix
161− absolute paths. Previously Windows pandoc was treating
162− `/foo/bar.jpg` as non-absolute.
163−
164− * Text.Pandoc.Logging: In rendering `LoadedResource`, use relative paths.
165−
166− * Docx writer: fix regression on captions (#7328). The "Table Caption"
167− style was no longer getting applied. (It was overwritten by "Compact.")
168−
169− * Use commonmark-extensions 0.2.1.2
170−
171−## pandoc 2.14 (2021-05-28)
172−
173− * Change reader types, allowing better tracking of source positions
174− [API change]. Previously, when multiple file arguments were provided,
175− pandoc simply concatenated them and passed the contents to the readers,
176− which took a Text argument. As a result, the readers had no way of knowing
177− which file was the source of any particular bit of text. This meant that
178− we couldn't report accurate source positions on errors or include accurate
179− source positions as attributes in the AST. More seriously, it meant that
180− we couldn't resolve resource paths relative to the files containing them
181− (see e.g. #5501, #6632, #6384, #3752).
182−
183− * Add `rebase_relative_paths` extension (#3752). When enabled, this
184− extension rewrites relative image and link paths by prepending
185− the (relative) directory of the containing file. This
186− behavior is useful when your input sources are split
187− into multiple files, across several directories, with files
188− referring to images stored in the same directory. The
189− extension can be enabled for all markdown and commonmark-based formats.
190−
191− * Add Text.Pandoc.Sources (exported module), with a `Sources` type and a
192− `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs
193− [API change]. A parsec `Stream` instance is provided for `Sources`. The
194− module also exports versions of parsec's `satisfy` and other Char parsers
195− that track source positions accurately from a `Sources` stream (or any
196− instance of the new `UpdateSourcePos` class).
197−
198− * Text.Pandoc.Parsing
199−
200− + Export the modified Char parsers defined in Text.Pandoc.Sources
201− instead of the ones parsec provides. Modified parsers to use a
202− `Sources` as stream [API change].
203− + Improve include file functions [API change]. Remove old
204− `insertIncludedFileF`. Give `insertIncludedFile` a more general type,
205− allowing it to be used where `insertIncludedFileF` was.
206− + Add parameter to the `citeKey` parser from
207− Text.Pandoc.Parsing, which controls whether the `@{..}`
208− syntax is allowed [API change].
209−
210− * Text.Pandoc.Error: Modified the constructor `PandocParsecError` to take a
211− `Sources` rather than a `Text` as first argument, so parse error locations
212− can be accurately reported.
213−
214− * Fix source position reporting for YAML bibliographies (#7273).
215−
216− * Issue error message when reader or writer format is malformed
217− (#7231). Previously we exited with an error status but (due to a bug)
218− no message.
219−
220− * Smarter smart quotes (#7216, #2103). Treat a leading `"` with no
221− closing `"` as a left curly quote. This supports the practice, in
222− fiction, of continuing paragraphs quoting the same speaker without an
223− end quote. It also helps with quotes that break over lines in line blocks.
224−
225− * Markdown reader:
226−
227− + Use MetaInlines not MetaBlocks for multimarkdown metadata fields.
228− This gives better results in converting to e.g. pandoc markdown.
229− + Implement curly-brace syntax for Markdown citation keys (#6026).
230− The change provides a way to use citation keys that contain
231− special characters not usable with the standard citation key syntax.
232− Example: `@{foo_bar{x}'}` for the key `foo_bar{x}`. It also allows
233− separating citation keys from immediately following text, e.g. `@{foo}A`.
234−
235− * RST reader:
236−
237− + Seek include files in the directory of the file
238− containing the include directive, as RST requires (#6632).
239− + Use `insertIncludedFile` from Text.Pandoc.Parsing
240− instead of reproducing much of its code.
241−
242− * Org reader: Resolve org includes relative to the directory containing the
243− file containing the INCLUDE directive (#5501).
244−
245− * ODT reader: Treat tabs as spaces (#7185, niszet).
246−
247− * Docx reader:
248−
249− + Add handling of vml image objects (#7257, mbrackeantidot).
250− + Support new table features (Emily Bourke, #6316): column
251− spans, row spans, multiple header rows, table description
252− (parsed as a simple caption), captions, column widths.
253−
254− * LaTeX reader:
255−
256− + Improved siunitx support (#6658, #6620).
257− + Better support for `\xspace` (#7299).
258− + Improve parsing of `\def` macros. We previously set "verbatim mode"
259− even for parsing the initial `\def`; this caused problems
260− for `\def` nested inside another `\def`.
261− + Implement `\newif`.
262−
263− * ConTeXt writer: improve ordered lists (#5016, Denis Maier).
264− Change ordered list from itemize to enumerate. Add new
265− itemgroup for ordered lists. Remove manual insertion of
266− width attributes. Use tabular figures in ordered list
267− enumerators.
268−
269− * HTML reader:
270−
271− + Don't fail on unmatched closing "script" tag (Albert Krenkel, #7282).
272− + Keep h1 tags as normal headers (#2293, Albert Krewinkel).
273− The tags `<title>` and `<h1 class="title">` often contain the same
274− information, so the latter was dropped from the document. However, as
275− this can lead to loss of information, the heading is now always
276− retained. Use `--shift-heading-level-by=-1` to turn the `<h1>`
277− into the document title, or a filter to restore the previous behavior.
278− + Handle relative lengths (e.g. `2*`) in HTML column widths (#4063).
279− See <https://www.w3.org/TR/html4/types.html#h-6.6>.
280−
281− * DocBook/JATS readers:
282−
283− + Fix mathml regression caused by the switch in XML libraries (#7173).
284− + Fix "phrase" in DocBook: take classes from "role" not "class" (#7195).
285−
286− * DocBook reader: ensure that first and last names are separated (#6541).
287−
288− * Jira reader (Albert Krewinkel, #7218):
289−
290− + Support "smart" links: `[alias|https://example.com|smart-card]` syntax.
291− + Allow spaces and most unicode characters in attachment links.
292− + No longer require a newline character after `{noformat}`.
293− + Only allow URI path segment characters in bare links.
294− + The `file:` schema is no longer allowed in bare links; these
295− rarely make sense.
296−
297− * Plain writer: handle superscript unicode minus (#7276).
298−
299− * LaTeX writer:
300−
301− + Better handling of line breaks in simple tables (#7272).
302− Now we also handle the case where they're embedded in other elements,
303− e.g. spans.
304− + For beamer output, support `exampleblock` and `alertblock` (#7278).
305− A block will be rendered as an `exampleblock` if the heading
306− has class `example` and an `alertblock` if it has class `alert`.
307− + Separate successive quote chars with thin space (#6958,
308− Albert Krewinkel). Successive quote characters are separated with
309− a thin space to improve readability and to prevent unwanted ligatures.
310− Detection of these quotes sometimes had failed if the second quote
311− was nested in a span element.
312− + Separate successive quote chars with thin space (#6958, Albert
313− Krewinkel).
314−
315− * EPUB Writer: Fix belongs-to-collection XML id choice (#7267, nuew).
316− The epub writer previously used the same XML id for both the book
317− identifier and the epub collection. This causes an error on epubcheck.
318−
319− * BibTeX/BibLaTeX writer: Handle `annote` field (#7266).
320−
321− * ZimWiki writer: allow links and emphasis in headers (#6605,
322− Albert Krewinkel).
323−
324− * ConTeXt writer:
325−
326− + Support blank lines in line blocks (#6564, Albert Krewinkel,
327− thanks to @denismaier).
328− + Use span identifiers as reference anchors (#7246, Albert Krewinkel).
329−
330− * HTML writer:
331−
332− + Keep attributes from code nested below `pre` tag (#7221,
333− Albert Krewinkel). If a code block is defined with `<pre><code
334− class="language-x">…</code></pre>`, where the `<pre>` element has no
335− attributes, then the attributes from the `<code>` element are used
336− instead. Any leading `language-` prefix is dropped in the code's
337− `class` attribute are dropped to improve syntax highlighting.
338− + Ensure headings only have valid attribs in HTML4 (#5944, Albert
339− Krewinkel).
340− + Parse `<header>` as a Div (Albert Krewinkel).
341−
342− * Org writer:
343−
344− + Inline latex envs need newlines (#7252, tecosaur).
345− As specified in https://orgmode.org/manual/LaTeX-fragments.html, an
346− inline \begin{}...\end{} LaTeX block must start on a new line.
347− + Use LaTeX style maths deliminators (#7196, tecosaur).
348−
349− * JATS writer (Albert Krewinkel):
350−
351− + Use either styled-content or named-content for spans (#7211).
352− If the element has a content-type attribute, or at least one class,
353− then that value is used as `content-type` and the span is put inside
354− a `<named-content>` element. Otherwise a `<styled-content>` element
355− is used instead.
356− + Reduce unnecessary use of `<p>` elements for wrapping (#7227).
357− The `<p>` element is used for wrapping in cases were the contents
358− would otherwise not be allowed in a certain context. Unnecessary
359− wrapping is avoided, especially around quotes (`<disp-quote>` elements).
360− + Convert spans to `<named-content>` elements (#7211). Spans with
361− attributes are converted to `<named-content>` elements instead of
362− being wrapped with `<milestone-start/>` and `<milestone-end>`
363− elements. Milestone elements are not allowed in documents using the
364− articleauthoring tag set, so this change ensures the creation of valid
365− documents.
366− + Add footnote number as label in backmatter (#7210). Footnotes in the
367− backmatter are given the footnote's number as a label. The
368− articleauthoring output is unaffected from this change, as footnotes
369− are placed inline there.
370− + Escape disallows chars in identifiers. XML identifiers must start
371− with an underscore or letter, and can contain only a limited set
372− of punctuation characters. Any IDs not adhering to these rules are
373− rewritten by writing the offending characters as `Uxxxx`,
374− where `xxxx` is the character's hex code.
375−
376− * Jira writer: use `{color}` when span has a color attribute
377− (Albert Krewinkel, tarleb/jira-wiki-markup#10).
378−
379− * Docx writer:
380−
381− + Autoset table width if no column has an explicit width (Albert
382− Krewinkel).
383− + Extract Table handling into separate module (Albert Krewinkel).
384− + Support colspans and rowspans in tables (Albert Krewinkel, #6315).
385− + Support multirow table headers (Albert Krewinkel).
386− + Improve integration of settings from reference.docx (#1209).
387− This change allows users to create a reference.docx that
388− sets `w:proofState` for spelling or grammar to `dirty`,
389− so that spell/grammar checking will be triggered on the
390− generated docx.
391− + Copy over more settings from reference.docx (#7240). From settings.xml
392− in the reference-doc, we now include: `zoom`, `embedSystemFonts`,
393− `doNotTrackMoves`, `defaultTabStop`, `drawingGridHorizontalSpacing`,
394− `drawingGridVerticalSpacing`, `displayHorizontalDrawingGridEvery`,
395− `displayVerticalDrawingGridEvery`, `characterSpacingControl`,
396− `savePreviewPicture`, `mathPr`, `themeFontLang`, `decimalSymbol`,
397− `listSeparator`, `autoHyphenation`, `compat`.
398− + Set zoom to 100% by default in settings.xml.
399− + Align math options more with current Word defaults (e.g. Cambria Math
400− font).
401− + Remove `rsid`s from default settings.xml. Word will add these
402− when revisions are made.
403−
404− * Ms writer: Handle tables with multiple paragraphs (#7288).
405− Previously they overflowed the table cell width. We now set line lengths
406− per-cell and restore them after the table has been written.
407−
408− * Markdown writer:
409−
410− + Use cleaner braceless syntax for code blocks with a
411− single class (#7242, Jan Tojnar).
412− + Add quotes properly in markdown YAML metadata fields (#7245).
413− This fixes a bug, which caused the writer to look at the *last*
414− rather than the *first* character in determining whether quotes
415− were needed. So we got spurious quotes in some cases and
416− didn't get necessary quotes in others.
417− + Use `@{..}` syntax for citations when needed.
418− + Use fewer unneeded escapes for `#` (see #6259).
419− + Improve escaping of `@`. We need to escape literal `@` before
420− `{` because of the new citation syntax.
421−
422− * Commonmark writer: Use backslash escapes for `<` and `|`...
423− instead of entities (#7208).
424−
425− * Powerpoint writer: allow `monofont` to be specified in metadata
426− (#7187).
427−
428− * LaTeX template:
429−
430− + Use non-starred names for xcolor color names (#6109).
431− This should make svgnames and x11names work properly.
432− + Fix bad vertical spacing after bibliography (#7234, badumont).
433− + List of figures before list of tables (#7235, Julien Dutant).
434− + Move CSL macro definitions before header-includes so they can be
435− overridden (#7286).
436− + Improve treatment of CSL `entry-spacing` (#7296).
437− Previously with the default template settings (`indent` variable
438− not set), we would get interparagraph spaces separating bib
439− entries even with `entry-spacing="0"`. On the other hand,
440− setting `entry-spacing="2"` gave ridiculously large spacing.
441− This change makes the spacing caused by `entry-spacing` a multiple
442− of `\parskip` by default, which gives aesthetically reasonable
443− output. Those who want a larger or smaller unit (e.g. because
444− they use `indent` which sets `\parskip` to 0) may
445− `\setlength{\cslentryspacingunit}{10pt}` in header-includes
446− to override the defaults.
447− + Move title, author, date up to top of preamble (#7295).
448− This allows header-includes to use them, and puts them
449− in a position where you can see them immediately.
450− + Define commands for zero width non-joiner character
451− (#6639, Albert Krewinkel). The zero-width non-joiner character
452− is used to avoid ligatures (e.g. in German).
453−
454− * ConTeXt template:
455−
456− + Define `enumerate` itemgroup (#5016, Denis Maier).
457− + List of figures before list of tables (#7235, Julien Dutant).
458−
459− * reveal.js template:
460−
461− + Support `toc-title` (#7171, Florian Kohrt).
462− + Use `hash: true` by default rather than `history: true` (#6968).
463−
464− * HTML-based slide shows: add support for `institute` (#7289, Thomas
465− Hodgson).
466−
467− * Text.Pandoc.Extensions: Add constructor `Ext_rebase_relative_paths` to
468− `Extensions` [API change].
469−
470− * Text.Pandoc.XML.Light: add Eq, Ord instances for Content,
471− Element, Attr, CDataKind [API change].
472−
473− * Text.Pandoc.MediaBag:
474−
475− + Change type to use a `Text` key instead of `[FilePath]`.
476− We normalize the path and use `/` separators for consistency.
477− + Export `MediaItem` type [API change].
478− + Change `MediaBag` type to a map from Text to MediaItem [API change].
479− + `lookupMedia` now returns a `MediaItem` [API change].
480− + Change `insertMedia` so it sets the `mediaPath` to a filename based on
481− the SHA1 hash of the contents. This will be used when contents
482− are extracted.
483−
484− * Text.Pandoc.Class.PandocMonad:
485−
486− + Remove `fetchMediaResource` [API change]. Use `fetchItem` to get
487− resources in `fillMediaBag`.
488− + Add informational message in `downloadOrRead` indicating what path
489− local resources have been loaded from.
490−
491− * Text.Pandoc.Logging:
492−
493− + Remove single quotes around paths in messages.
494− + Add LoadedResource constructor to LogMessage [API change].
495− This is for INFO-level messages telling where image data has been
496− loaded from. (This can vary because of the resource path.)
497−
498− * Text.Pandoc.Asciify: simplify code and export `toAsciiText` [API change].
499− Instead of encoding a giant (and incomplete) map, we now
500− just use unicode-transforms to normalize the text to
501− a canonical decomposition, and manipulate the result.
502−
503− * App: allow tabs expansion even if file-scope is used (Albert Krewinkel,
504− #6709). Tabs in plain-text inputs are now handled correctly, even if
505− the `--file-scope` flag is used.
506−
507− * Add new internal module Text.Pandoc.Writers.GridTable (Albert Krewinkel).
508−
509− * Text.Pandoc.Highlighting: Change type of `languagesByExtension`, adding
510− a parameter for a `SyntaxMap` [API change] (Jan Tojnar, #7241).
511− Languages defined using `--syntax-definition` were not recognized by
512− `languagesByExtension`. This patch corrects that, allowing the writers
513− to see all custom definitions. The LaTeX writer still uses the default
514− syntax map, but that's okay in that context, since
515− `--syntax-definition` won't create new listings styles.
516−
517− * Text.Pandoc.Citeproc:
518−
519− + Ensure that CSL-related attributes are passed on to a Div with id
520− 'refs'. Otherwise things like `entry-spacing` won't work when
521− such Divs are used.
522− + Use metadata's `lang` for the lang parameter of citeproc, overriding
523− `localeLanguage`.
524− + Recognize locators spelled with a capital letter (#7323).
525− + Add a comma and a space in front of the suffix if it doesn't start
526− with space or punctuation (#7324).
527− + Don't detect math elements as locators (#7321).
528−
529− * Remove Text.Pandoc.BCP47 module [API change]. Use types and functions
530− from UnicodeCollation.Lang instead. This is a richer implementation
531− of BCP 47.
532−
533− * Text.Pandoc.Shared:
534−
535− + Fix regression in grid tables for wide characters (#7214).
536− In the translation from String to Text, a char-width-sensitive
537− `splitAt'` was dropped. This commit reinstates it and uses it to make
538− `splitTextByInstances` char-width sensitive.
539− + Add `getLang` (formerly in the now-removed BCP47) [API change].
540−
541− * Text.Pandoc.SelfContained: use `application/octet-stream`
542− for unknown mime types instead of halting with an error (#7202).
543−
544− * Lua filters: respect Inlines/Blocks filter functions in `pandoc.walk_*`
545− (Albert Krewinkel).
546−
547− * Add text as build-depend for trypandoc (#7193, Roman Beránek).
548−
549− * Bump upper-bounds for network-uri, time, attoparsec.
550−
551− * Use citeproc 0.4.
552−
553− * Use texmath 0.12.3.
554−
555− * Use jira-wiki-markup 1.3.5 (Albert Krewinkel).
556−
557− * Require latest skylighting (fixes a bug in XML syntax highlighting).
558−
559− * Use latest xml-conduit.
560−
561− * Use latest commonmark, commonmark-extensions, commonmark-pandoc.
562−
563− * Use haddock-library-1.10.0 (Albert Krewinkel).
564−
565− * Allow compilation with base 4.15 (Albert Krewinkel).
566−
567− * MANUAL:
568−
569− + Add information about `lang` and bibliography sorting.
570− + Add info about YAML escape sequences, link to spec (#7152,
571− Albert Krewinkel).
572− + Note that `institute` variable works for HTML-based slides.
573− + Update documentation on citation syntax.
574− + Add citation example for locators and suffixes (Tristan Stenner)
575−
576− * Updated and fixed typos in documentation (Charanjit Singh,
577− Anti-Distinctlyminty, Tatiana Porras, obcat).
578−
579− * Add instructions for installing pandoc-types before compiling filter.
580−
581− * INSTALL: add note that parallel installations should be avoided
582− (#6865).
583−
584− * Remove `biblatex-nussbaum.md` test. It is basically the same
585− as `biblaetx-quotes.md`.
586−
587− * Command tests: fail if a file contains no tests---and fix a
588− test that failed in that way!
589−
590− * Use smaller images in tests, reducing the size of the source tarball by 8 MB.
591−
592−
593−## pandoc 2.13 (2021-03-21)
594−
595− * Support `yaml_metadata_block` extension for `commonmark`, `gfm` (#6537).
596− This supported is a bit more limited than with pandoc's
597− `markdown`. The YAML block must be the first thing in the input,
598− and the leaf notes are parsed in isolation from the rest of
599− the document. So, for example, you can't use reference
600− links if the references are defined later in the document.
601−
602− * Fix fallback to default partials when custom templates are used.
603− If the directory containing a template does not contain the partial,
604− it should be sought in the default templates, but this was not
605− working properly (#7164).
606−
607− * Handle `nocite` better with `--biblatex` and `--natbib` (#4585).
608− Previously the nocite metadata field was ignored with these formats.
609− Now it populates a `nocite-ids` template variable and causes a
610− `\nocite` command to be issued.
611−
612− * Text.Pandoc.Citeproc: apply `fixLinks` correctly (#7130). This is code
613− that incorporates a prefix like `https://doi.org/` into a following link
614− when appropriate.
615−
616− * Text.Pandoc.Shared:
617−
618− + Remove `backslashEscapes`, `escapeStringUsing` [API change]. Replace
619− these inefficient association list lookups with more efficient escaping
620− functions in the writers that used them (for a 10-25% performance boost
621− in org, haddock, rtf, texinfo writers).
622− + Remove `ToString`, `ToText` typeclasses [API change]. These were needed
623− for the transition from String to Text, but they are no longer used and
624− may clash with other things.
625− + Simplify `compactDL`.
626−
627− * Text.Pandoc.Parsing:
628−
629− + Change type of `readWithM` so that it is no longer polymorphic
630− [API change]. The `ToText` class has been removed, and now that we've
631− completed the transition to Text we no longer need this to operate
632− on Strings.
633− + Remove `F` type synonym [API change]. Muse and Org were defining their
634− own `F` anyway.
635−
636− * Text.Pandoc.Readers.Metadata:
637−
638− + Export `yamlMetaBlock` [API change].
639− + Make `yamlBsToMeta`, `yamlBsToRefs` polymorphic on the parser state
640− [API change].
641−
642− * Markdown reader: Fix regression with `tex_math_backslash` (#7155).
643−
644− * MediaWiki reader: Allow block-level content in notes (ref) (#7145).
645−
646− * Jira reader (Albert Krewinkel):
647−
648− + Fixed parsing of autolinks (i.e., of bare URLs in the text).
649− Previously an autolink would take up the rest of a line, as spaces
650− were allowed characters in these items.
651− + Emoji character sequences no longer cause parsing failures. This was
652− due to missing backtracking when emoji parsing fails.
653− + Mark divs created from panels with class "panel".
654−
655− * RST reader: fix logic for ending comments (#7134). Previously comments
656− sometimes got extended too far.
657−
658− * DocBook writer: include Header attributes as XML attributes on
659− section (Erik Rask). Attributes with key names that are not allowed
660− as XML attributes are dropped, as are attributes with invalid values
661− and `xml:id` (DocBook 5) and `id` (DocBook 4).
662−
663− * Docx writer:
664−
665− + Make `nsid` in `abstractNum` deterministic. Previously we assigned
666− a random number, but we don't need random values, so now we just
667− assign a value based on the list marker.
668− + Use integral values for `w:tblW` (#7141).
669−
670− * Jira writer (Albert Krewinkel):
671−
672− + Block quotes are only rendered as `bq.` if they do not contain a
673− linebreak.
674− + Jira writer: improve div/panel handling. Include div attributes in
675− panels, always render divs with class `panel` as panels, and
676− avoid nesting of panels.
677−
678− * HTML writer: Add warnings on duplicate attribute values.
679− This prevents emitting invalid HTML. Ultimately it would be good to
680− prevent this in the types themselves, but this is better for now.
681−
682− * Org writer: Prevent unintended creation of ordered list items (#7132,
683− Albert Krewinkel). Adjust line wrapping if default wrapping would cause
684− a line to be read as an ordered list item.
685−
686− * JATS templates: support 'equal-contrib' attrib for authors (Albert
687− Krewinkel). Authors who contributed equally to a paper may be marked
688− with `equal-contrib`.
689−
690− * reveal.js template: replace JS comment with HTML (#7154, Florian Kohrt).
691−
692− * Text.Pandoc.Logging: Add `DuplicateAttribute` constructor to `LogMessage`.
693− [API change]
694−
695− * Use `-j4` for linux release build. This speeds up the build dramatically
696− on arm.
697−
698− * cabal.project: remove ghcoptions. Move flags to top level, so they can
699− be set differently on the command line.
700−
701− * Require latest texmath, skylighting, citeproc, jira-wiki-markup.
702− (The latest skylighting fixes a bad bug with Haskell syntax highlighting.)
703− Narrow version bounds for texmath, skylighting, and citeproc, since
704− the test output depend on them.
705−
706− * Use doclayout 0.3.0.2. This significantly reduces the time and memory
707− needed to compile pandoc.
708−
709− * Use `foldl'` instead of `foldl` everywhere.
710−
711− * Update bounds for random (#7156, Alexey Kuleshevich).
712−
713− * Remove uses of some partial functions.
714−
715− * Don't bake in a larger stack size for the executable.
716−
717− * Test improvements:
718−
719− + Use `getExecutablePath` from base, avoiding the dependency on
720− `executable-path`.
721− + Factor out `setupEnvironment` in Helpers, to avoid code duplication.
722− + Fix finding of data files by setting teh `pandoc_datadir` environment
723− variable when we shell out to pandoc. This avoids the need to use
724− `--data-dir` for the tests, which caused problems finding `pandoc.lua`
725− when compiling without the `embed_data_files` flag (#7163).
726−
727− * Benchmark improvements:
728−
729− + Build `+RTS -A8m -RTS` into default ghc-options for benchmark.
730− This is necessary to get accurate benchmark results; otherwise we
731− are largely measuring garbage collecting, some not related to the
732− current benchmark.
733− + Allow specifying BASELINE file in 'make bench' for comparison
734− (otherwise the latest benchmark is chosen by default).
735− + Force `readFile` in benchmarks early (Bodigrim).
736−
737− * CONTRIBUTING: suggest using a `cabal.project.local` file (#7153,
738− Albert Krewinkel).
739−
740− * Add ghcid-test to Makefile. This loads the test suite in ghcid.
741−
742−
743−## pandoc 2.12 (2021-03-08)
744−
745− * `--resource-path` now accumulates if specified multiple
746− times (#6152). Resource paths specified later on the command line are
747− prepended to those specified earlier. Thus,
748− `--resource-path foo --resource-path bar:baz` is equivalent to
749− `--resource-path bar:bas:foo`. (The previous behavior was
750− for the last `--resource-path` to replace all the rest.)
751− `resource-path` in defaults files behaves the same way: it
752− will be prepended to the resource path set by earlier
753− command line options or defaults files. This change
754− facilitates the use of multiple defaults files: each can
755− specify a directory containing resources it refers to
756− without clobbering the resource paths set by the others.
757−
758− * Allow defaults files to refer to the home directory, the
759− user data directory, and the directory containing the defaults file
760− itself (#5871, #5982, #5977). In fields that expect file paths
761− (and only in these fields),
762−
763− + `${VARIABLE}` will expand to the value of the environment variable
764− `VARIABLE` (and in particular `${HOME}` will expand to the path
765− of the home directory). A warning will be raised for undefined
766− variables.
767− + `${USERDATA}` will expand to the path of the user data
768− directory in force when the defaults file is being processed.
769− + `${.}` will expand to the directory containing the defaults file.
770− (This allows default files to be placed in a directory containing
771− resources they make use of.)
772−
773− * When downloading content from URL arguments, be sensitive to
774− the character encoding (#5600). We can properly handle UTF-8 and latin1
775− (ISO-8859-1); for others we raise an error. Fall back to latin1 if
776− no charset is given in the mime type and UTF-8 decoding fails.
777−
778− * Allow abbreviations that don't end in a period to be
779− specified using `--abbreviations` (#7124).
780−
781− * Add new unexported module Text.Pandoc.XML.Light, as well
782− as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc,
783− Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).
784−
785− This module exports definitions of `Element` and `Content`
786− that are isomorphic to xml-light's, but with Text
787− instead of String. This allows us to keep most of the code in existing
788− readers that use xml-light, but avoid lots of unnecessary allocation.
789−
790− We also add versions of the functions from xml-light's
791− Text.XML.Light.Output and Text.XML.Light.Proc that operate on our
792− modified XML types, and functions that convert xml-light types to our
793− types (since some of our dependencies, like texmath, use xml-light).
794−
795− We export functions that use xml-conduit's parser to produce an
796− `Element` or `[Content]`. This allows existing pandoc code to use
797− a better parser without much modification.
798−
799− The new parser is used in all places where xml-light's parser was
800− previously used. Benchmarks show a significant performance improvement
801− in parsing XML-based formats (with docbook, opml, jats, and docx
802− almost twice as fast, odt and fb2 more than twice as fast).
803−
804− In addition, the new parser gives us better error reporting than
805− xml-light. We report XML errors, when possible, using the new
806− `PandocXMLError` constructor in `PandocError`.
807−
808− These changes revealed the need for some changes in the tests. The
809− docbook-reader.docbook test lacked definitions for the entities it used;
810− these have been added. And the docx golden tests have been updated,
811− because the new parser does not preserve the order of attributes.
812−
813− * DocBook reader:
814−
815− + Avoid expensive tree normalization step, as it is not necessary
816− with the new XML parser.
817− + Support `informalfigure` (#7079) (Nils Carlson).
818−
819− * Docx reader:
820−
821− + Use Map instead of list for Namespaces. This gives a speedup of
822− about 5-10%. With this and the XML parsing changes, the docx reader
823− is now about twice as fast as in the previous release.
824−
825− * HTML reader:
826−
827− + Small performance tweaks.
828− + Also, remove exported class `NamedTag(..)` [API change]. This was just
829− intended to smooth over the transition from String to Text and is no
830− longer needed.
831− + As a result, the functions `isInlineTag` and `isBlockTag`
832− are no longer polymorphic; they apply to a `Tag Text` [API change].
833− + Do a lookahead to find the right parser to use. This takes
834− benchmarks from 34ms to 23ms, with less allocation.
835− + Fix bad handling of empty `src` attribute in `iframe` (#7099).
836− If `src` is empty, we simply skip the `iframe`.
837− If `src` is invalid or cannot be fetched, we issue a warning
838− nd skip instead of failing with an error.
839−
840− * JATS reader:
841−
842− + Avoid tree normalization, which is no longer necessary given the
843− new XML parser.
844−
845− * LaTeX reader:
846−
847− + Don't export `tokenize`, `untokenize` [API change]. These are internal
848− implementation details, which were only exported for testing.
849− They don't belong in the public API.
850− + Improved efficiency of the parser. With these changes the reader
851− is almost twice as fast as in the last release in our benchmarks.
852− + Code cleanup, removing some unnecessary things.
853− + Rewrite `withRaw` so it doesn't rely on fragile assumptions
854− about token positions (which break when macros are expanded)
855− (#7092). This requires the addition of `sEnableWithRaw` and
856− `sRawTokens` in `LaTeXState`, and a new combinator `disablingWithRaw`
857− to disable collecting of raw tokens in certain contexts.
858− Add `parseFromToks` to Text.Pandoc.Readers.LaTeX.Parsing.
859− Fix parsing of single character tokens so it doesn't mess
860− up the new raw token collecting. These changes slightly increase
861− allocations and have a small performance impact.
862− + Handle some bibtex/biblatex-specific commands that used to be
863− dealt with in pandoc-citeproc (#7049).
864− + Optimize `satisfyTok`, avoiding unnecessary macro expansion steps.
865− Benchmarks after this change show 2/3 of the run time and 2/3 of the
866− allocation of the Feb. 10 benchmarks.
867− + Removed `sExpanded` in state. This isn't actually needed and checking
868− it doesn't change anything.
869− + Improve `braced'`. Remove the parameter, have it parse the
870− opening brace, and make it more efficient.
871− + Factor out pieces of the LaTeX reader to make the module smaller.
872− This reduces memory demands when compiling. Created
873− Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Inline}.
874− Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map
875− instead of individual commands.
876− + Handle table cells containing `&` in `\verb` (#7129).
877−
878− * Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change].
879−
880− * Markdown reader:
881−
882− + Improved handling of mmd link attributes in references (#7080).
883− Previously they only worked for links that had titles.
884− + Improved efficiency of the parser (benchmarks show a 15% speedup).
885−
886− * OPML reader:
887−
888− + Avoid tree normalization, which is no longer necessary with the
889− new XML parser.
890−
891− * ODT reader:
892−
893− + Finer-grained errors on parse failure (#7091).
894− + Give more information if the zip container can't be unpacked.
895−
896− * Org reader:
897−
898− + Support `task_lists` extension (Albert Krewinkel, #6336).
899− + Fix bug in org-ref citation parsing (Albert Krewinkel, #7101).
900− The org-ref syntax allows to list multiple citations separated by
901− comma. Previously commas were accepted as part of the citation id,
902− so all citation lists were parsed as one single citation.
903−
904− * RST reader:
905−
906− + Use `getTimestamp` instead of `getCurrentTime` to fetch timestamp.
907− Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
908− + RST reader: fix handling of header in CSV tables (#7064).
909− The interpretation of this line is not affected by the delim option.
910−
911− * Jira reader:
912−
913− + Modified the Doc parser to skip leading blank lines. This fixes
914− parsing of documents which start with multiple blank lines (Albert
915− Krewinkel, #7095).
916− + Prevent URLs within link aliases to be treated as autolinks (Albert
917− Krewinkel, #6944).
918−
919− * Text.Pandoc.Shared
920−
921− + Remove formerly exported functions that are no longer used in the
922− code base: `splitByIndices`, `splitStringByIndicies`, `substitute`,
923− and `underlineSpan` (which had been deprecated in April 2020)
924− [API change].
925− + Export `handleTaskListItem` (Albert Krewinkel) [API change].
926− + Change `defaultUserDataDirs` to `defaultUserDataDir` [API
927− change]. We determine what is the default user data directory
928− by seeing whether the XDG directory and/or legacy
929− directory exist.
930−
931− * BibTeX writer:
932−
933− + BibTeX writer: use doclayout and doctemplate. This change allows
934− bibtex/biblatex output to wrap as other formats do,
935− depending on the settings of `--wrap` and `--columns` (#7068).
936−
937− * CSL JSON writer:
938−
939− + Output `[]` if no references in input, instead of raising a
940− PandocAppError as before.
941−
942− * Docx writer:
943−
944− + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
945− Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
946−
947− * EPUB writer:
948−
949− + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
950− Setting `SOURCE_DATE_EPOCH` will allow reproducible builds (#7093).
951− This does not suffice to fully enable reproducible in EPUB, since
952− a unique id is still being generated for each build.
953− + Support `belongs-to-collection` metadata (#7063) (Nick Berendsen).
954−
955− * JATS writer:
956−
957− + Escape special chars in reference elements (Albert Krewinkel).
958− Prevents the generation of invalid markup if a citation element
959− contains an ampersand or another character with a special meaning
960− in XML.
961−
962− * Jira writer:
963−
964− + Use Span identifiers as anchors (Albert Krewinkel).
965− + Use `{noformat}` instead of `{code}` for unknown languages (Albert
966− Krewinkel). Code blocks which are not marked as a language supported
967− by Jira are rendered as preformatted text via `{noformat}` blocks.
968−
969− * LaTeX writer:
970−
971− + Adjust hypertargets to beginnings of paragraphs (#7078).
972− Use `\vadjust pre` so that the hypertarget takes you to the beginning
973− of the paragraph rather than one line down.
974− This makes a particular difference for links to citations using
975− `--citeproc` and `link-citations: true`.
976− + Change BCP47 lang tag from `jp` to `ja` (Mauro Bieg, #7047).
977− + Use function instead of map for accent lookup (should be
978− more efficient).
979− + Split the module to make it easier to compile on low-memory
980− systems: added Text.Pandoc.Writers.LaTeX.{Util,Citation,Lang}.
981−
982− * Markdown writer:
983−
984− + Handle math right before digit. We insert an HTML comment to
985− avoid a `$` right before a digit, which pandoc will not recognize
986− as a math delimiter.
987− + Split the module to make it easier to compile on low-memory
988− systems: added Text.Pandoc.Writers.Markdown.{Types,Inline}.
989−
990− * ODT writer:
991−
992− + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
993− Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
994− + Update default ODT style (Lorenzo). Previously, the "First paragraph"
995− style inherited from "Standard" but not from "Text body." Now
996− it is adjusted to inherit from "Text body", to avoid some ugly
997− spacing issues. It may be necessary to update a custom `reference.odt`
998− in light of this change.
999−
1000− * Org writer:
1001−
1002− + Support `task_lists` extension (Albert Krewinkel, #6336).
1003−
1004− * Pptx writer:
1005−
1006− + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
1007− Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
1008−
1009− * JATS templates: tag `author.name` as `string-name` (Albert Krewinkel).
1010− The partitioning the components of a name into surname, given names,
1011− etc. is not always possible or not available. Using `author.name`
1012− allows to give the full name as a fallback to be used when
1013− `author.surname` is not available.
1014−
1015− * Add default templates for bibtex and biblatex, so that
1016− the variables `header-include`, `include-before`, `include-after`
1017− (or alternatively the command line options
1018− `--include-in-header`, `--include-before-body`, `--include-after-body`)
1019− may be used.
1020−
1021− * LaTeX template:
1022−
1023− + Update to iftex package (#7073) (Andrew Dunning)
1024− + Wrap url colours in braces (#7121) (Loïc Grobol).
1025−
1026− * revealjs template: Add 'center' option for vertical slide centering.
1027− (maurerle, #7104).
1028−
1029− * Text.Pandoc.XML: Improve efficiency of `fromEntities`.
1030−
1031− * Text.Pandoc.MIME
1032−
1033− + Add exported function `getCharset` [API change].
1034−
1035− * Text.Pandoc.UTF8: change IO functions to return Text, not String
1036− [API change]. This affects `readFile`, `getContents`, `writeFileWith`,
1037− `writeFile`, `putStrWith`, `putStr`, `putStrLnWith`, `putStrLn`.
1038− `hPutStrWith`, `hPutStr`, `hPutStrLnWith`, `hPutStrLn`, `hGetContents`.
1039− This avoids the need to uselessly create a linked list of characters
1040− when emiting output.
1041−
1042− * Text.Pandoc.App
1043−
1044− + Add `parseOptionsFromArgs` [API change, new exported function].
1045− + Add fields for CSL options to `Opt` [API change]:
1046− `optCSL`, `optbibliography`, `optCitationAbbreviations`.
1047−
1048− * Text.Pandoc.Citeproc.BibTeX
1049−
1050− + `Text.Pandoc.Citeproc.writeBibTeXString` now returns
1051− `Doc Text` instead of `Text` (#7068).
1052− + Correctly handle `pages` (= `page` in CSL) (#7067).
1053− + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067).
1054− + In BibTeX output, protect foreign titles since there's no language
1055− field (#7067).
1056− + Clean up BibTeX parsing (#7049). Previously there was a messy code
1057− path that gave strange results in some cases, not passing through raw
1058− tex but trying to extract a string content. This was an artefact of
1059− trying to handle some special bibtex-specific commands in the BibTeX
1060− reader. Now we just handle these in the LaTeX reader and simplify
1061− parsing in the BibTeX reader. This does mean that more raw tex will
1062− be passed through (and currently this is not sensitive to the
1063− `raw_tex` extension; this should be fixed).
1064−
1065− * Text.Pandoc.Citeproc.MetaValue
1066−
1067− + Correctly parse "raw" date value in markdown references metadata.
1068− (See jgm/citeproc#53.)
1069−
1070− * Text.Pandoc.Citeproc
1071−
1072− + Use https URLs for links (Salim B, #7122).
1073−
1074− * Text.Pandoc.Class
1075−
1076− + Add `getTimestamp` [API change]. This attempts to read the
1077− `SOURCE_DATE_EPOCH` environment variable and parse a UTC time
1078− from it (treating it as a unix date stamp, see
1079− https://reproducible-builds.org/specs/source-date-epoch/). If the
1080− variable is not set or can't be parsed as a unix date stamp, then the
1081− function returns the current date.
1082−
1083− * Text.Pandoc.Error
1084−
1085− + Add `PandocUnsupportedCharsetError` constructor for
1086− `PandocError` [API change].
1087− + Export `renderError` [API change].
1088− + Refactor `handleError` to use `renderError`. This allows us render
1089− error messages without exiting.
1090−
1091− * Text.Pandoc.Extensions
1092−
1093− + `Ext_task_lists` is now supported by org (and turned
1094− on by default) (Albert Krewinkel, #6336).
1095− + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes
1096− (#7097). This attribute was listed as allowed, but it didn't actually
1097− do anything. Use `attributes` for code attributes and more.
1098−
1099− * Lua subsystem:
1100−
1101− + Always load built-in Lua scripts from default data-dir (Albert
1102− Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always
1103− loaded from the system's default data directory. Loading from a
1104− different directory by overriding the default path, e.g. via
1105− `--data-dir`, is no longer supported to avoid unexpected behavior
1106− and to address security concerns.
1107− + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565).
1108− The module allows to work with file paths in a convenient and
1109− platform-independent manner.
1110− + Use strict evaluation when retrieving AST value from the stack
1111− (Albert Krewinkel, #6674).
1112−
1113− * Text.Pandoc.PDF
1114−
1115− + Disable `smart` extension when building PDF via LaTeX.
1116− This is to prevent accidental creation of ligatures like
1117− `` ?` `` and `` !` `` (especially in languages with quotations like
1118− German), and similar ligature issues. (See jgm/citeproc#54.)
1119−
1120− * Text.Pandoc.CSV:
1121−
1122− + Fix parsing of unquoted values (#7112). Previously we didn't allow
1123− unescaped quotes in unquoted values, but they are allowed
1124− in CSV.
1125−
1126− * Test suite:
1127−
1128− + Use a more robust method for testing the executable. Many
1129− of our tests require running the pandoc executable. This
1130− is problematic for a few different reasons. First,
1131− cabal-install will sometimes run the test suite after
1132− building the library but before building the executable,
1133− which means the executable isn't in place for the tests.
1134− One can work around that by first building, then building
1135− and running the tests, but that's fragile. Second, we
1136− have to find the executable. So far, we've done that using
1137− a function `findPandoc` that attempts to locate it
1138− relative to the test executable (which can be located
1139− using findExecutablePath). But the logic here is delicate
1140− and work with every combination of options. To solve both
1141− problems, we add an `--emulate` option to the
1142− `test-pandoc` executable. When `--emulate` occurs as the
1143− first argument passed to `test-pandoc`, the program simply
1144− emulates the regular pandoc executable, using the rest of
1145− the arguments (after `--emulate`). Thus, `test-pandoc
1146− --emulate -f markdown -t latex` is just like `pandoc -f
1147− markdown -t latex`. Since all the work is done by library
1148− functions, implementing this emulation just takes a couple
1149− lines of code and should be entirely reliable. With this
1150− change, we can test the pandoc executable by running the
1151− test program itself (locatable using `findExecutablePath`)
1152− with the `--emulate` option. This removes the need for the
1153− fragile `findPandoc` step, and it means we can run our
1154− integration tests even when we're just building the
1155− library, not the executable. [Note: part of this change
1156− involved simplifying some complex handling to set
1157− environment variables for dynamic library paths. I have
1158− tested a build with `--enable-dynamic-executable`, and it
1159− works, but further testing may be needed.]
1160− + Print accurate location if a test fails (Albert
1161− Krewinkel). Ensures that tasty-hunit reports the location
1162− of the failing test instead of the location of the helper
1163− `test` function.
1164−
1165− * Documentation: Update URLs and use `https` where possible (#7122,
1166− Salim B).
1167−
1168− * Add `doc/libraries.md`, a description of libraries that support pandoc.
1169−
1170− * MANUAL.txt
1171−
1172− + MANUAL: block-level formatting is not allowed in line blocks (#7107).
1173− + Clarify `tex_math_dollars` extension. Note that no blank lines
1174− are allowed between the delimiters in display math.
1175− + Add MANUAL section on reproducible builds.
1176− + Document no template fallback for absolute path (#7077, Nixon
1177− Enraght-Moony.)
1178− + Improve docs for cite-method.
1179− + Update README and man page.
1180−
1181− * Makefile: in `make bench`, create CSV files for comparison and compare
1182− against previous benchmark run. Add timestamp to CSV filenames.
1183−
1184− * cabal.project: don't explicitly set -trypandoc.
1185− If we do, this can't be overridden on the cabal command line.
1186−
1187− * doc/lua-filters.md: improve documentation for
1188− `pandoc.mediabag.insert`, `pandoc.mediabag.fetch`,
1189− `directory`, `normalize` (Albert Krewinkel).
1190−
1191− * Allow base64-bytestring-1.2.* (Dmitrii Kovanikov)
1192−
1193− * Require jira-wiki-markup 1.3.3 (Albert Krewinkel)
1194−
1195− * Require citeproc 0.3.0.8, which correctly titlecases when titles
1196− contain non-ASCII characters.
1197−
1198− * Use skylighting 0.10.4. This version of skylighting uses xml-conduit
1199− rather than hxt. This speeds up parsing of XML syntax definitions
1200− fourfold, and removes four packages from pandoc's dependency graph:
1201− hxt-charproperties, hxt-unicode, hxt-regex-xmlschema, hxt.
1202−
1203− * Add script `tools/parseTimings.pl` to help pin down which
1204− modules take the most time and memory to compile.
1205−
1206− * Avoid unnecessary use of NoImplicitPrelude pragma (#7089) (Albert
1207− Krewinkel)
1208−
1209− * Benchmarks
1210−
1211− + Use the lighter-weight tasty-bench instead of criterion.
1212− + Run writer benchmarks for binary formats too.
1213− + Alphabetize benchmarks.
1214− + Don't run benchmarks for bibliography formats
1215− (yet; we need a special input for them).
1216− + Show allocation data
1217− + Clean up benchmark code.
1218− + Allow specifying patterns using `-p blah'.
1219−
1220− * trypandoc: add 2 second timeout.
1221−
1222− * Use `-split-sections` in creating linux release binary.
1223− This reduces executable size significantly (by about 30%).
1224−
1225− * Remove `weigh-pandoc`. It's not really useful any more, now that our
1226− regular benchmarks include data on allocation.
1227−
1228− * Improve linux package build process and add script to
1229− automate building an arm64 binary package.
1230−
1231−
1232−## pandoc 2.11.4 (2021-01-22)
1233−
1234− * Add `biblatex`, `bibtex` as output formats (closes #7040).
1235−
1236− * Recognize more extensions as markdown by default (#7034):
1237− `mkdn`, `mkd`, `mdwn`, `mdown`, `Rmd`.
1238−
1239− * Implement defaults file inheritance (#6924, David Martschenko).
1240− Allow defaults files to inherit options from other defaults files by
1241− specifying them with the following syntax:
1242− `defaults: [list of defaults files or single defaults file]`.
1243−
1244− * Fix infinite HTTP requests when writing epubs from URL source (#7013).
1245− Due to a bug in code added to avoid overwriting the cover image
1246− if it had the form `fileX.YYY`, pandoc made an endless sequence
1247− of HTTP requests when writing epub with input from a URL.
1248−
1249− * Org reader:
1250−
1251− + Allow multiple pipe chars in todo sequences (Albert Krewinkel, #7014).
1252− Additional pipe chars, used to separate "action" state from "no further
1253− action" states, are ignored. E.g., for the following sequence, both
1254− `DONE` and `FINISHED` are states with no further action required:
1255− `#+TODO: UNFINISHED | DONE | FINISHED`.
1256− + Restructure output of captioned code blocks (Albert Krewinkel, #6977).
1257− The Div wrapper of code blocks with captions now has the class
1258− "captioned-content". The caption itself is added as a Plain block
1259− inside a Div of class "caption". This makes it easier to write filters
1260− which match on captioned code blocks. Existing filters will need to be
1261− updated.
1262− + Mark verbatim code with class `verbatim` (Dimitri Sabadie, #6998).
1263−
1264− * LaTeX reader:
1265−
1266− + Handle `filecontents` environment (#7003).
1267− + Put contents of unknown environments in a Div when `raw_tex` is not
1268− enabled (#6997). (When `raw_tex` is enabled, the whole environment is
1269− parsed as a raw block.) The class name is the name of the environment.
1270− Previously, we just included the contents without the surrounding Div,
1271− but having a record of the environment's boundaries and name can be
1272− useful.
1273−
1274− * Mediawiki reader:
1275−
1276− + Allow space around storng/emph delimiters (#6993).
1277−
1278− * New module Text.Pandoc.Writers.BibTeX, exporting
1279− writeBibTeX and writeBibLaTeX. [API change]
1280−
1281− * LaTeX writer:
1282−
1283− + Revert table line height increase in 2.11.3 (#6996).
1284− In 2.11.3 we started adding `\addlinespace`, which produced less dense
1285− tables. This wasn't an intentional change; I misunderstood a comment in
1286− the discussion leading up to the change. This commit restores the earlier
1287− default table appearance. Note that if you want a less dense table, you
1288− can use something like `\def\arraystretch{1.5}` in your header.
1289−
1290− * EPUB writer:
1291−
1292− + Adjust internal links to identifiers defined in raw HTML sections
1293− after splitting into chapters (#7000).
1294− + Recognize `Format "html4"`, `Format "html5"` as raw HTML.
1295− + Adjust internal links to images, links, and tables after splitting into
1296− chapters. Previously we only did this for Div and Span and Header
1297− elements (see #7000).
1298−
1299− * Ms writer:
1300−
1301− + Don't justify text inside table cells.
1302−
1303− * JATS writer:
1304−
1305− + Use `<element-citation>` if `element_citations`
1306− extension is enabled (Albert Krewinkel).
1307− + Fix citations (Albert Krewinkel, #7018). By default
1308− we use formatted citations.
1309− + Ensure that `<disp-quote>` is always wrapped in `<p>` (#7041).
1310−
1311− * Markdown writer:
1312−
1313− + Cleaned up raw formats. We now react appropriately
1314− to `gfm`, `commonmark`, and `commonmark_x` as raw formats.
1315−
1316− * RST writer:
1317−
1318− + Fix bug with dropped content from inside spans with a class in
1319− some cases (#7039).
1320−
1321− * Docx writer:
1322−
1323− + Handle table header using styles (#7008). Instead of hard-coding
1324− the border and header cell vertical alignment, we now let this
1325− be determined by the Table style, making use of Word's
1326− "conditional formatting" for the table's first row. For
1327− headerless tables, we use the tblLook element to tell Word
1328− not to apply conditional first-row formatting.
1329−
1330− * Commonmark writer:
1331−
1332− + Implement start number on ordered lists (#7009). Previously they always
1333− started at 1, but according to the spec the start number is respected.
1334−
1335− * HTML writer:
1336−
1337− + Fix implicit_figure at end of footnotes (#7006).
1338−
1339− * ConTeXt template: Remove `\setupthinrules` from default template.
1340− The width parameter this used is not actually supported,
1341− and the command didn't do anything.
1342−
1343− * Text.Pandoc.Extensions:
1344−
1345− + Add `Ext_element_citations` constructor (Albert Krewinkel).
1346−
1347− * Text.Pandoc.Citeproc.BibTeX: New unexported function
1348− `writeBibtexString`.
1349−
1350− * Text.Pandoc.Citeproc:
1351−
1352− + Use finer grained imports (Albert Krewinkel).
1353− + Factor out and export `getStyle` [API change].
1354− + Export `getReferences` [API change, #7106].
1355− + Factor out getLang.
1356−
1357− * Text.Pandoc.Parsing: modify `gridTableWith'` for headerless tables.
1358− If the table lacks a header, the header row should be an empty
1359− list. Previously we got a list of empty cells, which caused
1360− an empty header to be emitted instead of no header. In LaTeX/PDF
1361− output that meant we got a double top line with space between.
1362−
1363− * ImageSize: use `viewBox` for SVG if no length, width attributes (#7045).
1364− This change allows pandoc to extract size information from more SVGs.
1365−
1366− * Add simple default.nix.
1367−
1368− * Use commonmark 0.1.1.3.
1369−
1370− * Use citeproc 0.3.0.5.
1371−
1372− * Update default CSL to use latest chicago-author-date.csl.
1373−
1374− * CONTRIBUTING.md: add note on GNU xargs.
1375−
1376− * MANUAL.txt:
1377−
1378− + Update description of `-L`/`--lua-filter`.
1379− + Document use of citations in note styles (#6828).
1380−
1381−## pandoc 2.11.3.2 (2020-12-29)
1382−
1383− * HTML reader: use renderTags' from Text.Pandoc.Shared (Albert Krewinkel).
1384− A side effect of this change is that empty `<col>` elements are written
1385− as self-closing tags in raw HTML blocks.
1386−
1387− * Asciidoc writer: Add support for writing nested tables (#6972, timo-a).
1388− Asciidoc supports one level of nesting. If deeper tables are to be
1389− written, they are omitted and a warning is issued.
1390−
1391− * Docx writer: fix nested tables with captions (#6983).
1392− Previously we got unreadable content, because docx seems
1393− to want a `<w:p>` element (even an empty one) at the end of
1394− every table cell.
1395−
1396− * Powerpoint writer: allow arbitrary OOXML in raw inline elements
1397− (Albert Krewinkel). The raw text is now included verbatim in the
1398− output. Previously is was parsed into XML elements, which prevented
1399− the inclusion of partial XML snippets.
1400−
1401− * LaTeX writer: support colspans and rowspans in tables (#6950,
1402− Albert Krewinkel). Note that the multirow package is needed for
1403− rowspans. It is included in the latex template under a variable,
1404− so that it won't be used unless needed for a table.
1405−
1406− * HTML writer: don't include p tags in CSL bibliography entries
1407− (#6966). Fixes a regression in 2.11.3.
1408−
1409− * Add `meta-description` variable to HTML templates (#6982). This
1410− is populated by the writer by stringifying the `description`
1411− field of metadata (Jerry Sky). The `description` meta tag will
1412− make the generated HTML documents more complete and SEO-friendly.
1413−
1414− * Citeproc: fix handling of empty URL variables (`DOI`, etc.).
1415− The `linkifyVariables` function was changing these to links
1416− which then got treated as non-empty by citeproc, leading
1417− to wrong results (e.g. ignoring nonempty URL when empty DOI is present).
1418− See jgm/citeproc#41.
1419−
1420− * Use citeproc 0.3.0.3. Fixes an issue in author-only citations when
1421− both an author and translator are present, and an issue with
1422− citation group delimiters.
1423−
1424− * Require texmath 0.12.1. This improves siunitx support in math,
1425− fixes bugs with `\*mod` family operators and arrays, and avoids
1426− italicizing symbols and operator names in docx output.
1427−
1428− * Ensure that the perl interpreter used for filters with `.pl`
1429− extension (wuffi).
1430−
1431− * MANUAL: note that textarea content is never parsed as Markdown
1432− (Albert Krewinkel).
1433−
1434−
1435−## pandoc 2.11.3.1 (2020-12-18)
1436−
1437− * Added some missing files to extra-source-files and data
1438− files, so they are included in the sdist tarball. Closes #6961.
1439− Cleaned up some extraneous data and test files, and added
1440− a CI check to ensure that the test and data files included
1441− in the sdist match what is in the git repository.
1442−
1443− * Use citeproc 0.3.0.1, which avoids removing nonbreaking
1444− space at the end of the `initialize-with` attribute. (Some
1445− journals require nonbreaking space after initials, and this
1446− makes that possible.)
1447−
1448−## pandoc 2.11.3 (2020-12-17)
1449−
1450− * With `--bibliography` (or `bibliography` in metadata), a
1451− URL may now be provided, and pandoc will fetch the resource.
1452− In addition, if a file path is provided and it is not
1453− found relative to the working directory, the resource
1454− path will be searched (#6940).
1455−
1456− * Add `sourcepos` extension for `commonmark`, `gfm`, `commonmark_x`
1457− (#4565). With the `sourcepos` extension set set, `data-pos`
1458− attributes are added to the AST by the commonmark reader. No other
1459− readers are affected. The `data-pos` attributes are put on elements
1460− that accept attributes; for other elements, an enlosing Div or Span
1461− is added to hold the attributes.
1462−
1463− * Change extensions for `commonmark_x`: replace `auto_identifiers`
1464− with `gfm_auto_identifiers` (#6863). `commonmark_x` never actually
1465− supported `auto_identifiers` (it didn't do anything), because the
1466− underlying library implements gfm-style identifiers only. Attempts
1467− to add the `auto_identifiers` extension to `commonmark` will now
1468− fail with an error.
1469−
1470− * HTML reader:
1471−
1472− + Split module into several submodules (Albert Krewinkel). Reducing
1473− module size should reduce memory use during compilation.
1474− + Support advanced table features (Albert Krewinkel):
1475− block level content in captions, row and colspans,
1476− body headers, row head columns, footers, attributes.
1477− + Disable round-trip testing for tables. Information for cell
1478− alignment in a column is not preserved during round-trips (Albert
1479− Krewinkel).
1480− + Allow finer grained options for tag omission (Albert Krewinkel).
1481− + Simplify list attribute handling (Albert Krewinkel).
1482− + Pay attention to `lang` attributes on body element (#6938).
1483− These (as well as `lang` attributes on the html element) should update
1484− lang in metadata.
1485− + Retain attribute prefixes and avoid duplicates (#6938).
1486− Previously we stripped attribute prefixes, reading `xml:lang` as
1487− `lang` for example. This resulted in two duplicate `lang`
1488− attributes when `xml:lang` and `lang` were both used. This commit
1489− causes the prefixes to be retained, and also avoids invald
1490− duplicate attributes.
1491−
1492− * Commonmark reader:
1493−
1494− + Refactor `specFor`.
1495− + Set input name to `""` to avoid clutter in sourcepos output.
1496−
1497− * Org reader:
1498−
1499− + Parse `#+LANGUAGE` into `lang` metadata field (#6845, Albert
1500− Krewinkel).
1501− + Preserve targets of spurious links (#6916, Albert
1502− Krewinkel). Links with (internal) targets that the reader doesn't
1503− know about are converted into emphasized text. Information on the
1504− link target is now preserved by wrapping the text in a Span of class
1505− `spurious-link`, with an attribute `target` set to the link's
1506− original target. This allows to recover and fix broken or unknown
1507− links with filters.
1508−
1509− * DocBook reader:
1510−
1511− + Table text width support (#6791, Nils Carlson).
1512− Table width in relation to text width is not natively supported
1513− by docbook but is by the docbook `fo` stylesheets through an XML
1514− processing instruction, `<?dbfo table-width="50%"?>`.
1515−
1516− * LaTeX reader:
1517−
1518− + Improve parsing of command options (#6869, #6873).
1519− In cases where we run into trouble parsing inlines til the
1520− closing `]`, e.g. quotes, we return a plain string with the
1521− option contents. Previously we mistakenly included the brackets
1522− in this string.
1523− + Preserve center environment (#6852, Igor Pashev).
1524− The contents of the `center` environment are put in a `Div`
1525− with class `center`.
1526− + Don't parse `\rule` with width 0 as horizontal rule. These are
1527− sometimes used as spacers in LaTeX.
1528− + Don't apply theorem default styling to a figure inside (#6925).
1529− If we put an image in italics, then when rendering to Markdown
1530− we no longer get an implicit figure.
1531−
1532− * Dokuwiki reader:
1533−
1534− + Handle unknown interwiki links better (#6932).
1535− DokuWiki lets the user define his own Interwiki links. Previously
1536− pandoc reacted to these by emitting a google search link, which is
1537− not helpful. Instead, we now just emit the full URL including the
1538− wikilink prefix, e.g. `faquk>FAQ-mathml`. This at least gives users
1539− the ability to modify the links using filters.
1540−
1541− * Markdown writer:
1542−
1543− + Properly handle boolean values in writing YAML metadata (#6388).
1544− + Ensure that a new csl-block begins on a new line (#6921).
1545− This just looks better and doesn't affect the semantics.
1546−
1547− * RST writer:
1548−
1549− + Better image handling (#6948). An image alone in its paragraph
1550− (but not a figure) is now rendered as an independent image, with an
1551− `alt` attribute if a description is supplied. An inline image that
1552− is not alone in its paragraph will be rendered, as before, using a
1553− substitution. Such an image cannot have a "center", "left", or
1554− "right" alignment, so the classes `align-center`, `align-left`, or
1555− `align-right` are ignored. However, `align-top`, `align-middle`,
1556− `align-bottom` will generate a corresponding `align` attribute.
1557−
1558− * Docx writer:
1559−
1560− + Keep raw openxml strings verbatim (#6933, Albert Krewinkel).
1561− + Use Content instead of Element. This allows us to inject
1562− raw OpenXML into the document without reparsing it into an
1563− Element, which is necessary if you want to inject an open
1564− tag or close tag.
1565− + Fix bullets/lists indentation, so that the first level is slightly
1566− indented to the right instead of right on the margin (cholonam).
1567− + Support bold and italic in "complex script" (#6911).
1568− Previously bold and italics didn't work properly in LTR
1569− text. This commit causes the w:bCs and w:iCs attributes
1570− to be used, in addition to w:b and w:i, for bold and
1571− italics respectively.
1572−
1573− * ICML writer:
1574−
1575− + Fix image bounding box for custom widths/heighta (Mauro Bieg, #6936).
1576−
1577− * LaTeX writer:
1578−
1579− + Improve table spacing (#6842, #6860).
1580− Remove the `\strut` that was added at the end of minipage
1581− environments in cells. Replace `\tabularnewline` with
1582− `\\ \addlinespace`.
1583− + Improve calculation of column spacing (#6883).
1584− + Extract table handling into separate module (Albert Krewinkel).
1585− + Fix bug with nested `csl-` display Spans (#6921).
1586− + Improve longtable output (#6883). Don't create minipages for
1587− regular paragraphs. Put width and alignment information in the
1588− longtable column descriptors.
1589−
1590− * OpenDocument writer:
1591−
1592− + Support for table width as a percentage of text width
1593− (#6792, Nils Carson).
1594− + Implement Div and Span ident support (#6755, Nils Carson).
1595− Spans and Divs containing an ident in the Attr will become bookmarks
1596− or sections with idents in OpenDocument format.
1597− + Add two extensions, `xrefs_name` and `xrefs_number` (#6774, Nils
1598− Carlson). Links to headings, figures and tables inside the
1599− document are substituted with cross-references that will use the
1600− name or caption of the referenced item for `xrefs_name` or the
1601− number for `xrefs_number`. For the `xrefs_number` to be useful
1602− heading numbers must be enabled in the generated document and
1603− table and figure captions must be enabled using for example the
1604− `native_numbering` extension. In order for numbers and reference
1605− text to be updated the generated document must be refreshed.
1606−
1607− * JATS writer:
1608−
1609− + Support advanced table features (Albert Krewinkel).
1610− + Support author affiliations (#6687, Albert Krewinkel).
1611−
1612− * Docbook writer:
1613−
1614− + Use correct id attribute consistently (Jan Tojnar).
1615− DocBook5 should always use `xml:id` instead of `id`.
1616− + Handle admonition titles better (Jan Tojnar).
1617− Docbook reader produces a `Div` with `title` class for `<title>`
1618− element within an “admonition” element. Markdown writer then turns
1619− this into a fenced div with `title` class attribute. Since fenced
1620− divs are block elements, their content is recognized as a
1621− paragraph by the Markdown reader. This is an issue for Docbook
1622− writer because it would produce an invalid DocBook document from
1623− such AST – the `<title>` element can only contain “inline”
1624− elements. Handle this special case separately by unwrapping
1625− the paragraph before creating the `<title>` element.
1626− + Add XML namespaces to top-level elements (#6923, Jan Tojnar).
1627− Previously, we only added `xmlns` attributes to chapter
1628− elements, even when running with `--top-level-division=section`.
1629− These namespaces are now added to part and section elements too,
1630− when they are the selected top-level divisions.
1631− We do not need to add namespaces to documents produced with
1632− `--standalone` flag, since those will already have xmlns attribute on
1633− the root element in the template.
1634−
1635− * HTML writer:
1636−
1637− + Fix handling of nested `csl-` display spans (#6921).
1638− Previously inner Spans used to represent CSL display attributes were
1639− not rendered as div tags as intended.
1640−
1641− * EPUB writer:
1642−
1643− + Include title page in landmarks (#6919).
1644− Note that the toc is also included if `--toc` is specified.
1645− + Add frontmatter type on body element for nav.xhtml (#6918).
1646−
1647− * EPUB templates: use preserveAspectRatio="xMidYMid" for cover image (#6895,
1648− Shin Sang-jae). This change affects both the epub2 and the epub3
1649− templates. It avoids distortion of the cover image by requiring that the
1650− aspect ratio be preserved.
1651−
1652− * LaTeX template:
1653−
1654− + Include `csquotes` package if `csquotes` variable set.
1655− + Put back `amssymb`. We need it for checkboxes in todo lists,
1656− and maybe for other things. In this location it seems compatible
1657− with the cases that prompted #6469 and PR #6762.
1658− + Disable language-specific shorthands in babel (#6817, #6887).
1659− Babel defines "shorthands" for some languages, and these can
1660− produce unexpected results. For example, in Spanish, `1.22`
1661− gets rendered as `122`, and `et~al.` as `etal`.
1662− One would think that babel's `shorthands=off` option (which
1663− we were using) would disable these, but it doesn't. So we
1664− remove `shorthands=off` and add some code that redefines
1665− the shorthands macro. Eventually this will be fixed in babel,
1666− I hope, and we can revert to something simpler.
1667−
1668− * JATS template: allow array of persistent institute ids in `pid`
1669− (Albert Krewinkel).
1670−
1671− * Text.Pandoc.Parsing: minor code and efficiency improvements.
1672−
1673− * Text.Pandoc.Extension:
1674−
1675− + Add `Ext_sourcepos` constructor for `Extension` [API change].
1676− + Add `Ext_xrefs_name` and `Ext_xrefs_number` constructors for
1677− `Extension` (Nils Carson) [API change].
1678−
1679− * Text.Pandoc.Citeproc:
1680−
1681− + Fix truncation of `[Citation]` list in `Cite` inside footnotes (#6890).
1682− This affected author-in-text citations in footnotes. It didn't cause
1683− problems for the printed output, but for filters that expected the
1684− citation id and other information.
1685− + Allow the use of both inline and external references (#6951),
1686− as with pandoc-citeproc. References defined in the document's
1687− metadata take priority over references with the same id defined in
1688− an external bibliography.
1689− + Use `fetchItem` to get external bibliography (#6940).
1690− + Ensure that BCP47 lang codes can be used. We ignore the variants
1691− and just use the base lang code and country code when passing off
1692− to citeproc.
1693− + Citeproc BibTeX parser: revert change in `getRawField`
1694− which was made (for reasons forgotten) when transferring
1695− this code from pandoc-citeproc. The change led to `--` in
1696− URLs being interpreted as en-dashes, which is unwanted (#6874).
1697−
1698− * Text.Pandoc.ImageSize:
1699−
1700− + Default to DPI 72 if the format specifies DPI of 0 (#6880).
1701− This shouldn't happen, in general, but it can happen with
1702− JPEGs that don't conform to the spec. Having a DPI of 0
1703− will blow up size calculations (division by 0).
1704− + ImageSize: use JuicyPixels to determine size for png, jpeg, and
1705− gif, instead of doing our own binary parsing (#6936). This
1706− gives more reliable results.
1707−
1708− * Text.Pandoc.CSS:
1709−
1710− + Remove `foldOrElse` (internal module) (Albert Krewinkel).
1711−
1712− * Use skylighting 0.10.2 (#6625).
1713−
1714− * Use citeproc 0.3. This fixes issues with references with
1715− duplicate ids (jgm/citeproc#36).
1716−
1717− * Use doctemplates 0.9. This fixes issues with boolean
1718− metadata values in the Markdown writer (#6388)
1719− and in `meta-json` (#6650). It also fixes
1720− issues with nested for loops in templates.
1721−
1722− * Add translations zh-Hans.yaml and zh-Hant.yaml (#6904, #6909,
1723− Kolen Cheung, taotieren).
1724−
1725− * Add tests: True to cabal.project.
1726− This fixes some CI failures for cabal.
1727−
1728− * Normalize test/tables/*.native (#6888, Kolen Cheung).
1729−
1730− * Move executable to `app` directory to avoid problems with cabal repl.
1731−
1732− * CONTRIBUTING: add section "How can I help?" (#6892, Albert Krewinkel).
1733− Also adds a paragraph aimed at highlighting the importance of feature
1734− maintenance.
1735−
1736− * MANUAL: Document that --number-sections works in `ms` (#6935).
1737−
1738−
17393## pandoc 2.11.2 (2020-11-19)
17404
17415 * Default to using ATX (`##`-style) headings for Markdown output
@@ -1820,11 +84,8 @@
182084 translated to `C:\/foo/bar`, which caused problems.
182185 With this fix, the backslashes are removed.
182286
1823− * Text.Pandoc.Logging:
1824−
1825− + Add constructor `ATXHeadingInLHS` to `LogMessage` [API change].
1826− + Add constructor `EnvironmentVariableUndefined` to
1827− `LogMessage` [API change].
87+ * Text.Pandoc.Logging: Add constructor `ATXHeadingInLHS` constructor
88+ to `LogMessage` [API change].
182889
182990 * Fix error that is given when people specify `doc` output (#6834,
183091 gison93).
data/default.csl+2−12
@@ -27,9 +27,6 @@
2727 <email>matthew.g.roth@yale.edu</email>
2828 <uri> https://orcid.org/0000-0001-7902-6331</uri>
2929 </contributor>
30− <contributor>
31− <name>Brenton M. Wiernik</name>
32− </contributor>
3330 <category citation-format="author-date"/>
3431 <category field="generic-base"/>
3532 <summary>The author-date variant of the Chicago style</summary>
@@ -512,15 +509,8 @@
512509 </choose>
513510 </macro>
514511 <macro name="event">
515− <group delimiter=" ">
516− <choose>
517− <if variable="genre">
518− <text term="presented at"/>
519− </if>
520− <else>
521− <text term="presented at" text-case="capitalize-first"/>
522− </else>
523− </choose>
512+ <group>
513+ <text term="presented at" suffix=" "/>
524514 <text variable="event"/>
525515 </group>
526516 </macro>
data/docx/word/settings.xml+16−6
@@ -1,6 +1,6 @@
11<?xml version="1.0" encoding="UTF-8"?>
22<w:settings xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:w10="urn:schemas-microsoft-com:office:word" xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:sl="http://schemas.openxmlformats.org/schemaLibrary/2006/main">
3− <w:zoom w:percent="100" />
3+ <w:zoom w:percent="90" />
44 <w:embedSystemFonts />
55 <w:proofState w:spelling="clean" w:grammar="clean" />
66 <w:stylePaneFormatFilter w:val="0004" />
@@ -17,21 +17,31 @@
1717 <w:characterSpacingControl w:val="doNotCompress" />
1818 <w:savePreviewPicture />
1919 <w:rsids>
20+ <w:rsidRoot w:val="00590D07" />
21+ <w:rsid w:val="00011C8B" />
22+ <w:rsid w:val="004E29B3" />
23+ <w:rsid w:val="00590D07" />
24+ <w:rsid w:val="00784D58" />
25+ <w:rsid w:val="008D6863" />
26+ <w:rsid w:val="00B86B75" />
27+ <w:rsid w:val="00BC48D5" />
28+ <w:rsid w:val="00C36279" />
29+ <w:rsid w:val="00E315A3" />
2030 </w:rsids>
2131 <m:mathPr>
22− <m:mathFont m:val="Cambria Math" />
32+ <m:mathFont m:val="Lucida Grande" />
2333 <m:brkBin m:val="before" />
2434 <m:brkBinSub m:val="--" />
25− <m:smallFrac m:val="0" />
26− <m:dispDef />
35+ <m:smallFrac m:val="false" />
36+ <m:dispDef m:val="false" />
2737 <m:lMargin m:val="0" />
2838 <m:rMargin m:val="0" />
2939 <m:wrapRight />
3040 <m:intLim m:val="subSup" />
31− <m:naryLim m:val="undOvr" />
41+ <m:naryLim m:val="subSup" />
3242 </m:mathPr>
3343 <w:themeFontLang w:val="en-US" />
3444 <w:clrSchemeMapping w:bg1="light1" w:t1="dark1" w:bg2="light2" w:t2="dark2" w:accent1="accent1" w:accent2="accent2" w:accent3="accent3" w:accent4="accent4" w:accent5="accent5" w:accent6="accent6" w:hyperlink="hyperlink" w:followedHyperlink="followedHyperlink" />
3545 <w:decimalSymbol w:val="." />
3646 <w:listSeparator w:val="," />
37−</w:settings>
47+</w:settings>
data/docx/word/styles.xml−15
@@ -350,21 +350,6 @@
350350 <w:right w:w="108" w:type="dxa" />
351351 </w:tblCellMar>
352352 </w:tblPr>
353− <w:tblStylePr w:type="firstRow">
354− <w:tblPr>
355− <w:jc w:val="left"/>
356− <w:tblInd w:w="0" w:type="dxa"/>
357− </w:tblPr>
358− <w:trPr>
359− <w:jc w:val="left"/>
360− </w:trPr>
361− <w:tcPr>
362− <w:vAlign w:val="bottom"/>
363− <w:tcBorders>
364− <w:bottom w:val="single"/>
365− </w:tcBorders>
366− </w:tcPr>
367− </w:tblStylePr>
368353 </w:style>
369354 <w:style w:type="paragraph" w:customStyle="1" w:styleId="DefinitionTerm">
370355 <w:name w:val="Definition Term" />
data/jats.cslnew file+203
@@ -0,0 +1,203 @@
1+<?xml version="1.0" encoding="utf-8"?>
2+<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" default-locale="en-US">
3+ <info>
4+ <title>Journal Article Tag Suite</title>
5+ <title-short>JATS</title-short>
6+ <id>http://www.zotero.org/styles/journal-article-tag-suite</id>
7+ <link href="https://github.com/MartinPaulEve/JATS-CSL/blob/master/jats.csl" rel="self"/>
8+ <link rel="documentation" href="http://jats.nlm.nih.gov/archiving/tag-library/1.0/index.html"/>
9+ <author>
10+ <name>Martin Paul Eve</name>
11+ <email>martin@martineve.com</email>
12+ </author>
13+ <category citation-format="numeric"/>
14+ <category field="medicine"/>
15+ <category field="biology"/>
16+ <summary>Use this style to generate bibliographic data in Journal Article Tagging Suite (JATS) 1.0 XML format</summary>
17+ <updated>2014-06-21T17:41:26+00:00</updated>
18+ <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. Originally by Martin Fenner.</rights>
19+ </info>
20+ <locale xml:lang="en">
21+ <terms>
22+ <term name="et-al">{{jats}}&lt;etal/&gt;{{/jats}}</term>
23+ </terms>
24+ </locale>
25+ <macro name="citation-number">
26+ <text variable="citation-number" prefix="{{jats}}id=&quot;ref-{{/jats}}" suffix="{{jats}}&quot;&gt;{{/jats}}"/>
27+ </macro>
28+ <macro name="author">
29+ <names variable="author" delimiter=" ">
30+ <name prefix="{{jats}}&lt;name&gt;{{/jats}}" suffix="{{jats}}&lt;/name&gt;{{/jats}}" name-as-sort-order="all" sort-separator=" ">
31+ <name-part name="family" text-case="capitalize-first" prefix="{{jats}}&lt;surname&gt;{{/jats}}" suffix="{{jats}}&lt;/surname&gt;{{/jats}}"/>
32+ <name-part name="given" text-case="capitalize-first" prefix="{{jats}}&lt;given-names&gt;{{/jats}}" suffix="{{jats}}&lt;/given-names&gt;{{/jats}}"/>
33+ </name>
34+ <substitute>
35+ <names variable="editor"/>
36+ </substitute>
37+ </names>
38+ </macro>
39+
40+ <macro name="editor" delimiter=" ">
41+ <names variable="editor" prefix="{{jats}}&lt;person-group person-group-type=&quot;editor&quot;&gt;{{/jats}}" suffix="{{jats}}&lt;/person-group&gt;{{/jats}}">
42+ <name prefix="{{jats}}&lt;name&gt;{{/jats}}" suffix="{{jats}}&lt;/name&gt;{{/jats}}" name-as-sort-order="all" sort-separator=" ">
43+ <name-part name="family" text-case="capitalize-first" prefix="{{jats}}&lt;surname&gt;{{/jats}}" suffix="{{jats}}&lt;/surname&gt;{{/jats}}"/>
44+ <name-part name="given" text-case="capitalize-first" prefix="{{jats}}&lt;given-names&gt;{{/jats}}" suffix="{{jats}}&lt;/given-names&gt;{{/jats}}"/>
45+ </name>
46+ <substitute>
47+ <names variable="editor"/>
48+ </substitute>
49+ </names>
50+ </macro>
51+
52+ <macro name="editor">
53+ <group delimiter=": ">
54+ <names variable="editor">
55+ <name prefix="{{jats}}&lt;name&gt;{{/jats}}" suffix="{{jats}}&lt;/name&gt;{{/jats}}" name-as-sort-order="all" sort-separator="">
56+ <name-part name="family" text-case="capitalize-first" prefix="{{jats}}&lt;surname&gt;{{/jats}}" suffix="{{jats}}&lt;/surname&gt;{{/jats}}"/>
57+ <name-part name="given" text-case="capitalize-first" prefix="{{jats}}&lt;given-names&gt;{{/jats}}" suffix="{{jats}}&lt;given-names&gt;{{/jats}}"/>
58+ </name>
59+ </names>
60+ </group>
61+ </macro>
62+ <macro name="title">
63+ <choose>
64+ <if type="book" match="any">
65+ <group prefix="{{jats}}&lt;source&gt;{{/jats}}" suffix="{{jats}}&lt;/source&gt;{{/jats}}">
66+ <text variable="title"/>
67+ </group>
68+ </if>
69+ <else>
70+ <group prefix="{{jats}}&lt;article-title&gt;{{/jats}}" suffix="{{jats}}&lt;/article-title&gt;{{/jats}}">
71+ <text variable="title"/>
72+ </group>
73+ </else>
74+ </choose>
75+ </macro>
76+ <macro name="container-title">
77+ <text variable="container-title" form="short" prefix="{{jats}}&lt;source&gt;{{/jats}}" suffix="{{jats}}&lt;/source&gt;{{/jats}}"/>
78+ </macro>
79+ <macro name="publisher">
80+ <text variable="publisher" prefix="{{jats}}&lt;publisher-name&gt;{{/jats}}" suffix="{{jats}}&lt;/publisher-name&gt;{{/jats}}"/>
81+ <text variable="publisher-place" prefix="{{jats}}&lt;publisher-loc&gt;{{/jats}}" suffix="{{jats}}&lt;/publisher-loc&gt;{{/jats}}"/>
82+ </macro>
83+ <macro name="link">
84+ <choose>
85+ <if match="any" variable="DOI">
86+ <group prefix="{{jats}}&lt;pub-id pub-id-type=&quot;doi&quot;&gt;{{/jats}}" suffix="{{jats}}&lt;/pub-id&gt;{{/jats}}">
87+ <text variable="DOI"/>
88+ </group>
89+ </if>
90+ </choose>
91+ <choose>
92+ <if match="any" variable="PMID">
93+ <group prefix="{{jats}}&lt;pub-id pub-id-type=&quot;pmid&quot;&gt;{{/jats}}" suffix="{{jats}}&lt;/pub-id&gt;{{/jats}}">
94+ <text variable="PMID"/>
95+ </group>
96+ </if>
97+ </choose>
98+ <choose>
99+ <if variable="URL" match="any">
100+ <text variable="URL" />
101+ </if>
102+ </choose>
103+ </macro>
104+ <macro name="date">
105+ <choose>
106+ <if type="article-journal article-magazine article-newspaper report patent book" match="any">
107+ <group prefix="{{jats}}&lt;date&gt;{{/jats}}" suffix="{{jats}}&lt;/date&gt;{{/jats}}">
108+ <date variable="issued">
109+ <date-part name="day" form="numeric-leading-zeros" prefix="{{jats}}&lt;day&gt;{{/jats}}" suffix="{{jats}}&lt;/day&gt;{{/jats}}"/>
110+ <date-part name="month" form="numeric-leading-zeros" prefix="{{jats}}&lt;month&gt;{{/jats}}" suffix="{{jats}}&lt;/month&gt;{{/jats}}"/>
111+ <date-part name="year" prefix="{{jats}}&lt;year&gt;{{/jats}}" suffix="{{jats}}&lt;/year&gt;{{/jats}}"/>
112+ </date>
113+ </group>
114+ </if>
115+ <else>
116+ <group prefix="{{jats}}&lt;date-in-citation content-type=&quot;access-date&quot;{{/jats}}" suffix="{{jats}}&lt;/date-in-citation&gt;{{/jats}}">
117+ <date variable="accessed" prefix="{{jats}} iso-8601-date=&quot;{{/jats}}" suffix="{{jats}}&quot;&gt;{{/jats}}">
118+ <date-part name="year"/>
119+ <date-part name="month" form="numeric-leading-zeros" prefix="{{jats}}-{{/jats}}"/>
120+ <date-part name="day" form="numeric-leading-zeros" prefix="{{jats}}-{{/jats}}"/>
121+ </date>
122+ <date variable="accessed">
123+ <date-part name="day" prefix="{{jats}}&lt;day&gt;{{/jats}}" suffix="{{jats}}&lt;/day&gt;{{/jats}}"/>
124+ <date-part name="month" form="numeric-leading-zeros" prefix="{{jats}}&lt;month&gt;{{/jats}}" suffix="{{jats}}&lt;/month&gt;{{/jats}}"/>
125+ <date-part name="year" prefix="{{jats}}&lt;year&gt;{{/jats}}" suffix="{{jats}}&lt;/year&gt;{{/jats}}"/>
126+ </date>
127+ </group>
128+ </else>
129+ </choose>
130+ </macro>
131+ <macro name="location">
132+ <choose>
133+ <if type="article-journal article-magazine" match="any">
134+ <text variable="volume" prefix="{{jats}}&lt;volume&gt;{{/jats}}" suffix="{{jats}}&lt;/volume&gt;{{/jats}}"/>
135+ <text variable="issue" prefix="{{jats}}&lt;issue&gt;{{/jats}}" suffix="{{jats}}&lt;/issue&gt;{{/jats}}"/>
136+ </if>
137+ </choose>
138+ <choose>
139+ <if type="article-journal article-magazine article-newspaper chapter" match="any">
140+ <text variable="page-first" prefix="{{jats}}&lt;fpage&gt;{{/jats}}" suffix="{{jats}}&lt;/fpage&gt;{{/jats}}"/>
141+ </if>
142+ </choose>
143+ </macro>
144+ <macro name="publication-type">
145+ <group prefix="{{jats}} publication-type=&quot;{{/jats}}" suffix="{{jats}}&quot;&gt;{{/jats}}">
146+ <choose>
147+ <if type="article-journal article-magazine article-newspaper" match="any">
148+ <text value="journal"/>
149+ </if>
150+ <else-if type="book" match="any">
151+ <text value="book"/>
152+ </else-if>
153+ <else-if type="chapter" match="any">
154+ <text value="bookchapter"/>
155+ </else-if>
156+ <else-if type="dataset" match="any">
157+ <text value="dataset"/>
158+ </else-if>
159+ <else-if type="patent" match="any">
160+ <text value="patent"/>
161+ </else-if>
162+ <else-if type="report" match="any">
163+ <text value="report"/>
164+ </else-if>
165+ <else-if type="review" match="any">
166+ <text value="review"/>
167+ </else-if>
168+ <else>
169+ <text value="standard"/>
170+ </else>
171+ </choose>
172+ </group>
173+ </macro>
174+ <citation collapse="citation-number">
175+ <sort>
176+ <key variable="citation-number"/>
177+ </sort>
178+ <layout delimiter=",">
179+ <group prefix="{{jats}}&lt;xref ref-type=&quot;bibr&quot; rid=&quot;{{/jats}}" suffix="{{jats}}&lt;/xref&gt;{{/jats}}">
180+ <text variable="citation-number" prefix="{{jats}}ref-{{/jats}}" suffix="{{jats}}&quot;&gt;{{/jats}}"/>
181+ <text variable="citation-number"/>
182+ </group>
183+ </layout>
184+ </citation>
185+ <bibliography sort-separator="">
186+ <layout>
187+ <group prefix="{{jats}}&lt;ref {{/jats}}" suffix="{{jats}}&lt;/ref&gt;{{/jats}}">
188+ <text macro="citation-number"/>
189+ <group prefix="{{jats}}&lt;element-citation{{/jats}}" suffix="{{jats}}&lt;/element-citation&gt;{{/jats}}">
190+ <text macro="publication-type"/>
191+ <text macro="author" prefix="{{jats}}&lt;person-group person-group-type=&quot;author&quot;&gt;{{/jats}}" suffix="{{jats}}&lt;/person-group&gt;{{/jats}}"/>
192+ <text macro="title" />
193+ <text macro="container-title"/>
194+ <text macro="editor"/>
195+ <text macro="publisher"/>
196+ <text macro="date"/>
197+ <text macro="location"/>
198+ <text macro="link"/>
199+ </group>
200+ </group>
201+ </layout>
202+ </bibliography>
203+</style>
data/make-reference-files.hsnew file+27
@@ -0,0 +1,27 @@
1+import System.Environment
2+import System.Directory
3+import Codec.Archive.Zip
4+import qualified Data.ByteString.Lazy as BS
5+import qualified Control.Exception as E
6+import System.IO.Error (isDoesNotExistError)
7+import System.FilePath
8+
9+mkzip :: String -> IO ()
10+mkzip fmt = do
11+ let dir = "data" </> fmt
12+ output = "data" </> "reference" <.> fmt
13+ cd <- getCurrentDirectory
14+ setCurrentDirectory dir
15+ archive <- addFilesToArchive [OptRecursive] emptyArchive ["."]
16+ setCurrentDirectory cd
17+ removeIfExists output
18+ BS.writeFile output $ fromArchive archive
19+
20+removeIfExists :: FilePath -> IO ()
21+removeIfExists fileName = removeFile fileName `E.catch` handleExists
22+ where handleExists e
23+ | isDoesNotExistError e = return ()
24+ | otherwise = E.throwIO e
25+
26+main :: IO ()
27+main = getArgs >>= mkzip . (!!0)
data/odt/styles.xml+1−1
@@ -350,7 +350,7 @@ xmlns:css3t="http://www.w3.org/TR/css3-text/" office:version="1.2">
350350 </style:style>
351351 <style:style style:name="First_20_paragraph"
352352 style:display-name="First paragraph" style:family="paragraph"
353− style:parent-style-name="Text_20_body"
353+ style:parent-style-name="Standard"
354354 style:next-style-name="Text_20_body" style:class="text" />
355355 <style:style style:name="Numbering_20_Symbols"
356356 style:display-name="Numbering Symbols" style:family="text" />
data/templates/.github/ISSUE_TEMPLATE.mdnew file+3
@@ -0,0 +1,3 @@
1+This repository is now maintained as a subtree of
2+[jgm/pandoc](https://github.com/jgm/pandoc). Please open
3+your issue there, not here.
data/templates/.github/PULL_REQUEST_TEMPLATE.mdnew file+4
@@ -0,0 +1,4 @@
1+This repository is now maintained as a subtree of
2+[jgm/pandoc](https://github.com/jgm/pandoc). Please open
3+your pull request there, not here.
4+
data/templates/README.markdownnew file+49
@@ -0,0 +1,49 @@
1+The [pandoc-templates] repository contains default templates for [pandoc].
2+
3+If you use custom templates, we recommend forking this
4+repository, so that you can integrate changes to the default
5+templates in future pandoc releases.
6+
7+This repository is a subtree of [pandoc]; it will be updated
8+each time there is a new pandoc release. However, changes to
9+templates are made first in [pandoc]; thus, issues and pull
10+requests should be made [pandoc]'s issue tracker rather than
11+here.
12+
13+[pandoc]: https://github.com/jgm/pandoc
14+[pandoc-templates]: https://github.com/jgm/pandoc-templates
15+
16+All of the templates in this repository are dual licensed, under both
17+the GPL (v2 or higher, same as pandoc) and the BSD 3-clause license
18+(included below).
19+
20+Copyright (c) 2014--2019, John MacFarlane
21+
22+All rights reserved.
23+
24+Redistribution and use in source and binary forms, with or without
25+modification, are permitted provided that the following conditions are met:
26+
27+* Redistributions of source code must retain the above copyright
28+ notice, this list of conditions and the following disclaimer.
29+
30+* Redistributions in binary form must reproduce the above
31+ copyright notice, this list of conditions and the following
32+ disclaimer in the documentation and/or other materials provided
33+ with the distribution.
34+
35+* Neither the name of John MacFarlane nor the names of other
36+ contributors may be used to endorse or promote products derived
37+ from this software without specific prior written permission.
38+
39+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
40+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
41+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
42+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
43+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
44+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
45+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
46+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
47+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
48+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
49+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
data/templates/article.jats_publishing+4−6
@@ -30,10 +30,10 @@ $if(journal.abbrev-title)$
3030$endif$
3131</journal-title-group>
3232$if(journal.pissn)$
33−<issn publication-format="print">$journal.pissn$</issn>
33+<issn pub-type="ppub">$journal.pissn$</issn>
3434$endif$
3535$if(journal.eissn)$
36−<issn publication-format="electronic">$journal.eissn$</issn>
36+<issn pub-type="epub">$journal.eissn$</issn>
3737$endif$
3838$-- At least one issn element is required; use empty issn as fallback
3939$if(journal.pissn)$
@@ -86,7 +86,7 @@ $endif$
8686$if(author)$
8787<contrib-group>
8888$for(author)$
89−<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
89+<contrib contrib-type="author">
9090$if(author.orcid)$
9191<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
9292$endif$
@@ -95,8 +95,6 @@ $if(author.surname)$
9595<surname>$author.surname$</surname>
9696<given-names>$author.given-names$</given-names>
9797</name>
98−$elseif(author.name)$
99−<string-name>$author.name$</string-name>
10098$else$
10199<string-name>$author$</string-name>
102100$endif$
@@ -130,7 +128,7 @@ $endif$
130128</author-notes>
131129$endif$
132130$if(date)$
133−<pub-date date-type="$if(date.type)$$date.type$$else$pub$endif$" publication-format="electronic"$if(date.iso-8601)$ iso-8601-date="$date.iso-8601$"$endif$>
131+<pub-date pub-type="epub"$if(date.iso-8601)$ iso-8601-date="$date.iso-8601$"$endif$>
134132$if(date.day)$
135133<day>$date.day$</day>
136134$endif$
data/templates/default.biblatexdeleted−10
@@ -1,10 +0,0 @@
1−$for(header-includes)$
2−$header-includes$
3−$endfor$
4−$for(include-before)$
5−$include-before$
6−$endfor$
7−$body$
8−$for(include-after)$
9−$include-after$
10−$endfor$
data/templates/default.bibtexdeleted−10
@@ -1,10 +0,0 @@
1−$for(header-includes)$
2−$header-includes$
3−$endfor$
4−$for(include-before)$
5−$include-before$
6−$endfor$
7−$body$
8−$for(include-after)$
9−$include-after$
10−$endfor$
data/templates/default.context+5−7
@@ -51,7 +51,6 @@ $endif$
5151
5252% use microtypography
5353\definefontfeature[default][default][script=latn, protrusion=quality, expansion=quality, itlc=yes, textitalics=yes, onum=yes, pnum=yes]
54−\definefontfeature[default:tnum][default][tnum=yes, pnum=no]
5554\definefontfeature[smallcaps][script=latn, protrusion=quality, expansion=quality, smcp=yes, onum=yes, pnum=yes]
5655\setupalign[hz,hanging]
5756\setupitaliccorrection[global, always]
@@ -98,12 +97,11 @@ $endif$
9897\setupitemize[autointro] % prevent orphan list intro
9998\setupitemize[indentnext=no]
10099
101−\defineitemgroup[enumerate]
102−\setupenumerate[each][fit][itemalign=left,distance=.5em,style={\feature[+][default:tnum]}]
103−
104100\setupfloat[figure][default={here,nonumber}]
105101\setupfloat[table][default={here,nonumber}]
106102
103+\setupthinrules[width=15em] % width of horizontal rules
104+
107105\setupxtable[frame=off]
108106\setupxtable[head][topframe=on,bottomframe=on]
109107\setupxtable[body][]
@@ -164,12 +162,12 @@ $endfor$
164162$if(toc)$
165163\completecontent
166164$endif$
167−$if(lof)$
168−\completelistoffigures
169−$endif$
170165$if(lot)$
171166\completelistoftables
172167$endif$
168+$if(lof)$
169+\completelistoffigures
170+$endif$
173171
174172$body$
175173
data/templates/default.dzslides+1−1
@@ -171,7 +171,7 @@ $if(subtitle)$
171171 <h1 class="subtitle">$subtitle$</h1>
172172$endif$
173173 <footer>
174− $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span> · $endif$$if(institute)$<span class="institute">$for(institute)$$institute$$sep$, $endfor$</span> · $endif$$if(date)$<span class="date">$date$</span>$endif$
174+ $if(author)$<span class="author">$for(author)$$author$$sep$, $endfor$</span>$endif$ · $if(date)$<span class="date">$date$</span>$endif$
175175 </footer>
176176</section>
177177$endif$
data/templates/default.epub2+1−1
@@ -48,7 +48,7 @@ $endif$
4848$else$
4949$if(coverpage)$
5050<div id="cover-image">
51−<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="100%" height="100%" viewBox="0 0 $cover-image-width$ $cover-image-height$" preserveAspectRatio="xMidYMid">
51+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="100%" height="100%" viewBox="0 0 $cover-image-width$ $cover-image-height$" preserveAspectRatio="none">
5252<image width="$cover-image-width$" height="$cover-image-height$" xlink:href="../media/$cover-image$" />
5353</svg>
5454</div>
data/templates/default.epub3+1−1
@@ -49,7 +49,7 @@ $endif$
4949$else$
5050$if(coverpage)$
5151<div id="cover-image">
52−<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="100%" height="100%" viewBox="0 0 $cover-image-width$ $cover-image-height$" preserveAspectRatio="xMidYMid">
52+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="100%" height="100%" viewBox="0 0 $cover-image-width$ $cover-image-height$" preserveAspectRatio="none">
5353<image width="$cover-image-width$" height="$cover-image-height$" xlink:href="../media/$cover-image$" />
5454</svg>
5555</div>
data/templates/default.html4−3
@@ -12,9 +12,6 @@ $if(date-meta)$
1212$endif$
1313$if(keywords)$
1414 <meta name="keywords" content="$for(keywords)$$keywords$$sep$, $endfor$" />
15−$endif$
16−$if(description-meta)$
17− <meta name="description" content="$description-meta$" />
1815$endif$
1916 <title>$if(title-prefix)$$title-prefix$ – $endif$$pagetitle$</title>
2017 <style type="text/css">
data/templates/default.html5−3
@@ -12,9 +12,6 @@ $if(date-meta)$
1212$endif$
1313$if(keywords)$
1414 <meta name="keywords" content="$for(keywords)$$keywords$$sep$, $endfor$" />
15−$endif$
16−$if(description-meta)$
17− <meta name="description" content="$description-meta$" />
1815$endif$
1916 <title>$if(title-prefix)$$title-prefix$ – $endif$$pagetitle$</title>
2017 <style>
data/templates/default.jats_articleauthoring+1−3
@@ -19,7 +19,7 @@ $endif$
1919$if(author)$
2020<contrib-group>
2121$for(author)$
22−<contrib contrib-type="author"$if(author.equal-contrib)$ equal-contrib="true"$endif$>
22+<contrib contrib-type="author">
2323$if(author.orcid)$
2424<contrib-id contrib-id-type="orcid">$author.orcid$</contrib-id>
2525$endif$
@@ -28,8 +28,6 @@ $if(author.surname)$
2828<surname>$author.surname$</surname>
2929<given-names>$author.given-names$</given-names>
3030</name>
31−$elseif(author.name)$
32−<string-name>$author.name$</string-name>
3331$else$
3432<string-name>$author$</string-name>
3533$endif$
data/templates/default.latex+40−71
@@ -2,7 +2,7 @@
22\PassOptionsToPackage{unicode$for(hyperrefoptions)$,$hyperrefoptions$$endfor$}{hyperref}
33\PassOptionsToPackage{hyphens}{url}
44$if(colorlinks)$
5−\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}
5+\PassOptionsToPackage{dvipsnames,svgnames*,x11names*}{xcolor}
66$endif$
77$if(dir)$
88$if(latex-dir-rtl)$
@@ -98,14 +98,14 @@ $endif$
9898$if(linestretch)$
9999\usepackage{setspace}
100100$endif$
101−\usepackage{iftex}
102−\ifPDFTeX
101+\usepackage{ifxetex,ifluatex}
102+\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
103103 \usepackage[$if(fontenc)$$fontenc$$else$T1$endif$]{fontenc}
104104 \usepackage[utf8]{inputenc}
105105 \usepackage{textcomp} % provide euro and other symbols
106106\else % if luatex or xetex
107107$if(mathspec)$
108− \ifXeTeX
108+ \ifxetex
109109 \usepackage{mathspec}
110110 \else
111111 \usepackage{unicode-math}
@@ -129,7 +129,7 @@ $for(fontfamilies)$
129129$endfor$
130130$if(mathfont)$
131131$if(mathspec)$
132− \ifXeTeX
132+ \ifxetex
133133 \setmathfont(Digits,Latin,Greek)[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}
134134 \else
135135 \setmathfont[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}
@@ -139,42 +139,23 @@ $else$
139139$endif$
140140$endif$
141141$if(CJKmainfont)$
142− \ifXeTeX
142+ \ifxetex
143143 \usepackage{xeCJK}
144144 \setCJKmainfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}
145145 \fi
146146$endif$
147147$if(luatexjapresetoptions)$
148− \ifLuaTeX
148+ \ifluatex
149149 \usepackage[$for(luatexjapresetoptions)$$luatexjapresetoptions$$sep$,$endfor$]{luatexja-preset}
150150 \fi
151151$endif$
152152$if(CJKmainfont)$
153− \ifLuaTeX
153+ \ifluatex
154154 \usepackage[$for(luatexjafontspecoptions)$$luatexjafontspecoptions$$sep$,$endfor$]{luatexja-fontspec}
155155 \setmainjfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}
156156 \fi
157157$endif$
158158\fi
159−$if(zero-width-non-joiner)$
160−%% Support for zero-width non-joiner characters.
161−\makeatletter
162−\def\zerowidthnonjoiner{%
163− % Prevent ligatures and adjust kerning, but still support hyphenating.
164− \texorpdfstring{%
165− \textormath{\nobreak\discretionary{-}{}{\kern.03em}%
166− \ifvmode\else\nobreak\hskip\z@skip\fi}{}%
167− }{}%
168−}
169−\makeatother
170−\ifPDFTeX
171− \DeclareUnicodeCharacter{200C}{\zerowidthnonjoiner}
172−\else
173− \catcode`^^^^200c=\active
174− \protected\def ^^^^200c{\zerowidthnonjoiner}
175−\fi
176−%% End of ZWNJ support
177−$endif$
178159$if(beamer)$
179160$if(theme)$
180161\usetheme[$for(themeoptions)$$themeoptions$$sep$,$endfor$]{$theme$}
@@ -238,10 +219,10 @@ $if(keywords)$
238219$endif$
239220$if(colorlinks)$
240221 colorlinks=true,
241− linkcolor={$if(linkcolor)$$linkcolor$$else$Maroon$endif$},
242− filecolor={$if(filecolor)$$filecolor$$else$Maroon$endif$},
243− citecolor={$if(citecolor)$$citecolor$$else$Blue$endif$},
244− urlcolor={$if(urlcolor)$$urlcolor$$else$Blue$endif$},
222+ linkcolor=$if(linkcolor)$$linkcolor$$else$Maroon$endif$,
223+ filecolor=$if(filecolor)$$filecolor$$else$Maroon$endif$,
224+ citecolor=$if(citecolor)$$citecolor$$else$Blue$endif$,
225+ urlcolor=$if(urlcolor)$$urlcolor$$else$Blue$endif$,
245226$else$
246227 hidelinks,
247228$endif$
@@ -274,9 +255,6 @@ $highlighting-macros$
274255$endif$
275256$if(tables)$
276257\usepackage{longtable,booktabs,array}
277−$if(multirow)$
278−\usepackage{multirow}
279−$endif$
280258\usepackage{calc} % for calculating minipage widths
281259$if(beamer)$
282260\usepackage{caption}
@@ -344,36 +322,11 @@ $endif$
344322$if(pagestyle)$
345323\pagestyle{$pagestyle$}
346324$endif$
347−$if(csl-refs)$
348−\newlength{\cslhangindent}
349−\setlength{\cslhangindent}{1.5em}
350−\newlength{\csllabelwidth}
351−\setlength{\csllabelwidth}{3em}
352−\newlength{\cslentryspacingunit} % times entry-spacing
353−\setlength{\cslentryspacingunit}{\parskip}
354−\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
355− {% don't indent paragraphs
356− \setlength{\parindent}{0pt}
357− % turn on hanging indent if param 1 is 1
358− \ifodd #1
359− \let\oldpar\par
360− \def\par{\hangindent=\cslhangindent\oldpar}
361− \fi
362− % set entry spacing
363− \setlength{\parskip}{#2\cslentryspacingunit}
364− }%
365− {}
366−\usepackage{calc}
367−\newcommand{\CSLBlock}[1]{#1\hfill\break}
368−\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
369−\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
370−\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
371−$endif$
372325$for(header-includes)$
373326$header-includes$
374327$endfor$
375328$if(lang)$
376−\ifXeTeX
329+\ifxetex
377330 % Load polyglossia as late as possible: uses bidi with RTL langages (e.g. Hebrew, Arabic)
378331 \usepackage{polyglossia}
379332 \setmainlanguage[$for(polyglossia-lang.options)$$polyglossia-lang.options$$sep$,$endfor$]{$polyglossia-lang.name$}
@@ -390,15 +343,15 @@ $if(babel-newcommands)$
390343$endif$
391344\fi
392345$endif$
393−\ifLuaTeX
346+\ifluatex
394347 \usepackage{selnolig} % disable illegal ligatures
395348\fi
396349$if(dir)$
397−\ifXeTeX
350+\ifxetex
398351 % Load bidi as late as possible as it modifies e.g. graphicx
399352 \usepackage{bidi}
400353\fi
401−\ifPDFTeX
354+\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
402355 \TeXXeTstate=1
403356 \newcommand{\RL}[1]{\beginR #1\endR}
404357 \newcommand{\LR}[1]{\beginL #1\endL}
@@ -416,11 +369,27 @@ $for(bibliography)$
416369\addbibresource{$bibliography$}
417370$endfor$
418371$endif$
419−$if(nocite-ids)$
420−\nocite{$for(nocite-ids)$$it$$sep$, $endfor$}
421−$endif$
422−$if(csquotes)$
423−\usepackage{csquotes}
372+$if(csl-refs)$
373+\newlength{\cslhangindent}
374+\setlength{\cslhangindent}{1.5em}
375+\newlength{\csllabelwidth}
376+\setlength{\csllabelwidth}{3em}
377+\newenvironment{CSLReferences}[2] % #1 hanging-ident, #2 entry spacing
378+ {% don't indent paragraphs
379+ \setlength{\parindent}{0pt}
380+ % turn on hanging indent if param 1 is 1
381+ \ifodd #1 \everypar{\setlength{\hangindent}{\cslhangindent}}\ignorespaces\fi
382+ % set entry spacing
383+ \ifnum #2 > 0
384+ \setlength{\parskip}{#2\baselineskip}
385+ \fi
386+ }%
387+ {}
388+\usepackage{calc}
389+\newcommand{\CSLBlock}[1]{#1\hfill\break}
390+\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}}
391+\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}\break}
392+\newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1}
424393$endif$
425394
426395$if(title)$
@@ -494,12 +463,12 @@ $endif$
494463}
495464$endif$
496465$endif$
497−$if(lof)$
498−\listoffigures
499−$endif$
500466$if(lot)$
501467\listoftables
502468$endif$
469+$if(lof)$
470+\listoffigures
471+$endif$
503472$if(linestretch)$
504473\setstretch{$linestretch$}
505474$endif$
data/templates/default.revealjs+125−79
@@ -52,9 +52,6 @@ $endif$
5252$for(author)$
5353 <p class="author">$author$</p>
5454$endfor$
55−$for(institute)$
56− <p class="institute">$institute$</p>
57−$endfor$
5855$if(date)$
5956 <p class="date">$date$</p>
6057$endif$
@@ -62,12 +59,7 @@ $endif$
6259$endif$
6360$if(toc)$
6461<section id="$idprefix$TOC">
65−<nav role="doc-toc">
66−$if(toc-title)$
67−<h2 id="$idprefix$toc-title">$toc-title$</h2>
68−$endif$
6962$table-of-contents$
70−</nav>
7163</section>
7264$endif$
7365
@@ -77,7 +69,7 @@ $body$
7769
7870 <script src="$revealjs-url$/dist/reveal.js"></script>
7971
80− <!-- reveal.js plugins -->
72+ // reveal.js plugins
8173 <script src="$revealjs-url$/plugin/notes/notes.js"></script>
8274 <script src="$revealjs-url$/plugin/search/search.js"></script>
8375 <script src="$revealjs-url$/plugin/zoom/zoom.js"></script>
@@ -90,198 +82,252 @@ $endif$
9082 // Full list of configuration options available at:
9183 // https://revealjs.com/config/
9284 Reveal.initialize({
85+$if(controls)$
9386 // Display controls in the bottom right corner
9487 controls: $controls$,
95−
88+$endif$
89+$if(controlsTutorial)$
9690 // Help the user learn the controls by providing hints, for example by
9791 // bouncing the down arrow when they first encounter a vertical slide
9892 controlsTutorial: $controlsTutorial$,
99−
93+$endif$
94+$if(controlsLayout)$
10095 // Determines where controls appear, "edges" or "bottom-right"
10196 controlsLayout: '$controlsLayout$',
102−
97+$endif$
98+$if(controlsBackArrows)$
10399 // Visibility rule for backwards navigation arrows; "faded", "hidden"
104100 // or "visible"
105101 controlsBackArrows: '$controlsBackArrows$',
106−
102+$endif$
103+$if(progress)$
107104 // Display a presentation progress bar
108105 progress: $progress$,
109−
106+$endif$
107+$if(slideNumber)$
110108 // Display the page number of the current slide
111109 slideNumber: $slideNumber$,
112−
110+$endif$
111+$if(showSlideNumber)$
113112 // 'all', 'print', or 'speaker'
114113 showSlideNumber: '$showSlideNumber$',
115−
114+$endif$
115+$if(hash)$
116116 // Add the current slide number to the URL hash so that reloading the
117117 // page/copying the URL will return you to the same slide
118118 hash: $hash$,
119−
119+$endif$
120+$if(hashOneBasedIndex)$
120121 // Start with 1 for the hash rather than 0
121122 hashOneBasedIndex: $hashOneBasedIndex$,
122−
123− // Flags if we should monitor the hash and change slides accordingly
124− respondToHashChanges: $respondToHashChanges$,
125−
123+$endif$
126124 // Push each slide change to the browser history
127− history: $history$,
128−
125+ history: $if(history)$$history$$else$true$endif$,
126+$if(keyboard)$
129127 // Enable keyboard shortcuts for navigation
130128 keyboard: $keyboard$,
131−
129+$endif$
130+$if(overview)$
132131 // Enable the slide overview mode
133132 overview: $overview$,
134−
135− // Disables the default reveal.js slide layout (scaling and centering)
136− // so that you can use custom CSS layout
137− disableLayout: false,
138−
133+$endif$
134+$if(center)$
139135 // Vertical centering of slides
140136 center: $center$,
141−
137+$endif$
138+$if(touch)$
142139 // Enables touch navigation on devices with touch input
143140 touch: $touch$,
144−
141+$endif$
142+$if(loop)$
145143 // Loop the presentation
146144 loop: $loop$,
147−
145+$endif$
146+$if(rtl)$
148147 // Change the presentation direction to be RTL
149148 rtl: $rtl$,
150−
149+$endif$
150+$if(navigationMode)$
151151 // see https://revealjs.com/vertical-slides/#navigation-mode
152152 navigationMode: '$navigationMode$',
153−
153+$endif$
154+$if(shuffle)$
154155 // Randomizes the order of slides each time the presentation loads
155156 shuffle: $shuffle$,
156−
157+$endif$
158+$if(fragments)$
157159 // Turns fragments on and off globally
158160 fragments: $fragments$,
159−
161+$endif$
162+$if(fragmentInURL)$
160163 // Flags whether to include the current fragment in the URL,
161164 // so that reloading brings you to the same fragment position
162165 fragmentInURL: $fragmentInURL$,
163−
166+$endif$
167+$if(embedded)$
164168 // Flags if the presentation is running in an embedded mode,
165169 // i.e. contained within a limited portion of the screen
166170 embedded: $embedded$,
167−
171+$endif$
172+$if(help)$
168173 // Flags if we should show a help overlay when the questionmark
169174 // key is pressed
170175 help: $help$,
171−
172− // Flags if it should be possible to pause the presentation (blackout)
173− pause: $pause$,
174−
176+$endif$
177+$if(pause)$
178+ // Flags if it should be possible to pause the presentation (blackout)
179+ pause: $pause$,
180+$endif$
181+$if(showNotes)$
175182 // Flags if speaker notes should be visible to all viewers
176183 showNotes: $showNotes$,
177−
178− // Global override for autoplaying embedded media (null/true/false)
184+$endif$
185+$if(autoPlayMedia)$
186+ // Global override for autoplaying embedded media (video/audio/iframe)
187+ // - null: Media will only autoplay if data-autoplay is present
188+ // - true: All media will autoplay, regardless of individual setting
189+ // - false: No media will autoplay, regardless of individual setting
179190 autoPlayMedia: $autoPlayMedia$,
180−
181− // Global override for preloading lazy-loaded iframes (null/true/false)
191+$endif$
192+$if(preloadIframes)$
193+ // Global override for preloading lazy-loaded iframes
194+ // - null: Iframes with data-src AND data-preload will be loaded when within
195+ // the viewDistance, iframes with only data-src will be loaded when visible
196+ // - true: All iframes with data-src will be loaded when within the viewDistance
197+ // - false: All iframes with data-src will be loaded only when visible
182198 preloadIframes: $preloadIframes$,
183−
199+$endif$
200+$if(autoSlide)$
184201 // Number of milliseconds between automatically proceeding to the
185202 // next slide, disabled when set to 0, this value can be overwritten
186203 // by using a data-autoslide attribute on your slides
187204 autoSlide: $autoSlide$,
188−
205+$endif$
206+$if(autoSlideStoppable)$
189207 // Stop auto-sliding after user input
190208 autoSlideStoppable: $autoSlideStoppable$,
191−
209+$endif$
210+$if(autoSlideMethod)$
192211 // Use this method for navigation when auto-sliding
193212 autoSlideMethod: $autoSlideMethod$,
194−
213+$endif$
214+$if(defaultTiming)$
195215 // Specify the average time in seconds that you think you will spend
196216 // presenting each slide. This is used to show a pacing timer in the
197217 // speaker view
198218 defaultTiming: $defaultTiming$,
199−
219+$endif$
220+$if(totalTime)$
221+ // Specify the total time in seconds that is available to
222+ // present. If this is set to a nonzero value, the pacing
223+ // timer will work out the time available for each slide,
224+ // instead of using the defaultTiming value
225+ totalTime: $totalTime$,
226+$endif$
227+$if(minimumTimePerSlide)$
228+ // Specify the minimum amount of time you want to allot to
229+ // each slide, if using the totalTime calculation method. If
230+ // the automated time allocation causes slide pacing to fall
231+ // below this threshold, then you will see an alert in the
232+ // speaker notes window
233+ minimumTimePerSlide: $minimumTimePerSlide$,
234+$endif$
235+$if(mouseWheel)$
200236 // Enable slide navigation via mouse wheel
201237 mouseWheel: $mouseWheel$,
202−
203− // The display mode that will be used to show slides
204− display: '$display$',
205−
238+$endif$
239+$if(rollingLinks)$
240+ // Apply a 3D roll to links on hover
241+ rollingLinks: $rollingLinks$,
242+$endif$
243+$if(hideInactiveCursor)$
206244 // Hide cursor if inactive
207245 hideInactiveCursor: $hideInactiveCursor$,
208−
246+$endif$
247+$if(hideCursorTime)$
209248 // Time before the cursor is hidden (in ms)
210249 hideCursorTime: $hideCursorTime$,
211−
250+$endif$
251+$if(hideAddressBar)$
252+ // Hides the address bar on mobile devices
253+ hideAddressBar: $hideAddressBar$,
254+$endif$
255+$if(previewLinks)$
212256 // Opens links in an iframe preview overlay
213257 previewLinks: $previewLinks$,
214−
215− // Transition style (none/fade/slide/convex/concave/zoom)
216− transition: '$transition$',
217−
218− // Transition speed (default/fast/slow)
219− transitionSpeed: '$transitionSpeed$',
220−
258+$endif$
259+$if(transition)$
260+ // Transition style
261+ transition: '$transition$', // none/fade/slide/convex/concave/zoom
262+$endif$
263+$if(transitionSpeed)$
264+ // Transition speed
265+ transitionSpeed: '$transitionSpeed$', // default/fast/slow
266+$endif$
267+$if(backgroundTransition)$
221268 // Transition style for full page slide backgrounds
222− // (none/fade/slide/convex/concave/zoom)
223− backgroundTransition: '$backgroundTransition$',
224−
269+ backgroundTransition: '$backgroundTransition$', // none/fade/slide/convex/concave/zoom
270+$endif$
271+$if(viewDistance)$
225272 // Number of slides away from the current that are visible
226273 viewDistance: $viewDistance$,
227−
274+$endif$
275+$if(mobileViewDistance)$
228276 // Number of slides away from the current that are visible on mobile
229277 // devices. It is advisable to set this to a lower number than
230278 // viewDistance in order to save resources.
231279 mobileViewDistance: $mobileViewDistance$,
280+$endif$
232281$if(parallaxBackgroundImage)$
233−
234282 // Parallax background image
235283 parallaxBackgroundImage: '$parallaxBackgroundImage$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
236284$else$
237285$if(background-image)$
238−
239286 // Parallax background image
240287 parallaxBackgroundImage: '$background-image$', // e.g. "'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'"
241288$endif$
242289$endif$
243290$if(parallaxBackgroundSize)$
244−
245291 // Parallax background size
246292 parallaxBackgroundSize: '$parallaxBackgroundSize$', // CSS syntax, e.g. "2100px 900px"
247293$endif$
248294$if(parallaxBackgroundHorizontal)$
249−
250295 // Amount to move parallax background (horizontal and vertical) on slide change
251296 // Number, e.g. 100
252297 parallaxBackgroundHorizontal: $parallaxBackgroundHorizontal$,
253298$endif$
254299$if(parallaxBackgroundVertical)$
255−
256300 parallaxBackgroundVertical: $parallaxBackgroundVertical$,
257301$endif$
258302$if(width)$
259−
260303 // The "normal" size of the presentation, aspect ratio will be preserved
261304 // when the presentation is scaled to fit different resolutions. Can be
262305 // specified using percentage units.
263306 width: $width$,
264307$endif$
265308$if(height)$
266−
267309 height: $height$,
268310$endif$
269311$if(margin)$
270−
271312 // Factor of the display size that should remain empty around the content
272313 margin: $margin$,
273314$endif$
274315$if(minScale)$
275−
276316 // Bounds for smallest/largest possible scale to apply to content
277317 minScale: $minScale$,
278318$endif$
279319$if(maxScale)$
280−
281320 maxScale: $maxScale$,
282321$endif$
322+$if(zoomKey)$
323+ // Modifier key used to click-zoom to part of the slide
324+ zoomKey: '$zoomKey$',
325+$endif$
326+$if(display)$
327+ // The display mode that will be used to show slides
328+ display: '$display$',
329+$endif$
283330$if(mathjax)$
284−
285331 math: {
286332 mathjax: '$mathjaxurl$',
287333 config: 'TeX-AMS_HTML-full',
data/templates/default.s5−3
@@ -61,9 +61,6 @@ $endif$
6161$if(author)$
6262 <h3 class="author">$for(author)$$author$$sep$<br/>$endfor$</h3>
6363$endif$
64−$if(institute)$
65− <h3 class="institute">$for(institute)$$institute$$sep$<br/>$endfor$</h3>
66−$endif$
6764$if(date)$
6865 <h4 class="date">$date$</h4>
6966$endif$
data/templates/default.slideous−7
@@ -63,16 +63,9 @@ $if(title)$
6363$if(subtitle)$
6464 <h1 class="subtitle">$subtitle$</h1>
6565$endif$
66−$if(author)$
6766 <p class="author">
6867$for(author)$$author$$sep$<br/>$endfor$
6968 </p>
70−$endif$
71−$if(institute)$
72− <p class="institute">
73−$for(institute)$$institute$$sep$<br/>$endfor$
74− </p>
75−$endif$
7669$if(date)$
7770 <p class="date">$date$</p>
7871$endif$
data/templates/default.slidy−5
@@ -52,11 +52,6 @@ $if(author)$
5252$for(author)$$author$$sep$<br/>$endfor$
5353 </p>
5454$endif$
55−$if(institute)$
56− <p class="institute">
57−$for(institute)$$institute$$sep$<br/>$endfor$
58− </p>
59−$endif$
6055$if(date)$
6156 <p class="date">$date$</p>
6257$endif$
data/templates/styles.html+1−2
@@ -14,7 +14,7 @@ body {
1414 padding-top: $if(margin-top)$$margin-top$$else$50px$endif$;
1515 padding-bottom: $if(margin-bottom)$$margin-bottom$$else$50px$endif$;
1616 hyphens: auto;
17− overflow-wrap: break-word;
17+ word-wrap: break-word;
1818 text-rendering: optimizeLegibility;
1919 font-kerning: normal;
2020}
@@ -93,7 +93,6 @@ $endif$
9393pre code {
9494 padding: 0;
9595 overflow: visible;
96− overflow-wrap: normal;
9796}
9897.sourceCode {
9998 background-color: transparent;
data/translations/zh-Hans.yamldeleted−22
@@ -1,22 +0,0 @@
1−Abstract: 摘要
2−Appendix: 附录
3−Bibliography: 文献目录
4−Cc: 副本
5−Chapter: 章
6−Contents: 目录
7−Encl: 附件
8−Figure: 图
9−Glossary: 术语
10−Index: 索引
11−Listing: 列表
12−ListOfFigures: 附图目录
13−ListOfTables: 表格索引
14−Page: 页
15−Part: 段
16−Preface: 序
17−Proof: 校对
18−References: 参考文献
19−See: 见
20−SeeAlso: 参见
21−Table: 表
22−To: 到
data/translations/zh-Hant.yamldeleted−22
@@ -1,22 +0,0 @@
1−Abstract: 摘要
2−Appendix: 附錄
3−Bibliography: 文獻目錄
4−Cc: 副本
5−Chapter: 章
6−Contents: 目錄
7−Encl: 附件
8−Figure: 圖
9−Glossary: 術語
10−Index: 索引
11−Listing: 列表
12−ListOfFigures: 附圖目錄
13−ListOfTables: 表格索引
14−Page: 頁
15−Part: 段
16−Preface: 序
17−Proof: 校對
18−References: 參考文獻
19−See: 見
20−SeeAlso: 參見
21−Table: 表
22−To: 到
data/vimwiki.cssnew file+82
@@ -0,0 +1,82 @@
1+pre{
2+ font-size: 1.5em
3+}
4+
5+img {vertical-align: middle}
6+
7+body {font-family: Tahoma, Geneva, sans-serif; margin: 1em 2em 1em 2em; font-size: 120%; line-height: 130%;}
8+h1, h2, h3, h4, h5, h6 {font-family: Trebuchet MS, Helvetica, sans-serif; font-weight: bold; line-height:100%; margin-top: 1.5em; margin-bottom: 0.5em;}
9+h1 {font-size: 2.6em; color: #000000;}
10+h2 {font-size: 2.2em; color: #404040;}
11+h3 {font-size: 1.8em; color: #707070;}
12+h4 {font-size: 1.4em; color: #909090;}
13+h5 {font-size: 1.3em; color: #989898;}
14+h6 {font-size: 1.2em; color: #9c9c9c;}
15+p, pre, blockquote, table, ul, ol, dl {margin-top: 1em; margin-bottom: 1em;}
16+ul ul, ul ol, ol ol, ol ul {margin-top: 0.5em; margin-bottom: 0.5em;}
17+li {margin: 0.3em auto;}
18+ul {margin-left: 2em; padding-left: 0.5em;}
19+dt {font-weight: bold;}
20+img {border: none;}
21+pre {border-left: 1px solid #ccc; margin-left: 2em; padding-left: 0.5em;}
22+blockquote {padding: 0.4em; background-color: #f6f5eb;}
23+th, td {border: 1px solid #ccc; padding: 0.3em;}
24+th {background-color: #f0f0f0;}
25+hr {border: none; border-top: 1px solid #ccc; width: 100%;}
26+del {text-decoration: line-through; color: #777777;}
27+.toc li {list-style-type: none;}
28+.todo {font-weight: bold; background-color: #f0ece8; color: #a03020;}
29+.justleft {text-align: left;}
30+.justright {text-align: right;}
31+.justcenter {text-align: center;}
32+.center {margin-left: auto; margin-right: auto;}
33+div.center > table {margin-left: auto; margin-right: auto;}
34+.tag {background-color: #eeeeee; font-family: monospace; padding: 2px;}
35+
36+/* classes for items of todo lists */
37+.done0 {
38+ /* list-style: none; */
39+ background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA8AAAAPCAYAAAA71pVKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAAxQAAAMUBHc26qAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAA7SURBVCiR7dMxEgAgCANBI3yVRzF5KxNbW6wsuH7LQ2YKQK1mkswBVERYF5Os3UV3gwd/jF2SkXy66gAZkxS6BniubAAAAABJRU5ErkJggg==);
40+ background-repeat: no-repeat;
41+ background-position: 0 .2em;
42+ padding-left: 1.5em;
43+}
44+.done1 {
45+ background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA8AAAAPCAYAAAA71pVKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAAxQAAAMUBHc26qAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAABtSURBVCiR1ZO7DYAwDER9BDmTeZQMFXmUbGYpOjrEryA0wOvO8itOslFrJYAug5BMM4BeSkmjsrv3aVTa8p48Xw1JSkSsWVUFwD05IqS1tmYzk5zzae9jnVVVzGyXb8sALjse+euRkEzu/uirFomVIdDGOLjuAAAAAElFTkSuQmCC);
46+ background-repeat: no-repeat;
47+ background-position: 0 .15em;
48+ padding-left: 1.5em;
49+}
50+.done2 {
51+ background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA8AAAAPCAYAAAA71pVKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAAxQAAAMUBHc26qAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAB1SURBVCiRzdO5DcAgDAVQGxjAYgTvxlDIu1FTIRYAp8qlFISkSH7l5kk+ZIwxKiI2mIyqWoeILYRgZ7GINDOLjnmF3VqklKCUMgTee2DmM661Qs55iI3Zm/1u5h9sm4ig9z4ERHTFzLyd4G4+nFlVrYg8+qoF/c0kdpeMsmcAAAAASUVORK5CYII=);
52+ background-repeat: no-repeat;
53+ background-position: 0 .15em;
54+ padding-left: 1.5em;
55+}
56+.done3 {
57+ background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA8AAAAPCAYAAAA71pVKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAAxQAAAMUBHc26qAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAABoSURBVCiR7dOxDcAgDATA/0DtUdiKoZC3YhLkHjkVKF3idJHiztKfvrHZWnOSE8Fx95RJzlprimJVnXktvXeY2S0SEZRSAAAbmxnGGKH2I5T+8VfxPhIReQSuuY3XyYWa3T2p6quvOgGrvSFGlewuUAAAAABJRU5ErkJggg==);
58+ background-repeat: no-repeat;
59+ background-position: 0 .15em;
60+ padding-left: 1.5em;
61+}
62+.done4 {
63+ background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABIAAAAQCAYAAAAbBi9cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAAzgAAAM4BlP6ToAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAIISURBVDiNnZQ9SFtRFMd/773kpTaGJoQk1im4VDpWQcTNODhkFBcVTCNCF0NWyeDiIIiCm82QoIMIUkHUxcFBg1SEQoZszSat6cdTn1qNue92CMbEr9Sey+XC/Z/zu+f8h6ukUil3sVg0+M+4cFxk42/jH2wAqqqKSCSiPQdwcHHAnDHH9s/tN1h8V28ETdP+eU8fT9Nt62ancYdIPvJNtsu87bmjrJlrTDVM4RROJs1JrHPrD4Bar7A6cpc54iKOaTdJXCUI2UMVrQZ0Js7YPN18ECKkYNQcJe/OE/4dZsw7VqNXQMvHy3QZXQypQ6ycrtwDjf8aJ+PNEDSCzLpn7+m2pD8ZKHlKarYhy6XjEoCYGcN95qansQeA3fNdki+SaJZGTMQIOoL3W/Z89rxv+tokubNajlvk/vm+LFpF2XnUKZHI0I+QrI7Dw0OZTqdzUkpsM7mZTyfy5OPGyw1tK7AFSvmB/Ks8w8YwbUYbe6/3QEKv0vugfxWPnMLJun+d/kI/WLdizpNjMbAIKrhMF4OuwadBALqqs+RfInwUvuNi+fBd+wjogfogAFVRmffO02q01mZZ0HHdgXIzdz0QQLPezIQygX6llxNKKgOFARYCC49CqhoHIUTlss/Vx2phlYwjw8j1CAlfAiwQiJpiy7o1VHnsG5FISkoJu7Q/2YmmaV+i0ei7v38L2CBguSi5AAAAAElFTkSuQmCC);
64+ background-repeat: no-repeat;
65+ background-position: 0 .15em;
66+ padding-left: 1.5em;
67+}
68+
69+*:not(pre) > code {
70+ font-family: Monaco,"Courier New","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace;
71+ -webkit-border-radius: 1px;
72+ -moz-border-radius: 1px;
73+ border-radius: 1px;
74+ -moz-background-clip: padding;
75+ -webkit-background-clip: padding-box;
76+ background-clip: padding-box;
77+ padding: 0px 3px;
78+ display: inline-block;
79+ color: #52595d;
80+ border: 1px solid #ccc;
81+ background-color: #f9f9f9;
82+}
default.nixdeleted−13
@@ -1,13 +0,0 @@
1−# default.nix
2−let
3− pkgs = import <nixpkgs> { };
4−in
5− pkgs.haskellPackages.developPackage {
6− root = ./.;
7− modifier = drv:
8− pkgs.haskell.lib.addBuildTools drv (with pkgs.haskellPackages;
9− [ cabal-install
10− ghcid
11− ]);
12− }
13−
doc/epub.md+2−9
@@ -141,15 +141,8 @@ Pandoc has an EPUB3 writer. It renders LaTeX math into MathML, which
141141EPUB3 readers are supposed to support (but unfortunately few do).
142142
143143Of course, this isn't much help if you want EPUB2 output (`pandoc -t epub2`)
144−or target readers that don't support MathML. Then you have two options:
145−
146−1. Use the [`--webtex`](https://pandoc.org/MANUAL.html#option--webtex) option,
147− which will use a web service to convert the TeX to an image.
148−2. Use the [`--gladtex`](https://pandoc.org/MANUAL.html#option--gladtex) option
149− to convert maths into SVG images on your local machine.
150−
151−Both GladTeX and WebTeX add the LaTeX source of the formula as alternative text
152−of the image, increasing accessibility for blind users.
144+or target readers that don't support MathML. Then you should try using the
145+`--webtex` option, which will use a web service to convert the TeX to an image.
153146
154147[KindleGen]: https://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000765211
155148[EPUB]: https://en.wikipedia.org/wiki/EPUB
doc/faqs.mddeleted−128
@@ -1,128 +0,0 @@
1−% FAQs
2−
3−::: faqs
4−
5−## How can I convert a whole directory of files from Markdown to RTF?
6−
7−On linux or OSX:
8−
9− for f in *.txt; do pandoc "$f" -s -o "${f%.txt}.rtf"; done
10−
11−
12−In Windows Powershell:
13−
14− gci -r -i *.txt |foreach{$rtf=$_.directoryname+"\"+$_.basename+".rtf";pandoc -f markdown -s $_.name -o $rtf}
15−
16−## I used pandoc to convert a document to ICML (or OPML or RTF), and when I try to open it I'm told it's invalid. What have I done wrong?
17−
18−Be sure to use the `-s` or `--standalone` flag, or you just get a
19−fragment, not a full document with the required header:
20−
21− pandoc -s -f markdown -t icml -o my.icml my.md
22−
23−## I get a blank document when I try to convert a markdown document in Chinese to PDF.
24−
25−By default, pandoc uses pdflatex to generate the PDF, and pdflatex
26−doesn't handle Chinese characters. But you can change the default to
27−use xelatex instead. You should also make sure you're using a font
28−with Chinese glyphs. For example:
29−
30− pandoc -o c.pdf --pdf-engine=xelatex -V mainfont='Adobe Ming Std'
31−
32−## The Windows installer does a single user install, rather than installing pandoc for all users. How can I install pandoc for all users?
33−
34−Run the following command as admin:
35−
36− msiexec /i pandoc-VERSION.msi ALLUSERS=1
37−
38−This will put pandoc in `C:\Program Files\Pandoc`.
39−You can install Pandoc to a different directory by setting APPLICATIONFOLDER parameter,
40−for example:
41−
42− msiexec /i pandoc-1.11.1.msi ALLUSERS=1 APPLICATIONFOLDER="C:\Pandoc"
43−
44−## How do I change the margins in PDF output?
45−
46−The option
47−
48− -V geometry:margin=1in
49−
50−will set the margins to one inch on each side. If you don't want uniform
51−margins, you can do something like
52−
53− -V geometry:"top=2cm, bottom=1.5cm, left=1cm, right=1cm"
54−
55−Or
56−
57− -V geometry:"left=3cm, width=10cm"
58−
59−For more options, see the documentation for the LaTeX [geometry
60−package](https://www.ctan.org/pkg/geometry).
61−
62−## How does pandoc compare to multimarkdown?
63−
64−Here is a [wiki
65−page](https://github.com/jgm/pandoc/wiki/Pandoc-vs-Multimarkdown)
66−comparing the two.
67−
68−## When I specify an image width of 50% and convert to LaTeX, pandoc sets the height to textheight and the aspect ratio isn't preserved. How can I prevent this?
69−
70−For example, if you convert an image with `{width="50%"}`, the LaTeX produced
71−will be `\includegraphics[width=0.5\textwidth,height=\textheight]`.
72−
73−This output presupposes the following code in pandoc's default latex
74−template:
75−
76−```
77−% Scale images if necessary, so that they will not overflow the page
78−% margins by default, and it is still possible to overwrite the defaults
79−% using explicit options in \includegraphics[width, height, ...]{}
80−\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
81−```
82−
83−If you don't have this in your custom template, you should
84−add it. If we didn't set the `height` explicitly in this way,
85−the image would not be resized correctly unless it was
86−being resized to smaller than its original size.
87−
88−## Pandoc sometimes uses too much memory. How can I limit the memory used by pandoc?
89−
90−`pandoc +RTS -M30m -RTS` will limit heap memory to 30MB.
91−When converting a document requires more than this, an out of
92−memory error will be issued.
93−
94−## When using `--include-in-header` with PDF or LaTeX output, how do I reference tex declarations coming after `$header-includes$` in the default template?
95−
96−For various reasons, the `$header-includes$` are not at the very
97−end of the LaTeX preamble. This poses a problem when the code
98−you are inserting depends on declarations in the preamble coming
99−after the `$header-includes$` location. For example, you might
100−want to reference the `\author` and `\title` metadata values
101−(set at the very bottom of the preamble) and present them in
102−margins. In that case you can wrap your code in `etoolbox`'s
103−`\AtEndPreamble`. The technique is demonstrated in [this
104−gist](https://gist.github.com/JohnLukeBentley/9dda6166b9ee5c4127afd2b8cd16b70a).
105−When using `\AtEndPreamble`, keep any `makeatletter` or
106−`makeatother` outside of the `\AtEndPreamble`, as shown in the
107−example.
108−
109−## How can I convert PDFs to other formats using pandoc?
110−
111−You can't. You can try opening the PDF in Word or Google Docs
112−and saving in a format from which pandoc can convert directly.
113−
114−## Do I really need to install a 1 GB TeX installation to produce a PDF using pandoc?
115−
116−No. You can get by with a relatively small TeX installation,
117−for example, by starting with MacTeX's Basic TeX distribution
118−and using the `tlmgr` tool to install a few packages required by pandoc
119−(see https://pandoc.org/MANUAL.html#creating-a-pdf).
120−
121−Or, you can produce PDFs via HTML and `wkhtmltopdf`,
122−or via groff ms and `pdfroff`. (These don't produce as nice
123−topography as TeX, particularly when it comes to math, but they
124−may be fine for many purposes.)
125−
126−
127−:::
128−
doc/filters.md+7−13
@@ -253,15 +253,12 @@ repository](https://github.com/jgm/pandocfilters).
253253
254254For a more Pythonic alternative to pandocfilters, see
255255the [panflute](https://pypi.org/project/panflute) library.
256−Don't like Python? There are also ports of pandocfilters in
257−
258−- [PHP](https://github.com/vinai/pandocfilters-php),
259−- [perl](https://metacpan.org/pod/Pandoc::Filter),
260−- TypeScript/JavaScript via Node.js
261− - [pandoc-filter](https://github.com/mvhenderson/pandoc-filter-node),
262− - [node-pandoc-filter](https://github.com/mu-io/node-pandoc-filter),
263−- [Groovy](https://github.com/dfrommi/groovy-pandoc), and
264−- [Ruby](https://heerdebeer.org/Software/markdown/paru/).
256+Don't like Python? There are also ports of pandocfilters in
257+[PHP](https://github.com/vinai/pandocfilters-php),
258+[perl](https://metacpan.org/pod/Pandoc::Filter),
259+[javascript/node.js](https://github.com/mvhenderson/pandoc-filter-node),
260+[Groovy](https://github.com/dfrommi/groovy-pandoc), and
261+[Ruby](https://heerdebeer.org/Software/markdown/paru/).
265262
266263Starting with pandoc 2.0, pandoc includes built-in support for
267264writing filters in lua. The lua interpreter is built in to
@@ -394,8 +391,6 @@ automatically assign it `Just` the target format or `Nothing`.
394391
395392We compile our script:
396393
397− # first, make sure pandoc-types is installed:
398− cabal install --lib pandoc-types --package-env .
399394 ghc --make handleRuby
400395
401396Then run it:
@@ -505,8 +500,7 @@ extension.
505500 ---------------- --------------
506501 .py `python`
507502 .hs `runhaskell`
508− .pl `perl`
509− .rb `ruby`
503+ .pl `ruby`
510504 .php `php`
511505 .js `node`
512506 .r `Rscript`
doc/jats.md+7−31
@@ -33,11 +33,6 @@ Metadata Values
3333
3434 See [`<given-names>`][elem:given-names].
3535
36− `name`
37− : full name of the author; included only as a fallback if
38− `author.surname` is not available. Tagged with
39− [`<string-name>`][elem:string-name].
40−
4136 `email`
4237 : the contributor's email address.
4338
@@ -50,13 +45,6 @@ Metadata Values
5045 list must also occur as the `id` of an affiliation listed in
5146 the top-level `affiliation` list.
5247
53− `equal-contrib`
54− : boolean attribute used to mark authors who contributed
55− equally to the work. The
56− [`equal-contrib`][attr:equal-contrib] attribute is added
57− to the author's [`<contrib>`] element if this is set to a
58− truthy value.
59−
6048 `cor-id`
6149 : identifier linking to the contributor's correspondence
6250 information. The info itself must be stored in as an item in
@@ -179,13 +167,12 @@ Metadata Values
179167 pass these components directly.
180168
181169 The publication date is recorded in the document via the
182− [`<pub-date>`] element and its sub-elements. The
183− [`publication-format`][attr:publication-format] attribute is
184− always set to `electronic`.
170+ [`<pub-date>`][elem:pub-date] element and its sub-elements. The
171+ [`pub-type`][attr:pub-type] attribute is always set to `epub`.
185172
186173 `iso-8601`
187174 : ISO-8601 representation of the publication date. Used as the
188− value of the [`<pub-date>`] element's
175+ value of the [`pub-date`][elem:pub-date] element's
189176 [`iso-8601-date`][attr:iso-8601-date] attribute.
190177
191178 This value is set automatically if pandoc can parse the
@@ -199,12 +186,6 @@ Metadata Values
199186 The values are set automatically if pandoc can parse the
200187 `date` value as a date.
201188
202− `type`
203− : The type of event marked by this date. The value is set as
204− the [`date-type`][attr:date-type] attribute on the
205− [`<pub-date>`] element and defaults to "pub" if not
206− specified.
207−
208189`article`
209190: information concerning the article that identifies or describes
210191 it. The key-value pairs within this map are typically used
@@ -314,14 +295,12 @@ Metadata Values
314295 `pissn`
315296 : ISSN identifier of the publication's print version. Used as
316297 content of the [`<issn>`][elem:issn] element with the
317− [`publication-format`][attr:publication-format] attribute
318− set to `print`.
298+ [`pub-type`][attr:pub-type] attribute set to `ppub`.
319299
320300 `eissn`
321301 : ISSN identifier of the publication's electronic version.
322302 Used as content of the [`<issn>`][elem:issn] element with
323− the [`publication-format`][attr:publication-format]
324− attribute set to `electronic`.
303+ the [`pub-type`][attr:pub-type] attribute set to `epub`.
325304
326305 `publisher-name`
327306 : name of the publishing entity (person, company, or other).
@@ -361,8 +340,6 @@ Required metadata values:
361340
362341[Ringgold]: https://ringgold.com/
363342[attr:content-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/content-type.html
364−[attr:date-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/date-type.html
365−[attr:equal-contrib]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/equal-contrib.html
366343[attr:fn-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/fn-type.html
367344[attr:institution-id-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/institution-id-type.html
368345[attr:iso-8601-date]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/iso-8601-date.html
@@ -370,7 +347,7 @@ Required metadata values:
370347[attr:kwd-group-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/kwd-group-type.html
371348[attr:license-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/license-type.html
372349[attr:pub-id-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/pub-id-type.html
373−[attr:publication-format]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/publication-format.html
350+[attr:pub-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/pub-type.html
374351[attr:ref-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/ref-type.html
375352[attr:rid]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/rid.html
376353[attr:subj-group-type]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/attribute/subj-group-type.html
@@ -395,6 +372,7 @@ Required metadata values:
395372[elem:license]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/license.html
396373[elem:notes]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/notes.html
397374[elem:permissions]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/permissions.html
375+[elem:pub-date]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/pub-date.html
398376[elem:publisher-loc]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/publisher-loc.html
399377[elem:publisher-name]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/publisher-name.html
400378[elem:string-name]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/string-name.html
@@ -406,9 +384,7 @@ Required metadata values:
406384[`<addr-line>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/addr-line.html
407385[`<aff>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/aff.html
408386[`<city>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/city.html
409−[`<contrib>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/contrib.html
410387[`<country>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/country.html
411388[`<institution-id>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-id.html
412389[`<institution-wrap>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution-wrap.html
413390[`<institution>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/institution.html
414−[`<pub-date>`]: https://jats.nlm.nih.gov/publishing/tag-library/1.2/element/pub-date.html
doc/libraries.mddeleted−63
@@ -1,63 +0,0 @@
1−---
2−title: The pandoc family of libraries
3−author: John MacFarlane
4−---
5−
6−The following Haskell libraries have been developed to support
7−pandoc:
8−
9−[skylighting-core] and [skylighting]
10−: Syntax highlighting engine supporting over 140 languages.
11−
12−[citeproc]
13−: Citation processing using CSL stylesheets.
14−
15−[texmath]
16−: Conversion of math between tex, Word equation, MathML, and GNU eqn.
17−
18−[unicode-collation]
19−: Proper Unicode collation (sorting).
20−
21−[doclayout]
22−: Combinators for laying out a textual document, with support
23− for line wrapping, tabular layout, and more.
24−
25−[doctemplates]
26−: Supports pandoc's templates.
27−
28−[commonmark], [commonmark-extensions], and [commonmark-pandoc]
29−: Efficient, standards-compliant parser for commonmark and extensions.
30−
31−[ipynb]
32−: Representation of Jupyter notebooks and conversion to and
33− from JSON.
34−
35−[zip-archive]
36−: A pure zip file creator and extractor, used by pandoc for
37− docx, ODT, and EPUB.
38−
39−[rfc5051]
40−: Simple unicode collation (used for citation sorting).
41−
42−[emojis]
43−: Conversion between emoji characters and aliases.
44−
45−[jira-wiki-markup]
46−: Support for parsing Jira wiki syntax.
47−
48−[skylighting]: https://hackage.haskell.org/package/skylighting
49−[skylighting-core]: https://hackage.haskell.org/package/skylighting-core
50−[citeproc]: https://hackage.haskell.org/package/citeproc
51−[texmath]: https://hackage.haskell.org/package/texmath
52−[doclayout]: https://hackage.haskell.org/package/doclayout
53−[doctemplates]: https://hackage.haskell.org/package/doctemplates
54−[commonmark]: https://hackage.haskell.org/package/commonmark
55−[commonmark-extensions]: https://hackage.haskell.org/package/commonmark-extensions
56−[commonmark-pandoc]: https://hackage.haskell.org/package/commonmark-pandoc
57−[ipynb]: https://hackage.haskell.org/package/ipynb
58−[zip-archive]: https://hackage.haskell.org/package/zip-archive
59−[rfc5051]: https://hackage.haskell.org/package/rfc5051
60−[emojis]: https://hackage.haskell.org/package/emojis
61−[jira-wiki-markup]: https://hackage.haskell.org/package/jira-wiki-markup
62−[unicode-collation]: https://hackage.haskell.org/package/unicode-collation
63−
doc/lua-filters.md+7−193
@@ -109,7 +109,7 @@ element filtering function. In other words, filter entries will
109109be called for each corresponding element in the document,
110110getting the respective element as input.
111111
112−The return of a filter function must be one of the following:
112+The return of a filter function must one of the following:
113113
114114- nil: this means that the object should remain unchanged.
115115- a pandoc object: this must be of the same type as the input
@@ -1513,7 +1513,7 @@ with the [`pandoc.Attr`](#pandoc.attr) constructor. For
15131513convenience, it is usually not necessary to construct the value
15141514directly if it is part of an element, and it is sufficient to
15151515pass an HTML-like table. E.g., to create a span with identifier
1516−"text" and classes "a" and "b", one can write:
1516+"text" and classes "a" and "b", on can write:
15171517
15181518 local span = pandoc.Span('text', {id = 'text', class = 'a b'})
15191519
@@ -3084,8 +3084,7 @@ Clear-out the media bag, deleting all items.
30843084
30853085`insert (filepath, mime_type, contents)`
30863086
3087−Adds a new entry to pandoc's media bag. Replaces any existing
3088−mediabag entry with the same `filepath`.
3087+Adds a new entry to pandoc's media bag.
30893088
30903089Parameters:
30913090
@@ -3093,7 +3092,7 @@ Parameters:
30933092: filename and path relative to the output folder.
30943093
30953094`mime_type`:
3096−: the file's MIME type; use `nil` if unknown or unavailable.
3095+: the file's MIME type
30973096
30983097`contents`:
30993098: the binary contents of the file.
@@ -3177,22 +3176,12 @@ Usage:
31773176
31783177### fetch {#pandoc.mediabag.fetch}
31793178
3180−`fetch (source)`
3179+`fetch (source, base_url)`
31813180
31823181Fetches the given source from a URL or local file. Returns two
31833182values: the contents of the file and the MIME type (or an empty
31843183string).
31853184
3186−The function will first try to retrieve `source` from the
3187−mediabag; if that fails, it will try to download it or read it
3188−from the local file system while respecting pandoc's "resource
3189−path" setting.
3190−
3191−Parameters:
3192−
3193−`source`:
3194−: path to a resource; either a local file path or URI
3195−
31963185Returns:
31973186
31983187- the entries MIME type, or nil if the file was not found.
@@ -3201,11 +3190,11 @@ Returns:
32013190Usage:
32023191
32033192 local diagram_url = "https://pandoc.org/diagram.jpg"
3204− local mt, contents = pandoc.mediabag.fetch(diagram_url)
3193+ local mt, contents = pandoc.mediabag.fetch(diagram_url, ".")
32053194
32063195# Module pandoc.List
32073196
3208−This module defines pandoc's list type. It comes with useful
3197+The this module defines pandoc's list type. It comes with useful
32093198methods and convenience functions.
32103199
32113200## Constructor
@@ -3387,181 +3376,6 @@ methods and convenience functions.
33873376 `comp`:
33883377 : Comparison function as described above.
33893378
3390−# Module pandoc.path
3391−
3392−Module for file path manipulations.
3393−
3394−## Static Fields {#pandoc.path-fields}
3395−
3396−### separator {#pandoc.path.separator}
3397−
3398−The character that separates directories.
3399−
3400−### search_path_separator {#pandoc.path.search_path_separator}
3401−
3402−The character that is used to separate the entries in the `PATH`
3403−environment variable.
3404−
3405−## Functions {#pandoc.path-functions}
3406−
3407−### directory (filepath) {#pandoc.path.directory}
3408−
3409−Gets the directory name, i.e., removes the last directory
3410−separator and everything after from the given path.
3411−
3412−Parameters:
3413−
3414−filepath
3415−: path (string)
3416−
3417−Returns:
3418−
3419−- The filepath up to the last directory separator. (string)
3420−
3421−### filename (filepath) {#pandoc.path.filename}
3422−
3423−Get the file name.
3424−
3425−Parameters:
3426−
3427−filepath
3428−: path (string)
3429−
3430−Returns:
3431−
3432−- File name part of the input path. (string)
3433−
3434−### is_absolute (filepath) {#pandoc.path.is_absolute}
3435−
3436−Checks whether a path is absolute, i.e. not fixed to a root.
3437−
3438−Parameters:
3439−
3440−filepath
3441−: path (string)
3442−
3443−Returns:
3444−
3445−- `true` iff `filepath` is an absolute path, `false` otherwise.
3446− (boolean)
3447−
3448−### is_relative (filepath) {#pandoc.path.is_relative}
3449−
3450−Checks whether a path is relative or fixed to a root.
3451−
3452−Parameters:
3453−
3454−filepath
3455−: path (string)
3456−
3457−Returns:
3458−
3459−- `true` iff `filepath` is a relative path, `false` otherwise.
3460− (boolean)
3461−
3462−### join (filepaths) {#pandoc.path.join}
3463−
3464−Join path elements back together by the directory separator.
3465−
3466−Parameters:
3467−
3468−filepaths
3469−: path components (list of strings)
3470−
3471−Returns:
3472−
3473−- The joined path. (string)
3474−
3475−### make_relative (path, root[, unsafe]) {#pandoc.path.make_relative}
3476−
3477−Contract a filename, based on a relative path. Note that the
3478−resulting path will usually not introduce `..` paths, as the
3479−presence of symlinks means `../b` may not reach `a/b` if it starts
3480−from `a/c`. For a worked example see [this blog
3481−post](https://neilmitchell.blogspot.co.uk/2015/10/filepaths-are-subtle-symlinks-are-hard.html).
3482−
3483−Set `unsafe` to a truthy value to a allow `..` in paths.
3484−
3485−Parameters:
3486−
3487−path
3488−: path to be made relative (string)
3489−
3490−root
3491−: root path (string)
3492−
3493−unsafe
3494−: whether to allow `..` in the result. (boolean)
3495−
3496−Returns:
3497−
3498−- contracted filename (string)
3499−
3500−### normalize (filepath) {#pandoc.path.normalize}
3501−
3502−Normalizes a path.
3503−
3504−- `//` makes sense only as part of a (Windows) network drive;
3505− elsewhere, multiple slashes are reduced to a single
3506− `path.separator` (platform dependent).
3507−- `/` becomes `path.separator` (platform dependent)
3508−- `./` -\> ''
3509−- an empty path becomes `.`
3510−
3511−Parameters:
3512−
3513−filepath
3514−: path (string)
3515−
3516−Returns:
3517−
3518−- The normalized path. (string)
3519−
3520−### split (filepath) {#pandoc.path.split}
3521−
3522−Splits a path by the directory separator.
3523−
3524−Parameters:
3525−
3526−filepath
3527−: path (string)
3528−
3529−Returns:
3530−
3531−- List of all path components. (list of strings)
3532−
3533−### split_extension (filepath) {#pandoc.path.split_extension}
3534−
3535−Splits the last extension from a file path and returns the parts. The
3536−extension, if present, includes the leading separator; if the path has
3537−no extension, then the empty string is returned as the extension.
3538−
3539−Parameters:
3540−
3541−filepath
3542−: path (string)
3543−
3544−Returns:
3545−
3546−- filepath without extension (string)
3547−
3548−- extension or empty string (string)
3549−
3550−### split_search_path (search_path) {#pandoc.path.split_search_path}
3551−
3552−Takes a string and splits it on the `search_path_separator` character.
3553−Blank items are ignored on Windows, and converted to `.` on Posix. On
3554−Windows path elements are stripped of quotes.
3555−
3556−Parameters:
3557−
3558−search_path
3559−: platform-specific search path (string)
3560−
3561−Returns:
3562−
3563−- list of directories in search path (list of strings)
3564−
35653379# Module pandoc.system
35663380
35673381Access to system information and functionality.
doc/org.md+2−16
@@ -25,7 +25,8 @@ The following export keywords are supported:
2525- LANGUAGE: document language; included as plain-text metadata
2626 field `lang`. The value should be a [BCP47 language tag].
2727
28−- SELECT_TAGS: tags which select a tree for export.
28+- SELECT_TAGS: tags which select a tree for export. Currently
29+ *unsupported*.
2930
3031- EXCLUDE\_TAGS: tags which prevent a subtree from being
3132 exported. Fully supported.
@@ -36,21 +37,6 @@ The following export keywords are supported:
3637 defaults to stdout unless a target has to be given as a command
3738 line option.
3839
39−::: {.alert .alert-info}
40−Pandoc tries to be compatible with org-mode when exporting an org document. If
41−you find some behavior confusing, please do refer to org-mode
42−[Export-Settings](https://orgmode.org/manual/Export-Settings.html)
43−documentation. For example, a common confusion
44−([#3214](https://github.com/jgm/pandoc/issues/3214 "Problem with headers lower
45−then 3 in org-mode reader"), [#5169](https://github.com/jgm/pandoc/issues/5169
46−"org mode headings past level three converted to numbered outline list"),
47−[#6145](https://github.com/jgm/pandoc/issues/6145 "Headers 4 levels deep render
48−differently"), [#7236](https://github.com/jgm/pandoc/issues/7236 "In Org mode,
49−Header with level > 3 are not recognized as headers")) is treatment of headers
50−with level > 3 differently because org-mode sets `org-export-headline-levels`
51−(configurable with `#+OPTIONS: H:3`) to 3 by default.
52−:::
53−
5440[BCP47 language tag]: https://tools.ietf.org/html/bcp47
5541
5642Format-specific options
doc/using-the-pandoc-api.md+12−25
@@ -14,7 +14,7 @@ and types is available at
1414Pandoc is structured as a set of *readers*, which translate
1515various input formats into an abstract syntax tree (the
1616Pandoc AST) representing a structured document, and a set of
17−*writers*, which render this AST into various output formats.
17+*writers*, which render this AST into various input formats.
1818Pictorially:
1919
2020```
@@ -90,14 +90,8 @@ Some notes:
9090Let's look at the types of `readMarkdown` and `writeRST`:
9191
9292```haskell
93−readMarkdown :: (PandocMonad m, ToSources a)
94− => ReaderOptions
95− -> a
96− -> m Pandoc
97−writeRST :: PandocMonad m
98− => WriterOptions
99− -> Pandoc
100− -> m Text
93+readMarkdown :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
94+writeRST :: PandocMonad m => WriterOptions -> Pandoc -> m Text
10195```
10296
10397The `PandocMonad m =>` part is a typeclass constraint.
@@ -139,7 +133,7 @@ report :: PandocMonad m => LogMessage -> m ()
139133-- | Fetch an image or other item from the local filesystem or the net.
140134-- Returns raw content and maybe mime type.
141135fetchItem :: PandocMonad m
142− => Text
136+ => String
143137 -> m (B.ByteString, Maybe MimeType)
144138
145139-- Set the resource path searched by 'fetchItem'.
@@ -161,13 +155,6 @@ Note that `PandocIO` is an instance of `MonadIO`, so you can
161155use `liftIO` to perform arbitrary IO operations inside a pandoc
162156conversion chain.
163157
164−`readMarkdown` is polymorphic in its second argument, which
165−can be any type that is an instance of the `ToSources`
166−typeclass. You can use `Text`, as in the example above.
167−But you can also use `[(FilePath, Text)]`, if the input comes
168−from multiple files and you want to track source positions
169−accurately.
170−
171158# Options
172159
173160The first argument of each reader or writer is for
@@ -213,8 +200,8 @@ of the Monoid typeclass and can easily be concatenated:
213200import Text.Pandoc.Builder
214201
215202mydoc :: Pandoc
216−mydoc = doc $ header 1 (text (T.pack "Hello!"))
217− <> para (emph (text (T.pack "hello world")) <> text (T.pack "."))
203+mydoc = doc $ header 1 (text "Hello!")
204+ <> para (emph (text "hello world") <> text ".")
218205
219206main :: IO ()
220207main = print mydoc
@@ -261,16 +248,16 @@ import qualified Data.Text as T
261248import Data.List (intersperse)
262249
263250data Station = Station{
264− address :: T.Text
265− , name :: T.Text
266− , cardsAccepted :: [T.Text]
251+ address :: String
252+ , name :: String
253+ , cardsAccepted :: [String]
267254 } deriving Show
268255
269256instance FromJSON Station where
270257 parseJSON (Object v) = Station <$>
271258 v .: "street_address" <*>
272259 v .: "station_name" <*>
273− (T.words <$> (v .:? "cards_accepted" .!= ""))
260+ (words <$> (v .:? "cards_accepted" .!= ""))
274261 parseJSON _ = mzero
275262
276263createLetter :: [Station] -> Pandoc
@@ -328,7 +315,7 @@ users to override the system defaults. If you want to disable
328315this behavior, use `setUserDataDir Nothing`.
329316
330317To render a template, use `renderTemplate'`, which takes two
331−arguments, a template (Text) and a context (any instance
318+arguments, a template (String) and a context (any instance
332319of ToJSON). If you want to create a context from the metadata
333320part of a Pandoc document, use `metaToJSON'` from
334321[Text.Pandoc.Writers.Shared]. If you also want to incorporate
@@ -426,7 +413,7 @@ concatenated together. Here's an example that returns a
426413list of the URLs linked to in a document:
427414
428415```haskell
429−listURLs :: Pandoc -> [Text]
416+listURLs :: Pandoc -> [String]
430417listURLs = query urls
431418 where urls (Link _ _ (src, _)) = [src]
432419 urls _ = []
linux/make_artifacts.sh+7−19
@@ -2,23 +2,13 @@ set -e
22
33MACHINE=$(uname -m)
44case "$MACHINE" in
5− x86_64) ARCHITECTURE=amd64;;
6− i686) ARCHITECTURE=i386;;
7− i386) ARCHITECTURE=i386;;
8− aarch64) ARCHITECTURE=arm64;;
9− *) ARCHITECTURE=unknown;;
5+ x86_64) ARCHITECTURE=amd64;;
6+ i686) ARCHITECTURE=i386;;
7+ i386) ARCHITECTURE=i386;;
108esac
119
1210ARTIFACTS="${ARTIFACTS:-/artifacts}"
1311
14−# This is our sentinel that tells us when we're done.
15−rm -f $ARTIFACTS/DONE
16−
17−clean_up() {
18− echo "All done!" > "$ARTIFACTS/DONE"
19−}
20−trap clean_up EXIT
21−
2212# build binaries
2313
2414cabal --version
@@ -26,9 +16,9 @@ ghc --version
2616
2717cabal v2-update
2818cabal v2-clean
29−cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-j4 +RTS -A256m -RTS -split-sections -optc-Os -optl=-pthread' pandoc
30−cabal v2-build -j4
31−cabal v2-test -j4
19+cabal v2-configure --enable-tests -f-export-dynamic -fembed_data_files --enable-executable-static --ghc-options '-optc-Os -optl=-pthread' pandoc
20+cabal v2-build
21+cabal v2-test -j1
3222for f in $(find dist-newstyle -name 'pandoc' -type f -perm /400); do cp $f /artifacts/; done
3323
3424# make deb
@@ -78,7 +68,5 @@ mv pandoc $TARGET/bin
7868strip $TARGET/bin/pandoc
7969gzip -9 $TARGET/share/man/man1/pandoc.1
8070
81−tar cvzf $TARGET-linux-$ARCHITECTURE.tar.gz $TARGET
71+tar cvzf $TARGET-linux-amd64.tar.gz $TARGET
8272rm -r $TARGET
83−
84−exit 0
man/pandoc.1+101−417
@@ -1,7 +1,7 @@
11'\" t
2−.\" Automatically generated by Pandoc 2.14.0.1
2+.\" Automatically generated by Pandoc 2.11.2
33.\"
4−.TH "Pandoc User\[cq]s Guide" "" "June 20, 2021" "pandoc 2.14.0.3" ""
4+.TH "Pandoc User\[cq]s Guide" "" "November 19, 2020" "pandoc 2.11.2" ""
55.hy
66.SH NAME
77pandoc - general markup converter
@@ -172,12 +172,13 @@ You can then test it with \f[C]pdflatex test.tex\f[R].
172172When using LaTeX, the following packages need to be available (they are
173173included with all recent versions of TeX Live): \f[C]amsfonts\f[R],
174174\f[C]amsmath\f[R], \f[C]lm\f[R], \f[C]unicode-math\f[R],
175−\f[C]iftex\f[R], \f[C]listings\f[R] (if the \f[C]--listings\f[R] option
176−is used), \f[C]fancyvrb\f[R], \f[C]longtable\f[R], \f[C]booktabs\f[R],
177−\f[C]graphicx\f[R] (if the document contains images),
178−\f[C]hyperref\f[R], \f[C]xcolor\f[R], \f[C]ulem\f[R], \f[C]geometry\f[R]
179−(with the \f[C]geometry\f[R] variable set), \f[C]setspace\f[R] (with
180−\f[C]linestretch\f[R]), and \f[C]babel\f[R] (with \f[C]lang\f[R]).
175+\f[C]ifxetex\f[R], \f[C]ifluatex\f[R], \f[C]listings\f[R] (if the
176+\f[C]--listings\f[R] option is used), \f[C]fancyvrb\f[R],
177+\f[C]longtable\f[R], \f[C]booktabs\f[R], \f[C]graphicx\f[R] (if the
178+document contains images), \f[C]hyperref\f[R], \f[C]xcolor\f[R],
179+\f[C]ulem\f[R], \f[C]geometry\f[R] (with the \f[C]geometry\f[R] variable
180+set), \f[C]setspace\f[R] (with \f[C]linestretch\f[R]), and
181+\f[C]babel\f[R] (with \f[C]lang\f[R]).
181182The use of \f[C]xelatex\f[R] or \f[C]lualatex\f[R] as the PDF engine
182183requires \f[C]fontspec\f[R].
183184\f[C]lualatex\f[R] uses \f[C]selnolig\f[R].
@@ -319,10 +320,6 @@ Specify output format.
319320.IP \[bu] 2
320321\f[C]beamer\f[R] (LaTeX beamer slide show)
321322.IP \[bu] 2
322−\f[C]bibtex\f[R] (BibTeX bibliography)
323−.IP \[bu] 2
324−\f[C]biblatex\f[R] (BibLaTeX bibliography)
325−.IP \[bu] 2
326323\f[C]commonmark\f[R] (CommonMark Markdown)
327324.IP \[bu] 2
328325\f[C]commonmark_x\f[R] (CommonMark Markdown with extensions)
@@ -457,15 +454,16 @@ On *nix and macOS systems this will be the \f[C]pandoc\f[R] subdirectory
457454of the XDG data directory (by default, \f[C]$HOME/.local/share\f[R],
458455overridable by setting the \f[C]XDG_DATA_HOME\f[R] environment
459456variable).
460−If that directory does not exist and \f[C]$HOME/.pandoc\f[R] exists, it
461−will be used (for backwards compatibility).
462−On Windows the default user data directory is
457+If that directory does not exist, \f[C]$HOME/.pandoc\f[R] will be used
458+(for backwards compatibility).
459+In Windows the default user data directory is
463460\f[C]C:\[rs]Users\[rs]USERNAME\[rs]AppData\[rs]Roaming\[rs]pandoc\f[R].
464461You can find the default user data directory on your system by looking
465462at the output of \f[C]pandoc --version\f[R].
466−Data files placed in this directory (for example,
467−\f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
468−\f[C]templates\f[R]) will override pandoc\[cq]s normal defaults.
463+A \f[C]reference.odt\f[R], \f[C]reference.docx\f[R], \f[C]epub.css\f[R],
464+\f[C]templates\f[R], \f[C]slidy\f[R], \f[C]slideous\f[R], or
465+\f[C]s5\f[R] directory placed in this directory will override
466+pandoc\[cq]s normal defaults.
469467.TP
470468\f[B]\f[CB]-d\f[B]\f[R] \f[I]FILE\f[R], \f[B]\f[CB]--defaults=\f[B]\f[R]\f[I]FILE\f[R]
471469Specify a set of default option settings.
@@ -625,8 +623,8 @@ above).
625623.IP "3." 3
626624\f[C]$PATH\f[R] (executable only)
627625.PP
628−Filters, Lua-filters, and citeproc processing are applied in the order
629−specified on the command line.
626+Filters and Lua-filters are applied in the order specified on the
627+command line.
630628.RE
631629.TP
632630\f[B]\f[CB]-L\f[B]\f[R] \f[I]SCRIPT\f[R], \f[B]\f[CB]--lua-filter=\f[B]\f[R]\f[I]SCRIPT\f[R]
@@ -643,17 +641,29 @@ The \f[C]pandoc\f[R] Lua module provides helper functions for element
643641creation.
644642It is always loaded into the script\[cq]s Lua environment.
645643.PP
646−See the Lua filters documentation for further details.
644+The following is an example Lua script for macro-expansion:
645+.IP
646+.nf
647+\f[C]
648+function expand_hello_world(inline)
649+ if inline.c == \[aq]{{helloworld}}\[aq] then
650+ return pandoc.Emph{ pandoc.Str \[dq]Hello, World\[dq] }
651+ else
652+ return inline
653+ end
654+end
655+
656+return {{Str = expand_hello_world}}
657+\f[R]
658+.fi
647659.PP
648660In order of preference, pandoc will look for Lua filters in
649661.IP "1." 3
650−a specified full or relative path
662+a specified full or relative path (executable or non-executable)
651663.IP "2." 3
652−\f[C]$DATADIR/filters\f[R] where \f[C]$DATADIR\f[R] is the user data
653−directory (see \f[C]--data-dir\f[R], above).
654−.PP
655−Filters, Lua filters, and citeproc processing are applied in the order
656−specified on the command line.
664+\f[C]$DATADIR/filters\f[R] (executable or non-executable) where
665+\f[C]$DATADIR\f[R] is the user data directory (see \f[C]--data-dir\f[R],
666+above).
657667.RE
658668.TP
659669\f[B]\f[CB]-M\f[B]\f[R] \f[I]KEY\f[R][\f[B]\f[CB]=\f[B]\f[R]\f[I]VAL\f[R]], \f[B]\f[CB]--metadata=\f[B]\f[R]\f[I]KEY\f[R][\f[B]\f[CB]:\f[B]\f[R]\f[I]VAL\f[R]]
@@ -716,11 +726,11 @@ Extract images and other media contained in or linked from the source
716726document to the path \f[I]DIR\f[R], creating it if necessary, and adjust
717727the images references in the document so they point to the extracted
718728files.
719−Media are downloaded, read from the file system, or extracted from a
720−binary container (e.g.\ docx), as needed.
721−The original file paths are used if they are relative paths not
722−containing \f[C]..\f[R].
723−Otherwise filenames are constructed from the SHA1 hash of the contents.
729+If the source format is a binary container (docx, epub, or odt), the
730+media is extracted from the container and the original filenames are
731+used.
732+Otherwise the media is read from the file system or downloaded, and new
733+filenames are constructed based on SHA1 hashes of the contents.
724734.TP
725735\f[B]\f[CB]--abbreviations=\f[B]\f[R]\f[I]FILE\f[R]
726736Specifies a custom abbreviations file, with abbreviations one to a line.
@@ -730,9 +740,9 @@ system default.
730740To see the system default, use
731741\f[C]pandoc --print-default-data-file=abbreviations\f[R].
732742The only use pandoc makes of this list is in the Markdown reader.
733−Strings found in this list will be followed by a nonbreaking space, and
734−the period will not produce sentence-ending space in formats like LaTeX.
735−The strings may not contain spaces.
743+Strings ending in a period that are found in this list will be followed
744+by a nonbreaking space, so that the period will not produce
745+sentence-ending space in formats like LaTeX.
736746.SS General writer options
737747.TP
738748\f[B]\f[CB]-s\f[B]\f[R], \f[B]\f[CB]--standalone\f[B]\f[R]
@@ -923,11 +933,13 @@ Note that, if \f[C]--resource-path\f[R] is specified, the working
923933directory must be explicitly listed or it will not be searched.
924934For example: \f[C]--resource-path=.:test\f[R] will search the working
925935directory and the \f[C]test\f[R] subdirectory, in that order.
926−This option can be used repeatedly.
927−Search path components that come later on the command line will be
928−searched before those that come earlier, so
929−\f[C]--resource-path foo:bar --resource-path baz:bim\f[R] is equivalent
930−to \f[C]--resource-path baz:bim:foo:bar\f[R].
936+.RS
937+.PP
938+\f[C]--resource-path\f[R] only has an effect if (a) the output format
939+embeds images (for example, \f[C]docx\f[R], \f[C]pdf\f[R], or
940+\f[C]html\f[R] with \f[C]--self-contained\f[R]) or (b) it is used
941+together with \f[C]--extract-media\f[R].
942+.RE
931943.TP
932944\f[B]\f[CB]--request-header=\f[B]\f[R]\f[I]NAME\f[R]\f[B]\f[CB]:\f[B]\f[R]\f[I]VAL\f[R]
933945Set the request header \f[I]NAME\f[R] to the value \f[I]VAL\f[R] when
@@ -1022,8 +1034,7 @@ become \f[C]\[rs]part{..}\f[R], while second-level headings remain as
10221034their default type.
10231035.TP
10241036\f[B]\f[CB]-N\f[B]\f[R], \f[B]\f[CB]--number-sections\f[B]\f[R]
1025−Number section headings in LaTeX, ConTeXt, HTML, Docx, ms, or EPUB
1026−output.
1037+Number section headings in LaTeX, ConTeXt, HTML, Docx, or EPUB output.
10271038By default, sections are not numbered.
10281039Sections with class \f[C]unnumbered\f[R] will never be numbered, even if
10291040\f[C]--number-sections\f[R] is specified.
@@ -1415,9 +1426,6 @@ Set the \f[C]bibliography\f[R] field in the document\[cq]s metadata to
14151426\f[I]FILE\f[R], overriding any value set in the metadata.
14161427If you supply this argument multiple times, each \f[I]FILE\f[R] will be
14171428added to bibliography.
1418−If \f[I]FILE\f[R] is a URL, it will be fetched via HTTP.
1419−If \f[I]FILE\f[R] is not found relative to the working directory, it
1420−will be sought in the resource path (see \f[C]--resource-path\f[R]).
14211429.TP
14221430\f[B]\f[CB]--csl=\f[B]\f[R]\f[I]FILE\f[R]
14231431Set the \f[C]csl\f[R] field in the document\[cq]s metadata to
@@ -1425,9 +1433,8 @@ Set the \f[C]csl\f[R] field in the document\[cq]s metadata to
14251433(This is equivalent to \f[C]--metadata csl=FILE\f[R].) If \f[I]FILE\f[R]
14261434is a URL, it will be fetched via HTTP.
14271435If \f[I]FILE\f[R] is not found relative to the working directory, it
1428−will be sought in the resource path (see \f[C]--resource-path\f[R]) and
1429−finally in the \f[C]csl\f[R] subdirectory of the pandoc user data
1430−directory.
1436+will be sought in the resource path and finally in the \f[C]csl\f[R]
1437+subdirectory of the pandoc user data directory.
14311438.TP
14321439\f[B]\f[CB]--citation-abbreviations=\f[B]\f[R]\f[I]FILE\f[R]
14331440Set the \f[C]citation-abbreviations\f[R] field in the document\[cq]s
@@ -1436,9 +1443,8 @@ metadata to \f[I]FILE\f[R], overriding any value set in the metadata.
14361443\f[C]--metadata citation-abbreviations=FILE\f[R].) If \f[I]FILE\f[R] is
14371444a URL, it will be fetched via HTTP.
14381445If \f[I]FILE\f[R] is not found relative to the working directory, it
1439−will be sought in the resource path (see \f[C]--resource-path\f[R]) and
1440−finally in the \f[C]csl\f[R] subdirectory of the pandoc user data
1441−directory.
1446+will be sought in the resource path and finally in the \f[C]csl\f[R]
1447+subdirectory of the pandoc user data directory.
14421448.TP
14431449\f[B]\f[CB]--natbib\f[B]\f[R]
14441450Use \f[C]natbib\f[R] for citations in LaTeX output.
@@ -1504,16 +1510,16 @@ inserted.
15041510.TP
15051511\f[B]\f[CB]--gladtex\f[B]\f[R]
15061512Enclose TeX math in \f[C]<eq>\f[R] tags in HTML output.
1507−The resulting HTML can then be processed by GladTeX to produce SVG
1508−images of the typeset formulas and an HTML file with these images
1509−embedded.
1513+The resulting HTML can then be processed by GladTeX to produce images of
1514+the typeset formulas and an HTML file with links to these images.
1515+So, the procedure is:
15101516.RS
15111517.IP
15121518.nf
15131519\f[C]
15141520pandoc -s --gladtex input.md -o myfile.htex
1515−gladtex -d image_dir myfile.htex
1516−# produces myfile.html and images in image_dir
1521+gladtex -d myfile-images myfile.htex
1522+# produces myfile.html and images in myfile-images
15171523\f[R]
15181524.fi
15191525.RE
@@ -1617,11 +1623,6 @@ T}@T{
16171623PandocPDFError
16181624T}
16191625T{
1620−44
1621−T}@T{
1622−PandocXMLError
1623−T}
1624−T{
1625162647
16261627T}@T{
16271628PandocPDFProgramNotFoundError
@@ -1682,11 +1683,6 @@ T}@T{
16821683PandocIpynbDecodingError
16831684T}
16841685T{
1685−94
1686−T}@T{
1687−PandocUnsupportedCharsetError
1688−T}
1689−T{
1690168697
16911687T}@T{
16921688PandocCouldNotFindDataFileError
@@ -1719,16 +1715,6 @@ input-files:
17191715- content.md
17201716# or you may use input-file: with a single value
17211717
1722−# Include options from the specified defaults files.
1723−# The files will be searched for first in the working directory
1724−# and then in the defaults subdirectory of the user data directory.
1725−# The files are included in the same order in which they appear in
1726−# the list. Options specified in this defaults file always have
1727−# priority over the included ones.
1728−defaults:
1729−- defsA
1730−- defsB
1731−
17321718template: letter
17331719standalone: true
17341720self-contained: false
@@ -1786,11 +1772,8 @@ data-dir:
17861772verbosity: INFO
17871773log-file: log.json
17881774
1789−# citeproc, natbib, or biblatex. This only affects LaTeX
1790−# output. If you want to use citeproc to format citations,
1791−# you should also set \[aq]citeproc: true\[aq] (see above).
1775+# citeproc, natbib, or biblatex
17921776cite-method: citeproc
1793−
17941777# part, chapter, section, or default:
17951778top-level-division: chapter
17961779abbreviations:
@@ -1891,38 +1874,6 @@ verbosity: INFO
18911874\f[R]
18921875.fi
18931876.PP
1894−In fields that expect a file path (or list of file paths), the following
1895−syntax may be used to interpolate environment variables:
1896−.IP
1897−.nf
1898−\f[C]
1899−csl: ${HOME}/mycsldir/special.csl
1900−\f[R]
1901−.fi
1902−.PP
1903−\f[C]${USERDATA}\f[R] may also be used; this will always resolve to the
1904−user data directory that is current when the defaults file is parsed,
1905−regardless of the setting of the environment variable
1906−\f[C]USERDATA\f[R].
1907−.PP
1908−\f[C]${.}\f[R] will resolve to the directory containing the default file
1909−itself.
1910−This allows you to refer to resources contained in that directory:
1911−.IP
1912−.nf
1913−\f[C]
1914−epub-cover-image: ${.}/cover.jpg
1915−epub-metadata: ${.}/meta.xml
1916−resource-path:
1917−- . # the working directory from which pandoc is run
1918−- ${.}/images # the images subdirectory of the directory
1919− # containing this defaults file
1920−\f[R]
1921−.fi
1922−.PP
1923−This environment variable interpolation syntax \f[I]only\f[R] works in
1924−fields that expect file paths.
1925−.PP
19261877Default files can be placed in the \f[C]defaults\f[R] subdirectory of
19271878the user data directory and used from any directory.
19281879For example, one could create a file specifying defaults for writing
@@ -2182,9 +2133,9 @@ ${ styles.html() }
21822133\f[R]
21832134.fi
21842135.PP
2185−(If a partial is not found in the directory of the template and the
2186−template path is given as a relative path, it will also be sought in the
2187−\f[C]templates\f[R] subdirectory of the user data directory.)
2136+(If a partial is not found in the directory of the template, it will
2137+also be sought in the \f[C]templates\f[R] subdirectory of the user data
2138+directory.)
21882139.PP
21892140Partials may optionally be applied to variables using a colon:
21902141.IP
@@ -2591,9 +2542,10 @@ YAML metadata or with \f[C]-M classoption=fleqn\f[R].
25912542.SS Variables for HTML slides
25922543.PP
25932544These affect HTML output when [producing slide shows with pandoc].
2594−.TP
2595−\f[B]\f[CB]institute\f[B]\f[R]
2596−author affiliations: can be a list when there are multiple authors
2545+.PP
2546+All reveal.js configuration options are available as variables.
2547+To turn off boolean flags that default to true in reveal.js, use
2548+\f[C]0\f[R].
25972549.TP
25982550\f[B]\f[CB]revealjs-url\f[B]\f[R]
25992551base URL for reveal.js documents (defaults to
@@ -2612,10 +2564,6 @@ base URL for Slideous documents (defaults to \f[C]slideous\f[R])
26122564\f[B]\f[CB]title-slide-attributes\f[B]\f[R]
26132565additional attributes for the title slide of reveal.js slide shows.
26142566See background in reveal.js and beamer for an example.
2615−.PP
2616−All reveal.js configuration options are available as variables.
2617−To turn off boolean flags that default to true in reveal.js, use
2618−\f[C]0\f[R].
26192567.SS Variables for Beamer slides
26202568.PP
26212569These variables change the appearance of PDF slides using
@@ -3428,43 +3376,6 @@ This extension can be enabled/disabled for the following formats:
34283376.TP
34293377output formats
34303378\f[C]odt\f[R], \f[C]opendocument\f[R]
3431−.SS Extension: \f[C]xrefs_name\f[R]
3432−.PP
3433−Links to headings, figures and tables inside the document are
3434−substituted with cross-references that will use the name or caption of
3435−the referenced item.
3436−The original link text is replaced once the generated document is
3437−refreshed.
3438−This extension can be combined with \f[C]xrefs_number\f[R] in which case
3439−numbers will appear before the name.
3440−.PP
3441−Text in cross-references is only made consistent with the referenced
3442−item once the document has been refreshed.
3443−.PP
3444−This extension can be enabled/disabled for the following formats:
3445−.TP
3446−output formats
3447−\f[C]odt\f[R], \f[C]opendocument\f[R]
3448−.SS Extension: \f[C]xrefs_number\f[R]
3449−.PP
3450−Links to headings, figures and tables inside the document are
3451−substituted with cross-references that will use the number of the
3452−referenced item.
3453−The original link text is discarded.
3454−This extension can be combined with \f[C]xrefs_name\f[R] in which case
3455−the name or caption numbers will appear after the number.
3456−.PP
3457−For the \f[C]xrefs_number\f[R] to be useful heading numbers must be
3458−enabled in the generated document, also table and figure captions must
3459−be enabled using for example the \f[C]native_numbering\f[R] extension.
3460−.PP
3461−Numbers in cross-references are only visible in the final document once
3462−it has been refreshed.
3463−.PP
3464−This extension can be enabled/disabled for the following formats:
3465−.TP
3466−output formats
3467−\f[C]odt\f[R], \f[C]opendocument\f[R]
34683379.SS Extension: \f[C]styles\f[R]
34693380.PP
34703381When converting from docx, read all docx styles as divs (for paragraph
@@ -3490,12 +3401,6 @@ output format.
34903401.PP
34913402Some aspects of Pandoc\[cq]s Markdown citation syntax are also accepted
34923403in \f[C]org\f[R] input.
3493−.SS Extension: \f[C]element_citations\f[R]
3494−.PP
3495−In the \f[C]jats\f[R] output formats, this causes reference items to be
3496−replaced with \f[C]<element-citation>\f[R] elements.
3497−These elements are not influenced by CSL styles, but all information on
3498−the item is included in tags.
34993404.SS Extension: \f[C]ntb\f[R]
35003405.PP
35013406In the \f[C]context\f[R] output format this enables the use of Natural
@@ -3979,9 +3884,6 @@ begin with a space.
39793884\f[R]
39803885.fi
39813886.PP
3982−Inline formatting (such as emphasis) is allowed in the content, but not
3983−block-level formatting (such as block quotes or lists).
3984−.PP
39853887This syntax is borrowed from reStructuredText.
39863888.SS Lists
39873889.SS Bullet lists
@@ -4275,7 +4177,7 @@ Term 2
42754177Note that space between items in a definition list is required.
42764178(A variant that loosens this requirement, but disallows \[lq]lazy\[rq]
42774179hard wrapping, can be activated with \f[C]compact_definition_lists\f[R]:
4278−see Non-default extensions, below.)
4180+see Non-pandoc extensions, below.)
42794181.SS Numbered example lists
42804182.SS Extension: \f[C]example_lists\f[R]
42814183.PP
@@ -4796,9 +4698,7 @@ All of the metadata will appear in a single block at the beginning of
47964698the document.
47974699.PP
47984700Note that YAML escaping rules must be followed.
4799−Thus, for example, if a title contains a colon, it must be quoted, and
4800−if it contains a backslash escape, then it must be ensured that it is
4801−not treated as a YAML escape sequence.
4701+Thus, for example, if a title contains a colon, it must be quoted.
48024702The pipe character (\f[C]|\f[R]) can be used to begin an indented block
48034703that will be interpreted literally, without need for escaping.
48044704This form is necessary when the field contains blank lines or
@@ -4820,13 +4720,6 @@ abstract: |
48204720\f[R]
48214721.fi
48224722.PP
4823−The literal block after the \f[C]|\f[R] must be indented relative to the
4824−line containing the \f[C]|\f[R].
4825−If it is not, the YAML will be invalid and pandoc will not interpret it
4826−as metadata.
4827−For an overview of the complex rules governing YAML, see the Wikipedia
4828−entry on YAML syntax.
4829−.PP
48304723Template variables will be set automatically from the metadata.
48314724Thus, for example, in writing HTML, the variable \f[C]abstract\f[R] will
48324725be set to the HTML equivalent of the Markdown in the \f[C]abstract\f[R]
@@ -4892,21 +4785,6 @@ header-includes:
48924785 \[ga]\[ga]\[ga]
48934786\f[R]
48944787.fi
4895−.PP
4896−Note: the \f[C]yaml_metadata_block\f[R] extension works with
4897−\f[C]commonmark\f[R] as well as \f[C]markdown\f[R] (and it is enabled by
4898−default in \f[C]gfm\f[R] and \f[C]commonmark_x\f[R]).
4899−However, in these formats the following restrictions apply:
4900−.IP \[bu] 2
4901−The YAML metadata block must occur at the beginning of the document (and
4902−there can be only one).
4903−If multiple files are given as arguments to pandoc, only the first can
4904−be a YAML metadata block.
4905−.IP \[bu] 2
4906−The leaf nodes of the YAML structure are parsed in isolation from each
4907−other and from the rest of the document.
4908−So, for example, you can\[cq]t use a reference link in these contexts if
4909−the link definition is somewhere else in the document.
49104788.SS Backslash escapes
49114789.SS Extension: \f[C]all_symbols_escapable\f[R]
49124790.PP
@@ -5126,9 +5004,7 @@ delimiters.
51265004.PP
51275005For display math, use \f[C]$$\f[R] delimiters.
51285006(In this case, the delimiters may be separated from the formula by
5129−whitespace.
5130−However, there can be no blank lines betwen the opening and closing
5131−\f[C]$$\f[R] delimiters.)
5007+whitespace.)
51325008.PP
51335009TeX math will be printed in all output formats.
51345010How it is rendered depends on the output format:
@@ -5254,9 +5130,8 @@ into
52545130.PP
52555131whereas \f[C]Markdown.pl\f[R] will preserve it as is.
52565132.PP
5257−There is one exception to this rule: text between \f[C]<script>\f[R],
5258−\f[C]<style>\f[R], and \f[C]<textarea>\f[R] tags is not interpreted as
5259−Markdown.
5133+There is one exception to this rule: text between \f[C]<script>\f[R] and
5134+\f[C]<style>\f[R] tags is not interpreted as Markdown.
52605135.PP
52615136This departure from standard Markdown should make it easier to mix
52625137Markdown with HTML block elements.
@@ -5787,70 +5662,27 @@ Inline and regular footnotes may be mixed freely.
57875662.SS Citation syntax
57885663.SS Extension: \f[C]citations\f[R]
57895664.PP
5790−To cite a bibliographic item with an identifier foo, use the syntax
5791−\f[C]\[at]foo\f[R].
5792−Normal citations should be included in square brackets, with semicolons
5793−separating distinct items:
5794−.IP
5795−.nf
5796−\f[C]
5797−Blah blah [\[at]doe99; \[at]smith2000; \[at]smith2004].
5798−\f[R]
5799−.fi
5800−.PP
5801−How this is rendered depends on the citation style.
5802−In an author-date style, it might render as
5803−.IP
5804−.nf
5805−\f[C]
5806−Blah blah (Doe 1999, Smith 2000, 2004).
5807−\f[R]
5808−.fi
5809−.PP
5810−In a footnote style, it might render as
5665+Markdown citations go inside square brackets and are separated by
5666+semicolons.
5667+Each citation must have a key, composed of `\[at]' + the citation
5668+identifier from the database, and may optionally have a prefix, a
5669+locator, and a suffix.
5670+The citation key must begin with a letter, digit, or \f[C]_\f[R], and
5671+may contain alphanumerics, \f[C]_\f[R], and internal punctuation
5672+characters (\f[C]:.#$%&-+?<>\[ti]/\f[R]).
5673+Here are some examples:
58115674.IP
58125675.nf
58135676\f[C]
5814−Blah blah.[\[ha]1]
5677+Blah blah [see \[at]doe99, pp. 33-35; also \[at]smith04, chap. 1].
58155678
5816−[\[ha]1]: John Doe, \[dq]Frogs,\[dq] *Journal of Amphibians* 44 (1999);
5817−Susan Smith, \[dq]Flies,\[dq] *Journal of Insects* (2000);
5818−Susan Smith, \[dq]Bees,\[dq] *Journal of Insects* (2004).
5819−\f[R]
5820−.fi
5821−.PP
5822−See the CSL user documentation for more information about CSL styles and
5823−how they affect rendering.
5824−.PP
5825−Unless a citation key start with a letter, digit, or \f[C]_\f[R], and
5826−contains only alphanumerics and internal punctuation characters
5827−(\f[C]:.#$%&-+?<>\[ti]/\f[R]), it must be surrounded by curly braces,
5828−which are not considered part of the key.
5829−In \f[C]\[at]Foo_bar.baz.\f[R], the key is \f[C]Foo_bar.baz\f[R].
5830−The final period is not \f[I]internal\f[R] punctuation, so it is not
5831−included in the key.
5832−In \f[C]\[at]{Foo_bar.baz.}\f[R], the key is \f[C]Foo_bar.baz.\f[R],
5833−including the final period.
5834−The curly braces are recommended if you use URLs as keys:
5835−\f[C][\[at]{https://example.com/bib?name=foobar&date=2000}, p. 33]\f[R].
5836−.PP
5837−Citation items may optionally include a prefix, a locator, and a suffix.
5838−In
5839−.IP
5840−.nf
5841−\f[C]
5842−Blah blah [see \[at]doe99, pp. 33-35 and *passim*; \[at]smith04, chap. 1].
5679+Blah blah [\[at]doe99, pp. 33-35, 38-39 and *passim*].
5680+
5681+Blah blah [\[at]smith04; \[at]doe99].
58435682\f[R]
58445683.fi
58455684.PP
5846−The first item (\f[C]doe99\f[R]) has prefix \f[C]see\f[R], locator
5847−\f[C]pp. 33-35\f[R], and suffix \f[C]and *passim*\f[R].
5848−The second item (\f[C]smith04\f[R]) has locator \f[C]chap. 1\f[R] and no
5849−prefix or suffix.
5850−.PP
5851−Pandoc uses some heuristics to separate the locator from the rest of the
5852−subject.
5853−It is sensitive to the locator terms defined in the CSL locale files.
5685+\f[C]pandoc\f[R] detects locator terms in the CSL locale files.
58545686Either abbreviated or unabbreviated forms are accepted.
58555687In the \f[C]en-US\f[R] locale, locator terms can be written in either
58565688singular or plural forms, as \f[C]book\f[R],
@@ -5872,15 +5704,14 @@ singular or plural forms, as \f[C]book\f[R],
58725704\f[C]\[sc]\f[R]/\f[C]\[sc]\[sc]\f[R].
58735705If no locator term is used, \[lq]page\[rq] is assumed.
58745706.PP
5875−In complex cases, you can force something to be treated as a locator by
5876−enclosing it in curly braces or prevent parsing the suffix as locator by
5877−prepending curly braces:
5707+\f[C]pandoc\f[R] will use heuristics to distinguish the locator from the
5708+suffix.
5709+In complex cases, the locator can be enclosed in curly braces:
58785710.IP
58795711.nf
58805712\f[C]
58815713[\[at]smith{ii, A, D-Z}, with a suffix]
58825714[\[at]smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here]
5883−[\[at]smith{}, 99 years later]
58845715\f[R]
58855716.fi
58865717.PP
@@ -5894,8 +5725,7 @@ Smith says blah [-\[at]smith04].
58945725\f[R]
58955726.fi
58965727.PP
5897−You can also write an author-in-text citation, by omitting the square
5898−brackets:
5728+You can also write an in-text citation, as follows:
58995729.IP
59005730.nf
59015731\f[C]
@@ -5904,64 +5734,13 @@ brackets:
59045734\[at]smith04 [p. 33] says blah.
59055735\f[R]
59065736.fi
5907−.PP
5908−This will cause the author\[cq]s name to be rendered, followed by the
5909−bibliographical details.
5910−Use this form when you want to make the citation the subject of a
5911−sentence.
5912−.PP
5913−When you are using a note style, it is usually better to let citeproc
5914−create the footnotes from citations rather than writing an explicit
5915−note.
5916−If you do write an explicit note that contains a citation, note that
5917−normal citations will be put in parentheses, while author-in-text
5918−citations will not.
5919−For this reason, it is sometimes preferable to use the author-in-text
5920−style inside notes when using a note style.
5921−.SS Non-default extensions
5737+.SS Non-pandoc extensions
59225738.PP
59235739The following Markdown syntax extensions are not enabled by default in
59245740pandoc, but may be enabled by adding \f[C]+EXTENSION\f[R] to the format
59255741name, where \f[C]EXTENSION\f[R] is the name of the extension.
59265742Thus, for example, \f[C]markdown+hard_line_breaks\f[R] is Markdown with
59275743hard line breaks.
5928−.SS Extension: \f[C]rebase_relative_paths\f[R]
5929−.PP
5930−Rewrite relative paths for Markdown links and images, depending on the
5931−path of the file containing the link or image link.
5932−For each link or image, pandoc will compute the directory of the
5933−containing file, relative to the working directory, and prepend the
5934−resulting path to the link or image path.
5935−.PP
5936−The use of this extension is best understood by example.
5937−Suppose you have a a subdirectory for each chapter of a book,
5938−\f[C]chap1\f[R], \f[C]chap2\f[R], \f[C]chap3\f[R].
5939−Each contains a file \f[C]text.md\f[R] and a number of images used in
5940−the chapter.
5941−You would like to have \f[C]![image](spider.jpg)\f[R] in
5942−\f[C]chap1/text.md\f[R] refer to \f[C]chap1/spider.jpg\f[R] and
5943−\f[C]![image](spider.jpg)\f[R] in \f[C]chap2/text.md\f[R] refer to
5944−\f[C]chap2/spider.jpg\f[R].
5945−To do this, use
5946−.IP
5947−.nf
5948−\f[C]
5949−pandoc chap*/*.md -f markdown+rebase_relative_paths
5950−\f[R]
5951−.fi
5952−.PP
5953−Without this extension, you would have to use
5954−\f[C]![image](chap1/spider.jpg)\f[R] in \f[C]chap1/text.md\f[R] and
5955−\f[C]![image](chap2/spider.jpg)\f[R] in \f[C]chap2/text.md\f[R].
5956−Links with relative paths will be rewritten in the same way as images.
5957−.PP
5958−Absolute paths and URLs are not changed.
5959−Neither are empty paths or paths consisting entirely of a fragment,
5960−e.g., \f[C]#foo\f[R].
5961−.PP
5962−Note that relative paths in reference links and images will be rewritten
5963−relative to the file containing the link reference definition, not the
5964−file containing the reference link or image itself, if these differ.
59655744.SS Extension: \f[C]attributes\f[R]
59665745.PP
59675746Allows attributes to be attached to any inline or block-level element.
@@ -6141,12 +5920,6 @@ be indented four spaces.
61415920Use Project Gutenberg conventions for \f[C]plain\f[R] output: all-caps
61425921for strong emphasis, surround by underscores for regular emphasis, add
61435922extra blank space around headings.
6144−.SS Extension: \f[C]sourcepos\f[R]
6145−.PP
6146−Include source position attributes when parsing \f[C]commonmark\f[R].
6147−For elements that accept attributes, a \f[C]data-pos\f[R] attribute is
6148−added; other elements are placed in a surrounding Div or Span elemnet
6149−with a \f[C]data-pos\f[R] attribute.
61505923.SS Markdown variants
61515924.PP
61525925In addition to pandoc\[cq]s extended Markdown, the following Markdown
@@ -6306,11 +6079,6 @@ references:
63066079\f[R]
63076080.fi
63086081.PP
6309−If both an external bibliography and inline (YAML metadata) references
6310−are provided, both will be used.
6311−In case of conflicting \f[C]id\f[R]s, the inline references will take
6312−precedence.
6313−.PP
63146082Note that \f[C]pandoc\f[R] can be used to produce such a YAML metadata
63156083section from a BibTeX, BibLaTeX, or CSL JSON bibliography:
63166084.IP
@@ -6321,13 +6089,13 @@ pandoc chem.json -s -f csljson -t markdown
63216089\f[R]
63226090.fi
63236091.PP
6324−Indeed, \f[C]pandoc\f[R] can convert between any of these citation
6325−formats:
6092+\f[C]pandoc\f[R] can also be used to produce CSL JSON bibliography from
6093+BibTeX, BibLaTeX, or markdown YAML:
63266094.IP
63276095.nf
63286096\f[C]
63296097pandoc chem.bib -s -f biblatex -t csljson
6330−pandoc chem.yaml -s -f markdown -t biblatex
6098+pandoc chem.yaml -s -f markdown -t csljson
63316099\f[R]
63326100.fi
63336101.PP
@@ -6456,41 +6224,6 @@ The format of the file can be illustrated with an example:
64566224}
64576225\f[R]
64586226.fi
6459−.SS Citations in note styles
6460−.PP
6461−Pandoc\[cq]s citation processing is designed to allow you to move
6462−between author-date, numerical, and note styles without modifying the
6463−markdown source.
6464−When you\[cq]re using a note style, avoid inserting footnotes manually.
6465−Instead, insert citations just as you would in an author-date
6466−style\[em]for example,
6467−.IP
6468−.nf
6469−\f[C]
6470−Blah blah [\[at]foo, p. 33].
6471−\f[R]
6472−.fi
6473−.PP
6474−The footnote will be created automatically.
6475−Pandoc will take care of removing the space and moving the note before
6476−or after the period, depending on the setting of
6477−\f[C]notes-after-punctuation\f[R], as described below in Other relevant
6478−metadata fields.
6479−.PP
6480−In some cases you may need to put a citation inside a regular footnote.
6481−Normal citations in footnotes (such as \f[C][\[at]foo, p. 33]\f[R]) will
6482−be rendered in parentheses.
6483−In-text citations (such as \f[C]\[at]foo [p. 33]\f[R]) will be rendered
6484−without parentheses.
6485−(A comma will be added if appropriate.) Thus:
6486−.IP
6487−.nf
6488−\f[C]
6489−[\[ha]1]: Some studies [\[at]foo; \[at]bar, p. 33] show that
6490−frubulicious zoosnaps are quantical. For a survey
6491−of the literature, see \[at]baz [chap. 1].
6492−\f[R]
6493−.fi
64946227.SS Raw content in a style
64956228.PP
64966229To include raw content in a prefix, suffix, delimiter, or term, surround
@@ -6591,29 +6324,9 @@ entries (for author-date and numerical styles only).
65916324.TP
65926325\f[B]\f[CB]lang\f[B]\f[R]
65936326The \f[C]lang\f[R] field will affect how the style is localized, for
6594−example in the translation of labels, the use of quotation marks, and
6595−the way items are sorted.
6327+example in the translation of labels and the use of quotation marks.
65966328(For backwards compatibility, \f[C]locale\f[R] may be used instead of
65976329\f[C]lang\f[R], but this use is deprecated.)
6598−.RS
6599−.PP
6600−A BCP 47 language tag is expected: for example, \f[C]en\f[R],
6601−\f[C]de\f[R], \f[C]en-US\f[R], \f[C]fr-CA\f[R], \f[C]ug-Cyrl\f[R].
6602−The unicode extension syntax (after \f[C]-u-\f[R]) may be used to
6603−specify options for collation (sorting) more precisely.
6604−Here are some examples:
6605−.IP \[bu] 2
6606−\f[C]zh-u-co-pinyin\f[R] \[en] Chinese with the Pinyin collation.
6607−.IP \[bu] 2
6608−\f[C]es-u-co-trad\f[R] \[en] Spanish with the traditional collation
6609−(with \f[C]Ch\f[R] sorting after \f[C]C\f[R]).
6610−.IP \[bu] 2
6611−\f[C]fr-u-kb\f[R] \[en] French with \[lq]backwards\[rq] accent sorting
6612−(with \f[C]cot\['e]\f[R] sorting after \f[C]c\[^o]te\f[R]).
6613−.IP \[bu] 2
6614−\f[C]en-US-u-kf-upper\f[R] \[en] English with uppercase letters sorting
6615−before lower (default is lower before upper).
6616−.RE
66176330.TP
66186331\f[B]\f[CB]notes-after-punctuation\f[B]\f[R]
66196332If true (the default), pandoc will put footnote citations after
@@ -6734,11 +6447,6 @@ A heading at the slide level always starts a new slide.
67346447.IP \[bu] 2
67356448Headings \f[I]below\f[R] the slide level in the hierarchy create
67366449headings \f[I]within\f[R] a slide.
6737−(In beamer, a \[lq]block\[rq] will be created.
6738−If the heading has the class \f[C]example\f[R], an
6739−\f[C]exampleblock\f[R] environment will be used; if it has the class
6740−\f[C]alert\f[R], an \f[C]alertblock\f[R] will be used; otherwise a
6741−regular \f[C]block\f[R] will be used.)
67426450.IP \[bu] 2
67436451Headings \f[I]above\f[R] the slide level in the hierarchy create
67446452\[lq]title slides,\[rq] which just contain the section title and help to
@@ -7139,16 +6847,6 @@ A string value.
71396847\f[B]\f[CB]rights\f[B]\f[R]
71406848A string value.
71416849.TP
7142−\f[B]\f[CB]belongs-to-collection\f[B]\f[R]
7143−A string value.
7144−identifies the name of a collection to which the EPUB Publication
7145−belongs.
7146−.TP
7147−\f[B]\f[CB]group-position\f[B]\f[R]
7148−The \f[C]group-position\f[R] field indicates the numeric position in
7149−which the EPUB Publication belongs relative to other works belonging to
7150−the same \f[C]belongs-to-collection\f[R] field.
7151−.TP
71526850\f[B]\f[CB]cover-image\f[B]\f[R]
71536851A string value (path to cover image).
71546852.TP
@@ -7664,20 +7362,6 @@ need to specify a template manually using \f[C]--template\f[R] or add a
76647362new default template with the name
76657363\f[C]default.NAME_OF_CUSTOM_WRITER.lua\f[R] to the \f[C]templates\f[R]
76667364subdirectory of your user data directory (see Templates).
7667−.SH REPRODUCIBLE BUILDS
7668−.PP
7669−Some of the document formats pandoc targets (such as EPUB, docx, and
7670−ODT) include build timestamps in the generated document.
7671−That means that the files generated on successive builds will differ,
7672−even if the source does not.
7673−To avoid this, set the \f[C]SOURCE_DATE_EPOCH\f[R] environment variable,
7674−and the timestamp will be taken from it instead of the current time.
7675−\f[C]SOURCE_DATE_EPOCH\f[R] should contain an integer unix timestamp
7676−(specifying the number of second since midnight UTC January 1, 1970).
7677−.PP
7678−Some document formats also include a unique identifier.
7679−For EPUB, this can be set explicitly by setting the \f[C]identifier\f[R]
7680−metadata field (see EPUB Metadata, above).
76817365.SH A NOTE ON SECURITY
76827366.PP
76837367If you use pandoc to convert user-contributed content in a web
@@ -7708,16 +7392,16 @@ The HTML generated by pandoc is not guaranteed to be safe.
77087392If \f[C]raw_html\f[R] is enabled for the Markdown input, users can
77097393inject arbitrary HTML.
77107394Even if \f[C]raw_html\f[R] is disabled, users can include dangerous
7711−content in URLs and attributes.
7395+content in attributes for headings, spans, and code blocks.
77127396To be safe, you should run all the generated HTML through an HTML
77137397sanitizer.
77147398.SH AUTHORS
77157399.PP
7716−Copyright 2006\[en]2021 John MacFarlane (jgm\[at]berkeley.edu).
7400+Copyright 2006\[en]2020 John MacFarlane (jgm\[at]berkeley.edu).
77177401Released under the GPL, version 2 or greater.
77187402This software carries no warranty of any kind.
77197403(See COPYRIGHT for full copyright and warranty notices.) For a full list
77207404of contributors, see the file AUTHORS.md in the pandoc source code.
77217405.PP
77227406The Pandoc source code and all documentation may be downloaded
7723−from <https://pandoc.org>.
7407+from <http://pandoc.org>.
man/pandoc.1.after+1−1
@@ -1,3 +1,3 @@
11.PP
22The Pandoc source code and all documentation may be downloaded
3−from <https://pandoc.org>.
3+from <http://pandoc.org>.
pandoc.cabal+171−123
@@ -1,10 +1,10 @@
11cabal-version: 2.2
22name: pandoc
3−version: 2.14.0.3
3+version: 2.11.2
44build-type: Simple
55license: GPL-2.0-or-later
66license-file: COPYING.md
7−copyright: (c) 2006-2021 John MacFarlane
7+copyright: (c) 2006-2020 John MacFarlane
88author: John MacFarlane <jgm@berkeley.edu>
99maintainer: John MacFarlane <jgm@berkeley.edu>
1010bug-reports: https://github.com/jgm/pandoc/issues
@@ -12,7 +12,7 @@ stability: alpha
1212homepage: https://pandoc.org
1313category: Text
1414tested-with: GHC == 8.2.2, GHC == 8.4.4, GHC == 8.6.5,
15− GHC == 8.8.4, GHC == 8.10.2, GHC == 9.0.1
15+ GHC == 8.8.4, GHC == 8.10.2
1616synopsis: Conversion between markup formats
1717description: Pandoc is a Haskell library for converting from one markup
1818 format to another, and a command-line tool that uses
@@ -56,8 +56,6 @@ data-files:
5656 data/templates/default.icml
5757 data/templates/default.opml
5858 data/templates/default.latex
59− data/templates/default.bibtex
60− data/templates/default.biblatex
6159 data/templates/default.context
6260 data/templates/default.texinfo
6361 data/templates/default.jira
@@ -86,7 +84,6 @@ data-files:
8684 data/templates/default.epub2
8785 data/templates/default.epub3
8886 data/templates/article.jats_publishing
89− data/templates/affiliations.jats
9087 -- translations
9188 data/translations/*.yaml
9289 -- source files for reference.docx
@@ -182,6 +179,8 @@ data-files:
182179 data/pandoc.List.lua
183180 -- bash completion template
184181 data/bash_completion.tpl
182+ -- jats csl
183+ data/jats.csl
185184 -- citeproc
186185 data/default.csl
187186 citeproc/biblatex-localization/*.lbx.strings
@@ -208,25 +207,14 @@ extra-source-files:
208207 test/command/*.md
209208 test/command/*.csl
210209 test/command/biblio.bib
211− test/command/averroes.bib
210+ test/command/biblatex-examples.bib
212211 test/command/A.txt
213212 test/command/B.txt
214213 test/command/C.txt
215214 test/command/D.txt
216215 test/command/01.csv
217− test/command/chap1/spider.png
218− test/command/chap2/spider.png
219− test/command/chap1/text.md
220− test/command/chap2/text.md
221216 test/command/defaults1.yaml
222217 test/command/defaults2.yaml
223− test/command/defaults3.yaml
224− test/command/defaults4.yaml
225− test/command/defaults5.yaml
226− test/command/defaults6.yaml
227− test/command/defaults7.yaml
228− test/command/defaults8.yaml
229− test/command/defaults9.yaml
230218 test/command/3533-rst-csv-tables.csv
231219 test/command/3880.txt
232220 test/command/5182.txt
@@ -298,7 +286,6 @@ extra-source-files:
298286 test/tables.opendocument
299287 test/tables.org
300288 test/tables.asciidoc
301− test/tables.asciidoctor
302289 test/tables.haddock
303290 test/tables.texinfo
304291 test/tables.rst
@@ -310,9 +297,7 @@ extra-source-files:
310297 test/tables.xwiki
311298 test/tables/*.html4
312299 test/tables/*.html5
313− test/tables/*.latex
314300 test/tables/*.native
315− test/tables/*.jats_archiving
316301 test/testsuite.txt
317302 test/writer.latex
318303 test/writer.context
@@ -333,7 +318,6 @@ extra-source-files:
333318 test/writer.opendocument
334319 test/writer.org
335320 test/writer.asciidoc
336− test/writer.asciidoctor
337321 test/writer.haddock
338322 test/writer.rst
339323 test/writer.icml
@@ -400,116 +384,104 @@ flag trypandoc
400384 Description: Build trypandoc cgi executable.
401385 Default: False
402386
403−common common-options
404− default-language: Haskell2010
405− build-depends: base >= 4.9 && < 5
406− ghc-options: -Wall -fno-warn-unused-do-bind
407− -Wincomplete-record-updates
408− -Wnoncanonical-monad-instances
409−
410− if impl(ghc < 8.4)
411− hs-source-dirs: prelude
412− other-modules: Prelude
413− build-depends: base-compat >= 0.9
414− other-extensions: NoImplicitPrelude
415−
416− if os(windows)
417− cpp-options: -D_WINDOWS
418−
419− -- Later:
420− -- -Wpartial-fields (currently used in Powerpoint writer)
421− -- -Wmissing-export-lists (currently some Odt modules violate this)
422− -- -Wredundant-constraints (problematic if we want to support older base)
423− if impl(ghc >= 8.2)
424− ghc-options: -Wcpp-undef
425− if impl(ghc >= 8.4)
426− ghc-options: -Wincomplete-uni-patterns
427− -Widentities
428− -fhide-source-paths
429−
430−common common-executable
431− import: common-options
432− build-depends: pandoc
433− ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
434−
435−
436387library
437− import: common-options
438− build-depends: Glob >= 0.7 && < 0.11,
388+ build-depends: base >= 4.9 && < 5,
389+ Glob >= 0.7 && < 0.11,
439390 HTTP >= 4000.0.5 && < 4000.4,
440391 HsYAML >= 0.2 && < 0.3,
441392 JuicyPixels >= 3.1.6.1 && < 3.4,
442393 SHA >= 1.6 && < 1.7,
443394 aeson >= 0.7 && < 1.6,
444395 aeson-pretty >= 0.8.5 && < 0.9,
445− array >= 0.5 && < 0.6,
446− attoparsec >= 0.12 && < 0.15,
447− base64-bytestring >= 0.1 && < 1.3,
448− binary >= 0.7 && < 0.11,
396+ attoparsec >= 0.12 && < 0.14,
397+ base64-bytestring >= 0.1 && < 1.2,
398+ binary >= 0.5 && < 0.11,
449399 blaze-html >= 0.9 && < 0.10,
450400 blaze-markup >= 0.8 && < 0.9,
451401 bytestring >= 0.9 && < 0.12,
452402 case-insensitive >= 1.2 && < 1.3,
453− citeproc >= 0.4.0.1 && < 0.4.1,
454− commonmark >= 0.2.1 && < 0.3,
455− commonmark-extensions >= 0.2.1.2 && < 0.3,
456− commonmark-pandoc >= 0.2.1.1 && < 0.3,
403+ citeproc >= 0.2 && < 0.3,
404+ commonmark >= 0.1.1.2 && < 0.2,
405+ commonmark-extensions >= 0.2.0.4 && < 0.3,
406+ commonmark-pandoc >= 0.2 && < 0.3,
457407 connection >= 0.3.1,
458408 containers >= 0.4.2.1 && < 0.7,
459409 data-default >= 0.4 && < 0.8,
460410 deepseq >= 1.3 && < 1.5,
461411 directory >= 1.2.3 && < 1.4,
462− doclayout >= 0.3.0.1 && < 0.4,
463− doctemplates >= 0.9 && < 0.10,
412+ doclayout >= 0.3 && < 0.4,
413+ doctemplates >= 0.8.2 && < 0.9,
464414 emojis >= 0.1 && < 0.2,
465415 exceptions >= 0.8 && < 0.11,
466416 file-embed >= 0.0 && < 0.1,
467417 filepath >= 1.1 && < 1.5,
468− haddock-library >= 1.10 && < 1.11,
418+ haddock-library >= 1.8 && < 1.10,
469419 hslua >= 1.1 && < 1.4,
470− hslua-module-path >= 0.1.0 && < 0.2.0,
471420 hslua-module-system >= 0.2 && < 0.3,
472421 hslua-module-text >= 0.2.1 && < 0.4,
473422 http-client >= 0.4.30 && < 0.8,
474423 http-client-tls >= 0.2.4 && < 0.4,
475424 http-types >= 0.8 && < 0.13,
476425 ipynb >= 0.1 && < 0.2,
477− jira-wiki-markup >= 1.4 && < 1.5,
426+ jira-wiki-markup >= 1.3.2 && < 1.4,
478427 mtl >= 2.2 && < 2.3,
479428 network >= 2.6,
480− network-uri >= 2.6 && < 2.8,
429+ network-uri >= 2.6 && < 2.7,
481430 pandoc-types >= 1.22 && < 1.23,
482431 parsec >= 3.1 && < 3.2,
483432 process >= 1.2.3 && < 1.7,
484− random >= 1 && < 1.3,
485− safe >= 0.3.18 && < 0.4,
433+ random >= 1 && < 1.2,
434+ safe >= 0.3 && < 0.4,
486435 scientific >= 0.3 && < 0.4,
487− skylighting >= 0.10.5.2 && < 0.10.6,
488− skylighting-core >= 0.10.5.2 && < 0.10.6,
436+ skylighting >= 0.10.1 && < 0.11,
437+ skylighting-core >= 0.10.1 && < 0.11,
489438 split >= 0.2 && < 0.3,
490439 syb >= 0.1 && < 0.8,
491440 tagsoup >= 0.14.6 && < 0.15,
492441 temporary >= 1.1 && < 1.4,
493− texmath >= 0.12.3 && < 0.12.4,
442+ texmath >= 0.12.0.2 && < 0.13,
494443 text >= 1.1.1.0 && < 1.3,
495444 text-conversions >= 0.3 && < 0.4,
496− time >= 1.5 && < 1.12,
445+ time >= 1.5 && < 1.10,
497446 unicode-transforms >= 0.3 && < 0.4,
498447 unordered-containers >= 0.2 && < 0.3,
499448 xml >= 1.3.12 && < 1.4,
500− xml-conduit >= 1.9.1.1 && < 1.10,
501− unicode-collation >= 0.1.1 && < 0.2,
502449 zip-archive >= 0.2.3.4 && < 0.5,
503450 zlib >= 0.5 && < 0.7
504451 if os(windows) && arch(i386)
505452 build-depends: basement >= 0.0.10,
506453 foundation >= 0.0.23
507454 -- basement 0.0.9 won't build on 32-bit windows.
508− if !os(windows)
455+ if impl(ghc < 8.4)
456+ hs-source-dirs: prelude
457+ other-modules: Prelude
458+ build-depends: base-compat >= 0.9
459+ if os(windows)
460+ cpp-options: -D_WINDOWS
461+ else
509462 build-depends: unix >= 2.4 && < 2.8
510463 if flag(embed_data_files)
511464 cpp-options: -DEMBED_DATA_FILES
512465 other-modules: Text.Pandoc.Data
466+ if os(windows)
467+ cpp-options: -D_WINDOWS
468+ ghc-options: -Wall -fno-warn-unused-do-bind
469+ -Wincomplete-record-updates
470+ -Wnoncanonical-monad-instances
471+ -- Later:
472+ -- -Wpartial-fields (currently used in Powerpoint writer)
473+ -- -Wmissing-export-lists (currently some Odt modules violate this)
474+ -- -Wredundant-constraints (problematic if we want to support older base)
475+ if impl(ghc >= 8.2)
476+ ghc-options: -Wcpp-undef
477+ if impl(ghc >= 8.4)
478+ ghc-options: -Wincomplete-uni-patterns
479+ -Widentities
480+ -Werror=missing-home-modules
481+ -fhide-source-paths
482+
483+ default-language: Haskell2010
484+ other-extensions: NoImplicitPrelude
513485 hs-source-dirs: src
514486
515487 exposed-modules: Text.Pandoc,
@@ -517,13 +489,13 @@ library
517489 Text.Pandoc.Options,
518490 Text.Pandoc.Extensions,
519491 Text.Pandoc.Shared,
520− Text.Pandoc.Sources,
521492 Text.Pandoc.MediaBag,
522493 Text.Pandoc.Error,
523494 Text.Pandoc.Filter,
524495 Text.Pandoc.Readers,
525496 Text.Pandoc.Readers.HTML,
526497 Text.Pandoc.Readers.LaTeX,
498+ Text.Pandoc.Readers.LaTeX.Types,
527499 Text.Pandoc.Readers.Markdown,
528500 Text.Pandoc.Readers.CommonMark,
529501 Text.Pandoc.Readers.Creole,
@@ -592,7 +564,6 @@ library
592564 Text.Pandoc.Writers.Shared,
593565 Text.Pandoc.Writers.OOXML,
594566 Text.Pandoc.Writers.AnnotatedTable,
595− Text.Pandoc.Writers.BibTeX,
596567 Text.Pandoc.Lua,
597568 Text.Pandoc.PDF,
598569 Text.Pandoc.UTF8,
@@ -607,6 +578,7 @@ library
607578 Text.Pandoc.Asciify,
608579 Text.Pandoc.Emoji,
609580 Text.Pandoc.ImageSize,
581+ Text.Pandoc.BCP47,
610582 Text.Pandoc.Class,
611583 Text.Pandoc.Citeproc
612584 other-modules: Text.Pandoc.App.CommandLineOptions,
@@ -631,15 +603,9 @@ library
631603 Text.Pandoc.Readers.HTML.Table,
632604 Text.Pandoc.Readers.HTML.TagCategories,
633605 Text.Pandoc.Readers.HTML.Types,
634− Text.Pandoc.Readers.LaTeX.Inline,
635− Text.Pandoc.Readers.LaTeX.Citation,
636− Text.Pandoc.Readers.LaTeX.Lang,
637− Text.Pandoc.Readers.LaTeX.Macro,
638− Text.Pandoc.Readers.LaTeX.Math,
639606 Text.Pandoc.Readers.LaTeX.Parsing,
607+ Text.Pandoc.Readers.LaTeX.Lang,
640608 Text.Pandoc.Readers.LaTeX.SIunitx,
641− Text.Pandoc.Readers.LaTeX.Table,
642− Text.Pandoc.Readers.LaTeX.Types,
643609 Text.Pandoc.Readers.Odt.Base,
644610 Text.Pandoc.Readers.Odt.Namespaces,
645611 Text.Pandoc.Readers.Odt.StyleReader,
@@ -663,21 +629,8 @@ library
663629 Text.Pandoc.Readers.Metadata,
664630 Text.Pandoc.Readers.Roff,
665631 Text.Pandoc.Writers.Docx.StyleMap,
666− Text.Pandoc.Writers.Docx.Table,
667− Text.Pandoc.Writers.Docx.Types,
668− Text.Pandoc.Writers.GridTable
669− Text.Pandoc.Writers.JATS.References,
670632 Text.Pandoc.Writers.JATS.Table,
671633 Text.Pandoc.Writers.JATS.Types,
672− Text.Pandoc.Writers.LaTeX.Caption,
673− Text.Pandoc.Writers.LaTeX.Notes,
674− Text.Pandoc.Writers.LaTeX.Table,
675− Text.Pandoc.Writers.LaTeX.Lang,
676− Text.Pandoc.Writers.LaTeX.Types,
677− Text.Pandoc.Writers.LaTeX.Citation,
678− Text.Pandoc.Writers.LaTeX.Util,
679− Text.Pandoc.Writers.Markdown.Types,
680− Text.Pandoc.Writers.Markdown.Inline,
681634 Text.Pandoc.Writers.Roff,
682635 Text.Pandoc.Writers.Powerpoint.Presentation,
683636 Text.Pandoc.Writers.Powerpoint.Output,
@@ -705,10 +658,6 @@ library
705658 Text.Pandoc.Lua.PandocLua,
706659 Text.Pandoc.Lua.Util,
707660 Text.Pandoc.Lua.Walk,
708− Text.Pandoc.XML.Light,
709− Text.Pandoc.XML.Light.Types,
710− Text.Pandoc.XML.Light.Proc,
711− Text.Pandoc.XML.Light.Output,
712661 Text.Pandoc.CSS,
713662 Text.Pandoc.CSV,
714663 Text.Pandoc.RoffChar,
@@ -727,50 +676,119 @@ library
727676 buildable: True
728677
729678executable pandoc
730− import: common-executable
731− hs-source-dirs: app
679+ build-depends: pandoc, base >= 4.8 && < 5
680+ if impl(ghc < 8.4)
681+ hs-source-dirs: prelude
682+ other-modules: Prelude
683+ build-depends: base-compat >= 0.9
684+ default-language: Haskell2010
685+ other-extensions: NoImplicitPrelude
686+ hs-source-dirs: .
732687 main-is: pandoc.hs
733688 buildable: True
734689 other-modules: Paths_pandoc
690+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
691+ -Wall -fno-warn-unused-do-bind
692+ -Wincomplete-record-updates
693+ -Wnoncanonical-monad-instances
694+ if impl(ghc >= 8.2)
695+ ghc-options: -Wcpp-undef
696+ if impl(ghc >= 8.4)
697+ ghc-options: -Wincomplete-uni-patterns
698+ -Widentities
699+ -Werror=missing-home-modules
700+ -fhide-source-paths
735701
736702executable trypandoc
737− import: common-executable
738703 main-is: trypandoc.hs
739704 hs-source-dirs: trypandoc
705+ default-language: Haskell2010
706+ other-extensions: NoImplicitPrelude
740707 if flag(trypandoc)
741− build-depends: aeson, http-types, text, wai >= 0.3, wai-extra >= 3.0.24
708+ build-depends: base, aeson, pandoc,
709+ text, wai-extra, wai >= 0.3, http-types
742710 buildable: True
743711 else
744712 buildable: False
713+ if impl(ghc < 8.4)
714+ hs-source-dirs: prelude
715+ other-modules: Prelude
716+ build-depends: base-compat >= 0.9
717+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
718+ -Wall -fno-warn-unused-do-bind
719+ -Wincomplete-record-updates
720+ -Wnoncanonical-monad-instances
721+ if impl(ghc >= 8.2)
722+ ghc-options: -Wcpp-undef
723+ if impl(ghc >= 8.4)
724+ ghc-options: -Wincomplete-uni-patterns
725+ -Widentities
726+ -Werror=missing-home-modules
727+ -fhide-source-paths
728+
729+benchmark weigh-pandoc
730+ type: exitcode-stdio-1.0
731+ main-is: weigh-pandoc.hs
732+ hs-source-dirs: benchmark
733+ build-depends: pandoc,
734+ base >= 4.8 && < 5,
735+ mtl >= 2.2 && < 2.3,
736+ text,
737+ weigh >= 0.0 && < 0.1,
738+ if impl(ghc < 8.4)
739+ hs-source-dirs: prelude
740+ other-modules: Prelude
741+ build-depends: base-compat >= 0.9
742+ default-language: Haskell2010
743+ other-extensions: NoImplicitPrelude
744+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
745+ -Wall -fno-warn-unused-do-bind
746+ -Wincomplete-record-updates
747+ -Wnoncanonical-monad-instances
748+ if impl(ghc >= 8.2)
749+ ghc-options: -Wcpp-undef
750+ if impl(ghc >= 8.4)
751+ ghc-options: -Wincomplete-uni-patterns
752+ -Widentities
753+ -Werror=missing-home-modules
754+ -fhide-source-paths
755+
745756
746757test-suite test-pandoc
747− import: common-executable
748758 type: exitcode-stdio-1.0
749759 main-is: test-pandoc.hs
750760 hs-source-dirs: test
751761 build-depends: pandoc,
762+ base >= 4.8 && < 5,
752763 Diff >= 0.2 && < 0.5,
753764 Glob >= 0.7 && < 0.11,
754765 QuickCheck >= 2.4 && < 2.15,
766+ base64-bytestring >= 0.1 && < 1.2,
755767 bytestring >= 0.9 && < 0.12,
756768 containers >= 0.4.2.1 && < 0.7,
757769 directory >= 1.2.3 && < 1.4,
758− doctemplates >= 0.9 && < 0.10,
770+ doctemplates >= 0.8.2 && < 0.9,
759771 exceptions >= 0.8 && < 0.11,
772+ executable-path >= 0.0 && < 0.1,
760773 filepath >= 1.1 && < 1.5,
761774 hslua >= 1.1 && < 1.4,
762775 mtl >= 2.2 && < 2.3,
763776 pandoc-types >= 1.22 && < 1.23,
764777 process >= 1.2.3 && < 1.7,
765− tasty >= 0.11 && < 1.5,
778+ tasty >= 0.11 && < 1.4,
766779 tasty-golden >= 2.3 && < 2.4,
767780 tasty-hunit >= 0.9 && < 0.11,
768781 tasty-lua >= 0.2 && < 0.3,
769782 tasty-quickcheck >= 0.8 && < 0.11,
770− text >= 1.1.1.0 && < 1.3,
771− time >= 1.5 && < 1.12,
783+ temporary >= 1.1 && < 1.4,
784+ text >= 1.1.1.0 && < 1.3,
785+ time >= 1.5 && < 1.10,
772786 xml >= 1.3.12 && < 1.4,
773787 zip-archive >= 0.2.3.4 && < 0.5
788+ if impl(ghc < 8.4)
789+ hs-source-dirs: prelude
790+ other-modules: Prelude
791+ build-depends: base-compat >= 0.9
774792 other-modules: Tests.Old
775793 Tests.Command
776794 Tests.Helpers
@@ -826,19 +844,49 @@ test-suite test-pandoc
826844 Tests.Writers.OOXML
827845 Tests.Writers.Ms
828846 Tests.Writers.AnnotatedTable
847+ if os(windows)
848+ cpp-options: -D_WINDOWS
849+ default-language: Haskell2010
850+ other-extensions: NoImplicitPrelude
851+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
852+ -Wall -fno-warn-unused-do-bind
853+ -Wincomplete-record-updates
854+ -Wnoncanonical-monad-instances
855+ if impl(ghc >= 8.2)
856+ ghc-options: -Wcpp-undef
857+ if impl(ghc >= 8.4)
858+ ghc-options: -Wincomplete-uni-patterns
859+ -Widentities
860+ -Werror=missing-home-modules
861+ -fhide-source-paths
862+
829863
830864benchmark benchmark-pandoc
831− import: common-executable
832865 type: exitcode-stdio-1.0
833866 main-is: benchmark-pandoc.hs
834867 hs-source-dirs: benchmark
835− build-depends: bytestring,
868+ build-depends: pandoc,
869+ base >= 4.8 && < 5,
870+ bytestring,
836871 containers,
837− -- gauge >= 0.2 && < 0.3,
838− tasty-bench >= 0.2 && <= 0.3,
872+ criterion >= 1.0 && < 1.6,
839873 mtl >= 2.2 && < 2.3,
840874 text >= 1.1.1.0 && < 1.3,
841− time,
842− deepseq
843− -- we increase heap size to avoid benchmarking garbage collection:
844− ghc-options: -rtsopts -with-rtsopts=-A8m -threaded
875+ time
876+ if impl(ghc < 8.4)
877+ hs-source-dirs: prelude
878+ other-modules: Prelude
879+ build-depends: base-compat >= 0.9
880+ default-language: Haskell2010
881+ other-extensions: NoImplicitPrelude
882+ ghc-options: -rtsopts -with-rtsopts=-K16m -threaded
883+ -Wall -fno-warn-unused-do-bind
884+ -Wincomplete-record-updates
885+ -Wnoncanonical-monad-instances
886+ if impl(ghc >= 8.2)
887+ ghc-options: -Wcpp-undef
888+ if impl(ghc >= 8.4)
889+ ghc-options: -Wincomplete-uni-patterns
890+ -Widentities
891+ -Werror=missing-home-modules
892+ -fhide-source-paths
app/pandoc.hs → pandoc.hsrenamed+3−1
@@ -1,6 +1,7 @@
1+{-# LANGUAGE NoImplicitPrelude #-}
12{- |
23 Module : Main
3− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
45 License : GNU GPL, version 2 or above
56
67 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -11,6 +12,7 @@ Parses command-line options and calls the appropriate readers and
1112writers.
1213-}
1314module Main where
15+import Prelude
1416import qualified Control.Exception as E
1517import Text.Pandoc.App (convertWithOpts, defaultOpts, options, parseOptions)
1618import Text.Pandoc.Error (handleError)
release.nixdeleted−4
@@ -1,4 +0,0 @@
1−let
2− pkgs = import <nixpkgs> { };
3−in
4− pkgs.haskellPackages.callPackage ./project.nix { }
shell.nixdeleted−106
@@ -1,106 +0,0 @@
1−{nixpkgs ? import <nixpkgs> {} }:
2−let
3− inherit (nixpkgs) pkgs;
4− inherit (pkgs) haskellPackages;
5−
6− haskellDeps = ps: with ps; [
7− Diff
8− Glob
9− HTTP
10− HTTP
11− HsYAML
12− JuicyPixels
13− QuickCheck
14− SHA
15− aeson
16− aeson-pretty
17− attoparsec
18− base
19− base-compat
20− base64-bytestring
21− binary
22− blaze-html
23− blaze-markup
24− bytestring
25− case-insensitive
26− citeproc
27− commonmark
28− commonmark-extensions
29− commonmark-pandoc
30− conduit-extra
31− connection
32− connection
33− containers
34− data-default
35− deepseq
36− directory
37− doclayout
38− doctemplates
39− emojis
40− exceptions
41− file-embed
42− filepath
43− haddock-library
44− hsc2hs
45− hslua
46− hslua-module-system
47− hslua-module-text
48− http-client
49− http-client
50− http-client-tls
51− http-client-tls
52− http-types
53− ipynb
54− jira-wiki-markup
55− mtl
56− network
57− network
58− network-uri
59− pandoc-types
60− parsec
61− process
62− random
63− safe
64− scientific
65− skylighting
66− skylighting-core
67− socks
68− split
69− streaming-commons
70− syb
71− tagsoup
72− tasty
73− tasty-golden
74− tasty-hunit
75− tasty-lua
76− tasty-quickcheck
77− temporary
78− texmath
79− text
80− text-conversions
81− time
82− tls
83− unicode-transforms
84− unordered-containers
85− weigh
86− xml
87− xml-conduit
88− zip-archive
89− zlib
90− ];
91−
92− ghc = haskellPackages.ghcWithPackages haskellDeps;
93−
94− nixPackages = [
95− pkgs.zlib
96− ghc
97− pkgs.gdb
98− haskellPackages.ghcid
99− haskellPackages.cabal2nix
100− haskellPackages.cabal-install
101− ];
102−in
103−pkgs.stdenv.mkDerivation {
104− name = "env";
105− buildInputs = nixPackages;
106−}
src/Text/Pandoc.hs+1−1
@@ -3,7 +3,7 @@
33{-# LANGUAGE ScopedTypeVariables #-}
44{- |
55 Module : Text.Pandoc
6− Copyright : Copyright (C) 2006-2021 John MacFarlane
6+ Copyright : Copyright (C) 2006-2020 John MacFarlane
77 License : GNU GPL, version 2 or above
88
99 Maintainer : John MacFarlane <jgm@berkeley.edu>
src/Text/Pandoc/App.hs+46−69
@@ -1,10 +1,9 @@
1−{-# LANGUAGE LambdaCase #-}
21{-# LANGUAGE OverloadedStrings #-}
32{-# LANGUAGE CPP #-}
43{-# LANGUAGE ScopedTypeVariables #-}
54{- |
65 Module : Text.Pandoc.App
7− Copyright : Copyright (C) 2006-2021 John MacFarlane
6+ Copyright : Copyright (C) 2006-2020 John MacFarlane
87 License : GNU GPL, version 2 or above
98
109 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -20,16 +19,14 @@ module Text.Pandoc.App (
2019 , Filter(..)
2120 , defaultOpts
2221 , parseOptions
23− , parseOptionsFromArgs
2422 , options
2523 , applyFilters
2624 ) where
2725import qualified Control.Exception as E
28−import Control.Monad ( (>=>), when )
29−import Control.Monad.Trans ( MonadIO(..) )
26+import Control.Monad
27+import Control.Monad.Trans
3028import Control.Monad.Except (throwError)
3129import qualified Data.ByteString as BS
32−import qualified Data.ByteString.Char8 as B8
3330import qualified Data.ByteString.Lazy as BL
3431import Data.Char (toLower)
3532import Data.Maybe (fromMaybe, isJust, isNothing)
@@ -43,25 +40,23 @@ import qualified Data.Text.Encoding.Error as TSE
4340import Network.URI (URI (..), parseURI)
4441import System.Directory (doesDirectoryExist)
4542import System.Exit (exitSuccess)
46−import System.FilePath ( takeBaseName, takeExtension )
43+import System.FilePath
4744import System.IO (nativeNewline, stdout)
4845import qualified System.IO as IO (Newline (..))
4946import Text.Pandoc
50−import Text.Pandoc.Builder (setMeta)
51−import Text.Pandoc.MIME (getCharset)
5247import Text.Pandoc.App.FormatHeuristics (formatFromFilePaths)
5348import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), defaultOpts,
54− IpynbOutput (..))
55−import Text.Pandoc.App.CommandLineOptions (parseOptions, parseOptionsFromArgs,
56− options)
49+ IpynbOutput (..) )
50+import Text.Pandoc.App.CommandLineOptions (parseOptions, options)
5751import Text.Pandoc.App.OutputSettings (OutputSettings (..), optToOutputSettings)
58−import Text.Collate.Lang (Lang (..), parseLang)
52+import Text.Pandoc.BCP47 (Lang (..), parseBCP47)
53+import Text.Pandoc.Builder (setMeta)
5954import Text.Pandoc.Filter (Filter (JSONFilter, LuaFilter), applyFilters)
6055import Text.Pandoc.PDF (makePDF)
61−import Text.Pandoc.SelfContained (makeSelfContained)
56+import Text.Pandoc.SelfContained (makeDataURI, makeSelfContained)
6257import Text.Pandoc.Shared (eastAsianLineBreakFilter, stripEmptyParagraphs,
6358 headerShift, isURI, tabFilter, uriPathToPath, filterIpynbOutput,
64− defaultUserDataDir, tshow)
59+ defaultUserDataDirs, tshow, findM)
6560import Text.Pandoc.Writers.Shared (lookupMetaString)
6661import Text.Pandoc.Readers.Markdown (yamlToMeta)
6762import qualified Text.Pandoc.UTF8 as UTF8
@@ -72,29 +67,25 @@ import System.Posix.Terminal (queryTerminal)
7267
7368convertWithOpts :: Opt -> IO ()
7469convertWithOpts opts = do
75− datadir <- case optDataDir opts of
76− Nothing -> do
77− d <- defaultUserDataDir
78− exists <- doesDirectoryExist d
79− return $ if exists
80− then Just d
81− else Nothing
82− Just _ -> return $ optDataDir opts
83−
8470 let outputFile = fromMaybe "-" (optOutputFile opts)
8571 let filters = optFilters opts
8672 let verbosity = optVerbosity opts
8773
8874 when (optDumpArgs opts) $
89− do UTF8.hPutStrLn stdout (T.pack outputFile)
90− mapM_ (UTF8.hPutStrLn stdout . T.pack)
91− (fromMaybe ["-"] $ optInputFiles opts)
75+ do UTF8.hPutStrLn stdout outputFile
76+ mapM_ (UTF8.hPutStrLn stdout) (fromMaybe ["-"] $ optInputFiles opts)
9277 exitSuccess
9378
9479 let sources = case optInputFiles opts of
9580 Just xs | not (optIgnoreArgs opts) -> xs
9681 _ -> ["-"]
9782
83+ datadir <- case optDataDir opts of
84+ Nothing -> do
85+ ds <- defaultUserDataDirs
86+ findM doesDirectoryExist ds
87+ Just _ -> return $ optDataDir opts
88+
9889 let runIO' :: PandocIO a -> IO a
9990 runIO' f = do
10091 (res, reports) <- runIOorExplode $ do
@@ -160,11 +151,9 @@ convertWithOpts opts = do
160151 else optTabStop opts)
161152
162153
163− let readSources :: [FilePath] -> PandocIO [(FilePath, Text)]
164− readSources srcs =
165− mapM (\fp -> do
166− t <- readSource fp
167− return (if fp == "-" then "" else fp, convertTabs t)) srcs
154+ let readSources :: [FilePath] -> PandocIO Text
155+ readSources srcs = convertTabs . T.intercalate (T.pack "\n") <$>
156+ mapM readSource srcs
168157
169158
170159 outputSettings <- optToOutputSettings opts
@@ -201,9 +190,20 @@ convertWithOpts opts = do
201190 Nothing -> readDataFile "abbreviations"
202191 Just f -> readFileStrict f
203192
193+ metadata <- if format == "jats" &&
194+ isNothing (lookupMeta "csl" (optMetadata opts)) &&
195+ isNothing (lookupMeta "citation-style"
196+ (optMetadata opts))
197+ then do
198+ jatsCSL <- readDataFile "jats.csl"
199+ let jatsEncoded = makeDataURI
200+ ("application/xml", jatsCSL)
201+ return $ setMeta "csl" jatsEncoded $ optMetadata opts
202+ else return $ optMetadata opts
203+
204204 case lookupMetaString "lang" (optMetadata opts) of
205− "" -> setTranslations $ Lang "en" Nothing (Just "US") [] [] []
206− l -> case parseLang l of
205+ "" -> setTranslations $ Lang "en" "" "US" []
206+ l -> case parseBCP47 l of
207207 Left _ -> report $ InvalidLang l
208208 Right l' -> setTranslations l'
209209
@@ -257,17 +257,13 @@ convertWithOpts opts = do
257257 let sourceToDoc :: [FilePath] -> PandocIO Pandoc
258258 sourceToDoc sources' =
259259 case reader of
260− TextReader r
261− | readerNameBase == "json" ->
262− mconcat <$> mapM (readSource >=> r readerOpts) sources'
263− | optFileScope opts ->
264− -- Read source and convert tabs (see #6709)
265− let readSource' = fmap convertTabs . readSource
266− in mconcat <$> mapM (readSource' >=> r readerOpts) sources'
267− | otherwise ->
268− readSources sources' >>= r readerOpts
269− ByteStringReader r ->
270− mconcat <$> mapM (readFile' >=> r readerOpts) sources'
260+ TextReader r
261+ | optFileScope opts || readerNameBase == "json" ->
262+ mconcat <$> mapM (readSource >=> r readerOpts) sources'
263+ | otherwise ->
264+ readSources sources' >>= r readerOpts
265+ ByteStringReader r ->
266+ mconcat <$> mapM (readFile' >=> r readerOpts) sources'
271267
272268
273269 when (readerNameBase == "markdown_github" ||
@@ -285,21 +281,12 @@ convertWithOpts opts = do
285281 report $ Deprecated "pandoc-citeproc filter"
286282 "Use --citeproc instead."
287283
288− let cslMetadata =
289− maybe id (setMeta "csl") (optCSL opts) .
290− (case optBibliography opts of
291− [] -> id
292− xs -> setMeta "bibliography" xs) .
293− maybe id (setMeta "citation-abbreviations")
294− (optCitationAbbreviations opts) $ mempty
295−
296284 doc <- sourceToDoc sources >>=
297285 ( (if isJust (optExtractMedia opts)
298286 then fillMediaBag
299287 else return)
300288 >=> return . adjustMetadata (metadataFromFile <>)
301− >=> return . adjustMetadata (<> optMetadata opts)
302− >=> return . adjustMetadata (<> cslMetadata)
289+ >=> return . adjustMetadata (<> metadata)
303290 >=> applyTransforms transforms
304291 >=> applyFilters readerOpts filters [T.unpack format]
305292 >=> maybe return extractMedia (optExtractMedia opts)
@@ -366,18 +353,7 @@ readSource src = case parseURI src of
366353 _ -> PandocAppError (tshow e))
367354
368355readURI :: FilePath -> PandocIO Text
369−readURI src = do
370− (bs, mt) <- openURL (T.pack src)
371− case mt >>= getCharset of
372− Just "UTF-8" -> return $ UTF8.toText bs
373− Just "ISO-8859-1" -> return $ T.pack $ B8.unpack bs
374− Just charset -> throwError $ PandocUnsupportedCharsetError charset
375− Nothing -> liftIO $ -- try first as UTF-8, then as latin1
376− E.catch (return $! UTF8.toText bs)
377− (\case
378− TSE.DecodeError{} ->
379− return $ T.pack $ B8.unpack bs
380− e -> E.throwIO e)
356+readURI src = UTF8.toText . fst <$> openURL (T.pack src)
381357
382358readFile' :: MonadIO m => FilePath -> m BL.ByteString
383359readFile' "-" = liftIO BL.getContents
@@ -388,5 +364,6 @@ writeFnBinary "-" = liftIO . BL.putStr
388364writeFnBinary f = liftIO . BL.writeFile (UTF8.encodePath f)
389365
390366writerFn :: MonadIO m => IO.Newline -> FilePath -> Text -> m ()
391−writerFn eol "-" = liftIO . UTF8.putStrWith eol
392−writerFn eol f = liftIO . UTF8.writeFileWith eol f
367+-- TODO this implementation isn't maximally efficient:
368+writerFn eol "-" = liftIO . UTF8.putStrWith eol . T.unpack
369+writerFn eol f = liftIO . UTF8.writeFileWith eol f . T.unpack
src/Text/Pandoc/App/CommandLineOptions.hs+51−64
@@ -6,7 +6,7 @@
66{-# LANGUAGE FlexibleContexts #-}
77{- |
88 Module : Text.Pandoc.App.CommandLineOptions
9− Copyright : Copyright (C) 2006-2021 John MacFarlane
9+ Copyright : Copyright (C) 2006-2020 John MacFarlane
1010 License : GNU GPL, version 2 or above
1111
1212 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -17,7 +17,6 @@ Does a pandoc conversion based on command-line options.
1717-}
1818module Text.Pandoc.App.CommandLineOptions (
1919 parseOptions
20− , parseOptionsFromArgs
2120 , options
2221 , engines
2322 , lookupHighlightStyle
@@ -26,12 +25,11 @@ module Text.Pandoc.App.CommandLineOptions (
2625import Control.Monad
2726import Control.Monad.Trans
2827import Control.Monad.Except (throwError)
29−import Control.Monad.State.Strict
3028import Data.Aeson.Encode.Pretty (encodePretty', Config(..), keyOrder,
3129 defConfig, Indent(..), NumberFormat(..))
3230import Data.Bifunctor (second)
3331import Data.Char (toLower)
34−import Data.List (intercalate, sort, foldl')
32+import Data.List (intercalate, sort)
3533#ifdef _WINDOWS
3634#if MIN_VERSION_base(4,12,0)
3735import Data.List (isPrefixOf)
@@ -48,13 +46,10 @@ import System.FilePath
4846import System.IO (stdout)
4947import Text.DocTemplates (Context (..), ToContext (toVal), Val (..))
5048import Text.Pandoc
51−import Text.Pandoc.Builder (setMeta)
52−import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), IpynbOutput (..),
53− DefaultsState (..), applyDefaults,
54− fullDefaultsPath)
49+import Text.Pandoc.App.Opt (Opt (..), LineEnding (..), IpynbOutput (..), addMeta)
5550import Text.Pandoc.Filter (Filter (..))
5651import Text.Pandoc.Highlighting (highlightingStyles)
57−import Text.Pandoc.Shared (ordNub, elemText, safeStrRead, defaultUserDataDir)
52+import Text.Pandoc.Shared (ordNub, elemText, safeStrRead, defaultUserDataDirs, findM)
5853import Text.Printf
5954
6055#ifdef EMBED_DATA_FILES
@@ -69,19 +64,16 @@ import qualified Data.ByteString as BS
6964import qualified Data.ByteString.Lazy as B
7065import qualified Data.Map as M
7166import qualified Data.Text as T
67+import qualified Data.YAML as Y
7268import qualified Text.Pandoc.UTF8 as UTF8
7369
7470parseOptions :: [OptDescr (Opt -> IO Opt)] -> Opt -> IO Opt
7571parseOptions options' defaults = do
7672 rawArgs <- map UTF8.decodeArg <$> getArgs
7773 prg <- getProgName
78− parseOptionsFromArgs options' defaults prg rawArgs
7974
80−parseOptionsFromArgs
81− :: [OptDescr (Opt -> IO Opt)] -> Opt -> String -> [String] -> IO Opt
82−parseOptionsFromArgs options' defaults prg rawArgs = do
8375 let (actions, args, unrecognizedOpts, errors) =
84− getOpt' Permute options' (map UTF8.decodeArg rawArgs)
76+ getOpt' Permute options' rawArgs
8577
8678 let unknownOptionErrors =
8779 foldr (handleUnrecognizedOption . takeWhile (/= '=')) []
@@ -93,7 +85,7 @@ parseOptionsFromArgs options' defaults prg rawArgs = do
9385 ("Try " ++ prg ++ " --help for more information.")
9486
9587 -- thread option data structure through all supplied option actions
96− opts <- foldl' (>>=) (return defaults) actions
88+ opts <- foldl (>>=) (return defaults) actions
9789 let mbArgs = case args of
9890 [] -> Nothing
9991 xs -> Just xs
@@ -174,11 +166,7 @@ options =
174166
175167 , Option "d" ["defaults"]
176168 (ReqArg
177− (\arg opt -> runIOorExplode $ do
178− let defsState = DefaultsState { curDefaults = Nothing,
179− inheritanceGraph = [] }
180− fp <- fullDefaultsPath (optDataDir opt) arg
181− evalStateT (applyDefaults opt fp) defsState
169+ (\arg opt -> applyDefaults opt arg
182170 )
183171 "FILE")
184172 ""
@@ -288,8 +276,7 @@ options =
288276 , Option "" ["resource-path"]
289277 (ReqArg
290278 (\arg opt -> return opt { optResourcePath =
291− splitSearchPath arg ++
292− optResourcePath opt })
279+ splitSearchPath arg })
293280 "SEARCHPATH")
294281 "" -- "Paths to search for images and other resources"
295282
@@ -814,10 +801,10 @@ options =
814801 map (\c -> ['-',c]) shorts ++
815802 map ("--" ++) longs
816803 let allopts = unwords (concatMap optnames options)
817− UTF8.hPutStrLn stdout $ T.pack $ printf tpl allopts
818− (T.unpack $ T.unwords readersNames)
819− (T.unpack $ T.unwords writersNames)
820− (T.unpack $ T.unwords $ map fst highlightingStyles)
804+ UTF8.hPutStrLn stdout $ printf tpl allopts
805+ (unwords readersNames)
806+ (unwords writersNames)
807+ (unwords $ map (T.unpack . fst) highlightingStyles)
821808 (unwords datafiles)
822809 exitSuccess ))
823810 "" -- "Print bash completion script"
@@ -856,7 +843,7 @@ options =
856843 else if extensionEnabled x allExts
857844 then '-'
858845 else ' ') : drop 4 (show x)
859− mapM_ (UTF8.hPutStrLn stdout . T.pack . showExt)
846+ mapM_ (UTF8.hPutStrLn stdout . showExt)
860847 [ex | ex <- extList, extensionEnabled ex allExts]
861848 exitSuccess )
862849 "FORMAT")
@@ -870,14 +857,14 @@ options =
870857 , sShortname s `notElem`
871858 [T.pack "Alert", T.pack "Alert_indent"]
872859 ]
873− mapM_ (UTF8.hPutStrLn stdout . T.pack) (sort langs)
860+ mapM_ (UTF8.hPutStrLn stdout) (sort langs)
874861 exitSuccess ))
875862 ""
876863
877864 , Option "" ["list-highlight-styles"]
878865 (NoArg
879866 (\_ -> do
880− mapM_ (UTF8.hPutStrLn stdout . fst) highlightingStyles
867+ mapM_ (UTF8.hPutStrLn stdout . T.unpack . fst) highlightingStyles
881868 exitSuccess ))
882869 ""
883870
@@ -895,7 +882,7 @@ options =
895882 | T.null t -> -- e.g. for docx, odt, json:
896883 E.throwIO $ PandocCouldNotFindDataFileError $ T.pack
897884 ("templates/default." ++ arg)
898− | otherwise -> write t
885+ | otherwise -> write . T.unpack $ t
899886 Left e -> E.throwIO e
900887 exitSuccess)
901888 "FORMAT")
@@ -941,13 +928,12 @@ options =
941928 (NoArg
942929 (\_ -> do
943930 prg <- getProgName
944− defaultDatadir <- defaultUserDataDir
945− UTF8.hPutStrLn stdout
946− $ T.pack
947− $ prg ++ " " ++ T.unpack pandocVersion ++
948− compileInfo ++
949− "\nUser data directory: " ++ defaultDatadir ++
950− ('\n':copyrightMessage)
931+ defaultDatadirs <- defaultUserDataDirs
932+ UTF8.hPutStrLn stdout (prg ++ " " ++ T.unpack pandocVersion ++
933+ compileInfo ++
934+ "\nUser data directory: " ++
935+ intercalate " or " defaultDatadirs ++
936+ ('\n':copyrightMessage))
951937 exitSuccess ))
952938 "" -- "Print version"
953939
@@ -955,7 +941,7 @@ options =
955941 (NoArg
956942 (\_ -> do
957943 prg <- getProgName
958− UTF8.hPutStr stdout (T.pack $ usageMessage prg options)
944+ UTF8.hPutStr stdout (usageMessage prg options)
959945 exitSuccess ))
960946 "" -- "Show help"
961947 ]
@@ -976,7 +962,7 @@ usageMessage programName = usageInfo (programName ++ " [OPTIONS] [FILES]")
976962
977963copyrightMessage :: String
978964copyrightMessage = intercalate "\n" [
979− "Copyright (C) 2006-2021 John MacFarlane. Web: https://pandoc.org",
965+ "Copyright (C) 2006-2020 John MacFarlane. Web: https://pandoc.org",
980966 "This is free software; see the source for copying conditions. There is no",
981967 "warranty, not even for merchantability or fitness for a particular purpose." ]
982968
@@ -1016,16 +1002,38 @@ handleUnrecognizedOption "-R" = handleUnrecognizedOption "--parse-raw"
10161002handleUnrecognizedOption x =
10171003 (("Unknown option " ++ x ++ ".") :)
10181004
1019−readersNames :: [Text]
1020−readersNames = sort (map fst (readers :: [(Text, Reader PandocIO)]))
1005+readersNames :: [String]
1006+readersNames = sort (map (T.unpack . fst) (readers :: [(Text, Reader PandocIO)]))
10211007
1022−writersNames :: [Text]
1008+writersNames :: [String]
10231009writersNames = sort
1024− ("pdf" : map fst (writers :: [(Text, Writer PandocIO)]))
1010+ ("pdf" : map (T.unpack . fst) (writers :: [(Text, Writer PandocIO)]))
10251011
10261012splitField :: String -> (String, String)
10271013splitField = second (tailDef "true") . break (`elemText` ":=")
10281014
1015+-- | Apply defaults from --defaults file.
1016+applyDefaults :: Opt -> FilePath -> IO Opt
1017+applyDefaults opt file = runIOorExplode $ do
1018+ let fp = if null (takeExtension file)
1019+ then addExtension file "yaml"
1020+ else file
1021+ setVerbosity $ optVerbosity opt
1022+ dataDirs <- liftIO defaultUserDataDirs
1023+ let fps = fp : case optDataDir opt of
1024+ Nothing -> map (</> ("defaults" </> fp))
1025+ dataDirs
1026+ Just dd -> [dd </> "defaults" </> fp]
1027+ fp' <- fromMaybe fp <$> findM fileExists fps
1028+ inp <- readFileLazy fp'
1029+ case Y.decode1 inp of
1030+ Right (f :: Opt -> Opt) -> return $ f opt
1031+ Left (errpos, errmsg) -> throwError $
1032+ PandocParseError $ T.pack $
1033+ "Error parsing " ++ fp' ++ " line " ++
1034+ show (Y.posLine errpos) ++ " column " ++
1035+ show (Y.posColumn errpos) ++ ":\n" ++ errmsg
1036+
10291037lookupHighlightStyle :: PandocMonad m => String -> m Style
10301038lookupHighlightStyle s
10311039 | takeExtension s == ".theme" = -- attempt to load KDE theme
@@ -1054,27 +1062,6 @@ setVariable key val (Context ctx) = Context $ M.alter go key ctx
10541062 go (Just (ListVal xs)) = Just $ ListVal $ xs ++ [toVal val]
10551063 go (Just x) = Just $ ListVal [x, toVal val]
10561064
1057−addMeta :: String -> String -> Meta -> Meta
1058−addMeta k v meta =
1059− case lookupMeta k' meta of
1060− Nothing -> setMeta k' v' meta
1061− Just (MetaList xs) ->
1062− setMeta k' (MetaList (xs ++ [v'])) meta
1063− Just x -> setMeta k' (MetaList [x, v']) meta
1064− where
1065− v' = readMetaValue v
1066− k' = T.pack k
1067−
1068−readMetaValue :: String -> MetaValue
1069−readMetaValue s
1070− | s == "true" = MetaBool True
1071− | s == "True" = MetaBool True
1072− | s == "TRUE" = MetaBool True
1073− | s == "false" = MetaBool False
1074− | s == "False" = MetaBool False
1075− | s == "FALSE" = MetaBool False
1076− | otherwise = MetaString $ T.pack s
1077−
10781065-- On Windows with ghc 8.6+, we need to rewrite paths
10791066-- beginning with \\ to \\?\UNC\. -- See #5127.
10801067normalizePath :: FilePath -> FilePath
src/Text/Pandoc/App/FormatHeuristics.hs+8−19
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.App.FormatHeuristics
4− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -15,24 +15,18 @@ module Text.Pandoc.App.FormatHeuristics
1515 ) where
1616
1717import Data.Char (toLower)
18−import Data.Foldable (asum)
1918import Data.Text (Text)
2019import System.FilePath (takeExtension)
2120
22−-- | Determines default format based on file extensions; uses the format
23−-- of the first extension that's associated with a format.
24−--
25−-- Examples:
26−--
27−-- > formatFromFilePaths ["text.unknown", "no-extension"]
28−-- Nothing
29−--
30−-- > formatFromFilePaths ["my.md", "other.rst"]
31−-- Just "markdown"
21+-- Determine default format based on file extensions.
3222formatFromFilePaths :: [FilePath] -> Maybe Text
33−formatFromFilePaths = asum . map formatFromFilePath
23+formatFromFilePaths [] = Nothing
24+formatFromFilePaths (x:xs) =
25+ case formatFromFilePath x of
26+ Just f -> Just f
27+ Nothing -> formatFromFilePaths xs
3428
35−-- | Determines format based on file extension.
29+-- Determine format based on file extension
3630formatFromFilePath :: FilePath -> Maybe Text
3731formatFromFilePath x =
3832 case takeExtension (map toLower x) of
@@ -54,11 +48,6 @@ formatFromFilePath x =
5448 ".lhs" -> Just "markdown+lhs"
5549 ".ltx" -> Just "latex"
5650 ".markdown" -> Just "markdown"
57− ".mkdn" -> Just "markdown"
58− ".mkd" -> Just "markdown"
59− ".mdwn" -> Just "markdown"
60− ".mdown" -> Just "markdown"
61− ".Rmd" -> Just "markdown"
6251 ".md" -> Just "markdown"
6352 ".ms" -> Just "ms"
6453 ".muse" -> Just "muse"
src/Text/Pandoc/App/Opt.hs+51−261
@@ -7,7 +7,7 @@
77{-# LANGUAGE FlexibleContexts #-}
88{- |
99 Module : Text.Pandoc.App.Opt
10− Copyright : Copyright (C) 2006-2021 John MacFarlane
10+ Copyright : Copyright (C) 2006-2020 John MacFarlane
1111 License : GNU GPL, version 2 or above
1212
1313 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -20,31 +20,21 @@ module Text.Pandoc.App.Opt (
2020 Opt(..)
2121 , LineEnding (..)
2222 , IpynbOutput (..)
23− , DefaultsState (..)
2423 , defaultOpts
25− , applyDefaults
26− , fullDefaultsPath
24+ , addMeta
2725 ) where
28−import Control.Monad.Except (MonadIO, liftIO, throwError, (>=>), foldM)
29−import Control.Monad.State.Strict (StateT, modify, gets)
30−import System.FilePath ( addExtension, (</>), takeExtension, takeDirectory )
31−import System.Directory ( canonicalizePath )
3226import Data.Char (isLower, toLower)
33−import Data.Maybe (fromMaybe)
3427import GHC.Generics hiding (Meta)
28+import Text.Pandoc.Builder (setMeta)
3529import Text.Pandoc.Filter (Filter (..))
36−import Text.Pandoc.Logging (Verbosity (WARNING), LogMessage(..))
30+import Text.Pandoc.Logging (Verbosity (WARNING))
3731import Text.Pandoc.Options (TopLevelDivision (TopLevelDefault),
3832 TrackChanges (AcceptChanges),
3933 WrapOption (WrapAuto), HTMLMathMethod (PlainMath),
4034 ReferenceLocation (EndOfDocument),
4135 ObfuscationMethod (NoObfuscation),
4236 CiteMethod (Citeproc))
43−import Text.Pandoc.Class (readFileLazy, fileExists, setVerbosity, report,
44− PandocMonad(lookupEnv), getUserDataDir)
45−import Text.Pandoc.Error (PandocError (PandocParseError, PandocSomeError))
46−import Text.Pandoc.Shared (camelCaseStrToHyphenated, defaultUserDataDir,
47− findM, ordNub)
37+import Text.Pandoc.Shared (camelCaseStrToHyphenated)
4838import qualified Text.Pandoc.Parsing as P
4939import Text.Pandoc.Readers.Metadata (yamlMap)
5040import Text.Pandoc.Class.PandocPure
@@ -53,7 +43,7 @@ import Data.Text (Text, unpack)
5343import Data.Default (def)
5444import qualified Data.Text as T
5545import qualified Data.Map as M
56−import Text.Pandoc.Definition (Meta(..), MetaValue(..))
46+import Text.Pandoc.Definition (Meta(..), MetaValue(..), lookupMeta)
5747import Data.Aeson (defaultOptions, Options(..))
5848import Data.Aeson.TH (deriveJSON)
5949import Control.Applicative ((<|>))
@@ -157,194 +147,19 @@ data Opt = Opt
157147 , optNoCheckCertificate :: Bool -- ^ Disable certificate validation
158148 , optEol :: LineEnding -- ^ Style of line-endings to use
159149 , optStripComments :: Bool -- ^ Skip HTML comments
160− , optCSL :: Maybe FilePath -- ^ CSL stylesheet
161− , optBibliography :: [FilePath] -- ^ Bibliography files
162− , optCitationAbbreviations :: Maybe FilePath -- ^ Citation abbreviations
163150 } deriving (Generic, Show)
164151
165152instance FromYAML (Opt -> Opt) where
166− parseYAML (Mapping _ _ m) = chain doOpt (M.toList m)
153+ parseYAML (Mapping _ _ m) =
154+ foldr (.) id <$> mapM doOpt (M.toList m)
167155 parseYAML n = failAtNode n "Expected a mapping"
168156
169−data DefaultsState = DefaultsState
170− {
171− curDefaults :: Maybe FilePath -- currently parsed file
172− , inheritanceGraph :: [[FilePath]] -- defaults file inheritance graph
173− } deriving (Show)
174−
175−instance (PandocMonad m, MonadIO m)
176− => FromYAML (Opt -> StateT DefaultsState m Opt) where
177− parseYAML (Mapping _ _ m) = do
178− let opts = M.mapKeys toText m
179− dataDir <- case M.lookup "data-dir" opts of
180− Nothing -> return Nothing
181− Just v -> Just . unpack <$> parseYAML v
182− f <- parseOptions (M.toList m)
183− case M.lookup "defaults" opts of
184− Just v -> do
185− g <- parseDefaults v dataDir
186− return $ g >=> f >=> resolveVarsInOpt
187− Nothing -> return $ f >=> resolveVarsInOpt
188− where
189− toText (Scalar _ (SStr s)) = s
190− toText _ = ""
191− parseYAML n = failAtNode n "Expected a mapping"
192−
193−resolveVarsInOpt :: forall m. (PandocMonad m, MonadIO m)
194− => Opt -> StateT DefaultsState m Opt
195−resolveVarsInOpt
196− opt@Opt
197− { optTemplate = oTemplate
198− , optMetadataFiles = oMetadataFiles
199− , optOutputFile = oOutputFile
200− , optInputFiles = oInputFiles
201− , optSyntaxDefinitions = oSyntaxDefinitions
202− , optAbbreviations = oAbbreviations
203− , optReferenceDoc = oReferenceDoc
204− , optEpubMetadata = oEpubMetadata
205− , optEpubFonts = oEpubFonts
206− , optEpubCoverImage = oEpubCoverImage
207− , optLogFile = oLogFile
208− , optFilters = oFilters
209− , optDataDir = oDataDir
210− , optExtractMedia = oExtractMedia
211− , optCss = oCss
212− , optIncludeBeforeBody = oIncludeBeforeBody
213− , optIncludeAfterBody = oIncludeAfterBody
214− , optIncludeInHeader = oIncludeInHeader
215− , optResourcePath = oResourcePath
216− , optCSL = oCSL
217− , optBibliography = oBibliography
218− , optCitationAbbreviations = oCitationAbbreviations
219− }
220− = do
221− oTemplate' <- mapM resolveVars oTemplate
222− oMetadataFiles' <- mapM resolveVars oMetadataFiles
223− oOutputFile' <- mapM resolveVars oOutputFile
224− oInputFiles' <- mapM (mapM resolveVars) oInputFiles
225− oSyntaxDefinitions' <- mapM resolveVars oSyntaxDefinitions
226− oAbbreviations' <- mapM resolveVars oAbbreviations
227− oReferenceDoc' <- mapM resolveVars oReferenceDoc
228− oEpubMetadata' <- mapM resolveVars oEpubMetadata
229− oEpubFonts' <- mapM resolveVars oEpubFonts
230− oEpubCoverImage' <- mapM resolveVars oEpubCoverImage
231− oLogFile' <- mapM resolveVars oLogFile
232− oFilters' <- mapM resolveVarsInFilter oFilters
233− oDataDir' <- mapM resolveVars oDataDir
234− oExtractMedia' <- mapM resolveVars oExtractMedia
235− oCss' <- mapM resolveVars oCss
236− oIncludeBeforeBody' <- mapM resolveVars oIncludeBeforeBody
237− oIncludeAfterBody' <- mapM resolveVars oIncludeAfterBody
238− oIncludeInHeader' <- mapM resolveVars oIncludeInHeader
239− oResourcePath' <- mapM resolveVars oResourcePath
240− oCSL' <- mapM resolveVars oCSL
241− oBibliography' <- mapM resolveVars oBibliography
242− oCitationAbbreviations' <- mapM resolveVars oCitationAbbreviations
243− return opt{ optTemplate = oTemplate'
244− , optMetadataFiles = oMetadataFiles'
245− , optOutputFile = oOutputFile'
246− , optInputFiles = oInputFiles'
247− , optSyntaxDefinitions = oSyntaxDefinitions'
248− , optAbbreviations = oAbbreviations'
249− , optReferenceDoc = oReferenceDoc'
250− , optEpubMetadata = oEpubMetadata'
251− , optEpubFonts = oEpubFonts'
252− , optEpubCoverImage = oEpubCoverImage'
253− , optLogFile = oLogFile'
254− , optFilters = oFilters'
255− , optDataDir = oDataDir'
256− , optExtractMedia = oExtractMedia'
257− , optCss = oCss'
258− , optIncludeBeforeBody = oIncludeBeforeBody'
259− , optIncludeAfterBody = oIncludeAfterBody'
260− , optIncludeInHeader = oIncludeInHeader'
261− , optResourcePath = oResourcePath'
262− , optCSL = oCSL'
263− , optBibliography = oBibliography'
264− , optCitationAbbreviations = oCitationAbbreviations'
265− }
266−
267− where
268− resolveVars :: FilePath -> StateT DefaultsState m FilePath
269− resolveVars [] = return []
270− resolveVars ('$':'{':xs) =
271− let (ys, zs) = break (=='}') xs
272− in if null zs
273− then return $ '$':'{':xs
274− else do
275− val <- lookupEnv' ys
276− (val ++) <$> resolveVars (drop 1 zs)
277− resolveVars (c:cs) = (c:) <$> resolveVars cs
278− lookupEnv' :: String -> StateT DefaultsState m String
279− lookupEnv' "." = do
280− mbCurDefaults <- gets curDefaults
281− maybe (return "")
282− (fmap takeDirectory . liftIO . canonicalizePath)
283− mbCurDefaults
284− lookupEnv' "USERDATA" = do
285− mbodatadir <- mapM resolveVars oDataDir
286− mbdatadir <- getUserDataDir
287− defdatadir <- liftIO defaultUserDataDir
288− return $ fromMaybe defdatadir (mbodatadir <|> mbdatadir)
289− lookupEnv' v = do
290− mbval <- fmap T.unpack <$> lookupEnv (T.pack v)
291− case mbval of
292− Nothing -> do
293− report $ EnvironmentVariableUndefined (T.pack v)
294− return mempty
295− Just x -> return x
296− resolveVarsInFilter (JSONFilter fp) =
297− JSONFilter <$> resolveVars fp
298− resolveVarsInFilter (LuaFilter fp) =
299− LuaFilter <$> resolveVars fp
300− resolveVarsInFilter CiteprocFilter = return CiteprocFilter
301−
302−
303−
304−parseDefaults :: (PandocMonad m, MonadIO m)
305− => Node Pos
306− -> Maybe FilePath
307− -> Parser (Opt -> StateT DefaultsState m Opt)
308−parseDefaults n dataDir = parseDefsNames n >>= \ds -> return $ \o -> do
309− -- get parent defaults:
310− defsParent <- gets $ fromMaybe "" . curDefaults
311− -- get child defaults:
312− defsChildren <- mapM (fullDefaultsPath dataDir) ds
313− -- expand parent in defaults inheritance graph by children:
314− defsGraph <- gets inheritanceGraph
315− let defsGraphExp = expand defsGraph defsChildren defsParent
316− modify $ \defsState -> defsState{ inheritanceGraph = defsGraphExp }
317− -- check for cyclic inheritance:
318− if cyclic defsGraphExp
319− then throwError $
320− PandocSomeError $ T.pack $
321− "Error: Circular defaults file reference in " ++
322− "'" ++ defsParent ++ "'"
323− else foldM applyDefaults o defsChildren
324− where parseDefsNames x = (parseYAML x >>= \xs -> return $ map unpack xs)
325− <|> (parseYAML x >>= \x' -> return [unpack x'])
326−
327−parseOptions :: Monad m
328− => [(Node Pos, Node Pos)]
329− -> Parser (Opt -> StateT DefaultsState m Opt)
330−parseOptions ns = do
331− f <- chain doOpt' ns
332− return $ return . f
333−
334−chain :: Monad m => (a -> m (b -> b)) -> [a] -> m (b -> b)
335−chain f = foldM g id
336− where g o n = f n >>= \o' -> return $ o' . o
337−
338−doOpt' :: (Node Pos, Node Pos) -> Parser (Opt -> Opt)
339−doOpt' (k',v) = do
340− k <- parseStringKey k'
341− case k of
342− "defaults" -> return id
343− _ -> doOpt (k',v)
344−
345157doOpt :: (Node Pos, Node Pos) -> Parser (Opt -> Opt)
346158doOpt (k',v) = do
347− k <- parseStringKey k'
159+ k <- case k' of
160+ Scalar _ (SStr t) -> return t
161+ Scalar _ _ -> failAtNode k' "Non-string key"
162+ _ -> failAtNode k' "Non-scalar key"
348163 case k of
349164 "tab-stop" ->
350165 parseYAML v >>= \x -> return (\o -> o{ optTabStop = x })
@@ -543,18 +358,26 @@ doOpt (k',v) = do
543358 (parseYAML v >>= \x -> return (\o -> o{ optCss = optCss o <>
544359 [unpack x] }))
545360 "bibliography" ->
546− (parseYAML v >>= \x -> return (\o ->
547− o{ optBibliography = optBibliography o <>
548− map unpack x }))
549− <|>
550− (parseYAML v >>= \x -> return (\o ->
551− o{ optBibliography = optBibliography o <>
552− [unpack x] }))
361+ do let addItem x o = o{ optMetadata =
362+ addMeta "bibliography" (T.unpack x)
363+ (optMetadata o) }
364+ (parseYAML v >>= \(xs :: [Text]) -> return $ \o ->
365+ foldr addItem o xs)
366+ <|>
367+ (parseYAML v >>= \(x :: Text) -> return $ \o -> addItem x o)
553368 "csl" ->
554− parseYAML v >>= \x -> return (\o -> o{ optCSL = unpack <$> x })
369+ do let addItem x o = o{ optMetadata =
370+ addMeta "csl" (T.unpack x)
371+ (optMetadata o) }
372+ (parseYAML v >>= \(xs :: [Text]) -> return $ \o ->
373+ foldr addItem o xs)
374+ <|>
375+ (parseYAML v >>= \(x :: Text) -> return $ \o -> addItem x o)
555376 "citation-abbreviations" ->
556− parseYAML v >>= \x -> return (\o -> o{ optCitationAbbreviations =
557− unpack <$> x })
377+ parseYAML v >>= \x ->
378+ return (\o -> o{ optMetadata =
379+ addMeta "citation-abbreviations" (T.unpack x)
380+ (optMetadata o) })
558381 "ipynb-output" ->
559382 parseYAML v >>= \x -> return (\o -> o{ optIpynbOutput = x })
560383 "include-before-body" ->
@@ -583,8 +406,7 @@ doOpt (k',v) = do
583406 optIncludeInHeader o <> [unpack x] }))
584407 "resource-path" ->
585408 parseYAML v >>= \x ->
586− return (\o -> o{ optResourcePath = map unpack x <>
587− optResourcePath o })
409+ return (\o -> o{ optResourcePath = map unpack x })
588410 "request-headers" ->
589411 parseYAML v >>= \x ->
590412 return (\o -> o{ optRequestHeaders = x })
@@ -670,70 +492,38 @@ defaultOpts = Opt
670492 , optNoCheckCertificate = False
671493 , optEol = Native
672494 , optStripComments = False
673− , optCSL = Nothing
674− , optBibliography = []
675− , optCitationAbbreviations = Nothing
676495 }
677496
678−parseStringKey :: Node Pos -> Parser Text
679−parseStringKey k = case k of
680− Scalar _ (SStr t) -> return t
681− Scalar _ _ -> failAtNode k "Non-string key"
682− _ -> failAtNode k "Non-scalar key"
683−
684497yamlToMeta :: Node Pos -> Parser Meta
685498yamlToMeta (Mapping _ _ m) =
686499 either (fail . show) return $ runEverything (yamlMap pMetaString m)
687500 where
688501 pMetaString = pure . MetaString <$> P.manyChar P.anyChar
689− runEverything p =
690− runPure (P.readWithM p (def :: P.ParserState) ("" :: Text))
502+ runEverything p = runPure (P.readWithM p def "")
691503 >>= fmap (Meta . flip P.runF def)
692504yamlToMeta _ = return mempty
693505
694−-- | Apply defaults from --defaults file.
695−applyDefaults :: (PandocMonad m, MonadIO m)
696− => Opt
697− -> FilePath
698− -> StateT DefaultsState m Opt
699−applyDefaults opt file = do
700− setVerbosity $ optVerbosity opt
701− modify $ \defsState -> defsState{ curDefaults = Just file }
702− inp <- readFileLazy file
703− case decode1 inp of
704− Right f -> f opt
705− Left (errpos, errmsg) -> throwError $
706− PandocParseError $ T.pack $
707− "Error parsing " ++ file ++ " line " ++
708− show (posLine errpos) ++ " column " ++
709− show (posColumn errpos) ++ ":\n" ++ errmsg
506+addMeta :: String -> String -> Meta -> Meta
507+addMeta k v meta =
508+ case lookupMeta k' meta of
509+ Nothing -> setMeta k' v' meta
510+ Just (MetaList xs) ->
511+ setMeta k' (MetaList (xs ++ [v'])) meta
512+ Just x -> setMeta k' (MetaList [x, v']) meta
513+ where
514+ v' = readMetaValue v
515+ k' = T.pack k
710516
711−fullDefaultsPath :: (PandocMonad m, MonadIO m)
712− => Maybe FilePath
713− -> FilePath
714− -> m FilePath
715−fullDefaultsPath dataDir file = do
716− let fp = if null (takeExtension file)
717− then addExtension file "yaml"
718− else file
719− defaultDataDir <- liftIO defaultUserDataDir
720− let defaultFp = fromMaybe defaultDataDir dataDir </> "defaults" </> fp
721− fromMaybe fp <$> findM fileExists [fp, defaultFp]
517+readMetaValue :: String -> MetaValue
518+readMetaValue s
519+ | s == "true" = MetaBool True
520+ | s == "True" = MetaBool True
521+ | s == "TRUE" = MetaBool True
522+ | s == "false" = MetaBool False
523+ | s == "False" = MetaBool False
524+ | s == "FALSE" = MetaBool False
525+ | otherwise = MetaString $ T.pack s
722526
723−-- | In a list of lists, append another list in front of every list which
724−-- starts with specific element.
725−expand :: Ord a => [[a]] -> [a] -> a -> [[a]]
726−expand [] ns n = fmap (\x -> x : [n]) ns
727−expand ps ns n = concatMap (ext n ns) ps
728− where
729− ext x xs p = case p of
730− (l : _) | x == l -> fmap (: p) xs
731− _ -> [p]
732−
733−cyclic :: Ord a => [[a]] -> Bool
734−cyclic = any hasDuplicate
735− where
736− hasDuplicate xs = length (ordNub xs) /= length xs
737527
738528-- see https://github.com/jgm/pandoc/pull/4083
739529-- using generic deriving caused long compilation times
src/Text/Pandoc/App/OutputSettings.hs+3−3
@@ -5,7 +5,7 @@
55{-# LANGUAGE TupleSections #-}
66{- |
77 Module : Text.Pandoc.App
8− Copyright : Copyright (C) 2006-2021 John MacFarlane
8+ Copyright : Copyright (C) 2006-2020 John MacFarlane
99 License : GNU GPL, version 2 or above
1010
1111 Maintainer : John MacFarlane <jgm@berkeley@edu>
@@ -59,8 +59,8 @@ optToOutputSettings opts = do
5959 let outputFile = fromMaybe "-" (optOutputFile opts)
6060
6161 when (optDumpArgs opts) . liftIO $ do
62− UTF8.hPutStrLn stdout (T.pack outputFile)
63− mapM_ (UTF8.hPutStrLn stdout . T.pack) (fromMaybe [] $ optInputFiles opts)
62+ UTF8.hPutStrLn stdout outputFile
63+ mapM_ (UTF8.hPutStrLn stdout) (fromMaybe [] $ optInputFiles opts)
6464 exitSuccess
6565
6666 epubMetadata <- traverse readUtf8File $ optEpubMetadata opts
src/Text/Pandoc/Asciify.hs+391−14
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Asciify
3− Copyright : Copyright (C) 2013-2021 John MacFarlane
3+ Copyright : Copyright (C) 2013-2020 John MacFarlane
44 License : GNU GPL, version 2 or above
55
66 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -10,19 +10,396 @@
1010Function to convert accented latin letters to their unaccented
1111ascii equivalents (used in constructing HTML identifiers).
1212-}
13−module Text.Pandoc.Asciify (toAsciiChar, toAsciiText)
13+module Text.Pandoc.Asciify (toAsciiChar)
1414where
15−import Data.Char (isAscii, isMark)
16−import qualified Data.Text.Normalize as TN
17−import Data.Text (Text)
18−import qualified Data.Text as T
19−
20−toAsciiText :: Text -> Text
21−toAsciiText = T.filter isAscii . TN.normalize (TN.NFD)
15+import Data.Char (isAscii)
16+import qualified Data.Map as M
2217
2318toAsciiChar :: Char -> Maybe Char
24−toAsciiChar c = case T.unpack (TN.normalize TN.NFD (T.singleton c)) of
25− (x:xs) | isAscii x
26− , all isMark xs
27− -> Just x
28− _ -> Nothing
19+toAsciiChar c | isAscii c = Just c
20+ | otherwise = M.lookup c asciiMap
21+
22+asciiMap :: M.Map Char Char
23+asciiMap = M.fromList
24+ [('\192','A')
25+ ,('\193','A')
26+ ,('\194','A')
27+ ,('\195','A')
28+ ,('\196','A')
29+ ,('\197','A')
30+ ,('\199','C')
31+ ,('\200','E')
32+ ,('\201','E')
33+ ,('\202','E')
34+ ,('\203','E')
35+ ,('\204','I')
36+ ,('\205','I')
37+ ,('\206','I')
38+ ,('\207','I')
39+ ,('\209','N')
40+ ,('\210','O')
41+ ,('\211','O')
42+ ,('\212','O')
43+ ,('\213','O')
44+ ,('\214','O')
45+ ,('\217','U')
46+ ,('\218','U')
47+ ,('\219','U')
48+ ,('\220','U')
49+ ,('\221','Y')
50+ ,('\224','a')
51+ ,('\225','a')
52+ ,('\226','a')
53+ ,('\227','a')
54+ ,('\228','a')
55+ ,('\229','a')
56+ ,('\231','c')
57+ ,('\232','e')
58+ ,('\233','e')
59+ ,('\234','e')
60+ ,('\235','e')
61+ ,('\236','i')
62+ ,('\237','i')
63+ ,('\238','i')
64+ ,('\239','i')
65+ ,('\241','n')
66+ ,('\242','o')
67+ ,('\243','o')
68+ ,('\244','o')
69+ ,('\245','o')
70+ ,('\246','o')
71+ ,('\249','u')
72+ ,('\250','u')
73+ ,('\251','u')
74+ ,('\252','u')
75+ ,('\253','y')
76+ ,('\255','y')
77+ ,('\256','A')
78+ ,('\257','a')
79+ ,('\258','A')
80+ ,('\259','a')
81+ ,('\260','A')
82+ ,('\261','a')
83+ ,('\262','C')
84+ ,('\263','c')
85+ ,('\264','C')
86+ ,('\265','c')
87+ ,('\266','C')
88+ ,('\267','c')
89+ ,('\268','C')
90+ ,('\269','c')
91+ ,('\270','D')
92+ ,('\271','d')
93+ ,('\274','E')
94+ ,('\275','e')
95+ ,('\276','E')
96+ ,('\277','e')
97+ ,('\278','E')
98+ ,('\279','e')
99+ ,('\280','E')
100+ ,('\281','e')
101+ ,('\282','E')
102+ ,('\283','e')
103+ ,('\284','G')
104+ ,('\285','g')
105+ ,('\286','G')
106+ ,('\287','g')
107+ ,('\288','G')
108+ ,('\289','g')
109+ ,('\290','G')
110+ ,('\291','g')
111+ ,('\292','H')
112+ ,('\293','h')
113+ ,('\296','I')
114+ ,('\297','i')
115+ ,('\298','I')
116+ ,('\299','i')
117+ ,('\300','I')
118+ ,('\301','i')
119+ ,('\302','I')
120+ ,('\303','i')
121+ ,('\304','I')
122+ ,('\305','i')
123+ ,('\308','J')
124+ ,('\309','j')
125+ ,('\310','K')
126+ ,('\311','k')
127+ ,('\313','L')
128+ ,('\314','l')
129+ ,('\315','L')
130+ ,('\316','l')
131+ ,('\317','L')
132+ ,('\318','l')
133+ ,('\323','N')
134+ ,('\324','n')
135+ ,('\325','N')
136+ ,('\326','n')
137+ ,('\327','N')
138+ ,('\328','n')
139+ ,('\332','O')
140+ ,('\333','o')
141+ ,('\334','O')
142+ ,('\335','o')
143+ ,('\336','O')
144+ ,('\337','o')
145+ ,('\340','R')
146+ ,('\341','r')
147+ ,('\342','R')
148+ ,('\343','r')
149+ ,('\344','R')
150+ ,('\345','r')
151+ ,('\346','S')
152+ ,('\347','s')
153+ ,('\348','S')
154+ ,('\349','s')
155+ ,('\350','S')
156+ ,('\351','s')
157+ ,('\352','S')
158+ ,('\353','s')
159+ ,('\354','T')
160+ ,('\355','t')
161+ ,('\356','T')
162+ ,('\357','t')
163+ ,('\360','U')
164+ ,('\361','u')
165+ ,('\362','U')
166+ ,('\363','u')
167+ ,('\364','U')
168+ ,('\365','u')
169+ ,('\366','U')
170+ ,('\367','u')
171+ ,('\368','U')
172+ ,('\369','u')
173+ ,('\370','U')
174+ ,('\371','u')
175+ ,('\372','W')
176+ ,('\373','w')
177+ ,('\374','Y')
178+ ,('\375','y')
179+ ,('\376','Y')
180+ ,('\377','Z')
181+ ,('\378','z')
182+ ,('\379','Z')
183+ ,('\380','z')
184+ ,('\381','Z')
185+ ,('\382','z')
186+ ,('\416','O')
187+ ,('\417','o')
188+ ,('\431','U')
189+ ,('\432','u')
190+ ,('\461','A')
191+ ,('\462','a')
192+ ,('\463','I')
193+ ,('\464','i')
194+ ,('\465','O')
195+ ,('\466','o')
196+ ,('\467','U')
197+ ,('\468','u')
198+ ,('\486','G')
199+ ,('\487','g')
200+ ,('\488','K')
201+ ,('\489','k')
202+ ,('\490','O')
203+ ,('\491','o')
204+ ,('\496','j')
205+ ,('\500','G')
206+ ,('\501','g')
207+ ,('\504','N')
208+ ,('\505','n')
209+ ,('\512','A')
210+ ,('\513','a')
211+ ,('\514','A')
212+ ,('\515','a')
213+ ,('\516','E')
214+ ,('\517','e')
215+ ,('\518','E')
216+ ,('\519','e')
217+ ,('\520','I')
218+ ,('\521','i')
219+ ,('\522','I')
220+ ,('\523','i')
221+ ,('\524','O')
222+ ,('\525','o')
223+ ,('\526','O')
224+ ,('\527','o')
225+ ,('\528','R')
226+ ,('\529','r')
227+ ,('\530','R')
228+ ,('\531','r')
229+ ,('\532','U')
230+ ,('\533','u')
231+ ,('\534','U')
232+ ,('\535','u')
233+ ,('\536','S')
234+ ,('\537','s')
235+ ,('\538','T')
236+ ,('\539','t')
237+ ,('\542','H')
238+ ,('\543','h')
239+ ,('\550','A')
240+ ,('\551','a')
241+ ,('\552','E')
242+ ,('\553','e')
243+ ,('\558','O')
244+ ,('\559','o')
245+ ,('\562','Y')
246+ ,('\563','y')
247+ ,('\894',';')
248+ ,('\7680','A')
249+ ,('\7681','a')
250+ ,('\7682','B')
251+ ,('\7683','b')
252+ ,('\7684','B')
253+ ,('\7685','b')
254+ ,('\7686','B')
255+ ,('\7687','b')
256+ ,('\7690','D')
257+ ,('\7691','d')
258+ ,('\7692','D')
259+ ,('\7693','d')
260+ ,('\7694','D')
261+ ,('\7695','d')
262+ ,('\7696','D')
263+ ,('\7697','d')
264+ ,('\7698','D')
265+ ,('\7699','d')
266+ ,('\7704','E')
267+ ,('\7705','e')
268+ ,('\7706','E')
269+ ,('\7707','e')
270+ ,('\7710','F')
271+ ,('\7711','f')
272+ ,('\7712','G')
273+ ,('\7713','g')
274+ ,('\7714','H')
275+ ,('\7715','h')
276+ ,('\7716','H')
277+ ,('\7717','h')
278+ ,('\7718','H')
279+ ,('\7719','h')
280+ ,('\7720','H')
281+ ,('\7721','h')
282+ ,('\7722','H')
283+ ,('\7723','h')
284+ ,('\7724','I')
285+ ,('\7725','i')
286+ ,('\7728','K')
287+ ,('\7729','k')
288+ ,('\7730','K')
289+ ,('\7731','k')
290+ ,('\7732','K')
291+ ,('\7733','k')
292+ ,('\7734','L')
293+ ,('\7735','l')
294+ ,('\7738','L')
295+ ,('\7739','l')
296+ ,('\7740','L')
297+ ,('\7741','l')
298+ ,('\7742','M')
299+ ,('\7743','m')
300+ ,('\7744','M')
301+ ,('\7745','m')
302+ ,('\7746','M')
303+ ,('\7747','m')
304+ ,('\7748','N')
305+ ,('\7749','n')
306+ ,('\7750','N')
307+ ,('\7751','n')
308+ ,('\7752','N')
309+ ,('\7753','n')
310+ ,('\7754','N')
311+ ,('\7755','n')
312+ ,('\7764','P')
313+ ,('\7765','p')
314+ ,('\7766','P')
315+ ,('\7767','p')
316+ ,('\7768','R')
317+ ,('\7769','r')
318+ ,('\7770','R')
319+ ,('\7771','r')
320+ ,('\7774','R')
321+ ,('\7775','r')
322+ ,('\7776','S')
323+ ,('\7777','s')
324+ ,('\7778','S')
325+ ,('\7779','s')
326+ ,('\7786','T')
327+ ,('\7787','t')
328+ ,('\7788','T')
329+ ,('\7789','t')
330+ ,('\7790','T')
331+ ,('\7791','t')
332+ ,('\7792','T')
333+ ,('\7793','t')
334+ ,('\7794','U')
335+ ,('\7795','u')
336+ ,('\7796','U')
337+ ,('\7797','u')
338+ ,('\7798','U')
339+ ,('\7799','u')
340+ ,('\7804','V')
341+ ,('\7805','v')
342+ ,('\7806','V')
343+ ,('\7807','v')
344+ ,('\7808','W')
345+ ,('\7809','w')
346+ ,('\7810','W')
347+ ,('\7811','w')
348+ ,('\7812','W')
349+ ,('\7813','w')
350+ ,('\7814','W')
351+ ,('\7815','w')
352+ ,('\7816','W')
353+ ,('\7817','w')
354+ ,('\7818','X')
355+ ,('\7819','x')
356+ ,('\7820','X')
357+ ,('\7821','x')
358+ ,('\7822','Y')
359+ ,('\7823','y')
360+ ,('\7824','Z')
361+ ,('\7825','z')
362+ ,('\7826','Z')
363+ ,('\7827','z')
364+ ,('\7828','Z')
365+ ,('\7829','z')
366+ ,('\7830','h')
367+ ,('\7831','t')
368+ ,('\7832','w')
369+ ,('\7833','y')
370+ ,('\7840','A')
371+ ,('\7841','a')
372+ ,('\7842','A')
373+ ,('\7843','a')
374+ ,('\7864','E')
375+ ,('\7865','e')
376+ ,('\7866','E')
377+ ,('\7867','e')
378+ ,('\7868','E')
379+ ,('\7869','e')
380+ ,('\7880','I')
381+ ,('\7881','i')
382+ ,('\7882','I')
383+ ,('\7883','i')
384+ ,('\7884','O')
385+ ,('\7885','o')
386+ ,('\7886','O')
387+ ,('\7887','o')
388+ ,('\7908','U')
389+ ,('\7909','u')
390+ ,('\7910','U')
391+ ,('\7911','u')
392+ ,('\7922','Y')
393+ ,('\7923','y')
394+ ,('\7924','Y')
395+ ,('\7925','y')
396+ ,('\7926','Y')
397+ ,('\7927','y')
398+ ,('\7928','Y')
399+ ,('\7929','y')
400+ ,('\8175','`')
401+ ,('\8490','K')
402+ ,('\8800','=')
403+ ,('\8814','<')
404+ ,('\8815','>')
405+ ]
src/Text/Pandoc/BCP47.hsnew file+112
@@ -0,0 +1,112 @@
1+{-# LANGUAGE OverloadedStrings #-}
2+{- |
3+ Module : Text.Pandoc.BCP47
4+ Copyright : Copyright (C) 2017–2020 John MacFarlane
5+ License : GNU GPL, version 2 or above
6+
7+ Maintainer : John MacFarlane <jgm@berkeley.edu>
8+ Stability : alpha
9+ Portability : portable
10+
11+Functions for parsing and rendering BCP47 language identifiers.
12+-}
13+module Text.Pandoc.BCP47 (
14+ getLang
15+ , parseBCP47
16+ , Lang(..)
17+ , renderLang
18+ )
19+where
20+import Control.Monad (guard)
21+import Data.Char (isAlphaNum, isAscii, isLetter, isLower, isUpper)
22+import Text.Pandoc.Definition
23+import Text.Pandoc.Options
24+import Text.DocTemplates (FromContext(..))
25+import qualified Data.Text as T
26+import qualified Text.Parsec as P
27+
28+-- | Represents BCP 47 language/country code.
29+data Lang = Lang{ langLanguage :: T.Text
30+ , langScript :: T.Text
31+ , langRegion :: T.Text
32+ , langVariants :: [T.Text] }
33+ deriving (Eq, Ord, Show)
34+
35+-- | Render a Lang as BCP 47.
36+renderLang :: Lang -> T.Text
37+renderLang lang = T.intercalate "-" (langLanguage lang : filter (not . T.null)
38+ ([langScript lang, langRegion lang] ++ langVariants lang))
39+
40+-- | Get the contents of the `lang` metadata field or variable.
41+getLang :: WriterOptions -> Meta -> Maybe T.Text
42+getLang opts meta =
43+ case lookupContext "lang" (writerVariables opts) of
44+ Just s -> Just s
45+ _ ->
46+ case lookupMeta "lang" meta of
47+ Just (MetaBlocks [Para [Str s]]) -> Just s
48+ Just (MetaBlocks [Plain [Str s]]) -> Just s
49+ Just (MetaInlines [Str s]) -> Just s
50+ Just (MetaString s) -> Just s
51+ _ -> Nothing
52+
53+-- | Parse a BCP 47 string as a Lang. Currently we parse
54+-- extensions and private-use fields as "variants," even
55+-- though officially they aren't.
56+parseBCP47 :: T.Text -> Either T.Text Lang
57+parseBCP47 lang =
58+ case P.parse bcp47 "lang" lang of
59+ Right r -> Right r
60+ Left e -> Left $ T.pack $ show e
61+ where bcp47 = do
62+ language <- pLanguage
63+ script <- P.option "" pScript
64+ region <- P.option "" pRegion
65+ variants <- P.many (pVariant P.<|> pExtension P.<|> pPrivateUse)
66+ P.eof
67+ return Lang{ langLanguage = language
68+ , langScript = script
69+ , langRegion = region
70+ , langVariants = variants }
71+ asciiLetter = P.satisfy (\c -> isAscii c && isLetter c)
72+ pLanguage = do
73+ cs <- P.many1 asciiLetter
74+ let lcs = length cs
75+ guard $ lcs == 2 || lcs == 3
76+ return $ T.toLower $ T.pack cs
77+ pScript = P.try $ do
78+ P.char '-'
79+ x <- P.satisfy (\c -> isAscii c && isLetter c && isUpper c)
80+ xs <- P.count 3
81+ (P.satisfy (\c -> isAscii c && isLetter c && isLower c))
82+ return $ T.toLower $ T.pack (x:xs)
83+ pRegion = P.try $ do
84+ P.char '-'
85+ cs <- P.many1 asciiLetter
86+ let lcs = length cs
87+ guard $ lcs == 2 || lcs == 3
88+ return $ T.toUpper $ T.pack cs
89+ pVariant = P.try $ do
90+ P.char '-'
91+ ds <- P.option "" (P.count 1 P.digit)
92+ cs <- P.many1 asciiLetter
93+ let var = ds ++ cs
94+ lv = length var
95+ guard $ if null ds
96+ then lv >= 5 && lv <= 8
97+ else lv == 4
98+ return $ T.toLower $ T.pack var
99+ pExtension = P.try $ do
100+ P.char '-'
101+ cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
102+ let lcs = length cs
103+ guard $ lcs >= 2 && lcs <= 8
104+ return $ T.toLower $ T.pack cs
105+ pPrivateUse = P.try $ do
106+ P.char '-'
107+ P.char 'x'
108+ P.char '-'
109+ cs <- P.many1 $ P.satisfy (\c -> isAscii c && isAlphaNum c)
110+ guard $ not (null cs) && length cs <= 8
111+ let var = "x-" ++ cs
112+ return $ T.toLower $ T.pack var
src/Text/Pandoc/CSS.hs+17−22
@@ -1,6 +1,6 @@
11{- |
22Module : Text.Pandoc.CSS
3−Copyright : © 2006-2021 John MacFarlane <jgm@berkeley.edu>,
3+Copyright : © 2006-2020 John MacFarlane <jgm@berkeley.edu>,
44 2015-2016 Mauro Bieg,
55 2015 Ophir Lifshitz <hangfromthefloor@gmail.com>
66License : GNU GPL, version 2 or above
@@ -11,46 +11,41 @@ Portability : portable
1111
1212Tools for working with CSS.
1313-}
14−module Text.Pandoc.CSS
15− ( cssAttributes
16− , pickStyleAttrProps
17− , pickStylesToKVs
18− )
14+module Text.Pandoc.CSS ( pickStyleAttrProps
15+ , pickStylesToKVs
16+ )
1917where
2018
19+import qualified Data.Text as T
2120import Data.Maybe (mapMaybe, listToMaybe)
22−import Data.Text (Text, pack)
2321import Text.Pandoc.Shared (trim)
2422import Text.Parsec
2523import Text.Parsec.Text
2624
27−ruleParser :: Parser (Text, Text)
25+ruleParser :: Parser (T.Text, T.Text)
2826ruleParser = do
2927 p <- many1 (noneOf ":") <* char ':'
3028 v <- many1 (noneOf ":;") <* optional (char ';') <* spaces
31− return (trim $ pack p, trim $ pack v)
29+ return (trim $ T.pack p, trim $ T.pack v)
3230
33−styleAttrParser :: Parser [(Text, Text)]
31+styleAttrParser :: Parser [(T.Text, T.Text)]
3432styleAttrParser = many1 ruleParser
3533
36−-- | Parses a style string, returning the CSS attributes.
37−-- Returns an empty list on failure.
38−cssAttributes :: Text -> [(Text, Text)]
39−cssAttributes styleString =
40− -- Use Data.Either.fromRight once GHC 8.0 is no longer supported
41− case parse styleAttrParser "" styleString of
42− Left _ -> []
43− Right x -> x
34+eitherToMaybe :: Either a b -> Maybe b
35+eitherToMaybe (Right x) = Just x
36+eitherToMaybe _ = Nothing
4437
4538-- | takes a list of keys/properties and a CSS string and
4639-- returns the corresponding key-value-pairs.
47−pickStylesToKVs :: [Text] -> Text -> [(Text, Text)]
40+pickStylesToKVs :: [T.Text] -> T.Text -> [(T.Text, T.Text)]
4841pickStylesToKVs props styleAttr =
49− filter (\s -> fst s `elem` props) $ cssAttributes styleAttr
42+ case parse styleAttrParser "" styleAttr of
43+ Left _ -> []
44+ Right styles -> filter (\s -> fst s `elem` props) styles
5045
5146-- | takes a list of key/property synonyms and a CSS string and maybe
5247-- returns the value of the first match (in order of the supplied list)
53−pickStyleAttrProps :: [Text] -> Text -> Maybe Text
48+pickStyleAttrProps :: [T.Text] -> T.Text -> Maybe T.Text
5449pickStyleAttrProps lookupProps styleAttr = do
55− styles <- either (const Nothing) Just $ parse styleAttrParser "" styleAttr
50+ styles <- eitherToMaybe $ parse styleAttrParser "" styleAttr
5651 listToMaybe $ mapMaybe (`lookup` styles) lookupProps
src/Text/Pandoc/CSV.hs+3−2
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.CSV
3− Copyright : Copyright (C) 2017-2021 John MacFarlane <jgm@berkeley.edu>
3+ Copyright : Copyright (C) 2017–2020 John MacFarlane <jgm@berkeley.edu>
44 License : GNU GPL, version 2 or above
55 Maintainer : John MacFarlane <jgm@berkeley.edu>
66 Stability : alpha
@@ -68,7 +68,8 @@ escaped opts = try $
6868
6969pCSVUnquotedCell :: CSVOptions -> Parser Text
7070pCSVUnquotedCell opts = T.pack <$>
71− many (satisfy (\c -> c /= csvDelim opts && c /= '\r' && c /= '\n'))
71+ many (satisfy (\c -> c /= csvDelim opts && c /= '\r' && c /= '\n'
72+ && c /= csvQuote opts))
7273
7374pCSVDelim :: CSVOptions -> Parser ()
7475pCSVDelim opts = do
src/Text/Pandoc/Citeproc.hs+133−213
@@ -5,10 +5,7 @@
55{-# LANGUAGE FlexibleInstances #-}
66{-# LANGUAGE OverloadedStrings #-}
77module Text.Pandoc.Citeproc
8− ( processCitations,
9− getReferences,
10− getStyle
11− )
8+ ( processCitations )
129where
1310
1411import Citeproc
@@ -18,104 +15,38 @@ import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
1815import Text.Pandoc.Citeproc.BibTeX (readBibtexString, Variant(..))
1916import Text.Pandoc.Citeproc.MetaValue (metaValueToReference, metaValueToText)
2017import Text.Pandoc.Readers.Markdown (yamlToRefs)
21−import Text.Pandoc.Builder (Inlines, Many(..), deleteMeta, setMeta)
22−import qualified Text.Pandoc.Builder as B
18+import Text.Pandoc.Class (setResourcePath, getResourcePath, getUserDataDir)
19+import Data.ByteString (ByteString)
20+import qualified Data.ByteString.Lazy as L
2321import Text.Pandoc.Definition as Pandoc
24−import Text.Pandoc.Class (PandocMonad(..), getResourcePath, getUserDataDir,
25− fetchItem, readDataFile, report, setResourcePath)
26−import Text.Pandoc.Error (PandocError(..))
27−import Text.Pandoc.Extensions (pandocExtensions)
28−import Text.Pandoc.Logging (LogMessage(..))
29−import Text.Pandoc.Options (ReaderOptions(..))
30−import Text.Pandoc.Shared (stringify, ordNub, tshow)
22+import Text.Pandoc.Walk
23+import Text.Pandoc.Builder as B
24+import Text.Pandoc (PandocMonad(..), PandocError(..),
25+ readDataFile, ReaderOptions(..), pandocExtensions,
26+ report, LogMessage(..), fetchItem)
27+import Text.Pandoc.Shared (stringify, ordNub, blocksToInlines, tshow)
3128import qualified Text.Pandoc.UTF8 as UTF8
32−import Text.Pandoc.Walk (query, walk, walkM)
33−import Control.Applicative ((<|>))
34−import Control.Monad.Except (catchError, throwError)
35−import Control.Monad.State (State, evalState, get, put, runState)
3629import Data.Aeson (eitherDecode)
37−import Data.ByteString (ByteString)
38−import qualified Data.ByteString.Lazy as L
39−import Data.Char (isPunctuation, isUpper)
40−import Data.Default (Default(def))
41−import qualified Data.Foldable as Foldable
42−import qualified Data.Map as M
43−import Data.Maybe (mapMaybe, fromMaybe)
30+import Data.Default
4431import Data.Ord ()
45−import qualified Data.Sequence as Seq
32+import qualified Data.Map as M
4633import qualified Data.Set as Set
34+import Data.Char (isPunctuation, isUpper)
4735import Data.Text (Text)
4836import qualified Data.Text as T
49−import System.FilePath (takeExtension)
37+import Control.Monad.State
38+import qualified Data.Sequence as Seq
39+import qualified Data.Foldable as Foldable
40+import System.FilePath
41+import Control.Applicative
42+import Control.Monad.Except
43+import Data.Maybe (mapMaybe, fromMaybe)
5044import Safe (lastMay, initSafe)
45+-- import Debug.Trace as Trace (trace, traceShowId)
5146
5247
53−processCitations :: PandocMonad m => Pandoc -> m Pandoc
48+processCitations :: PandocMonad m => Pandoc -> m Pandoc
5449processCitations (Pandoc meta bs) = do
55− style <- getStyle (Pandoc meta bs)
56−
57− mblang <- getLang meta
58− let locale = Citeproc.mergeLocales mblang style
59−
60− refs <- getReferences (Just locale) (Pandoc meta bs)
61−
62− let otherIdsMap = foldr (\ref m ->
63− case T.words . extractText <$>
64− M.lookup "other-ids"
65− (referenceVariables ref) of
66− Nothing -> m
67− Just ids -> foldr
68− (\id' ->
69− M.insert id' (referenceId ref)) m ids)
70− M.empty refs
71− let meta' = deleteMeta "nocite" meta
72− let citations = getCitations locale otherIdsMap $ Pandoc meta' bs
73−
74−
75− let linkCites = maybe False truish $ lookupMeta "link-citations" meta
76− let opts = defaultCiteprocOptions{ linkCitations = linkCites }
77− let result = Citeproc.citeproc opts style mblang refs citations
78− mapM_ (report . CiteprocWarning) (resultWarnings result)
79− let sopts = styleOptions style
80− let classes = "references" : -- TODO remove this or keep for compatibility?
81− "csl-bib-body" :
82− ["hanging-indent" | styleHangingIndent sopts]
83− let refkvs = (case styleEntrySpacing sopts of
84− Just es | es > 0 -> (("entry-spacing",T.pack $ show es):)
85− _ -> id) .
86− (case styleLineSpacing sopts of
87− Just ls | ls > 1 -> (("line-spacing",T.pack $ show ls):)
88− _ -> id) $ []
89− let bibs = mconcat $ map (\(ident, out) ->
90− B.divWith ("ref-" <> ident,["csl-entry"],[]) . B.para .
91− walk (convertQuotes locale) .
92− insertSpace $ out)
93− (resultBibliography result)
94− let moveNotes = styleIsNoteStyle sopts &&
95− maybe True truish (lookupMeta "notes-after-punctuation" meta)
96− let cits = map (walk (convertQuotes locale)) $
97− resultCitations result
98−
99− let metanocites = lookupMeta "nocite" meta
100− let Pandoc meta'' bs' =
101− maybe id (setMeta "nocite") metanocites .
102− walk (mvPunct moveNotes locale) .
103− (if styleIsNoteStyle sopts
104− then walk addNote . walk deNote
105− else id) .
106− evalState (walkM insertResolvedCitations $ Pandoc meta' bs)
107− $ cits
108− return $ Pandoc meta''
109− $ insertRefs refkvs classes meta''
110− (walk fixLinks $ B.toList bibs) bs'
111−
112−-- | Retrieve the CSL style specified by the csl or citation-style
113−-- metadata field in a pandoc document, or the default CSL style
114−-- if none is specified. Retrieve the parent style
115−-- if the style is a dependent style. Add abbreviations defined
116−-- in an abbreviation file if one has been specified.
117−getStyle :: PandocMonad m => Pandoc -> m (Style Inlines)
118−getStyle (Pandoc meta _) = do
11950 let cslfile = (lookupMeta "csl" meta <|> lookupMeta "citation-style" meta)
12051 >>= metaValueToText
12152
@@ -156,60 +87,86 @@ getStyle (Pandoc meta _) = do
15687 catchError (getFile ".csl" basename) (\_ -> fst <$> fetchItem url)
15788
15889 styleRes <- Citeproc.parseStyle getParentStyle cslContents
159− case styleRes of
160− Left err -> throwError $ PandocAppError $ prettyCiteprocError err
161− Right style -> return style{ styleAbbreviations = mbAbbrevs }
162−
163−
164−-- Retrieve citeproc lang based on metadata.
165−getLang :: PandocMonad m => Meta -> m (Maybe Lang)
166−getLang meta = maybe (return Nothing) bcp47LangToIETF
167− ((lookupMeta "lang" meta <|> lookupMeta "locale" meta) >>=
168− metaValueToText)
169−
170−-- | Get references defined inline in the metadata and via an external
171−-- bibliography. Only references that are actually cited in the document
172−-- (either with a genuine citation or with `nocite`) are returned.
173−-- URL variables are converted to links.
174−getReferences :: PandocMonad m
175− => Maybe Locale -> Pandoc -> m [Reference Inlines]
176−getReferences mblocale (Pandoc meta bs) = do
177− locale <- case mblocale of
178− Just l -> return l
179− Nothing -> do
180− mblang <- getLang meta
181− case mblang of
182− Just lang -> return $ either mempty id $ getLocale lang
183− Nothing -> return mempty
184−
90+ style <-
91+ case styleRes of
92+ Left err -> throwError $ PandocAppError $ prettyCiteprocError err
93+ Right style -> return style{ styleAbbreviations = mbAbbrevs }
94+ let mblang = parseLang <$>
95+ ((lookupMeta "lang" meta <|> lookupMeta "locale" meta) >>= metaValueToText)
96+ let locale = Citeproc.mergeLocales mblang style
18597 let getCiteId (Cite cs _) = Set.fromList $ map B.citationId cs
18698 getCiteId _ = mempty
18799 let metanocites = lookupMeta "nocite" meta
100+ let meta' = deleteMeta "nocite" meta
188101 let nocites = maybe mempty (query getCiteId) metanocites
189102 let citeIds = query getCiteId (Pandoc meta bs)
190103 let idpred = if "*" `Set.member` nocites
191104 then const True
192105 else (`Set.member` citeIds)
193− let inlineRefs = case lookupMeta "references" meta of
194− Just (MetaList rs) ->
195− filter (idpred . unItemId . referenceId)
196− $ mapMaybe metaValueToReference rs
197− _ -> []
198− externalRefs <- case lookupMeta "bibliography" meta of
199− Just (MetaList xs) ->
200− mconcat <$>
201− mapM (getRefsFromBib locale idpred)
202− (mapMaybe metaValueToText xs)
203− Just x ->
204− case metaValueToText x of
205− Just fp -> getRefsFromBib locale idpred fp
206− Nothing -> return []
207− Nothing -> return []
208− return $ map (linkifyVariables . legacyDateRanges)
209− (externalRefs ++ inlineRefs)
210− -- note that inlineRefs can override externalRefs
106+ refs <- map (linkifyVariables . legacyDateRanges) <$>
107+ case lookupMeta "references" meta of
108+ Just (MetaList rs) -> return $ mapMaybe metaValueToReference rs
109+ _ ->
110+ case lookupMeta "bibliography" meta of
111+ Just (MetaList xs) ->
112+ mconcat <$>
113+ mapM (getRefsFromBib locale idpred)
114+ (mapMaybe metaValueToText xs)
115+ Just x ->
116+ case metaValueToText x of
117+ Just fp -> getRefsFromBib locale idpred fp
118+ Nothing -> return []
119+ Nothing -> return []
120+ let otherIdsMap = foldr (\ref m ->
121+ case T.words . extractText <$>
122+ M.lookup "other-ids"
123+ (referenceVariables ref) of
124+ Nothing -> m
125+ Just ids -> foldr
126+ (\id' ->
127+ M.insert id' (referenceId ref)) m ids)
128+ M.empty refs
129+ -- TODO: issue warning if no refs defined
130+ let citations = getCitations locale otherIdsMap $ Pandoc meta' bs
131+ let linkCites = maybe False truish $ lookupMeta "link-citations" meta
132+ let opts = defaultCiteprocOptions{ linkCitations = linkCites }
133+ let result = Citeproc.citeproc opts style (localeLanguage locale)
134+ refs citations
135+ mapM_ (report . CiteprocWarning) (resultWarnings result)
136+ let sopts = styleOptions style
137+ let classes = "references" : -- TODO remove this or keep for compatibility?
138+ "csl-bib-body" :
139+ ["hanging-indent" | styleHangingIndent sopts]
140+ let refkvs = (case styleEntrySpacing sopts of
141+ Just es | es > 0 -> (("entry-spacing",T.pack $ show es):)
142+ _ -> id) .
143+ (case styleLineSpacing sopts of
144+ Just ls | ls > 1 -> (("line-spacing",T.pack $ show ls):)
145+ _ -> id) $ []
146+ let bibs = mconcat $ map (\(ident, out) ->
147+ B.divWith ("ref-" <> ident,["csl-entry"],[]) . B.para .
148+ walk (convertQuotes locale) . insertSpace $ out)
149+ (resultBibliography result)
150+ let moveNotes = maybe True truish $
151+ lookupMeta "notes-after-punctuation" meta
152+ let cits = map (walk fixLinks . walk (convertQuotes locale)) $
153+ resultCitations result
211154
155+ let fixQuotes = case localePunctuationInQuote locale of
156+ Just True ->
157+ B.toList . movePunctuationInsideQuotes . B.fromList
158+ _ -> id
212159
160+ let Pandoc meta'' bs' =
161+ maybe id (setMeta "nocite") metanocites .
162+ walk (map capitalizeNoteCitation .
163+ fixQuotes . mvPunct moveNotes locale) .
164+ walk deNote .
165+ evalState (walkM insertResolvedCitations $ Pandoc meta' bs)
166+ $ cits
167+ return $ Pandoc meta''
168+ $ insertRefs refkvs classes meta''
169+ (walk fixLinks $ B.toList bibs) bs'
213170
214171-- If we have a span.csl-left-margin followed by span.csl-right-inline,
215172-- we insert a space. This ensures that they will be separated by a space,
@@ -230,23 +187,24 @@ insertSpace ils =
230187
231188getRefsFromBib :: PandocMonad m
232189 => Locale -> (Text -> Bool) -> Text -> m [Reference Inlines]
233−getRefsFromBib locale idpred fp = do
234− (raw, _) <- fetchItem fp
235− case formatFromExtension (T.unpack fp) of
190+getRefsFromBib locale idpred t = do
191+ let fp = T.unpack t
192+ raw <- readFileStrict fp
193+ case formatFromExtension fp of
236194 Just f -> getRefs locale f idpred (Just fp) raw
237195 Nothing -> throwError $ PandocAppError $
238− "Could not determine bibliography format for " <> fp
196+ "Could not determine bibliography format for " <> t
239197
240198getRefs :: PandocMonad m
241199 => Locale
242200 -> BibFormat
243201 -> (Text -> Bool)
244− -> Maybe Text
202+ -> Maybe FilePath
245203 -> ByteString
246204 -> m [Reference Inlines]
247205getRefs locale format idpred mbfp raw = do
248206 let err' = throwError .
249− PandocBibliographyError (fromMaybe mempty mbfp)
207+ PandocBibliographyError (maybe mempty T.pack mbfp)
250208 case format of
251209 Format_bibtex ->
252210 either (err' . tshow) return .
@@ -261,7 +219,7 @@ getRefs locale format idpred mbfp raw = do
261219 Format_yaml -> do
262220 rs <- yamlToRefs idpred
263221 def{ readerExtensions = pandocExtensions }
264− (T.unpack <$> mbfp)
222+ mbfp
265223 (L.fromStrict raw)
266224 return $ mapMaybe metaValueToReference rs
267225
@@ -290,7 +248,7 @@ insertResolvedCitations (Cite cs ils) = do
290248 [] -> return (Cite cs ils)
291249 (x:xs) -> do
292250 put xs
293− return $ Cite cs (walk fixLinks $ B.toList x)
251+ return $ Cite cs (B.toList x)
294252insertResolvedCitations x = return x
295253
296254getCitations :: Locale
@@ -372,6 +330,7 @@ formatFromExtension fp = case dropWhile (== '.') $ takeExtension fp of
372330
373331
374332isNote :: Inline -> Bool
333+isNote (Note _) = True
375334isNote (Cite _ [Note _]) = True
376335 -- the following allows citation styles that are "in-text" but use superscript
377336 -- references to be treated as if they are "notes" for the purposes of moving
@@ -384,12 +343,6 @@ isSpacy Space = True
384343isSpacy SoftBreak = True
385344isSpacy _ = False
386345
387−movePunctInsideQuotes :: Locale -> [Inline] -> [Inline]
388−movePunctInsideQuotes locale
389− | localePunctuationInQuote locale == Just True
390− = B.toList . movePunctuationInsideQuotes . B.fromList
391− | otherwise
392− = id
393346
394347mvPunct :: Bool -> Locale -> [Inline] -> [Inline]
395348mvPunct moveNotes locale (x : xs)
@@ -402,8 +355,7 @@ mvPunct moveNotes locale (q : s : x : ys)
402355 in if moveNotes
403356 then if T.null spunct
404357 then q : x : mvPunct moveNotes locale ys
405− else movePunctInsideQuotes locale
406− [q , Str spunct , x] ++ mvPunct moveNotes locale
358+ else q : Str spunct : x : mvPunct moveNotes locale
407359 (B.toList
408360 (dropTextWhile isPunctuation (B.fromList ys)))
409361 else q : x : mvPunct moveNotes locale ys
@@ -415,10 +367,9 @@ mvPunct moveNotes locale (Cite cs ils : ys)
415367 , moveNotes
416368 = let s = stringify ys
417369 spunct = T.takeWhile isPunctuation s
418− in Cite cs (movePunctInsideQuotes locale $
419− init ils
420− ++ [Str spunct | not (endWithPunct False (init ils))]
421− ++ [last ils]) :
370+ in Cite cs (init ils
371+ ++ [Str spunct | not (endWithPunct False (init ils))]
372+ ++ [last ils]) :
422373 mvPunct moveNotes locale
423374 (B.toList (dropTextWhile isPunctuation (B.fromList ys)))
424375mvPunct moveNotes locale (s : x : ys) | isSpacy s, isNote x =
@@ -434,7 +385,7 @@ mvPunct _ _ [] = []
434385-- move https://doi.org etc. prefix inside link text (#6723):
435386fixLinks :: [Inline] -> [Inline]
436387fixLinks (Str t : Link attr [Str u1] (u2,tit) : xs)
437− | u2 == t <> u1
388+ | t <> u1 == u2
438389 = Link attr [Str (t <> u1)] (u2,tit) : fixLinks xs
439390fixLinks (x:xs) = x : fixLinks xs
440391fixLinks [] = []
@@ -504,8 +455,7 @@ insertRefs refkvs refclasses meta refs bs =
504455 put True
505456 -- refHeader isn't used if you have an explicit references div
506457 let cs' = ordNub $ cs ++ refclasses
507− let kvs' = ordNub $ kvs ++ refkvs
508− return $ Div ("refs",cs',kvs') (xs ++ refs)
458+ return $ Div ("refs",cs',kvs) (xs ++ refs)
509459 go x = return x
510460
511461refTitle :: Meta -> Maybe [Inline]
@@ -548,15 +498,13 @@ linkifyVariables ref =
548498 fixShortDOI x = let x' = extractText x
549499 in if "10/" `T.isPrefixOf` x'
550500 then TextVal $ T.drop 3 x'
551− -- see https://shortdoi.org
501+ -- see http://shortdoi.org
552502 else TextVal x'
553503 tolink pref x = let x' = extractText x
554504 x'' = if "://" `T.isInfixOf` x'
555505 then x'
556506 else pref <> x'
557− in if T.null x'
558− then x
559− else FancyVal (B.link x'' "" (B.str x'))
507+ in FancyVal (B.link x'' "" (B.str x'))
560508
561509extractText :: Val Inlines -> Text
562510extractText (TextVal x) = x
@@ -564,62 +512,42 @@ extractText (FancyVal x) = toText x
564512extractText (NumVal n) = T.pack (show n)
565513extractText _ = mempty
566514
567−-- Here we take the Spans with class csl-note that are left
568−-- after deNote has removed nested ones, and convert them
569−-- into real notes.
570−addNote :: Inline -> Inline
571−addNote (Span ("",["csl-note"],[]) ils) =
572− Note [Para $
573− B.toList . addTextCase Nothing CapitalizeFirst . B.fromList $ ils]
574−addNote x = x
575−
576−-- Here we handle citation notes that occur inside footnotes
577−-- or other citation notes, in a note style. We don't want
578−-- notes inside notes, so we convert these to parenthesized
579−-- or comma-separated citations.
580−deNote :: Inline -> Inline
581−deNote (Note bs) =
582− case bs of
583− [Para (cit@(Cite (c:_) _) : ils)]
584− | citationMode c /= AuthorInText ->
585− -- if citation is first in note, no need to parenthesize.
586− Note [Para (walk removeNotes $ cit : walk addParens ils)]
587− _ -> Note (walk removeNotes . walk addParens $ bs)
588−
515+capitalizeNoteCitation :: Inline -> Inline
516+capitalizeNoteCitation (Cite cs [Note [Para ils]]) =
517+ Cite cs
518+ [Note [Para $ B.toList $ addTextCase Nothing CapitalizeFirst
519+ $ B.fromList ils]]
520+capitalizeNoteCitation x = x
521+
522+deNote :: [Inline] -> [Inline]
523+deNote [] = []
524+deNote (Note bs:rest) =
525+ Note (walk go bs) : deNote rest
589526 where
590− addParens [] = []
591− addParens (Cite (c:cs) ils : zs)
527+ go [] = []
528+ go (Cite (c:cs) ils : zs)
592529 | citationMode c == AuthorInText
593− = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) :
594− addParens zs
530+ = Cite (c:cs) (concatMap (noteAfterComma (needsPeriod zs)) ils) : go zs
595531 | otherwise
596− = Cite (c:cs) (concatMap noteInParens ils) : addParens zs
597− addParens (x:xs) = x : addParens xs
598−
599− removeNotes (Span ("",["csl-note"],[]) ils) = Span ("",[],[]) ils
600− removeNotes x = x
601−
532+ = Cite (c:cs) (concatMap noteInParens ils) : go zs
533+ go (x:xs) = x : go xs
602534 needsPeriod [] = True
603535 needsPeriod (Str t:_) = case T.uncons t of
604536 Nothing -> False
605537 Just (c,_) -> isUpper c
606538 needsPeriod (Space:zs) = needsPeriod zs
607539 needsPeriod _ = False
608−
609− noteInParens (Span ("",["csl-note"],[]) ils)
540+ noteInParens (Note bs')
610541 = Space : Str "(" :
611− removeFinalPeriod ils ++ [Str ")"]
542+ removeFinalPeriod (blocksToInlines bs') ++ [Str ")"]
612543 noteInParens x = [x]
613−
614− noteAfterComma needsPer (Span ("",["csl-note"],[]) ils)
615− | not (null ils)
544+ noteAfterComma needsPer (Note bs')
616545 = Str "," : Space :
617− if needsPer
618− then ils
619− else removeFinalPeriod ils
546+ (if needsPer
547+ then id
548+ else removeFinalPeriod) (blocksToInlines bs')
620549 noteAfterComma _ x = [x]
621−
622−deNote x = x
550+deNote (x:xs) = x : deNote xs
623551
624552-- Note: we can't use dropTextWhileEnd indiscriminately,
625553-- because this would remove the final period on abbreviations like Ibid.
@@ -651,11 +579,3 @@ removeFinalPeriod ils =
651579 isRightQuote "\8217" = True
652580 isRightQuote "\187" = True
653581 isRightQuote _ = False
654−
655−bcp47LangToIETF :: PandocMonad m => Text -> m (Maybe Lang)
656−bcp47LangToIETF bcplang =
657− case parseLang bcplang of
658− Left _ -> do
659− report $ InvalidLang bcplang
660− return Nothing
661− Right lang -> return $ Just lang
src/Text/Pandoc/Citeproc/BibTeX.hs+58−299
@@ -1,5 +1,4 @@
11{-# LANGUAGE ViewPatterns #-}
2−{-# LANGUAGE LambdaCase #-}
32{-# LANGUAGE OverloadedStrings #-}
43{-# LANGUAGE ScopedTypeVariables #-}
54{-# LANGUAGE FlexibleContexts #-}
@@ -18,7 +17,6 @@
1817module Text.Pandoc.Citeproc.BibTeX
1918 ( Variant(..)
2019 , readBibtexString
21− , writeBibtexString
2220 )
2321 where
2422
@@ -26,11 +24,10 @@ import Text.Pandoc.Definition
2624import Text.Pandoc.Builder as B
2725import Text.Pandoc.Readers.LaTeX (readLaTeX)
2826import Text.Pandoc.Extensions (Extension(..), extensionsFromList)
29−import Text.Pandoc.Options (ReaderOptions(..), WriterOptions)
27+import Text.Pandoc.Options (ReaderOptions(..))
28+import Text.Pandoc.Class (runPure)
3029import Text.Pandoc.Error (PandocError)
3130import Text.Pandoc.Shared (stringify)
32−import Text.Pandoc.Writers.LaTeX (writeLaTeX)
33−import Text.Pandoc.Class (runPure)
3431import qualified Text.Pandoc.Walk as Walk
3532import Citeproc.Types
3633import Citeproc.Pandoc ()
@@ -49,21 +46,17 @@ import qualified Data.Sequence as Seq
4946import Data.Char (isAlphaNum, isDigit, isLetter,
5047 isUpper, toLower, toUpper,
5148 isLower, isPunctuation)
52−import Data.List (foldl', intercalate, intersperse)
49+import Data.List (foldl', intercalate)
5350import Safe (readMay)
54−import Text.Printf (printf)
55−import Text.DocLayout (literal, hsep, nest, hang, Doc(..),
56− braces, ($$), cr)
5751
5852data Variant = Bibtex | Biblatex
5953 deriving (Show, Eq, Ord)
6054
6155-- | Parse BibTeX or BibLaTeX into a list of 'Reference's.
62−readBibtexString :: ToSources a
63− => Variant -- ^ bibtex or biblatex
56+readBibtexString :: Variant -- ^ bibtex or biblatex
6457 -> Locale -- ^ Locale
6558 -> (Text -> Bool) -- ^ Filter on citation ids
66− -> a -- ^ bibtex/biblatex text
59+ -> Text -- ^ bibtex/biblatex text
6760 -> Either ParseError [Reference Inlines]
6861readBibtexString variant locale idpred contents = do
6962 case runParser (((resolveCrossRefs variant <$> bibEntries) <* eof) >>=
@@ -71,280 +64,17 @@ readBibtexString variant locale idpred contents = do
7164 filter (\item -> idpred (identifier item) &&
7265 entryType item /= "xdata"))
7366 (fromMaybe defaultLang $ localeLanguage locale, Map.empty)
74− (initialSourceName sources) sources of
67+ "" contents of
7568 Left err -> Left err
7669 Right xs -> return xs
77− where
78− sources = toSources contents
79−
80−-- | Write BibTeX or BibLaTeX given given a 'Reference'.
81−writeBibtexString :: WriterOptions -- ^ options (for writing LaTex)
82− -> Variant -- ^ bibtex or biblatex
83− -> Maybe Lang -- ^ Language
84− -> Reference Inlines -- ^ Reference to write
85− -> Doc Text
86−writeBibtexString opts variant mblang ref =
87− "@" <> bibtexType <> "{" <> literal (unItemId (referenceId ref)) <> ","
88− $$ nest 2 (renderFields fs)
89− $$ "}" <> cr
90−
91− where
92− bibtexType =
93− case referenceType ref of
94− "article-magazine" -> "article"
95− "article-newspaper" -> "article"
96− "article-journal" -> "article"
97− "book" -> "book"
98− "pamphlet" -> "booklet"
99− "dataset" | variant == Biblatex -> "dataset"
100− "webpage" | variant == Biblatex -> "online"
101− "chapter" -> case getVariable "editor" of
102− Just _ -> "incollection"
103− Nothing -> "inbook"
104− "entry-encyclopedia" | variant == Biblatex -> "inreference"
105− | otherwise -> "inbook"
106− "paper-conference" -> "inproceedings"
107− "thesis" -> case getVariableAsText "genre" of
108− Just "mathesis" -> "mastersthesis"
109− _ -> "phdthesis"
110− "patent" | variant == Biblatex -> "patent"
111− "report" | variant == Biblatex -> "report"
112− | otherwise -> "techreport"
113− "speech" -> "unpublished"
114− "manuscript" -> "unpublished"
115− "graphic" | variant == Biblatex -> "artwork"
116− "song" | variant == Biblatex -> "music"
117− "legal_case" | variant == Biblatex -> "jurisdictionN"
118− "legislation" | variant == Biblatex -> "legislation"
119− "treaty" | variant == Biblatex -> "legal"
120− "personal_communication" | variant == Biblatex -> "letter"
121− "motion_picture" | variant == Biblatex -> "movie"
122− "review" | variant == Biblatex -> "review"
123− _ -> "misc"
124−
125− mbSubtype =
126− case referenceType ref of
127− "article-magazine" -> Just "magazine"
128− "article-newspaper" -> Just "newspaper"
129− _ -> Nothing
130−
131− fs =
132− case variant of
133− Biblatex ->
134− [ "author"
135− , "editor"
136− , "translator"
137− , "publisher"
138− , "title"
139− , "booktitle"
140− , "journal"
141− , "series"
142− , "edition"
143− , "volume"
144− , "volumes"
145− , "number"
146− , "pages"
147− , "date"
148− , "eventdate"
149− , "urldate"
150− , "address"
151− , "url"
152− , "doi"
153− , "isbn"
154− , "issn"
155− , "type"
156− , "entrysubtype"
157− , "note"
158− , "langid"
159− , "abstract"
160− , "keywords"
161− , "annote"
162− ]
163− Bibtex ->
164− [ "author"
165− , "editor"
166− , "translator"
167− , "publisher"
168− , "title"
169− , "booktitle"
170− , "journal"
171− , "series"
172− , "edition"
173− , "volume"
174− , "number"
175− , "pages"
176− , "year"
177− , "month"
178− , "address"
179− , "type"
180− , "note"
181− , "annote"
182− ]
183−
184− valToInlines (TextVal t) = B.text t
185− valToInlines (FancyVal ils) = ils
186− valToInlines (NumVal n) = B.text (T.pack $ show n)
187− valToInlines (NamesVal names) =
188− mconcat $ intersperse (B.space <> B.text "and" <> B.space)
189− $ map renderName names
190− valToInlines (DateVal date) = B.text $
191− case dateLiteral date of
192− Just t -> t
193− Nothing -> T.intercalate "/" (map renderDatePart (dateParts date)) <>
194− (if dateCirca date then "~" else mempty)
195−
196− renderDatePart (DateParts xs) = T.intercalate "-" $
197− map (T.pack . printf "%02d") xs
198−
199− renderName name =
200− case nameLiteral name of
201− Just t -> B.text t
202− Nothing -> spacedMaybes
203− [ nameNonDroppingParticle name
204− , nameFamily name
205− , if nameCommaSuffix name
206− then (", " <>) <$> nameSuffix name
207− else nameSuffix name ]
208− <>
209− spacedMaybes
210− [ (", " <>) <$> nameGiven name,
211− nameDroppingParticle name ]
212−
213− mblang' = case getVariableAsText "language" of
214− Just l -> either (const Nothing) Just $ parseLang l
215− Nothing -> mblang
216−
217− titlecase = case mblang' of
218− Just lang | langLanguage lang == "en"
219− -> titlecase'
220− Nothing -> titlecase'
221− _ ->
222− case variant of
223− Bibtex -> B.spanWith nullAttr
224− -- BibTex lacks a language field, so we wrap non-English
225− -- titles in {} to protect case.
226− Biblatex -> id
227−
228− titlecase' = addTextCase mblang' TitleCase .
229− (\ils -> B.fromList
230− (case B.toList ils of
231− Str t : xs -> Str t : Walk.walk spanAroundCapitalizedWords xs
232− xs -> Walk.walk spanAroundCapitalizedWords xs))
233−
234− -- protect capitalized words when we titlecase
235− spanAroundCapitalizedWords (Str t)
236− | not (T.all (\c -> isLower c || not (isLetter c)) t) =
237− Span ("",["nocase"],[]) [Str t]
238− spanAroundCapitalizedWords x = x
239−
240− spacedMaybes = mconcat . intersperse B.space . mapMaybe (fmap B.text)
241−
242− toLaTeX x =
243− case runPure (writeLaTeX opts $ doc (B.plain x)) of
244− Left _ -> Nothing
245− Right t -> Just $ hsep . map literal $ T.words t
246−
247− renderField :: Text -> Maybe (Doc Text)
248− renderField name =
249− (((literal name) <>) . hang 2 " = " . braces)
250− <$> getContentsFor name
251−
252− getVariable v = lookupVariable (toVariable v) ref
253−
254− getVariableAsText v = (stringify . valToInlines) <$> getVariable v
255−
256− getYear val =
257− case val of
258− DateVal date ->
259− case dateLiteral date of
260− Just t -> toLaTeX (B.text t)
261− Nothing ->
262− case dateParts date of
263− [DateParts (y1:_), DateParts (y2:_)] ->
264− Just $ literal (T.pack (printf "%04d" y1) <> "--" <>
265− T.pack (printf "%04d" y2))
266− [DateParts (y1:_)] ->
267− Just $ literal (T.pack (printf "%04d" y1))
268− _ -> Nothing
269− _ -> Nothing
270−
271− toMonth 1 = "jan"
272− toMonth 2 = "feb"
273− toMonth 3 = "mar"
274− toMonth 4 = "apr"
275− toMonth 5 = "may"
276− toMonth 6 = "jun"
277− toMonth 7 = "jul"
278− toMonth 8 = "aug"
279− toMonth 9 = "sep"
280− toMonth 10 = "oct"
281− toMonth 11 = "nov"
282− toMonth 12 = "dec"
283− toMonth x = T.pack $ show x
284−
285− getMonth val =
286− case val of
287− DateVal date ->
288− case dateParts date of
289− [DateParts (_:m1:_), DateParts (_:m2:_)] ->
290− Just $ literal (toMonth m1 <> "--" <> toMonth m2)
291− [DateParts (_:m1:_)] -> Just $ literal (toMonth m1)
292− _ -> Nothing
293− _ -> Nothing
294−
295− getContentsFor :: Text -> Maybe (Doc Text)
296− getContentsFor "type" =
297− getVariableAsText "genre" >>=
298− \case
299− "mathesis" -> Just "mastersthesis"
300− "phdthesis" -> Just "phdthesis"
301− _ -> Nothing
302− getContentsFor "entrysubtype" = literal <$> mbSubtype
303− getContentsFor "journal"
304− | bibtexType `elem` ["article", "periodical", "suppperiodical", "review"]
305− = getVariable "container-title" >>= toLaTeX . valToInlines
306− | otherwise = Nothing
307− getContentsFor "booktitle"
308− | bibtexType `elem`
309− ["inbook","incollection","inproceedings","inreference","bookinbook"]
310− = (getVariable "volume-title" <|> getVariable "container-title")
311− >>= toLaTeX . valToInlines
312− | otherwise = Nothing
313− getContentsFor "series" = getVariable "collection-title"
314− >>= toLaTeX . valToInlines
315− getContentsFor "address" = getVariable "publisher-place"
316− >>= toLaTeX . valToInlines
317− getContentsFor "date" = getVariable "issued" >>= toLaTeX . valToInlines
318− getContentsFor "eventdate" = getVariable "event-date" >>= toLaTeX . valToInlines
319− getContentsFor "urldate" = getVariable "accessed" >>= toLaTeX . valToInlines
320− getContentsFor "year" = getVariable "issued" >>= getYear
321− getContentsFor "month" = getVariable "issued" >>= getMonth
322− getContentsFor "pages" = getVariable "page" >>= toLaTeX . valToInlines
323− getContentsFor "langid" = getVariable "language" >>= toLaTeX . valToInlines
324− getContentsFor "number" = (getVariable "number"
325− <|> getVariable "collection-number"
326− <|> getVariable "issue") >>= toLaTeX . valToInlines
327−
328− getContentsFor x = getVariable x >>=
329− if isURL x
330− then Just . literal . stringify . valToInlines
331− else toLaTeX .
332− (if x == "title"
333− then titlecase
334− else id) .
335− valToInlines
336−
337− isURL x = x `elem` ["url","doi","issn","isbn"]
338−
339− renderFields = mconcat . intersperse ("," <> cr) . mapMaybe renderField
34070
34171defaultLang :: Lang
342−defaultLang = Lang "en" Nothing (Just "US") [] [] []
72+defaultLang = Lang "en" (Just "US")
34373
34474-- a map of bibtex "string" macros
34575type StringMap = Map.Map Text Text
34676
347−type BibParser = Parser Sources (Lang, StringMap)
77+type BibParser = Parser Text (Lang, StringMap)
34878
34979data Item = Item{ identifier :: Text
35080 , sourcePos :: SourcePos
@@ -359,7 +89,9 @@ itemToReference locale variant item = do
35989 bib item $ do
36090 let lang = fromMaybe defaultLang $ localeLanguage locale
36191 modify $ \st -> st{ localeLang = lang,
362− untitlecase = langLanguage lang == "en" }
92+ untitlecase = case lang of
93+ (Lang "en" _) -> True
94+ _ -> False }
36395
36496 id' <- asks identifier
36597 otherIds <- (Just <$> getRawField "ids")
@@ -583,10 +315,10 @@ itemToReference locale variant item = do
583315 eprint <- getRawField "eprint"
584316 let baseUrl =
585317 case T.toLower etype of
586− "arxiv" -> "https://arxiv.org/abs/"
587− "jstor" -> "https://www.jstor.org/stable/"
588− "pubmed" -> "https://www.ncbi.nlm.nih.gov/pubmed/"
589− "googlebooks" -> "https://books.google.com?id="
318+ "arxiv" -> "http://arxiv.org/abs/"
319+ "jstor" -> "http://www.jstor.org/stable/"
320+ "pubmed" -> "http://www.ncbi.nlm.nih.gov/pubmed/"
321+ "googlebooks" -> "http://books.google.com?id="
590322 _ -> ""
591323 if T.null baseUrl
592324 then mzero
@@ -717,7 +449,7 @@ itemToReference locale variant item = do
717449
718450
719451bib :: Item -> Bib a -> BibParser a
720−bib entry m = fst <$> evalRWST m entry (BibState True defaultLang)
452+bib entry m = fst <$> evalRWST m entry (BibState True (Lang "en" (Just "US")))
721453
722454resolveCrossRefs :: Variant -> [Item] -> [Item]
723455resolveCrossRefs variant entries =
@@ -770,10 +502,41 @@ blocksToInlines bs =
770502 _ -> B.fromList $ Walk.query (:[]) bs
771503
772504adjustSpans :: Lang -> Inline -> Inline
773−adjustSpans lang (Span ("",[],[("bibstring",s)]) _) = Str $ resolveKey' lang s
505+adjustSpans lang (RawInline (Format "latex") s)
506+ | s == "\\hyphen" || s == "\\hyphen " = Str "-"
507+ | otherwise = parseRawLaTeX lang s
774508adjustSpans _ SoftBreak = Space
775509adjustSpans _ x = x
776510
511+parseRawLaTeX :: Lang -> Text -> Inline
512+parseRawLaTeX lang t@(T.stripPrefix "\\" -> Just xs) =
513+ case parseLaTeX lang contents of
514+ Right [Para ys] -> f command ys
515+ Right [Plain ys] -> f command ys
516+ Right [] -> f command []
517+ _ -> RawInline (Format "latex") t
518+ where (command', contents') = T.break (\c -> c =='{' || c =='\\') xs
519+ command = T.strip command'
520+ contents = T.drop 1 $ T.dropEnd 1 contents'
521+ f "mkbibquote" ils = Span nullAttr [Quoted DoubleQuote ils]
522+ f "mkbibemph" ils = Span nullAttr [Emph ils]
523+ f "mkbibitalic" ils = Span nullAttr [Emph ils]
524+ f "mkbibbold" ils = Span nullAttr [Strong ils]
525+ f "mkbibparens" ils = Span nullAttr $
526+ [Str "("] ++ ils ++ [Str ")"]
527+ f "mkbibbrackets" ils = Span nullAttr $
528+ [Str "["] ++ ils ++ [Str "]"]
529+ -- ... both should be nestable & should work in year fields
530+ f "autocap" ils = Span nullAttr ils
531+ -- TODO: should work in year fields
532+ f "textnormal" ils = Span ("",["nodecor"],[]) ils
533+ f "bibstring" [Str s] = Str $ resolveKey' lang s
534+ f "adddot" [] = Str "."
535+ f "adddotspace" [] = Span nullAttr [Str ".", Space]
536+ f "addabbrvspace" [] = Space
537+ f _ ils = Span nullAttr ils
538+parseRawLaTeX _ t = RawInline (Format "latex") t
539+
777540latex' :: Text -> Bib [Block]
778541latex' t = do
779542 lang <- gets localeLang
@@ -809,7 +572,7 @@ bibEntries = do
809572 (bibComment <|> bibPreamble <|> bibString))
810573
811574bibSkip :: BibParser ()
812−bibSkip = skipMany1 (satisfy (/='@'))
575+bibSkip = () <$ take1WhileP (/='@')
813576
814577bibComment :: BibParser ()
815578bibComment = do
@@ -834,14 +597,11 @@ bibString = do
834597 updateState (\(l,m) -> (l, Map.insert k v m))
835598 return ()
836599
837−take1WhileP :: Monad m => (Char -> Bool) -> ParserT Sources u m Text
838−take1WhileP f = T.pack <$> many1 (satisfy f)
839−
840600inBraces :: BibParser Text
841601inBraces = do
842602 char '{'
843603 res <- manyTill
844− ( take1WhileP (\c -> c /= '{' && c /= '}' && c /= '\\')
604+ ( (T.pack <$> many1 (noneOf "{}\\"))
845605 <|> (char '\\' >> ( (char '{' >> return "\\{")
846606 <|> (char '}' >> return "\\}")
847607 <|> return "\\"))
@@ -856,7 +616,7 @@ inQuotes :: BibParser Text
856616inQuotes = do
857617 char '"'
858618 T.concat <$> manyTill
859− ( take1WhileP (\c -> c /= '{' && c /= '"' && c /= '\\')
619+ ( (T.pack <$> many1 (noneOf "\"\\{"))
860620 <|> (char '\\' >> T.cons '\\' . T.singleton <$> anyChar)
861621 <|> braced <$> inBraces
862622 ) (char '"')
@@ -868,7 +628,7 @@ fieldName = resolveAlias . T.toLower
868628
869629isBibtexKeyChar :: Char -> Bool
870630isBibtexKeyChar c =
871− isAlphaNum c || c `elem` (".:;?!`'()$/*@_+=-[]*&" :: [Char])
631+ isAlphaNum c || c `elem` (".:;?!`'()/*@_+=-[]*&" :: [Char])
872632
873633bibItem :: BibParser Item
874634bibItem = do
@@ -1052,14 +812,14 @@ getOldDate prefix = do
1052812 let dateparts = filter (\x -> x /= DateParts [])
1053813 $ map toDateParts [(year',month',day'),
1054814 (endyear',endmonth',endday')]
1055− literal' <- if null dateparts
1056− then Just <$> getRawField (prefix <> "year")
1057− else return Nothing
815+ literal <- if null dateparts
816+ then Just <$> getRawField (prefix <> "year")
817+ else return Nothing
1058818 return $
1059819 Date { dateParts = dateparts
1060820 , dateCirca = False
1061821 , dateSeason = Nothing
1062− , dateLiteral = literal' }
822+ , dateLiteral = literal }
1063823
1064824getRawField :: Text -> Bib Text
1065825getRawField f = do
@@ -1465,9 +1225,8 @@ resolveKey lang ils = Walk.walk go ils
14651225 go x = x
14661226
14671227resolveKey' :: Lang -> Text -> Text
1468−resolveKey' lang k =
1469− case Map.lookup (langLanguage lang) biblatexStringMap >>=
1470− Map.lookup (T.toLower k) of
1228+resolveKey' lang@(Lang l _) k =
1229+ case Map.lookup l biblatexStringMap >>= Map.lookup (T.toLower k) of
14711230 Nothing -> k
14721231 Just (x, _) -> either (const k) stringify $ parseLaTeX lang x
14731232
src/Text/Pandoc/Citeproc/Data.hs+6−7
@@ -10,7 +10,7 @@ import qualified Data.Text.Encoding as TE
1010import qualified Data.Text as T
1111import Data.Text (Text)
1212import Text.Pandoc.Citeproc.Util (toIETF)
13−import Text.Collate.Lang (Lang(..), parseLang)
13+import Citeproc (Lang(..), parseLang)
1414
1515biblatexLocalizations :: [(FilePath, ByteString)]
1616biblatexLocalizations = $(embedDir "citeproc/biblatex-localization")
@@ -21,12 +21,11 @@ biblatexStringMap :: M.Map Text (M.Map Text (Text, Text))
2121biblatexStringMap = foldr go mempty biblatexLocalizations
2222 where
2323 go (fp, bs) =
24− let ls = T.lines $ TE.decodeUtf8 bs
25− in case parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp) of
26− Right lang | length ls > 4
27− -> M.insert (langLanguage lang)
28− (toStringMap $ map (T.splitOn "|") ls)
29− _ -> id
24+ let Lang lang _ = parseLang (toIETF $ T.takeWhile (/= '.') $ T.pack fp)
25+ ls = T.lines $ TE.decodeUtf8 bs
26+ in if length ls > 4
27+ then M.insert lang (toStringMap $ map (T.splitOn "|") ls)
28+ else id
3029 toStringMap = foldr go' mempty
3130 go' [term, x, y] = M.insert term (x, y)
3231 go' _ = id
src/Text/Pandoc/Citeproc/Locator.hs+4−20
@@ -7,7 +7,6 @@ where
77import Citeproc.Types
88import Data.Text (Text)
99import qualified Data.Text as T
10−import Data.List (foldl')
1110import Text.Parsec
1211import Text.Pandoc.Definition
1312import Text.Pandoc.Parsing (romanNumeral)
@@ -20,7 +19,7 @@ parseLocator :: Locale -> [Inline] -> (Maybe (Text, Text), [Inline])
2019parseLocator locale inp =
2120 case parse (pLocatorWords (toLocatorMap locale)) "suffix" $ splitInp inp of
2221 Right r -> r
23− Left _ -> (Nothing, maybeAddComma inp)
22+ Left _ -> (Nothing, inp)
2423
2524splitInp :: [Inline] -> [Inline]
2625splitInp = splitStrWhen (\c -> isSpace c || (isPunctuation c && c /= ':'))
@@ -42,17 +41,9 @@ pLocatorWords locMap = do
4241 -- i.e. the first one will be " 9"
4342 return $
4443 if T.null la && T.null lo
45− then (Nothing, maybeAddComma s)
44+ then (Nothing, s)
4645 else (Just (la, T.strip lo), s)
4746
48−maybeAddComma :: [Inline] -> [Inline]
49−maybeAddComma [] = []
50−maybeAddComma ils@(Space : _) = ils
51−maybeAddComma ils@(Str t : _)
52− | Just (c, _) <- T.uncons t
53− , isPunctuation c = ils
54−maybeAddComma ils = Str "," : Space : ils
55−
5647pLocatorDelimited :: LocatorMap -> LocatorParser (Text, Text)
5748pLocatorDelimited locMap = try $ do
5849 _ <- pMatchChar "{" (== '{')
@@ -105,7 +96,7 @@ pLocatorLabel' locMap lim = go ""
10596 t <- anyToken
10697 ts <- manyTill anyToken (try $ lookAhead lim)
10798 let s = acc <> stringify (t:ts)
108− case M.lookup (T.toCaseFold $ T.strip s) locMap of
99+ case M.lookup (T.strip s) locMap of
109100 -- try to find a longer one, or return this one
110101 Just l -> go s <|> return (l, False)
111102 Nothing -> go s
@@ -148,7 +139,7 @@ pBalancedBraces braces p = try $ do
148139 where
149140 except = notFollowedBy pBraces >> p
150141 -- outer and inner
151− surround = foldl' (\a (open, close) -> sur open close except <|> a)
142+ surround = foldl (\a (open, close) -> sur open close except <|> a)
152143 except
153144 braces
154145
@@ -189,7 +180,6 @@ pPageUnit = roman <|> plainUnit
189180 plainUnit = do
190181 ts <- many1 (notFollowedBy pSpace >>
191182 notFollowedBy pLocatorPunct >>
192− notFollowedBy pMath >>
193183 anyToken)
194184 let s = stringify ts
195185 -- otherwise look for actual digits or -s
@@ -220,12 +210,6 @@ pMatchChar msg f = satisfyTok f' <?> msg
220210pSpace :: LocatorParser Inline
221211pSpace = satisfyTok (\t -> isSpacey t || t == Str "\160") <?> "space"
222212
223−pMath :: LocatorParser Inline
224−pMath = satisfyTok isMath
225− where
226− isMath (Math{}) = True
227− isMath _ = False
228−
229213satisfyTok :: (Inline -> Bool) -> LocatorParser Inline
230214satisfyTok f = tokenPrim show (\sp _ _ -> sp) (\tok -> if f tok
231215 then Just tok
src/Text/Pandoc/Citeproc/MetaValue.hs+3−5
@@ -135,13 +135,12 @@ metaValueToVal k v
135135 MetaMap _ -> TextVal mempty
136136
137137metaValueToDate :: MetaValue -> Date
138−metaValueToDate (MetaMap m) = fromMaybe
139− (Date
138+metaValueToDate (MetaMap m) =
139+ Date
140140 { dateParts = dateparts
141141 , dateCirca = circa
142142 , dateSeason = season
143− , dateLiteral = literal })
144− rawdate
143+ , dateLiteral = literal }
145144 where
146145 dateparts = case M.lookup "date-parts" m of
147146 Just (MetaList xs) ->
@@ -153,7 +152,6 @@ metaValueToDate (MetaMap m) = fromMaybe
153152 M.lookup "circa" m >>= metaValueToBool
154153 season = M.lookup "season" m >>= metaValueToInt
155154 literal = M.lookup "literal" m >>= metaValueToText
156− rawdate = M.lookup "raw" m >>= metaValueToText >>= rawDateEDTF
157155metaValueToDate (MetaList xs) =
158156 Date{ dateParts = mapMaybe metaValueToDateParts xs
159157 , dateCirca = False
src/Text/Pandoc/Class/CommonState.hs+1−1
@@ -19,7 +19,7 @@ where
1919
2020import Data.Default (Default (def))
2121import Data.Text (Text)
22−import Text.Collate.Lang (Lang)
22+import Text.Pandoc.BCP47 (Lang)
2323import Text.Pandoc.MediaBag (MediaBag)
2424import Text.Pandoc.Logging (LogMessage, Verbosity (WARNING))
2525import Text.Pandoc.Translations (Translations)
src/Text/Pandoc/Class/IO.hs+25−24
@@ -62,7 +62,7 @@ import Text.Pandoc.Definition (Pandoc, Inline (Image))
6262import Text.Pandoc.Error (PandocError (..))
6363import Text.Pandoc.Logging (LogMessage (..), messageVerbosity, showLogMessage)
6464import Text.Pandoc.MIME (MimeType)
65−import Text.Pandoc.MediaBag (MediaBag, MediaItem(..), lookupMedia, mediaItems)
65+import Text.Pandoc.MediaBag (MediaBag, lookupMedia, mediaDirectory)
6666import Text.Pandoc.Walk (walk)
6767import qualified Control.Exception as E
6868import qualified Data.ByteString as B
@@ -183,7 +183,7 @@ getModificationTime = liftIOError System.Directory.getModificationTime
183183logOutput :: (PandocMonad m, MonadIO m) => LogMessage -> m ()
184184logOutput msg = liftIO $ do
185185 UTF8.hPutStr stderr $
186− "[" <> T.pack (show (messageVerbosity msg)) <> "] "
186+ "[" ++ show (messageVerbosity msg) ++ "] "
187187 alertIndent $ T.lines $ showLogMessage msg
188188
189189-- | Prints the list of lines to @stderr@, indenting every but the first
@@ -191,41 +191,42 @@ logOutput msg = liftIO $ do
191191alertIndent :: [Text] -> IO ()
192192alertIndent [] = return ()
193193alertIndent (l:ls) = do
194− UTF8.hPutStrLn stderr l
194+ UTF8.hPutStrLn stderr $ unpack l
195195 mapM_ go ls
196196 where go l' = do UTF8.hPutStr stderr " "
197− UTF8.hPutStrLn stderr l'
197+ UTF8.hPutStrLn stderr $ unpack l'
198198
199199-- | Extract media from the mediabag into a directory.
200200extractMedia :: (PandocMonad m, MonadIO m) => FilePath -> Pandoc -> m Pandoc
201201extractMedia dir d = do
202202 media <- getMediaBag
203− let items = mediaItems media
204− if null items
205− then return d
206− else do
207− mapM_ (writeMedia dir) items
208− return $ walk (adjustImagePath dir media) d
203+ case [fp | (fp, _, _) <- mediaDirectory media] of
204+ [] -> return d
205+ fps -> do
206+ mapM_ (writeMedia dir media) fps
207+ return $ walk (adjustImagePath dir fps) d
209208
210209-- | Write the contents of a media bag to a path.
211210writeMedia :: (PandocMonad m, MonadIO m)
212− => FilePath
213− -> (FilePath, MimeType, BL.ByteString)
211+ => FilePath -> MediaBag -> FilePath
214212 -> m ()
215−writeMedia dir (fp, _mt, bs) = do
216− -- we normalize to get proper path separators for the platform
217− let fullpath = normalise $ dir </> fp
218− liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
219− logIOError $ BL.writeFile fullpath bs
213+writeMedia dir mediabag subpath = do
214+ -- we join and split to convert a/b/c to a\b\c on Windows;
215+ -- in zip containers all paths use /
216+ let fullpath = dir </> unEscapeString (normalise subpath)
217+ let mbcontents = lookupMedia subpath mediabag
218+ case mbcontents of
219+ Nothing -> throwError $ PandocResourceNotFound $ pack subpath
220+ Just (_, bs) -> do
221+ report $ Extracting $ pack fullpath
222+ liftIOError (createDirectoryIfMissing True) (takeDirectory fullpath)
223+ logIOError $ BL.writeFile fullpath bs
220224
221225-- | If the given Inline element is an image with a @src@ path equal to
222226-- one in the list of @paths@, then prepends @dir@ to the image source;
223227-- returns the element unchanged otherwise.
224−adjustImagePath :: FilePath -> MediaBag -> Inline -> Inline
225−adjustImagePath dir mediabag (Image attr lab (src, tit)) =
226− case lookupMedia (T.unpack src) mediabag of
227− Nothing -> Image attr lab (src, tit)
228− Just item ->
229− let fullpath = dir </> mediaPath item
230− in Image attr lab (T.pack fullpath, tit)
228+adjustImagePath :: FilePath -> [FilePath] -> Inline -> Inline
229+adjustImagePath dir paths (Image attr lab (src, tit))
230+ | unpack src `elem` paths
231+ = Image attr lab (pack (normalise $ dir </> unpack src), tit)
231232adjustImagePath _ _ x = x
src/Text/Pandoc/Class/PandocMonad.hs+36−43
@@ -1,5 +1,4 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE TupleSections #-}
32{-# LANGUAGE FlexibleContexts #-}
43{-# LANGUAGE FlexibleInstances #-}
54{-# LANGUAGE MultiParamTypeClasses #-}
@@ -38,6 +37,7 @@ module Text.Pandoc.Class.PandocMonad
3837 , setUserDataDir
3938 , getUserDataDir
4039 , fetchItem
40+ , fetchMediaResource
4141 , getInputFiles
4242 , setInputFiles
4343 , getOutputFile
@@ -51,31 +51,30 @@ module Text.Pandoc.Class.PandocMonad
5151 , setTranslations
5252 , translateTerm
5353 , makeCanonical
54− , getTimestamp
5554 ) where
5655
5756import Codec.Archive.Zip
5857import Control.Monad.Except (MonadError (catchError, throwError),
5958 MonadTrans, lift, when)
60−import Data.List (foldl')
59+import Data.Digest.Pure.SHA (sha1, showDigest)
60+import Data.Maybe (fromMaybe)
6161import Data.Time (UTCTime)
62−import Data.Time.Clock.POSIX (POSIXTime, utcTimeToPOSIXSeconds,
63− posixSecondsToUTCTime)
62+import Data.Time.Clock.POSIX (POSIXTime, utcTimeToPOSIXSeconds)
6463import Data.Time.LocalTime (TimeZone, ZonedTime, utcToZonedTime)
6564import Network.URI ( escapeURIString, nonStrictRelativeTo,
6665 unEscapeString, parseURIReference, isAllowedInURI,
6766 parseURI, URI(..) )
68−import System.FilePath ((</>), takeExtension, dropExtension,
69− isRelative, splitDirectories, makeRelative)
67+import System.FilePath ((</>), (<.>), takeExtension, dropExtension,
68+ isRelative, splitDirectories)
7069import System.Random (StdGen)
71−import Text.Collate.Lang (Lang(..), parseLang, renderLang)
70+import Text.Pandoc.BCP47 (Lang(..), parseBCP47, renderLang)
7271import Text.Pandoc.Class.CommonState (CommonState (..))
7372import Text.Pandoc.Definition
7473import Text.Pandoc.Error
7574import Text.Pandoc.Logging
76−import Text.Pandoc.MIME (MimeType, getMimeType)
77−import Text.Pandoc.MediaBag (MediaBag, lookupMedia, MediaItem(..))
78−import Text.Pandoc.Shared (uriPathToPath, safeRead)
75+import Text.Pandoc.MIME (MimeType, getMimeType, extensionFromMimeType)
76+import Text.Pandoc.MediaBag (MediaBag, lookupMedia)
77+import Text.Pandoc.Shared (uriPathToPath)
7978import Text.Pandoc.Translations (Term(..), Translations, lookupTerm,
8079 readTranslations)
8180import Text.Pandoc.Walk (walkM)
@@ -176,21 +175,6 @@ report msg = do
176175 when (level <= verbosity) $ logOutput msg
177176 modifyCommonState $ \st -> st{ stLog = msg : stLog st }
178177
179−-- | Get the time from the @SOURCE_DATE_EPOCH@
180−-- environment variable. The variable should contain a
181−-- unix time stamp, the number of seconds since midnight Jan 01
182−-- 1970 UTC. If the variable is not set or cannot be
183−-- parsed as a unix time stamp, the current time is returned.
184−-- This function is designed to make possible reproducible
185−-- builds in formats that include a creation timestamp.
186−getTimestamp :: PandocMonad m => m UTCTime
187−getTimestamp = do
188− mbSourceDateEpoch <- lookupEnv "SOURCE_DATE_EPOCH"
189− case mbSourceDateEpoch >>= safeRead of
190− Just (epoch :: Integer) ->
191− return $ posixSecondsToUTCTime $ fromIntegral epoch
192− Nothing -> getCurrentTime
193−
194178-- | Determine whether tracing is enabled. This affects
195179-- the behavior of 'trace'. If tracing is not enabled,
196180-- 'trace' does nothing.
@@ -283,7 +267,7 @@ readFileFromDirs (d:ds) f = catchError
283267toLang :: PandocMonad m => Maybe T.Text -> m (Maybe Lang)
284268toLang Nothing = return Nothing
285269toLang (Just s) =
286− case parseLang s of
270+ case parseBCP47 s of
287271 Left _ -> do
288272 report $ InvalidLang s
289273 return Nothing
@@ -374,8 +358,7 @@ fetchItem :: PandocMonad m
374358fetchItem s = do
375359 mediabag <- getMediaBag
376360 case lookupMedia (T.unpack s) mediabag of
377− Just item -> return (BL.toStrict (mediaContents item),
378− Just (mediaMimeType item))
361+ Just (mime, bs) -> return (BL.toStrict bs, Just mime)
379362 Nothing -> downloadOrRead s
380363
381364-- | Returns the content and, if available, the MIME type of a resource.
@@ -410,10 +393,9 @@ downloadOrRead s = do
410393 _ -> readLocalFile fp -- get from local file system
411394 where readLocalFile f = do
412395 resourcePath <- getResourcePath
413− (fp', cont) <- if isRelative f
414− then withPaths resourcePath readFileStrict f
415− else (f,) <$> readFileStrict f
416− report $ LoadedResource f (makeRelative "." fp')
396+ cont <- if isRelative f
397+ then withPaths resourcePath readFileStrict f
398+ else readFileStrict f
417399 return (cont, mime)
418400 httpcolon = URI{ uriScheme = "http:",
419401 uriAuthority = Nothing,
@@ -613,7 +595,7 @@ checkExistence fn = do
613595-- | Canonicalizes a file path by removing redundant @.@ and @..@.
614596makeCanonical :: FilePath -> FilePath
615597makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
616− where transformPathParts = reverse . foldl' go []
598+ where transformPathParts = reverse . foldl go []
617599 go as "." = as
618600 go (_:as) ".." = as
619601 go as x = x : as
@@ -623,13 +605,25 @@ makeCanonical = Posix.joinPath . transformPathParts . splitDirectories
623605-- that filepath. Returns the result of the first successful execution
624606-- of the action, or throws a @PandocResourceNotFound@ exception if the
625607-- action errors for all filepaths.
626−withPaths :: PandocMonad m
627− => [FilePath] -> (FilePath -> m a) -> FilePath -> m (FilePath, a)
608+withPaths :: PandocMonad m => [FilePath] -> (FilePath -> m a) -> FilePath -> m a
628609withPaths [] _ fp = throwError $ PandocResourceNotFound $ T.pack fp
629610withPaths (p:ps) action fp =
630− catchError ((p </> fp,) <$> action (p </> fp))
611+ catchError (action (p </> fp))
631612 (\_ -> withPaths ps action fp)
632613
614+-- | Fetch local or remote resource (like an image) and provide data suitable
615+-- for adding it to the MediaBag.
616+fetchMediaResource :: PandocMonad m
617+ => T.Text -> m (FilePath, Maybe MimeType, BL.ByteString)
618+fetchMediaResource src = do
619+ (bs, mt) <- downloadOrRead src
620+ let ext = fromMaybe (T.pack $ takeExtension $ T.unpack src)
621+ (mt >>= extensionFromMimeType)
622+ let bs' = BL.fromChunks [bs]
623+ let basename = showDigest $ sha1 bs'
624+ let fname = basename <.> T.unpack ext
625+ return (fname, mt, bs')
626+
633627-- | Traverse tree, filling media bag for any images that
634628-- aren't already in the media bag.
635629fillMediaBag :: PandocMonad m => Pandoc -> m Pandoc
@@ -637,13 +631,12 @@ fillMediaBag d = walkM handleImage d
637631 where handleImage :: PandocMonad m => Inline -> m Inline
638632 handleImage (Image attr lab (src, tit)) = catchError
639633 (do mediabag <- getMediaBag
640− let fp = T.unpack src
641− case lookupMedia fp mediabag of
642− Just _ -> return ()
634+ case lookupMedia (T.unpack src) mediabag of
635+ Just (_, _) -> return $ Image attr lab (src, tit)
643636 Nothing -> do
644− (bs, mt) <- fetchItem src
645− insertMedia fp mt (BL.fromStrict bs)
646− return $ Image attr lab (src, tit))
637+ (fname, mt, bs) <- fetchMediaResource src
638+ insertMedia fname mt bs
639+ return $ Image attr lab (T.pack fname, tit))
647640 (\e ->
648641 case e of
649642 PandocResourceNotFound _ -> do
src/Text/Pandoc/Data.hs+1−1
@@ -1,7 +1,7 @@
11{-# LANGUAGE TemplateHaskell #-}
22{- |
33Module : Text.Pandoc.Data
4−Copyright : Copyright (C) 2013-2021 John MacFarlane
4+Copyright : Copyright (C) 2013-2020 John MacFarlane
55License : GNU GPL, version 2 or above
66
77Maintainer : John MacFarlane <jgm@berkeley@edu>
src/Text/Pandoc/Error.hs+49−102
@@ -3,7 +3,7 @@
33{-# LANGUAGE OverloadedStrings #-}
44{- |
55 Module : Text.Pandoc.Error
6− Copyright : Copyright (C) 2006-2021 John MacFarlane
6+ Copyright : Copyright (C) 2006-2020 John MacFarlane
77 License : GNU GPL, version 2 or above
88
99 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -16,41 +16,38 @@ encountered during parsing.
1616-}
1717module Text.Pandoc.Error (
1818 PandocError(..),
19− renderError,
2019 handleError) where
2120
22−import Control.Exception (Exception, displayException)
21+import Control.Exception (Exception)
2322import Data.Typeable (Typeable)
2423import Data.Word (Word8)
2524import Data.Text (Text)
26−import Data.List (sortOn)
2725import qualified Data.Text as T
28−import Data.Ord (Down(..))
2926import GHC.Generics (Generic)
3027import Network.HTTP.Client (HttpException)
3128import System.Exit (ExitCode (..), exitWith)
3229import System.IO (stderr)
3330import qualified Text.Pandoc.UTF8 as UTF8
34−import Text.Pandoc.Sources (Sources(..))
3531import Text.Printf (printf)
3632import Text.Parsec.Error
3733import Text.Parsec.Pos hiding (Line)
3834import Text.Pandoc.Shared (tshow)
3935import Citeproc (CiteprocError, prettyCiteprocError)
4036
37+type Input = Text
38+
4139data PandocError = PandocIOError Text IOError
4240 | PandocHttpError Text HttpException
4341 | PandocShouldNeverHappenError Text
4442 | PandocSomeError Text
4543 | PandocParseError Text
46− | PandocParsecError Sources ParseError
44+ | PandocParsecError Input ParseError
4745 | PandocMakePDFError Text
4846 | PandocOptionError Text
4947 | PandocSyntaxMapError Text
5048 | PandocFailOnWarningError
5149 | PandocPDFProgramNotFoundError Text
5250 | PandocPDFError Text
53− | PandocXMLError Text Text
5451 | PandocFilterError Text Text
5552 | PandocLuaError Text
5653 | PandocCouldNotFindDataFileError Text
@@ -61,7 +58,6 @@ data PandocError = PandocIOError Text IOError
6158 | PandocMacroLoop Text
6259 | PandocUTF8DecodingError Text Int Word8
6360 | PandocIpynbDecodingError Text
64− | PandocUnsupportedCharsetError Text
6561 | PandocUnknownReaderError Text
6662 | PandocUnknownWriterError Text
6763 | PandocUnsupportedExtensionError Text Text
@@ -71,70 +67,62 @@ data PandocError = PandocIOError Text IOError
7167
7268instance Exception PandocError
7369
74−renderError :: PandocError -> Text
75−renderError e =
70+-- | Handle PandocError by exiting with an error message.
71+handleError :: Either PandocError a -> IO a
72+handleError (Right r) = return r
73+handleError (Left e) =
7674 case e of
77− PandocIOError _ err' -> T.pack $ displayException err'
78− PandocHttpError u err' ->
75+ PandocIOError _ err' -> ioError err'
76+ PandocHttpError u err' -> err 61 $
7977 "Could not fetch " <> u <> "\n" <> tshow err'
80− PandocShouldNeverHappenError s ->
78+ PandocShouldNeverHappenError s -> err 62 $
8179 "Something we thought was impossible happened!\n" <>
8280 "Please report this to pandoc's developers: " <> s
83− PandocSomeError s -> s
84− PandocParseError s -> s
85− PandocParsecError (Sources inputs) err' ->
81+ PandocSomeError s -> err 63 s
82+ PandocParseError s -> err 64 s
83+ PandocParsecError input err' ->
8684 let errPos = errorPos err'
8785 errLine = sourceLine errPos
8886 errColumn = sourceColumn errPos
89− errFile = sourceName errPos
90− errorInFile =
91− case sortOn (Down . sourceLine . fst)
92− [ (pos,t)
93− | (pos,t) <- inputs
94− , sourceName pos == errFile
95− , sourceLine pos <= errLine
96− ] of
97− [] -> ""
98− ((pos,txt):_) ->
99− let ls = T.lines txt <> [""]
100− ln = (errLine - sourceLine pos) + 1
101− in if length ls > ln && ln >= 1
102− then T.concat ["\n", ls !! (ln - 1)
103− ,"\n", T.replicate (errColumn - 1) " "
104− ,"^"]
105− else ""
106− in "Error at " <> tshow err' <> errorInFile
107− PandocMakePDFError s -> s
108− PandocOptionError s -> s
109− PandocSyntaxMapError s -> s
110− PandocFailOnWarningError -> "Failing because there were warnings."
111− PandocPDFProgramNotFoundError pdfprog ->
87+ ls = T.lines input <> [""]
88+ errorInFile = if length ls > errLine - 1
89+ then T.concat ["\n", ls !! (errLine - 1)
90+ ,"\n", T.replicate (errColumn - 1) " "
91+ ,"^"]
92+ else ""
93+ in err 65 $ "\nError at " <> tshow err' <>
94+ -- if error comes from a chunk or included file,
95+ -- then we won't get the right text this way:
96+ if sourceName errPos == "source"
97+ then errorInFile
98+ else ""
99+ PandocMakePDFError s -> err 66 s
100+ PandocOptionError s -> err 6 s
101+ PandocSyntaxMapError s -> err 67 s
102+ PandocFailOnWarningError -> err 3 "Failing because there were warnings."
103+ PandocPDFProgramNotFoundError pdfprog -> err 47 $
112104 pdfprog <> " not found. Please select a different --pdf-engine or install " <> pdfprog
113− PandocPDFError logmsg -> "Error producing PDF.\n" <> logmsg
114− PandocXMLError fp logmsg -> "Invalid XML" <>
115− (if T.null fp then "" else " in " <> fp) <> ":\n" <> logmsg
116− PandocFilterError filtername msg -> "Error running filter " <>
105+ PandocPDFError logmsg -> err 43 $ "Error producing PDF.\n" <> logmsg
106+ PandocFilterError filtername msg -> err 83 $ "Error running filter " <>
117107 filtername <> ":\n" <> msg
118− PandocLuaError msg -> "Error running Lua:\n" <> msg
119− PandocCouldNotFindDataFileError fn ->
108+ PandocLuaError msg -> err 84 $ "Error running Lua:\n" <> msg
109+ PandocCouldNotFindDataFileError fn -> err 97 $
120110 "Could not find data file " <> fn
121− PandocResourceNotFound fn ->
111+ PandocResourceNotFound fn -> err 99 $
122112 "File " <> fn <> " not found in resource path"
123− PandocTemplateError s -> "Error compiling template " <> s
124− PandocAppError s -> s
125− PandocEpubSubdirectoryError s ->
113+ PandocTemplateError s -> err 5 $ "Error compiling template " <> s
114+ PandocAppError s -> err 4 s
115+ PandocEpubSubdirectoryError s -> err 31 $
126116 "EPUB subdirectory name '" <> s <> "' contains illegal characters"
127− PandocMacroLoop s ->
117+ PandocMacroLoop s -> err 91 $
128118 "Loop encountered in expanding macro " <> s
129− PandocUTF8DecodingError f offset w ->
119+ PandocUTF8DecodingError f offset w -> err 92 $
130120 "UTF-8 decoding error in " <> f <> " at byte offset " <> tshow offset <>
131121 " (" <> T.pack (printf "%2x" w) <> ").\n" <>
132122 "The input must be a UTF-8 encoded text."
133− PandocIpynbDecodingError w ->
123+ PandocIpynbDecodingError w -> err 93 $
134124 "ipynb decoding error: " <> w
135− PandocUnsupportedCharsetError charset ->
136− "Unsupported charset " <> charset
137− PandocUnknownReaderError r ->
125+ PandocUnknownReaderError r -> err 21 $
138126 "Unknown input format " <> r <>
139127 case r of
140128 "doc" -> "\nPandoc can convert from DOCX, but not from DOC." <>
@@ -142,7 +130,7 @@ renderError e =
142130 " and convert that with pandoc."
143131 "pdf" -> "\nPandoc can convert to PDF, but not from PDF."
144132 _ -> ""
145− PandocUnknownWriterError w ->
133+ PandocUnknownWriterError w -> err 22 $
146134 "Unknown output format " <> w <>
147135 case w of
148136 "pdf" -> "To create a pdf using pandoc, use" <>
@@ -151,57 +139,16 @@ renderError e =
151139 ".pdf extension (-o filename.pdf)."
152140 "doc" -> "\nPandoc can convert to DOCX, but not to DOC."
153141 _ -> ""
154− PandocUnsupportedExtensionError ext f ->
142+ PandocUnsupportedExtensionError ext f -> err 23 $
155143 "The extension " <> ext <> " is not supported " <>
156144 "for " <> f
157− PandocCiteprocError e' ->
145+ PandocCiteprocError e' -> err 24 $
158146 prettyCiteprocError e'
159− PandocBibliographyError fp msg ->
147+ PandocBibliographyError fp msg -> err 25 $
160148 "Error reading bibliography file " <> fp <> ":\n" <> msg
161149
162−
163−-- | Handle PandocError by exiting with an error message.
164−handleError :: Either PandocError a -> IO a
165−handleError (Right r) = return r
166−handleError (Left e) =
167− case e of
168− PandocIOError _ err' -> ioError err'
169− _ -> err exitCode (renderError e)
170− where
171− exitCode =
172− case e of
173− PandocIOError{} -> 1
174− PandocHttpError{} -> 61
175− PandocShouldNeverHappenError{} -> 62
176− PandocSomeError{} -> 63
177− PandocParseError{} -> 64
178− PandocParsecError{} -> 65
179− PandocMakePDFError{} -> 66
180− PandocOptionError{} -> 6
181− PandocSyntaxMapError{} -> 67
182− PandocFailOnWarningError{} -> 3
183− PandocPDFProgramNotFoundError{} -> 47
184− PandocPDFError{} -> 43
185− PandocXMLError{} -> 44
186− PandocFilterError{} -> 83
187− PandocLuaError{} -> 84
188− PandocCouldNotFindDataFileError{} -> 97
189− PandocResourceNotFound{} -> 99
190− PandocTemplateError{} -> 5
191− PandocAppError{} -> 4
192− PandocEpubSubdirectoryError{} -> 31
193− PandocMacroLoop{} -> 91
194− PandocUTF8DecodingError{} -> 92
195− PandocIpynbDecodingError{} -> 93
196− PandocUnsupportedCharsetError{} -> 94
197− PandocUnknownReaderError{} -> 21
198− PandocUnknownWriterError{} -> 22
199− PandocUnsupportedExtensionError{} -> 23
200− PandocCiteprocError{} -> 24
201− PandocBibliographyError{} -> 25
202−
203150err :: Int -> Text -> IO a
204151err exitCode msg = do
205− UTF8.hPutStrLn stderr msg
152+ UTF8.hPutStrLn stderr (T.unpack msg)
206153 exitWith $ ExitFailure exitCode
207154 return undefined
src/Text/Pandoc/Extensions.hs+4−32
@@ -6,7 +6,7 @@
66{-# LANGUAGE OverloadedStrings #-}
77{- |
88 Module : Text.Pandoc.Extensions
9− Copyright : Copyright (C) 2012-2021 John MacFarlane
9+ Copyright : Copyright (C) 2012-2020 John MacFarlane
1010 License : GNU GPL, version 2 or above
1111
1212 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -34,7 +34,6 @@ module Text.Pandoc.Extensions ( Extension(..)
3434where
3535import Data.Bits (clearBit, setBit, testBit, (.|.))
3636import Data.Data (Data)
37−import Data.List (foldl')
3837import qualified Data.Text as T
3938import Data.Typeable (Typeable)
4039import GHC.Generics (Generic)
@@ -89,7 +88,6 @@ data Extension =
8988 -- does not affect readers/writers directly; it causes
9089 -- the eastAsianLineBreakFilter to be applied after
9190 -- parsing, in Text.Pandoc.App.convertWithOpts.
92− | Ext_element_citations -- ^ Use element-citation elements for JATS citations
9391 | Ext_emoji -- ^ Support emoji like :smile:
9492 | Ext_empty_paragraphs -- ^ Allow empty paragraphs
9593 | Ext_epub_html_exts -- ^ Recognise the EPUB extended version of HTML
@@ -136,8 +134,6 @@ data Extension =
136134 | Ext_raw_html -- ^ Allow raw HTML
137135 | Ext_raw_tex -- ^ Allow raw TeX (other than math)
138136 | Ext_raw_markdown -- ^ Parse markdown in ipynb as raw markdown
139− | Ext_rebase_relative_paths -- ^ Rebase relative image and link paths,
140− -- relative to directory of containing file
141137 | Ext_shortcut_reference_links -- ^ Shortcut reference links
142138 | Ext_simple_tables -- ^ Pandoc-style simple tables
143139 | Ext_smart -- ^ "Smart" quotes, apostrophes, ellipses, dashes
@@ -153,12 +149,9 @@ data Extension =
153149 | Ext_tex_math_dollars -- ^ TeX math between $..$ or $$..$$
154150 | Ext_tex_math_double_backslash -- ^ TeX math btw \\(..\\) \\[..\\]
155151 | Ext_tex_math_single_backslash -- ^ TeX math btw \(..\) \[..\]
156− | Ext_xrefs_name -- ^ Use xrefs with names
157− | Ext_xrefs_number -- ^ Use xrefs with numbers
158152 | Ext_yaml_metadata_block -- ^ YAML metadata block
159153 | Ext_gutenberg -- ^ Use Project Gutenberg conventions for plain
160154 | Ext_attributes -- ^ Generic attribute syntax
161− | Ext_sourcepos -- ^ Include source position attributes
162155 deriving (Show, Read, Enum, Eq, Ord, Bounded, Data, Typeable, Generic)
163156
164157-- | Extensions to be used with pandoc-flavored markdown.
@@ -356,7 +349,6 @@ getDefaultExtensions "gfm" = extensionsFromList
356349 , Ext_strikeout
357350 , Ext_task_lists
358351 , Ext_emoji
359− , Ext_yaml_metadata_block
360352 ]
361353getDefaultExtensions "commonmark" = extensionsFromList
362354 [Ext_raw_html]
@@ -382,11 +374,10 @@ getDefaultExtensions "commonmark_x" = extensionsFromList
382374 , Ext_raw_attribute
383375 , Ext_implicit_header_references
384376 , Ext_attributes
385− , Ext_yaml_metadata_block
377+ , Ext_fenced_code_attributes
386378 ]
387379getDefaultExtensions "org" = extensionsFromList
388380 [Ext_citations,
389− Ext_task_lists,
390381 Ext_auto_identifiers]
391382getDefaultExtensions "html" = extensionsFromList
392383 [Ext_auto_identifiers,
@@ -418,11 +409,6 @@ getDefaultExtensions "textile" = extensionsFromList
418409 Ext_smart,
419410 Ext_raw_html,
420411 Ext_auto_identifiers]
421−getDefaultExtensions "jats" = extensionsFromList
422− [Ext_auto_identifiers]
423−getDefaultExtensions "jats_archiving" = getDefaultExtensions "jats"
424−getDefaultExtensions "jats_publishing" = getDefaultExtensions "jats"
425−getDefaultExtensions "jats_articleauthoring" = getDefaultExtensions "jats"
426412getDefaultExtensions "opml" = pandocExtensions -- affects notes
427413getDefaultExtensions _ = extensionsFromList
428414 [Ext_auto_identifiers]
@@ -464,7 +450,6 @@ getAllExtensions f = universalExtensions <> getAll f
464450 , Ext_gutenberg
465451 , Ext_smart
466452 , Ext_literate_haskell
467− , Ext_rebase_relative_paths
468453 ]
469454 getAll "markdown_strict" = allMarkdownExtensions
470455 getAll "markdown_phpextra" = allMarkdownExtensions
@@ -480,8 +465,6 @@ getAllExtensions f = universalExtensions <> getAll f
480465 getAll "opendocument" = extensionsFromList
481466 [ Ext_empty_paragraphs
482467 , Ext_native_numbering
483− , Ext_xrefs_name
484− , Ext_xrefs_number
485468 ]
486469 getAll "odt" = getAll "opendocument" <> autoIdExtensions
487470 getAll "muse" = autoIdExtensions <>
@@ -515,16 +498,13 @@ getAllExtensions f = universalExtensions <> getAll f
515498 , Ext_raw_attribute
516499 , Ext_implicit_header_references
517500 , Ext_attributes
518− , Ext_sourcepos
519− , Ext_yaml_metadata_block
520− , Ext_rebase_relative_paths
501+ , Ext_fenced_code_attributes
521502 ]
522503 getAll "commonmark_x" = getAll "commonmark"
523504 getAll "org" = autoIdExtensions <>
524505 extensionsFromList
525506 [ Ext_citations
526507 , Ext_smart
527− , Ext_task_lists
528508 ]
529509 getAll "html" = autoIdExtensions <>
530510 extensionsFromList
@@ -568,14 +548,6 @@ getAllExtensions f = universalExtensions <> getAll f
568548 , Ext_smart
569549 , Ext_raw_tex
570550 ]
571− getAll "jats" =
572− extensionsFromList
573− [ Ext_auto_identifiers
574− , Ext_element_citations
575− ]
576− getAll "jats_archiving" = getAll "jats"
577− getAll "jats_publishing" = getAll "jats"
578− getAll "jats_articleauthoring" = getAll "jats"
579551 getAll "opml" = allMarkdownExtensions -- affects notes
580552 getAll "twiki" = autoIdExtensions <>
581553 extensionsFromList
@@ -601,7 +573,7 @@ parseFormatSpec :: T.Text
601573parseFormatSpec = parse formatSpec ""
602574 where formatSpec = do
603575 name <- formatName
604− (extsToEnable, extsToDisable) <- foldl' (flip ($)) ([],[]) <$>
576+ (extsToEnable, extsToDisable) <- foldl (flip ($)) ([],[]) <$>
605577 many extMod
606578 return (T.pack name, reverse extsToEnable, reverse extsToDisable)
607579 formatName = many1 $ noneOf "-+"
src/Text/Pandoc/Filter.hs+1−1
@@ -4,7 +4,7 @@
44{-# LANGUAGE OverloadedStrings #-}
55{- |
66 Module : Text.Pandoc.Filter
7− Copyright : Copyright (C) 2006-2021 John MacFarlane
7+ Copyright : Copyright (C) 2006-2020 John MacFarlane
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : John MacFarlane <jgm@berkeley@edu>
src/Text/Pandoc/Filter/JSON.hs+1−1
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Filter
4− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley@edu>
src/Text/Pandoc/Filter/Lua.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Filter.Lua
3− Copyright : Copyright (C) 2006-2021 John MacFarlane
3+ Copyright : Copyright (C) 2006-2020 John MacFarlane
44 License : GNU GPL, version 2 or above
55
66 Maintainer : John MacFarlane <jgm@berkeley@edu>
src/Text/Pandoc/Filter/Path.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Filter.Path
3− Copyright : Copyright (C) 2006-2021 John MacFarlane
3+ Copyright : Copyright (C) 2006-2020 John MacFarlane
44 License : GNU GPL, version 2 or above
55
66 Maintainer : John MacFarlane <jgm@berkeley@edu>
src/Text/Pandoc/Highlighting.hs+6−6
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Highlighting
4− Copyright : Copyright (C) 2008-2021 John MacFarlane
4+ Copyright : Copyright (C) 2008-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -52,12 +52,12 @@ highlightingStyles =
5252 ("breezedark", breezeDark),
5353 ("haddock", haddock)]
5454
55−languages :: SyntaxMap -> [T.Text]
56−languages syntaxmap = [T.toLower (sName s) | s <- M.elems syntaxmap]
55+languages :: [T.Text]
56+languages = [T.toLower (sName s) | s <- M.elems defaultSyntaxMap]
5757
58−languagesByExtension :: SyntaxMap -> T.Text -> [T.Text]
59−languagesByExtension syntaxmap ext =
60− [T.toLower (sName s) | s <- syntaxesByExtension syntaxmap (T.unpack ext)]
58+languagesByExtension :: T.Text -> [T.Text]
59+languagesByExtension ext =
60+ [T.toLower (sName s) | s <- syntaxesByExtension defaultSyntaxMap (T.unpack ext)]
6161
6262highlight :: SyntaxMap
6363 -> (FormatOptions -> [SourceLine] -> a) -- ^ Formatter
src/Text/Pandoc/Image.hs+1−1
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings, ScopedTypeVariables, CPP #-}
22{- |
33Module : Text.Pandoc.Image
4−Copyright : Copyright (C) 2020-2021 John MacFarlane
4+Copyright : Copyright (C) 2020 John MacFarlane
55License : GNU GPL, version 2 or above
66
77Maintainer : John MacFarlane <jgm@berkeley.edu>
src/Text/Pandoc/ImageSize.hs+341−55
@@ -1,9 +1,9 @@
1−{-# LANGUAGE OverloadedStrings, ScopedTypeVariables #-}
1+{-# LANGUAGE OverloadedStrings, ScopedTypeVariables, CPP #-}
22{-# LANGUAGE ViewPatterns #-}
33{-# OPTIONS_GHC -fno-warn-type-defaults #-}
44{- |
55Module : Text.Pandoc.ImageSize
6−Copyright : Copyright (C) 2011-2021 John MacFarlane
6+Copyright : Copyright (C) 2011-2020 John MacFarlane
77License : GNU GPL, version 2 or above
88
99Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -32,33 +32,33 @@ module Text.Pandoc.ImageSize ( ImageType(..)
3232 , showInPixel
3333 , showFl
3434 ) where
35−import Data.ByteString (ByteString)
35+import Data.ByteString (ByteString, unpack)
3636import qualified Data.ByteString.Char8 as B
3737import qualified Data.ByteString.Lazy as BL
38−import Data.Binary.Get
3938import Data.Char (isDigit)
4039import Control.Monad
40+import Data.Bits
41+import Data.Binary
42+import Data.Binary.Get
4143import Text.Pandoc.Shared (safeRead)
4244import Data.Default (Default)
4345import Numeric (showFFloat)
4446import Text.Pandoc.Definition
4547import Text.Pandoc.Options
4648import qualified Text.Pandoc.UTF8 as UTF8
47−import Text.Pandoc.XML.Light hiding (Attr)
49+import qualified Text.XML.Light as Xml
50+import qualified Data.Map as M
4851import qualified Data.Text as T
49−import qualified Data.Text.Lazy as TL
5052import qualified Data.Text.Encoding as TE
53+import Control.Monad.Except
5154import Control.Applicative
55+import Data.Maybe (fromMaybe)
5256import qualified Data.Attoparsec.ByteString.Char8 as A
53−import qualified Codec.Picture.Metadata as Metadata
54−import qualified Codec.Picture.Metadata.Exif as Exif
55−import Codec.Picture (decodeImageWithMetadata)
5657
5758-- quick and dirty functions to get image sizes
5859-- algorithms borrowed from wwwis.pl
5960
60−data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf | Tiff
61− deriving Show
61+data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf deriving Show
6262data Direction = Width | Height
6363instance Show Direction where
6464 show Width = "width"
@@ -101,8 +101,6 @@ imageType :: ByteString -> Maybe ImageType
101101imageType img = case B.take 4 img of
102102 "\x89\x50\x4e\x47" -> return Png
103103 "\x47\x49\x46\x38" -> return Gif
104− "\x49\x49\x2a\x00" -> return Tiff
105− "\x4D\x4D\x00\x2a" -> return Tiff
106104 "\xff\xd8\xff\xe0" -> return Jpeg -- JFIF
107105 "\xff\xd8\xff\xe1" -> return Jpeg -- Exif
108106 "%PDF" -> return Pdf
@@ -124,10 +122,9 @@ findSvgTag img = "<svg" `B.isInfixOf` img || "<SVG" `B.isInfixOf` img
124122imageSize :: WriterOptions -> ByteString -> Either T.Text ImageSize
125123imageSize opts img = checkDpi <$>
126124 case imageType img of
127− Just Png -> getSize img
128− Just Gif -> getSize img
129− Just Jpeg -> getSize img
130− Just Tiff -> getSize img
125+ Just Png -> mbToEither "could not determine PNG size" $ pngSize img
126+ Just Gif -> mbToEither "could not determine GIF size" $ gifSize img
127+ Just Jpeg -> jpegSize img
131128 Just Svg -> mbToEither "could not determine SVG size" $ svgSize opts img
132129 Just Eps -> mbToEither "could not determine EPS size" $ epsSize img
133130 Just Pdf -> mbToEither "could not determine PDF size" $ pdfSize img
@@ -142,6 +139,9 @@ imageSize opts img = checkDpi <$>
142139 , dpiY = if dpiY size == 0 then 72 else dpiY size }
143140
144141
142+defaultSize :: (Integer, Integer)
143+defaultSize = (72, 72)
144+
145145sizeInPixels :: ImageSize -> (Integer, Integer)
146146sizeInPixels s = (pxX s, pxY s)
147147
@@ -300,50 +300,59 @@ pPdfSize = do
300300 , dpiY = 72 }
301301 ) <|> pPdfSize
302302
303−getSize :: ByteString -> Either T.Text ImageSize
304−getSize img =
305− case decodeImageWithMetadata img of
306− Left e -> Left (T.pack e)
307− Right (_, meta) -> do
308− pxx <- maybe (Left "Could not determine width") Right $
309− -- first look for exif image width, then width
310− (Metadata.lookup
311− (Metadata.Exif (Exif.TagUnknown 0xA002)) meta >>=
312− exifDataToWord) <|>
313− Metadata.lookup Metadata.Width meta
314− pxy <- maybe (Left "Could not determine height") Right $
315− -- first look for exif image height, then height
316− (Metadata.lookup
317− (Metadata.Exif (Exif.TagUnknown 0xA003)) meta >>=
318− exifDataToWord) <|>
319− Metadata.lookup Metadata.Height meta
320− dpix <- maybe (Right 72) Right $ Metadata.lookup Metadata.DpiX meta
321− dpiy <- maybe (Right 72) Right $ Metadata.lookup Metadata.DpiY meta
322− return $ ImageSize
323− { pxX = fromIntegral pxx
324− , pxY = fromIntegral pxy
325− , dpiX = fromIntegral dpix
326− , dpiY = fromIntegral dpiy }
327− where
328− exifDataToWord (Exif.ExifLong x) = Just $ fromIntegral x
329− exifDataToWord (Exif.ExifShort x) = Just $ fromIntegral x
330− exifDataToWord _ = Nothing
331−
303+pngSize :: ByteString -> Maybe ImageSize
304+pngSize img = do
305+ let (h, rest) = B.splitAt 8 img
306+ guard $ h == "\x8a\x4d\x4e\x47\x0d\x0a\x1a\x0a" ||
307+ h == "\x89\x50\x4e\x47\x0d\x0a\x1a\x0a"
308+ let (i, rest') = B.splitAt 4 $ B.drop 4 rest
309+ guard $ i == "MHDR" || i == "IHDR"
310+ let (sizes, rest'') = B.splitAt 8 rest'
311+ (x,y) <- case map fromIntegral $unpack sizes of
312+ ([w1,w2,w3,w4,h1,h2,h3,h4] :: [Integer]) -> return
313+ (shift w1 24 + shift w2 16 + shift w3 8 + w4,
314+ shift h1 24 + shift h2 16 + shift h3 8 + h4)
315+ _ -> Nothing -- "PNG parse error"
316+ (dpix, dpiy) <- findpHYs rest''
317+ return ImageSize { pxX = x, pxY = y, dpiX = dpix, dpiY = dpiy }
318+
319+findpHYs :: ByteString -> Maybe (Integer, Integer)
320+findpHYs x
321+ | B.null x || "IDAT" `B.isPrefixOf` x = return (72,72)
322+ | "pHYs" `B.isPrefixOf` x =
323+ case map fromIntegral $ unpack $ B.take 9 $ B.drop 4 x of
324+ [x1,x2,x3,x4,y1,y2,y3,y4,u] -> do
325+ let factor = if u == 1 -- dots per meter
326+ then \z -> z * 254 `div` 10000
327+ else const 72
328+ return
329+ ( factor $ shift x1 24 + shift x2 16 + shift x3 8 + x4,
330+ factor $ shift y1 24 + shift y2 16 + shift y3 8 + y4 )
331+ _ -> mzero
332+ | otherwise = findpHYs $ B.drop 1 x -- read another byte
333+
334+gifSize :: ByteString -> Maybe ImageSize
335+gifSize img = do
336+ let (h, rest) = B.splitAt 6 img
337+ guard $ h == "GIF87a" || h == "GIF89a"
338+ case map fromIntegral $ unpack $ B.take 4 rest of
339+ [w2,w1,h2,h1] -> return ImageSize {
340+ pxX = shift w1 8 + w2,
341+ pxY = shift h1 8 + h2,
342+ dpiX = 72,
343+ dpiY = 72
344+ }
345+ _ -> Nothing -- "GIF parse error"
332346
333347svgSize :: WriterOptions -> ByteString -> Maybe ImageSize
334348svgSize opts img = do
335− doc <- either (const mzero) return $ parseXMLElement
336− $ TL.fromStrict $ UTF8.toText img
337− let viewboxSize = do
338− vb <- findAttrBy (== QName "viewBox" Nothing Nothing) doc
339− [_,_,w,h] <- mapM safeRead (T.words vb)
340− return (w,h)
349+ doc <- Xml.parseXMLDoc $ UTF8.toString img
341350 let dpi = fromIntegral $ writerDpi opts
342351 let dirToInt dir = do
343− dim <- findAttrBy (== QName dir Nothing Nothing) doc >>= lengthToDim
352+ dim <- Xml.findAttrBy (== Xml.QName dir Nothing Nothing) doc >>= lengthToDim . T.pack
344353 return $ inPixel opts dim
345− w <- dirToInt "width" <|> (fst <$> viewboxSize)
346− h <- dirToInt "height" <|> (snd <$> viewboxSize)
354+ w <- dirToInt "width"
355+ h <- dirToInt "height"
347356 return ImageSize {
348357 pxX = w
349358 , pxY = h
@@ -381,3 +390,280 @@ emfSize img =
381390 case parseheader . BL.fromStrict $ img of
382391 Left _ -> Nothing
383392 Right (_, _, size) -> Just size
393+
394+
395+jpegSize :: ByteString -> Either T.Text ImageSize
396+jpegSize img =
397+ let (hdr, rest) = B.splitAt 4 img
398+ in if B.length rest < 14
399+ then Left "unable to determine JPEG size"
400+ else case hdr of
401+ "\xff\xd8\xff\xe0" -> jfifSize rest
402+ "\xff\xd8\xff\xe1" -> exifSize rest
403+ _ -> Left "unable to determine JPEG size"
404+
405+jfifSize :: ByteString -> Either T.Text ImageSize
406+jfifSize rest =
407+ case map fromIntegral $ unpack $ B.take 5 $ B.drop 9 rest of
408+ [dpiDensity,dpix1,dpix2,dpiy1,dpiy2] ->
409+ let factor = case dpiDensity of
410+ 1 -> id
411+ 2 -> \x -> x * 254 `div` 10
412+ _ -> const 72
413+ dpix = factor (shift dpix1 8 + dpix2)
414+ dpiy = factor (shift dpiy1 8 + dpiy2)
415+ in case findJfifSize rest of
416+ Left msg -> Left msg
417+ Right (w,h) -> Right ImageSize { pxX = w
418+ , pxY = h
419+ , dpiX = dpix
420+ , dpiY = dpiy }
421+ _ -> Left "unable to determine JFIF size"
422+
423+findJfifSize :: ByteString -> Either T.Text (Integer,Integer)
424+findJfifSize bs =
425+ let bs' = B.dropWhile (=='\xff') $ B.dropWhile (/='\xff') bs
426+ in case B.uncons bs' of
427+ Just (c,bs'') | c >= '\xc0' && c <= '\xc3' ->
428+ case map fromIntegral $ unpack $ B.take 4 $ B.drop 3 bs'' of
429+ [h1,h2,w1,w2] -> Right (shift w1 8 + w2, shift h1 8 + h2)
430+ _ -> Left "JFIF parse error"
431+ Just (_,bs'') ->
432+ case map fromIntegral $ unpack $ B.take 2 bs'' of
433+ [c1,c2] ->
434+ let len = shift c1 8 + c2
435+ -- skip variables
436+ in findJfifSize $ B.drop len bs''
437+ _ -> Left "JFIF parse error"
438+ Nothing -> Left "Did not find JFIF length record"
439+
440+runGet' :: Get (Either T.Text a) -> BL.ByteString -> Either T.Text a
441+runGet' p bl =
442+#if MIN_VERSION_binary(0,7,0)
443+ case runGetOrFail p bl of
444+ Left (_,_,msg) -> Left $ T.pack msg
445+ Right (_,_,x) -> x
446+#else
447+ runGet p bl
448+#endif
449+
450+exifSize :: ByteString -> Either T.Text ImageSize
451+exifSize bs = runGet' header bl
452+ where bl = BL.fromChunks [bs]
453+ header = runExceptT $ exifHeader bl
454+-- NOTE: It would be nicer to do
455+-- runGet ((Just <$> exifHeader) <|> return Nothing)
456+-- which would prevent pandoc from raising an error when an exif header can't
457+-- be parsed. But we only get an Alternative instance for Get in binary 0.6,
458+-- and binary 0.5 ships with ghc 7.6.
459+
460+exifHeader :: BL.ByteString -> ExceptT T.Text Get ImageSize
461+exifHeader hdr = do
462+ _app1DataSize <- lift getWord16be
463+ exifHdr <- lift getWord32be
464+ unless (exifHdr == 0x45786966) $ throwError "Did not find exif header"
465+ zeros <- lift getWord16be
466+ unless (zeros == 0) $ throwError "Expected zeros after exif header"
467+ -- beginning of tiff header -- we read whole thing to use
468+ -- in getting data from offsets:
469+ let tiffHeader = BL.drop 8 hdr
470+ byteAlign <- lift getWord16be
471+ let bigEndian = byteAlign == 0x4d4d
472+ let (getWord16, getWord32, getWord64) =
473+ if bigEndian
474+ then (getWord16be, getWord32be, getWord64be)
475+ else (getWord16le, getWord32le, getWord64le)
476+ let getRational = do
477+ num <- getWord32
478+ den <- getWord32
479+ return $ fromIntegral num / fromIntegral den
480+ tagmark <- lift getWord16
481+ unless (tagmark == 0x002a) $ throwError "Failed alignment sanity check"
482+ ifdOffset <- lift getWord32
483+ lift $ skip (fromIntegral ifdOffset - 8) -- skip to IDF
484+ numentries <- lift getWord16
485+ let ifdEntry :: ExceptT T.Text Get (TagType, DataFormat)
486+ ifdEntry = do
487+ tag <- fromMaybe UnknownTagType . flip M.lookup tagTypeTable
488+ <$> lift getWord16
489+ dataFormat <- lift getWord16
490+ numComponents <- lift getWord32
491+ (fmt, bytesPerComponent) <-
492+ case dataFormat of
493+ 1 -> return (UnsignedByte <$> getWord8, 1)
494+ 2 -> return (AsciiString <$>
495+ getLazyByteString
496+ (fromIntegral numComponents), 1)
497+ 3 -> return (UnsignedShort <$> getWord16, 2)
498+ 4 -> return (UnsignedLong <$> getWord32, 4)
499+ 5 -> return (UnsignedRational <$> getRational, 8)
500+ 6 -> return (SignedByte <$> getWord8, 1)
501+ 7 -> return (Undefined <$> getLazyByteString
502+ (fromIntegral numComponents), 1)
503+ 8 -> return (SignedShort <$> getWord16, 2)
504+ 9 -> return (SignedLong <$> getWord32, 4)
505+ 10 -> return (SignedRational <$> getRational, 8)
506+ 11 -> return (SingleFloat <$> getWord32 {- TODO -}, 4)
507+ 12 -> return (DoubleFloat <$> getWord64 {- TODO -}, 8)
508+ _ -> throwError $ "Unknown data format " <> T.pack (show dataFormat)
509+ let totalBytes = fromIntegral $ numComponents * bytesPerComponent
510+ payload <- if totalBytes <= 4 -- data is right here
511+ then lift $ fmt <* skip (4 - totalBytes)
512+ else do -- get data from offset
513+ offs <- lift getWord32
514+ let bytesAtOffset =
515+ BL.take (fromIntegral totalBytes)
516+ $ BL.drop (fromIntegral offs) tiffHeader
517+ case runGet' (Right <$> fmt) bytesAtOffset of
518+ Left msg -> throwError msg
519+ Right x -> return x
520+ return (tag, payload)
521+ entries <- replicateM (fromIntegral numentries) ifdEntry
522+ subentries <- case lookup ExifOffset entries of
523+ Just (UnsignedLong offset') -> do
524+ pos <- lift bytesRead
525+ lift $ skip (fromIntegral offset' - (fromIntegral pos - 8))
526+ numsubentries <- lift getWord16
527+ replicateM (fromIntegral numsubentries) ifdEntry
528+ _ -> return []
529+ let allentries = entries ++ subentries
530+ (wdth, hght) <- case (lookup ExifImageWidth allentries,
531+ lookup ExifImageHeight allentries) of
532+ (Just (UnsignedLong w), Just (UnsignedLong h)) ->
533+ return (fromIntegral w, fromIntegral h)
534+ _ -> return defaultSize
535+ -- we return a default width and height when
536+ -- the exif header doesn't contain these
537+ let resfactor = case lookup ResolutionUnit allentries of
538+ Just (UnsignedShort 1) -> 100 / 254
539+ _ -> 1
540+ let xres = case lookup XResolution allentries of
541+ Just (UnsignedRational x) -> floor (x * resfactor)
542+ _ -> 72
543+ let yres = case lookup YResolution allentries of
544+ Just (UnsignedRational y) -> floor (y * resfactor)
545+ _ -> 72
546+ return ImageSize{
547+ pxX = wdth
548+ , pxY = hght
549+ , dpiX = xres
550+ , dpiY = yres }
551+
552+data DataFormat = UnsignedByte Word8
553+ | AsciiString BL.ByteString
554+ | UnsignedShort Word16
555+ | UnsignedLong Word32
556+ | UnsignedRational Rational
557+ | SignedByte Word8
558+ | Undefined BL.ByteString
559+ | SignedShort Word16
560+ | SignedLong Word32
561+ | SignedRational Rational
562+ | SingleFloat Word32
563+ | DoubleFloat Word64
564+ deriving (Show)
565+
566+data TagType = ImageDescription
567+ | Make
568+ | Model
569+ | Orientation
570+ | XResolution
571+ | YResolution
572+ | ResolutionUnit
573+ | Software
574+ | DateTime
575+ | WhitePoint
576+ | PrimaryChromaticities
577+ | YCbCrCoefficients
578+ | YCbCrPositioning
579+ | ReferenceBlackWhite
580+ | Copyright
581+ | ExifOffset
582+ | ExposureTime
583+ | FNumber
584+ | ExposureProgram
585+ | ISOSpeedRatings
586+ | ExifVersion
587+ | DateTimeOriginal
588+ | DateTimeDigitized
589+ | ComponentConfiguration
590+ | CompressedBitsPerPixel
591+ | ShutterSpeedValue
592+ | ApertureValue
593+ | BrightnessValue
594+ | ExposureBiasValue
595+ | MaxApertureValue
596+ | SubjectDistance
597+ | MeteringMode
598+ | LightSource
599+ | Flash
600+ | FocalLength
601+ | MakerNote
602+ | UserComment
603+ | FlashPixVersion
604+ | ColorSpace
605+ | ExifImageWidth
606+ | ExifImageHeight
607+ | RelatedSoundFile
608+ | ExifInteroperabilityOffset
609+ | FocalPlaneXResolution
610+ | FocalPlaneYResolution
611+ | FocalPlaneResolutionUnit
612+ | SensingMethod
613+ | FileSource
614+ | SceneType
615+ | UnknownTagType
616+ deriving (Show, Eq, Ord)
617+
618+tagTypeTable :: M.Map Word16 TagType
619+tagTypeTable = M.fromList
620+ [ (0x010e, ImageDescription)
621+ , (0x010f, Make)
622+ , (0x0110, Model)
623+ , (0x0112, Orientation)
624+ , (0x011a, XResolution)
625+ , (0x011b, YResolution)
626+ , (0x0128, ResolutionUnit)
627+ , (0x0131, Software)
628+ , (0x0132, DateTime)
629+ , (0x013e, WhitePoint)
630+ , (0x013f, PrimaryChromaticities)
631+ , (0x0211, YCbCrCoefficients)
632+ , (0x0213, YCbCrPositioning)
633+ , (0x0214, ReferenceBlackWhite)
634+ , (0x8298, Copyright)
635+ , (0x8769, ExifOffset)
636+ , (0x829a, ExposureTime)
637+ , (0x829d, FNumber)
638+ , (0x8822, ExposureProgram)
639+ , (0x8827, ISOSpeedRatings)
640+ , (0x9000, ExifVersion)
641+ , (0x9003, DateTimeOriginal)
642+ , (0x9004, DateTimeDigitized)
643+ , (0x9101, ComponentConfiguration)
644+ , (0x9102, CompressedBitsPerPixel)
645+ , (0x9201, ShutterSpeedValue)
646+ , (0x9202, ApertureValue)
647+ , (0x9203, BrightnessValue)
648+ , (0x9204, ExposureBiasValue)
649+ , (0x9205, MaxApertureValue)
650+ , (0x9206, SubjectDistance)
651+ , (0x9207, MeteringMode)
652+ , (0x9208, LightSource)
653+ , (0x9209, Flash)
654+ , (0x920a, FocalLength)
655+ , (0x927c, MakerNote)
656+ , (0x9286, UserComment)
657+ , (0xa000, FlashPixVersion)
658+ , (0xa001, ColorSpace)
659+ , (0xa002, ExifImageWidth)
660+ , (0xa003, ExifImageHeight)
661+ , (0xa004, RelatedSoundFile)
662+ , (0xa005, ExifInteroperabilityOffset)
663+ , (0xa20e, FocalPlaneXResolution)
664+ , (0xa20f, FocalPlaneYResolution)
665+ , (0xa210, FocalPlaneResolutionUnit)
666+ , (0xa217, SensingMethod)
667+ , (0xa300, FileSource)
668+ , (0xa301, SceneType)
669+ ]
src/Text/Pandoc/Logging.hs+10−33
@@ -4,7 +4,7 @@
44{-# LANGUAGE OverloadedStrings #-}
55{- |
66 Module : Text.Pandoc.Logging
7− Copyright : Copyright (C) 2006-2021 John MacFarlane
7+ Copyright : Copyright (C) 2006-2020 John MacFarlane
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -36,7 +36,6 @@ import Data.Typeable (Typeable)
3636import GHC.Generics (Generic)
3737import Text.Pandoc.Definition
3838import Text.Parsec.Pos
39−import Text.Pandoc.Shared (tshow)
4039
4140-- | Verbosity level.
4241data Verbosity = ERROR | WARNING | INFO
@@ -85,7 +84,6 @@ data LogMessage =
8584 | CouldNotParseCSS Text
8685 | Fetching Text
8786 | Extracting Text
88− | LoadedResource FilePath FilePath
8987 | NoTitleElement Text
9088 | NoLangSpecified
9189 | InvalidLang Text
@@ -102,8 +100,6 @@ data LogMessage =
102100 | FilterCompleted FilePath Integer
103101 | CiteprocWarning Text
104102 | ATXHeadingInLHS Int Text
105− | EnvironmentVariableUndefined Text
106− | DuplicateAttribute Text Text
107103 deriving (Show, Eq, Data, Ord, Typeable, Generic)
108104
109105instance ToJSON LogMessage where
@@ -196,9 +192,6 @@ instance ToJSON LogMessage where
196192 ["path" .= fp]
197193 Extracting fp ->
198194 ["path" .= fp]
199− LoadedResource orig found ->
200− ["for" .= orig
201− ,"from" .= found]
202195 NoTitleElement fallback ->
203196 ["fallback" .= fallback]
204197 NoLangSpecified -> []
@@ -236,20 +229,13 @@ instance ToJSON LogMessage where
236229 ATXHeadingInLHS lvl contents ->
237230 ["level" .= lvl
238231 ,"contents" .= contents]
239− EnvironmentVariableUndefined var ->
240− ["variable" .= var ]
241− DuplicateAttribute attr val ->
242− ["attribute" .= attr
243− ,"value" .= val]
244232
245233showPos :: SourcePos -> Text
246234showPos pos = Text.pack $ sn ++ "line " ++
247235 show (sourceLine pos) ++ " column " ++ show (sourceColumn pos)
248− where
249− sn' = sourceName pos
250− sn = if sn' == "source" || sn' == "" || sn' == "-"
251− then ""
252− else sn' ++ " "
236+ where sn = if sourceName pos == "source" || sourceName pos == ""
237+ then ""
238+ else sourceName pos ++ " "
253239
254240encodeLogMessages :: [LogMessage] -> BL.ByteString
255241encodeLogMessages ms =
@@ -282,7 +268,7 @@ showLogMessage msg =
282268 ParsingUnescaped s pos ->
283269 "Parsing unescaped '" <> s <> "' at " <> showPos pos
284270 CouldNotLoadIncludeFile fp pos ->
285− "Could not load include file " <> fp <> " at " <> showPos pos
271+ "Could not load include file '" <> fp <> "' at " <> showPos pos
286272 MacroAlreadyDefined name pos ->
287273 "Macro '" <> name <> "' already defined, ignoring at " <> showPos pos
288274 InlineNotRendered il ->
@@ -294,18 +280,18 @@ showLogMessage msg =
294280 IgnoredIOError s ->
295281 "IO Error (ignored): " <> s
296282 CouldNotFetchResource fp s ->
297− "Could not fetch resource " <> fp <>
283+ "Could not fetch resource '" <> fp <> "'" <>
298284 if Text.null s then "" else ": " <> s
299285 CouldNotDetermineImageSize fp s ->
300− "Could not determine image size for " <> fp <>
286+ "Could not determine image size for '" <> fp <> "'" <>
301287 if Text.null s then "" else ": " <> s
302288 CouldNotConvertImage fp s ->
303− "Could not convert image " <> fp <>
289+ "Could not convert image '" <> fp <> "'" <>
304290 if Text.null s then "" else ": " <> s
305291 CouldNotDetermineMimeType fp ->
306− "Could not determine mime type for " <> fp
292+ "Could not determine mime type for '" <> fp <> "'"
307293 CouldNotConvertTeXMath s m ->
308− "Could not convert TeX math " <> s <> ", rendering as TeX" <>
294+ "Could not convert TeX math '" <> s <> "', rendering as TeX" <>
309295 if Text.null m then "" else ":\n" <> m
310296 CouldNotParseCSS m ->
311297 "Could not parse CSS" <> if Text.null m then "" else ":\n" <> m
@@ -313,8 +299,6 @@ showLogMessage msg =
313299 "Fetching " <> fp <> "..."
314300 Extracting fp ->
315301 "Extracting " <> fp <> "..."
316− LoadedResource orig found ->
317− "Loaded " <> Text.pack orig <> " from " <> Text.pack found
318302 NoTitleElement fallback ->
319303 "This document format requires a nonempty <title> element.\n" <>
320304 "Defaulting to '" <> fallback <> "' as the title.\n" <>
@@ -361,10 +345,6 @@ showLogMessage msg =
361345 if lvl < 3
362346 then " Consider using --markdown-headings=setext."
363347 else ""
364− EnvironmentVariableUndefined var ->
365− "Undefined environment variable " <> var <> " in defaults file."
366− DuplicateAttribute attr val ->
367− "Ignoring duplicate attribute " <> attr <> "=" <> tshow val <> "."
368348
369349messageVerbosity :: LogMessage -> Verbosity
370350messageVerbosity msg =
@@ -395,7 +375,6 @@ messageVerbosity msg =
395375 CouldNotParseCSS{} -> WARNING
396376 Fetching{} -> INFO
397377 Extracting{} -> INFO
398− LoadedResource{} -> INFO
399378 NoTitleElement{} -> WARNING
400379 NoLangSpecified -> INFO
401380 InvalidLang{} -> WARNING
@@ -412,5 +391,3 @@ messageVerbosity msg =
412391 FilterCompleted{} -> INFO
413392 CiteprocWarning{} -> WARNING
414393 ATXHeadingInLHS{} -> WARNING
415− EnvironmentVariableUndefined{}-> WARNING
416− DuplicateAttribute{} -> WARNING
src/Text/Pandoc/Lua.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua
3− Copyright : Copyright © 2017-2021 Albert Krewinkel
3+ Copyright : Copyright © 2017–2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/ErrorConversion.hs+1−1
@@ -3,7 +3,7 @@
33{-# LANGUAGE ScopedTypeVariables #-}
44{- |
55 Module : Text.Pandoc.Lua.ErrorConversion
6− Copyright : © 2020-2021 Albert Krewinkel
6+ Copyright : © 2020 Albert Krewinkel
77 License : GNU GPL, version 2 or above
88
99 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Filter.hs+3−6
@@ -1,8 +1,8 @@
11{-# LANGUAGE FlexibleContexts #-}
22{- |
33Module : Text.Pandoc.Lua.Filter
4−Copyright : © 2012-2021 John MacFarlane,
5− © 2017-2021 Albert Krewinkel
4+Copyright : © 2012–2020 John MacFarlane,
5+ © 2017-2020 Albert Krewinkel
66License : GNU GPL, version 2 or above
77Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
88Stability : alpha
@@ -13,9 +13,7 @@ module Text.Pandoc.Lua.Filter ( LuaFilterFunction
1313 , LuaFilter
1414 , runFilterFile
1515 , walkInlines
16− , walkInlineLists
1716 , walkBlocks
18− , walkBlockLists
1917 , module Text.Pandoc.Lua.Walk
2018 ) where
2119import Control.Applicative ((<|>))
@@ -24,7 +22,6 @@ import Control.Monad.Catch (finally, try)
2422import Data.Data (Data, DataType, dataTypeConstrs, dataTypeName, dataTypeOf,
2523 showConstr, toConstr, tyconUQname)
2624import Data.Foldable (foldrM)
27−import Data.List (foldl')
2825import Data.Map (Map)
2926import Data.Maybe (fromMaybe)
3027import Foreign.Lua (Lua, Peekable, Pushable, StackIndex)
@@ -207,7 +204,7 @@ walkMeta lf (Pandoc m bs) = do
207204
208205walkPandoc :: LuaFilter -> Pandoc -> Lua Pandoc
209206walkPandoc (LuaFilter fnMap) =
210− case foldl' mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
207+ case foldl mplus Nothing (map (`Map.lookup` fnMap) pandocFilterNames) of
211208 Just fn -> \x -> runFilterFunction fn x *> singleElement x
212209 Nothing -> return
213210
src/Text/Pandoc/Lua/Global.hs+1−1
@@ -1,7 +1,7 @@
11{-# LANGUAGE DeriveDataTypeable #-}
22{- |
33 Module : Text.Pandoc.Lua
4− Copyright : Copyright © 2017-2021 Albert Krewinkel
4+ Copyright : Copyright © 2017-2020 Albert Krewinkel
55 License : GNU GPL, version 2 or above
66
77 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Init.hs+5−15
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua
3− Copyright : Copyright © 2017-2021 Albert Krewinkel
3+ Copyright : Copyright © 2017-2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -12,18 +12,17 @@ module Text.Pandoc.Lua.Init
1212 ( runLua
1313 ) where
1414
15−import Control.Monad (when)
1615import Control.Monad.Catch (try)
1716import Control.Monad.Trans (MonadIO (..))
1817import Data.Data (Data, dataTypeConstrs, dataTypeOf, showConstr)
1918import Foreign.Lua (Lua)
2019import GHC.IO.Encoding (getForeignEncoding, setForeignEncoding, utf8)
21−import Text.Pandoc.Class.PandocMonad (readDataFile)
2220import Text.Pandoc.Class.PandocIO (PandocIO)
2321import Text.Pandoc.Error (PandocError)
2422import Text.Pandoc.Lua.Packages (installPandocPackageSearcher)
25−import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua, runPandocLua)
26−import Text.Pandoc.Lua.Util (throwTopMessageAsError')
23+import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua,
24+ loadScriptFromDataDir, runPandocLua)
25+
2726import qualified Foreign.Lua as Lua
2827import qualified Text.Pandoc.Definition as Pandoc
2928import qualified Text.Pandoc.Lua.Module.Pandoc as ModulePandoc
@@ -45,7 +44,7 @@ initLuaState = do
4544 liftPandocLua Lua.openlibs
4645 installPandocPackageSearcher
4746 initPandocModule
48− loadInitScript "init.lua"
47+ loadScriptFromDataDir "init.lua"
4948 where
5049 initPandocModule :: PandocLua ()
5150 initPandocModule = do
@@ -62,15 +61,6 @@ initLuaState = do
6261 -- assign module to global variable
6362 liftPandocLua $ Lua.setglobal "pandoc"
6463
65− loadInitScript :: FilePath -> PandocLua ()
66− loadInitScript scriptFile = do
67− script <- readDataFile scriptFile
68− status <- liftPandocLua $ Lua.dostring script
69− when (status /= Lua.OK) . liftPandocLua $
70− throwTopMessageAsError'
71− (("Couldn't load '" ++ scriptFile ++ "'.\n") ++)
72−
73−
7464-- | AST elements are marshaled via normal constructor functions in the
7565-- @pandoc@ module. However, accessing Lua globals from Haskell is
7666-- expensive (due to error handling). Accessing the Lua registry is much
src/Text/Pandoc/Lua/Marshaling.hs+2−2
@@ -1,7 +1,7 @@
11{- |
22 Module : Text.Pandoc.Lua.Marshaling
3− Copyright : © 2012-2021 John MacFarlane
4− © 2017-2021 Albert Krewinkel
3+ Copyright : © 2012-2020 John MacFarlane
4+ © 2017-2020 Albert Krewinkel
55 License : GNU GPL, version 2 or above
66
77 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Marshaling/AST.hs+81−79
@@ -1,10 +1,9 @@
11{-# OPTIONS_GHC -fno-warn-orphans #-}
2−{-# LANGUAGE BangPatterns #-}
32{-# LANGUAGE LambdaCase #-}
43{- |
54 Module : Text.Pandoc.Lua.Marshaling.AST
6− Copyright : © 2012-2021 John MacFarlane
7− © 2017-2021 Albert Krewinkel
5+ Copyright : © 2012-2020 John MacFarlane
6+ © 2017-2020 Albert Krewinkel
87 License : GNU GPL, version 2 or above
98
109 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -18,7 +17,6 @@ module Text.Pandoc.Lua.Marshaling.AST
1817 ) where
1918
2019import Control.Applicative ((<|>))
21−import Control.Monad ((<$!>))
2220import Foreign.Lua (Lua, Peekable, Pushable, StackIndex)
2321import Text.Pandoc.Definition
2422import Text.Pandoc.Error (PandocError)
@@ -34,16 +32,17 @@ instance Pushable Pandoc where
3432 pushViaConstructor "Pandoc" blocks meta
3533
3634instance Peekable Pandoc where
37− peek idx = defineHowTo "get Pandoc value" $! Pandoc
38− <$!> LuaUtil.rawField idx "meta"
39− <*> LuaUtil.rawField idx "blocks"
35+ peek idx = defineHowTo "get Pandoc value" $ do
36+ blocks <- LuaUtil.rawField idx "blocks"
37+ meta <- LuaUtil.rawField idx "meta"
38+ return $ Pandoc meta blocks
4039
4140instance Pushable Meta where
4241 push (Meta mmap) =
4342 pushViaConstructor "Meta" mmap
4443instance Peekable Meta where
45− peek idx = defineHowTo "get Meta value" $!
46− Meta <$!> Lua.peek idx
44+ peek idx = defineHowTo "get Meta value" $
45+ Meta <$> Lua.peek idx
4746
4847instance Pushable MetaValue where
4948 push = pushMetaValue
@@ -69,13 +68,14 @@ instance Pushable Citation where
6968 pushViaConstructor "Citation" cid mode prefix suffix noteNum hash
7069
7170instance Peekable Citation where
72− peek idx = Citation
73− <$!> LuaUtil.rawField idx "id"
74− <*> LuaUtil.rawField idx "prefix"
75− <*> LuaUtil.rawField idx "suffix"
76− <*> LuaUtil.rawField idx "mode"
77− <*> LuaUtil.rawField idx "note_num"
78− <*> LuaUtil.rawField idx "hash"
71+ peek idx = do
72+ id' <- LuaUtil.rawField idx "id"
73+ prefix <- LuaUtil.rawField idx "prefix"
74+ suffix <- LuaUtil.rawField idx "suffix"
75+ mode <- LuaUtil.rawField idx "mode"
76+ num <- LuaUtil.rawField idx "note_num"
77+ hash <- LuaUtil.rawField idx "hash"
78+ return $ Citation id' prefix suffix mode num hash
7979
8080instance Pushable Alignment where
8181 push = Lua.push . show
@@ -90,7 +90,7 @@ instance Peekable CitationMode where
9090instance Pushable Format where
9191 push (Format f) = Lua.push f
9292instance Peekable Format where
93− peek idx = Format <$!> Lua.peek idx
93+ peek idx = Format <$> Lua.peek idx
9494
9595instance Pushable ListNumberDelim where
9696 push = Lua.push . show
@@ -130,26 +130,26 @@ peekMetaValue idx = defineHowTo "get MetaValue" $ do
130130 elementContent = Lua.peek idx
131131 luatype <- Lua.ltype idx
132132 case luatype of
133− Lua.TypeBoolean -> MetaBool <$!> Lua.peek idx
134− Lua.TypeString -> MetaString <$!> Lua.peek idx
133+ Lua.TypeBoolean -> MetaBool <$> Lua.peek idx
134+ Lua.TypeString -> MetaString <$> Lua.peek idx
135135 Lua.TypeTable -> do
136136 tag <- try $ LuaUtil.getTag idx
137137 case tag of
138− Right "MetaBlocks" -> MetaBlocks <$!> elementContent
139− Right "MetaBool" -> MetaBool <$!> elementContent
140− Right "MetaMap" -> MetaMap <$!> elementContent
141− Right "MetaInlines" -> MetaInlines <$!> elementContent
142− Right "MetaList" -> MetaList <$!> elementContent
143− Right "MetaString" -> MetaString <$!> elementContent
138+ Right "MetaBlocks" -> MetaBlocks <$> elementContent
139+ Right "MetaBool" -> MetaBool <$> elementContent
140+ Right "MetaMap" -> MetaMap <$> elementContent
141+ Right "MetaInlines" -> MetaInlines <$> elementContent
142+ Right "MetaList" -> MetaList <$> elementContent
143+ Right "MetaString" -> MetaString <$> elementContent
144144 Right t -> Lua.throwMessage ("Unknown meta tag: " <> t)
145145 Left _ -> do
146146 -- no meta value tag given, try to guess.
147147 len <- Lua.rawlen idx
148148 if len <= 0
149− then MetaMap <$!> Lua.peek idx
150− else (MetaInlines <$!> Lua.peek idx)
151− <|> (MetaBlocks <$!> Lua.peek idx)
152− <|> (MetaList <$!> Lua.peek idx)
149+ then MetaMap <$> Lua.peek idx
150+ else (MetaInlines <$> Lua.peek idx)
151+ <|> (MetaBlocks <$> Lua.peek idx)
152+ <|> (MetaList <$> Lua.peek idx)
153153 _ -> Lua.throwMessage "could not get meta value"
154154
155155-- | Push a block element to the top of the Lua stack.
@@ -174,25 +174,25 @@ pushBlock = \case
174174
175175-- | Return the value at the given index as block if possible.
176176peekBlock :: StackIndex -> Lua Block
177−peekBlock idx = defineHowTo "get Block value" $! do
177+peekBlock idx = defineHowTo "get Block value" $ do
178178 tag <- LuaUtil.getTag idx
179179 case tag of
180− "BlockQuote" -> BlockQuote <$!> elementContent
181− "BulletList" -> BulletList <$!> elementContent
182− "CodeBlock" -> withAttr CodeBlock <$!> elementContent
183− "DefinitionList" -> DefinitionList <$!> elementContent
184− "Div" -> withAttr Div <$!> elementContent
180+ "BlockQuote" -> BlockQuote <$> elementContent
181+ "BulletList" -> BulletList <$> elementContent
182+ "CodeBlock" -> withAttr CodeBlock <$> elementContent
183+ "DefinitionList" -> DefinitionList <$> elementContent
184+ "Div" -> withAttr Div <$> elementContent
185185 "Header" -> (\(lvl, LuaAttr attr, lst) -> Header lvl attr lst)
186− <$!> elementContent
186+ <$> elementContent
187187 "HorizontalRule" -> return HorizontalRule
188− "LineBlock" -> LineBlock <$!> elementContent
188+ "LineBlock" -> LineBlock <$> elementContent
189189 "OrderedList" -> (\(LuaListAttributes lstAttr, lst) ->
190190 OrderedList lstAttr lst)
191− <$!> elementContent
191+ <$> elementContent
192192 "Null" -> return Null
193− "Para" -> Para <$!> elementContent
194− "Plain" -> Plain <$!> elementContent
195− "RawBlock" -> uncurry RawBlock <$!> elementContent
193+ "Para" -> Para <$> elementContent
194+ "Plain" -> Plain <$> elementContent
195+ "RawBlock" -> uncurry RawBlock <$> elementContent
196196 "Table" -> (\(attr, capt, colSpecs, thead, tbodies, tfoot) ->
197197 Table (fromLuaAttr attr)
198198 capt
@@ -200,7 +200,7 @@ peekBlock idx = defineHowTo "get Block value" $! do
200200 thead
201201 tbodies
202202 tfoot)
203− <$!> elementContent
203+ <$> elementContent
204204 _ -> Lua.throwMessage ("Unknown block type: " <> tag)
205205 where
206206 -- Get the contents of an AST element.
@@ -222,14 +222,15 @@ pushCaption (Caption shortCaption longCaption) = do
222222
223223-- | Peek Caption element
224224peekCaption :: StackIndex -> Lua Caption
225−peekCaption idx = Caption
226− <$!> (Lua.fromOptional <$!> LuaUtil.rawField idx "short")
227− <*> LuaUtil.rawField idx "long"
225+peekCaption idx = do
226+ short <- Lua.fromOptional <$> LuaUtil.rawField idx "short"
227+ long <- LuaUtil.rawField idx "long"
228+ return $ Caption short long
228229
229230instance Peekable ColWidth where
230231 peek idx = do
231− width <- Lua.fromOptional <$!> Lua.peek idx
232− return $! maybe ColWidthDefault ColWidth width
232+ width <- Lua.fromOptional <$> Lua.peek idx
233+ return $ maybe ColWidthDefault ColWidth width
233234
234235instance Pushable ColWidth where
235236 push = \case
@@ -251,11 +252,12 @@ instance Pushable TableBody where
251252 LuaUtil.addField "body" body
252253
253254instance Peekable TableBody where
254− peek idx = TableBody
255− <$!> LuaUtil.rawField idx "attr"
256− <*> (RowHeadColumns <$!> LuaUtil.rawField idx "row_head_columns")
257− <*> LuaUtil.rawField idx "head"
258− <*> LuaUtil.rawField idx "body"
255+ peek idx = do
256+ attr <- LuaUtil.rawField idx "attr"
257+ rowHeadColumns <- LuaUtil.rawField idx "row_head_columns"
258+ head' <- LuaUtil.rawField idx "head"
259+ body <- LuaUtil.rawField idx "body"
260+ return $ TableBody attr (RowHeadColumns rowHeadColumns) head' body
259261
260262instance Pushable TableHead where
261263 push (TableHead attr rows) = Lua.push (attr, rows)
@@ -285,12 +287,13 @@ pushCell (Cell attr align (RowSpan rowSpan) (ColSpan colSpan) contents) = do
285287 LuaUtil.addField "contents" contents
286288
287289peekCell :: StackIndex -> Lua Cell
288−peekCell idx = Cell
289− <$!> (fromLuaAttr <$!> LuaUtil.rawField idx "attr")
290− <*> LuaUtil.rawField idx "alignment"
291− <*> (RowSpan <$!> LuaUtil.rawField idx "row_span")
292− <*> (ColSpan <$!> LuaUtil.rawField idx "col_span")
293− <*> LuaUtil.rawField idx "contents"
290+peekCell idx = do
291+ attr <- fromLuaAttr <$> LuaUtil.rawField idx "attr"
292+ align <- LuaUtil.rawField idx "alignment"
293+ rowSpan <- LuaUtil.rawField idx "row_span"
294+ colSpan <- LuaUtil.rawField idx "col_span"
295+ contents <- LuaUtil.rawField idx "contents"
296+ return $ Cell attr align (RowSpan rowSpan) (ColSpan colSpan) contents
294297
295298-- | Push an inline element to the top of the lua stack.
296299pushInline :: Inline -> Lua ()
@@ -321,29 +324,28 @@ peekInline :: StackIndex -> Lua Inline
321324peekInline idx = defineHowTo "get Inline value" $ do
322325 tag <- LuaUtil.getTag idx
323326 case tag of
324− "Cite" -> uncurry Cite <$!> elementContent
325− "Code" -> withAttr Code <$!> elementContent
326− "Emph" -> Emph <$!> elementContent
327− "Underline" -> Underline <$!> elementContent
328− "Image" -> (\(LuaAttr !attr, !lst, !tgt) -> Image attr lst tgt)
329− <$!> elementContent
330− "Link" -> (\(LuaAttr !attr, !lst, !tgt) -> Link attr lst tgt)
331− <$!> elementContent
327+ "Cite" -> uncurry Cite <$> elementContent
328+ "Code" -> withAttr Code <$> elementContent
329+ "Emph" -> Emph <$> elementContent
330+ "Underline" -> Underline <$> elementContent
331+ "Image" -> (\(LuaAttr attr, lst, tgt) -> Image attr lst tgt)
332+ <$> elementContent
333+ "Link" -> (\(LuaAttr attr, lst, tgt) -> Link attr lst tgt)
334+ <$> elementContent
332335 "LineBreak" -> return LineBreak
333− "Note" -> Note <$!> elementContent
334− "Math" -> uncurry Math <$!> elementContent
335− "Quoted" -> uncurry Quoted <$!> elementContent
336− "RawInline" -> uncurry RawInline <$!> elementContent
337− "SmallCaps" -> SmallCaps <$!> elementContent
336+ "Note" -> Note <$> elementContent
337+ "Math" -> uncurry Math <$> elementContent
338+ "Quoted" -> uncurry Quoted <$> elementContent
339+ "RawInline" -> uncurry RawInline <$> elementContent
340+ "SmallCaps" -> SmallCaps <$> elementContent
338341 "SoftBreak" -> return SoftBreak
339342 "Space" -> return Space
340− "Span" -> withAttr Span <$!> elementContent
341− -- strict to Lua string is copied before gc
342− "Str" -> Str <$!> elementContent
343− "Strikeout" -> Strikeout <$!> elementContent
344− "Strong" -> Strong <$!> elementContent
345− "Subscript" -> Subscript <$!> elementContent
346− "Superscript"-> Superscript <$!> elementContent
343+ "Span" -> withAttr Span <$> elementContent
344+ "Str" -> Str <$> elementContent
345+ "Strikeout" -> Strikeout <$> elementContent
346+ "Strong" -> Strong <$> elementContent
347+ "Subscript" -> Subscript <$> elementContent
348+ "Superscript"-> Superscript <$> elementContent
347349 _ -> Lua.throwMessage ("Unknown inline type: " <> tag)
348350 where
349351 -- Get the contents of an AST element.
@@ -364,7 +366,7 @@ instance Pushable LuaAttr where
364366 pushViaConstructor "Attr" id' classes kv
365367
366368instance Peekable LuaAttr where
367− peek idx = defineHowTo "get Attr value" $! (LuaAttr <$!> Lua.peek idx)
369+ peek idx = defineHowTo "get Attr value" (LuaAttr <$> Lua.peek idx)
368370
369371-- | Wrapper for ListAttributes
370372newtype LuaListAttributes = LuaListAttributes ListAttributes
src/Text/Pandoc/Lua/Marshaling/AnyValue.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua.Marshaling.AnyValue
3− Copyright : © 2017-2021 Albert Krewinkel
3+ Copyright : © 2017-2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Marshaling/CommonState.hs+2−2
@@ -3,8 +3,8 @@
33{-# LANGUAGE OverloadedStrings #-}
44{- |
55 Module : Text.Pandoc.Lua.Marshaling.CommonState
6− Copyright : © 2012-2021 John MacFarlane
7− © 2017-2021 Albert Krewinkel
6+ Copyright : © 2012-2020 John MacFarlane
7+ © 2017-2020 Albert Krewinkel
88 License : GNU GPL, version 2 or above
99 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
1010 Stability : alpha
src/Text/Pandoc/Lua/Marshaling/Context.hs+2−3
@@ -1,8 +1,8 @@
11{-# OPTIONS_GHC -fno-warn-orphans #-}
22{- |
33 Module : Text.Pandoc.Lua.Marshaling.Context
4− Copyright : © 2012-2021 John MacFarlane
5− © 2017-2021 Albert Krewinkel
4+ Copyright : © 2012-2020 John MacFarlane
5+ © 2017-2020 Albert Krewinkel
66 License : GNU GPL, version 2 or above
77
88 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -22,7 +22,6 @@ instance (TemplateTarget a, Pushable a) => Pushable (Context a) where
2222
2323instance (TemplateTarget a, Pushable a) => Pushable (Val a) where
2424 push NullVal = Lua.push ()
25− push (BoolVal b) = Lua.push b
2625 push (MapVal ctx) = Lua.push ctx
2726 push (ListVal xs) = Lua.push xs
2827 push (SimpleVal d) = Lua.push $ render Nothing d
src/Text/Pandoc/Lua/Marshaling/List.hs+2−2
@@ -4,8 +4,8 @@
44{-# LANGUAGE UndecidableInstances #-}
55{- |
66Module : Text.Pandoc.Lua.Marshaling.List
7−Copyright : © 2012-2021 John MacFarlane
8− © 2017-2021 Albert Krewinkel
7+Copyright : © 2012-2020 John MacFarlane
8+ © 2017-2020 Albert Krewinkel
99License : GNU GPL, version 2 or above
1010Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
1111Stability : alpha
src/Text/Pandoc/Lua/Marshaling/MediaBag.hs+2−2
@@ -1,7 +1,7 @@
11{- |
22 Module : Text.Pandoc.Lua.Marshaling.MediaBag
3− Copyright : © 2012-2021 John MacFarlane
4− © 2017-2021 Albert Krewinkel
3+ Copyright : © 2012-2020 John MacFarlane
4+ © 2017-2020 Albert Krewinkel
55 License : GNU GPL, version 2 or above
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
77 Stability : alpha
src/Text/Pandoc/Lua/Marshaling/PandocError.hs+1−1
@@ -4,7 +4,7 @@
44{-# LANGUAGE ScopedTypeVariables #-}
55{- |
66 Module : Text.Pandoc.Lua.Marshaling.PandocError
7− Copyright : © 2020-2021 Albert Krewinkel
7+ Copyright : © 2020 Albert Krewinkel
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs+2−2
@@ -4,8 +4,8 @@
44{-# OPTIONS_GHC -fno-warn-orphans #-}
55{- |
66 Module : Text.Pandoc.Lua.Marshaling.ReaderOptions
7− Copyright : © 2012-2021 John MacFarlane
8− © 2017-2021 Albert Krewinkel
7+ Copyright : © 2012-2020 John MacFarlane
8+ © 2017-2020 Albert Krewinkel
99 License : GNU GPL, version 2 or above
1010
1111 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Marshaling/SimpleTable.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua.Marshaling.SimpleTable
3− Copyright : © 2020-2021 Albert Krewinkel
3+ Copyright : © 2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Marshaling/Version.hs+1−1
@@ -4,7 +4,7 @@
44{-# LANGUAGE ScopedTypeVariables #-}
55{- |
66 Module : Text.Pandoc.Lua.Marshaling.Version
7− Copyright : © 2019-2021 Albert Krewinkel
7+ Copyright : © 2019-2020 Albert Krewinkel
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Module/MediaBag.hs+17−18
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Lua.Module.MediaBag
4− Copyright : Copyright © 2017-2021 Albert Krewinkel
4+ Copyright : Copyright © 2017-2020 Albert Krewinkel
55 License : GNU GPL, version 2 or above
66
77 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -13,7 +13,6 @@ module Text.Pandoc.Lua.Module.MediaBag
1313 ( pushModule
1414 ) where
1515
16−import Prelude hiding (lookup)
1716import Control.Monad (zipWithM_)
1817import Foreign.Lua (Lua, NumResults, Optional)
1918import Text.Pandoc.Class.CommonState (CommonState (..))
@@ -37,10 +36,10 @@ pushModule = do
3736 liftPandocLua Lua.newtable
3837 addFunction "delete" delete
3938 addFunction "empty" empty
40− addFunction "insert" insert
39+ addFunction "insert" insertMediaFn
4140 addFunction "items" items
42− addFunction "lookup" lookup
43− addFunction "list" list
41+ addFunction "lookup" lookupMediaFn
42+ addFunction "list" mediaDirectoryFn
4443 addFunction "fetch" fetch
4544 return 1
4645
@@ -54,11 +53,11 @@ empty :: PandocLua NumResults
5453empty = 0 <$ modifyCommonState (\st -> st { stMediaBag = mempty })
5554
5655-- | Insert a new item into the media bag.
57−insert :: FilePath
58− -> Optional MimeType
59− -> BL.ByteString
60− -> PandocLua NumResults
61−insert fp optionalMime contents = do
56+insertMediaFn :: FilePath
57+ -> Optional MimeType
58+ -> BL.ByteString
59+ -> PandocLua NumResults
60+insertMediaFn fp optionalMime contents = do
6261 mb <- getMediaBag
6362 setMediaBag $ MB.insertMedia fp (Lua.fromOptional optionalMime) contents mb
6463 return (Lua.NumResults 0)
@@ -67,19 +66,19 @@ insert fp optionalMime contents = do
6766items :: PandocLua NumResults
6867items = getMediaBag >>= liftPandocLua . pushIterator
6968
70−lookup :: FilePath
71− -> PandocLua NumResults
72−lookup fp = do
69+lookupMediaFn :: FilePath
70+ -> PandocLua NumResults
71+lookupMediaFn fp = do
7372 res <- MB.lookupMedia fp <$> getMediaBag
7473 liftPandocLua $ case res of
7574 Nothing -> 1 <$ Lua.pushnil
76− Just item -> do
77− Lua.push $ MB.mediaMimeType item
78− Lua.push $ MB.mediaContents item
75+ Just (mimeType, contents) -> do
76+ Lua.push mimeType
77+ Lua.push contents
7978 return 2
8079
81−list :: PandocLua NumResults
82−list = do
80+mediaDirectoryFn :: PandocLua NumResults
81+mediaDirectoryFn = do
8382 dirContents <- MB.mediaDirectory <$> getMediaBag
8483 liftPandocLua $ do
8584 Lua.newtable
src/Text/Pandoc/Lua/Module/Pandoc.hs+23−29
@@ -2,7 +2,7 @@
22{-# LANGUAGE OverloadedStrings #-}
33{- |
44 Module : Text.Pandoc.Lua.Module.Pandoc
5− Copyright : Copyright © 2017-2021 Albert Krewinkel
5+ Copyright : Copyright © 2017-2020 Albert Krewinkel
66 License : GNU GPL, version 2 or above
77
88 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -14,7 +14,6 @@ module Text.Pandoc.Lua.Module.Pandoc
1414 ( pushModule
1515 ) where
1616
17−import Prelude hiding (read)
1817import Control.Monad (when)
1918import Control.Monad.Except (throwError)
2019import Data.Default (Default (..))
@@ -23,12 +22,10 @@ import Foreign.Lua (Lua, NumResults, Optional, Peekable, Pushable)
2322import System.Exit (ExitCode (..))
2423import Text.Pandoc.Class.PandocIO (runIO)
2524import Text.Pandoc.Definition (Block, Inline)
26−import Text.Pandoc.Lua.Filter (LuaFilter, SingletonsList (..), walkInlines,
27− walkInlineLists, walkBlocks, walkBlockLists)
25+import Text.Pandoc.Lua.Filter (walkInlines, walkBlocks, LuaFilter, SingletonsList (..))
2826import Text.Pandoc.Lua.Marshaling ()
29−import Text.Pandoc.Lua.Marshaling.List (List (..))
3027import Text.Pandoc.Lua.PandocLua (PandocLua, addFunction, liftPandocLua,
31− loadDefaultModule)
28+ loadScriptFromDataDir)
3229import Text.Pandoc.Walk (Walkable)
3330import Text.Pandoc.Options (ReaderOptions (readerExtensions))
3431import Text.Pandoc.Process (pipeProcess)
@@ -41,33 +38,30 @@ import qualified Foreign.Lua as Lua
4138import qualified Text.Pandoc.Lua.Util as LuaUtil
4239import Text.Pandoc.Error
4340
44−-- | Push the "pandoc" package to the Lua stack. Requires the `List`
45−-- module to be loadable.
41+-- | Push the "pandoc" on the lua stack. Requires the `list` module to be
42+-- loaded.
4643pushModule :: PandocLua NumResults
4744pushModule = do
48− loadDefaultModule "pandoc"
49− addFunction "read" read
50− addFunction "pipe" pipe
51− addFunction "walk_block" walk_block
52− addFunction "walk_inline" walk_inline
45+ loadScriptFromDataDir "pandoc.lua"
46+ addFunction "read" readDoc
47+ addFunction "pipe" pipeFn
48+ addFunction "walk_block" walkBlock
49+ addFunction "walk_inline" walkInline
5350 return 1
5451
5552walkElement :: (Walkable (SingletonsList Inline) a,
56− Walkable (SingletonsList Block) a,
57− Walkable (List Inline) a,
58− Walkable (List Block) a)
53+ Walkable (SingletonsList Block) a)
5954 => a -> LuaFilter -> PandocLua a
60−walkElement x f = liftPandocLua $
61− walkInlines f x >>= walkInlineLists f >>= walkBlocks f >>= walkBlockLists f
55+walkElement x f = liftPandocLua $ walkInlines f x >>= walkBlocks f
6256
63−walk_inline :: Inline -> LuaFilter -> PandocLua Inline
64−walk_inline = walkElement
57+walkInline :: Inline -> LuaFilter -> PandocLua Inline
58+walkInline = walkElement
6559
66−walk_block :: Block -> LuaFilter -> PandocLua Block
67−walk_block = walkElement
60+walkBlock :: Block -> LuaFilter -> PandocLua Block
61+walkBlock = walkElement
6862
69−read :: T.Text -> Optional T.Text -> PandocLua NumResults
70−read content formatSpecOrNil = liftPandocLua $ do
63+readDoc :: T.Text -> Optional T.Text -> PandocLua NumResults
64+readDoc content formatSpecOrNil = liftPandocLua $ do
7165 let formatSpec = fromMaybe "markdown" (Lua.fromOptional formatSpecOrNil)
7266 res <- Lua.liftIO . runIO $
7367 getReader formatSpec >>= \(rdr,es) ->
@@ -85,11 +79,11 @@ read content formatSpecOrNil = liftPandocLua $ do
8579 Left e -> Lua.raiseError $ show e
8680
8781-- | Pipes input through a command.
88−pipe :: String -- ^ path to executable
89− -> [String] -- ^ list of arguments
90− -> BL.ByteString -- ^ input passed to process via stdin
91− -> PandocLua NumResults
92−pipe command args input = liftPandocLua $ do
82+pipeFn :: String
83+ -> [String]
84+ -> BL.ByteString
85+ -> PandocLua NumResults
86+pipeFn command args input = liftPandocLua $ do
9387 (ec, output) <- Lua.liftIO $ pipeProcess Nothing command args input
9488 case ec of
9589 ExitSuccess -> 1 <$ Lua.push output
src/Text/Pandoc/Lua/Module/System.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua.Module.System
3− Copyright : © 2019-2021 Albert Krewinkel
3+ Copyright : © 2019-2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Module/Types.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Lua.Module.Types
3− Copyright : © 2019-2021 Albert Krewinkel
3+ Copyright : © 2019-2020 Albert Krewinkel
44 License : GNU GPL, version 2 or above
55
66 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Module/Utils.hs+2−2
@@ -2,7 +2,7 @@
22{-# LANGUAGE ScopedTypeVariables #-}
33{- |
44 Module : Text.Pandoc.Lua.Module.Utils
5− Copyright : Copyright © 2017-2021 Albert Krewinkel
5+ Copyright : Copyright © 2017-2020 Albert Krewinkel
66 License : GNU GPL, version 2 or above
77
88 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -146,7 +146,7 @@ from_simple_table (SimpleTable capt aligns widths head' body) = do
146146 nullAttr
147147 (Caption Nothing [Plain capt])
148148 (zipWith (\a w -> (a, toColWidth w)) aligns widths)
149− (TableHead nullAttr [blockListToRow head' | not (null head') ])
149+ (TableHead nullAttr [blockListToRow head'])
150150 [TableBody nullAttr 0 [] $ map blockListToRow body]
151151 (TableFoot nullAttr [])
152152 return (NumResults 1)
src/Text/Pandoc/Lua/Packages.hs+27−10
@@ -1,6 +1,9 @@
1+{-# LANGUAGE FlexibleContexts #-}
2+{-# LANGUAGE LambdaCase #-}
3+{-# LANGUAGE ScopedTypeVariables #-}
14{- |
25 Module : Text.Pandoc.Lua.Packages
3− Copyright : Copyright © 2017-2021 Albert Krewinkel
6+ Copyright : Copyright © 2017-2020 Albert Krewinkel
47 License : GNU GPL, version 2 or above
58
69 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -12,12 +15,15 @@ module Text.Pandoc.Lua.Packages
1215 ( installPandocPackageSearcher
1316 ) where
1417
18+import Control.Monad.Catch (try)
1519import Control.Monad (forM_)
16−import Foreign.Lua (NumResults)
17−import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua, loadDefaultModule)
20+import Data.ByteString (ByteString)
21+import Foreign.Lua (Lua, NumResults)
22+import Text.Pandoc.Error (PandocError)
23+import Text.Pandoc.Class.PandocMonad (readDataFile)
24+import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua)
1825
1926import qualified Foreign.Lua as Lua
20−import qualified Foreign.Lua.Module.Path as Path
2127import qualified Foreign.Lua.Module.Text as Text
2228import qualified Text.Pandoc.Lua.Module.Pandoc as Pandoc
2329import qualified Text.Pandoc.Lua.Module.MediaBag as MediaBag
@@ -44,17 +50,28 @@ pandocPackageSearcher pkgName =
4450 case pkgName of
4551 "pandoc" -> pushWrappedHsFun Pandoc.pushModule
4652 "pandoc.mediabag" -> pushWrappedHsFun MediaBag.pushModule
47− "pandoc.path" -> pushWrappedHsFun Path.pushModule
4853 "pandoc.system" -> pushWrappedHsFun System.pushModule
4954 "pandoc.types" -> pushWrappedHsFun Types.pushModule
5055 "pandoc.utils" -> pushWrappedHsFun Utils.pushModule
5156 "text" -> pushWrappedHsFun Text.pushModule
52− "pandoc.List" -> pushWrappedHsFun (loadDefaultModule pkgName)
53− _ -> reportPandocSearcherFailure
57+ _ -> searchPureLuaLoader
5458 where
5559 pushWrappedHsFun f = liftPandocLua $ do
5660 Lua.pushHaskellFunction f
5761 return 1
58− reportPandocSearcherFailure = liftPandocLua $ do
59− Lua.push ("\n\t" <> pkgName <> "is not one of pandoc's default packages")
60− return (1 :: NumResults)
62+ searchPureLuaLoader = do
63+ let filename = pkgName ++ ".lua"
64+ try (readDataFile filename) >>= \case
65+ Right script -> pushWrappedHsFun (loadStringAsPackage pkgName script)
66+ Left (_ :: PandocError) -> liftPandocLua $ do
67+ Lua.push ("\n\tno file '" ++ filename ++ "' in pandoc's datadir")
68+ return (1 :: NumResults)
69+
70+loadStringAsPackage :: String -> ByteString -> Lua NumResults
71+loadStringAsPackage pkgName script = do
72+ status <- Lua.dostring script
73+ if status == Lua.OK
74+ then return (1 :: NumResults)
75+ else do
76+ msg <- Lua.popValue
77+ Lua.raiseError ("Error while loading `" <> pkgName <> "`.\n" <> msg)
src/Text/Pandoc/Lua/PandocLua.hs+13−20
@@ -9,7 +9,7 @@
99{-# OPTIONS_GHC -fno-warn-orphans #-}
1010{- |
1111 Module : Text.Pandoc.Lua.PandocLua
12− Copyright : Copyright © 2020-2021 Albert Krewinkel
12+ Copyright : Copyright © 2020 Albert Krewinkel
1313 License : GNU GPL, version 2 or above
1414
1515 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -23,23 +23,24 @@ module Text.Pandoc.Lua.PandocLua
2323 , runPandocLua
2424 , liftPandocLua
2525 , addFunction
26− , loadDefaultModule
26+ , loadScriptFromDataDir
2727 ) where
2828
29+import Control.Monad (when)
2930import Control.Monad.Catch (MonadCatch, MonadMask, MonadThrow)
3031import Control.Monad.Except (MonadError (catchError, throwError))
3132import Control.Monad.IO.Class (MonadIO (liftIO))
3233import Foreign.Lua (Lua (..), NumResults, Pushable, ToHaskellFunction)
3334import Text.Pandoc.Class.PandocIO (PandocIO)
34−import Text.Pandoc.Class.PandocMonad (PandocMonad (..), readDefaultDataFile)
35−import Text.Pandoc.Error (PandocError (PandocLuaError))
35+import Text.Pandoc.Class.PandocMonad (PandocMonad (..), readDataFile)
36+import Text.Pandoc.Error (PandocError)
3637import Text.Pandoc.Lua.Global (Global (..), setGlobals)
3738import Text.Pandoc.Lua.ErrorConversion (errorConversion)
3839
3940import qualified Control.Monad.Catch as Catch
40−import qualified Data.Text as T
4141import qualified Foreign.Lua as Lua
4242import qualified Text.Pandoc.Class.IO as IO
43+import qualified Text.Pandoc.Lua.Util as LuaUtil
4344
4445-- | Type providing access to both, pandoc and Lua operations.
4546newtype PandocLua a = PandocLua { unPandocLua :: Lua a }
@@ -85,22 +86,14 @@ addFunction name fn = liftPandocLua $ do
8586 Lua.pushHaskellFunction fn
8687 Lua.rawset (-3)
8788
88−-- | Load a pure Lua module included with pandoc. Leaves the result on
89−-- the stack and returns @NumResults 1@.
90−--
91−-- The script is loaded from the default data directory. We do not load
92−-- from data directories supplied via command line, as this could cause
93−-- scripts to be executed even though they had not been passed explicitly.
94−loadDefaultModule :: String -> PandocLua NumResults
95−loadDefaultModule name = do
96− script <- readDefaultDataFile (name <> ".lua")
89+-- | Load a file from pandoc's data directory.
90+loadScriptFromDataDir :: FilePath -> PandocLua ()
91+loadScriptFromDataDir scriptFile = do
92+ script <- readDataFile scriptFile
9793 status <- liftPandocLua $ Lua.dostring script
98− if status == Lua.OK
99− then return (1 :: NumResults)
100− else do
101− msg <- liftPandocLua Lua.popValue
102− let err = "Error while loading `" <> name <> "`.\n" <> msg
103− throwError $ PandocLuaError (T.pack err)
94+ when (status /= Lua.OK) . liftPandocLua $
95+ LuaUtil.throwTopMessageAsError'
96+ (("Couldn't load '" ++ scriptFile ++ "'.\n") ++)
10497
10598-- | Global variables which should always be set.
10699defaultGlobals :: PandocIO [Global]
src/Text/Pandoc/Lua/Util.hs+2−2
@@ -3,8 +3,8 @@
33{-# LANGUAGE OverloadedStrings #-}
44{- |
55 Module : Text.Pandoc.Lua.Util
6− Copyright : © 2012-2021 John MacFarlane,
7− © 2017-2021 Albert Krewinkel
6+ Copyright : © 2012–2020 John MacFarlane,
7+ © 2017-2020 Albert Krewinkel
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
src/Text/Pandoc/Lua/Walk.hs+2−2
@@ -4,8 +4,8 @@
44{-# LANGUAGE MultiParamTypeClasses #-}
55{- |
66Module : Text.Pandoc.Lua.Walk
7−Copyright : © 2012-2021 John MacFarlane,
8− © 2017-2021 Albert Krewinkel
7+Copyright : © 2012–2020 John MacFarlane,
8+ © 2017-2020 Albert Krewinkel
99License : GNU GPL, version 2 or above
1010Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
1111Stability : alpha
src/Text/Pandoc/MIME.hs+3−26
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.MIME
4− Copyright : Copyright (C) 2011-2021 John MacFarlane
4+ Copyright : Copyright (C) 2011-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -10,13 +10,8 @@
1010
1111Mime type lookup.
1212-}
13−module Text.Pandoc.MIME (
14− MimeType,
15− getMimeType,
16− getMimeTypeDef,
17− getCharset,
18− extensionFromMimeType,
19− mediaCategory ) where
13+module Text.Pandoc.MIME ( MimeType, getMimeType, getMimeTypeDef,
14+ extensionFromMimeType, mediaCategory ) where
2015import Data.List (isPrefixOf, isSuffixOf)
2116import qualified Data.Map as M
2217import qualified Data.Text as T
@@ -43,16 +38,6 @@ getMimeTypeDef :: FilePath -> MimeType
4338getMimeTypeDef = fromMaybe "application/octet-stream" . getMimeType
4439
4540extensionFromMimeType :: MimeType -> Maybe T.Text
46−-- few special cases, where there are multiple options:
47−extensionFromMimeType "text/plain" = Just "txt"
48−extensionFromMimeType "video/quicktime" = Just "mov"
49−extensionFromMimeType "video/mpeg" = Just "mpeg"
50−extensionFromMimeType "video/dv" = Just "dv"
51−extensionFromMimeType "image/vnd.djvu" = Just "djvu"
52−extensionFromMimeType "image/tiff" = Just "tiff"
53−extensionFromMimeType "image/jpeg" = Just "jpg"
54−extensionFromMimeType "application/xml" = Just "xml"
55−extensionFromMimeType "application/ogg" = Just "ogg"
5641extensionFromMimeType mimetype =
5742 M.lookup (T.takeWhile (/=';') mimetype) reverseMimeTypes
5843 -- note: we just look up the basic mime type, dropping the content-encoding etc.
@@ -69,14 +54,6 @@ reverseMimeTypes = M.fromList $ map swap mimeTypesList
6954mimeTypes :: M.Map T.Text MimeType
7055mimeTypes = M.fromList mimeTypesList
7156
72−-- | Get the charset from a mime type, if one is present.
73−getCharset :: MimeType -> Maybe T.Text
74−getCharset mt =
75− let (_,y) = T.breakOn "charset=" mt
76− in if T.null y
77− then Nothing
78− else Just $ T.toUpper $ T.takeWhile (/= ';') $ T.drop 8 y
79−
8057-- | Collection of common mime types.
8158-- Except for first entry, list borrowed from
8259-- <https://github.com/Happstack/happstack-server/blob/master/src/Happstack/Server/FileServe/BuildingBlocks.hs happstack-server>
src/Text/Pandoc/MediaBag.hs+15−46
@@ -1,10 +1,9 @@
11{-# LANGUAGE CPP #-}
2−{-# LANGUAGE OverloadedStrings #-}
32{-# LANGUAGE DeriveDataTypeable #-}
43{-# LANGUAGE GeneralizedNewtypeDeriving #-}
54{- |
65 Module : Text.Pandoc.MediaBag
7− Copyright : Copyright (C) 2014-2015, 2017-2021 John MacFarlane
6+ Copyright : Copyright (C) 2014-2015, 2017–2020 John MacFarlane
87 License : GNU GPL, version 2 or above
98
109 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -15,7 +14,6 @@ Definition of a MediaBag object to hold binary resources, and an
1514interface for interacting with it.
1615-}
1716module Text.Pandoc.MediaBag (
18− MediaItem(..),
1917 MediaBag,
2018 deleteMedia,
2119 lookupMedia,
@@ -26,43 +24,29 @@ module Text.Pandoc.MediaBag (
2624import qualified Data.ByteString.Lazy as BL
2725import Data.Data (Data)
2826import qualified Data.Map as M
29−import Data.Maybe (fromMaybe, isNothing)
27+import Data.Maybe (fromMaybe)
3028import Data.Typeable (Typeable)
3129import System.FilePath
32−import Text.Pandoc.MIME (MimeType, getMimeTypeDef, extensionFromMimeType)
33−import Data.Text (Text)
34−import qualified Data.Text as T
35−import Data.Digest.Pure.SHA (sha1, showDigest)
36−import Network.URI (URI (..), parseURI)
37−
38−data MediaItem =
39− MediaItem
40− { mediaMimeType :: MimeType
41− , mediaPath :: FilePath
42− , mediaContents :: BL.ByteString
43− } deriving (Eq, Ord, Show, Data, Typeable)
30+import qualified System.FilePath.Posix as Posix
31+import Text.Pandoc.MIME (MimeType, getMimeTypeDef)
4432
4533-- | A container for a collection of binary resources, with names and
4634-- mime types. Note that a 'MediaBag' is a Monoid, so 'mempty'
4735-- can be used for an empty 'MediaBag', and '<>' can be used to append
4836-- two 'MediaBag's.
49−newtype MediaBag = MediaBag (M.Map Text MediaItem)
37+newtype MediaBag = MediaBag (M.Map [FilePath] (MimeType, BL.ByteString))
5038 deriving (Semigroup, Monoid, Data, Typeable)
5139
5240instance Show MediaBag where
5341 show bag = "MediaBag " ++ show (mediaDirectory bag)
5442
55−-- | We represent paths with /, in normalized form.
56−canonicalize :: FilePath -> Text
57−canonicalize = T.replace "\\" "/" . T.pack . normalise
58−
5943-- | Delete a media item from a 'MediaBag', or do nothing if no item corresponds
6044-- to the given path.
6145deleteMedia :: FilePath -- ^ relative path and canonical name of resource
6246 -> MediaBag
6347 -> MediaBag
6448deleteMedia fp (MediaBag mediamap) =
65− MediaBag $ M.delete (canonicalize fp) mediamap
49+ MediaBag $ M.delete (splitDirectories fp) mediamap
6650
6751-- | Insert a media item into a 'MediaBag', replacing any existing
6852-- value with the same name.
@@ -72,41 +56,26 @@ insertMedia :: FilePath -- ^ relative path and canonical name of resource
7256 -> MediaBag
7357 -> MediaBag
7458insertMedia fp mbMime contents (MediaBag mediamap) =
75− MediaBag (M.insert fp' mediaItem mediamap)
76− where mediaItem = MediaItem{ mediaPath = newpath
77− , mediaContents = contents
78− , mediaMimeType = mt }
79− fp' = canonicalize fp
80− uri = parseURI fp
81− newpath = if isRelative fp
82− && isNothing uri
83− && ".." `notElem` splitPath fp
84− then T.unpack fp'
85− else showDigest (sha1 contents) <> "." <> ext
59+ MediaBag (M.insert (splitDirectories fp) (mime, contents) mediamap)
60+ where mime = fromMaybe fallback mbMime
8661 fallback = case takeExtension fp of
8762 ".gz" -> getMimeTypeDef $ dropExtension fp
8863 _ -> getMimeTypeDef fp
89− mt = fromMaybe fallback mbMime
90− path = maybe fp uriPath uri
91− ext = case takeExtension path of
92− '.':e -> e
93− _ -> maybe "" T.unpack $ extensionFromMimeType mt
94−
9564
9665-- | Lookup a media item in a 'MediaBag', returning mime type and contents.
9766lookupMedia :: FilePath
9867 -> MediaBag
99− -> Maybe MediaItem
100−lookupMedia fp (MediaBag mediamap) = M.lookup (canonicalize fp) mediamap
68+ -> Maybe (MimeType, BL.ByteString)
69+lookupMedia fp (MediaBag mediamap) = M.lookup (splitDirectories fp) mediamap
10170
10271-- | Get a list of the file paths stored in a 'MediaBag', with
10372-- their corresponding mime types and the lengths in bytes of the contents.
10473mediaDirectory :: MediaBag -> [(FilePath, MimeType, Int)]
105−mediaDirectory mediabag =
106− map (\(fp, mt, bs) -> (fp, mt, fromIntegral (BL.length bs)))
107− (mediaItems mediabag)
74+mediaDirectory (MediaBag mediamap) =
75+ M.foldrWithKey (\fp (mime,contents) ->
76+ ((Posix.joinPath fp, mime, fromIntegral $ BL.length contents):)) [] mediamap
10877
10978mediaItems :: MediaBag -> [(FilePath, MimeType, BL.ByteString)]
11079mediaItems (MediaBag mediamap) =
111− map (\item -> (mediaPath item, mediaMimeType item, mediaContents item))
112− (M.elems mediamap)
80+ M.foldrWithKey (\fp (mime,contents) ->
81+ ((Posix.joinPath fp, mime, contents):)) [] mediamap
src/Text/Pandoc/Options.hs+18−19
@@ -6,7 +6,7 @@
66{-# LANGUAGE TemplateHaskell #-}
77{- |
88 Module : Text.Pandoc.Options
9− Copyright : Copyright (C) 2012-2021 John MacFarlane
9+ Copyright : Copyright (C) 2012-2020 John MacFarlane
1010 License : GNU GPL, version 2 or above
1111
1212 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -65,7 +65,6 @@ data ReaderOptions = ReaderOptions{
6565 , readerDefaultImageExtension :: Text -- ^ Default extension for images
6666 , readerTrackChanges :: TrackChanges -- ^ Track changes setting for docx
6767 , readerStripComments :: Bool -- ^ Strip HTML comments instead of parsing as raw HTML
68− -- (only implemented in commonmark)
6968} deriving (Show, Read, Data, Typeable, Generic)
7069
7170instance HasSyntaxExtensions ReaderOptions where
@@ -315,23 +314,6 @@ defaultMathJaxURL = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.j
315314defaultKaTeXURL :: Text
316315defaultKaTeXURL = "https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/"
317316
318−-- Update documentation in doc/filters.md if this is changed.
319−$(deriveJSON defaultOptions{ constructorTagModifier =
320− camelCaseStrToHyphenated
321− } ''TrackChanges)
322−
323−$(deriveJSON defaultOptions{ constructorTagModifier =
324− camelCaseStrToHyphenated
325− } ''WrapOption)
326−
327−$(deriveJSON defaultOptions{ constructorTagModifier =
328− camelCaseStrToHyphenated . drop 8
329− } ''TopLevelDivision)
330−
331−$(deriveJSON defaultOptions{ constructorTagModifier =
332− camelCaseStrToHyphenated
333− } ''ReferenceLocation)
334−
335317-- Update documentation in doc/filters.md if this is changed.
336318$(deriveJSON defaultOptions ''ReaderOptions)
337319
@@ -355,3 +337,20 @@ $(deriveJSON defaultOptions{ constructorTagModifier =
355337 } ''ObfuscationMethod)
356338
357339$(deriveJSON defaultOptions ''HTMLSlideVariant)
340+
341+-- Update documentation in doc/filters.md if this is changed.
342+$(deriveJSON defaultOptions{ constructorTagModifier =
343+ camelCaseStrToHyphenated
344+ } ''TrackChanges)
345+
346+$(deriveJSON defaultOptions{ constructorTagModifier =
347+ camelCaseStrToHyphenated
348+ } ''WrapOption)
349+
350+$(deriveJSON defaultOptions{ constructorTagModifier =
351+ camelCaseStrToHyphenated . drop 8
352+ } ''TopLevelDivision)
353+
354+$(deriveJSON defaultOptions{ constructorTagModifier =
355+ camelCaseStrToHyphenated
356+ } ''ReferenceLocation)
src/Text/Pandoc/PDF.hs+16−34
@@ -4,7 +4,7 @@
44{-# LANGUAGE ScopedTypeVariables #-}
55{- |
66 Module : Text.Pandoc.PDF
7− Copyright : Copyright (C) 2012-2021 John MacFarlane
7+ Copyright : Copyright (C) 2012-2020 John MacFarlane
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -43,7 +43,6 @@ import Text.Pandoc.Definition
4343import Text.Pandoc.Error (PandocError (PandocPDFProgramNotFoundError))
4444import Text.Pandoc.MIME (getMimeType)
4545import Text.Pandoc.Options (HTMLMathMethod (..), WriterOptions (..))
46−import Text.Pandoc.Extensions (disableExtension, Extension(Ext_smart))
4746import Text.Pandoc.Process (pipeProcess)
4847import System.Process (readProcessWithExitCode)
4948import Text.Pandoc.Shared (inDirectory, stringify, tshow)
@@ -115,10 +114,7 @@ makePDF program pdfargs writer opts doc =
115114 runIOorExplode $ do
116115 putCommonState commonState
117116 doc' <- handleImages opts tmpdir doc
118− source <- writer opts{ writerExtensions = -- disable use of quote
119− -- ligatures to avoid bad ligatures like ?`
120− disableExtension Ext_smart
121− (writerExtensions opts) } doc'
117+ source <- writer opts doc'
122118 res <- case baseProg of
123119 "context" -> context2pdf verbosity program pdfargs tmpdir source
124120 "tectonic" -> tectonic2pdf verbosity program pdfargs tmpdir source
@@ -202,7 +198,7 @@ convertImage opts tmpdir fname = do
202198 Just "image/svg+xml" -> E.catch (do
203199 (exit, _) <- pipeProcess Nothing "rsvg-convert"
204200 ["-f","pdf","-a","--dpi-x",dpi,"--dpi-y",dpi,
205− "-o",pdfOut,svgIn] BL.empty
201+ "-o",pdfOut,fname] BL.empty
206202 if exit == ExitSuccess
207203 then return $ Right pdfOut
208204 else return $ Left "conversion from SVG failed")
@@ -217,7 +213,6 @@ convertImage opts tmpdir fname = do
217213 where
218214 pngOut = replaceDirectory (replaceExtension fname ".png") tmpdir
219215 pdfOut = replaceDirectory (replaceExtension fname ".pdf") tmpdir
220− svgIn = tmpdir </> fname
221216 mime = getMimeType fname
222217 doNothing = return (Right fname)
223218
@@ -271,7 +266,7 @@ missingCharacterWarnings verbosity log' = do
271266 | isAscii c = T.singleton c
272267 | otherwise = T.pack $ c : " (U+" ++ printf "%04X" (ord c) ++ ")"
273268 let addCodePoint = T.concatMap toCodePoint
274− let warnings = [ addCodePoint (utf8ToText (BC.drop 19 l))
269+ let warnings = [ addCodePoint (T.pack $ utf8ToString (BC.drop 19 l))
275270 | l <- ls
276271 , isMissingCharacterWarning l
277272 ]
@@ -315,7 +310,7 @@ runTectonic verbosity program args' tmpDir' source = do
315310 env <- liftIO getEnvironment
316311 when (verbosity >= INFO) $ liftIO $
317312 showVerboseInfo (Just tmpDir) program programArgs env
318− (utf8ToText sourceBL)
313+ (utf8ToString sourceBL)
319314 (exit, out) <- liftIO $ E.catch
320315 (pipeProcess (Just env) program programArgs sourceBL)
321316 (handlePDFProgramNotFound program)
@@ -386,7 +381,7 @@ runTeXProgram verbosity program args numRuns tmpDir' source = do
386381 (pipeProcess (Just env'') program programArgs BL.empty)
387382 (handlePDFProgramNotFound program)
388383 when (verbosity >= INFO) $ liftIO $ do
389− UTF8.hPutStrLn stderr $ "[makePDF] Run #" <> tshow runNumber
384+ UTF8.hPutStrLn stderr $ "[makePDF] Run #" ++ show runNumber
390385 BL.hPutStr stderr out
391386 UTF8.hPutStr stderr "\n"
392387 if runNumber < numRuns
@@ -406,7 +401,7 @@ generic2pdf :: Verbosity
406401generic2pdf verbosity program args source = do
407402 env' <- getEnvironment
408403 when (verbosity >= INFO) $
409− showVerboseInfo Nothing program args env' source
404+ showVerboseInfo Nothing program args env' (T.unpack source)
410405 (exit, out) <- E.catch
411406 (pipeProcess (Just env') program args
412407 (BL.fromStrict $ UTF8.fromText source))
@@ -495,32 +490,19 @@ showVerboseInfo :: Maybe FilePath
495490 -> String
496491 -> [String]
497492 -> [(String, String)]
498− -> Text
493+ -> String
499494 -> IO ()
500495showVerboseInfo mbTmpDir program programArgs env source = do
501496 case mbTmpDir of
502497 Just tmpDir -> do
503498 UTF8.hPutStrLn stderr "[makePDF] temp dir:"
504− UTF8.hPutStrLn stderr (T.pack tmpDir)
499+ UTF8.hPutStrLn stderr tmpDir
505500 Nothing -> return ()
506501 UTF8.hPutStrLn stderr "[makePDF] Command line:"
507− UTF8.hPutStrLn stderr $
508− T.pack program <> " " <> T.pack (unwords (map show programArgs))
502+ UTF8.hPutStrLn stderr $ program ++ " " ++ unwords (map show programArgs)
509503 UTF8.hPutStr stderr "\n"
510− UTF8.hPutStrLn stderr "[makePDF] Relevant environment variables:"
511− -- we filter out irrelevant stuff to avoid leaking passwords and keys!
512− let isRelevant ("PATH",_) = True
513− isRelevant ("TMPDIR",_) = True
514− isRelevant ("PWD",_) = True
515− isRelevant ("LANG",_) = True
516− isRelevant ("HOME",_) = True
517− isRelevant ("LUA_PATH",_) = True
518− isRelevant ("LUA_CPATH",_) = True
519− isRelevant ("SHELL",_) = True
520− isRelevant ("TEXINPUTS",_) = True
521− isRelevant ("TEXMFOUTPUT",_) = True
522− isRelevant _ = False
523− mapM_ (UTF8.hPutStrLn stderr . tshow) (filter isRelevant env)
504+ UTF8.hPutStrLn stderr "[makePDF] Environment:"
505+ mapM_ (UTF8.hPutStrLn stderr . show) env
524506 UTF8.hPutStr stderr "\n"
525507 UTF8.hPutStrLn stderr "[makePDF] Source:"
526508 UTF8.hPutStrLn stderr source
@@ -531,8 +513,8 @@ handlePDFProgramNotFound program e
531513 E.throwIO $ PandocPDFProgramNotFoundError $ T.pack program
532514 | otherwise = E.throwIO e
533515
534−utf8ToText :: ByteString -> Text
535−utf8ToText lbs =
516+utf8ToString :: ByteString -> String
517+utf8ToString lbs =
536518 case decodeUtf8' lbs of
537− Left _ -> T.pack $ BC.unpack lbs -- if decoding fails, treat as latin1
538− Right t -> TL.toStrict t
519+ Left _ -> BC.unpack lbs -- if decoding fails, treat as latin1
520+ Right t -> TL.unpack t
src/Text/Pandoc/Parsing.hs+264−405
@@ -5,10 +5,11 @@
55{-# LANGUAGE GeneralizedNewtypeDeriving #-}
66{-# LANGUAGE IncoherentInstances #-}
77{-# LANGUAGE MultiParamTypeClasses #-}
8+{-# LANGUAGE ViewPatterns #-}
89{-# LANGUAGE OverloadedStrings #-}
910{- |
1011 Module : Text.Pandoc.Parsing
11− Copyright : Copyright (C) 2006-2021 John MacFarlane
12+ Copyright : Copyright (C) 2006-2020 John MacFarlane
1213 License : GNU GPL, version 2 or above
1314
1415 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -18,7 +19,8 @@
1819A utility library with parsers used in pandoc readers.
1920-}
2021
21−module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
22+module Text.Pandoc.Parsing ( take1WhileP,
23+ takeP,
2224 countChar,
2325 textStr,
2426 anyLine,
@@ -103,14 +105,14 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
103105 singleQuoteEnd,
104106 doubleQuoteStart,
105107 doubleQuoteEnd,
106− apostrophe,
107− doubleCloseQuote,
108108 ellipses,
109+ apostrophe,
109110 dash,
110111 nested,
111112 citeKey,
112113 Parser,
113114 ParserT,
115+ F,
114116 Future(..),
115117 runF,
116118 askF,
@@ -121,6 +123,7 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
121123 (<+?>),
122124 extractIdClass,
123125 insertIncludedFile,
126+ insertIncludedFileF,
124127 -- * Re-exports from Text.Parsec
125128 Stream,
126129 runParser,
@@ -131,10 +134,22 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
131134 getInput,
132135 setInput,
133136 unexpected,
137+ char,
138+ letter,
139+ digit,
140+ alphaNum,
134141 skipMany,
135142 skipMany1,
143+ spaces,
144+ space,
145+ anyChar,
146+ satisfy,
147+ newline,
148+ string,
136149 count,
137150 eof,
151+ noneOf,
152+ oneOf,
138153 lookAhead,
139154 notFollowedBy,
140155 many,
@@ -159,8 +174,6 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
159174 SourcePos,
160175 getPosition,
161176 setPosition,
162− sourceName,
163− setSourceName,
164177 sourceColumn,
165178 sourceLine,
166179 setSourceColumn,
@@ -176,141 +189,48 @@ module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,
176189where
177190
178191import Control.Monad.Identity
179− ( guard,
180− join,
181− unless,
182− when,
183− void,
184− liftM2,
185− liftM,
186− Identity(..),
187− MonadPlus(mzero) )
188192import Control.Monad.Reader
189− ( asks, runReader, MonadReader(ask), Reader, ReaderT(ReaderT) )
190−import Data.Char (chr, isAlphaNum, isAscii, isAsciiUpper, isAsciiLower,
191− isSpace, ord, toLower, toUpper)
192−import Data.Default ( Default(..) )
193+import Data.Char (chr, isAlphaNum, isAscii, isAsciiUpper,
194+ isPunctuation, isSpace, ord, toLower, toUpper)
195+import Data.Default
193196import Data.Functor (($>))
194197import Data.List (intercalate, transpose)
195198import qualified Data.Map as M
196−import Data.Maybe (fromMaybe)
199+import Data.Maybe (mapMaybe, fromMaybe)
197200import qualified Data.Set as Set
201+import Data.String
198202import Data.Text (Text)
199203import qualified Data.Text as T
200204import Text.HTML.TagSoup.Entity (lookupEntity)
201−import Text.Pandoc.Asciify (toAsciiText)
205+import Text.Pandoc.Asciify (toAsciiChar)
202206import Text.Pandoc.Builder (Blocks, HasMeta (..), Inlines, trimInlines)
203207import qualified Text.Pandoc.Builder as B
204208import Text.Pandoc.Class.PandocMonad (PandocMonad, readFileFromDirs, report)
205209import Text.Pandoc.Definition
206− ( Target,
207− nullMeta,
208− nullAttr,
209− Meta,
210− ColWidth(ColWidthDefault, ColWidth),
211− TableFoot(TableFoot),
212− TableBody(TableBody),
213− Attr,
214− TableHead(TableHead),
215− Row(..),
216− Alignment(..),
217− Inline(Str),
218− ListNumberDelim(..),
219− ListAttributes,
220− ListNumberStyle(..) )
221210import Text.Pandoc.Logging
222− ( LogMessage(CouldNotLoadIncludeFile, DuplicateIdentifier) )
223211import Text.Pandoc.Options
224− ( extensionEnabled,
225− Extension(Ext_old_dashes, Ext_tex_math_dollars,
226− Ext_tex_math_single_backslash, Ext_tex_math_double_backslash,
227− Ext_auto_identifiers, Ext_ascii_identifiers, Ext_smart),
228− ReaderOptions(readerTabStop, readerColumns, readerExtensions) )
229212import Text.Pandoc.Readers.LaTeX.Types (Macro)
230213import Text.Pandoc.Shared
231− ( uniqueIdent,
232− tshow,
233− mapLeft,
234− compactify,
235− trim,
236− trimr,
237− splitTextByIndices,
238− safeRead,
239− trimMath,
240− schemes,
241− escapeURI )
242−import Text.Pandoc.Sources
243214import qualified Text.Pandoc.UTF8 as UTF8 (putStrLn)
244215import Text.Pandoc.XML (fromEntities)
245−import Text.Parsec
246− ( between,
247− setSourceName,
248− Parsec,
249− Column,
250− Line,
251− incSourceLine,
252− incSourceColumn,
253− setSourceLine,
254− setSourceColumn,
255− sourceLine,
256− sourceColumn,
257− sourceName,
258− setSourceName,
259− setPosition,
260− getPosition,
261− updateState,
262− setState,
263− getState,
264− optionMaybe,
265− optional,
266− option,
267− endBy1,
268− endBy,
269− sepEndBy1,
270− sepEndBy,
271− sepBy1,
272− sepBy,
273− try,
274− choice,
275− (<?>),
276− (<|>),
277− manyTill,
278− many1,
279− many,
280− notFollowedBy,
281− lookAhead,
282− eof,
283− count,
284− skipMany1,
285− skipMany,
286− unexpected,
287− setInput,
288− getInput,
289− anyToken,
290− tokenPrim,
291− parse,
292− runParserT,
293− runParser,
294− ParseError,
295− ParsecT,
296− SourcePos,
297− Stream(..) )
298−import Text.Parsec.Pos (initialPos, newPos)
299−import Control.Monad.Except ( MonadError(throwError) )
216+import Text.Parsec hiding (token)
217+import Text.Parsec.Pos (initialPos, newPos, updatePosString)
218+
219+import Control.Monad.Except
300220import Text.Pandoc.Error
301− ( PandocError(PandocParseError, PandocParsecError) )
302221
303222type Parser t s = Parsec t s
304223
305224type ParserT = ParsecT
306225
307−
308226-- | Reader monad wrapping the parser state. This is used to possibly delay
309227-- evaluation until all relevant information has been parsed and made available
310228-- in the parser state.
311229newtype Future s a = Future { runDelayed :: Reader s a }
312230 deriving (Monad, Applicative, Functor)
313231
232+type F = Future ParserState
233+
314234runF :: Future s a -> s -> a
315235runF = runReader . runDelayed
316236
@@ -333,48 +253,70 @@ instance (Semigroup a, Monoid a) => Monoid (Future s a) where
333253 mappend = (<>)
334254
335255-- | Like @count@, but packs its result
336−countChar :: (Stream s m Char, UpdateSourcePos s Char, Monad m)
256+countChar :: (Stream s m Char, Monad m)
337257 => Int
338258 -> ParsecT s st m Char
339259 -> ParsecT s st m Text
340260countChar n = fmap T.pack . count n
341261
342262-- | Like @string@, but uses @Text@.
343−textStr :: (Stream s m Char, UpdateSourcePos s Char)
344− => Text -> ParsecT s u m Text
263+textStr :: Stream s m Char => Text -> ParsecT s u m Text
345264textStr t = string (T.unpack t) $> t
346265
347−
348−-- | Parse any line of text, returning the contents without the
349−-- final newline.
350−anyLine :: Monad m => ParserT Sources st m Text
266+-- | Parse characters while a predicate is true.
267+take1WhileP :: Monad m
268+ => (Char -> Bool)
269+ -> ParserT Text st m Text
270+take1WhileP f = do
271+ -- needed to persuade parsec that this won't match an empty string:
272+ c <- satisfy f
273+ inp <- getInput
274+ pos <- getPosition
275+ let (t, rest) = T.span f inp
276+ setInput rest
277+ setPosition $
278+ if f '\t' || f '\n'
279+ then updatePosString pos $ T.unpack t
280+ else incSourceColumn pos (T.length t)
281+ return $ T.singleton c <> t
282+
283+-- Parse n characters of input (or the rest of the input if
284+-- there aren't n characters).
285+takeP :: Monad m => Int -> ParserT Text st m Text
286+takeP n = do
287+ guard (n > 0)
288+ -- faster than 'count n anyChar'
289+ inp <- getInput
290+ pos <- getPosition
291+ let (xs, rest) = T.splitAt n inp
292+ -- needed to persuade parsec that this won't match an empty string:
293+ anyChar
294+ setInput rest
295+ setPosition $ updatePosString pos $ T.unpack xs
296+ return xs
297+
298+-- | Parse any line of text
299+anyLine :: Monad m => ParserT Text st m Text
351300anyLine = do
352301 -- This is much faster than:
353302 -- manyTill anyChar newline
354303 inp <- getInput
355− case inp of
356− Sources [] -> mzero
357− Sources ((fp,t):inps) ->
358− -- we assume that lines don't span different input files
359− case T.break (=='\n') t of
360− (this, rest)
361− | T.null rest
362− , not (null inps) ->
363− -- line may span different input files, so do it
364− -- character by character
365− T.pack <$> manyTill anyChar newline
366− | otherwise -> do -- either end of inputs or newline in rest
367− setInput $ Sources ((fp, rest):inps)
368− char '\n' -- needed so parsec knows we won't match empty string
369− -- and so source pos is updated
370− return this
304+ pos <- getPosition
305+ case T.break (=='\n') inp of
306+ (this, T.uncons -> Just ('\n', rest)) -> do
307+ -- needed to persuade parsec that this won't match an empty string:
308+ anyChar
309+ setInput rest
310+ setPosition $ incSourceLine (setSourceColumn pos 1) 1
311+ return this
312+ _ -> mzero
371313
372314-- | Parse any line, include the final newline in the output
373−anyLineNewline :: Monad m => ParserT Sources st m Text
315+anyLineNewline :: Monad m => ParserT Text st m Text
374316anyLineNewline = (<> "\n") <$> anyLine
375317
376318-- | Parse indent by specified number of spaces (or equiv. tabs)
377−indentWith :: (Stream s m Char, UpdateSourcePos s Char)
319+indentWith :: Stream s m Char
378320 => HasReaderOptions st
379321 => Int -> ParserT s st m Text
380322indentWith num = do
@@ -459,13 +401,11 @@ notFollowedBy' p = try $ join $ do a <- try p
459401 return (return ())
460402-- (This version due to Andrew Pimlott on the Haskell mailing list.)
461403
462−oneOfStrings' :: (Stream s m Char, UpdateSourcePos s Char)
463− => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
404+oneOfStrings' :: Stream s m Char => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text
464405oneOfStrings' f = fmap T.pack . oneOfStrings'' f . fmap T.unpack
465406
466407-- TODO: This should be re-implemented in a Text-aware way
467−oneOfStrings'' :: (Stream s m Char, UpdateSourcePos s Char)
468− => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
408+oneOfStrings'' :: Stream s m Char => (Char -> Char -> Bool) -> [String] -> ParserT s st m String
469409oneOfStrings'' _ [] = Prelude.fail "no strings"
470410oneOfStrings'' matches strs = try $ do
471411 c <- anyChar
@@ -480,16 +420,14 @@ oneOfStrings'' matches strs = try $ do
480420-- | Parses one of a list of strings. If the list contains
481421-- two strings one of which is a prefix of the other, the longer
482422-- string will be matched if possible.
483−oneOfStrings :: (Stream s m Char, UpdateSourcePos s Char)
484− => [Text] -> ParserT s st m Text
423+oneOfStrings :: Stream s m Char => [Text] -> ParserT s st m Text
485424oneOfStrings = oneOfStrings' (==)
486425
487426-- | Parses one of a list of strings (tried in order), case insensitive.
488427
489428-- TODO: This will not be accurate with general Unicode (neither
490429-- Text.toLower nor Text.toCaseFold can be implemented with a map)
491−oneOfStringsCI :: (Stream s m Char, UpdateSourcePos s Char)
492− => [Text] -> ParserT s st m Text
430+oneOfStringsCI :: Stream s m Char => [Text] -> ParserT s st m Text
493431oneOfStringsCI = oneOfStrings' ciMatch
494432 where ciMatch x y = toLower' x == toLower' y
495433 -- this optimizes toLower by checking common ASCII case
@@ -500,41 +438,29 @@ oneOfStringsCI = oneOfStrings' ciMatch
500438 | otherwise = toLower c
501439
502440-- | Parses a space or tab.
503−spaceChar :: (Stream s m Char, UpdateSourcePos s Char)
504− => ParserT s st m Char
441+spaceChar :: Stream s m Char => ParserT s st m Char
505442spaceChar = satisfy $ \c -> c == ' ' || c == '\t'
506443
507444-- | Parses a nonspace, nonnewline character.
508−nonspaceChar :: (Stream s m Char, UpdateSourcePos s Char)
509− => ParserT s st m Char
510−nonspaceChar = satisfy (not . isSpaceChar)
511−
512−isSpaceChar :: Char -> Bool
513−isSpaceChar ' ' = True
514−isSpaceChar '\t' = True
515−isSpaceChar '\n' = True
516−isSpaceChar '\r' = True
517−isSpaceChar _ = False
445+nonspaceChar :: Stream s m Char => ParserT s st m Char
446+nonspaceChar = noneOf ['\t', '\n', ' ', '\r']
518447
519448-- | Skips zero or more spaces or tabs.
520−skipSpaces :: (Stream s m Char, UpdateSourcePos s Char)
521− => ParserT s st m ()
449+skipSpaces :: Stream s m Char => ParserT s st m ()
522450skipSpaces = skipMany spaceChar
523451
524452-- | Skips zero or more spaces or tabs, then reads a newline.
525−blankline :: (Stream s m Char, UpdateSourcePos s Char)
526− => ParserT s st m Char
453+blankline :: Stream s m Char => ParserT s st m Char
527454blankline = try $ skipSpaces >> newline
528455
529456-- | Parses one or more blank lines and returns a string of newlines.
530−blanklines :: (Stream s m Char, UpdateSourcePos s Char)
531− => ParserT s st m Text
457+blanklines :: Stream s m Char => ParserT s st m Text
532458blanklines = T.pack <$> many1 blankline
533459
534460-- | Gobble n spaces; if tabs are encountered, expand them
535461-- and gobble some or all of their spaces, leaving the rest.
536462gobbleSpaces :: (HasReaderOptions st, Monad m)
537− => Int -> ParserT Sources st m ()
463+ => Int -> ParserT Text st m ()
538464gobbleSpaces 0 = return ()
539465gobbleSpaces n
540466 | n < 0 = error "gobbleSpaces called with negative number"
@@ -542,26 +468,18 @@ gobbleSpaces n
542468 char ' ' <|> eatOneSpaceOfTab
543469 gobbleSpaces (n - 1)
544470
545−eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Sources st m Char
471+eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Text st m Char
546472eatOneSpaceOfTab = do
547− lookAhead (char '\t')
548− pos <- getPosition
473+ char '\t'
549474 tabstop <- getOption readerTabStop
550− -- replace the tab on the input stream with spaces
551− let numSpaces = tabstop - ((sourceColumn pos - 1) `mod` tabstop)
552475 inp <- getInput
553− setInput $
554− case inp of
555− Sources [] -> error "eatOneSpaceOfTab - empty Sources list"
556− Sources ((fp,t):rest) ->
557− -- drop the tab and add spaces
558− Sources ((fp, T.replicate numSpaces " " <> T.drop 1 t):rest)
559− char ' '
476+ setInput $ T.replicate (tabstop - 1) " " <> inp
477+ return ' '
560478
561479-- | Gobble up to n spaces; if tabs are encountered, expand them
562480-- and gobble some or all of their spaces, leaving the rest.
563481gobbleAtMostSpaces :: (HasReaderOptions st, Monad m)
564− => Int -> ParserT Sources st m Int
482+ => Int -> ParserT Text st m Int
565483gobbleAtMostSpaces 0 = return 0
566484gobbleAtMostSpaces n
567485 | n < 0 = error "gobbleAtMostSpaces called with negative number"
@@ -570,8 +488,7 @@ gobbleAtMostSpaces n
570488 (+ 1) <$> gobbleAtMostSpaces (n - 1)
571489
572490-- | Parses material enclosed between start and end parsers.
573−enclosed :: (Show end, Stream s m Char, UpdateSourcePos s Char)
574− => ParserT s st m t -- ^ start parser
491+enclosed :: (Show end, Stream s m Char) => ParserT s st m t -- ^ start parser
575492 -> ParserT s st m end -- ^ end parser
576493 -> ParserT s st m a -- ^ content parser (to be used repeatedly)
577494 -> ParserT s st m [a]
@@ -579,41 +496,39 @@ enclosed start end parser = try $
579496 start >> notFollowedBy space >> many1Till parser end
580497
581498-- | Parse string, case insensitive.
582−stringAnyCase :: (Stream s m Char, UpdateSourcePos s Char)
583− => Text -> ParserT s st m Text
499+stringAnyCase :: Stream s m Char => Text -> ParserT s st m Text
584500stringAnyCase = fmap T.pack . stringAnyCase' . T.unpack
585501
586−stringAnyCase' :: (Stream s m Char, UpdateSourcePos s Char)
587− => String -> ParserT s st m String
502+stringAnyCase' :: Stream s m Char => String -> ParserT s st m String
588503stringAnyCase' [] = string ""
589504stringAnyCase' (x:xs) = do
590505 firstChar <- char (toUpper x) <|> char (toLower x)
591506 rest <- stringAnyCase' xs
592507 return (firstChar:rest)
593508
594−-- TODO rewrite by just adding to Sources stream?
595509-- | Parse contents of 'str' using 'parser' and return result.
596−parseFromString :: Monad m
597− => ParserT Sources st m r
510+parseFromString :: (Stream s m Char, IsString s)
511+ => ParserT s st m r
598512 -> Text
599− -> ParserT Sources st m r
513+ -> ParserT s st m r
600514parseFromString parser str = do
601515 oldPos <- getPosition
602516 setPosition $ initialPos "chunk"
603517 oldInput <- getInput
604− setInput $ toSources str
518+ setInput $ fromString $ T.unpack str
605519 result <- parser
606520 spaces
521+ eof
607522 setInput oldInput
608523 setPosition oldPos
609524 return result
610525
611526-- | Like 'parseFromString' but specialized for 'ParserState'.
612527-- This resets 'stateLastStrPos', which is almost always what we want.
613−parseFromString' :: (Monad m, HasLastStrPosition u)
614− => ParserT Sources u m a
528+parseFromString' :: (Stream s m Char, IsString s, HasLastStrPosition u)
529+ => ParserT s u m a
615530 -> Text
616− -> ParserT Sources u m a
531+ -> ParserT s u m a
617532parseFromString' parser str = do
618533 oldLastStrPos <- getLastStrPos <$> getState
619534 updateState $ setLastStrPos Nothing
@@ -622,7 +537,7 @@ parseFromString' parser str = do
622537 return res
623538
624539-- | Parse raw line block up to and including blank lines.
625−lineClump :: Monad m => ParserT Sources st m Text
540+lineClump :: Monad m => ParserT Text st m Text
626541lineClump = blanklines
627542 <|> (T.unlines <$> many1 (notFollowedBy blankline >> anyLine))
628543
@@ -631,7 +546,7 @@ lineClump = blanklines
631546-- pairs of open and close, which must be different. For example,
632547-- @charsInBalanced '(' ')' anyChar@ will parse "(hello (there))"
633548-- and return "hello (there)".
634−charsInBalanced :: (Stream s m Char, UpdateSourcePos s Char) => Char -> Char -> ParserT s st m Char
549+charsInBalanced :: Stream s m Char => Char -> Char -> ParserT s st m Char
635550 -> ParserT s st m Text
636551charsInBalanced open close parser = try $ do
637552 char open
@@ -650,7 +565,7 @@ charsInBalanced open close parser = try $ do
650565-- Auxiliary functions for romanNumeral:
651566
652567-- | Parses a roman numeral (uppercase or lowercase), returns number.
653−romanNumeral :: (Stream s m Char, UpdateSourcePos s Char) => Bool -- ^ Uppercase if true
568+romanNumeral :: Stream s m Char => Bool -- ^ Uppercase if true
654569 -> ParserT s st m Int
655570romanNumeral upperCase = do
656571 let rchar uc = char $ if upperCase then uc else toLower uc
@@ -686,19 +601,20 @@ romanNumeral upperCase = do
686601
687602-- | Parses an email address; returns original and corresponding
688603-- escaped mailto: URI.
689−emailAddress :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
604+emailAddress :: Stream s m Char => ParserT s st m (Text, Text)
690605emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
691606 where toResult mbox dom = let full = fromEntities $ T.pack $ mbox ++ '@':dom
692607 in (full, escapeURI $ "mailto:" <> full)
693608 mailbox = intercalate "." <$> (emailWord `sepBy1'` dot)
694609 domain = intercalate "." <$> (subdomain `sepBy1'` dot)
695610 dot = char '.'
696− subdomain = many1 $ alphaNum <|> innerPunct (=='-')
611+ subdomain = many1 $ alphaNum <|> innerPunct
697612 -- this excludes some valid email addresses, since an
698613 -- email could contain e.g. '__', but gives better results
699614 -- for our purposes, when combined with markdown parsing:
700− innerPunct f = try (satisfy f
701− <* notFollowedBy (satisfy (not . isAlphaNum)))
615+ innerPunct = try (satisfy (\c -> isEmailPunct c || c == '@')
616+ <* notFollowedBy space
617+ <* notFollowedBy (satisfy isPunctuation))
702618 -- technically an email address could begin with a symbol,
703619 -- but allowing this creates too many problems.
704620 -- See e.g. https://github.com/jgm/pandoc/issues/2940
@@ -709,16 +625,16 @@ emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)
709625 isEmailPunct c = T.any (== c) "!\"#$%&'*+-/=?^_{|}~;"
710626
711627
712−uriScheme :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Text
628+uriScheme :: Stream s m Char => ParserT s st m Text
713629uriScheme = oneOfStringsCI (Set.toList schemes)
714630
715631-- | Parses a URI. Returns pair of original and URI-escaped version.
716−uri :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)
632+uri :: Stream s m Char => ParserT s st m (Text, Text)
717633uri = try $ do
718634 scheme <- uriScheme
719635 char ':'
720636 -- Avoid parsing e.g. "**Notes:**" as a raw URI:
721− notFollowedBy $ satisfy (\c -> c == '*' || c == '_' || c == ']')
637+ notFollowedBy (oneOf "*_]")
722638 -- We allow sentence punctuation except at the end, since
723639 -- we don't want the trailing '.' in 'http://google.com.' We want to allow
724640 -- http://en.wikipedia.org/wiki/State_of_emergency_(disambiguation)
@@ -732,20 +648,7 @@ uri = try $ do
732648 let uri' = scheme <> ":" <> fromEntities str'
733649 return (uri', escapeURI uri')
734650 where
735− isWordChar '#' = True
736− isWordChar '$' = True
737− isWordChar '%' = True
738− isWordChar '+' = True
739− isWordChar '/' = True
740− isWordChar '@' = True
741− isWordChar '\\' = True
742− isWordChar '_' = True
743− isWordChar '-' = True
744− isWordChar '&' = True
745− isWordChar '=' = True
746− isWordChar c = isAlphaNum c
747−
748− wordChar = satisfy isWordChar
651+ wordChar = alphaNum <|> oneOf "#$%+/@\\_-&="
749652 percentEscaped = try $ (:) <$> char '%' <*> many1 hexDigit
750653 entity = try $ pure <$> characterReference
751654 punct = try $ many1 (char ',') <|> fmap pure (satisfy (\c -> not (isSpace c) && c /= '<' && c /= '>'))
@@ -756,13 +659,11 @@ uri = try $ do
756659 uriChunkBetween l r = try $ do chunk <- between (char l) (char r) uriChunk
757660 return (T.pack $ [l] ++ chunk ++ [r])
758661
759−mathInlineWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
662+mathInlineWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
760663mathInlineWith op cl = try $ do
761664 textStr op
762665 when (op == "$") $ notFollowedBy space
763− words' <- many1Till (
764− (T.singleton <$>
765− satisfy (\c -> not (isSpaceChar c || c == '\\')))
666+ words' <- many1Till (countChar 1 (noneOf " \t\n\\")
766667 <|> (char '\\' >>
767668 -- This next clause is needed because \text{..} can
768669 -- contain $, \(\), etc.
@@ -770,17 +671,17 @@ mathInlineWith op cl = try $ do
770671 (("\\text" <>) <$> inBalancedBraces 0 ""))
771672 <|> (\c -> T.pack ['\\',c]) <$> anyChar))
772673 <|> do (blankline <* notFollowedBy' blankline) <|>
773− (spaceChar <* skipMany spaceChar)
674+ (oneOf " \t" <* skipMany (oneOf " \t"))
774675 notFollowedBy (char '$')
775676 return " "
776677 ) (try $ textStr cl)
777678 notFollowedBy digit -- to prevent capture of $5
778679 return $ trimMath $ T.concat words'
779680 where
780− inBalancedBraces :: (Stream s m Char, UpdateSourcePos s Char) => Int -> Text -> ParserT s st m Text
681+ inBalancedBraces :: Stream s m Char => Int -> Text -> ParserT s st m Text
781682 inBalancedBraces n = fmap T.pack . inBalancedBraces' n . T.unpack
782683
783− inBalancedBraces' :: (Stream s m Char, UpdateSourcePos s Char) => Int -> String -> ParserT s st m String
684+ inBalancedBraces' :: Stream s m Char => Int -> String -> ParserT s st m String
784685 inBalancedBraces' 0 "" = do
785686 c <- anyChar
786687 if c == '{'
@@ -797,13 +698,12 @@ mathInlineWith op cl = try $ do
797698 '{' -> inBalancedBraces' (numOpen + 1) (c:xs)
798699 _ -> inBalancedBraces' numOpen (c:xs)
799700
800−mathDisplayWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text
701+mathDisplayWith :: Stream s m Char => Text -> Text -> ParserT s st m Text
801702mathDisplayWith op cl = try $ fmap T.pack $ do
802703 textStr op
803− many1Till (satisfy (/= '\n') <|> (newline <* notFollowedBy' blankline))
804− (try $ textStr cl)
704+ many1Till (noneOf "\n" <|> (newline <* notFollowedBy' blankline)) (try $ textStr cl)
805705
806−mathDisplay :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
706+mathDisplay :: (HasReaderOptions st, Stream s m Char)
807707 => ParserT s st m Text
808708mathDisplay =
809709 (guardEnabled Ext_tex_math_dollars >> mathDisplayWith "$$" "$$")
@@ -812,7 +712,7 @@ mathDisplay =
812712 <|> (guardEnabled Ext_tex_math_double_backslash >>
813713 mathDisplayWith "\\\\[" "\\\\]")
814714
815−mathInline :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
715+mathInline :: (HasReaderOptions st , Stream s m Char)
816716 => ParserT s st m Text
817717mathInline =
818718 (guardEnabled Ext_tex_math_dollars >> mathInlineWith "$" "$")
@@ -825,7 +725,7 @@ mathInline =
825725-- displacement (the difference between the source column at the end
826726-- and the source column at the beginning). Vertical displacement
827727-- (source row) is ignored.
828−withHorizDisplacement :: (Stream s m Char, UpdateSourcePos s Char)
728+withHorizDisplacement :: Stream s m Char
829729 => ParserT s st m a -- ^ Parser to apply
830730 -> ParserT s st m (a, Int) -- ^ (result, displacement)
831731withHorizDisplacement parser = do
@@ -837,37 +737,30 @@ withHorizDisplacement parser = do
837737-- | Applies a parser and returns the raw string that was parsed,
838738-- along with the value produced by the parser.
839739withRaw :: Monad m
840− => ParsecT Sources st m a
841− -> ParsecT Sources st m (a, Text)
740+ => ParsecT Text st m a
741+ -> ParsecT Text st m (a, Text)
842742withRaw parser = do
843− inps1 <- getInput
743+ pos1 <- getPosition
744+ inp <- getInput
844745 result <- parser
845− inps2 <- getInput
846− -- 'raw' is the difference between inps1 and inps2
847− return (result, sourcesDifference inps1 inps2)
848−
849−sourcesDifference :: Sources -> Sources -> Text
850−sourcesDifference (Sources is1) (Sources is2) = go is1 is2
851− where
852− go inps1 inps2 =
853− case (inps1, inps2) of
854− ([], _) -> mempty
855− (_, []) -> mconcat $ map snd inps1
856− ((p1,t1):rest1, (p2, t2):rest2)
857− | p1 == p2
858− , t1 == t2 -> go rest1 rest2
859− | p1 == p2
860− , t1 /= t2 -> fromMaybe mempty $ T.stripSuffix t2 t1
861− | otherwise -> t1 <> go rest1 inps2
746+ pos2 <- getPosition
747+ let (l1,c1) = (sourceLine pos1, sourceColumn pos1)
748+ let (l2,c2) = (sourceLine pos2, sourceColumn pos2)
749+ let inplines = take ((l2 - l1) + 1) $ T.lines inp
750+ let raw = case inplines of
751+ [] -> ""
752+ [l] -> T.take (c2 - c1) l
753+ ls -> T.unlines (init ls) <> T.take (c2 - 1) (last ls)
754+ return (result, raw)
862755
863756-- | Parses backslash, then applies character parser.
864−escaped :: (Stream s m Char, UpdateSourcePos s Char)
757+escaped :: Stream s m Char
865758 => ParserT s st m Char -- ^ Parser for character to escape
866759 -> ParserT s st m Char
867760escaped parser = try $ char '\\' >> parser
868761
869762-- | Parse character entity.
870−characterReference :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
763+characterReference :: Stream s m Char => ParserT s st m Char
871764characterReference = try $ do
872765 char '&'
873766 ent <- many1Till nonspaceChar (char ';')
@@ -880,19 +773,19 @@ characterReference = try $ do
880773 _ -> Prelude.fail "entity not found"
881774
882775-- | Parses an uppercase roman numeral and returns (UpperRoman, number).
883−upperRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
776+upperRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
884777upperRoman = do
885778 num <- romanNumeral True
886779 return (UpperRoman, num)
887780
888781-- | Parses a lowercase roman numeral and returns (LowerRoman, number).
889−lowerRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
782+lowerRoman :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
890783lowerRoman = do
891784 num <- romanNumeral False
892785 return (LowerRoman, num)
893786
894787-- | Parses a decimal numeral and returns (Decimal, number).
895−decimal :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
788+decimal :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
896789decimal = do
897790 num <- many1 digit
898791 return (Decimal, fromMaybe 1 $ safeRead $ T.pack num)
@@ -901,7 +794,7 @@ decimal = do
901794-- returns (DefaultStyle, [next example number]). The next
902795-- example number is incremented in parser state, and the label
903796-- (if present) is added to the label table.
904−exampleNum :: (Stream s m Char, UpdateSourcePos s Char)
797+exampleNum :: Stream s m Char
905798 => ParserT s ParserState m (ListNumberStyle, Int)
906799exampleNum = do
907800 char '@'
@@ -920,37 +813,37 @@ exampleNum = do
920813 return (Example, num)
921814
922815-- | Parses a '#' returns (DefaultStyle, 1).
923−defaultNum :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
816+defaultNum :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
924817defaultNum = do
925818 char '#'
926819 return (DefaultStyle, 1)
927820
928821-- | Parses a lowercase letter and returns (LowerAlpha, number).
929−lowerAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
822+lowerAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
930823lowerAlpha = do
931− ch <- satisfy isAsciiLower
824+ ch <- oneOf ['a'..'z']
932825 return (LowerAlpha, ord ch - ord 'a' + 1)
933826
934827-- | Parses an uppercase letter and returns (UpperAlpha, number).
935−upperAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
828+upperAlpha :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
936829upperAlpha = do
937− ch <- satisfy isAsciiUpper
830+ ch <- oneOf ['A'..'Z']
938831 return (UpperAlpha, ord ch - ord 'A' + 1)
939832
940833-- | Parses a roman numeral i or I
941−romanOne :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)
834+romanOne :: Stream s m Char => ParserT s st m (ListNumberStyle, Int)
942835romanOne = (char 'i' >> return (LowerRoman, 1)) <|>
943836 (char 'I' >> return (UpperRoman, 1))
944837
945838-- | Parses an ordered list marker and returns list attributes.
946−anyOrderedListMarker :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s ParserState m ListAttributes
839+anyOrderedListMarker :: Stream s m Char => ParserT s ParserState m ListAttributes
947840anyOrderedListMarker = choice
948841 [delimParser numParser | delimParser <- [inPeriod, inOneParen, inTwoParens],
949842 numParser <- [decimal, exampleNum, defaultNum, romanOne,
950843 lowerAlpha, lowerRoman, upperAlpha, upperRoman]]
951844
952845-- | Parses a list number (num) followed by a period, returns list attributes.
953−inPeriod :: (Stream s m Char, UpdateSourcePos s Char)
846+inPeriod :: Stream s m Char
954847 => ParserT s st m (ListNumberStyle, Int)
955848 -> ParserT s st m ListAttributes
956849inPeriod num = try $ do
@@ -962,7 +855,7 @@ inPeriod num = try $ do
962855 return (start, style, delim)
963856
964857-- | Parses a list number (num) followed by a paren, returns list attributes.
965−inOneParen :: (Stream s m Char, UpdateSourcePos s Char)
858+inOneParen :: Stream s m Char
966859 => ParserT s st m (ListNumberStyle, Int)
967860 -> ParserT s st m ListAttributes
968861inOneParen num = try $ do
@@ -971,7 +864,7 @@ inOneParen num = try $ do
971864 return (start, style, OneParen)
972865
973866-- | Parses a list number (num) enclosed in parens, returns list attributes.
974−inTwoParens :: (Stream s m Char, UpdateSourcePos s Char)
867+inTwoParens :: Stream s m Char
975868 => ParserT s st m (ListNumberStyle, Int)
976869 -> ParserT s st m ListAttributes
977870inTwoParens num = try $ do
@@ -982,7 +875,7 @@ inTwoParens num = try $ do
982875
983876-- | Parses an ordered list marker with a given style and delimiter,
984877-- returns number.
985−orderedListMarker :: (Stream s m Char, UpdateSourcePos s Char)
878+orderedListMarker :: Stream s m Char
986879 => ListNumberStyle
987880 -> ListNumberDelim
988881 -> ParserT s ParserState m Int
@@ -1005,10 +898,10 @@ orderedListMarker style delim = do
1005898 return start
1006899
1007900-- | Parses a character reference and returns a Str element.
1008−charRef :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inline
901+charRef :: Stream s m Char => ParserT s st m Inline
1009902charRef = Str . T.singleton <$> characterReference
1010903
1011−lineBlockLine :: Monad m => ParserT Sources st m Text
904+lineBlockLine :: Monad m => ParserT Text st m Text
1012905lineBlockLine = try $ do
1013906 char '|'
1014907 char ' '
@@ -1018,11 +911,11 @@ lineBlockLine = try $ do
1018911 continuations <- many (try $ char ' ' >> anyLine)
1019912 return $ white <> T.unwords (line : continuations)
1020913
1021−blankLineBlockLine :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char
914+blankLineBlockLine :: Stream s m Char => ParserT s st m Char
1022915blankLineBlockLine = try (char '|' >> blankline)
1023916
1024917-- | Parses an RST-style line block and returns a list of strings.
1025−lineBlockLines :: Monad m => ParserT Sources st m [Text]
918+lineBlockLines :: Monad m => ParserT Text st m [Text]
1026919lineBlockLines = try $ do
1027920 lines' <- many1 (lineBlockLine <|> (T.singleton <$> blankLineBlockLine))
1028921 skipMany blankline
@@ -1030,8 +923,7 @@ lineBlockLines = try $ do
1030923
1031924-- | Parse a table using 'headerParser', 'rowParser',
1032925-- 'lineParser', and 'footerParser'.
1033−tableWith :: (Stream s m Char, UpdateSourcePos s Char,
1034− HasReaderOptions st, Monad mf)
926+tableWith :: (Stream s m Char, HasReaderOptions st, Monad mf)
1035927 => ParserT s st m (mf [Blocks], [Alignment], [Int])
1036928 -> ([Int] -> ParserT s st m (mf [Blocks]))
1037929 -> ParserT s st m sep
@@ -1051,8 +943,7 @@ tableWith headerParser rowParser lineParser footerParser = try $ do
1051943
1052944type TableComponents mf = ([Alignment], [Double], mf [Row], mf [Row])
1053945
1054−tableWith' :: (Stream s m Char, UpdateSourcePos s Char,
1055− HasReaderOptions st, Monad mf)
946+tableWith' :: (Stream s m Char, HasReaderOptions st, Monad mf)
1056947 => ParserT s st m (mf [Blocks], [Alignment], [Int])
1057948 -> ([Int] -> ParserT s st m (mf [Blocks]))
1058949 -> ParserT s st m sep
@@ -1101,19 +992,20 @@ widthsFromIndices numColumns' indices =
1101992-- (which may be grid), then the rows,
1102993-- which may be grid, separated by blank lines, and
1103994-- ending with a footer (dashed line followed by blank line).
1104−gridTableWith :: (Monad m, HasReaderOptions st, HasLastStrPosition st, Monad mf)
1105− => ParserT Sources st m (mf Blocks) -- ^ Block list parser
995+gridTableWith :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
996+ Monad mf, IsString s)
997+ => ParserT s st m (mf Blocks) -- ^ Block list parser
1106998 -> Bool -- ^ Headerless table
1107− -> ParserT Sources st m (mf Blocks)
999+ -> ParserT s st m (mf Blocks)
11081000gridTableWith blocks headless =
11091001 tableWith (gridTableHeader headless blocks) (gridTableRow blocks)
11101002 (gridTableSep '-') gridTableFooter
11111003
1112−gridTableWith' :: (Monad m, HasReaderOptions st, HasLastStrPosition st,
1113− Monad mf)
1114− => ParserT Sources st m (mf Blocks) -- ^ Block list parser
1004+gridTableWith' :: (Stream s m Char, HasReaderOptions st, HasLastStrPosition st,
1005+ Monad mf, IsString s)
1006+ => ParserT s st m (mf Blocks) -- ^ Block list parser
11151007 -> Bool -- ^ Headerless table
1116− -> ParserT Sources st m (TableComponents mf)
1008+ -> ParserT s st m (TableComponents mf)
11171009gridTableWith' blocks headless =
11181010 tableWith' (gridTableHeader headless blocks) (gridTableRow blocks)
11191011 (gridTableSep '-') gridTableFooter
@@ -1122,7 +1014,7 @@ gridTableSplitLine :: [Int] -> Text -> [Text]
11221014gridTableSplitLine indices line = map removeFinalBar $ tail $
11231015 splitTextByIndices (init indices) $ trimr line
11241016
1125−gridPart :: Monad m => Char -> ParserT Sources st m ((Int, Int), Alignment)
1017+gridPart :: Stream s m Char => Char -> ParserT s st m ((Int, Int), Alignment)
11261018gridPart ch = do
11271019 leftColon <- option False (True <$ char ':')
11281020 dashes <- many1 (char ch)
@@ -1137,7 +1029,7 @@ gridPart ch = do
11371029 (False, False) -> AlignDefault
11381030 return ((lengthDashes, lengthDashes + 1), alignment)
11391031
1140−gridDashedLines :: Monad m => Char -> ParserT Sources st m [((Int, Int), Alignment)]
1032+gridDashedLines :: Stream s m Char => Char -> ParserT s st m [((Int, Int), Alignment)]
11411033gridDashedLines ch = try $ char '+' >> many1 (gridPart ch) <* blankline
11421034
11431035removeFinalBar :: Text -> Text
@@ -1146,47 +1038,47 @@ removeFinalBar = T.dropWhileEnd go . T.dropWhileEnd (=='|')
11461038 go c = T.any (== c) " \t"
11471039
11481040-- | Separator between rows of grid table.
1149−gridTableSep :: Monad m => Char -> ParserT Sources st m Char
1041+gridTableSep :: Stream s m Char => Char -> ParserT s st m Char
11501042gridTableSep ch = try $ gridDashedLines ch >> return '\n'
11511043
11521044-- | Parse header for a grid table.
1153−gridTableHeader :: (Monad m, Monad mf, HasLastStrPosition st)
1045+gridTableHeader :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
11541046 => Bool -- ^ Headerless table
1155− -> ParserT Sources st m (mf Blocks)
1156− -> ParserT Sources st m (mf [Blocks], [Alignment], [Int])
1157−gridTableHeader True _ = do
1158− optional blanklines
1159− dashes <- gridDashedLines '-'
1160− let aligns = map snd dashes
1161− let lines' = map (snd . fst) dashes
1162− let indices = scanl (+) 0 lines'
1163− return (return [], aligns, indices)
1164−gridTableHeader False blocks = try $ do
1047+ -> ParserT s st m (mf Blocks)
1048+ -> ParserT s st m (mf [Blocks], [Alignment], [Int])
1049+gridTableHeader headless blocks = try $ do
11651050 optional blanklines
11661051 dashes <- gridDashedLines '-'
1167− rawContent <- many1 (notFollowedBy (gridTableSep '=') >> char '|' >>
1052+ rawContent <- if headless
1053+ then return $ repeat ""
1054+ else many1
1055+ (notFollowedBy (gridTableSep '=') >> char '|' >>
11681056 T.pack <$> many1Till anyChar newline)
1169− underDashes <- gridDashedLines '='
1057+ underDashes <- if headless
1058+ then return dashes
1059+ else gridDashedLines '='
11701060 guard $ length dashes == length underDashes
11711061 let lines' = map (snd . fst) underDashes
11721062 let indices = scanl (+) 0 lines'
11731063 let aligns = map snd underDashes
1174− let rawHeads = map (T.unlines . map trim) $ transpose
1064+ let rawHeads = if headless
1065+ then replicate (length underDashes) ""
1066+ else map (T.unlines . map trim) $ transpose
11751067 $ map (gridTableSplitLine indices) rawContent
11761068 heads <- sequence <$> mapM (parseFromString' blocks . trim) rawHeads
11771069 return (heads, aligns, indices)
11781070
1179−gridTableRawLine :: (Stream s m Char, UpdateSourcePos s Char) => [Int] -> ParserT s st m [Text]
1071+gridTableRawLine :: Stream s m Char => [Int] -> ParserT s st m [Text]
11801072gridTableRawLine indices = do
11811073 char '|'
11821074 line <- many1Till anyChar newline
11831075 return (gridTableSplitLine indices $ T.pack line)
11841076
11851077-- | Parse row of grid table.
1186−gridTableRow :: (Monad m, Monad mf, HasLastStrPosition st)
1187− => ParserT Sources st m (mf Blocks)
1078+gridTableRow :: (Stream s m Char, Monad mf, IsString s, HasLastStrPosition st)
1079+ => ParserT s st m (mf Blocks)
11881080 -> [Int]
1189− -> ParserT Sources st m (mf [Blocks])
1081+ -> ParserT s st m (mf [Blocks])
11901082gridTableRow blocks indices = do
11911083 colLines <- many1 (gridTableRawLine indices)
11921084 let cols = map ((<> "\n") . T.unlines . removeOneLeadingSpace) $
@@ -1207,38 +1099,34 @@ removeOneLeadingSpace xs =
12071099 Just (c, _) -> c == ' '
12081100
12091101-- | Parse footer for a grid table.
1210−gridTableFooter :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
1102+gridTableFooter :: Stream s m Char => ParserT s st m ()
12111103gridTableFooter = optional blanklines
12121104
12131105---
12141106
12151107-- | Removes the ParsecT layer from the monad transformer stack
1216−readWithM :: (Monad m, ToSources t)
1217− => ParserT Sources st m a -- ^ parser
1218− -> st -- ^ initial state
1219− -> t -- ^ input
1108+readWithM :: (Stream s m Char, ToText s)
1109+ => ParserT s st m a -- ^ parser
1110+ -> st -- ^ initial state
1111+ -> s -- ^ input
12201112 -> m (Either PandocError a)
12211113readWithM parser state input =
1222− mapLeft (PandocParsecError sources)
1223− <$> runParserT parser state (initialSourceName sources) sources
1224− where
1225− sources = toSources input
1114+ mapLeft (PandocParsecError $ toText input) `liftM` runParserT parser state "source" input
12261115
12271116-- | Parse a string with a given parser and state
1228−readWith :: ToSources t
1229− => Parser Sources st a
1117+readWith :: Parser Text st a
12301118 -> st
1231− -> t
1119+ -> Text
12321120 -> Either PandocError a
12331121readWith p t inp = runIdentity $ readWithM p t inp
12341122
12351123-- | Parse a string with @parser@ (for testing).
12361124testStringWith :: Show a
1237− => ParserT Sources ParserState Identity a
1125+ => ParserT Text ParserState Identity a
12381126 -> Text
12391127 -> IO ()
1240−testStringWith parser str = UTF8.putStrLn $ tshow $
1241− readWith parser defaultParserState (toSources str)
1128+testStringWith parser str = UTF8.putStrLn $ show $
1129+ readWith parser defaultParserState str
12421130
12431131-- | Parsing options.
12441132data ParserState = ParserState
@@ -1258,7 +1146,7 @@ data ParserState = ParserState
12581146 stateInNote :: Bool, -- ^ True if parsing note contents
12591147 stateNoteNumber :: Int, -- ^ Last note number for citations
12601148 stateMeta :: Meta, -- ^ Document metadata
1261− stateMeta' :: Future ParserState Meta, -- ^ Document metadata
1149+ stateMeta' :: F Meta, -- ^ Document metadata
12621150 stateCitations :: M.Map Text Text, -- ^ RST-style citations
12631151 stateHeaderTable :: [HeaderType], -- ^ Ordered list of header types used
12641152 stateIdentifiers :: Set.Set Text, -- ^ Header identifiers used
@@ -1437,7 +1325,7 @@ data QuoteContext
14371325
14381326type NoteTable = [(Text, Text)]
14391327
1440−type NoteTable' = M.Map Text (SourcePos, Future ParserState Blocks)
1328+type NoteTable' = M.Map Text (SourcePos, F Blocks)
14411329-- used in markdown reader
14421330
14431331newtype Key = Key Text deriving (Show, Read, Eq, Ord)
@@ -1472,7 +1360,7 @@ registerHeader (ident,classes,kvs) header' = do
14721360 then do
14731361 let id' = uniqueIdent exts (B.toList header') ids
14741362 let id'' = if Ext_ascii_identifiers `extensionEnabled` exts
1475− then toAsciiText id'
1363+ then T.pack $ mapMaybe toAsciiChar $ T.unpack id'
14761364 else id'
14771365 updateState $ updateIdentifierList $ Set.insert id'
14781366 updateState $ updateIdentifierList $ Set.insert id''
@@ -1485,42 +1373,34 @@ registerHeader (ident,classes,kvs) header' = do
14851373 updateState $ updateIdentifierList $ Set.insert ident
14861374 return (ident,classes,kvs)
14871375
1488−smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st,
1489− HasQuoteContext st m,
1490− Stream s m Char, UpdateSourcePos s Char)
1376+smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
14911377 => ParserT s st m Inlines
14921378 -> ParserT s st m Inlines
14931379smartPunctuation inlineParser = do
14941380 guardEnabled Ext_smart
1495− choice [ quoted inlineParser, apostrophe, doubleCloseQuote, dash, ellipses ]
1381+ choice [ quoted inlineParser, apostrophe, dash, ellipses ]
1382+
1383+apostrophe :: Stream s m Char => ParserT s st m Inlines
1384+apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\x2019")
14961385
1497−quoted :: (HasLastStrPosition st, HasQuoteContext st m,
1498− Stream s m Char, UpdateSourcePos s Char)
1386+quoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
14991387 => ParserT s st m Inlines
15001388 -> ParserT s st m Inlines
15011389quoted inlineParser = doubleQuoted inlineParser <|> singleQuoted inlineParser
15021390
1503−singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m,
1504− Stream s m Char, UpdateSourcePos s Char)
1391+singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
15051392 => ParserT s st m Inlines
15061393 -> ParserT s st m Inlines
1507−singleQuoted inlineParser = do
1508− singleQuoteStart
1509− (B.singleQuoted . mconcat <$>
1510− try
1511− (withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)))
1512− <|> pure "\8217"
1513−
1514−doubleQuoted :: (HasQuoteContext st m, HasLastStrPosition st,
1515− Stream s m Char, UpdateSourcePos s Char)
1394+singleQuoted inlineParser = try $ B.singleQuoted . mconcat
1395+ <$ singleQuoteStart
1396+ <*> withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)
1397+
1398+doubleQuoted :: (HasQuoteContext st m, Stream s m Char)
15161399 => ParserT s st m Inlines
15171400 -> ParserT s st m Inlines
1518−doubleQuoted inlineParser = do
1519− doubleQuoteStart
1520− (B.doubleQuoted . mconcat <$>
1521− try
1522− (withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)))
1523− <|> pure (B.str "\8220")
1401+doubleQuoted inlineParser = try $ B.doubleQuoted . mconcat
1402+ <$ doubleQuoteStart
1403+ <*> withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)
15241404
15251405failIfInQuoteContext :: (HasQuoteContext st m, Stream s m t)
15261406 => QuoteContext
@@ -1529,14 +1409,13 @@ failIfInQuoteContext context = do
15291409 context' <- getQuoteContext
15301410 when (context' == context) $ Prelude.fail "already inside quotes"
15311411
1532−charOrRef :: (Stream s m Char, UpdateSourcePos s Char) => [Char] -> ParserT s st m Char
1412+charOrRef :: Stream s m Char => [Char] -> ParserT s st m Char
15331413charOrRef cs =
15341414 oneOf cs <|> try (do c <- characterReference
15351415 guard (c `elem` cs)
15361416 return c)
15371417
1538−singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m,
1539− Stream s m Char, UpdateSourcePos s Char)
1418+singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m, Stream s m Char)
15401419 => ParserT s st m ()
15411420singleQuoteStart = do
15421421 failIfInQuoteContext InSingleQuote
@@ -1544,39 +1423,30 @@ singleQuoteStart = do
15441423 guard =<< notAfterString
15451424 try $ do
15461425 charOrRef "'\8216\145"
1547− void $ lookAhead (satisfy (not . isSpaceChar))
1426+ notFollowedBy (oneOf [' ', '\t', '\n'])
15481427
1549−singleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
1428+singleQuoteEnd :: Stream s m Char
15501429 => ParserT s st m ()
15511430singleQuoteEnd = try $ do
15521431 charOrRef "'\8217\146"
15531432 notFollowedBy alphaNum
15541433
1555−doubleQuoteStart :: (HasLastStrPosition st,
1556− HasQuoteContext st m,
1557− Stream s m Char, UpdateSourcePos s Char)
1434+doubleQuoteStart :: (HasQuoteContext st m, Stream s m Char)
15581435 => ParserT s st m ()
15591436doubleQuoteStart = do
15601437 failIfInQuoteContext InDoubleQuote
1561− guard =<< notAfterString
15621438 try $ do charOrRef "\"\8220\147"
1563− void $ lookAhead (satisfy (not . isSpaceChar))
1439+ notFollowedBy (oneOf [' ', '\t', '\n'])
15641440
1565−doubleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)
1441+doubleQuoteEnd :: Stream s m Char
15661442 => ParserT s st m ()
15671443doubleQuoteEnd = void (charOrRef "\"\8221\148")
15681444
1569−apostrophe :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1570−apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\8217")
1571−
1572−doubleCloseQuote :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines
1573−doubleCloseQuote = B.str "\8221" <$ char '"'
1574−
1575−ellipses :: (Stream s m Char, UpdateSourcePos s Char)
1445+ellipses :: Stream s m Char
15761446 => ParserT s st m Inlines
15771447ellipses = try (string "..." >> return (B.str "\8230"))
15781448
1579−dash :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)
1449+dash :: (HasReaderOptions st, Stream s m Char)
15801450 => ParserT s st m Inlines
15811451dash = try $ do
15821452 oldDashes <- extensionEnabled Ext_old_dashes <$> getOption readerExtensions
@@ -1603,28 +1473,20 @@ nested p = do
16031473 updateState $ \st -> st{ stateMaxNestingLevel = nestlevel }
16041474 return res
16051475
1606−citeKey :: (Stream s m Char, UpdateSourcePos s Char, HasLastStrPosition st)
1607− => Bool -- ^ If True, allow expanded @{..} syntax.
1608− -> ParserT s st m (Bool, Text)
1609−citeKey allowBraced = try $ do
1476+citeKey :: (Stream s m Char, HasLastStrPosition st)
1477+ => ParserT s st m (Bool, Text)
1478+citeKey = try $ do
16101479 guard =<< notAfterString
16111480 suppress_author <- option False (True <$ char '-')
16121481 char '@'
1613− key <- simpleCiteIdentifier
1614− <|> if allowBraced
1615− then charsInBalanced '{' '}' (satisfy (not . isSpace))
1616− else mzero
1617− return (suppress_author, key)
1618−
1619−simpleCiteIdentifier :: (Stream s m Char, UpdateSourcePos s Char)
1620− => ParserT s st m Text
1621−simpleCiteIdentifier = do
16221482 firstChar <- alphaNum <|> char '_' <|> char '*' -- @* for wildcard in nocite
16231483 let regchar = satisfy (\c -> isAlphaNum c || c == '_')
16241484 let internal p = try $ p <* lookAhead regchar
16251485 rest <- many $ regchar <|> internal (oneOf ":.#$%&-+?<>~/") <|>
16261486 try (oneOf ":/" <* lookAhead (char '/'))
1627− return $ T.pack $ firstChar:rest
1487+ let key = firstChar:rest
1488+ return (suppress_author, T.pack key)
1489+
16281490
16291491token :: (Stream s m t)
16301492 => (t -> Text)
@@ -1644,15 +1506,12 @@ extractIdClass (ident, cls, kvs) = (ident', cls', kvs')
16441506 cls' = maybe cls T.words $ lookup "class" kvs
16451507 kvs' = filter (\(k,_) -> k /= "id" || k /= "class") kvs
16461508
1647−insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1648− => ParserT a st m b -- ^ parser to apply
1649− -> (Text -> a) -- ^ convert Text to stream type
1650− -> [FilePath] -- ^ search path (directories)
1651− -> FilePath -- ^ path of file to include
1652− -> Maybe Int -- ^ start line (negative counts from end)
1653− -> Maybe Int -- ^ end line (negative counts from end)
1654− -> ParserT a st m b
1655−insertIncludedFile parser toStream dirs f mbstartline mbendline = do
1509+insertIncludedFile' :: (PandocMonad m, HasIncludeFiles st)
1510+ => ParserT a st m (mf Blocks)
1511+ -> (Text -> a)
1512+ -> [FilePath] -> FilePath
1513+ -> ParserT a st m (mf Blocks)
1514+insertIncludedFile' blocks totoks dirs f = do
16561515 oldPos <- getPosition
16571516 oldInput <- getInput
16581517 containers <- getIncludeFiles <$> getState
@@ -1661,32 +1520,32 @@ insertIncludedFile parser toStream dirs f mbstartline mbendline = do
16611520 updateState $ addIncludeFile $ T.pack f
16621521 mbcontents <- readFileFromDirs dirs f
16631522 contents <- case mbcontents of
1664− Just s -> return $ exciseLines mbstartline mbendline s
1523+ Just s -> return s
16651524 Nothing -> do
16661525 report $ CouldNotLoadIncludeFile (T.pack f) oldPos
16671526 return ""
1668− setInput $ toStream contents
1669− setPosition $ newPos f (fromMaybe 1 mbstartline) 1
1670− result <- parser
1527+ setPosition $ newPos f 1 1
1528+ setInput $ totoks contents
1529+ bs <- blocks
16711530 setInput oldInput
16721531 setPosition oldPos
16731532 updateState dropLatestIncludeFile
1674− return result
1675−
1676−exciseLines :: Maybe Int -> Maybe Int -> Text -> Text
1677−exciseLines Nothing Nothing t = t
1678−exciseLines mbstartline mbendline t =
1679− T.unlines $ take (endline' - (startline' - 1))
1680− $ drop (startline' - 1) contentLines
1681− where
1682− contentLines = T.lines t
1683− numLines = length contentLines
1684− startline' = case mbstartline of
1685− Nothing -> 1
1686− Just x | x >= 0 -> x
1687− | otherwise -> numLines + x -- negative from end
1688− endline' = case mbendline of
1689− Nothing -> numLines
1690− Just x | x >= 0 -> x
1691− | otherwise -> numLines + x -- negative from end
1533+ return bs
16921534
1535+-- | Parse content of include file as blocks. Circular includes result in an
1536+-- @PandocParseError@.
1537+insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)
1538+ => ParserT [a] st m Blocks
1539+ -> (Text -> [a])
1540+ -> [FilePath] -> FilePath
1541+ -> ParserT [a] st m Blocks
1542+insertIncludedFile blocks totoks dirs f =
1543+ runIdentity <$> insertIncludedFile' (Identity <$> blocks) totoks dirs f
1544+
1545+-- | Parse content of include file as future blocks. Circular includes result in
1546+-- an @PandocParseError@.
1547+insertIncludedFileF :: (PandocMonad m, HasIncludeFiles st)
1548+ => ParserT Text st m (Future st Blocks)
1549+ -> [FilePath] -> FilePath
1550+ -> ParserT Text st m (Future st Blocks)
1551+insertIncludedFileF p = insertIncludedFile' p id
src/Text/Pandoc/Process.hs+1−1
@@ -1,6 +1,6 @@
11{- |
22 Module : Text.Pandoc.Process
3− Copyright : Copyright (C) 2013-2021 John MacFarlane
3+ Copyright : Copyright (C) 2013-2020 John MacFarlane
44 License : GNU GPL, version 2 or above
55
66 Maintainer : John MacFarlane <jgm@berkeley.edu>
src/Text/Pandoc/Readers.hs+49−56
@@ -1,10 +1,9 @@
11{-# LANGUAGE MonoLocalBinds #-}
2−{-# LANGUAGE RankNTypes #-}
32{-# LANGUAGE ScopedTypeVariables #-}
43{-# LANGUAGE OverloadedStrings #-}
54{- |
65 Module : Text.Pandoc.Readers
7− Copyright : Copyright (C) 2006-2021 John MacFarlane
6+ Copyright : Copyright (C) 2006-2020 John MacFarlane
87 License : GNU GPL, version 2 or above
98
109 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -66,14 +65,12 @@ import Data.Aeson
6665import qualified Data.ByteString.Lazy as BL
6766import Data.Text (Text)
6867import qualified Data.Text as T
69−import Text.Pandoc.Shared (tshow)
7068import Text.Pandoc.Class
7169import Text.Pandoc.Definition
7270import Text.Pandoc.Error
7371import Text.Pandoc.Extensions
7472import Text.Pandoc.Options
7573import Text.Pandoc.Readers.CommonMark
76−import Text.Pandoc.Readers.Markdown
7774import Text.Pandoc.Readers.Creole
7875import Text.Pandoc.Readers.DocBook
7976import Text.Pandoc.Readers.Docx
@@ -86,6 +83,7 @@ import Text.Pandoc.Readers.HTML (readHtml)
8683import Text.Pandoc.Readers.JATS (readJATS)
8784import Text.Pandoc.Readers.Jira (readJira)
8885import Text.Pandoc.Readers.LaTeX
86+import Text.Pandoc.Readers.Markdown
8987import Text.Pandoc.Readers.MediaWiki
9088import Text.Pandoc.Readers.Muse
9189import Text.Pandoc.Readers.Native
@@ -103,60 +101,59 @@ import Text.Pandoc.Readers.CSV
103101import Text.Pandoc.Readers.CslJson
104102import Text.Pandoc.Readers.BibTeX
105103import qualified Text.Pandoc.UTF8 as UTF8
106−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
104+import Text.Parsec.Error
107105
108−data Reader m = TextReader (forall a . ToSources a =>
109− ReaderOptions -> a -> m Pandoc)
106+data Reader m = TextReader (ReaderOptions -> Text -> m Pandoc)
110107 | ByteStringReader (ReaderOptions -> BL.ByteString -> m Pandoc)
111108
112109-- | Association list of formats and readers.
113110readers :: PandocMonad m => [(Text, Reader m)]
114−readers = [("native" , TextReader readNative)
115− ,("json" , TextReader readJSON)
116− ,("markdown" , TextReader readMarkdown)
117− ,("markdown_strict" , TextReader readMarkdown)
118− ,("markdown_phpextra" , TextReader readMarkdown)
119− ,("markdown_github" , TextReader readMarkdown)
120− ,("markdown_mmd", TextReader readMarkdown)
121− ,("commonmark" , TextReader readCommonMark)
122− ,("commonmark_x" , TextReader readCommonMark)
123− ,("creole" , TextReader readCreole)
124− ,("dokuwiki" , TextReader readDokuWiki)
125− ,("gfm" , TextReader readCommonMark)
126− ,("rst" , TextReader readRST)
127− ,("mediawiki" , TextReader readMediaWiki)
128− ,("vimwiki" , TextReader readVimwiki)
129− ,("docbook" , TextReader readDocBook)
130− ,("opml" , TextReader readOPML)
131− ,("org" , TextReader readOrg)
132− ,("textile" , TextReader readTextile) -- TODO : textile+lhs
133− ,("html" , TextReader readHtml)
134− ,("jats" , TextReader readJATS)
135− ,("jira" , TextReader readJira)
136− ,("latex" , TextReader readLaTeX)
137− ,("haddock" , TextReader readHaddock)
138− ,("twiki" , TextReader readTWiki)
139− ,("tikiwiki" , TextReader readTikiWiki)
140− ,("docx" , ByteStringReader readDocx)
141− ,("odt" , ByteStringReader readOdt)
142− ,("t2t" , TextReader readTxt2Tags)
143− ,("epub" , ByteStringReader readEPUB)
144− ,("muse" , TextReader readMuse)
145− ,("man" , TextReader readMan)
146− ,("fb2" , TextReader readFB2)
147− ,("ipynb" , TextReader readIpynb)
148− ,("csv" , TextReader readCSV)
149− ,("csljson" , TextReader readCslJson)
150− ,("bibtex" , TextReader readBibTeX)
151− ,("biblatex" , TextReader readBibLaTeX)
111+readers = [ ("native" , TextReader readNative)
112+ ,("json" , TextReader readJSON)
113+ ,("markdown" , TextReader readMarkdown)
114+ ,("markdown_strict" , TextReader readMarkdown)
115+ ,("markdown_phpextra" , TextReader readMarkdown)
116+ ,("markdown_github" , TextReader readMarkdown)
117+ ,("markdown_mmd", TextReader readMarkdown)
118+ ,("commonmark" , TextReader readCommonMark)
119+ ,("commonmark_x" , TextReader readCommonMark)
120+ ,("creole" , TextReader readCreole)
121+ ,("dokuwiki" , TextReader readDokuWiki)
122+ ,("gfm" , TextReader readCommonMark)
123+ ,("rst" , TextReader readRST)
124+ ,("mediawiki" , TextReader readMediaWiki)
125+ ,("vimwiki" , TextReader readVimwiki)
126+ ,("docbook" , TextReader readDocBook)
127+ ,("opml" , TextReader readOPML)
128+ ,("org" , TextReader readOrg)
129+ ,("textile" , TextReader readTextile) -- TODO : textile+lhs
130+ ,("html" , TextReader readHtml)
131+ ,("jats" , TextReader readJATS)
132+ ,("jira" , TextReader readJira)
133+ ,("latex" , TextReader readLaTeX)
134+ ,("haddock" , TextReader readHaddock)
135+ ,("twiki" , TextReader readTWiki)
136+ ,("tikiwiki" , TextReader readTikiWiki)
137+ ,("docx" , ByteStringReader readDocx)
138+ ,("odt" , ByteStringReader readOdt)
139+ ,("t2t" , TextReader readTxt2Tags)
140+ ,("epub" , ByteStringReader readEPUB)
141+ ,("muse" , TextReader readMuse)
142+ ,("man" , TextReader readMan)
143+ ,("fb2" , TextReader readFB2)
144+ ,("ipynb" , TextReader readIpynb)
145+ ,("csv" , TextReader readCSV)
146+ ,("csljson" , TextReader readCslJson)
147+ ,("bibtex" , TextReader readBibTeX)
148+ ,("biblatex" , TextReader readBibLaTeX)
152149 ]
153150
154151-- | Retrieve reader, extensions based on formatSpec (format+extensions).
155152getReader :: PandocMonad m => Text -> m (Reader m, Extensions)
156153getReader s =
157154 case parseFormatSpec s of
158− Left e -> throwError $ PandocAppError $
159− "Error parsing reader format " <> tshow s <> ": " <> tshow e
155+ Left e -> throwError $ PandocAppError
156+ $ T.intercalate "\n" [T.pack m | Message m <- errorMessages e]
160157 Right (readerName, extsToEnable, extsToDisable) ->
161158 case lookup readerName readers of
162159 Nothing -> throwError $ PandocUnknownReaderError
@@ -176,13 +173,9 @@ getReader s =
176173 return (r, exts)
177174
178175-- | Read pandoc document from JSON format.
179−readJSON :: (PandocMonad m, ToSources a)
180− => ReaderOptions
181− -> a
182− -> m Pandoc
183−readJSON _ s =
184− case eitherDecode' . BL.fromStrict . UTF8.fromText
185− . sourcesToText . toSources $ s of
176+readJSON :: PandocMonad m
177+ => ReaderOptions -> Text -> m Pandoc
178+readJSON _ t =
179+ case eitherDecode' . BL.fromStrict . UTF8.fromText $ t of
186180 Right doc -> return doc
187− Left e -> throwError $ PandocParseError ("JSON parse error: "
188− <> T.pack e)
181+ Left e -> throwError $ PandocParseError ("JSON parse error: " <> T.pack e)
src/Text/Pandoc/Readers/BibTeX.hs+9−15
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Readers.BibTeX
4− Copyright : Copyright (C) 2020-2021 John MacFarlane
4+ Copyright : Copyright (C) 2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -23,47 +23,41 @@ where
2323import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
26+import Data.Text (Text)
2627import Citeproc (Lang(..), parseLang)
2728import Citeproc.Locale (getLocale)
2829import Text.Pandoc.Error (PandocError(..))
2930import Text.Pandoc.Class (PandocMonad, lookupEnv)
3031import Text.Pandoc.Citeproc.BibTeX as BibTeX
3132import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
32−import Text.Pandoc.Sources (ToSources(..))
3333import Control.Monad.Except (throwError)
3434
3535-- | Read BibTeX from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readBibTeX :: (PandocMonad m, ToSources a)
40− => ReaderOptions -> a -> m Pandoc
39+readBibTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
4140readBibTeX = readBibTeX' BibTeX.Bibtex
4241
4342-- | Read BibLaTeX from an input string and return a Pandoc document.
4443-- The document will have only metadata, with an empty body.
4544-- The metadata will contain a `references` field with the
4645-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
47−readBibLaTeX :: (PandocMonad m, ToSources a)
48− => ReaderOptions -> a -> m Pandoc
46+readBibLaTeX :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
4947readBibLaTeX = readBibTeX' BibTeX.Biblatex
5048
51−readBibTeX' :: (PandocMonad m, ToSources a)
52− => Variant -> ReaderOptions -> a -> m Pandoc
49+readBibTeX' :: PandocMonad m => Variant -> ReaderOptions -> Text -> m Pandoc
5350readBibTeX' variant _opts t = do
54− mblangEnv <- lookupEnv "LANG"
55− let defaultLang = Lang "en" Nothing (Just "US") [] [] []
56− let lang = case mblangEnv of
57− Nothing -> defaultLang
58− Just l -> either (const defaultLang) id $ parseLang l
51+ lang <- maybe (Lang "en" (Just "US")) parseLang
52+ <$> lookupEnv "LANG"
5953 locale <- case getLocale lang of
6054 Left e ->
61− case getLocale (Lang "en" Nothing (Just "US") [] [] []) of
55+ case getLocale (Lang "en" (Just "US")) of
6256 Right l -> return l
6357 Left _ -> throwError $ PandocCiteprocError e
6458 Right l -> return l
6559 case BibTeX.readBibtexString variant locale (const True) t of
66− Left e -> throwError $ PandocParsecError (toSources t) e
60+ Left e -> throwError $ PandocParsecError t e
6761 Right refs -> return $ setMeta "references"
6862 (map referenceToMetaValue refs)
6963 . setMeta "nocite"
src/Text/Pandoc/Readers/CSV.hs+9−9
@@ -2,8 +2,8 @@
22
33{-# LANGUAGE ScopedTypeVariables #-}
44{- |
5− Module : Text.Pandoc.Readers.CSV
6− Copyright : Copyright (C) 2006-2021 John MacFarlane
5+ Module : Text.Pandoc.Readers.RST
6+ Copyright : Copyright (C) 2006-2020 John MacFarlane
77 License : GNU GPL, version 2 or above
88
99 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -13,23 +13,23 @@
1313Conversion from CSV to a 'Pandoc' table.
1414-}
1515module Text.Pandoc.Readers.CSV ( readCSV ) where
16+import Data.Text (Text)
1617import qualified Data.Text as T
1718import Text.Pandoc.CSV (parseCSV, defaultCSVOptions)
1819import Text.Pandoc.Definition
1920import qualified Text.Pandoc.Builder as B
2021import Text.Pandoc.Class (PandocMonad)
22+import Text.Pandoc.Shared (crFilter)
2123import Text.Pandoc.Error
22−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
2324import Text.Pandoc.Options (ReaderOptions)
2425import Control.Monad.Except (throwError)
2526
26−readCSV :: (PandocMonad m, ToSources a)
27+readCSV :: PandocMonad m
2728 => ReaderOptions -- ^ Reader options
28− -> a
29+ -> Text -- ^ Text to parse (assuming @'\n'@ line endings)
2930 -> m Pandoc
30−readCSV _opts s = do
31− let txt = sourcesToText $ toSources s
32− case parseCSV defaultCSVOptions txt of
31+readCSV _opts s =
32+ case parseCSV defaultCSVOptions (crFilter s) of
3333 Right (r:rs) -> return $ B.doc $ B.table capt
3434 (zip aligns widths)
3535 (TableHead nullAttr hdrs)
@@ -45,4 +45,4 @@ readCSV _opts s = do
4545 aligns = replicate numcols AlignDefault
4646 widths = replicate numcols ColWidthDefault
4747 Right [] -> return $ B.doc mempty
48− Left e -> throwError $ PandocParsecError (toSources [("",txt)]) e
48+ Left e -> throwError $ PandocParsecError s e
src/Text/Pandoc/Readers/CommonMark.hs+9−61
@@ -1,9 +1,8 @@
11{-# LANGUAGE OverloadedStrings #-}
2−{-# LANGUAGE FlexibleContexts #-}
32{-# LANGUAGE ScopedTypeVariables #-}
43{- |
54 Module : Text.Pandoc.Readers.CommonMark
6− Copyright : Copyright (C) 2015-2021 John MacFarlane
5+ Copyright : Copyright (C) 2015-2020 John MacFarlane
76 License : GNU GPL, version 2 or above
87
98 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -26,66 +25,17 @@ import Text.Pandoc.Definition
2625import Text.Pandoc.Builder as B
2726import Text.Pandoc.Options
2827import Text.Pandoc.Error
29−import Text.Pandoc.Readers.Metadata (yamlMetaBlock)
3028import Control.Monad.Except
3129import Data.Functor.Identity (runIdentity)
32−import Data.Typeable
33−import Text.Pandoc.Parsing (runParserT, getInput,
34− runF, defaultParserState, option, many1, anyChar,
35− Sources(..), ToSources(..), ParserT, Future,
36− sourceName)
37−import qualified Data.Text as T
3830
3931-- | Parse a CommonMark formatted string into a 'Pandoc' structure.
40−readCommonMark :: (PandocMonad m, ToSources a)
41− => ReaderOptions -> a -> m Pandoc
42−readCommonMark opts s
43− | isEnabled Ext_yaml_metadata_block opts = do
44− let sources = toSources s
45− let toks = concatMap sourceToToks (unSources sources)
46− res <- runParserT (do meta <- yamlMetaBlock (metaValueParser opts)
47− rest <- getInput
48− return (meta, rest))
49− defaultParserState "YAML metadata" (toSources s)
50− case res of
51− Left _ -> readCommonMarkBody opts sources toks
52− Right (meta, rest) -> do
53− -- strip off metadata section and parse body
54− let body = concatMap sourceToToks (unSources rest)
55− Pandoc _ bs <- readCommonMarkBody opts sources body
56− return $ Pandoc (runF meta defaultParserState) bs
57− | otherwise = do
58− let sources = toSources s
59− let toks = concatMap sourceToToks (unSources sources)
60− readCommonMarkBody opts sources toks
61−
62−sourceToToks :: (SourcePos, Text) -> [Tok]
63−sourceToToks (pos, s) = tokenize (sourceName pos) s
64−
65−metaValueParser :: Monad m
66− => ReaderOptions -> ParserT Sources st m (Future st MetaValue)
67−metaValueParser opts = do
68− inp <- option "" $ T.pack <$> many1 anyChar
69− let toks = concatMap sourceToToks (unSources (toSources inp))
70− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
71− Left _ -> mzero
72− Right (Cm bls :: Cm () Blocks) -> return $ return $ B.toMetaValue bls
73−
74−readCommonMarkBody :: PandocMonad m => ReaderOptions -> Sources -> [Tok] -> m Pandoc
75−readCommonMarkBody opts s toks
76− | isEnabled Ext_sourcepos opts =
77− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
78− Left err -> throwError $ PandocParsecError s err
79− Right (Cm bls :: Cm SourceRange Blocks) -> return $ B.doc bls
80− | otherwise =
81− case runIdentity (parseCommonmarkWith (specFor opts) toks) of
82− Left err -> throwError $ PandocParsecError s err
83− Right (Cm bls :: Cm () Blocks) -> return $ B.doc bls
84−
85−specFor :: (Monad m, Typeable m, Typeable a,
86− Rangeable (Cm a Inlines), Rangeable (Cm a Blocks))
87− => ReaderOptions -> SyntaxSpec m (Cm a Inlines) (Cm a Blocks)
88−specFor opts = foldr ($) defaultSyntaxSpec exts
32+readCommonMark :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
33+readCommonMark opts s = do
34+ let res = runIdentity $
35+ commonmarkWith (foldr ($) defaultSyntaxSpec exts) "input" s
36+ case res of
37+ Left err -> throwError $ PandocParsecError s err
38+ Right (Cm bls :: Cm () Blocks) -> return $ B.doc bls
8939 where
9040 exts = [ (hardLineBreaksSpec <>) | isEnabled Ext_hard_line_breaks opts ] ++
9141 [ (smartPunctuationSpec <>) | isEnabled Ext_smart opts ] ++
@@ -112,7 +62,5 @@ specFor opts = foldr ($) defaultSyntaxSpec exts
11262 | isEnabled Ext_implicit_header_references opts ] ++
11363 [ (footnoteSpec <>) | isEnabled Ext_footnotes opts ] ++
11464 [ (definitionListSpec <>) | isEnabled Ext_definition_lists opts ] ++
115− [ (taskListSpec <>) | isEnabled Ext_task_lists opts ] ++
116− [ (rebaseRelativePathsSpec <>)
117− | isEnabled Ext_rebase_relative_paths opts ]
65+ [ (taskListSpec <>) | isEnabled Ext_task_lists opts ]
11866
src/Text/Pandoc/Readers/Creole.hs+6−5
@@ -23,20 +23,21 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad (..))
2323import Text.Pandoc.Definition
2424import Text.Pandoc.Options
2525import Text.Pandoc.Parsing hiding (enclosed)
26+import Text.Pandoc.Shared (crFilter)
27+
2628
2729-- | Read creole from an input string and return a Pandoc document.
28−readCreole :: (PandocMonad m, ToSources a)
30+readCreole :: PandocMonad m
2931 => ReaderOptions
30− -> a
32+ -> Text
3133 -> m Pandoc
3234readCreole opts s = do
33− let sources = ensureFinalNewlines 2 (toSources s)
34− res <- readWithM parseCreole def{ stateOptions = opts } sources
35+ res <- readWithM parseCreole def{ stateOptions = opts } $ crFilter s <> "\n\n"
3536 case res of
3637 Left e -> throwError e
3738 Right d -> return d
3839
39−type CRLParser = ParserT Sources ParserState
40+type CRLParser = ParserT Text ParserState
4041
4142--
4243-- Utility functions
src/Text/Pandoc/Readers/CslJson.hs+5−6
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Readers.CslJson
4− Copyright : Copyright (C) 2020-2021 John MacFarlane
4+ Copyright : Copyright (C) 2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -24,22 +24,21 @@ import Text.Pandoc.Options
2424import Text.Pandoc.Definition
2525import Text.Pandoc.Builder (setMeta, cite, str)
2626import qualified Text.Pandoc.UTF8 as UTF8
27+import Data.Text (Text)
2728import qualified Data.Text as T
2829import Text.Pandoc.Error (PandocError(..))
2930import Text.Pandoc.Class (PandocMonad)
3031import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences)
3132import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)
3233import Control.Monad.Except (throwError)
33−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3434
3535-- | Read CSL JSON from an input string and return a Pandoc document.
3636-- The document will have only metadata, with an empty body.
3737-- The metadata will contain a `references` field with the
3838-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.
39−readCslJson :: (PandocMonad m, ToSources a)
40− => ReaderOptions -> a -> m Pandoc
41−readCslJson _opts x =
42− case cslJsonToReferences (UTF8.fromText $ sourcesToText $ toSources x) of
39+readCslJson :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
40+readCslJson _opts t =
41+ case cslJsonToReferences (UTF8.fromText t) of
4342 Left e -> throwError $ PandocParseError $ T.pack e
4443 Right refs -> return $ setMeta "references"
4544 (map referenceToMetaValue refs)
src/Text/Pandoc/Readers/DocBook.hs+82−95
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Readers.DocBook
4− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -12,28 +12,23 @@ Conversion of DocBook XML to 'Pandoc' document.
1212-}
1313module Text.Pandoc.Readers.DocBook ( readDocBook ) where
1414import Control.Monad.State.Strict
15−import Data.Char (isSpace, isLetter)
15+import Data.Char (isSpace, toUpper)
1616import Data.Default
1717import Data.Either (rights)
1818import Data.Foldable (asum)
1919import Data.Generics
2020import Data.List (intersperse,elemIndex)
21−import Data.List.NonEmpty (nonEmpty)
2221import Data.Maybe (fromMaybe,mapMaybe)
2322import Data.Text (Text)
2423import qualified Data.Text as T
25−import qualified Data.Text.Lazy as TL
26−import Control.Monad.Except (throwError)
2724import Text.HTML.TagSoup.Entity (lookupEntity)
28−import Text.Pandoc.Error (PandocError(..))
2925import Text.Pandoc.Builder
3026import Text.Pandoc.Class.PandocMonad (PandocMonad, report)
3127import Text.Pandoc.Options
3228import Text.Pandoc.Logging (LogMessage(..))
33−import Text.Pandoc.Shared (safeRead, extractSpaces)
34−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
29+import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
3530import Text.TeXMath (readMathML, writeTeX)
36−import Text.Pandoc.XML.Light
31+import Text.XML.Light
3732
3833{-
3934
@@ -134,7 +129,6 @@ List of all DocBook tags, with [x] indicating implemented,
134129[ ] corpcredit - A corporation or organization credited in a document
135130[ ] corpname - The name of a corporation
136131[ ] country - The name of a country
137−[x] danger - An admonition set off from the text indicating hazardous situation
138132[ ] database - The name of a database, or part of a database
139133[x] date - The date of publication or revision of a document
140134[ ] dedication - A wrapper for the dedication section of a book
@@ -212,7 +206,7 @@ List of all DocBook tags, with [x] indicating implemented,
212206[x] info - A wrapper for information about a component or other block. (DocBook v5)
213207[x] informalequation - A displayed mathematical equation without a title
214208[x] informalexample - A displayed example without a title
215−[x] informalfigure - An untitled figure
209+[ ] informalfigure - A untitled figure
216210[ ] informaltable - A table without a title
217211[ ] initializer - The initializer for a FieldSynopsis
218212[x] inlineequation - A mathematical equation or expression occurring inline
@@ -541,32 +535,24 @@ instance Default DBState where
541535 , dbContent = [] }
542536
543537
544−readDocBook :: (PandocMonad m, ToSources a)
545− => ReaderOptions
546− -> a
547− -> m Pandoc
538+readDocBook :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
548539readDocBook _ inp = do
549− let sources = toSources inp
550− tree <- either (throwError . PandocXMLError "") return $
551− parseXMLContents
552− (TL.fromStrict . handleInstructions . sourcesToText $ sources)
540+ let tree = normalizeTree . parseXML . handleInstructions $ crFilter inp
553541 (bs, st') <- flip runStateT (def{ dbContent = tree }) $ mapM parseBlock tree
554542 return $ Pandoc (dbMeta st') (toList . mconcat $ bs)
555543
556−-- We treat certain processing instructions by converting them to tags
557−-- beginning "pi-".
544+-- We treat <?asciidoc-br?> specially (issue #1236), converting it
545+-- to <br/>, since xml-light doesn't parse the instruction correctly.
546+-- Other xml instructions are simply removed from the input stream.
558547handleInstructions :: Text -> Text
559−handleInstructions t =
560− let (x,y) = T.breakOn "<?" t
561− in if T.null y
562− then x
563− else
564− let (w,z) = T.breakOn "?>" y
565− in (if T.takeWhile (\c -> isLetter c || c == '-')
566− (T.drop 2 w) `elem` ["asciidoc-br", "dbfo"]
567− then x <> "<pi-" <> T.drop 2 w <> "/>"
568− else x <> w <> T.take 2 z) <>
569− handleInstructions (T.drop 2 z)
548+handleInstructions = T.pack . handleInstructions' . T.unpack
549+
550+handleInstructions' :: String -> String
551+handleInstructions' ('<':'?':'a':'s':'c':'i':'i':'d':'o':'c':'-':'b':'r':'?':'>':xs) = '<':'b':'r':'/':'>': handleInstructions' xs
552+handleInstructions' xs = case break (=='<') xs of
553+ (ys, []) -> ys
554+ ([], '<':zs) -> '<' : handleInstructions' zs
555+ (ys, zs) -> ys ++ handleInstructions' zs
570556
571557getFigure :: PandocMonad m => Element -> DB m Blocks
572558getFigure e = do
@@ -579,14 +565,32 @@ getFigure e = do
579565 modify $ \st -> st{ dbFigureTitle = mempty, dbFigureId = mempty }
580566 return res
581567
568+-- normalize input, consolidating adjacent Text and CRef elements
569+normalizeTree :: [Content] -> [Content]
570+normalizeTree = everywhere (mkT go)
571+ where go :: [Content] -> [Content]
572+ go (Text (CData CDataRaw _ _):xs) = xs
573+ go (Text (CData CDataText s1 z):Text (CData CDataText s2 _):xs) =
574+ Text (CData CDataText (s1 ++ s2) z):xs
575+ go (Text (CData CDataText s1 z):CRef r:xs) =
576+ Text (CData CDataText (s1 ++ convertEntity r) z):xs
577+ go (CRef r:Text (CData CDataText s1 z):xs) =
578+ Text (CData CDataText (convertEntity r ++ s1) z):xs
579+ go (CRef r1:CRef r2:xs) =
580+ Text (CData CDataText (convertEntity r1 ++ convertEntity r2) Nothing):xs
581+ go xs = xs
582+
583+convertEntity :: String -> String
584+convertEntity e = Data.Maybe.fromMaybe (map toUpper e) (lookupEntity e)
585+
582586-- convenience function to get an attribute value, defaulting to ""
583−attrValue :: Text -> Element -> Text
587+attrValue :: String -> Element -> Text
584588attrValue attr elt =
585− fromMaybe "" (lookupAttrBy (\x -> qName x == attr) (elAttribs elt))
589+ maybe "" T.pack (lookupAttrBy (\x -> qName x == attr) (elAttribs elt))
586590
587591-- convenience function
588592named :: Text -> Element -> Bool
589−named s e = qName (elName e) == s
593+named s e = qName (elName e) == T.unpack s
590594
591595--
592596
@@ -601,24 +605,16 @@ addMetadataFromElement e = do
601605 Nothing -> return ()
602606 Just z -> addMetaField "author" z
603607 addMetaField "subtitle" e
604− addAuthor e
608+ addMetaField "author" e
605609 addMetaField "date" e
606610 addMetaField "release" e
607611 addMetaField "releaseinfo" e
608612 return mempty
609− where
610− addAuthor elt =
611− case filterChildren (named "author") elt of
612− [] -> return ()
613− [z] -> fromAuthor z >>= addMeta "author"
614− zs -> mapM fromAuthor zs >>= addMeta "author"
615− fromAuthor elt =
616− mconcat . intersperse space <$> mapM getInlines (elChildren elt)
617− addMetaField fieldname elt =
618− case filterChildren (named fieldname) elt of
619− [] -> return ()
620− [z] -> getInlines z >>= addMeta fieldname
621− zs -> mapM getInlines zs >>= addMeta fieldname
613+ where addMetaField fieldname elt =
614+ case filterChildren (named fieldname) elt of
615+ [] -> return ()
616+ [z] -> getInlines z >>= addMeta fieldname
617+ zs -> mapM getInlines zs >>= addMeta fieldname
622618
623619addMeta :: PandocMonad m => ToMetaValue a => Text -> a -> DB m ()
624620addMeta field val = modify (setMeta field val)
@@ -631,7 +627,7 @@ isBlockElement :: Content -> Bool
631627isBlockElement (Elem e) = qName (elName e) `elem` blockTags
632628isBlockElement _ = False
633629
634−blockTags :: [Text]
630+blockTags :: [String]
635631blockTags =
636632 [ "abstract"
637633 , "ackno"
@@ -673,7 +669,6 @@ blockTags =
673669 , "index"
674670 , "info"
675671 , "informalexample"
676− , "informalfigure"
677672 , "informaltable"
678673 , "itemizedlist"
679674 , "linegroup"
@@ -718,8 +713,8 @@ blockTags =
718713 , "variablelist"
719714 ] ++ admonitionTags
720715
721−admonitionTags :: [Text]
722−admonitionTags = ["caution","danger","important","note","tip","warning"]
716+admonitionTags :: [String]
717+admonitionTags = ["important","caution","note","tip","warning"]
723718
724719-- Trim leading and trailing newline characters
725720trimNl :: Text -> Text
@@ -776,10 +771,10 @@ getBlocks e = mconcat <$>
776771
777772parseBlock :: PandocMonad m => Content -> DB m Blocks
778773parseBlock (Text (CData CDataRaw _ _)) = return mempty -- DOCTYPE
779−parseBlock (Text (CData _ s _)) = if T.all isSpace s
774+parseBlock (Text (CData _ s _)) = if all isSpace s
780775 then return mempty
781− else return $ plain $ trimInlines $ text s
782−parseBlock (CRef x) = return $ plain $ str $ T.toUpper x
776+ else return $ plain $ trimInlines $ text $ T.pack s
777+parseBlock (CRef x) = return $ plain $ str $ T.toUpper $ T.pack x
783778parseBlock (Elem e) =
784779 case qName (elName e) of
785780 "toc" -> skip -- skip TOC, since in pandoc it's autogenerated
@@ -834,7 +829,7 @@ parseBlock (Elem e) =
834829 "refsect2" -> sect 2
835830 "refsect3" -> sect 3
836831 "refsection" -> gets dbSectionLevel >>= sect . (+1)
837− l | l `elem` admonitionTags -> parseAdmonition l
832+ l | l `elem` admonitionTags -> parseAdmonition $ T.pack l
838833 "area" -> skip
839834 "areaset" -> skip
840835 "areaspec" -> skip
@@ -860,7 +855,6 @@ parseBlock (Elem e) =
860855 "variablelist" -> definitionList <$> deflistitems
861856 "procedure" -> bulletList <$> steps
862857 "figure" -> getFigure e
863− "informalfigure" -> getFigure e
864858 "mediaobject" -> para <$> getMediaobject e
865859 "caption" -> skip
866860 "info" -> addMetadataFromElement e
@@ -896,11 +890,7 @@ parseBlock (Elem e) =
896890 "subtitle" -> return mempty -- handled in parent element
897891 _ -> skip >> getBlocks e
898892 where skip = do
899− let qn = qName $ elName e
900− let name = if "pi-" `T.isPrefixOf` qn
901− then "<?" <> qn <> "?>"
902− else qn
903− lift $ report $ IgnoredElement name
893+ lift $ report $ IgnoredElement $ T.pack $ qName (elName e)
904894 return mempty
905895
906896 codeBlockWithLang = do
@@ -908,7 +898,7 @@ parseBlock (Elem e) =
908898 "" -> []
909899 x -> [x]
910900 return $ codeBlockWith (attrValue "id" e, classes', [])
911− $ trimNl $ strContentRecursive e
901+ $ trimNl $ T.pack $ strContentRecursive e
912902 parseBlockquote = do
913903 attrib <- case filterChild (named "attribution") e of
914904 Nothing -> return mempty
@@ -962,16 +952,17 @@ parseBlock (Elem e) =
962952 w <- findAttr (unqual "colwidth") c
963953 n <- safeRead $ "0" <> T.filter (\x ->
964954 (x >= '0' && x <= '9')
965− || x == '.') w
955+ || x == '.') (T.pack w)
966956 if n > 0 then Just n else Nothing
967− let numrows = maybe 0 maximum $ nonEmpty
968− $ map length bodyrows
957+ let numrows = case bodyrows of
958+ [] -> 0
959+ xs -> maximum $ map length xs
969960 let aligns = case colspecs of
970961 [] -> replicate numrows AlignDefault
971962 cs -> map toAlignment cs
972963 let parseWidth s = safeRead (T.filter (\x -> (x >= '0' && x <= '9')
973964 || x == '.') s)
974− let textWidth = case filterChild (named "pi-dbfo") e of
965+ let textWidth = case filterChild (named "?dbfo") e of
975966 Just d -> case attrValue "table-width" d of
976967 "" -> 1.0
977968 w -> fromMaybe 100.0 (parseWidth w) / 100.0
@@ -1044,12 +1035,12 @@ parseMixed container conts = do
10441035 x <- parseMixed container rs
10451036 return $ p <> b <> x
10461037
1047−parseRow :: PandocMonad m => [Text] -> Element -> DB m [Cell]
1038+parseRow :: PandocMonad m => [String] -> Element -> DB m [Cell]
10481039parseRow cn = do
10491040 let isEntry x = named "entry" x || named "td" x || named "th" x
10501041 mapM (parseEntry cn) . filterChildren isEntry
10511042
1052−parseEntry :: PandocMonad m => [Text] -> Element -> DB m Cell
1043+parseEntry :: PandocMonad m => [String] -> Element -> DB m Cell
10531044parseEntry cn el = do
10541045 let colDistance sa ea = do
10551046 let iStrt = elemIndex sa cn
@@ -1071,7 +1062,7 @@ getInlines :: PandocMonad m => Element -> DB m Inlines
10711062getInlines e' = trimInlines . mconcat <$>
10721063 mapM parseInline (elContent e')
10731064
1074−strContentRecursive :: Element -> Text
1065+strContentRecursive :: Element -> String
10751066strContentRecursive = strContent .
10761067 (\e' -> e'{ elContent = map elementToStr $ elContent e' })
10771068
@@ -1080,16 +1071,16 @@ elementToStr (Elem e') = Text $ CData CDataText (strContentRecursive e') Nothing
10801071elementToStr x = x
10811072
10821073parseInline :: PandocMonad m => Content -> DB m Inlines
1083−parseInline (Text (CData _ s _)) = return $ text s
1074+parseInline (Text (CData _ s _)) = return $ text $ T.pack s
10841075parseInline (CRef ref) =
1085− return $ text $ maybe (T.toUpper ref) T.pack $ lookupEntity (T.unpack ref)
1076+ return $ text $ maybe (T.toUpper $ T.pack ref) T.pack $ lookupEntity ref
10861077parseInline (Elem e) =
10871078 case qName (elName e) of
10881079 "anchor" -> do
10891080 return $ spanWith (attrValue "id" e, [], []) mempty
10901081 "phrase" -> do
10911082 let ident = attrValue "id" e
1092− let classes = T.words $ attrValue "role" e
1083+ let classes = T.words $ attrValue "class" e
10931084 if ident /= "" || classes /= []
10941085 then innerInlines (spanWith (ident,classes,[]))
10951086 else innerInlines id
@@ -1134,7 +1125,7 @@ parseInline (Elem e) =
11341125 "userinput" -> codeWithLang
11351126 "systemitem" -> codeWithLang
11361127 "varargs" -> return $ code "(...)"
1137− "keycap" -> return (str $ strContent e)
1128+ "keycap" -> return (str $ T.pack $ strContent e)
11381129 "keycombo" -> keycombo <$>
11391130 mapM parseInline (elContent e)
11401131 "menuchoice" -> menuchoice <$>
@@ -1146,17 +1137,17 @@ parseInline (Elem e) =
11461137 let title = case attrValue "endterm" e of
11471138 "" -> maybe "???" xrefTitleByElem
11481139 (findElementById linkend content)
1149− endterm -> maybe "???" strContent
1140+ endterm -> maybe "???" (T.pack . strContent)
11501141 (findElementById endterm content)
11511142 return $ link ("#" <> linkend) "" (text title)
1152− "email" -> return $ link ("mailto:" <> strContent e) ""
1153− $ str $ strContent e
1154− "uri" -> return $ link (strContent e) "" $ str $ strContent e
1143+ "email" -> return $ link ("mailto:" <> T.pack (strContent e)) ""
1144+ $ str $ T.pack $ strContent e
1145+ "uri" -> return $ link (T.pack $ strContent e) "" $ str $ T.pack $ strContent e
11551146 "ulink" -> innerInlines (link (attrValue "url" e) "")
11561147 "link" -> do
11571148 ils <- innerInlines id
11581149 let href = case findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e of
1159− Just h -> h
1150+ Just h -> T.pack h
11601151 _ -> "#" <> attrValue "linkend" e
11611152 let ils' = if ils == mempty then str href else ils
11621153 let attr = (attrValue "id" e, T.words $ attrValue "role" e, [])
@@ -1172,15 +1163,12 @@ parseInline (Elem e) =
11721163 "title" -> return mempty
11731164 "affiliation" -> skip
11741165 -- Note: this isn't a real docbook tag; it's what we convert
1175− -- <?asciidor-br?> to in handleInstructions, above.
1176− "pi-asciidoc-br" -> return linebreak
1166+ -- <?asciidor-br?> to in handleInstructions, above. A kludge to
1167+ -- work around xml-light's inability to parse an instruction.
1168+ "br" -> return linebreak
11771169 _ -> skip >> innerInlines id
11781170 where skip = do
1179− let qn = qName $ elName e
1180− let name = if "pi-" `T.isPrefixOf` qn
1181− then "<?" <> qn <> "?>"
1182− else qn
1183− lift $ report $ IgnoredElement name
1171+ lift $ report $ IgnoredElement $ T.pack $ qName (elName e)
11841172 return mempty
11851173
11861174 innerInlines f = extractSpaces f . mconcat <$>
@@ -1189,7 +1177,7 @@ parseInline (Elem e) =
11891177 let classes' = case attrValue "language" e of
11901178 "" -> []
11911179 l -> [l]
1192− return $ codeWith (attrValue "id" e,classes',[]) $ strContentRecursive e
1180+ return $ codeWith (attrValue "id" e,classes',[]) $ T.pack $ strContentRecursive e
11931181 simpleList = mconcat . intersperse (str "," <> space) <$> mapM getInlines
11941182 (filterChildren (named "member") e)
11951183 segmentedList = do
@@ -1230,10 +1218,10 @@ parseInline (Elem e) =
12301218 "sect5" -> descendantContent "title" el
12311219 "cmdsynopsis" -> descendantContent "command" el
12321220 "funcsynopsis" -> descendantContent "function" el
1233− _ -> qName (elName el) <> "_title"
1221+ _ -> T.pack $ qName (elName el) ++ "_title"
12341222 where
12351223 xrefLabel = attrValue "xreflabel" el
1236− descendantContent name = maybe "???" strContent
1224+ descendantContent name = maybe "???" (T.pack . strContent)
12371225 . filterElementName (\n -> qName n == name)
12381226
12391227-- | Extract a math equation from an element
@@ -1253,9 +1241,8 @@ equation e constructor =
12531241 where
12541242 mathMLEquations :: [Text]
12551243 mathMLEquations = map writeTeX $ rights $ readMath
1256− (\x -> qName (elName x) == "math" &&
1257− qURI (elName x) == Just "http://www.w3.org/1998/Math/MathML")
1258− (readMathML . showElement)
1244+ (\x -> qName (elName x) == "math" && qPrefix (elName x) == Just "mml")
1245+ (readMathML . T.pack . showElement)
12591246
12601247 latexEquations :: [Text]
12611248 latexEquations = readMath (\x -> qName (elName x) == "mathphrase")
@@ -1269,8 +1256,8 @@ equation e constructor =
12691256-- | Get the actual text stored in a CData block. 'showContent'
12701257-- returns the text still surrounded by the [[CDATA]] tags.
12711258showVerbatimCData :: Content -> Text
1272−showVerbatimCData (Text (CData _ d _)) = d
1273−showVerbatimCData c = showContent c
1259+showVerbatimCData (Text (CData _ d _)) = T.pack d
1260+showVerbatimCData c = T.pack $ showContent c
12741261
12751262
12761263-- | Set the prefix of a name to 'Nothing'
src/Text/Pandoc/Readers/Docx.hs+42−59
@@ -61,14 +61,13 @@ module Text.Pandoc.Readers.Docx
6161import Codec.Archive.Zip
6262import Control.Monad.Reader
6363import Control.Monad.State.Strict
64−import Data.Bifunctor (bimap, first)
6564import qualified Data.ByteString.Lazy as B
6665import Data.Default (Default)
67−import Data.List (delete, intersect, foldl')
66+import Data.List (delete, intersect)
6867import Data.Char (isSpace)
6968import qualified Data.Map as M
7069import qualified Data.Text as T
71−import Data.Maybe (catMaybes, isJust, fromMaybe)
70+import Data.Maybe (isJust, fromMaybe)
7271import Data.Sequence (ViewL (..), viewl)
7372import qualified Data.Sequence as Seq
7473import qualified Data.Set as Set
@@ -86,7 +85,6 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
8685import qualified Text.Pandoc.Class.PandocMonad as P
8786import Text.Pandoc.Error
8887import Text.Pandoc.Logging
89−import Data.List.NonEmpty (nonEmpty)
9088
9189readDocx :: PandocMonad m
9290 => ReaderOptions
@@ -114,7 +112,6 @@ data DState = DState { docxAnchorMap :: M.Map T.Text T.Text
114112 -- restarting
115113 , docxListState :: M.Map (T.Text, T.Text) Integer
116114 , docxPrevPara :: Inlines
117− , docxTableCaptions :: [Blocks]
118115 }
119116
120117instance Default DState where
@@ -125,7 +122,6 @@ instance Default DState where
125122 , docxDropCap = mempty
126123 , docxListState = M.empty
127124 , docxPrevPara = mempty
128− , docxTableCaptions = []
129125 }
130126
131127data DEnv = DEnv { docxOptions :: ReaderOptions
@@ -494,32 +490,15 @@ singleParaToPlain blks
494490 singleton $ Plain ils
495491singleParaToPlain blks = blks
496492
497−cellToCell :: PandocMonad m => RowSpan -> Docx.Cell -> DocxContext m Pandoc.Cell
498−cellToCell rowSpan (Docx.Cell gridSpan _ bps) = do
493+cellToBlocks :: PandocMonad m => Docx.Cell -> DocxContext m Blocks
494+cellToBlocks (Docx.Cell bps) = do
499495 blks <- smushBlocks <$> mapM bodyPartToBlocks bps
500− let blks' = singleParaToPlain $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
501− return (cell AlignDefault rowSpan (ColSpan (fromIntegral gridSpan)) blks')
502−
503−rowsToRows :: PandocMonad m => [Docx.Row] -> DocxContext m [Pandoc.Row]
504−rowsToRows rows = do
505− let rowspans = (fmap . fmap) (first RowSpan) (Docx.rowsToRowspans rows)
506− cells <- traverse (traverse (uncurry cellToCell)) rowspans
507− return (fmap (Pandoc.Row nullAttr) cells)
508−
509−splitHeaderRows :: Bool -> [Docx.Row] -> ([Docx.Row], [Docx.Row])
510−splitHeaderRows hasFirstRowFormatting rs = bimap reverse reverse $ fst
511− $ if hasFirstRowFormatting
512− then foldl' f ((take 1 rs, []), True) (drop 1 rs)
513− else foldl' f (([], []), False) rs
514− where
515− f ((headerRows, bodyRows), previousRowWasHeader) r@(Docx.Row h cs)
516− | h == HasTblHeader || (previousRowWasHeader && any isContinuationCell cs)
517− = ((r : headerRows, bodyRows), True)
518− | otherwise
519− = ((headerRows, r : bodyRows), False)
520−
521− isContinuationCell (Docx.Cell _ vm _) = vm == Docx.Continue
496+ return $ fromList $ blocksToDefinitions $ blocksToBullets $ toList blks
522497
498+rowToBlocksList :: PandocMonad m => Docx.Row -> DocxContext m [Blocks]
499+rowToBlocksList (Docx.Row cells) = do
500+ blksList <- mapM cellToBlocks cells
501+ return $ map singleParaToPlain blksList
523502
524503-- like trimInlines, but also take out linebreaks
525504trimSps :: Inlines -> Inlines
@@ -566,11 +545,6 @@ normalizeToClassName = T.map go . fromStyleName
566545 where go c | isSpace c = '-'
567546 | otherwise = c
568547
569−bodyPartToTableCaption :: PandocMonad m => BodyPart -> DocxContext m (Maybe Blocks)
570−bodyPartToTableCaption (TblCaption pPr parparts) =
571− Just <$> bodyPartToBlocks (Paragraph pPr parparts)
572−bodyPartToTableCaption _ = pure Nothing
573−
574548bodyPartToBlocks :: PandocMonad m => BodyPart -> DocxContext m Blocks
575549bodyPartToBlocks (Paragraph pPr parparts)
576550 | Just True <- pBidi pPr = do
@@ -662,43 +636,54 @@ bodyPartToBlocks (ListItem pPr _ _ _ parparts) =
662636 let pPr' = pPr {pStyle = constructBogusParStyleData "list-paragraph": pStyle pPr}
663637 in
664638 bodyPartToBlocks $ Paragraph pPr' parparts
665−bodyPartToBlocks (TblCaption _ _) =
666− return $ para mempty -- collected separately
667639bodyPartToBlocks (Tbl _ _ _ []) =
668640 return $ para mempty
669−bodyPartToBlocks (Tbl cap grid look parts) = do
670− captions <- gets docxTableCaptions
671− fullCaption <- case captions of
672− c : cs -> do
673− modify (\s -> s { docxTableCaptions = cs })
674− return c
675− [] -> return $ if T.null cap then mempty else plain (text cap)
676− let shortCaption = if T.null cap then Nothing else Just (toList (text cap))
677− cap' = caption shortCaption fullCaption
678− (hdr, rows) = splitHeaderRows (firstRowFormatting look) parts
641+bodyPartToBlocks (Tbl cap _ look parts@(r:rs)) = do
642+ let cap' = simpleCaption $ plain $ text cap
643+ (hdr, rows) = case firstRowFormatting look of
644+ True | null rs -> (Nothing, [r])
645+ | otherwise -> (Just r, rs)
646+ False -> (Nothing, r:rs)
647+
648+ cells <- mapM rowToBlocksList rows
679649
680650 let width = maybe 0 maximum $ nonEmpty $ map rowLength parts
651+ -- Data.List.NonEmpty is not available with ghc 7.10 so we roll out
652+ -- our own, see
653+ -- https://github.com/jgm/pandoc/pull/4361#issuecomment-365416155
654+ nonEmpty [] = Nothing
655+ nonEmpty l = Just l
681656 rowLength :: Docx.Row -> Int
682− rowLength (Docx.Row _ c) = sum (fmap (\(Docx.Cell gridSpan _ _) -> fromIntegral gridSpan) c)
657+ rowLength (Docx.Row c) = length c
683658
684− headerCells <- rowsToRows hdr
685− bodyCells <- rowsToRows rows
659+ let toRow = Pandoc.Row nullAttr . map simpleCell
660+ toHeaderRow l = [toRow l | not (null l)]
686661
687− -- Horizontal column alignment goes to the default at the moment. Getting
688− -- it might be difficult, since there doesn't seem to be a column entity
689− -- in docx.
662+ -- pad cells. New Text.Pandoc.Builder will do that for us,
663+ -- so this is for compatibility while we switch over.
664+ let cells' = map (\row -> toRow $ take width (row ++ repeat mempty)) cells
665+
666+ hdrCells <- case hdr of
667+ Just r' -> toHeaderRow <$> rowToBlocksList r'
668+ Nothing -> return []
669+
670+ -- The two following variables (horizontal column alignment and
671+ -- relative column widths) go to the default at the
672+ -- moment. Width information is in the TblGrid field of the Tbl,
673+ -- so should be possible. Alignment might be more difficult,
674+ -- since there doesn't seem to be a column entity in docx.
690675 let alignments = replicate width AlignDefault
691− totalWidth = sum grid
692− widths = (\w -> ColWidth (fromInteger w / fromInteger totalWidth)) <$> grid
676+ widths = replicate width ColWidthDefault
693677
694678 return $ table cap'
695679 (zip alignments widths)
696− (TableHead nullAttr headerCells)
697− [TableBody nullAttr 0 [] bodyCells]
680+ (TableHead nullAttr hdrCells)
681+ [TableBody nullAttr 0 [] cells']
698682 (TableFoot nullAttr [])
699683bodyPartToBlocks (OMathPara e) =
700684 return $ para $ displayMath (writeTeX e)
701685
686+
702687-- replace targets with generated anchors.
703688rewriteLink' :: PandocMonad m => Inline -> DocxContext m Inline
704689rewriteLink' l@(Link attr ils (T.uncons -> Just ('#',target), title)) = do
@@ -734,8 +719,6 @@ bodyToOutput :: PandocMonad m => Body -> DocxContext m (Meta, [Block])
734719bodyToOutput (Body bps) = do
735720 let (metabps, blkbps) = sepBodyParts bps
736721 meta <- bodyPartsToMeta metabps
737− captions <- catMaybes <$> mapM bodyPartToTableCaption blkbps
738− modify (\s -> s { docxTableCaptions = captions })
739722 blks <- smushBlocks <$> mapM bodyPartToBlocks blkbps
740723 blks' <- rewriteLinks $ blocksToDefinitions $ blocksToBullets $ toList blks
741724 blks'' <- removeOrphanAnchors blks'
src/Text/Pandoc/Readers/Docx/Combine.hs+6−6
@@ -2,7 +2,7 @@
22{- |
33 Module : Text.Pandoc.Readers.Docx.Combine
44 Copyright : © 2014-2020 Jesse Rosenthal <jrosenthal@jhu.edu>,
5− 2014-2021 John MacFarlane <jgm@berkeley.edu>,
5+ 2014-2020 John MacFarlane <jgm@berkeley.edu>,
66 2020 Nikolay Yakimov <root@livid.pp.ru>
77 License : GNU GPL, version 2 or above
88
@@ -61,7 +61,7 @@ import Data.List
6161import Data.Bifunctor
6262import Data.Sequence ( ViewL (..), ViewR (..), viewl, viewr, spanr, spanl
6363 , (><), (|>) )
64−import Text.Pandoc.Builder as B
64+import Text.Pandoc.Builder
6565
6666data Modifier a = Modifier (a -> a)
6767 | AttrModifier (Attr -> a -> a) Attr
@@ -116,12 +116,12 @@ ilModifierAndInnards ils = case viewl $ unMany ils of
116116
117117inlinesL :: Inlines -> (Inlines, Inlines)
118118inlinesL ils = case viewl $ unMany ils of
119− (s :< sq) -> (B.singleton s, Many sq)
119+ (s :< sq) -> (singleton s, Many sq)
120120 _ -> (mempty, ils)
121121
122122inlinesR :: Inlines -> (Inlines, Inlines)
123123inlinesR ils = case viewr $ unMany ils of
124− (sq :> s) -> (Many sq, B.singleton s)
124+ (sq :> s) -> (Many sq, singleton s)
125125 _ -> (ils, mempty)
126126
127127combineInlines :: Inlines -> Inlines -> Inlines
@@ -182,7 +182,7 @@ isAttrModifier _ = False
182182
183183smushInlines :: [Inlines] -> Inlines
184184smushInlines xs = combineInlines xs' mempty
185− where xs' = foldl' combineInlines mempty xs
185+ where xs' = foldl combineInlines mempty xs
186186
187187smushBlocks :: [Blocks] -> Blocks
188−smushBlocks xs = foldl' combineBlocks mempty xs
188+smushBlocks xs = foldl combineBlocks mempty xs
src/Text/Pandoc/Readers/Docx/Parse.hs+120−214
@@ -33,9 +33,7 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
3333 , ParStyle
3434 , CharStyle(cStyleData)
3535 , Row(..)
36− , TblHeader(..)
3736 , Cell(..)
38− , VMerge(..)
3937 , TrackedChange(..)
4038 , ChangeType(..)
4139 , ChangeInfo(..)
@@ -52,7 +50,6 @@ module Text.Pandoc.Readers.Docx.Parse ( Docx(..)
5250 , pHeading
5351 , constructBogusParStyleData
5452 , leftBiasedMergeRunStyle
55− , rowsToRowspans
5653 ) where
5754import Text.Pandoc.Readers.Docx.Parse.Styles
5855import Codec.Archive.Zip
@@ -66,7 +63,6 @@ import Data.Char (chr, ord, readLitChar)
6663import Data.List
6764import qualified Data.Map as M
6865import qualified Data.Text as T
69−import Data.Text (Text)
7066import Data.Maybe
7167import System.FilePath
7268import Text.Pandoc.Readers.Docx.Util
@@ -76,7 +72,8 @@ import qualified Text.Pandoc.UTF8 as UTF8
7672import Text.TeXMath (Exp)
7773import Text.TeXMath.Readers.OMML (readOMML)
7874import Text.TeXMath.Unicode.Fonts (Font (..), getUnicode, textToFont)
79−import Text.Pandoc.XML.Light
75+import Text.XML.Light
76+import qualified Text.XML.Light.Cursor as XMLC
8077
8178data ReaderEnv = ReaderEnv { envNotes :: Notes
8279 , envComments :: Comments
@@ -130,23 +127,37 @@ mapD f xs =
130127 in
131128 concatMapM handler xs
132129
133−unwrapElement :: NameSpaces -> Element -> [Element]
134−unwrapElement ns element
130+unwrap :: NameSpaces -> Content -> [Content]
131+unwrap ns (Elem element)
135132 | isElem ns "w" "sdt" element
136133 , Just sdtContent <- findChildByName ns "w" "sdtContent" element
137− = concatMap (unwrapElement ns) (elChildren sdtContent)
134+ = concatMap (unwrap ns . Elem) (elChildren sdtContent)
138135 | isElem ns "w" "smartTag" element
139− = concatMap (unwrapElement ns) (elChildren element)
140− | otherwise
141− = [element{ elContent = concatMap (unwrapContent ns) (elContent element) }]
136+ = concatMap (unwrap ns . Elem) (elChildren element)
137+unwrap _ content = [content]
142138
143−unwrapContent :: NameSpaces -> Content -> [Content]
144−unwrapContent ns (Elem element) = map Elem $ unwrapElement ns element
145−unwrapContent _ content = [content]
139+unwrapChild :: NameSpaces -> Content -> Content
140+unwrapChild ns (Elem element) =
141+ Elem $ element { elContent = concatMap (unwrap ns) (elContent element) }
142+unwrapChild _ content = content
146143
147−walkDocument :: NameSpaces -> Element -> Element
144+walkDocument' :: NameSpaces -> XMLC.Cursor -> XMLC.Cursor
145+walkDocument' ns cur =
146+ let modifiedCur = XMLC.modifyContent (unwrapChild ns) cur
147+ in
148+ case XMLC.nextDF modifiedCur of
149+ Just cur' -> walkDocument' ns cur'
150+ Nothing -> XMLC.root modifiedCur
151+
152+walkDocument :: NameSpaces -> Element -> Maybe Element
148153walkDocument ns element =
149− element{ elContent = concatMap (unwrapContent ns) (elContent element) }
154+ let cur = XMLC.fromContent (Elem element)
155+ cur' = walkDocument' ns cur
156+ in
157+ case XMLC.toTree cur' of
158+ Elem element' -> Just element'
159+ _ -> Nothing
160+
150161
151162newtype Docx = Docx Document
152163 deriving Show
@@ -228,7 +239,6 @@ defaultParagraphStyle = ParagraphStyle { pStyle = []
228239data BodyPart = Paragraph ParagraphStyle [ParPart]
229240 | ListItem ParagraphStyle T.Text T.Text (Maybe Level) [ParPart]
230241 | Tbl T.Text TblGrid TblLook [Row]
231− | TblCaption ParagraphStyle [ParPart]
232242 | OMathPara [Exp]
233243 deriving Show
234244
@@ -240,61 +250,12 @@ newtype TblLook = TblLook {firstRowFormatting::Bool}
240250defaultTblLook :: TblLook
241251defaultTblLook = TblLook{firstRowFormatting = False}
242252
243−data Row = Row TblHeader [Cell] deriving Show
244−
245−data TblHeader = HasTblHeader | NoTblHeader deriving (Show, Eq)
253+newtype Row = Row [Cell]
254+ deriving Show
246255
247−data Cell = Cell GridSpan VMerge [BodyPart]
256+newtype Cell = Cell [BodyPart]
248257 deriving Show
249258
250−type GridSpan = Integer
251−
252−data VMerge = Continue
253− -- ^ This cell should be merged with the one above it
254− | Restart
255− -- ^ This cell should not be merged with the one above it
256− deriving (Show, Eq)
257−
258−rowsToRowspans :: [Row] -> [[(Int, Cell)]]
259−rowsToRowspans rows = let
260− removeMergedCells = fmap (filter (\(_, Cell _ vmerge _) -> vmerge == Restart))
261− in removeMergedCells (foldr f [] rows)
262− where
263− f :: Row -> [[(Int, Cell)]] -> [[(Int, Cell)]]
264− f (Row _ cells) acc = let
265− spans = g cells Nothing (listToMaybe acc)
266− in spans : acc
267−
268− g ::
269− -- | The current row
270− [Cell] ->
271− -- | Number of columns left below
272− Maybe Integer ->
273− -- | (rowspan so far, cell) for the row below this one
274− Maybe [(Int, Cell)] ->
275− -- | (rowspan so far, cell) for this row
276− [(Int, Cell)]
277− g cells _ Nothing = zip (repeat 1) cells
278− g cells columnsLeftBelow (Just rowBelow) =
279− case cells of
280− [] -> []
281− thisCell@(Cell thisGridSpan _ _) : restOfRow -> case rowBelow of
282− [] -> zip (repeat 1) cells
283− (spanSoFarBelow, Cell gridSpanBelow vmerge _) : _ ->
284− let spanSoFar = case vmerge of
285− Restart -> 1
286− Continue -> 1 + spanSoFarBelow
287− columnsToDrop = thisGridSpan + (gridSpanBelow - fromMaybe gridSpanBelow columnsLeftBelow)
288− (newColumnsLeftBelow, restOfRowBelow) = dropColumns columnsToDrop rowBelow
289− in (spanSoFar, thisCell) : g restOfRow (Just newColumnsLeftBelow) (Just restOfRowBelow)
290−
291− dropColumns :: Integer -> [(a, Cell)] -> (Integer, [(a, Cell)])
292− dropColumns n [] = (n, [])
293− dropColumns n cells@((_, Cell gridSpan _ _) : otherCells) =
294− if n < gridSpan
295− then (gridSpan - n, cells)
296− else dropColumns (n - gridSpan) otherCells
297−
298259leftBiasedMergeRunStyle :: RunStyle -> RunStyle -> RunStyle
299260leftBiasedMergeRunStyle a b = RunStyle
300261 { isBold = isBold a <|> isBold b
@@ -382,16 +343,10 @@ archiveToDocxWithWarnings archive = do
382343 Right doc -> Right (Docx doc, stateWarnings st)
383344 Left e -> Left e
384345
385−parseXMLFromEntry :: Entry -> Maybe Element
386−parseXMLFromEntry entry =
387− case parseXMLElement (UTF8.toTextLazy (fromEntry entry)) of
388− Left _ -> Nothing
389− Right el -> Just el
390−
391346getDocumentXmlPath :: Archive -> Maybe FilePath
392347getDocumentXmlPath zf = do
393348 entry <- findEntryByPath "_rels/.rels" zf
394− relsElem <- parseXMLFromEntry entry
349+ relsElem <- (parseXMLDoc . UTF8.toStringLazy . fromEntry) entry
395350 let rels = filterChildrenName (\n -> qName n == "Relationship") relsElem
396351 rel <- find (\e -> findAttr (QName "Type" Nothing Nothing) e ==
397352 Just "http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument")
@@ -399,18 +354,18 @@ getDocumentXmlPath zf = do
399354 fp <- findAttr (QName "Target" Nothing Nothing) rel
400355 -- sometimes there will be a leading slash, which windows seems to
401356 -- have trouble with.
402− return $ case T.unpack fp of
357+ return $ case fp of
403358 '/' : fp' -> fp'
404− fp' -> fp'
359+ _ -> fp
405360
406361archiveToDocument :: Archive -> D Document
407362archiveToDocument zf = do
408363 docPath <- asks envDocXmlPath
409364 entry <- maybeToD $ findEntryByPath docPath zf
410− docElem <- maybeToD $ parseXMLFromEntry entry
365+ docElem <- maybeToD $ (parseXMLDoc . UTF8.toStringLazy . fromEntry) entry
411366 let namespaces = elemToNameSpaces docElem
412367 bodyElem <- maybeToD $ findChildByName namespaces "w" "body" docElem
413− let bodyElem' = walkDocument namespaces bodyElem
368+ let bodyElem' = fromMaybe bodyElem (walkDocument namespaces bodyElem)
414369 body <- elemToBody namespaces bodyElem'
415370 return $ Document namespaces body
416371
@@ -446,24 +401,23 @@ constructBogusParStyleData stName = ParStyle
446401archiveToNotes :: Archive -> Notes
447402archiveToNotes zf =
448403 let fnElem = findEntryByPath "word/footnotes.xml" zf
449− >>= parseXMLFromEntry
404+ >>= (parseXMLDoc . UTF8.toStringLazy . fromEntry)
450405 enElem = findEntryByPath "word/endnotes.xml" zf
451− >>= parseXMLFromEntry
452− fn_namespaces = maybe mempty elemToNameSpaces fnElem
453− en_namespaces = maybe mempty elemToNameSpaces enElem
454− ns = M.union fn_namespaces en_namespaces
455− fn = fnElem >>= elemToNotes ns "footnote" . walkDocument ns
456− en = enElem >>= elemToNotes ns "endnote" . walkDocument ns
406+ >>= (parseXMLDoc . UTF8.toStringLazy . fromEntry)
407+ fn_namespaces = maybe [] elemToNameSpaces fnElem
408+ en_namespaces = maybe [] elemToNameSpaces enElem
409+ ns = unionBy (\x y -> fst x == fst y) fn_namespaces en_namespaces
410+ fn = fnElem >>= walkDocument ns >>= elemToNotes ns "footnote"
411+ en = enElem >>= walkDocument ns >>= elemToNotes ns "endnote"
457412 in
458413 Notes ns fn en
459414
460415archiveToComments :: Archive -> Comments
461416archiveToComments zf =
462417 let cmtsElem = findEntryByPath "word/comments.xml" zf
463− >>= parseXMLFromEntry
464− cmts_namespaces = maybe mempty elemToNameSpaces cmtsElem
465− cmts = elemToComments cmts_namespaces . walkDocument cmts_namespaces <$>
466− cmtsElem
418+ >>= (parseXMLDoc . UTF8.toStringLazy . fromEntry)
419+ cmts_namespaces = maybe [] elemToNameSpaces cmtsElem
420+ cmts = elemToComments cmts_namespaces <$> (cmtsElem >>= walkDocument cmts_namespaces)
467421 in
468422 case cmts of
469423 Just c -> Comments cmts_namespaces c
@@ -479,26 +433,20 @@ filePathToRelType path docXmlPath =
479433 then Just InDocument
480434 else Nothing
481435
482−relElemToRelationship :: FilePath -> DocumentLocation -> Element
483− -> Maybe Relationship
484−relElemToRelationship fp relType element | qName (elName element) == "Relationship" =
436+relElemToRelationship :: DocumentLocation -> Element -> Maybe Relationship
437+relElemToRelationship relType element | qName (elName element) == "Relationship" =
485438 do
486− relId <- findAttr (QName "Id" Nothing Nothing) element
487− target <- findAttr (QName "Target" Nothing Nothing) element
488− -- target may be relative (media/image1.jpeg) or absolute
489− -- (/word/media/image1.jpeg); we need to relativize it (see #7374)
490− let frontOfFp = T.pack $ takeWhile (/= '_') fp
491− let target' = fromMaybe target $
492− T.stripPrefix frontOfFp $ T.dropWhile (== '/') target
493− return $ Relationship relType relId target'
494−relElemToRelationship _ _ _ = Nothing
439+ relId <- findAttrText (QName "Id" Nothing Nothing) element
440+ target <- findAttrText (QName "Target" Nothing Nothing) element
441+ return $ Relationship relType relId target
442+relElemToRelationship _ _ = Nothing
495443
496444filePathToRelationships :: Archive -> FilePath -> FilePath -> [Relationship]
497445filePathToRelationships ar docXmlPath fp
498446 | Just relType <- filePathToRelType fp docXmlPath
499447 , Just entry <- findEntryByPath fp ar
500− , Just relElems <- parseXMLFromEntry entry =
501− mapMaybe (relElemToRelationship fp relType) $ elChildren relElems
448+ , Just relElems <- (parseXMLDoc . UTF8.toStringLazy . fromEntry) entry =
449+ mapMaybe (relElemToRelationship relType) $ elChildren relElems
502450filePathToRelationships _ _ _ = []
503451
504452archiveToRelationships :: Archive -> FilePath -> [Relationship]
@@ -530,10 +478,10 @@ lookupLevel numId ilvl (Numbering _ numbs absNumbs) = do
530478loElemToLevelOverride :: NameSpaces -> Element -> Maybe LevelOverride
531479loElemToLevelOverride ns element
532480 | isElem ns "w" "lvlOverride" element = do
533− ilvl <- findAttrByName ns "w" "ilvl" element
481+ ilvl <- findAttrTextByName ns "w" "ilvl" element
534482 let startOverride = findChildByName ns "w" "startOverride" element
535483 >>= findAttrByName ns "w" "val"
536− >>= stringToInteger
484+ >>= (\s -> listToMaybe (map fst (reads s :: [(Integer, String)])))
537485 lvl = findChildByName ns "w" "lvl" element
538486 >>= levelElemToLevel ns
539487 return $ LevelOverride ilvl startOverride lvl
@@ -542,9 +490,9 @@ loElemToLevelOverride _ _ = Nothing
542490numElemToNum :: NameSpaces -> Element -> Maybe Numb
543491numElemToNum ns element
544492 | isElem ns "w" "num" element = do
545− numId <- findAttrByName ns "w" "numId" element
493+ numId <- findAttrTextByName ns "w" "numId" element
546494 absNumId <- findChildByName ns "w" "abstractNumId" element
547− >>= findAttrByName ns "w" "val"
495+ >>= findAttrTextByName ns "w" "val"
548496 let lvlOverrides = mapMaybe
549497 (loElemToLevelOverride ns)
550498 (findChildrenByName ns "w" "lvlOverride" element)
@@ -554,7 +502,7 @@ numElemToNum _ _ = Nothing
554502absNumElemToAbsNum :: NameSpaces -> Element -> Maybe AbstractNumb
555503absNumElemToAbsNum ns element
556504 | isElem ns "w" "abstractNum" element = do
557− absNumId <- findAttrByName ns "w" "abstractNumId" element
505+ absNumId <- findAttrTextByName ns "w" "abstractNumId" element
558506 let levelElems = findChildrenByName ns "w" "lvl" element
559507 levels = mapMaybe (levelElemToLevel ns) levelElems
560508 return $ AbstractNumb absNumId levels
@@ -563,23 +511,23 @@ absNumElemToAbsNum _ _ = Nothing
563511levelElemToLevel :: NameSpaces -> Element -> Maybe Level
564512levelElemToLevel ns element
565513 | isElem ns "w" "lvl" element = do
566− ilvl <- findAttrByName ns "w" "ilvl" element
514+ ilvl <- findAttrTextByName ns "w" "ilvl" element
567515 fmt <- findChildByName ns "w" "numFmt" element
568− >>= findAttrByName ns "w" "val"
516+ >>= findAttrTextByName ns "w" "val"
569517 txt <- findChildByName ns "w" "lvlText" element
570− >>= findAttrByName ns "w" "val"
518+ >>= findAttrTextByName ns "w" "val"
571519 let start = findChildByName ns "w" "start" element
572520 >>= findAttrByName ns "w" "val"
573− >>= stringToInteger
521+ >>= (\s -> listToMaybe (map fst (reads s :: [(Integer, String)])))
574522 return (Level ilvl fmt txt start)
575523levelElemToLevel _ _ = Nothing
576524
577525archiveToNumbering' :: Archive -> Maybe Numbering
578526archiveToNumbering' zf =
579527 case findEntryByPath "word/numbering.xml" zf of
580− Nothing -> Just $ Numbering mempty [] []
528+ Nothing -> Just $ Numbering [] [] []
581529 Just entry -> do
582− numberingElem <- parseXMLFromEntry entry
530+ numberingElem <- (parseXMLDoc . UTF8.toStringLazy . fromEntry) entry
583531 let namespaces = elemToNameSpaces numberingElem
584532 numElems = findChildrenByName namespaces "w" "num" numberingElem
585533 absNumElems = findChildrenByName namespaces "w" "abstractNum" numberingElem
@@ -589,13 +537,13 @@ archiveToNumbering' zf =
589537
590538archiveToNumbering :: Archive -> Numbering
591539archiveToNumbering archive =
592− fromMaybe (Numbering mempty [] []) (archiveToNumbering' archive)
540+ fromMaybe (Numbering [] [] []) (archiveToNumbering' archive)
593541
594−elemToNotes :: NameSpaces -> Text -> Element -> Maybe (M.Map T.Text Element)
542+elemToNotes :: NameSpaces -> String -> Element -> Maybe (M.Map T.Text Element)
595543elemToNotes ns notetype element
596544 | isElem ns "w" (notetype <> "s") element =
597545 let pairs = mapMaybe
598− (\e -> findAttrByName ns "w" "id" e >>=
546+ (\e -> findAttrTextByName ns "w" "id" e >>=
599547 (\a -> Just (a, e)))
600548 (findChildrenByName ns "w" notetype element)
601549 in
@@ -607,7 +555,7 @@ elemToComments :: NameSpaces -> Element -> M.Map T.Text Element
607555elemToComments ns element
608556 | isElem ns "w" "comments" element =
609557 let pairs = mapMaybe
610− (\e -> findAttrByName ns "w" "id" e >>=
558+ (\e -> findAttrTextByName ns "w" "id" e >>=
611559 (\a -> Just (a, e)))
612560 (findChildrenByName ns "w" "comment" element)
613561 in
@@ -622,7 +570,7 @@ elemToTblGrid :: NameSpaces -> Element -> D TblGrid
622570elemToTblGrid ns element | isElem ns "w" "tblGrid" element =
623571 let cols = findChildrenByName ns "w" "gridCol" element
624572 in
625− mapD (\e -> maybeToD (findAttrByName ns "w" "w" e >>= stringToInteger))
573+ mapD (\e -> maybeToD (findAttrByName ns "w" "val" e >>= stringToInteger))
626574 cols
627575elemToTblGrid _ _ = throwError WrongElem
628576
@@ -646,31 +594,14 @@ elemToRow ns element | isElem ns "w" "tr" element =
646594 do
647595 let cellElems = findChildrenByName ns "w" "tc" element
648596 cells <- mapD (elemToCell ns) cellElems
649− let hasTblHeader = maybe NoTblHeader (const HasTblHeader)
650− (findChildByName ns "w" "trPr" element
651− >>= findChildByName ns "w" "tblHeader")
652− return $ Row hasTblHeader cells
597+ return $ Row cells
653598elemToRow _ _ = throwError WrongElem
654599
655600elemToCell :: NameSpaces -> Element -> D Cell
656601elemToCell ns element | isElem ns "w" "tc" element =
657602 do
658− let properties = findChildByName ns "w" "tcPr" element
659− let gridSpan = properties
660− >>= findChildByName ns "w" "gridSpan"
661− >>= findAttrByName ns "w" "val"
662− >>= stringToInteger
663− let vMerge = case properties >>= findChildByName ns "w" "vMerge" of
664− Nothing -> Restart
665− Just e ->
666− fromMaybe Continue $ do
667− s <- findAttrByName ns "w" "val" e
668− case s of
669− "continue" -> Just Continue
670− "restart" -> Just Restart
671− _ -> Nothing
672603 cellContents <- mapD (elemToBodyPart ns) (elChildren element)
673− return $ Cell (fromMaybe 1 gridSpan) vMerge cellContents
604+ return $ Cell cellContents
674605elemToCell _ _ = throwError WrongElem
675606
676607elemToParIndentation :: NameSpaces -> Element -> Maybe ParIndentation
@@ -684,12 +615,12 @@ elemToParIndentation ns element | isElem ns "w" "ind" element =
684615 stringToInteger
685616 , hangingParIndent =
686617 findAttrByName ns "w" "hanging" element >>=
687− stringToInteger }
618+ stringToInteger}
688619elemToParIndentation _ _ = Nothing
689620
690−testBitMask :: Text -> Int -> Bool
621+testBitMask :: String -> Int -> Bool
691622testBitMask bitMaskS n =
692− case (reads ("0x" ++ T.unpack bitMaskS) :: [(Int, String)]) of
623+ case (reads ("0x" ++ bitMaskS) :: [(Int, String)]) of
693624 [] -> False
694625 ((n', _) : _) -> (n' .|. n) /= 0
695626
@@ -702,9 +633,10 @@ pNumInfo = getParStyleField numInfo . pStyle
702633elemToBodyPart :: NameSpaces -> Element -> D BodyPart
703634elemToBodyPart ns element
704635 | isElem ns "w" "p" element
705− , (c:_) <- findChildrenByName ns "m" "oMathPara" element = do
706− expsLst <- eitherToD $ readOMML $ showElement c
707− return $ OMathPara expsLst
636+ , (c:_) <- findChildrenByName ns "m" "oMathPara" element =
637+ do
638+ expsLst <- eitherToD $ readOMML $ T.pack $ showElement c
639+ return $ OMathPara expsLst
708640elemToBodyPart ns element
709641 | isElem ns "w" "p" element
710642 , Just (numId, lvl) <- getNumInfo ns element = do
@@ -722,31 +654,13 @@ elemToBodyPart ns element
722654 Nothing | Just (numId, lvl) <- pNumInfo parstyle -> do
723655 levelInfo <- lookupLevel numId lvl <$> asks envNumbering
724656 return $ ListItem parstyle numId lvl levelInfo parparts
725− _ -> let
726− hasCaptionStyle = elem "Caption" (pStyleId <$> pStyle parstyle)
727−
728− hasSimpleTableField = fromMaybe False $ do
729− fldSimple <- findChildByName ns "w" "fldSimple" element
730− instr <- findAttrByName ns "w" "instr" fldSimple
731− pure ("Table" `elem` T.words instr)
732−
733− hasComplexTableField = fromMaybe False $ do
734− instrText <- findElementByName ns "w" "instrText" element
735− pure ("Table" `elem` T.words (strContent instrText))
736−
737− in if hasCaptionStyle && (hasSimpleTableField || hasComplexTableField)
738− then return $ TblCaption parstyle parparts
739− else return $ Paragraph parstyle parparts
740−
657+ _ -> return $ Paragraph parstyle parparts
741658elemToBodyPart ns element
742659 | isElem ns "w" "tbl" element = do
743− let tblProperties = findChildByName ns "w" "tblPr" element
744− caption = fromMaybe "" $ tblProperties
660+ let caption' = findChildByName ns "w" "tblPr" element
745661 >>= findChildByName ns "w" "tblCaption"
746− >>= findAttrByName ns "w" "val"
747− description = fromMaybe "" $ tblProperties
748− >>= findChildByName ns "w" "tblDescription"
749− >>= findAttrByName ns "w" "val"
662+ >>= findAttrTextByName ns "w" "val"
663+ caption = fromMaybe "" caption'
750664 grid' = case findChildByName ns "w" "tblGrid" element of
751665 Just g -> elemToTblGrid ns g
752666 Nothing -> return []
@@ -759,7 +673,7 @@ elemToBodyPart ns element
759673 grid <- grid'
760674 tblLook <- tblLook'
761675 rows <- mapD (elemToRow ns) (elChildren element)
762− return $ Tbl (caption <> description) grid tblLook rows
676+ return $ Tbl caption grid tblLook rows
763677elemToBodyPart _ _ = throwError WrongElem
764678
765679lookupRelationship :: DocumentLocation -> RelId -> [Relationship] -> Maybe Target
@@ -784,8 +698,8 @@ getTitleAndAlt :: NameSpaces -> Element -> (T.Text, T.Text)
784698getTitleAndAlt ns element =
785699 let mbDocPr = findChildByName ns "wp" "inline" element >>=
786700 findChildByName ns "wp" "docPr"
787− title = fromMaybe "" (mbDocPr >>= findAttrByName ns "" "title")
788− alt = fromMaybe "" (mbDocPr >>= findAttrByName ns "" "descr")
701+ title = fromMaybe "" (mbDocPr >>= findAttrTextByName ns "" "title")
702+ alt = fromMaybe "" (mbDocPr >>= findAttrTextByName ns "" "descr")
789703 in (title, alt)
790704
791705elemToParPart :: NameSpaces -> Element -> D ParPart
@@ -797,29 +711,22 @@ elemToParPart ns element
797711 = let (title, alt) = getTitleAndAlt ns drawingElem
798712 a_ns = "http://schemas.openxmlformats.org/drawingml/2006/main"
799713 drawing = findElement (QName "blip" (Just a_ns) (Just "a")) picElem
800− >>= findAttrByName ns "r" "embed"
714+ >>= findAttrTextByName ns "r" "embed"
801715 in
802716 case drawing of
803717 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp title alt bs $ elemToExtent drawingElem)
804718 Nothing -> throwError WrongElem
805−-- The two cases below are an attempt to deal with images in deprecated vml format.
806−-- Todo: check out title and attr for deprecated format.
719+-- The below is an attempt to deal with images in deprecated vml format.
807720elemToParPart ns element
808721 | isElem ns "w" "r" element
809722 , Just _ <- findChildByName ns "w" "pict" element =
810723 let drawing = findElement (elemName ns "v" "imagedata") element
811− >>= findAttrByName ns "r" "id"
724+ >>= findAttrTextByName ns "r" "id"
812725 in
813726 case drawing of
727+ -- Todo: check out title and attr for deprecated format.
814728 Just s -> expandDrawingId s >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
815729 Nothing -> throwError WrongElem
816−elemToParPart ns element
817− | isElem ns "w" "r" element
818− , Just objectElem <- findChildByName ns "w" "object" element
819− , Just shapeElem <- findChildByName ns "v" "shape" objectElem
820− , Just imagedataElem <- findChildByName ns "v" "imagedata" shapeElem
821− , Just drawingId <- findAttrByName ns "r" "id" imagedataElem
822− = expandDrawingId drawingId >>= (\(fp, bs) -> return $ Drawing fp "" "" bs Nothing)
823730-- Chart
824731elemToParPart ns element
825732 | isElem ns "w" "r" element
@@ -883,7 +790,7 @@ elemToParPart ns element
883790 fldCharState <- gets stateFldCharState
884791 case fldCharState of
885792 FldCharOpen -> do
886− info <- eitherToD $ parseFieldInfo $ strContent instrText
793+ info <- eitherToD $ parseFieldInfo $ T.pack $ strContent instrText
887794 modify $ \st -> st{stateFldCharState = FldCharFieldInfo info}
888795 return NullParPart
889796 _ -> return NullParPart
@@ -904,48 +811,48 @@ elemToParPart ns element
904811 return $ ChangedRuns change runs
905812elemToParPart ns element
906813 | isElem ns "w" "bookmarkStart" element
907− , Just bmId <- findAttrByName ns "w" "id" element
908− , Just bmName <- findAttrByName ns "w" "name" element =
814+ , Just bmId <- findAttrTextByName ns "w" "id" element
815+ , Just bmName <- findAttrTextByName ns "w" "name" element =
909816 return $ BookMark bmId bmName
910817elemToParPart ns element
911818 | isElem ns "w" "hyperlink" element
912− , Just relId <- findAttrByName ns "r" "id" element = do
819+ , Just relId <- findAttrTextByName ns "r" "id" element = do
913820 location <- asks envLocation
914821 runs <- mapD (elemToRun ns) (elChildren element)
915822 rels <- asks envRelationships
916823 case lookupRelationship location relId rels of
917824 Just target ->
918− case findAttrByName ns "w" "anchor" element of
825+ case findAttrTextByName ns "w" "anchor" element of
919826 Just anchor -> return $ ExternalHyperLink (target <> "#" <> anchor) runs
920827 Nothing -> return $ ExternalHyperLink target runs
921828 Nothing -> return $ ExternalHyperLink "" runs
922829elemToParPart ns element
923830 | isElem ns "w" "hyperlink" element
924− , Just anchor <- findAttrByName ns "w" "anchor" element = do
831+ , Just anchor <- findAttrTextByName ns "w" "anchor" element = do
925832 runs <- mapD (elemToRun ns) (elChildren element)
926833 return $ InternalHyperLink anchor runs
927834elemToParPart ns element
928835 | isElem ns "w" "commentRangeStart" element
929− , Just cmtId <- findAttrByName ns "w" "id" element = do
836+ , Just cmtId <- findAttrTextByName ns "w" "id" element = do
930837 (Comments _ commentMap) <- asks envComments
931838 case M.lookup cmtId commentMap of
932839 Just cmtElem -> elemToCommentStart ns cmtElem
933840 Nothing -> throwError WrongElem
934841elemToParPart ns element
935842 | isElem ns "w" "commentRangeEnd" element
936− , Just cmtId <- findAttrByName ns "w" "id" element =
843+ , Just cmtId <- findAttrTextByName ns "w" "id" element =
937844 return $ CommentEnd cmtId
938845elemToParPart ns element
939846 | isElem ns "m" "oMath" element =
940− fmap PlainOMath (eitherToD $ readOMML $ showElement element)
847+ fmap PlainOMath (eitherToD $ readOMML $ T.pack $ showElement element)
941848elemToParPart _ _ = throwError WrongElem
942849
943850elemToCommentStart :: NameSpaces -> Element -> D ParPart
944851elemToCommentStart ns element
945852 | isElem ns "w" "comment" element
946− , Just cmtId <- findAttrByName ns "w" "id" element
947− , Just cmtAuthor <- findAttrByName ns "w" "author" element
948− , cmtDate <- findAttrByName ns "w" "date" element = do
853+ , Just cmtId <- findAttrTextByName ns "w" "id" element
854+ , Just cmtAuthor <- findAttrTextByName ns "w" "author" element
855+ , cmtDate <- findAttrTextByName ns "w" "date" element = do
949856 bps <- mapD (elemToBodyPart ns) (elChildren element)
950857 return $ CommentStart cmtId cmtAuthor cmtDate bps
951858elemToCommentStart _ _ = throwError WrongElem
@@ -964,7 +871,7 @@ elemToExtent drawingElem =
964871 where
965872 wp_ns = "http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"
966873 getDim at = findElement (QName "extent" (Just wp_ns) (Just "wp")) drawingElem
967− >>= findAttr (QName at Nothing Nothing) >>= safeRead
874+ >>= findAttr (QName at Nothing Nothing) >>= safeRead . T.pack
968875
969876
970877childElemToRun :: NameSpaces -> Element -> D Run
@@ -975,7 +882,7 @@ childElemToRun ns element
975882 = let (title, alt) = getTitleAndAlt ns element
976883 a_ns = "http://schemas.openxmlformats.org/drawingml/2006/main"
977884 drawing = findElement (QName "blip" (Just a_ns) (Just "a")) picElem
978− >>= findAttr (QName "embed" (M.lookup "r" ns) (Just "r"))
885+ >>= findAttrText (QName "embed" (lookup "r" ns) (Just "r"))
979886 in
980887 case drawing of
981888 Just s -> expandDrawingId s >>=
@@ -988,7 +895,7 @@ childElemToRun ns element
988895 = return InlineChart
989896childElemToRun ns element
990897 | isElem ns "w" "footnoteReference" element
991− , Just fnId <- findAttrByName ns "w" "id" element = do
898+ , Just fnId <- findAttrTextByName ns "w" "id" element = do
992899 notes <- asks envNotes
993900 case lookupFootnote fnId notes of
994901 Just e -> do bps <- local (\r -> r {envLocation=InFootnote}) $ mapD (elemToBodyPart ns) (elChildren e)
@@ -996,7 +903,7 @@ childElemToRun ns element
996903 Nothing -> return $ Footnote []
997904childElemToRun ns element
998905 | isElem ns "w" "endnoteReference" element
999− , Just enId <- findAttrByName ns "w" "id" element = do
906+ , Just enId <- findAttrTextByName ns "w" "id" element = do
1000907 notes <- asks envNotes
1001908 case lookupEndnote enId notes of
1002909 Just e -> do bps <- local (\r -> r {envLocation=InEndnote}) $ mapD (elemToBodyPart ns) (elChildren e)
@@ -1049,15 +956,15 @@ getParStyleField _ _ = Nothing
1049956getTrackedChange :: NameSpaces -> Element -> Maybe TrackedChange
1050957getTrackedChange ns element
1051958 | isElem ns "w" "ins" element || isElem ns "w" "moveTo" element
1052− , Just cId <- findAttrByName ns "w" "id" element
1053− , Just cAuthor <- findAttrByName ns "w" "author" element
1054− , mcDate <- findAttrByName ns "w" "date" element =
959+ , Just cId <- findAttrTextByName ns "w" "id" element
960+ , Just cAuthor <- findAttrTextByName ns "w" "author" element
961+ , mcDate <- findAttrTextByName ns "w" "date" element =
1055962 Just $ TrackedChange Insertion (ChangeInfo cId cAuthor mcDate)
1056963getTrackedChange ns element
1057964 | isElem ns "w" "del" element || isElem ns "w" "moveFrom" element
1058− , Just cId <- findAttrByName ns "w" "id" element
1059− , Just cAuthor <- findAttrByName ns "w" "author" element
1060− , mcDate <- findAttrByName ns "w" "date" element =
965+ , Just cId <- findAttrTextByName ns "w" "id" element
966+ , Just cAuthor <- findAttrTextByName ns "w" "author" element
967+ , mcDate <- findAttrTextByName ns "w" "date" element =
1061968 Just $ TrackedChange Deletion (ChangeInfo cId cAuthor mcDate)
1062969getTrackedChange _ _ = Nothing
1063970
@@ -1066,7 +973,7 @@ elemToParagraphStyle ns element sty
1066973 | Just pPr <- findChildByName ns "w" "pPr" element =
1067974 let style =
1068975 mapMaybe
1069− (fmap ParaStyleId . findAttrByName ns "w" "val")
976+ (fmap ParaStyleId . findAttrTextByName ns "w" "val")
1070977 (findChildrenByName ns "w" "pStyle" pPr)
1071978 in ParagraphStyle
1072979 {pStyle = mapMaybe (`M.lookup` sty) style
@@ -1098,7 +1005,7 @@ elemToRunStyleD ns element
10981005 charStyles <- asks envCharStyles
10991006 let parentSty =
11001007 findChildByName ns "w" "rStyle" rPr >>=
1101− findAttrByName ns "w" "val" >>=
1008+ findAttrTextByName ns "w" "val" >>=
11021009 flip M.lookup charStyles . CharStyleId
11031010 return $ elemToRunStyle ns element parentSty
11041011elemToRunStyleD _ _ = return defaultRunStyle
@@ -1108,7 +1015,7 @@ elemToRunElem ns element
11081015 | isElem ns "w" "t" element
11091016 || isElem ns "w" "delText" element
11101017 || isElem ns "m" "t" element = do
1111− let str = strContent element
1018+ let str = T.pack $ strContent element
11121019 font <- asks envFont
11131020 case font of
11141021 Nothing -> return $ TextRun str
@@ -1130,14 +1037,14 @@ getSymChar :: NameSpaces -> Element -> RunElem
11301037getSymChar ns element
11311038 | Just s <- lowerFromPrivate <$> getCodepoint
11321039 , Just font <- getFont =
1133− case readLitChar ("\\x" ++ T.unpack s) of
1040+ case readLitChar ("\\x" ++ s) of
11341041 [(char, _)] -> TextRun . maybe "" T.singleton $ getUnicode font char
11351042 _ -> TextRun ""
11361043 where
11371044 getCodepoint = findAttrByName ns "w" "char" element
1138− getFont = textToFont =<< findAttrByName ns "w" "font" element
1139− lowerFromPrivate t | "F" `T.isPrefixOf` t = "0" <> T.drop 1 t
1140− | otherwise = t
1045+ getFont = textToFont . T.pack =<< findAttrByName ns "w" "font" element
1046+ lowerFromPrivate ('F':xs) = '0':xs
1047+ lowerFromPrivate xs = xs
11411048getSymChar _ _ = TextRun ""
11421049
11431050elemToRunElems :: NameSpaces -> Element -> D [RunElem]
@@ -1147,9 +1054,8 @@ elemToRunElems ns element
11471054 let qualName = elemName ns "w"
11481055 let font = do
11491056 fontElem <- findElement (qualName "rFonts") element
1150− textToFont =<<
1151− foldr ((<|>) . (flip findAttr fontElem . qualName))
1152− Nothing ["ascii", "hAnsi"]
1057+ textToFont . T.pack =<<
1058+ foldr ((<|>) . (flip findAttr fontElem . qualName)) Nothing ["ascii", "hAnsi"]
11531059 local (setFont font) (mapD (elemToRunElem ns) (elChildren element))
11541060elemToRunElems _ _ = throwError WrongElem
11551061
src/Text/Pandoc/Readers/Docx/Parse/Styles.hs+25−32
@@ -48,13 +48,11 @@ import Data.Function (on)
4848import Data.String (IsString(..))
4949import qualified Data.Map as M
5050import qualified Data.Text as T
51−import qualified Data.Text.Read
52−import Data.Text (Text)
5351import Data.Maybe
5452import Data.Coerce
5553import Text.Pandoc.Readers.Docx.Util
5654import qualified Text.Pandoc.UTF8 as UTF8
57−import Text.Pandoc.XML.Light
55+import Text.XML.Light
5856
5957newtype CharStyleId = CharStyleId T.Text
6058 deriving (Show, Eq, Ord, IsString, FromStyleId)
@@ -110,7 +108,7 @@ data RunStyle = RunStyle { isBold :: Maybe Bool
110108 , isRTL :: Maybe Bool
111109 , isForceCTL :: Maybe Bool
112110 , rVertAlign :: Maybe VertAlign
113− , rUnderline :: Maybe Text
111+ , rUnderline :: Maybe String
114112 , rParentStyle :: Maybe CharStyle
115113 }
116114 deriving Show
@@ -137,22 +135,19 @@ defaultRunStyle = RunStyle { isBold = Nothing
137135 , rParentStyle = Nothing
138136 }
139137
140−archiveToStyles'
141− :: (Ord k1, Ord k2, ElemToStyle a1, ElemToStyle a2)
142− => (a1 -> k1) -> (a2 -> k2) -> Archive -> (M.Map k1 a1, M.Map k2 a2)
138+archiveToStyles' :: (Ord k1, Ord k2, ElemToStyle a1, ElemToStyle a2) =>
139+ (a1 -> k1) -> (a2 -> k2) -> Archive -> (M.Map k1 a1, M.Map k2 a2)
143140archiveToStyles' conv1 conv2 zf =
144− case findEntryByPath "word/styles.xml" zf of
145− Nothing -> (M.empty, M.empty)
146− Just entry ->
147− case parseXMLElement . UTF8.toTextLazy . fromEntry $ entry of
148− Left _ -> (M.empty, M.empty)
149− Right styElem ->
150− let namespaces = elemToNameSpaces styElem
151− in
152− ( M.fromList $ map (\r -> (conv1 r, r)) $
153− buildBasedOnList namespaces styElem Nothing,
154− M.fromList $ map (\p -> (conv2 p, p)) $
155− buildBasedOnList namespaces styElem Nothing)
141+ let stylesElem = findEntryByPath "word/styles.xml" zf >>=
142+ (parseXMLDoc . UTF8.toStringLazy . fromEntry)
143+ in
144+ case stylesElem of
145+ Nothing -> (M.empty, M.empty)
146+ Just styElem ->
147+ let namespaces = elemToNameSpaces styElem
148+ in
149+ ( M.fromList $ map (\r -> (conv1 r, r)) $ buildBasedOnList namespaces styElem Nothing,
150+ M.fromList $ map (\p -> (conv2 p, p)) $ buildBasedOnList namespaces styElem Nothing)
156151
157152isBasedOnStyle :: (ElemToStyle a, FromStyleId (StyleId a)) => NameSpaces -> Element -> Maybe a -> Bool
158153isBasedOnStyle ns element parentStyle
@@ -160,7 +155,7 @@ isBasedOnStyle ns element parentStyle
160155 , Just styleType <- findAttrByName ns "w" "type" element
161156 , styleType == cStyleType parentStyle
162157 , Just basedOnVal <- findChildByName ns "w" "basedOn" element >>=
163− findAttrByName ns "w" "val"
158+ findAttrTextByName ns "w" "val"
164159 , Just ps <- parentStyle = basedOnVal == fromStyleId (getStyleId ps)
165160 | isElem ns "w" "style" element
166161 , Just styleType <- findAttrByName ns "w" "type" element
@@ -170,7 +165,7 @@ isBasedOnStyle ns element parentStyle
170165 | otherwise = False
171166
172167class HasStyleId a => ElemToStyle a where
173− cStyleType :: Maybe a -> Text
168+ cStyleType :: Maybe a -> String
174169 elemToStyle :: NameSpaces -> Element -> Maybe a -> Maybe a
175170
176171class FromStyleId (StyleId a) => HasStyleId a where
@@ -227,10 +222,8 @@ buildBasedOnList ns element rootStyle =
227222 stys -> stys ++
228223 concatMap (buildBasedOnList ns element . Just) stys
229224
230−stringToInteger :: Text -> Maybe Integer
231−stringToInteger s = case Data.Text.Read.decimal s of
232− Right (x,_) -> Just x
233− Left _ -> Nothing
225+stringToInteger :: String -> Maybe Integer
226+stringToInteger s = listToMaybe $ map fst (reads s :: [(Integer, String)])
234227
235228checkOnOff :: NameSpaces -> Element -> QName -> Maybe Bool
236229checkOnOff ns rPr tag
@@ -250,7 +243,7 @@ checkOnOff _ _ _ = Nothing
250243elemToCharStyle :: NameSpaces
251244 -> Element -> Maybe CharStyle -> Maybe CharStyle
252245elemToCharStyle ns element parentStyle
253− = CharStyle <$> (CharStyleId <$> findAttrByName ns "w" "styleId" element)
246+ = CharStyle <$> (CharStyleId <$> findAttrTextByName ns "w" "styleId" element)
254247 <*> getElementStyleName ns element
255248 <*> Just (elemToRunStyle ns element parentStyle)
256249
@@ -284,7 +277,7 @@ elemToRunStyle _ _ _ = defaultRunStyle
284277getHeaderLevel :: NameSpaces -> Element -> Maybe (ParaStyleName, Int)
285278getHeaderLevel ns element
286279 | Just styleName <- getElementStyleName ns element
287− , Just n <- stringToInteger =<<
280+ , Just n <- stringToInteger . T.unpack =<<
288281 (T.stripPrefix "heading " . T.toLower $
289282 fromStyleName styleName)
290283 , n > 0 = Just (styleName, fromInteger n)
@@ -292,8 +285,8 @@ getHeaderLevel _ _ = Nothing
292285
293286getElementStyleName :: Coercible T.Text a => NameSpaces -> Element -> Maybe a
294287getElementStyleName ns el = coerce <$>
295− ((findChildByName ns "w" "name" el >>= findAttrByName ns "w" "val")
296− <|> findAttrByName ns "w" "styleId" el)
288+ ((findChildByName ns "w" "name" el >>= findAttrTextByName ns "w" "val")
289+ <|> findAttrTextByName ns "w" "styleId" el)
297290
298291getNumInfo :: NameSpaces -> Element -> Maybe (T.Text, T.Text)
299292getNumInfo ns element = do
@@ -301,15 +294,15 @@ getNumInfo ns element = do
301294 findChildByName ns "w" "numPr"
302295 lvl = fromMaybe "0" (numPr >>=
303296 findChildByName ns "w" "ilvl" >>=
304− findAttrByName ns "w" "val")
297+ findAttrTextByName ns "w" "val")
305298 numId <- numPr >>=
306299 findChildByName ns "w" "numId" >>=
307− findAttrByName ns "w" "val"
300+ findAttrTextByName ns "w" "val"
308301 return (numId, lvl)
309302
310303elemToParStyleData :: NameSpaces -> Element -> Maybe ParStyle -> Maybe ParStyle
311304elemToParStyleData ns element parentStyle
312− | Just styleId <- findAttrByName ns "w" "styleId" element
305+ | Just styleId <- findAttrTextByName ns "w" "styleId" element
313306 , Just styleName <- getElementStyleName ns element
314307 = Just $ ParStyle
315308 {
src/Text/Pandoc/Readers/Docx/Util.hs+26−28
@@ -1,8 +1,7 @@
1−{-# LANGUAGE OverloadedStrings #-}
21{- |
32 Module : Text.Pandoc.Readers.Docx.StyleMaps
43 Copyright : © 2014-2020 Jesse Rosenthal <jrosenthal@jhu.edu>,
5− 2014-2021 John MacFarlane <jgm@berkeley.edu>,
4+ 2014-2020 John MacFarlane <jgm@berkeley.edu>,
65 2015 Nikolay Yakimov <root@livid.pp.ru>
76 License : GNU GPL, version 2 or above
87
@@ -19,52 +18,51 @@ module Text.Pandoc.Readers.Docx.Util (
1918 , elemToNameSpaces
2019 , findChildByName
2120 , findChildrenByName
22− , findElementByName
21+ , findAttrText
2322 , findAttrByName
23+ , findAttrTextByName
2424 ) where
2525
26+import Data.Maybe (mapMaybe)
2627import qualified Data.Text as T
27−import Data.Text (Text)
28−import Text.Pandoc.XML.Light
29−import qualified Data.Map as M
28+import Text.XML.Light
3029
31−type NameSpaces = M.Map Text Text
30+type NameSpaces = [(String, String)]
3231
3332elemToNameSpaces :: Element -> NameSpaces
34−elemToNameSpaces = foldr (\(Attr qn val) ->
35− case qn of
36− QName s _ (Just "xmlns") -> M.insert s val
37− _ -> id) mempty . elAttribs
33+elemToNameSpaces = mapMaybe attrToNSPair . elAttribs
3834
39−elemName :: NameSpaces -> Text -> Text -> QName
35+attrToNSPair :: Attr -> Maybe (String, String)
36+attrToNSPair (Attr (QName s _ (Just "xmlns")) val) = Just (s, val)
37+attrToNSPair _ = Nothing
38+
39+elemName :: NameSpaces -> String -> String -> QName
4040elemName ns prefix name =
41− QName name (M.lookup prefix ns)
42− (if T.null prefix then Nothing else Just prefix)
41+ QName name (lookup prefix ns) (if null prefix then Nothing else Just prefix)
4342
44−isElem :: NameSpaces -> Text -> Text -> Element -> Bool
43+isElem :: NameSpaces -> String -> String -> Element -> Bool
4544isElem ns prefix name element =
46− let ns' = ns <> elemToNameSpaces element
45+ let ns' = ns ++ elemToNameSpaces element
4746 in qName (elName element) == name &&
48− qURI (elName element) == M.lookup prefix ns'
47+ qURI (elName element) == lookup prefix ns'
4948
50−findChildByName :: NameSpaces -> Text -> Text -> Element -> Maybe Element
49+findChildByName :: NameSpaces -> String -> String -> Element -> Maybe Element
5150findChildByName ns pref name el =
52− let ns' = ns <> elemToNameSpaces el
51+ let ns' = ns ++ elemToNameSpaces el
5352 in findChild (elemName ns' pref name) el
5453
55−findChildrenByName :: NameSpaces -> Text -> Text -> Element -> [Element]
54+findChildrenByName :: NameSpaces -> String -> String -> Element -> [Element]
5655findChildrenByName ns pref name el =
57− let ns' = ns <> elemToNameSpaces el
56+ let ns' = ns ++ elemToNameSpaces el
5857 in findChildren (elemName ns' pref name) el
5958
60−-- | Like 'findChildrenByName', but searches descendants.
61−findElementByName :: NameSpaces -> Text -> Text -> Element -> Maybe Element
62−findElementByName ns pref name el =
63− let ns' = ns <> elemToNameSpaces el
64− in findElement (elemName ns' pref name) el
59+findAttrText :: QName -> Element -> Maybe T.Text
60+findAttrText x = fmap T.pack . findAttr x
6561
66−findAttrByName :: NameSpaces -> Text -> Text -> Element -> Maybe Text
62+findAttrByName :: NameSpaces -> String -> String -> Element -> Maybe String
6763findAttrByName ns pref name el =
68− let ns' = ns <> elemToNameSpaces el
64+ let ns' = ns ++ elemToNameSpaces el
6965 in findAttr (elemName ns' pref name) el
7066
67+findAttrTextByName :: NameSpaces -> String -> String -> Element -> Maybe T.Text
68+findAttrTextByName a b c = fmap T.pack . findAttrByName a b c
src/Text/Pandoc/Readers/DokuWiki.hs+9−10
@@ -29,27 +29,26 @@ import Text.Pandoc.Definition
2929import Text.Pandoc.Error (PandocError (PandocParsecError))
3030import Text.Pandoc.Options
3131import Text.Pandoc.Parsing hiding (enclosed, nested)
32−import Text.Pandoc.Shared (trim, stringify, tshow)
32+import Text.Pandoc.Shared (crFilter, trim, stringify, tshow)
3333
3434-- | Read DokuWiki from an input string and return a Pandoc document.
35−readDokuWiki :: (PandocMonad m, ToSources a)
35+readDokuWiki :: PandocMonad m
3636 => ReaderOptions
37− -> a
37+ -> Text
3838 -> m Pandoc
3939readDokuWiki opts s = do
40− let sources = toSources s
41− res <- runParserT parseDokuWiki def {stateOptions = opts }
42− (initialSourceName sources) sources
40+ let input = crFilter s
41+ res <- runParserT parseDokuWiki def {stateOptions = opts } "source" input
4342 case res of
44− Left e -> throwError $ PandocParsecError sources e
43+ Left e -> throwError $ PandocParsecError input e
4544 Right d -> return d
4645
47−type DWParser = ParserT Sources ParserState
46+type DWParser = ParserT Text ParserState
4847
4948-- * Utility functions
5049
5150-- | Parse end-of-line, which can be either a newline or end-of-file.
52−eol :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()
51+eol :: Stream s m Char => ParserT s st m ()
5352eol = void newline <|> eof
5453
5554nested :: PandocMonad m => DWParser m a -> DWParser m a
@@ -318,7 +317,7 @@ interwikiToUrl "wpes" page = "https://es.wikipedia.org/wiki/" <> page
318317interwikiToUrl "wpfr" page = "https://fr.wikipedia.org/wiki/" <> page
319318interwikiToUrl "wpjp" page = "https://jp.wikipedia.org/wiki/" <> page
320319interwikiToUrl "wppl" page = "https://pl.wikipedia.org/wiki/" <> page
321−interwikiToUrl unknown page = unknown <> ">" <> page
320+interwikiToUrl _ page = "https://www.google.com/search?q=" <> page <> "&btnI=lucky"
322321
323322linkText :: PandocMonad m => DWParser m B.Inlines
324323linkText = parseLink fromRaw "[[" "]]"
src/Text/Pandoc/Readers/EPUB.hs+36−44
@@ -17,14 +17,14 @@ module Text.Pandoc.Readers.EPUB
1717 (readEPUB)
1818 where
1919
20−import Codec.Archive.Zip (Archive (..), Entry(..), findEntryByPath, fromEntry,
20+import Codec.Archive.Zip (Archive (..), Entry, findEntryByPath, fromEntry,
2121 toArchiveOrFail)
2222import Control.DeepSeq (NFData, deepseq)
2323import Control.Monad (guard, liftM, liftM2, mplus)
2424import Control.Monad.Except (throwError)
2525import qualified Data.ByteString.Lazy as BL (ByteString)
26+import Data.List (isInfixOf)
2627import qualified Data.Text as T
27−import Data.Text (Text)
2828import qualified Data.Map as M (Map, elems, fromList, lookup)
2929import Data.Maybe (mapMaybe)
3030import qualified Data.Text.Lazy as TL
@@ -40,12 +40,12 @@ import Text.Pandoc.Extensions (Extension (Ext_raw_html), enableExtension)
4040import Text.Pandoc.MIME (MimeType)
4141import Text.Pandoc.Options (ReaderOptions (..))
4242import Text.Pandoc.Readers.HTML (readHtml)
43−import Text.Pandoc.Shared (addMetaField, collapseFilePath, escapeURI, tshow)
44−import qualified Text.Pandoc.UTF8 as UTF8 (toTextLazy)
43+import Text.Pandoc.Shared (addMetaField, collapseFilePath, escapeURI)
44+import qualified Text.Pandoc.UTF8 as UTF8 (toStringLazy)
4545import Text.Pandoc.Walk (query, walk)
46−import Text.Pandoc.XML.Light
46+import Text.XML.Light
4747
48−type Items = M.Map Text (FilePath, MimeType)
48+type Items = M.Map String (FilePath, MimeType)
4949
5050readEPUB :: PandocMonad m => ReaderOptions -> BL.ByteString -> m Pandoc
5151readEPUB opts bytes = case toArchiveOrFail bytes of
@@ -125,27 +125,26 @@ imageToPandoc s = B.doc . B.para $ B.image (T.pack s) "" mempty
125125imageMimes :: [MimeType]
126126imageMimes = ["image/gif", "image/jpeg", "image/png"]
127127
128−type CoverId = Text
128+type CoverId = String
129129
130130type CoverImage = FilePath
131131
132−parseManifest :: (PandocMonad m)
133− => Element -> Maybe CoverId -> m (Maybe CoverImage, Items)
132+parseManifest :: (PandocMonad m) => Element -> Maybe CoverId -> m (Maybe CoverImage, Items)
134133parseManifest content coverId = do
135134 manifest <- findElementE (dfName "manifest") content
136135 let items = findChildren (dfName "item") manifest
137136 r <- mapM parseItem items
138137 let cover = findAttr (emptyName "href") =<< filterChild findCover manifest
139− return (T.unpack <$> (cover `mplus` coverId), M.fromList r)
138+ return (cover `mplus` coverId, M.fromList r)
140139 where
141− findCover e = maybe False (T.isInfixOf "cover-image")
140+ findCover e = maybe False (isInfixOf "cover-image")
142141 (findAttr (emptyName "properties") e)
143142 || Just True == liftM2 (==) coverId (findAttr (emptyName "id") e)
144143 parseItem e = do
145144 uid <- findAttrE (emptyName "id") e
146145 href <- findAttrE (emptyName "href") e
147146 mime <- findAttrE (emptyName "media-type") e
148− return (uid, (T.unpack href, mime))
147+ return (uid, (href, T.pack mime))
149148
150149parseSpine :: PandocMonad m => Items -> Element -> m [(FilePath, MimeType)]
151150parseSpine is e = do
@@ -173,25 +172,25 @@ parseMeta content = do
173172-- http://www.idpf.org/epub/30/spec/epub30-publications.html#sec-metadata-elem
174173parseMetaItem :: Element -> Meta -> Meta
175174parseMetaItem e@(stripNamespace . elName -> field) meta =
176− addMetaField (renameMeta field) (B.str $ strContent e) meta
175+ addMetaField (renameMeta field) (B.str $ T.pack $ strContent e) meta
177176
178−renameMeta :: Text -> Text
177+renameMeta :: String -> T.Text
179178renameMeta "creator" = "author"
180−renameMeta s = s
179+renameMeta s = T.pack s
181180
182181getManifest :: PandocMonad m => Archive -> m (String, Element)
183182getManifest archive = do
184183 metaEntry <- findEntryByPathE ("META-INF" </> "container.xml") archive
185− docElem <- parseXMLDocE metaEntry
184+ docElem <- (parseXMLDocE . UTF8.toStringLazy . fromEntry) metaEntry
186185 let namespaces = mapMaybe attrToNSPair (elAttribs docElem)
187186 ns <- mkE "xmlns not in namespaces" (lookup "xmlns" namespaces)
188187 as <- fmap (map attrToPair . elAttribs)
189188 (findElementE (QName "rootfile" (Just ns) Nothing) docElem)
190− manifestFile <- T.unpack <$> mkE "Root not found" (lookup "full-path" as)
189+ manifestFile <- mkE "Root not found" (lookup "full-path" as)
191190 let rootdir = dropFileName manifestFile
192191 --mime <- lookup "media-type" as
193192 manifest <- findEntryByPathE manifestFile archive
194− (rootdir,) <$> parseXMLDocE manifest
193+ (rootdir,) <$> (parseXMLDocE . UTF8.toStringLazy . fromEntry $ manifest)
195194
196195-- Fixup
197196
@@ -201,8 +200,7 @@ fixInternalReferences pathToFile =
201200 . walk (fixBlockIRs filename)
202201 . walk (fixInlineIRs filename)
203202 where
204− (root, T.unpack . escapeURI . T.pack -> filename) =
205− splitFileName pathToFile
203+ (root, T.unpack . escapeURI . T.pack -> filename) = splitFileName pathToFile
206204
207205fixInlineIRs :: String -> Inline -> Inline
208206fixInlineIRs s (Span as v) =
@@ -215,7 +213,7 @@ fixInlineIRs s (Link as is t) =
215213 Link (fixAttrs s as) is t
216214fixInlineIRs _ v = v
217215
218−prependHash :: [Text] -> Inline -> Inline
216+prependHash :: [T.Text] -> Inline -> Inline
219217prependHash ps l@(Link attr is (url, tit))
220218 | or [s `T.isPrefixOf` url | s <- ps] =
221219 Link attr is ("#" <> url, tit)
@@ -232,17 +230,16 @@ fixBlockIRs s (CodeBlock as code) =
232230fixBlockIRs _ b = b
233231
234232fixAttrs :: FilePath -> B.Attr -> B.Attr
235−fixAttrs s (ident, cs, kvs) =
236− (addHash s ident, filter (not . T.null) cs, removeEPUBAttrs kvs)
233+fixAttrs s (ident, cs, kvs) = (addHash s ident, filter (not . T.null) cs, removeEPUBAttrs kvs)
237234
238−addHash :: FilePath -> Text -> Text
235+addHash :: String -> T.Text -> T.Text
239236addHash _ "" = ""
240237addHash s ident = T.pack (takeFileName s) <> "#" <> ident
241238
242−removeEPUBAttrs :: [(Text, Text)] -> [(Text, Text)]
239+removeEPUBAttrs :: [(T.Text, T.Text)] -> [(T.Text, T.Text)]
243240removeEPUBAttrs kvs = filter (not . isEPUBAttr) kvs
244241
245−isEPUBAttr :: (Text, a) -> Bool
242+isEPUBAttr :: (T.Text, a) -> Bool
246243isEPUBAttr (k, _) = "epub:" `T.isPrefixOf` k
247244
248245-- Library
@@ -259,44 +256,39 @@ uncurry3 f (a, b, c) = f a b c
259256
260257-- Utility
261258
262−stripNamespace :: QName -> Text
259+stripNamespace :: QName -> String
263260stripNamespace (QName v _ _) = v
264261
265−attrToNSPair :: Attr -> Maybe (Text, Text)
262+attrToNSPair :: Attr -> Maybe (String, String)
266263attrToNSPair (Attr (QName "xmlns" _ _) val) = Just ("xmlns", val)
267264attrToNSPair _ = Nothing
268265
269−attrToPair :: Attr -> (Text, Text)
266+attrToPair :: Attr -> (String, String)
270267attrToPair (Attr (QName name _ _) val) = (name, val)
271268
272−defaultNameSpace :: Maybe Text
269+defaultNameSpace :: Maybe String
273270defaultNameSpace = Just "http://www.idpf.org/2007/opf"
274271
275−dfName :: Text -> QName
272+dfName :: String -> QName
276273dfName s = QName s defaultNameSpace Nothing
277274
278−emptyName :: Text -> QName
275+emptyName :: String -> QName
279276emptyName s = QName s Nothing Nothing
280277
281278-- Convert Maybe interface to Either
282279
283−findAttrE :: PandocMonad m => QName -> Element -> m Text
280+findAttrE :: PandocMonad m => QName -> Element -> m String
284281findAttrE q e = mkE "findAttr" $ findAttr q e
285282
286283findEntryByPathE :: PandocMonad m => FilePath -> Archive -> m Entry
287284findEntryByPathE (normalise . unEscapeString -> path) a =
288− mkE ("No entry on path: " <> T.pack path) $ findEntryByPath path a
285+ mkE ("No entry on path: " ++ path) $ findEntryByPath path a
289286
290−parseXMLDocE :: PandocMonad m => Entry -> m Element
291−parseXMLDocE entry =
292− either (throwError . PandocXMLError fp) return $ parseXMLElement doc
293− where
294− doc = UTF8.toTextLazy . fromEntry $ entry
295− fp = T.pack $ eRelativePath entry
287+parseXMLDocE :: PandocMonad m => String -> m Element
288+parseXMLDocE doc = mkE "Unable to parse XML doc" $ parseXMLDoc doc
296289
297290findElementE :: PandocMonad m => QName -> Element -> m Element
298−findElementE e x =
299− mkE ("Unable to find element: " <> tshow e) $ findElement e x
291+findElementE e x = mkE ("Unable to find element: " ++ show e) $ findElement e x
300292
301−mkE :: PandocMonad m => Text -> Maybe a -> m a
302−mkE s = maybe (throwError . PandocParseError $ s) return
293+mkE :: PandocMonad m => String -> Maybe a -> m a
294+mkE s = maybe (throwError . PandocParseError $ T.pack s) return
src/Text/Pandoc/Readers/FB2.hs+51−57
@@ -25,13 +25,13 @@ TODO:
2525module Text.Pandoc.Readers.FB2 ( readFB2 ) where
2626import Control.Monad.Except (throwError)
2727import Control.Monad.State.Strict
28+import Data.ByteString.Lazy.Char8 ( pack )
2829import Data.ByteString.Base64.Lazy
2930import Data.Functor
3031import Data.List (intersperse)
3132import qualified Data.Map as M
3233import Data.Text (Text)
3334import qualified Data.Text as T
34−import qualified Data.Text.Lazy as TL
3535import Data.Default
3636import Data.Maybe
3737import Text.HTML.TagSoup.Entity (lookupEntity)
@@ -40,9 +40,8 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad, insertMedia, report)
4040import Text.Pandoc.Error
4141import Text.Pandoc.Logging
4242import Text.Pandoc.Options
43−import Text.Pandoc.XML.Light
44−import qualified Text.Pandoc.UTF8 as UTF8
45−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
43+import Text.Pandoc.Shared (crFilter)
44+import Text.XML.Light
4645
4746type FB2 m = StateT FB2State m
4847
@@ -63,15 +62,12 @@ instance HasMeta FB2State where
6362 setMeta field v s = s {fb2Meta = setMeta field v (fb2Meta s)}
6463 deleteMeta field s = s {fb2Meta = deleteMeta field (fb2Meta s)}
6564
66−readFB2 :: (PandocMonad m, ToSources a)
67− => ReaderOptions
68− -> a
69− -> m Pandoc
65+readFB2 :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
7066readFB2 _ inp =
71− case parseXMLElement $ TL.fromStrict $ sourcesToText $ toSources inp of
72− Left msg -> throwError $ PandocXMLError "" msg
73− Right el -> do
74− (bs, st) <- runStateT (parseRootElement el) def
67+ case parseXMLDoc $ crFilter inp of
68+ Nothing -> throwError $ PandocParseError "Not an XML document"
69+ Just e -> do
70+ (bs, st) <- runStateT (parseRootElement e) def
7571 let authors = if null $ fb2Authors st
7672 then id
7773 else setMeta "author" (map text $ reverse $ fb2Authors st)
@@ -87,12 +83,12 @@ removeHash t = case T.uncons t of
8783 Just ('#', xs) -> xs
8884 _ -> t
8985
90−convertEntity :: Text -> Text
91−convertEntity e = maybe (T.toUpper e) T.pack $ lookupEntity (T.unpack e)
86+convertEntity :: String -> Text
87+convertEntity e = maybe (T.toUpper $ T.pack e) T.pack $ lookupEntity e
9288
9389parseInline :: PandocMonad m => Content -> FB2 m Inlines
9490parseInline (Elem e) =
95− case qName $ elName e of
91+ case T.pack $ qName $ elName e of
9692 "strong" -> strong <$> parseStyleType e
9793 "emphasis" -> emph <$> parseStyleType e
9894 "style" -> parseNamedStyle e
@@ -100,12 +96,12 @@ parseInline (Elem e) =
10096 "strikethrough" -> strikeout <$> parseStyleType e
10197 "sub" -> subscript <$> parseStyleType e
10298 "sup" -> superscript <$> parseStyleType e
103− "code" -> pure $ code $ strContent e
99+ "code" -> pure $ code $ T.pack $ strContent e
104100 "image" -> parseInlineImageElement e
105101 name -> do
106102 report $ IgnoredElement name
107103 pure mempty
108−parseInline (Text x) = pure $ text $ cdData x
104+parseInline (Text x) = pure $ text $ T.pack $ cdData x
109105parseInline (CRef r) = pure $ str $ convertEntity r
110106
111107parseSubtitle :: PandocMonad m => Element -> FB2 m Blocks
@@ -115,7 +111,7 @@ parseSubtitle e = headerWith ("", ["unnumbered"], []) <$> gets fb2SectionLevel <
115111
116112parseRootElement :: PandocMonad m => Element -> FB2 m Blocks
117113parseRootElement e =
118− case qName $ elName e of
114+ case T.pack $ qName $ elName e of
119115 "FictionBook" -> do
120116 -- Parse notes before parsing the rest of the content.
121117 case filterChild isNotesBody e of
@@ -148,7 +144,7 @@ parseNote e =
148144 Just sectionId -> do
149145 content <- mconcat <$> mapM parseSectionChild (dropTitle $ elChildren e)
150146 oldNotes <- gets fb2Notes
151− modify $ \s -> s { fb2Notes = M.insert ("#" <> sectionId) content oldNotes }
147+ modify $ \s -> s { fb2Notes = M.insert ("#" <> T.pack sectionId) content oldNotes }
152148 pure ()
153149 where
154150 isTitle x = qName (elName x) == "title"
@@ -160,7 +156,7 @@ parseNote e =
160156-- | Parse a child of @\<FictionBook>@ element.
161157parseFictionBookChild :: PandocMonad m => Element -> FB2 m Blocks
162158parseFictionBookChild e =
163− case qName $ elName e of
159+ case T.pack $ qName $ elName e of
164160 "stylesheet" -> pure mempty -- stylesheet is ignored
165161 "description" -> mempty <$ mapM_ parseDescriptionChild (elChildren e)
166162 "body" -> if isNotesBody e
@@ -172,7 +168,7 @@ parseFictionBookChild e =
172168-- | Parse a child of @\<description>@ element.
173169parseDescriptionChild :: PandocMonad m => Element -> FB2 m ()
174170parseDescriptionChild e =
175− case qName $ elName e of
171+ case T.pack $ qName $ elName e of
176172 "title-info" -> mapM_ parseTitleInfoChild (elChildren e)
177173 "src-title-info" -> pure () -- ignore
178174 "document-info" -> pure ()
@@ -186,7 +182,7 @@ parseDescriptionChild e =
186182-- | Parse a child of @\<body>@ element.
187183parseBodyChild :: PandocMonad m => Element -> FB2 m Blocks
188184parseBodyChild e =
189− case qName $ elName e of
185+ case T.pack $ qName $ elName e of
190186 "image" -> parseImageElement e
191187 "title" -> header <$> gets fb2SectionLevel <*> parseTitleType (elContent e)
192188 "epigraph" -> parseEpigraph e
@@ -200,10 +196,7 @@ parseBinaryElement e =
200196 (Nothing, _) -> report $ IgnoredElement "binary without id attribute"
201197 (Just _, Nothing) ->
202198 report $ IgnoredElement "binary without content-type attribute"
203− (Just filename, contentType) ->
204− insertMedia (T.unpack filename) contentType
205− (decodeLenient
206− (UTF8.fromTextLazy . TL.fromStrict . strContent $ e))
199+ (Just filename, contentType) -> insertMedia filename (T.pack <$> contentType) (decodeLenient (pack (strContent e)))
207200
208201-- * Type parsers
209202
@@ -213,13 +206,13 @@ parseAuthor e = T.unwords . catMaybes <$> mapM parseAuthorChild (elChildren e)
213206
214207parseAuthorChild :: PandocMonad m => Element -> FB2 m (Maybe Text)
215208parseAuthorChild e =
216− case qName $ elName e of
217− "first-name" -> pure $ Just $ strContent e
218− "middle-name" -> pure $ Just $ strContent e
219− "last-name" -> pure $ Just $ strContent e
220− "nickname" -> pure $ Just $ strContent e
221− "home-page" -> pure $ Just $ strContent e
222− "email" -> pure $ Just $ strContent e
209+ case T.pack $ qName $ elName e of
210+ "first-name" -> pure $ Just $ T.pack $ strContent e
211+ "middle-name" -> pure $ Just $ T.pack $ strContent e
212+ "last-name" -> pure $ Just $ T.pack $ strContent e
213+ "nickname" -> pure $ Just $ T.pack $ strContent e
214+ "home-page" -> pure $ Just $ T.pack $ strContent e
215+ "email" -> pure $ Just $ T.pack $ strContent e
223216 name -> do
224217 report $ IgnoredElement $ name <> " in author"
225218 pure Nothing
@@ -243,13 +236,13 @@ parseTitleContent _ = pure Nothing
243236parseImageElement :: PandocMonad m => Element -> FB2 m Blocks
244237parseImageElement e =
245238 case href of
246− Just src -> pure $ para $ imageWith (imgId, [], []) (removeHash src) title alt
239+ Just src -> pure $ para $ imageWith (imgId, [], []) (removeHash $ T.pack src) title alt
247240 Nothing -> do
248241 report $ IgnoredElement " image without href"
249242 pure mempty
250− where alt = maybe mempty str $ findAttr (unqual "alt") e
251− title = fromMaybe "" $ findAttr (unqual "title") e
252− imgId = fromMaybe "" $ findAttr (unqual "id") e
243+ where alt = maybe mempty (str . T.pack) $ findAttr (unqual "alt") e
244+ title = maybe "" T.pack $ findAttr (unqual "title") e
245+ imgId = maybe "" T.pack $ findAttr (unqual "id") e
253246 href = findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e
254247
255248-- | Parse @pType@
@@ -263,7 +256,7 @@ parseCite e = blockQuote . mconcat <$> mapM parseCiteChild (elChildren e)
263256-- | Parse @citeType@ child
264257parseCiteChild :: PandocMonad m => Element -> FB2 m Blocks
265258parseCiteChild e =
266− case qName $ elName e of
259+ case T.pack $ qName $ elName e of
267260 "p" -> para <$> parsePType e
268261 "poem" -> parsePoem e
269262 "empty-line" -> pure horizontalRule
@@ -278,13 +271,13 @@ parsePoem e = mconcat <$> mapM parsePoemChild (elChildren e)
278271
279272parsePoemChild :: PandocMonad m => Element -> FB2 m Blocks
280273parsePoemChild e =
281− case qName $ elName e of
274+ case T.pack $ qName $ elName e of
282275 "title" -> parseTitle e
283276 "subtitle" -> parseSubtitle e
284277 "epigraph" -> parseEpigraph e
285278 "stanza" -> parseStanza e
286279 "text-author" -> para <$> parsePType e
287− "date" -> pure $ para $ text $ strContent e
280+ "date" -> pure $ para $ text $ T.pack $ strContent e
288281 name -> report (UnexpectedXmlElement name "poem") $> mempty
289282
290283parseStanza :: PandocMonad m => Element -> FB2 m Blocks
@@ -297,7 +290,7 @@ joinLineBlocks [] = []
297290
298291parseStanzaChild :: PandocMonad m => Element -> FB2 m Blocks
299292parseStanzaChild e =
300− case qName $ elName e of
293+ case T.pack $ qName $ elName e of
301294 "title" -> parseTitle e
302295 "subtitle" -> parseSubtitle e
303296 "v" -> lineBlock . (:[]) <$> parsePType e
@@ -307,11 +300,11 @@ parseStanzaChild e =
307300parseEpigraph :: PandocMonad m => Element -> FB2 m Blocks
308301parseEpigraph e =
309302 divWith (divId, ["epigraph"], []) . mconcat <$> mapM parseEpigraphChild (elChildren e)
310− where divId = fromMaybe "" $ findAttr (unqual "id") e
303+ where divId = maybe "" T.pack $ findAttr (unqual "id") e
311304
312305parseEpigraphChild :: PandocMonad m => Element -> FB2 m Blocks
313306parseEpigraphChild e =
314− case qName $ elName e of
307+ case T.pack $ qName $ elName e of
315308 "p" -> para <$> parsePType e
316309 "poem" -> parsePoem e
317310 "cite" -> parseCite e
@@ -325,7 +318,7 @@ parseAnnotation e = mconcat <$> mapM parseAnnotationChild (elChildren e)
325318
326319parseAnnotationChild :: PandocMonad m => Element -> FB2 m Blocks
327320parseAnnotationChild e =
328− case qName $ elName e of
321+ case T.pack $ qName $ elName e of
329322 "p" -> para <$> parsePType e
330323 "poem" -> parsePoem e
331324 "cite" -> parseCite e
@@ -339,14 +332,14 @@ parseSection :: PandocMonad m => Element -> FB2 m Blocks
339332parseSection e = do
340333 n <- gets fb2SectionLevel
341334 modify $ \st -> st{ fb2SectionLevel = n + 1 }
342− let sectionId = fromMaybe "" $ findAttr (unqual "id") e
335+ let sectionId = maybe "" T.pack $ findAttr (unqual "id") e
343336 bs <- divWith (sectionId, ["section"], []) . mconcat <$> mapM parseSectionChild (elChildren e)
344337 modify $ \st -> st{ fb2SectionLevel = n }
345338 pure bs
346339
347340parseSectionChild :: PandocMonad m => Element -> FB2 m Blocks
348341parseSectionChild e =
349− case qName $ elName e of
342+ case T.pack $ qName $ elName e of
350343 "title" -> parseBodyChild e
351344 "epigraph" -> parseEpigraph e
352345 "image" -> parseImageElement e
@@ -368,16 +361,16 @@ parseStyleType e = mconcat <$> mapM parseInline (elContent e)
368361parseNamedStyle :: PandocMonad m => Element -> FB2 m Inlines
369362parseNamedStyle e = do
370363 content <- mconcat <$> mapM parseNamedStyleChild (elContent e)
371− let lang = maybeToList $ ("lang",) <$> findAttr (QName "lang" Nothing (Just "xml")) e
364+ let lang = maybeToList $ ("lang",) . T.pack <$> findAttr (QName "lang" Nothing (Just "xml")) e
372365 case findAttr (unqual "name") e of
373− Just name -> pure $ spanWith ("", [name], lang) content
366+ Just name -> pure $ spanWith ("", [T.pack name], lang) content
374367 Nothing -> do
375368 report $ IgnoredElement "link without required name"
376369 pure mempty
377370
378371parseNamedStyleChild :: PandocMonad m => Content -> FB2 m Inlines
379372parseNamedStyleChild (Elem e) =
380− case qName (elName e) of
373+ case T.pack $ qName (elName e) of
381374 "strong" -> strong <$> parseStyleType e
382375 "emphasis" -> emph <$> parseStyleType e
383376 "style" -> parseNamedStyle e
@@ -385,7 +378,7 @@ parseNamedStyleChild (Elem e) =
385378 "strikethrough" -> strikeout <$> parseStyleType e
386379 "sub" -> subscript <$> parseStyleType e
387380 "sup" -> superscript <$> parseStyleType e
388− "code" -> pure $ code $ strContent e
381+ "code" -> pure $ code $ T.pack $ strContent e
389382 "image" -> parseInlineImageElement e
390383 name -> do
391384 report $ IgnoredElement $ name <> " in style"
@@ -397,7 +390,7 @@ parseLinkType :: PandocMonad m => Element -> FB2 m Inlines
397390parseLinkType e = do
398391 content <- mconcat <$> mapM parseStyleLinkType (elContent e)
399392 notes <- gets fb2Notes
400− case findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e of
393+ case T.pack <$> findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e of
401394 Just href -> case findAttr (QName "type" Nothing Nothing) e of
402395 Just "note" -> case M.lookup href notes of
403396 Nothing -> pure $ link href "" content
@@ -424,14 +417,15 @@ parseTable _ = pure mempty -- TODO: tables are not supported yet
424417-- | Parse @title-infoType@
425418parseTitleInfoChild :: PandocMonad m => Element -> FB2 m ()
426419parseTitleInfoChild e =
427− case qName (elName e) of
420+ case T.pack $ qName (elName e) of
428421 "genre" -> pure ()
429422 "author" -> parseAuthor e >>= \author -> modify (\st -> st {fb2Authors = author:fb2Authors st})
430− "book-title" -> modify (setMeta "title" (text $ strContent e))
423+ "book-title" -> modify (setMeta "title" (text $ T.pack $ strContent e))
431424 "annotation" -> parseAnnotation e >>= modify . setMeta "abstract"
432425 "keywords" -> modify (setMeta "keywords" (map (MetaString . trim) $ T.splitOn ","
426+ $ T.pack
433427 $ strContent e))
434− "date" -> modify (setMeta "date" (text $ strContent e))
428+ "date" -> modify (setMeta "date" (text $ T.pack $ strContent e))
435429 "coverpage" -> parseCoverPage e
436430 "lang" -> pure ()
437431 "src-lang" -> pure ()
@@ -445,7 +439,7 @@ parseCoverPage e =
445439 Just img -> case href of
446440 Just src -> modify (setMeta "cover-image" (MetaString $ removeHash src))
447441 Nothing -> pure ()
448− where href = findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) img
442+ where href = T.pack <$> findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) img
449443 Nothing -> pure ()
450444
451445-- | Parse @inlineImageType@ element
@@ -458,5 +452,5 @@ parseInlineImageElement e =
458452 Nothing -> do
459453 report $ IgnoredElement "inline image without href"
460454 pure mempty
461− where alt = maybe mempty str $ findAttr (unqual "alt") e
462− href = findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e
455+ where alt = maybe mempty (str . T.pack) $ findAttr (unqual "alt") e
456+ href = T.pack <$> findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e
src/Text/Pandoc/Readers/HTML.hs+216−222
@@ -5,7 +5,7 @@
55{-# LANGUAGE ViewPatterns #-}
66{- |
77 Module : Text.Pandoc.Readers.HTML
8− Copyright : Copyright (C) 2006-2021 John MacFarlane
8+ Copyright : Copyright (C) 2006-2020 John MacFarlane
99 License : GNU GPL, version 2 or above
1010
1111 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -19,20 +19,21 @@ module Text.Pandoc.Readers.HTML ( readHtml
1919 , htmlInBalanced
2020 , isInlineTag
2121 , isBlockTag
22+ , NamedTag(..)
2223 , isTextTag
2324 , isCommentTag
2425 ) where
2526
2627import Control.Applicative ((<|>))
27−import Control.Monad (guard, msum, mzero, unless, void)
28−import Control.Monad.Except (throwError, catchError)
28+import Control.Arrow (first)
29+import Control.Monad (guard, mplus, msum, mzero, unless, void)
30+import Control.Monad.Except (throwError)
2931import Control.Monad.Reader (ask, asks, lift, local, runReaderT)
3032import Data.ByteString.Base64 (encode)
3133import Data.Char (isAlphaNum, isLetter)
3234import Data.Default (Default (..), def)
3335import Data.Foldable (for_)
3436import Data.List.Split (splitWhen)
35−import Data.List (foldl')
3637import qualified Data.Map as M
3738import Data.Maybe (fromMaybe, isJust, isNothing)
3839import Data.Monoid (First (..))
@@ -49,7 +50,7 @@ import Text.Pandoc.CSS (pickStyleAttrProps)
4950import qualified Text.Pandoc.UTF8 as UTF8
5051import Text.Pandoc.Definition
5152import Text.Pandoc.Readers.HTML.Parsing
52−import Text.Pandoc.Readers.HTML.Table (pTable)
53+import Text.Pandoc.Readers.HTML.Table (pTable')
5354import Text.Pandoc.Readers.HTML.TagCategories
5455import Text.Pandoc.Readers.HTML.Types
5556import Text.Pandoc.Readers.LaTeX (rawLaTeXInline)
@@ -61,22 +62,22 @@ import Text.Pandoc.Options (
6162 ReaderOptions (readerExtensions, readerStripComments),
6263 extensionEnabled)
6364import Text.Pandoc.Parsing hiding ((<|>))
64−import Text.Pandoc.Shared (
65− addMetaField, blocksToInlines', escapeURI, extractSpaces,
66− htmlSpanLikeElements, renderTags', safeRead, tshow)
65+import Text.Pandoc.Shared (addMetaField, blocksToInlines', crFilter, escapeURI,
66+ extractSpaces, htmlSpanLikeElements, elemText, splitTextBy,
67+ safeRead, tshow)
6768import Text.Pandoc.Walk
6869import Text.Parsec.Error
6970import Text.TeXMath (readMathML, writeTeX)
7071
7172-- | Convert HTML-formatted string to 'Pandoc' document.
72−readHtml :: (PandocMonad m, ToSources a)
73+readHtml :: PandocMonad m
7374 => ReaderOptions -- ^ Reader options
74− -> a -- ^ Input to parse
75+ -> Text -- ^ String to parse (assumes @'\n'@ line endings)
7576 -> m Pandoc
7677readHtml opts inp = do
77− let tags = stripPrefixes $ canonicalizeTags $
78+ let tags = stripPrefixes . canonicalizeTags $
7879 parseTagsOptions parseOptions{ optTagPosition = True }
79− (sourcesToText $ toSources inp)
80+ (crFilter inp)
8081 parseDoc = do
8182 blocks <- fixPlains False . mconcat <$> manyTill block eof
8283 meta <- stateMeta . parserState <$> getState
@@ -95,15 +96,6 @@ readHtml opts inp = do
9596 Right doc -> return doc
9697 Left err -> throwError $ PandocParseError $ T.pack $ getError err
9798
98−-- Strip namespace prefixes on tags (not attributes)
99−stripPrefixes :: [Tag Text] -> [Tag Text]
100−stripPrefixes = map stripPrefix
101−
102−stripPrefix :: Tag Text -> Tag Text
103−stripPrefix (TagOpen s as) = TagOpen (T.takeWhileEnd (/=':') s) as
104−stripPrefix (TagClose s) = TagClose (T.takeWhileEnd (/=':') s)
105−stripPrefix x = x
106−
10799replaceNotes :: PandocMonad m => [Block] -> TagParser m [Block]
108100replaceNotes bs = do
109101 st <- getState
@@ -121,18 +113,14 @@ setInPlain :: PandocMonad m => HTMLParser m s a -> HTMLParser m s a
121113setInPlain = local (\s -> s {inPlain = True})
122114
123115pHtml :: PandocMonad m => TagParser m Blocks
124−pHtml = do
116+pHtml = try $ do
125117 (TagOpen "html" attr) <- lookAhead pAny
126− for_ (lookup "lang" attr <|> lookup "xml:lang" attr) $
118+ for_ (lookup "lang" attr) $
127119 updateState . B.setMeta "lang" . B.text
128120 pInTags "html" block
129121
130122pBody :: PandocMonad m => TagParser m Blocks
131−pBody = do
132− (TagOpen "body" attr) <- lookAhead pAny
133− for_ (lookup "lang" attr <|> lookup "xml:lang" attr) $
134− updateState . B.setMeta "lang" . B.text
135− pInTags "body" block
123+pBody = pInTags "body" block
136124
137125pHead :: PandocMonad m => TagParser m Blocks
138126pHead = pInTags "head" $ pTitle <|> pMetaTag <|> pBaseTag <|> (mempty <$ pAny)
@@ -158,65 +146,32 @@ pHead = pInTags "head" $ pTitle <|> pMetaTag <|> pBaseTag <|> (mempty <$ pAny)
158146 return mempty
159147
160148block :: PandocMonad m => TagParser m Blocks
161−block = ((do
162− tag <- lookAhead (pSatisfy isBlockTag)
163− exts <- getOption readerExtensions
164− case tag of
165− TagOpen name attr ->
166− let type' = fromMaybe "" $
167− lookup "type" attr <|> lookup "epub:type" attr
168− epubExts = extensionEnabled Ext_epub_html_exts exts
169− in
170− case name of
171− _ | name `elem` sectioningContent
172− , epubExts
173− , "chapter" `T.isInfixOf` type'
174− -> eSection
175− _ | epubExts
176− , type' `elem` ["footnote", "rearnote"]
177− -> mempty <$ eFootnote
178− _ | epubExts
179− , type' == "toc"
180− -> mempty <$ eTOC
181− _ | "titlepage" `T.isInfixOf` type'
182− , name `elem` ("section" : groupingContent)
183− -> mempty <$ eTitlePage
184− "p" -> pPara
185− "h1" -> pHeader
186− "h2" -> pHeader
187− "h3" -> pHeader
188− "h4" -> pHeader
189− "h5" -> pHeader
190− "h6" -> pHeader
191− "blockquote" -> pBlockQuote
192− "pre" -> pCodeBlock
193− "ul" -> pBulletList
194− "ol" -> pOrderedList
195− "dl" -> pDefinitionList
196− "table" -> pTable block
197− "hr" -> pHrule
198− "html" -> pHtml
199− "head" -> pHead
200− "body" -> pBody
201− "div"
202− | extensionEnabled Ext_line_blocks exts
203− , Just "line-block" <- lookup "class" attr
204− -> pLineBlock
205− | otherwise
206− -> pDiv
207− "section" -> pDiv
208− "header" -> pDiv
209− "main" -> pDiv
210− "figure" -> pFigure
211− "iframe" -> pIframe
212− "style" -> pRawHtmlBlock
213− "textarea" -> pRawHtmlBlock
214− "switch"
215− | epubExts
216− -> eSwitch B.para block
217− _ -> mzero
218− _ -> mzero) <|> pPlain <|> pRawHtmlBlock) >>= \res ->
219− res <$ trace (T.take 60 $ tshow $ B.toList res)
149+block = do
150+ res <- choice
151+ [ eSection
152+ , eSwitch B.para block
153+ , mempty <$ eFootnote
154+ , mempty <$ eTOC
155+ , mempty <$ eTitlePage
156+ , pPara
157+ , pHeader
158+ , pBlockQuote
159+ , pCodeBlock
160+ , pList
161+ , pHrule
162+ , pTable
163+ , pHtml
164+ , pHead
165+ , pBody
166+ , pLineBlock
167+ , pDiv
168+ , pPlain
169+ , pFigure
170+ , pIframe
171+ , pRawHtmlBlock
172+ ]
173+ trace (T.take 60 $ tshow $ B.toList res)
174+ return res
220175
221176namespaces :: PandocMonad m => [(Text, TagParser m Inlines)]
222177namespaces = [(mathMLNamespace, pMath True)]
@@ -289,6 +244,9 @@ eTOC = try $ do
289244 guard $ (lookup "type" attr <|> lookup "epub:type" attr) == Just "toc"
290245 void (pInTags tag block)
291246
247+pList :: PandocMonad m => TagParser m Blocks
248+pList = pBulletList <|> pOrderedList <|> pDefinitionList
249+
292250pBulletList :: PandocMonad m => TagParser m Blocks
293251pBulletList = try $ do
294252 pSatisfy (matchTagOpen "ul" [])
@@ -362,10 +320,7 @@ pDefListItem = try $ do
362320 terms <- many1 (try $ skipMany nonItem >> pInTags "dt" inline)
363321 defs <- many1 (try $ skipMany nonItem >> pInTags "dd" block)
364322 skipMany nonItem
365− let term = foldl' (\x y -> if null x
366− then trimInlines y
367− else x <> B.linebreak <> trimInlines y)
368− mempty terms
323+ let term = foldl1 (\x y -> x <> B.linebreak <> y) $ map trimInlines terms
369324 return (term, map (fixPlains True) defs)
370325
371326fixPlains :: Bool -> Blocks -> Blocks
@@ -402,16 +357,13 @@ pLineBlock = try $ do
402357 B.toList ils
403358 return $ B.lineBlock lns
404359
405−isDivLike :: Text -> Bool
406−isDivLike "div" = True
407−isDivLike "section" = True
408−isDivLike "header" = True
409−isDivLike "main" = True
410−isDivLike _ = False
411−
412360pDiv :: PandocMonad m => TagParser m Blocks
413361pDiv = try $ do
414362 guardEnabled Ext_native_divs
363+ let isDivLike "div" = True
364+ isDivLike "section" = True
365+ isDivLike "main" = True
366+ isDivLike _ = False
415367 TagOpen tag attr' <- lookAhead $ pSatisfy $ tagOpen isDivLike (const True)
416368 let (ident, classes, kvs) = toAttr attr'
417369 contents <- pInTags tag block
@@ -429,17 +381,11 @@ pIframe = try $ do
429381 tag <- pSatisfy (tagOpen (=="iframe") (isJust . lookup "src"))
430382 pCloses "iframe" <|> eof
431383 url <- canonicalizeUrl $ fromAttrib "src" tag
432− if T.null url
433− then ignore $ renderTags' [tag, TagClose "iframe"]
434− else catchError
435− (do (bs, _) <- openURL url
436− let inp = UTF8.toText bs
437− opts <- readerOpts <$> getState
438− Pandoc _ contents <- readHtml opts inp
439− return $ B.divWith ("",["iframe"],[]) $ B.fromList contents)
440− (\e -> do
441− logMessage $ CouldNotFetchResource url (renderError e)
442− ignore $ renderTags' [tag, TagClose "iframe"])
384+ (bs, _) <- openURL url
385+ let inp = UTF8.toText bs
386+ opts <- readerOpts <$> getState
387+ Pandoc _ contents <- readHtml opts inp
388+ return $ B.divWith ("",["iframe"],[]) $ B.fromList contents
443389
444390pRawHtmlBlock :: PandocMonad m => TagParser m Blocks
445391pRawHtmlBlock = do
@@ -501,19 +447,48 @@ pHeader = try $ do
501447 tagOpen (`elem` ["h1","h2","h3","h4","h5","h6"])
502448 (const True)
503449 let attr = toStringAttr attr'
450+ let bodyTitle = TagOpen tagtype attr' ~== TagOpen ("h1" :: Text)
451+ [("class","title")]
504452 level <- headerLevel tagtype
505453 contents <- trimInlines . mconcat <$> manyTill inline (pCloses tagtype <|> eof)
506454 let ident = fromMaybe "" $ lookup "id" attr
507455 let classes = maybe [] T.words $ lookup "class" attr
508456 let keyvals = [(k,v) | (k,v) <- attr, k /= "class", k /= "id"]
509457 attr'' <- registerHeader (ident, classes, keyvals) contents
510− return $ B.headerWith attr'' level contents
458+ return $ if bodyTitle
459+ then mempty -- skip a representation of the title in the body
460+ else B.headerWith attr'' level contents
511461
512462pHrule :: PandocMonad m => TagParser m Blocks
513463pHrule = do
514464 pSelfClosing (=="hr") (const True)
515465 return B.horizontalRule
516466
467+pTable :: PandocMonad m => TagParser m Blocks
468+pTable = pTable' block pCell
469+
470+pCell :: PandocMonad m => Text -> TagParser m [Cell]
471+pCell celltype = try $ do
472+ skipMany pBlank
473+ tag <- lookAhead $ pSatisfy (\t -> t ~== TagOpen celltype [])
474+ let extractAlign' [] = ""
475+ extractAlign' ("text-align":x:_) = x
476+ extractAlign' (_:xs) = extractAlign' xs
477+ let extractAlign = extractAlign' . splitTextBy (`elemText` " \t;:")
478+ let align = case maybeFromAttrib "align" tag `mplus`
479+ (extractAlign <$> maybeFromAttrib "style" tag) of
480+ Just "left" -> AlignLeft
481+ Just "right" -> AlignRight
482+ Just "center" -> AlignCenter
483+ _ -> AlignDefault
484+ let rowspan = RowSpan . fromMaybe 1 $
485+ safeRead =<< maybeFromAttrib "rowspan" tag
486+ let colspan = ColSpan . fromMaybe 1 $
487+ safeRead =<< maybeFromAttrib "colspan" tag
488+ res <- pInTags celltype block
489+ skipMany pBlank
490+ return [B.cell align rowspan colspan res]
491+
517492pBlockQuote :: PandocMonad m => TagParser m Blocks
518493pBlockQuote = do
519494 contents <- pInTags "blockquote" block
@@ -539,7 +514,7 @@ pFigure = try $ do
539514 TagOpen _ _ <- pSatisfy (matchTagOpen "figure" [])
540515 skipMany pBlank
541516 let pImg = (\x -> (Just x, Nothing)) <$>
542− (pInTag TagsOmittable "p" pImage <* skipMany pBlank)
517+ (pInTag True "p" pImage <* skipMany pBlank)
543518 pCapt = (\x -> (Nothing, Just x)) <$> do
544519 bs <- pInTags "figcaption" block
545520 return $ blocksToInlines' $ B.toList bs
@@ -557,18 +532,7 @@ pFigure = try $ do
557532pCodeBlock :: PandocMonad m => TagParser m Blocks
558533pCodeBlock = try $ do
559534 TagOpen _ attr' <- pSatisfy (matchTagOpen "pre" [])
560− -- if the `pre` has no attributes, try if it is followed by a `code`
561− -- element and use those attributes if possible.
562− attr <- case attr' of
563− _:_ -> pure (toAttr attr')
564− [] -> option nullAttr $ do
565− TagOpen _ codeAttr <- pSatisfy (matchTagOpen "code" [])
566− pure $ toAttr
567− [ (k, v') | (k, v) <- codeAttr
568− -- strip language from class
569− , let v' = if k == "class"
570− then fromMaybe v (T.stripPrefix "language-" v)
571− else v ]
535+ let attr = toAttr attr'
572536 contents <- manyTill pAny (pCloses "pre" <|> eof)
573537 let rawText = T.concat $ map tagToText contents
574538 -- drop leading newline if any
@@ -587,47 +551,31 @@ tagToText (TagOpen "br" _) = "\n"
587551tagToText _ = ""
588552
589553inline :: PandocMonad m => TagParser m Inlines
590−inline = pTagText <|> do
591− tag <- lookAhead (pSatisfy isInlineTag)
592− exts <- getOption readerExtensions
593− case tag of
594− TagOpen name attr ->
595− case name of
596− "a" | extensionEnabled Ext_epub_html_exts exts
597− , Just "noteref" <- lookup "type" attr <|> lookup "epub:type" attr
598− , Just ('#',_) <- lookup "href" attr >>= T.uncons
599− -> eNoteref
600− | otherwise -> pLink
601− "switch" -> eSwitch id inline
602− "q" -> pQ
603− "em" -> pEmph
604− "i" -> pEmph
605− "strong" -> pStrong
606− "b" -> pStrong
607− "sup" -> pSuperscript
608− "sub" -> pSubscript
609− "small" -> pSmall
610− "s" -> pStrikeout
611− "strike" -> pStrikeout
612− "del" -> pStrikeout
613− "u" -> pUnderline
614− "ins" -> pUnderline
615− "br" -> pLineBreak
616− "img" -> pImage
617− "svg" -> pSvg
618− "bdo" -> pBdo
619− "code" -> pCode
620− "samp" -> pCodeWithClass "samp" "sample"
621− "var" -> pCodeWithClass "var" "variable"
622− "span" -> pSpan
623− "math" -> pMath False
624− "script"
625− | Just x <- lookup "type" attr
626− , "math/tex" `T.isPrefixOf` x -> pScriptMath
627− _ | name `elem` htmlSpanLikeElements -> pSpanLike
628− _ -> pRawHtmlInline
629− TagText _ -> pTagText
630− _ -> pRawHtmlInline
554+inline = choice
555+ [ eNoteref
556+ , eSwitch id inline
557+ , pTagText
558+ , pQ
559+ , pEmph
560+ , pStrong
561+ , pSuperscript
562+ , pSubscript
563+ , pSpanLike
564+ , pSmall
565+ , pStrikeout
566+ , pUnderline
567+ , pLineBreak
568+ , pLink
569+ , pImage
570+ , pSvg
571+ , pBdo
572+ , pCode
573+ , pCodeWithClass [("samp","sample"),("var","variable")]
574+ , pSpan
575+ , pMath False
576+ , pScriptMath
577+ , pRawHtmlInline
578+ ]
631579
632580pSelfClosing :: PandocMonad m
633581 => (Text -> Bool) -> ([Attribute Text] -> Bool)
@@ -638,25 +586,27 @@ pSelfClosing f g = do
638586 return open
639587
640588pQ :: PandocMonad m => TagParser m Inlines
641−pQ = do
642− TagOpen _ attrs <- pSatisfy $ tagOpenLit "q" (const True)
643− case lookup "cite" attrs of
644− Just url -> do
645− let uid = fromMaybe mempty $
646− lookup "name" attrs <> lookup "id" attrs
647− let cls = maybe [] T.words $ lookup "class" attrs
648− url' <- canonicalizeUrl url
649− makeQuote $ B.spanWith (uid, cls, [("cite", escapeURI url')])
650− Nothing -> makeQuote id
651− where
652− makeQuote wrapper = do
653− ctx <- asks quoteContext
654− let (constructor, innerContext) = case ctx of
655− InDoubleQuote -> (B.singleQuoted, InSingleQuote)
656− _ -> (B.doubleQuoted, InDoubleQuote)
657− content <- withQuoteContext innerContext
658− (mconcat <$> manyTill inline (pCloses "q"))
659− return $ extractSpaces (constructor . wrapper) content
589+pQ = choice $ map try [citedQuote, normalQuote]
590+ where citedQuote = do
591+ tag <- pSatisfy $ tagOpenLit "q" (any ((=="cite") . fst))
592+
593+ url <- canonicalizeUrl $ fromAttrib "cite" tag
594+ let uid = fromMaybe (fromAttrib "name" tag) $
595+ maybeFromAttrib "id" tag
596+ let cls = T.words $ fromAttrib "class" tag
597+
598+ makeQuote $ B.spanWith (uid, cls, [("cite", escapeURI url)])
599+ normalQuote = do
600+ pSatisfy $ tagOpenLit "q" (const True)
601+ makeQuote id
602+ makeQuote wrapper = do
603+ ctx <- asks quoteContext
604+ let (constructor, innerContext) = case ctx of
605+ InDoubleQuote -> (B.singleQuoted, InSingleQuote)
606+ _ -> (B.doubleQuoted, InDoubleQuote)
607+
608+ content <- withQuoteContext innerContext (mconcat <$> manyTill inline (pCloses "q"))
609+ return $ extractSpaces (constructor . wrapper) content
660610
661611pEmph :: PandocMonad m => TagParser m Inlines
662612pEmph = pInlinesInTags "em" B.emph <|> pInlinesInTags "i" B.emph
@@ -703,6 +653,12 @@ pLineBreak = do
703653 pSelfClosing (=="br") (const True)
704654 return B.linebreak
705655
656+-- Unlike fromAttrib from tagsoup, this distinguishes
657+-- between a missing attribute and an attribute with empty content.
658+maybeFromAttrib :: Text -> Tag Text -> Maybe Text
659+maybeFromAttrib name (TagOpen _ attrs) = lookup name attrs
660+maybeFromAttrib _ _ = Nothing
661+
706662pLink :: PandocMonad m => TagParser m Inlines
707663pLink = try $ do
708664 tag <- pSatisfy $ tagOpenLit "a" (const True)
@@ -747,12 +703,13 @@ pSvg = do
747703 UTF8.toText (encode $ UTF8.fromText rawText)
748704 return $ B.imageWith (ident,cls,[]) svgData mempty mempty
749705
750−pCodeWithClass :: PandocMonad m => Text -> Text -> TagParser m Inlines
751−pCodeWithClass name class' = try $ do
752− TagOpen open attr' <- pSatisfy $ tagOpen (== name) (const True)
706+pCodeWithClass :: PandocMonad m => [(T.Text,Text)] -> TagParser m Inlines
707+pCodeWithClass elemToClass = try $ do
708+ let tagTest = flip elem . fmap fst $ elemToClass
709+ TagOpen open attr' <- pSatisfy $ tagOpen tagTest (const True)
753710 result <- manyTill pAny (pCloses open)
754711 let (ids,cs,kvs) = toAttr attr'
755− cs' = class' : cs
712+ cs' = maybe cs (:cs) . lookup open $ elemToClass
756713 return . B.codeWith (ids,cs',kvs) .
757714 T.unwords . T.lines . innerText $ result
758715
@@ -839,19 +796,17 @@ pInlinesInTags tagtype f = extractSpaces f <$> pInTags tagtype inline
839796
840797pTagText :: PandocMonad m => TagParser m Inlines
841798pTagText = try $ do
842− pos <- getPosition
843799 (TagText str) <- pSatisfy isTagText
844800 st <- getState
845801 qu <- ask
846802 parsed <- lift $ lift $
847− flip runReaderT qu $ runParserT (many pTagContents) st "text"
848− (Sources [(pos, str)])
803+ flip runReaderT qu $ runParserT (many pTagContents) st "text" str
849804 case parsed of
850805 Left _ -> throwError $ PandocParseError $
851806 "Could not parse `" <> str <> "'"
852807 Right result -> return $ mconcat result
853808
854−type InlinesParser m = HTMLParser m Sources
809+type InlinesParser m = HTMLParser m Text
855810
856811pTagContents :: PandocMonad m => InlinesParser m Inlines
857812pTagContents =
@@ -945,23 +900,27 @@ pSpace = many1 (satisfy isSpace) >>= \xs ->
945900 then return B.softbreak
946901 else return B.space
947902
948−getTagName :: Tag Text -> Maybe Text
949−getTagName (TagOpen t _) = Just t
950−getTagName (TagClose t) = Just t
951−getTagName _ = Nothing
952−
953−isInlineTag :: Tag Text -> Bool
954−isInlineTag t = isCommentTag t || case t of
955− TagOpen "script" _ -> "math/tex" `T.isPrefixOf` fromAttrib "type" t
956− TagClose "script" -> True
957− TagOpen name _ -> isInlineTagName name
958− TagClose name -> isInlineTagName name
959− _ -> False
960− where isInlineTagName x =
961− x `Set.notMember` blockTags ||
962− T.take 1 x == "?" -- processing instr.
963−
964−isBlockTag :: Tag Text -> Bool
903+class NamedTag a where
904+ getTagName :: a -> Maybe Text
905+
906+instance NamedTag (Tag Text) where
907+ getTagName (TagOpen t _) = Just t
908+ getTagName (TagClose t) = Just t
909+ getTagName _ = Nothing
910+
911+instance NamedTag (Tag String) where
912+ getTagName (TagOpen t _) = Just (T.pack t)
913+ getTagName (TagClose t) = Just (T.pack t)
914+ getTagName _ = Nothing
915+
916+isInlineTag :: NamedTag (Tag a) => Tag a -> Bool
917+isInlineTag t =
918+ isCommentTag t || case getTagName t of
919+ Nothing -> False
920+ Just x -> x `Set.notMember` blockTags ||
921+ T.take 1 x == "?" -- processing instr.
922+
923+isBlockTag :: NamedTag (Tag a) => Tag a -> Bool
965924isBlockTag t = isBlockTagName || isTagComment t
966925 where isBlockTagName =
967926 case getTagName t of
@@ -972,10 +931,10 @@ isBlockTag t = isBlockTagName || isTagComment t
972931 || x `Set.member` eitherBlockOrInline
973932 Nothing -> False
974933
975−isTextTag :: Tag Text -> Bool
934+isTextTag :: Tag a -> Bool
976935isTextTag = tagText (const True)
977936
978−isCommentTag :: Tag Text -> Bool
937+isCommentTag :: Tag a -> Bool
979938isCommentTag = tagComment (const True)
980939
981940--- parsers for use in markdown, textile readers
@@ -983,14 +942,13 @@ isCommentTag = tagComment (const True)
983942-- | Matches a stretch of HTML in balanced tags.
984943htmlInBalanced :: Monad m
985944 => (Tag Text -> Bool)
986− -> ParserT Sources st m Text
945+ -> ParserT Text st m Text
987946htmlInBalanced f = try $ do
988947 lookAhead (char '<')
989− sources <- getInput
990− let ts = canonicalizeTags
991− $ parseTagsOptions parseOptions{ optTagWarning = True,
992− optTagPosition = True }
993− $ sourcesToText sources
948+ inp <- getInput
949+ let ts = canonicalizeTags $
950+ parseTagsOptions parseOptions{ optTagWarning = True,
951+ optTagPosition = True } inp
994952 case ts of
995953 (TagPosition sr sc : t@(TagOpen tn _) : rest) -> do
996954 guard $ f t
@@ -1025,24 +983,22 @@ htmlInBalanced' tagname ts = fromMaybe [] $ go 0 ts
1025983 go n (t:ts') = (t :) <$> go n ts'
1026984 go _ [] = mzero
1027985
1028−hasTagWarning :: [Tag Text] -> Bool
986+hasTagWarning :: [Tag a] -> Bool
1029987hasTagWarning (TagWarning _:_) = True
1030988hasTagWarning _ = False
1031989
1032990-- | Matches a tag meeting a certain condition.
1033991htmlTag :: (HasReaderOptions st, Monad m)
1034992 => (Tag Text -> Bool)
1035− -> ParserT Sources st m (Tag Text, Text)
993+ -> ParserT Text st m (Tag Text, Text)
1036994htmlTag f = try $ do
1037995 lookAhead (char '<')
1038996 startpos <- getPosition
1039− sources <- getInput
1040− let inp = sourcesToText sources
997+ inp <- getInput
1041998 let ts = canonicalizeTags $ parseTagsOptions
1042999 parseOptions{ optTagWarning = False
10431000 , optTagPosition = True }
1044− (inp <> " ")
1045− -- add space to ensure that
1001+ (inp <> " ") -- add space to ensure that
10461002 -- we get a TagPosition after the tag
10471003 (next, ln, col) <- case ts of
10481004 (TagPosition{} : next : TagPosition ln col : _)
@@ -1100,6 +1056,21 @@ htmlTag f = try $ do
11001056 handleTag tagname
11011057 _ -> mzero
11021058
1059+-- Strip namespace prefixes
1060+stripPrefixes :: [Tag Text] -> [Tag Text]
1061+stripPrefixes = map stripPrefix
1062+
1063+stripPrefix :: Tag Text -> Tag Text
1064+stripPrefix (TagOpen s as) =
1065+ TagOpen (stripPrefix' s) (map (first stripPrefix') as)
1066+stripPrefix (TagClose s) = TagClose (stripPrefix' s)
1067+stripPrefix x = x
1068+
1069+stripPrefix' :: Text -> Text
1070+stripPrefix' s =
1071+ if T.null t then s else T.drop 1 t
1072+ where (_, t) = T.span (/= ':') s
1073+
11031074-- Utilities
11041075
11051076-- | Adjusts a url according to the document's base URL.
@@ -1109,3 +1080,26 @@ canonicalizeUrl url = do
11091080 return $ case (parseURIReference (T.unpack url), mbBaseHref) of
11101081 (Just rel, Just bs) -> tshow (rel `nonStrictRelativeTo` bs)
11111082 _ -> url
1083+
1084+-- For now we need a special version here; the one in Shared has String type
1085+renderTags' :: [Tag Text] -> Text
1086+renderTags' = renderTagsOptions
1087+ renderOptions{ optMinimize = matchTags ["hr", "br", "img",
1088+ "meta", "link"]
1089+ , optRawTag = matchTags ["script", "style"] }
1090+ where matchTags tags = flip elem tags . T.toLower
1091+
1092+
1093+-- EPUB Specific
1094+--
1095+--
1096+{-
1097+
1098+types :: [(String, ([String], Int))]
1099+types = -- Document divisions
1100+ map (\s -> (s, (["section", "body"], 0)))
1101+ ["volume", "part", "chapter", "division"]
1102+ <> -- Document section and components
1103+ [
1104+ ("abstract", ([], 0))]
1105+-}
src/Text/Pandoc/Readers/HTML/Parsing.hs+19−62
@@ -1,8 +1,7 @@
1−{-# LANGUAGE LambdaCase #-}
21{-# LANGUAGE OverloadedStrings #-}
32{- |
43 Module : Text.Pandoc.Readers.HTML.Parsing
5− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
65 License : GNU GPL, version 2 or above
76
87 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -12,11 +11,9 @@
1211Parsing functions and utilities.
1312-}
1413module Text.Pandoc.Readers.HTML.Parsing
15− ( TagOmission (..)
16− , pInTags
14+ ( pInTags
1715 , pInTags'
1816 , pInTag
19− , pInTagWithAttribs
2017 , pAny
2118 , pCloses
2219 , pSatisfy
@@ -24,7 +21,6 @@ module Text.Pandoc.Readers.HTML.Parsing
2421 , matchTagClose
2522 , matchTagOpen
2623 , isSpace
27− , maybeFromAttrib
2824 , toAttr
2925 , toStringAttr
3026 )
@@ -34,11 +30,11 @@ import Control.Monad (guard, void, mzero)
3430import Data.Maybe (fromMaybe)
3531import Data.Text (Text)
3632import Text.HTML.TagSoup
37− ( Attribute, Tag (..), isTagText, isTagPosition, isTagOpen, isTagClose, (~==) )
33+ ( Tag (..), (~==), isTagText, isTagPosition, isTagOpen, isTagClose )
3834import Text.Pandoc.Class.PandocMonad (PandocMonad (..))
3935import Text.Pandoc.Definition (Attr)
4036import Text.Pandoc.Parsing
41− ( (<|>), eof, getPosition, lookAhead, manyTill, newPos, option, optional
37+ ( (<|>), eof, getPosition, lookAhead, manyTill, newPos, optional
4238 , skipMany, setPosition, token, try)
4339import Text.Pandoc.Readers.HTML.TagCategories
4440import Text.Pandoc.Readers.HTML.Types
@@ -47,13 +43,6 @@ import Text.Pandoc.XML (html5Attributes, html4Attributes, rdfaAttributes)
4743import qualified Data.Set as Set
4844import qualified Data.Text as T
4945
50−-- | Whether no tag, the closing tag, or both tags can be omitted.
51−data TagOmission
52− = TagsRequired -- ^ Opening and closing tags are both required
53− | ClosingTagOptional -- ^ The closing tag can be omitted
54− | TagsOmittable -- ^ Both tags, opening and closing, can be omitted.
55− deriving (Eq)
56−
5746pInTags :: (PandocMonad m, Monoid a) => Text -> TagParser m a -> TagParser m a
5847pInTags tagtype parser = pInTags' tagtype (const True) parser
5948
@@ -63,41 +52,21 @@ pInTags' :: (PandocMonad m, Monoid a)
6352 -> TagParser m a
6453 -> TagParser m a
6554pInTags' tagtype tagtest parser = try $ do
66− pSatisfy $ \t -> matchTagOpen tagtype [] t && tagtest t
55+ pSatisfy (\t -> t ~== TagOpen tagtype [] && tagtest t)
6756 mconcat <$> manyTill parser (pCloses tagtype <|> eof)
6857
69−pInTag :: PandocMonad m
70− => TagOmission -- ^ Whether some tags can be omitted
71− -> Text -- ^ @tagtype@ Tag name
72− -> TagParser m a -- ^ @p@ Content parser
73− -> TagParser m a
74−pInTag tagOmission tagtype = fmap snd . pInTagWithAttribs tagOmission tagtype
75−
76−-- | Returns the contents of a tag together with its attributes; parses
77−-- @p@, preceded by an opening tag (optional if TagsOmittable) and
78−-- followed by a closing tag (optional unless TagsRequired).
79−pInTagWithAttribs :: PandocMonad m
80− => TagOmission -- ^ Whether some tags can be omitted
81− -> Text -- ^ @tagtype@ Tag name
82− -> TagParser m a -- ^ @p@ Content parser
83− -> TagParser m ([Attribute Text], a)
84−pInTagWithAttribs tagOmission tagtype p = try $ do
85− let openingOptional = tagOmission == TagsOmittable
86− let closingOptional = tagOmission /= TagsRequired
58+-- parses p, preceded by an opening tag (optional if tagsOptional)
59+-- and followed by a closing tag (optional if tagsOptional)
60+pInTag :: PandocMonad m => Bool -> Text -> TagParser m a -> TagParser m a
61+pInTag tagsOptional tagtype p = try $ do
8762 skipMany pBlank
88− attribs <- (if openingOptional then option [] else id)
89− (getAttribs <$> pSatisfy (matchTagOpen tagtype []))
63+ (if tagsOptional then optional else void) $ pSatisfy (matchTagOpen tagtype [])
9064 skipMany pBlank
9165 x <- p
9266 skipMany pBlank
93− (if closingOptional then optional else void) $
94− pSatisfy (matchTagClose tagtype)
67+ (if tagsOptional then optional else void) $ pSatisfy (matchTagClose tagtype)
9568 skipMany pBlank
96− return (attribs, x)
97− where
98− getAttribs = \case
99− TagOpen _ attribs -> attribs
100− _ -> []
69+ return x
10170
10271pCloses :: PandocMonad m => Text -> TagParser m ()
10372pCloses tagtype = try $ do
@@ -193,26 +162,14 @@ t1 `closes` t2 |
193162_ `closes` _ = False
194163
195164toStringAttr :: [(Text, Text)] -> [(Text, Text)]
196−toStringAttr = foldr go []
165+toStringAttr = map go
197166 where
198− go :: (Text, Text) -> [(Text, Text)] -> [(Text, Text)]
199− -- treat xml:lang as lang
200− go ("xml:lang",y) ats = go ("lang",y) ats
201− -- prevent duplicate attributes
202− go (x,y) ats
203− | any (\(x',_) -> x == x') ats = ats
204− | otherwise =
205− case T.stripPrefix "data-" x of
206− Just x' | x' `Set.notMember` (html5Attributes <>
207− html4Attributes <> rdfaAttributes)
208− -> go (x',y) ats
209− _ -> (x,y):ats
210−
211−-- Unlike fromAttrib from tagsoup, this distinguishes
212−-- between a missing attribute and an attribute with empty content.
213−maybeFromAttrib :: Text -> Tag Text -> Maybe Text
214−maybeFromAttrib name (TagOpen _ attrs) = lookup name attrs
215−maybeFromAttrib _ _ = Nothing
167+ go (x,y) =
168+ case T.stripPrefix "data-" x of
169+ Just x' | x' `Set.notMember` (html5Attributes <>
170+ html4Attributes <> rdfaAttributes)
171+ -> (x',y)
172+ _ -> (x,y)
216173
217174mkAttr :: [(Text, Text)] -> Attr
218175mkAttr attr = (attribsId, attribsClasses, attribsKV)
src/Text/Pandoc/Readers/HTML/Table.hs+70−240
@@ -1,10 +1,9 @@
1−{-# LANGUAGE LambdaCase #-}
21{-# LANGUAGE OverloadedStrings #-}
3−{-# LANGUAGE ViewPatterns #-}
2+{-# LANGUAGE ViewPatterns #-}
43{- |
54 Module : Text.Pandoc.Readers.HTML.Table
6− Copyright : © 2006-2021 John MacFarlane,
7− 2020-2021 Albert Krewinkel
5+ Copyright : © 2006-2020 John MacFarlane,
6+ 2020 Albert Krewinkel
87 License : GNU GPL, version 2 or above
98
109 Maintainer : Albert Krewinkel <albert@zeitkraut.de>
@@ -13,271 +12,102 @@
1312
1413HTML table parser.
1514-}
16−module Text.Pandoc.Readers.HTML.Table (pTable) where
15+module Text.Pandoc.Readers.HTML.Table (pTable') where
1716
18−import Control.Applicative ((<|>))
17+import Control.Monad (guard)
1918import Data.Maybe (fromMaybe)
20−import Data.Either (lefts, rights)
21−import Data.List.NonEmpty (nonEmpty)
2219import Data.Text (Text)
2320import Text.HTML.TagSoup
2421import Text.Pandoc.Builder (Blocks)
25−import Text.Pandoc.CSS (cssAttributes)
2622import Text.Pandoc.Definition
2723import Text.Pandoc.Class.PandocMonad (PandocMonad (..))
2824import Text.Pandoc.Parsing
29− ( eof, lookAhead, many, many1, manyTill, option, optional
30− , optionMaybe, skipMany, try)
25+ ( (<|>), eof, many, many1, manyTill, option, optional, skipMany, try)
3126import Text.Pandoc.Readers.HTML.Parsing
3227import Text.Pandoc.Readers.HTML.Types (TagParser)
3328import Text.Pandoc.Shared (onlySimpleTableCells, safeRead)
3429import qualified Data.Text as T
3530import qualified Text.Pandoc.Builder as B
3631
37−-- | Parses a @<col>@ element, returning the column's width.
38−-- An Either value is used: Left i means a "relative length" with
39−-- integral value i (see https://www.w3.org/TR/html4/types.html#h-6.6);
40−-- Right w means a regular width. Defaults to @'Right ColWidthDefault'@
41−-- if the width is not set or cannot be determined.
42−pCol :: PandocMonad m => TagParser m (Either Int ColWidth)
32+-- | Parses a @<col>@ element, returning the column's width. Defaults to
33+-- @'ColWidthDefault'@ if the width is not set or cannot be determined.
34+pCol :: PandocMonad m => TagParser m ColWidth
4335pCol = try $ do
4436 TagOpen _ attribs' <- pSatisfy (matchTagOpen "col" [])
4537 let attribs = toStringAttr attribs'
4638 skipMany pBlank
4739 optional $ pSatisfy (matchTagClose "col")
4840 skipMany pBlank
49− return $ case lookup "width" attribs of
41+ let width = case lookup "width" attribs of
5042 Nothing -> case lookup "style" attribs of
5143 Just (T.stripPrefix "width:" -> Just xs) | T.any (== '%') xs ->
52− maybe (Right ColWidthDefault) (Right . ColWidth . (/ 100.0))
53− $ safeRead (T.filter
54− (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
55− _ -> Right ColWidthDefault
56− Just (T.unsnoc -> Just (xs, '*')) ->
57− maybe (Left 1) Left $ safeRead xs
44+ fromMaybe 0.0 $ safeRead (T.filter
45+ (`notElem` (" \t\r\n%'\";" :: [Char])) xs)
46+ _ -> 0.0
5847 Just (T.unsnoc -> Just (xs, '%')) ->
59− maybe (Right ColWidthDefault)
60− (Right . ColWidth . (/ 100.0)) $ safeRead xs
61− _ -> Right ColWidthDefault
48+ fromMaybe 0.0 $ safeRead xs
49+ _ -> 0.0
50+ if width > 0.0
51+ then return $ ColWidth $ width / 100.0
52+ else return ColWidthDefault
6253
63−pColgroup :: PandocMonad m => TagParser m [Either Int ColWidth]
54+pColgroup :: PandocMonad m => TagParser m [ColWidth]
6455pColgroup = try $ do
6556 pSatisfy (matchTagOpen "colgroup" [])
6657 skipMany pBlank
6758 manyTill pCol (pCloses "colgroup" <|> eof) <* skipMany pBlank
6859
69−resolveRelativeLengths :: [Either Int ColWidth] -> [ColWidth]
70−resolveRelativeLengths ws =
71− let remaining = 1 - sum (map getColWidth $ rights ws)
72− relatives = sum $ lefts ws
73− relUnit = remaining / fromIntegral relatives
74− toColWidth (Right x) = x
75− toColWidth (Left i) = ColWidth (fromIntegral i * relUnit)
76− in map toColWidth ws
77−
78−getColWidth :: ColWidth -> Double
79−getColWidth ColWidthDefault = 0
80−getColWidth (ColWidth w) = w
81−
82−data CellType
83− = HeaderCell
84− | BodyCell
85− deriving Eq
86−
87−pCell :: PandocMonad m
88− => TagParser m Blocks
89− -> CellType
90− -> TagParser m (CellType, Cell)
91−pCell block celltype = try $ do
92− let celltype' = case celltype of
93− HeaderCell -> "th"
94− BodyCell -> "td"
95− skipMany pBlank
96− TagOpen _ attribs <- lookAhead $ pSatisfy (matchTagOpen celltype' [])
97− let cssAttribs = maybe [] cssAttributes $ lookup "style" attribs
98− let align = case lookup "align" attribs <|>
99− lookup "text-align" cssAttribs of
100− Just "left" -> AlignLeft
101− Just "right" -> AlignRight
102− Just "center" -> AlignCenter
103− _ -> AlignDefault
104− let rowspan = RowSpan . fromMaybe 1 $
105− safeRead =<< lookup "rowspan" attribs
106− let colspan = ColSpan . fromMaybe 1 $
107− safeRead =<< lookup "colspan" attribs
108− res <- pInTags celltype' block
109− skipMany pBlank
110− let handledAttribs = ["align", "colspan", "rowspan", "text-align"]
111− attribs' = foldr go [] attribs
112− go kv@(k, _) acc = case k of
113− "style" -> case filter ((/= "text-align") . fst) cssAttribs of
114− [] -> acc
115− cs -> ("style", toStyleString cs) : acc
116− -- drop attrib if it's already handled
117− _ | k `elem` handledAttribs -> acc
118− _ -> kv : acc
119− return (celltype, B.cellWith (toAttr attribs') align rowspan colspan res)
120−
121−-- | Create a style attribute string from a list of CSS attributes
122−toStyleString :: [(Text, Text)] -> Text
123−toStyleString = T.intercalate "; " . map (\(k, v) -> k <> ": " <> v)
124−
125−-- | Parses a normal table row; returns the row together with the number
126−-- of header cells at the beginning of the row.
127−pRow :: PandocMonad m
128− => TagParser m Blocks
129− -> TagParser m (RowHeadColumns, B.Row)
130−pRow block = try $ do
131− skipMany pBlank
132− TagOpen _ attribs <- pSatisfy (matchTagOpen "tr" []) <* skipMany pBlank
133− cells <- many (pCell block BodyCell <|> pCell block HeaderCell)
134− TagClose _ <- pSatisfy (matchTagClose "tr")
135− return ( RowHeadColumns $ length (takeWhile ((== HeaderCell) . fst) cells)
136− , Row (toAttr attribs) $ map snd cells
137− )
138−
139−-- | Parses a header row, i.e., a row which containing nothing but
140−-- @<th>@ elements.
141−pHeaderRow :: PandocMonad m
142− => TagParser m Blocks
143− -> TagParser m B.Row
144−pHeaderRow block = try $ do
145− skipMany pBlank
146− let pThs = map snd <$> many (pCell block HeaderCell)
147− let mkRow (attribs, cells) = Row (toAttr attribs) cells
148− mkRow <$> pInTagWithAttribs TagsRequired "tr" pThs
149−
150−-- | Parses a table head. If there is no @thead@ element, this looks for
151−-- a row of @<th>@-only elements as the first line of the table.
152−pTableHead :: PandocMonad m
153− => TagParser m Blocks
154− -> TagParser m TableHead
155−pTableHead block = try $ do
156− skipMany pBlank
157− let pRows = many (pRow block)
158− let pThead = pInTagWithAttribs ClosingTagOptional "thead" pRows
159− optionMaybe pThead >>= \case
160− Just (attribs, rows) ->
161− return $ TableHead (toAttr attribs) $ map snd rows
162− Nothing -> mkTableHead <$> optionMaybe (pHeaderRow block)
163− where
164− mkTableHead = TableHead nullAttr . \case
165− -- Use row as header only if it's non-empty
166− Just row@(Row _ (_:_)) -> [row]
167− _ -> []
168−
169−-- | Parses a table foot
170−pTableFoot :: PandocMonad m
171− => TagParser m Blocks
172− -> TagParser m TableFoot
173−pTableFoot block = try $ do
174− skipMany pBlank
175− TagOpen _ attribs <- pSatisfy (matchTagOpen "tfoot" []) <* skipMany pBlank
176− rows <- many (fmap snd $ pRow block <* skipMany pBlank)
177− optional $ pSatisfy (matchTagClose "tfoot")
178− return $ TableFoot (toAttr attribs) rows
179−
180−-- | Parses a table body
181−pTableBody :: PandocMonad m
182− => TagParser m Blocks
183− -> TagParser m TableBody
184−pTableBody block = try $ do
185− skipMany pBlank
186− attribs <- option [] $ getAttribs <$> pSatisfy (matchTagOpen "tbody" [])
187− <* skipMany pBlank
188− bodyheads <- many (pHeaderRow block)
189− (rowheads, rows) <- unzip <$> many1 (pRow block <* skipMany pBlank)
190− optional $ pSatisfy (matchTagClose "tbody")
191− return $ TableBody (toAttr attribs) (foldr max 0 rowheads) bodyheads rows
192− where
193− getAttribs (TagOpen _ attribs) = attribs
194− getAttribs _ = []
195−
19660-- | Parses a simple HTML table
197−pTable :: PandocMonad m
198− => TagParser m Blocks -- ^ Caption and cell contents parser
199− -> TagParser m Blocks
200−pTable block = try $ do
201− TagOpen _ attribs <- pSatisfy (matchTagOpen "table" []) <* skipMany pBlank
202− caption <- option mempty $ pInTags "caption" block <* skipMany pBlank
203− widths <- resolveRelativeLengths <$>
204− ((mconcat <$> many1 pColgroup) <|> many pCol) <* skipMany pBlank
205− thead <- pTableHead block <* skipMany pBlank
206− topfoot <- optionMaybe (pTableFoot block) <* skipMany pBlank
207− tbodies <- many (pTableBody block) <* skipMany pBlank
208− botfoot <- optionMaybe (pTableFoot block) <* skipMany pBlank
61+pTable' :: PandocMonad m
62+ => TagParser m Blocks -- ^ Caption parser
63+ -> (Text -> TagParser m [Cell]) -- ^ Table cell parser
64+ -> TagParser m Blocks
65+pTable' block pCell = try $ do
66+ TagOpen _ attribs' <- pSatisfy (matchTagOpen "table" [])
67+ let attribs = toAttr attribs'
68+ skipMany pBlank
69+ caption <- option mempty $ pInTags "caption" block <* skipMany pBlank
70+ widths' <- (mconcat <$> many1 pColgroup) <|> many pCol
71+ let pTh = option [] $ pInTags "tr" (pCell "th")
72+ pTr = try $ skipMany pBlank >>
73+ pInTags "tr" (pCell "td" <|> pCell "th")
74+ pTBody = pInTag True "tbody" $ many1 pTr
75+ head'' <- pInTag False "thead" (option [] pTr) <|> pInTag True "thead" pTh
76+ head' <- pInTag True "tbody"
77+ (if null head'' then pTh else return head'')
78+ topfoot <- option [] $ pInTag False "tfoot" $ many pTr
79+ rowsLs <- many pTBody
80+ bottomfoot <- option [] $ pInTag False "tfoot" $ many pTr
20981 TagClose _ <- pSatisfy (matchTagClose "table")
210− let tfoot = fromMaybe (TableFoot nullAttr []) $ topfoot <|> botfoot
211− case normalize widths thead tbodies tfoot of
212− Left err -> fail err
213− Right (colspecs, thead', tbodies', tfoot') -> return $
214− B.tableWith (toAttr attribs)
215− (B.simpleCaption caption)
216− colspecs
217− thead'
218− tbodies'
219− tfoot'
220−data TableType
221− = SimpleTable
222− | NormalTable
223−
224−tableType :: [[Cell]] -> TableType
225−tableType cells =
226− if onlySimpleTableCells $ map (map cellContents) cells
227− then SimpleTable
228− else NormalTable
229− where
230− cellContents :: Cell -> [Block]
231− cellContents (Cell _ _ _ _ bs) = bs
232−
233−normalize :: [ColWidth] -> TableHead -> [TableBody] -> TableFoot
234− -> Either String ([ColSpec], TableHead, [TableBody], TableFoot)
235−normalize widths head' bodies foot = do
236− let rows = headRows head' <> concatMap bodyRows bodies <> footRows foot
237− let cellWidth (Cell _ _ _ (ColSpan cs) _) = cs
238− let rowLength = foldr (\cell acc -> cellWidth cell + acc) 0 . rowCells
239− let ncols = maybe 0 maximum $ nonEmpty $ map rowLength rows
240− let tblType = tableType (map rowCells rows)
82+ let rows = concat rowsLs <> topfoot <> bottomfoot
83+ rows''' = map (map cellContents) rows
24184 -- fail on empty table
242− if null rows
243− then Left "empty table"
244− else Right
245− ( zip (calculateAlignments ncols bodies)
246− (normalizeColWidths ncols tblType widths)
247− , head'
248− , bodies
249− , foot
250− )
251−
252−normalizeColWidths :: Int -> TableType -> [ColWidth] -> [ColWidth]
253−normalizeColWidths ncols tblType = \case
254− [] -> case tblType of
255− SimpleTable -> replicate ncols ColWidthDefault
256− NormalTable -> replicate ncols (ColWidth $ 1 / fromIntegral ncols)
257− widths -> widths
258−
259−calculateAlignments :: Int -> [TableBody] -> [Alignment]
260−calculateAlignments cols tbodies =
261− case cells of
262− cs:_ -> take cols $ concatMap cellAligns cs ++ repeat AlignDefault
263− _ -> replicate cols AlignDefault
264− where
265− cells :: [[Cell]]
266− cells = concatMap bodyRowCells tbodies
267− cellAligns :: Cell -> [Alignment]
268− cellAligns (Cell _ align _ (ColSpan cs) _) = replicate cs align
269−
270−bodyRowCells :: TableBody -> [[Cell]]
271−bodyRowCells = map rowCells . bodyRows
272−
273−headRows :: TableHead -> [B.Row]
274−headRows (TableHead _ rows) = rows
275−
276−bodyRows :: TableBody -> [B.Row]
277−bodyRows (TableBody _ _ headerRows bodyRows') = headerRows <> bodyRows'
278−
279−footRows :: TableFoot -> [B.Row]
280−footRows (TableFoot _ rows) = rows
281−
282−rowCells :: B.Row -> [Cell]
283−rowCells (Row _ cells) = cells
85+ guard $ not $ null head' && null rows'''
86+ let isSimple = onlySimpleTableCells $
87+ map cellContents head' : rows'''
88+ let cols = if null head'
89+ then maximum (map length rows''')
90+ else length head'
91+ let aligns = case rows of
92+ (cs:_) -> take cols $
93+ concatMap cellAligns cs ++ repeat AlignDefault
94+ _ -> replicate cols AlignDefault
95+ let widths = if null widths'
96+ then if isSimple
97+ then replicate cols ColWidthDefault
98+ else replicate cols (ColWidth (1.0 / fromIntegral cols))
99+ else widths'
100+ let toRow = Row nullAttr
101+ toHeaderRow l = [toRow l | not (null l)]
102+ return $ B.tableWith attribs
103+ (B.simpleCaption caption)
104+ (zip aligns widths)
105+ (TableHead nullAttr $ toHeaderRow head')
106+ [TableBody nullAttr 0 [] $ map toRow rows]
107+ (TableFoot nullAttr [])
108+
109+cellContents :: Cell -> [Block]
110+cellContents (Cell _ _ _ _ bs) = bs
111+
112+cellAligns :: Cell -> [Alignment]
113+cellAligns (Cell _ align _ (ColSpan cs) _) = replicate cs align
src/Text/Pandoc/Readers/HTML/TagCategories.hs+1−1
@@ -1,7 +1,7 @@
11{-# LANGUAGE OverloadedStrings #-}
22{- |
33 Module : Text.Pandoc.Readers.HTML.TagCategories
4− Copyright : Copyright (C) 2006-2021 John MacFarlane
4+ Copyright : Copyright (C) 2006-2020 John MacFarlane
55 License : GNU GPL, version 2 or above
66
77 Maintainer : John MacFarlane <jgm@berkeley.edu>
src/Text/Pandoc/Readers/HTML/Types.hs+1−1
@@ -2,7 +2,7 @@
22{-# LANGUAGE MultiParamTypeClasses #-}
33{- |
44 Module : Text.Pandoc.Readers.HTML.Types
5− Copyright : Copyright (C) 2006-2021 John MacFarlane
5+ Copyright : Copyright (C) 2006-2020 John MacFarlane
66 License : GNU GPL, version 2 or above
77
88 Maintainer : John MacFarlane <jgm@berkeley.edu>
src/Text/Pandoc/Readers/Haddock.hs+7−11
@@ -17,9 +17,8 @@ module Text.Pandoc.Readers.Haddock
1717
1818import Control.Monad.Except (throwError)
1919import Data.List (intersperse)
20−import Data.List.NonEmpty (nonEmpty)
2120import Data.Maybe (fromMaybe)
22−import Data.Text (unpack)
21+import Data.Text (Text, unpack)
2322import qualified Data.Text as T
2423import Documentation.Haddock.Parser
2524import Documentation.Haddock.Types as H
@@ -29,17 +28,15 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad)
2928import Text.Pandoc.Definition
3029import Text.Pandoc.Error
3130import Text.Pandoc.Options
32−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
33−import Text.Pandoc.Shared (splitTextBy, trim)
31+import Text.Pandoc.Shared (crFilter, splitTextBy, trim)
3432
3533
3634-- | Parse Haddock markup and return a 'Pandoc' document.
37−readHaddock :: (PandocMonad m, ToSources a)
35+readHaddock :: PandocMonad m
3836 => ReaderOptions
39− -> a
37+ -> Text
4038 -> m Pandoc
41−readHaddock opts s = case readHaddockEither opts
42− (unpack . sourcesToText . toSources $ s) of
39+readHaddock opts s = case readHaddockEither opts (unpack (crFilter s)) of
4340 Right result -> return result
4441 Left e -> throwError e
4542
@@ -95,7 +92,7 @@ docHToBlocks d' =
9592 then ([], map toCells bodyRows)
9693 else (toCells (head headerRows),
9794 map toCells (tail headerRows ++ bodyRows))
98− colspecs = replicate (maybe 0 maximum (nonEmpty (map length body)))
95+ colspecs = replicate (maximum (map length body))
9996 (AlignDefault, ColWidthDefault)
10097 in B.table B.emptyCaption
10198 colspecs
@@ -131,8 +128,7 @@ docHToInlines isCode d' =
131128 DocIdentifier s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
132129 _ -> mempty
133130 DocIdentifierUnchecked s -> B.codeWith ("",["haskell","identifier"],[]) $ T.pack s
134− DocModule s -> B.codeWith ("",["haskell","module"],[]) $
135− T.pack (modLinkName s)
131+ DocModule s -> B.codeWith ("",["haskell","module"],[]) $ T.pack s
136132 DocWarning _ -> mempty -- TODO
137133 DocEmphasis d -> B.emph (docHToInlines isCode d)
138134 DocMonospaced (DocString s) -> B.code $ T.pack s
src/Text/Pandoc/Readers/Ipynb.hs+4−6
@@ -4,7 +4,7 @@
44{-# LANGUAGE ScopedTypeVariables #-}
55{- |
66 Module : Text.Pandoc.Readers.Ipynb
7− Copyright : Copyright (C) 2019-2021 John MacFarlane
7+ Copyright : Copyright (C) 2019-2020 John MacFarlane
88 License : GNU GPL, version 2 or above
99
1010 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -39,12 +39,10 @@ import Data.Aeson as Aeson
3939import Control.Monad.Except (throwError)
4040import Text.Pandoc.Readers.Markdown (readMarkdown)
4141import qualified Text.Pandoc.UTF8 as UTF8
42−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
4342
44−readIpynb :: (PandocMonad m, ToSources a)
45− => ReaderOptions -> a -> m Pandoc
46−readIpynb opts x = do
47− let src = BL.fromStrict . TE.encodeUtf8 . sourcesToText $ toSources x
43+readIpynb :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
44+readIpynb opts t = do
45+ let src = BL.fromStrict (TE.encodeUtf8 t)
4846 case eitherDecode src of
4947 Right (notebook4 :: Notebook NbV4) -> notebookToPandoc opts notebook4
5048 Left _ ->
src/Text/Pandoc/Readers/JATS.hs+51−37
@@ -14,9 +14,7 @@ Conversion of JATS XML to 'Pandoc' document.
1414
1515module Text.Pandoc.Readers.JATS ( readJATS ) where
1616import Control.Monad.State.Strict
17−import Control.Monad.Except (throwError)
18−import Text.Pandoc.Error (PandocError(..))
19−import Data.Char (isDigit, isSpace)
17+import Data.Char (isDigit, isSpace, toUpper)
2018import Data.Default
2119import Data.Generics
2220import Data.List (foldl', intersperse)
@@ -24,17 +22,15 @@ import qualified Data.Map as Map
2422import Data.Maybe (maybeToList, fromMaybe)
2523import Data.Text (Text)
2624import qualified Data.Text as T
27−import qualified Data.Text.Lazy as TL
2825import Text.HTML.TagSoup.Entity (lookupEntity)
2926import Text.Pandoc.Builder
3027import Text.Pandoc.Class.PandocMonad (PandocMonad)
3128import Text.Pandoc.Options
32−import Text.Pandoc.Shared (safeRead, extractSpaces)
29+import Text.Pandoc.Shared (crFilter, safeRead, extractSpaces)
3330import Text.TeXMath (readMathML, writeTeX)
34−import Text.Pandoc.XML.Light
31+import Text.XML.Light
3532import qualified Data.Set as S (fromList, member)
3633import Data.Set ((\\))
37−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
3834
3935type JATS m = StateT JATSState m
4036
@@ -53,28 +49,42 @@ instance Default JATSState where
5349 , jatsContent = [] }
5450
5551
56−readJATS :: (PandocMonad m, ToSources a)
57− => ReaderOptions
58− -> a
59− -> m Pandoc
52+readJATS :: PandocMonad m => ReaderOptions -> Text -> m Pandoc
6053readJATS _ inp = do
61− let sources = toSources inp
62− tree <- either (throwError . PandocXMLError "") return $
63− parseXMLContents (TL.fromStrict . sourcesToText $ sources)
54+ let tree = normalizeTree . parseXML
55+ $ T.unpack $ crFilter inp
6456 (bs, st') <- flip runStateT (def{ jatsContent = tree }) $ mapM parseBlock tree
6557 return $ Pandoc (jatsMeta st') (toList . mconcat $ bs)
6658
59+-- normalize input, consolidating adjacent Text and CRef elements
60+normalizeTree :: [Content] -> [Content]
61+normalizeTree = everywhere (mkT go)
62+ where go :: [Content] -> [Content]
63+ go (Text (CData CDataRaw _ _):xs) = xs
64+ go (Text (CData CDataText s1 z):Text (CData CDataText s2 _):xs) =
65+ Text (CData CDataText (s1 ++ s2) z):xs
66+ go (Text (CData CDataText s1 z):CRef r:xs) =
67+ Text (CData CDataText (s1 ++ convertEntity r) z):xs
68+ go (CRef r:Text (CData CDataText s1 z):xs) =
69+ Text (CData CDataText (convertEntity r ++ s1) z):xs
70+ go (CRef r1:CRef r2:xs) =
71+ Text (CData CDataText (convertEntity r1 ++ convertEntity r2) Nothing):xs
72+ go xs = xs
73+
74+convertEntity :: String -> String
75+convertEntity e = Data.Maybe.fromMaybe (map toUpper e) (lookupEntity e)
76+
6777-- convenience function to get an attribute value, defaulting to ""
68−attrValue :: Text -> Element -> Text
78+attrValue :: String -> Element -> Text
6979attrValue attr =
7080 fromMaybe "" . maybeAttrValue attr
7181
72−maybeAttrValue :: Text -> Element -> Maybe Text
82+maybeAttrValue :: String -> Element -> Maybe Text
7383maybeAttrValue attr elt =
74− lookupAttrBy (\x -> qName x == attr) (elAttribs elt)
84+ T.pack <$> lookupAttrBy (\x -> qName x == attr) (elAttribs elt)
7585
7686-- convenience function
77−named :: Text -> Element -> Bool
87+named :: String -> Element -> Bool
7888named s e = qName (elName e) == s
7989
8090--
@@ -140,10 +150,10 @@ getBlocks e = mconcat <$>
140150
141151parseBlock :: PandocMonad m => Content -> JATS m Blocks
142152parseBlock (Text (CData CDataRaw _ _)) = return mempty -- DOCTYPE
143−parseBlock (Text (CData _ s _)) = if T.all isSpace s
153+parseBlock (Text (CData _ s _)) = if all isSpace s
144154 then return mempty
145− else return $ plain $ trimInlines $ text s
146−parseBlock (CRef x) = return $ plain $ str $ T.toUpper x
155+ else return $ plain $ trimInlines $ text $ T.pack s
156+parseBlock (CRef x) = return $ plain $ str $ T.toUpper $ T.pack x
147157parseBlock (Elem e) =
148158 case qName (elName e) of
149159 "p" -> parseMixed para (elContent e)
@@ -192,7 +202,7 @@ parseBlock (Elem e) =
192202 "" -> []
193203 x -> [x]
194204 return $ codeBlockWith (attrValue "id" e, classes', [])
195− $ trimNl $ strContentRecursive e
205+ $ trimNl $ textContentRecursive e
196206 parseBlockquote = do
197207 attrib <- case filterChild (named "attribution") e of
198208 Nothing -> return mempty
@@ -256,7 +266,7 @@ parseBlock (Elem e) =
256266 Just "center" -> AlignCenter
257267 _ -> AlignDefault
258268 let toWidth c = do
259− w <- findAttr (unqual "colwidth") c
269+ w <- findAttrText (unqual "colwidth") c
260270 n <- safeRead $ "0" <> T.filter (\x -> isDigit x || x == '.') w
261271 if n > 0 then Just n else Nothing
262272 let numrows = foldl' max 0 $ map length bodyrows
@@ -427,10 +437,16 @@ parseRef e = do
427437 Nothing -> return $ Map.insert "id" (toMetaValue refId) mempty
428438 -- TODO handle mixed-citation
429439
440+findAttrText :: QName -> Element -> Maybe Text
441+findAttrText x = fmap T.pack . findAttr x
442+
430443textContent :: Element -> Text
431−textContent = strContent
444+textContent = T.pack . strContent
445+
446+textContentRecursive :: Element -> Text
447+textContentRecursive = T.pack . strContentRecursive
432448
433−strContentRecursive :: Element -> Text
449+strContentRecursive :: Element -> String
434450strContentRecursive = strContent .
435451 (\e' -> e'{ elContent = map elementToStr $ elContent e' })
436452
@@ -439,9 +455,9 @@ elementToStr (Elem e') = Text $ CData CDataText (strContentRecursive e') Nothing
439455elementToStr x = x
440456
441457parseInline :: PandocMonad m => Content -> JATS m Inlines
442−parseInline (Text (CData _ s _)) = return $ text s
443−parseInline (CRef ref) = return $ maybe (text $ T.toUpper ref) (text . T.pack)
444− $ lookupEntity (T.unpack ref)
458+parseInline (Text (CData _ s _)) = return $ text $ T.pack s
459+parseInline (CRef ref) =
460+ return . text . maybe (T.toUpper $ T.pack ref) T.pack $ lookupEntity ref
445461parseInline (Elem e) =
446462 case qName (elName e) of
447463 "italic" -> innerInlines emph
@@ -486,9 +502,9 @@ parseInline (Elem e) =
486502 else linkWith attr ("#" <> rid) "" ils
487503 "ext-link" -> do
488504 ils <- innerInlines id
489− let title = fromMaybe "" $ findAttr (QName "title" (Just "http://www.w3.org/1999/xlink") Nothing) e
505+ let title = fromMaybe "" $ findAttrText (QName "title" (Just "http://www.w3.org/1999/xlink") Nothing) e
490506 let href = case findAttr (QName "href" (Just "http://www.w3.org/1999/xlink") Nothing) e of
491− Just h -> h
507+ Just h -> T.pack h
492508 _ -> "#" <> attrValue "rid" e
493509 let ils' = if ils == mempty then str href else ils
494510 let attr = (attrValue "id" e, [], [])
@@ -496,8 +512,7 @@ parseInline (Elem e) =
496512
497513 "disp-formula" -> formula displayMath
498514 "inline-formula" -> formula math
499− "math" | qURI (elName e) == Just "http://www.w3.org/1998/Math/MathML"
500− -> return . math $ mathML e
515+ "math" | qPrefix (elName e) == Just "mml" -> return . math $ mathML e
501516 "tex-math" -> return . math $ textContent e
502517
503518 "email" -> return $ link ("mailto:" <> textContent e) ""
@@ -509,7 +524,7 @@ parseInline (Elem e) =
509524 where innerInlines f = extractSpaces f . mconcat <$>
510525 mapM parseInline (elContent e)
511526 mathML x =
512− case readMathML . showElement $ everywhere (mkT removePrefix) x of
527+ case readMathML . T.pack . showElement $ everywhere (mkT removePrefix) x of
513528 Left _ -> mempty
514529 Right m -> writeTeX m
515530 formula constructor = do
@@ -520,12 +535,11 @@ parseInline (Elem e) =
520535 filterChildren isMathML whereToLook
521536 return . mconcat . take 1 . map constructor $ texMaths ++ mathMLs
522537
523− isMathML x = qName (elName x) == "math" &&
524− qURI (elName x) ==
525− Just "http://www.w3.org/1998/Math/MathML"
538+ isMathML x = qName (elName x) == "math" &&
539+ qPrefix (elName x) == Just "mml"
526540 removePrefix elname = elname { qPrefix = Nothing }
527541 codeWithLang = do
528542 let classes' = case attrValue "language" e of
529543 "" -> []
530544 l -> [l]
531− return $ codeWith (attrValue "id" e,classes',[]) $ strContentRecursive e
545+ return $ codeWith (attrValue "id" e,classes',[]) $ textContentRecursive e
src/Text/Pandoc/Readers/Jira.hs+10−14
@@ -2,7 +2,7 @@
22{-# LANGUAGE OverloadedStrings #-}
33{- |
44 Module : Text.Pandoc.Readers.Org
5− Copyright : © 2019-2021 Albert Krewinkel
5+ Copyright : © 2019-2020 Albert Krewinkel
66 License : GNU GPL, version 2 or above
77
88 Maintainer : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>
@@ -20,20 +20,18 @@ import Text.Pandoc.Builder hiding (cell)
2020import Text.Pandoc.Error (PandocError (PandocParseError))
2121import Text.Pandoc.Options (ReaderOptions)
2222import Text.Pandoc.Shared (stringify)
23−import Text.Pandoc.Sources (ToSources(..), sourcesToText)
23+
2424import qualified Text.Jira.Markup as Jira
2525
2626-- | Read Jira wiki markup.
27−readJira :: (PandocMonad m, ToSources a)
27+readJira :: PandocMonad m
2828 => ReaderOptions
29− -> a
29+ -> Text
3030 -> m Pandoc
31−readJira _opts inp = do
32− let sources = toSources inp
33− case parse (sourcesToText sources) of
34− Right d -> return $ jiraToPandoc d
35− Left e -> throwError . PandocParseError $
36− "Jira parse error" `append` pack (show e)
31+readJira _opts s = case parse s of
32+ Right d -> return $ jiraToPandoc d
33+ Left e -> throwError . PandocParseError $
34+ "Jira parse error" `append` pack (show e)
3735
3836jiraToPandoc :: Jira.Doc -> Pandoc
3937jiraToPandoc (Jira.Doc blks) = doc $ foldMap jiraToPandocBlocks blks
@@ -73,10 +71,10 @@ toPandocCodeBlocks langMay params txt =
7371 Nothing -> []
7472 in codeBlockWith ("", classes, map paramToPair params) txt
7573
76−-- | Create a pandoc @'Div'@ from a panel.
74+-- | Create a pandoc @'Div'@
7775toPandocDiv :: [Jira.Parameter] -> [Jira.Block] -> Blocks
7876toPandocDiv params =
79− divWith ("", ["panel"], map paramToPair params) . foldMap jiraToPandocBlocks
77+ divWith ("", [], map paramToPair params) . foldMap jiraToPandocBlocks
8078
8179paramToPair :: Jira.Parameter -> (Text, Text)
8280paramToPair (Jira.Parameter key value) = (key, value)
@@ -172,8 +170,6 @@ jiraLinkToPandoc linkType alias url =
172170 Jira.Email -> link ("mailto:" <> url') "" alias'
173171 Jira.Attachment -> linkWith ("", ["attachment"], []) url' "" alias'
174172 Jira.User -> linkWith ("", ["user-account"], []) url' "" alias'
175− Jira.SmartCard -> linkWith ("", ["smart-card"], []) url' "" alias'
176− Jira.SmartLink -> linkWith ("", ["smart-link"], []) url' "" alias'
177173
178174-- | Get unicode representation of a Jira icon.
179175iconUnicode :: Jira.Icon -> Text
src/Text/Pandoc/Readers/LaTeX.hs+1416−264
@@ -1,10 +1,14 @@
1+{-# LANGUAGE BangPatterns #-}
2+{-# LANGUAGE CPP #-}
3+{-# LANGUAGE FlexibleInstances #-}
4+{-# LANGUAGE MultiParamTypeClasses #-}
15{-# LANGUAGE OverloadedStrings #-}
26{-# LANGUAGE PatternGuards #-}
37{-# LANGUAGE ScopedTypeVariables #-}
48{-# LANGUAGE ViewPatterns #-}
59{- |
610 Module : Text.Pandoc.Readers.LaTeX
7− Copyright : Copyright (C) 2006-2021 John MacFarlane
11+ Copyright : Copyright (C) 2006-2020 John MacFarlane
812 License : GNU GPL, version 2 or above
913
1014 Maintainer : John MacFarlane <jgm@berkeley.edu>
@@ -18,58 +22,50 @@ module Text.Pandoc.Readers.LaTeX ( readLaTeX,
1822 applyMacros,
1923 rawLaTeXInline,
2024 rawLaTeXBlock,
21− inlineCommand
25+ inlineCommand,
26+ tokenize,
27+ untokenize
2228 ) where
2329
2430import Control.Applicative (many, optional, (<|>))
2531import Control.Monad
2632import Control.Monad.Except (throwError)
27−import Data.Char (isDigit, isLetter, isAlphaNum, toUpper, chr)
33+import Data.Char (isDigit, isLetter, toUpper, chr)
2834import Data.Default
35+import Data.Functor (($>))
2936import Data.List (intercalate)
3037import qualified Data.Map as M
3138import Data.Maybe (fromMaybe, maybeToList)
3239import qualified Data.Set as Set
40+import qualified Data.Sequence as Seq
3341import Data.Text (Text)
3442import qualified Data.Text as T
35−import Skylighting (defaultSyntaxMap)
3643import System.FilePath (addExtension, replaceExtension, takeExtension)
37−import Text.Collate.Lang (renderLang)
38−import Text.Pandoc.Builder as B
44+import Text.Pandoc.BCP47 (Lang (..), renderLang)
45+import Text.Pandoc.Builder
3946import Text.Pandoc.Class.PandocPure (PandocPure)
4047import Text.Pandoc.Class.PandocMonad (PandocMonad (..), getResourcePath,
41− readFileFromDirs, report,
42− setResourcePath)
48+ readFileFromDirs, report, setResourcePath,
49+ setTranslations, translateTerm)
4350import Text.Pandoc.Error (PandocError (PandocParseError, PandocParsecError))
44−import Text.Pandoc.Highlighting (languagesByExtension)
51+import Text.Pandoc.Highlighting (fromListingsLanguage, languagesByExtension)
4552import Text.Pandoc.ImageSize (numUnit, showFl)
4653import Text.Pandoc.Logging
4754import Text.Pandoc.Options
4855import Text.Pandoc.Parsing hiding (blankline, many, mathDisplay, mathInline,
4956 optional, space, spaces, withRaw, (<|>))
50−import Text.Pandoc.Readers.LaTeX.Types (Tok (..), TokType (..))
57+import Text.Pandoc.Readers.LaTeX.Types (ExpansionPoint (..), Macro (..),
58+ ArgSpec (..), Tok (..), TokType (..))
5159import Text.Pandoc.Readers.LaTeX.Parsing
52−import Text.Pandoc.Readers.LaTeX.Citation (citationCommands, cites)
53−import Text.Pandoc.Readers.LaTeX.Math (dollarsMath, inlineEnvironments,
54− inlineEnvironment,
55− mathDisplay, mathInline,
56− newtheorem, theoremstyle, proof,
57− theoremEnvironment)
58−import Text.Pandoc.Readers.LaTeX.Table (tableEnvironments)
59−import Text.Pandoc.Readers.LaTeX.Macro (macroDef)
60−import Text.Pandoc.Readers.LaTeX.Lang (inlineLanguageCommands,
61− enquoteCommands,
62− babelLangToBCP47, setDefaultLanguage)
63−import Text.Pandoc.Readers.LaTeX.SIunitx (siunitxCommands)
64−import Text.Pandoc.Readers.LaTeX.Inline (acronymCommands, refCommands,
65− nameCommands, charCommands,
66− accentCommands,
67− biblatexInlineCommands,
68− verbCommands, rawInlineOr,
69− listingsLanguage)
60+import Text.Pandoc.Readers.LaTeX.Lang (polyglossiaLangToBCP47,
61+ babelLangToBCP47)
62+import Text.Pandoc.Readers.LaTeX.SIunitx
7063import Text.Pandoc.Shared
64+import qualified Text.Pandoc.Translations as Translations
7165import Text.Pandoc.Walk
72−import Data.List.NonEmpty (nonEmpty)
66+import qualified Text.Pandoc.Builder as B
67+import qualified Data.Text.Normalize as Normalize
68+import Safe
7369
7470-- for debugging:
7571-- import Text.Pandoc.Extensions (getDefaultExtensions)
@@ -77,17 +73,16 @@ import Data.List.NonEmpty (nonEmpty)
7773-- import Debug.Trace (traceShowId)
7874
7975-- | Parse LaTeX from string and return 'Pandoc' document.
80−readLaTeX :: (PandocMonad m, ToSources a)
76+readLaTeX :: PandocMonad m
8177 => ReaderOptions -- ^ Reader options
82− -> a -- ^ Input to parse
78+ -> Text -- ^ String to parse (assumes @'\n'@ line endings)
8379 -> m Pandoc
8480readLaTeX opts ltx = do
85− let sources = toSources ltx
8681 parsed <- runParserT parseLaTeX def{ sOptions = opts } "source"
87− (tokenizeSources sources)
82+ (tokenize "source" (crFilter ltx))
8883 case parsed of
8984 Right result -> return result
90− Left e -> throwError $ PandocParsecError sources e
85+ Left e -> throwError $ PandocParsecError ltx e
9186
9287parseLaTeX :: PandocMonad m => LP m Pandoc
9388parseLaTeX = do
@@ -98,7 +93,11 @@ parseLaTeX = do
9893 let doc' = doc bs
9994 let headerLevel (Header n _ _) = [n]
10095 headerLevel _ = []
101− let bottomLevel = maybe 1 minimum $ nonEmpty $ query headerLevel doc'
96+#if MIN_VERSION_safe(0,3,18)
97+ let bottomLevel = minimumBound 1 $ query headerLevel doc'
98+#else
99+ let bottomLevel = minimumDef 1 $ query headerLevel doc'
100+#endif
102101 let adjustHeaders m (Header n attr ils) = Header (n+m) attr ils
103102 adjustHeaders _ x = x
104103 let (Pandoc _ bs') =
@@ -133,11 +132,11 @@ resolveRefs _ x = x
133132
134133
135134rawLaTeXBlock :: (PandocMonad m, HasMacros s, HasReaderOptions s)
136− => ParserT Sources s m Text
135+ => ParserT Text s m Text
137136rawLaTeXBlock = do
138137 lookAhead (try (char '\\' >> letter))
139138 inp <- getInput
140− let toks = tokenizeSources inp
139+ let toks = tokenize "source" inp
141140 snd <$> (rawLaTeXParser toks False (macroDef (const mempty)) blocks
142141 <|> rawLaTeXParser toks True
143142 (do choice (map controlSeq
@@ -164,14 +163,14 @@ beginOrEndCommand = try $ do
164163 (txt <> untokenize rawargs)
165164
166165rawLaTeXInline :: (PandocMonad m, HasMacros s, HasReaderOptions s)
167− => ParserT Sources s m Text
166+ => ParserT Text s m Text
168167rawLaTeXInline = do
169168 lookAhead (try (char '\\' >> letter))
170169 inp <- getInput
171− let toks = tokenizeSources inp
170+ let toks = tokenize "source" inp
172171 raw <- snd <$>
173172 ( rawLaTeXParser toks True
174− (mempty <$ (controlSeq "input" >> skipMany rawopt >> braced))
173+ (mempty <$ (controlSeq "input" >> skipMany opt >> braced))
175174 inlines
176175 <|> rawLaTeXParser toks True (inlineEnvironment <|> inlineCommand')
177176 inlines
@@ -179,11 +178,11 @@ rawLaTeXInline = do
179178 finalbraces <- mconcat <$> many (try (string "{}")) -- see #5439
180179 return $ raw <> T.pack finalbraces
181180
182−inlineCommand :: PandocMonad m => ParserT Sources ParserState m Inlines
181+inlineCommand :: PandocMonad m => ParserT Text ParserState m Inlines
183182inlineCommand = do
184183 lookAhead (try (char '\\' >> letter))
185184 inp <- getInput
186− let toks = tokenizeSources inp
185+ let toks = tokenize "source" inp
187186 fst <$> rawLaTeXParser toks True (inlineEnvironment <|> inlineCommand')
188187 inlines
189188
@@ -192,6 +191,12 @@ inlineCommand = do
192191word :: PandocMonad m => LP m Inlines
193192word = str . untoken <$> satisfyTok isWordTok
194193
194+regularSymbol :: PandocMonad m => LP m Inlines
195+regularSymbol = str . untoken <$> satisfyTok isRegularSymbol
196+ where isRegularSymbol (Tok _ Symbol t) = not $ T.any isSpecial t
197+ isRegularSymbol _ = False
198+ isSpecial c = c `Set.member` specialChars
199+
195200inlineGroup :: PandocMonad m => LP m Inlines
196201inlineGroup = do
197202 ils <- grouped inline
@@ -232,6 +237,19 @@ mkImage options (T.unpack -> src) = do
232237 _ -> return src
233238 return $ imageWith attr (T.pack src') "" alt
234239
240+doxspace :: PandocMonad m => LP m Inlines
241+doxspace =
242+ (space <$ lookAhead (satisfyTok startsWithLetter)) <|> return mempty
243+ where startsWithLetter (Tok _ Word t) =
244+ case T.uncons t of
245+ Just (c, _) | isLetter c -> True
246+ _ -> False
247+ startsWithLetter _ = False
248+
249+
250+lit :: Text -> LP m Inlines
251+lit = pure . str
252+
235253removeDoubleQuotes :: Text -> Text
236254removeDoubleQuotes t =
237255 Data.Maybe.fromMaybe t $ T.stripPrefix "\"" t >>= T.stripSuffix "\""
@@ -278,14 +296,23 @@ quoted' f starter ender = do
278296 cs -> cs)
279297 else lit startchs
280298
281−lit :: Text -> LP m Inlines
282−lit = pure . str
299+enquote :: PandocMonad m => Bool -> Maybe Text -> LP m Inlines
300+enquote starred mblang = do
301+ skipopts
302+ let lang = mblang >>= babelLangToBCP47
303+ let langspan = case lang of
304+ Nothing -> id
305+ Just l -> spanWith ("",[],[("lang", renderLang l)])
306+ quoteContext <- sQuoteContext <$> getState
307+ if starred || quoteContext == InDoubleQuote
308+ then singleQuoted . langspan <$> withQuoteContext InSingleQuote tok
309+ else doubleQuoted . langspan <$> withQuoteContext InDoubleQuote tok
283310
284311blockquote :: PandocMonad m => Bool -> Maybe Text -> LP m Blocks
285312blockquote cvariant mblang = do
286313 citepar <- if cvariant
287314 then (\xs -> para (cite xs mempty))
288− <$> cites inline NormalCitation False
315+ <$> cites NormalCitation False
289316 else option mempty $ para <$> bracketed inline
290317 let lang = mblang >>= babelLangToBCP47
291318 let langdiv = case lang of
@@ -296,13 +323,224 @@ blockquote cvariant mblang = do
296323 optional $ symbolIn (".:;?!" :: [Char]) -- currently ignored
297324 return $ blockQuote . langdiv $ (bs <> citepar)
298325
326+doAcronym :: PandocMonad m => Text -> LP m Inlines
327+doAcronym form = do
328+ acro <- braced
329+ return . mconcat $ [spanWith ("",[],[("acronym-label", untokenize acro),
330+ ("acronym-form", "singular+" <> form)])
331+ $ str $ untokenize acro]
332+
333+doAcronymPlural :: PandocMonad m => Text -> LP m Inlines
334+doAcronymPlural form = do
335+ acro <- braced
336+ plural <- lit "s"
337+ return . mconcat $ [spanWith ("",[],[("acronym-label", untokenize acro),
338+ ("acronym-form", "plural+" <> form)]) $
339+ mconcat [str $ untokenize acro, plural]]
340+
341+doverb :: PandocMonad m => LP m Inlines
342+doverb = do
343+ Tok _ Symbol t <- anySymbol
344+ marker <- case T.uncons t of
345+ Just (c, ts) | T.null ts -> return c
346+ _ -> mzero
347+ withVerbatimMode $
348+ code . untokenize <$>
349+ manyTill (notFollowedBy newlineTok >> verbTok marker) (symbol marker)
350+
351+verbTok :: PandocMonad m => Char -> LP m Tok
352+verbTok stopchar = do
353+ t@(Tok pos toktype txt) <- anyTok
354+ case T.findIndex (== stopchar) txt of
355+ Nothing -> return t
356+ Just i -> do
357+ let (t1, t2) = T.splitAt i txt
358+ inp <- getInput
359+ setInput $ Tok (incSourceColumn pos i) Symbol (T.singleton stopchar)
360+ : totoks (incSourceColumn pos (i + 1)) (T.drop 1 t2) ++ inp
361+ return $ Tok pos toktype t1
362+
363+listingsLanguage :: [(Text, Text)] -> Maybe Text
364+listingsLanguage opts =
365+ case lookup "language" opts of
366+ Nothing -> Nothing
367+ Just l -> fromListingsLanguage l `mplus` Just l
368+
369+dolstinline :: PandocMonad m => LP m Inlines
370+dolstinline = do
371+ options <- option [] keyvals
372+ let classes = maybeToList $ listingsLanguage options
373+ doinlinecode classes
374+
375+domintinline :: PandocMonad m => LP m Inlines
376+domintinline = do
377+ skipopts
378+ cls <- untokenize <$> braced
379+ doinlinecode [cls]
380+
381+doinlinecode :: PandocMonad m => [Text] -> LP m Inlines
382+doinlinecode classes = do
383+ Tok _ Symbol t <- anySymbol
384+ marker <- case T.uncons t of
385+ Just (c, ts) | T.null ts -> return c
386+ _ -> mzero
387+ let stopchar = if marker == '{' then '}' else marker
388+ withVerbatimMode $
389+ codeWith ("",classes,[]) . T.map nlToSpace . untokenize <$>
390+ manyTill (verbTok stopchar) (symbol stopchar)
391+
392+nlToSpace :: Char -> Char
393+nlToSpace '\n' = ' '
394+nlToSpace x = x
395+
396+mathDisplay :: Text -> Inlines
397+mathDisplay = displayMath . trimMath
398+
399+mathInline :: Text -> Inlines
400+mathInline = math . trimMath
401+
402+dollarsMath :: PandocMonad m => LP m Inlines
403+dollarsMath = do
404+ symbol '$'
405+ display <- option False (True <$ symbol '$')
406+ (do contents <- try $ untokenize <$> pDollarsMath 0
407+ if display
408+ then mathDisplay contents <$ symbol '$'
409+ else return $ mathInline contents)
410+ <|> (guard display >> return (mathInline ""))
411+
412+-- Int is number of embedded groupings
413+pDollarsMath :: PandocMonad m => Int -> LP m [Tok]
414+pDollarsMath n = do
415+ tk@(Tok _ toktype t) <- anyTok
416+ case toktype of
417+ Symbol | t == "$"
418+ , n == 0 -> return []
419+ | t == "\\" -> do
420+ tk' <- anyTok
421+ (tk :) . (tk' :) <$> pDollarsMath n
422+ | t == "{" -> (tk :) <$> pDollarsMath (n+1)
423+ | t == "}" ->
424+ if n > 0
425+ then (tk :) <$> pDollarsMath (n-1)
426+ else mzero
427+ _ -> (tk :) <$> pDollarsMath n
428+
429+-- citations
430+
431+addPrefix :: [Inline] -> [Citation] -> [Citation]
432+addPrefix p (k:ks) = k {citationPrefix = p ++ citationPrefix k} : ks
433+addPrefix _ _ = []
434+
435+addSuffix :: [Inline] -> [Citation] -> [Citation]
436+addSuffix s ks@(_:_) =
437+ let k = last ks
438+ in init ks ++ [k {citationSuffix = citationSuffix k ++ s}]
439+addSuffix _ _ = []
440+
441+simpleCiteArgs :: PandocMonad m => LP m [Citation]
442+simpleCiteArgs = try $ do
443+ first <- optionMaybe $ toList <$> opt
444+ second <- optionMaybe $ toList <$> opt
445+ keys <- try $ bgroup *> manyTill citationLabel egroup
446+ let (pre, suf) = case (first , second ) of
447+ (Just s , Nothing) -> (mempty, s )
448+ (Just s , Just t ) -> (s , t )
449+ _ -> (mempty, mempty)
450+ conv k = Citation { citationId = k
451+ , citationPrefix = []
452+ , citationSuffix = []
453+ , citationMode = NormalCitation
454+ , citationHash = 0
455+ , citationNoteNum = 0
456+ }
457+ return $ addPrefix pre $ addSuffix suf $ map conv keys
458+
459+citationLabel :: PandocMonad m => LP m Text
460+citationLabel = do
461+ sp
462+ untokenize <$>
463+ (many1 (satisfyTok isWordTok <|> symbolIn bibtexKeyChar)
464+ <* sp
465+ <* optional (symbol ',')
466+ <* sp)
467+ where bibtexKeyChar = ".:;?!`'()/*@_+=-&[]" :: [Char]
468+
469+cites :: PandocMonad m => CitationMode -> Bool -> LP m [Citation]
470+cites mode multi = try $ do
471+ cits <- if multi
472+ then do
473+ multiprenote <- optionMaybe $ toList <$> paropt
474+ multipostnote <- optionMaybe $ toList <$> paropt
475+ let (pre, suf) = case (multiprenote, multipostnote) of
476+ (Just s , Nothing) -> (mempty, s)
477+ (Nothing , Just t) -> (mempty, t)
478+ (Just s , Just t ) -> (s, t)
479+ _ -> (mempty, mempty)
480+ tempCits <- many1 simpleCiteArgs
481+ case tempCits of
482+ (k:ks) -> case ks of
483+ (_:_) -> return $ (addMprenote pre k : init ks) ++
484+ [addMpostnote suf (last ks)]
485+ _ -> return [addMprenote pre (addMpostnote suf k)]
486+ _ -> return [[]]
487+ else count 1 simpleCiteArgs
488+ let cs = concat cits
489+ return $ case mode of
490+ AuthorInText -> case cs of
491+ (c:rest) -> c {citationMode = mode} : rest
492+ [] -> []
493+ _ -> map (\a -> a {citationMode = mode}) cs
494+ where mprenote (k:ks) = (k:ks) ++ [Space]
495+ mprenote _ = mempty
496+ mpostnote (k:ks) = [Str ",", Space] ++ (k:ks)
497+ mpostnote _ = mempty
498+ addMprenote mpn (k:ks) =
499+ let mpnfinal = case citationPrefix k of
500+ (_:_) -> mprenote mpn
501+ _ -> mpn
502+ in addPrefix mpnfinal (k:ks)
503+ addMprenote _ _ = []
504+ addMpostnote = addSuffix . mpostnote
505+
506+citation :: PandocMonad m => Text -> CitationMode -> Bool -> LP m Inlines
507+citation name mode multi = do
508+ (c,raw) <- withRaw $ cites mode multi
509+ return $ cite c (rawInline "latex" $ "\\" <> name <> untokenize raw)
510+
511+handleCitationPart :: Inlines -> [Citation]
512+handleCitationPart ils =
513+ let isCite Cite{} = True
514+ isCite _ = False
515+ (pref, rest) = break isCite (toList ils)
516+ in case rest of
517+ (Cite cs _:suff) -> addPrefix pref $ addSuffix suff cs
518+ _ -> []
519+
520+complexNatbibCitation :: PandocMonad m => CitationMode -> LP m Inlines
521+complexNatbibCitation mode = try $ do
522+ (cs, raw) <-
523+ withRaw $ concat <$> do
524+ bgroup
525+ items <- mconcat <$>
526+ many1 (notFollowedBy (symbol ';') >> inline)
527+ `sepBy1` symbol ';'
528+ egroup
529+ return $ map handleCitationPart items
530+ case cs of
531+ [] -> mzero
532+ (c:cits) -> return $ cite (c{ citationMode = mode }:cits)
533+ (rawInline "latex" $ "\\citetext" <> untokenize raw)
534+
535+inNote :: Inlines -> Inlines
536+inNote ils =
537+ note $ para $ ils <> str "."
538+
299539inlineCommand' :: PandocMonad m => LP m Inlines
300540inlineCommand' = try $ do
301541 Tok _ (CtrlSeq name) cmd <- anyControlSeq
302542 guard $ name /= "begin" && name /= "end" && name /= "and"
303− star <- if T.all isAlphaNum name
304− then option "" ("*" <$ symbol '*' <* sp)
305− else pure ""
543+ star <- option "" ("*" <$ symbol '*' <* sp)
306544 overlay <- option "" overlaySpecification
307545 let name' = name <> star <> overlay
308546 let names = ordNub [name', name] -- check non-starred as fallback
@@ -313,8 +551,28 @@ inlineCommand' = try $ do
313551 <|> ignore rawcommand
314552 lookupListDefault raw names inlineCommands
315553
554+
316555tok :: PandocMonad m => LP m Inlines
317−tok = tokWith inline
556+tok = try $ spaces >> grouped inline <|> inlineCommand' <|> singleChar'
557+ where singleChar' = do
558+ Tok _ _ t <- singleChar
559+ return $ str t
560+
561+opt :: PandocMonad m => LP m Inlines
562+opt = do
563+ toks <- try (sp *> bracketedToks <* sp)
564+ -- now parse the toks as inlines
565+ st <- getState
566+ parsed <- runParserT (mconcat <$> many inline) st "bracketed option" toks
567+ case parsed of
568+ Right result -> return result
569+ Left e -> throwError $ PandocParsecError (untokenize toks) e
570+
571+paropt :: PandocMonad m => LP m Inlines
572+paropt = parenWrapped inline
573+
574+inBrackets :: Inlines -> Inlines
575+inBrackets x = str "[" <> x <> str "]"
318576
319577unescapeURL :: Text -> Text
320578unescapeURL = T.concat . go . T.splitOn "\\"
@@ -327,107 +585,379 @@ unescapeURL = T.concat . go . T.splitOn "\\"
327585 , isEscapable c = t
328586 | otherwise = "\\" <> t
329587
588+mathEnvWith :: PandocMonad m
589+ => (Inlines -> a) -> Maybe Text -> Text -> LP m a
590+mathEnvWith f innerEnv name = f . mathDisplay . inner <$> mathEnv name
591+ where inner x = case innerEnv of
592+ Nothing -> x
593+ Just y -> "\\begin{" <> y <> "}\n" <> x <>
594+ "\\end{" <> y <> "}"
595+
596+mathEnv :: PandocMonad m => Text -> LP m Text
597+mathEnv name = do
598+ skipopts
599+ optional blankline
600+ res <- manyTill anyTok (end_ name)
601+ return $ stripTrailingNewlines $ untokenize res
602+
603+inlineEnvironment :: PandocMonad m => LP m Inlines
604+inlineEnvironment = try $ do
605+ controlSeq "begin"
606+ name <- untokenize <$> braced
607+ M.findWithDefault mzero name inlineEnvironments
608+
609+inlineEnvironments :: PandocMonad m => M.Map Text (LP m Inlines)
610+inlineEnvironments = M.fromList [
611+ ("displaymath", mathEnvWith id Nothing "displaymath")
612+ , ("math", math <$> mathEnv "math")
613+ , ("equation", mathEnvWith id Nothing "equation")
614+ , ("equation*", mathEnvWith id Nothing "equation*")
615+ , ("gather", mathEnvWith id (Just "gathered") "gather")
616+ , ("gather*", mathEnvWith id (Just "gathered") "gather*")
617+ , ("multline", mathEnvWith id (Just "gathered") "multline")
618+ , ("multline*", mathEnvWith id (Just "gathered") "multline*")
619+ , ("eqnarray", mathEnvWith id (Just "aligned") "eqnarray")
620+ , ("eqnarray*", mathEnvWith id (Just "aligned") "eqnarray*")
621+ , ("align", mathEnvWith id (Just "aligned") "align")
622+ , ("align*", mathEnvWith id (Just "aligned") "align*")
623+ , ("alignat", mathEnvWith id (Just "aligned") "alignat")
624+ , ("alignat*", mathEnvWith id (Just "aligned") "alignat*")
625+ , ("dmath", mathEnvWith id Nothing "dmath")
626+ , ("dmath*", mathEnvWith id Nothing "dmath*")
627+ , ("dgroup", mathEnvWith id (Just "aligned") "dgroup")
628+ , ("dgroup*", mathEnvWith id (Just "aligned") "dgroup*")
629+ , ("darray", mathEnvWith id (Just "aligned") "darray")
630+ , ("darray*", mathEnvWith id (Just "aligned") "darray*")
631+ ]
632+
330633inlineCommands :: PandocMonad m => M.Map Text (LP m Inlines)
331−inlineCommands = M.unions
332− [ accentCommands tok
333− , citationCommands inline
334− , siunitxCommands tok
335− , acronymCommands
336− , refCommands
337− , nameCommands
338− , verbCommands
339− , charCommands
340− , enquoteCommands tok
341− , inlineLanguageCommands tok
342− , biblatexInlineCommands tok
343− , rest ]
344− where
345− rest = M.fromList
346− [ ("emph", extractSpaces emph <$> tok)
347− , ("textit", extractSpaces emph <$> tok)
348− , ("textsl", extractSpaces emph <$> tok)
349− , ("textsc", extractSpaces smallcaps <$> tok)
350− , ("textsf", extractSpaces (spanWith ("",["sans-serif"],[])) <$> tok)
351− , ("textmd", extractSpaces (spanWith ("",["medium"],[])) <$> tok)
352− , ("textrm", extractSpaces (spanWith ("",["roman"],[])) <$> tok)
353− , ("textup", extractSpaces (spanWith ("",["upright"],[])) <$> tok)
354− , ("texttt", ttfamily)
355− , ("sout", extractSpaces strikeout <$> tok)
356− , ("alert", skipopts >> spanWith ("",["alert"],[]) <$> tok) -- beamer
357− , ("textsuperscript", extractSpaces superscript <$> tok)
358− , ("textsubscript", extractSpaces subscript <$> tok)
359− , ("textbf", extractSpaces strong <$> tok)
360− , ("textnormal", extractSpaces (spanWith ("",["nodecor"],[])) <$> tok)
361− , ("underline", underline <$> tok)
362− , ("mbox", rawInlineOr "mbox" $ processHBox <$> tok)
363− , ("hbox", rawInlineOr "hbox" $ processHBox <$> tok)
364− , ("lettrine", rawInlineOr "lettrine" lettrine)
365− , ("(", mathInline . untokenize <$> manyTill anyTok (controlSeq ")"))
366− , ("[", mathDisplay . untokenize <$> manyTill anyTok (controlSeq "]"))
367− , ("ensuremath", mathInline . untokenize <$> braced)
368− , ("texorpdfstring", const <$> tok <*> tok)
369− -- old TeX commands
370− , ("em", extractSpaces emph <$> inlines)
371− , ("it", extractSpaces emph <$> inlines)
372− , ("sl", extractSpaces emph <$> inlines)
373− , ("bf", extractSpaces strong <$> inlines)
374− , ("tt", code . stringify . toList <$> inlines)
375− , ("rm", inlines)
376− , ("itshape", extractSpaces emph <$> inlines)
377− , ("slshape", extractSpaces emph <$> inlines)
378− , ("scshape", extractSpaces smallcaps <$> inlines)
379− , ("bfseries", extractSpaces strong <$> inlines)
380− , ("MakeUppercase", makeUppercase <$> tok)
381− , ("MakeTextUppercase", makeUppercase <$> tok) -- textcase
382− , ("uppercase", makeUppercase <$> tok)
383− , ("MakeLowercase", makeLowercase <$> tok)
384− , ("MakeTextLowercase", makeLowercase <$> tok)
385− , ("lowercase", makeLowercase <$> tok)
386− , ("thanks", skipopts >> note <$> grouped block)
387− , ("footnote", skipopts >> note <$> grouped block)
388− , ("passthrough", tok) -- \passthrough macro used by latex writer
389− -- for listings
390− , ("includegraphics", do options <- option [] keyvals
391− src <- braced
392− mkImage options .
393− unescapeURL .
394− removeDoubleQuotes $ untokenize src)
395− -- hyperref
396− , ("url", (\url -> link url "" (str url)) . unescapeURL . untokenize <$>
397− bracedUrl)
398− , ("nolinkurl", code . unescapeURL . untokenize <$> bracedUrl)
399− , ("href", do url <- bracedUrl
400− sp
401− link (unescapeURL $ untokenize url) "" <$> tok)
402− , ("hyperlink", hyperlink)
403− , ("hyperref", hyperref)
404− , ("hypertarget", hypertargetInline)
405− -- hyphenat
406− , ("nohyphens", tok)
407− , ("textnhtt", ttfamily)
408− , ("nhttfamily", ttfamily)
409− -- LaTeX colors
410− , ("textcolor", coloredInline "color")
411− , ("colorbox", coloredInline "background-color")
412− -- etoolbox
413− , ("ifstrequal", ifstrequal)
414− , ("newtoggle", braced >>= newToggle)
415− , ("toggletrue", braced >>= setToggle True)
416− , ("togglefalse", braced >>= setToggle False)
417− , ("iftoggle", try $ ifToggle >> inline)
418− -- include
419− , ("input", rawInlineOr "input" $ include "input")
420− -- soul package
421− , ("ul", underline <$> tok)
422− -- ulem package
423− , ("uline", underline <$> tok)
424− -- plain tex stuff that should just be passed through as raw tex
425− , ("ifdim", ifdim)
426− ]
634+inlineCommands = M.union inlineLanguageCommands $ M.fromList
635+ [ ("emph", extractSpaces emph <$> tok)
636+ , ("textit", extractSpaces emph <$> tok)
637+ , ("textsl", extractSpaces emph <$> tok)
638+ , ("textsc", extractSpaces smallcaps <$> tok)
639+ , ("textsf", extractSpaces (spanWith ("",["sans-serif"],[])) <$> tok)
640+ , ("textmd", extractSpaces (spanWith ("",["medium"],[])) <$> tok)
641+ , ("textrm", extractSpaces (spanWith ("",["roman"],[])) <$> tok)
642+ , ("textup", extractSpaces (spanWith ("",["upright"],[])) <$> tok)
643+ , ("texttt", ttfamily)
644+ , ("sout", extractSpaces strikeout <$> tok)
645+ , ("alert", skipopts >> spanWith ("",["alert"],[]) <$> tok) -- beamer
646+ , ("lq", return (str "‘"))
647+ , ("rq", return (str "’"))
648+ , ("textquoteleft", return (str "‘"))
649+ , ("textquoteright", return (str "’"))
650+ , ("textquotedblleft", return (str "“"))
651+ , ("textquotedblright", return (str "”"))
652+ , ("textsuperscript", extractSpaces superscript <$> tok)
653+ , ("textsubscript", extractSpaces subscript <$> tok)
654+ , ("textbackslash", lit "\\")
655+ , ("backslash", lit "\\")
656+ , ("slash", lit "/")
657+ , ("textbf", extractSpaces strong <$> tok)
658+ , ("textnormal", extractSpaces (spanWith ("",["nodecor"],[])) <$> tok)
659+ , ("underline", underline <$> tok)
660+ , ("ldots", lit "…")
661+ , ("vdots", lit "\8942")
662+ , ("dots", lit "…")
663+ , ("mdots", lit "…")
664+ , ("sim", lit "~")
665+ , ("sep", lit ",")
666+ , ("label", rawInlineOr "label" dolabel)
667+ , ("ref", rawInlineOr "ref" $ doref "ref")
668+ , ("cref", rawInlineOr "cref" $ doref "ref") -- from cleveref.sty
669+ , ("vref", rawInlineOr "vref" $ doref "ref+page") -- from varioref.sty
670+ , ("eqref", rawInlineOr "eqref" $ doref "eqref") -- from amsmath.sty
671+ , ("mbox", rawInlineOr "mbox" $ processHBox <$> tok)
672+ , ("hbox", rawInlineOr "hbox" $ processHBox <$> tok)
673+ , ("lettrine", rawInlineOr "lettrine" lettrine)
674+ , ("(", mathInline . untokenize <$> manyTill anyTok (controlSeq ")"))
675+ , ("[", mathDisplay . untokenize <$> manyTill anyTok (controlSeq "]"))
676+ , ("ensuremath", mathInline . untokenize <$> braced)
677+ , ("texorpdfstring", const <$> tok <*> tok)
678+ , ("P", lit "¶")
679+ , ("S", lit "§")
680+ , ("$", lit "$")
681+ , ("%", lit "%")
682+ , ("&", lit "&")
683+ , ("#", lit "#")
684+ , ("_", lit "_")
685+ , ("{", lit "{")
686+ , ("}", lit "}")
687+ , ("qed", lit "\a0\x25FB")
688+ -- old TeX commands
689+ , ("em", extractSpaces emph <$> inlines)
690+ , ("it", extractSpaces emph <$> inlines)
691+ , ("sl", extractSpaces emph <$> inlines)
692+ , ("bf", extractSpaces strong <$> inlines)
693+ , ("tt", code . stringify . toList <$> inlines)
694+ , ("rm", inlines)
695+ , ("itshape", extractSpaces emph <$> inlines)
696+ , ("slshape", extractSpaces emph <$> inlines)
697+ , ("scshape", extractSpaces smallcaps <$> inlines)
698+ , ("bfseries", extractSpaces strong <$> inlines)
699+ , ("MakeUppercase", makeUppercase <$> tok)
700+ , ("MakeTextUppercase", makeUppercase <$> tok) -- textcase
701+ , ("uppercase", makeUppercase <$> tok)
702+ , ("MakeLowercase", makeLowercase <$> tok)
703+ , ("MakeTextLowercase", makeLowercase <$> tok)
704+ , ("lowercase", makeLowercase <$> tok)
705+ , ("/", pure mempty) -- italic correction
706+ , ("aa", lit "å")
707+ , ("AA", lit "Å")
708+ , ("ss", lit "ß")
709+ , ("o", lit "ø")
710+ , ("O", lit "Ø")
711+ , ("L", lit "Ł")
712+ , ("l", lit "ł")
713+ , ("ae", lit "æ")
714+ , ("AE", lit "Æ")
715+ , ("oe", lit "œ")
716+ , ("OE", lit "Œ")
717+ , ("pounds", lit "£")
718+ , ("euro", lit "€")
719+ , ("copyright", lit "©")
720+ , ("textasciicircum", lit "^")
721+ , ("textasciitilde", lit "~")
722+ , ("H", accent '\779' Nothing) -- hungarumlaut
723+ , ("`", accent '\768' (Just '`')) -- grave
724+ , ("'", accent '\769' (Just '\'')) -- acute
725+ , ("^", accent '\770' (Just '^')) -- circ
726+ , ("~", accent '\771' (Just '~')) -- tilde
727+ , ("\"", accent '\776' Nothing) -- umlaut
728+ , (".", accent '\775' Nothing) -- dot
729+ , ("=", accent '\772' Nothing) -- macron
730+ , ("|", accent '\781' Nothing) -- vertical line above
731+ , ("b", accent '\817' Nothing) -- macron below
732+ , ("c", accent '\807' Nothing) -- cedilla
733+ , ("G", accent '\783' Nothing) -- doublegrave
734+ , ("h", accent '\777' Nothing) -- hookabove
735+ , ("d", accent '\803' Nothing) -- dotbelow
736+ , ("f", accent '\785' Nothing) -- inverted breve
737+ , ("r", accent '\778' Nothing) -- ringabove
738+ , ("t", accent '\865' Nothing) -- double inverted breve
739+ , ("U", accent '\782' Nothing) -- double vertical line above
740+ , ("v", accent '\780' Nothing) -- hacek
741+ , ("u", accent '\774' Nothing) -- breve
742+ , ("k", accent '\808' Nothing) -- ogonek
743+ , ("textogonekcentered", accent '\808' Nothing) -- ogonek
744+ , ("i", lit "ı") -- dotless i
745+ , ("j", lit "ȷ") -- dotless j
746+ , ("newtie", accent '\785' Nothing) -- inverted breve
747+ , ("textcircled", accent '\8413' Nothing) -- combining circle
748+ , ("\\", linebreak <$ (do inTableCell <- sInTableCell <$> getState
749+ guard $ not inTableCell
750+ optional opt
751+ spaces))
752+ , (",", lit "\8198")
753+ , ("@", pure mempty)
754+ , (" ", lit "\160")
755+ , ("ps", pure $ str "PS." <> space)
756+ , ("TeX", lit "TeX")
757+ , ("LaTeX", lit "LaTeX")
758+ , ("bar", lit "|")
759+ , ("textless", lit "<")
760+ , ("textgreater", lit ">")
761+ , ("thanks", skipopts >> note <$> grouped block)
762+ , ("footnote", skipopts >> note <$> grouped block)
763+ , ("passthrough", tok) -- \passthrough macro used by latex writer
764+ -- for listings
765+ , ("verb", doverb)
766+ , ("lstinline", dolstinline)
767+ , ("mintinline", domintinline)
768+ , ("Verb", doverb)
769+ , ("url", (\url -> link url "" (str url)) . unescapeURL . untokenize <$>
770+ bracedUrl)
771+ , ("nolinkurl", code . unescapeURL . untokenize <$> bracedUrl)
772+ , ("href", do url <- bracedUrl
773+ sp
774+ link (unescapeURL $ untokenize url) "" <$> tok)
775+ , ("includegraphics", do options <- option [] keyvals
776+ src <- braced
777+ mkImage options . unescapeURL . removeDoubleQuotes $
778+ untokenize src)
779+ , ("enquote*", enquote True Nothing)
780+ , ("enquote", enquote False Nothing)
781+ -- foreignquote is supposed to use native quote marks
782+ , ("foreignquote*", braced >>= enquote True . Just . untokenize)
783+ , ("foreignquote", braced >>= enquote False . Just . untokenize)
784+ -- hypehnquote uses regular quotes
785+ , ("hyphenquote*", braced >>= enquote True . Just . untokenize)
786+ , ("hyphenquote", braced >>= enquote False . Just . untokenize)
787+ , ("figurename", doTerm Translations.Figure)
788+ , ("prefacename", doTerm Translations.Preface)
789+ , ("refname", doTerm Translations.References)
790+ , ("bibname", doTerm Translations.Bibliography)
791+ , ("chaptername", doTerm Translations.Chapter)
792+ , ("partname", doTerm Translations.Part)
793+ , ("contentsname", doTerm Translations.Contents)
794+ , ("listfigurename", doTerm Translations.ListOfFigures)
795+ , ("listtablename", doTerm Translations.ListOfTables)
796+ , ("indexname", doTerm Translations.Index)
797+ , ("abstractname", doTerm Translations.Abstract)
798+ , ("tablename", doTerm Translations.Table)
799+ , ("enclname", doTerm Translations.Encl)
800+ , ("ccname", doTerm Translations.Cc)
801+ , ("headtoname", doTerm Translations.To)
802+ , ("pagename", doTerm Translations.Page)
803+ , ("seename", doTerm Translations.See)
804+ , ("seealsoname", doTerm Translations.SeeAlso)
805+ , ("proofname", doTerm Translations.Proof)
806+ , ("glossaryname", doTerm Translations.Glossary)
807+ , ("lstlistingname", doTerm Translations.Listing)
808+ , ("cite", citation "cite" NormalCitation False)
809+ , ("Cite", citation "Cite" NormalCitation False)
810+ , ("citep", citation "citep" NormalCitation False)
811+ , ("citep*", citation "citep*" NormalCitation False)
812+ , ("citeal", citation "citeal" NormalCitation False)
813+ , ("citealp", citation "citealp" NormalCitation False)
814+ , ("citealp*", citation "citealp*" NormalCitation False)
815+ , ("autocite", citation "autocite" NormalCitation False)
816+ , ("smartcite", citation "smartcite" NormalCitation False)
817+ , ("footcite", inNote <$> citation "footcite" NormalCitation False)
818+ , ("parencite", citation "parencite" NormalCitation False)
819+ , ("supercite", citation "supercite" NormalCitation False)
820+ , ("footcitetext", inNote <$> citation "footcitetext" NormalCitation False)
821+ , ("citeyearpar", citation "citeyearpar" SuppressAuthor False)
822+ , ("citeyear", citation "citeyear" SuppressAuthor False)
823+ , ("autocite*", citation "autocite*" SuppressAuthor False)
824+ , ("cite*", citation "cite*" SuppressAuthor False)
825+ , ("parencite*", citation "parencite*" SuppressAuthor False)
826+ , ("textcite", citation "textcite" AuthorInText False)
827+ , ("citet", citation "citet" AuthorInText False)
828+ , ("citet*", citation "citet*" AuthorInText False)
829+ , ("citealt", citation "citealt" AuthorInText False)
830+ , ("citealt*", citation "citealt*" AuthorInText False)
831+ , ("textcites", citation "textcites" AuthorInText True)
832+ , ("cites", citation "cites" NormalCitation True)
833+ , ("autocites", citation "autocites" NormalCitation True)
834+ , ("footcites", inNote <$> citation "footcites" NormalCitation True)
835+ , ("parencites", citation "parencites" NormalCitation True)
836+ , ("supercites", citation "supercites" NormalCitation True)
837+ , ("footcitetexts", inNote <$> citation "footcitetexts" NormalCitation True)
838+ , ("Autocite", citation "Autocite" NormalCitation False)
839+ , ("Smartcite", citation "Smartcite" NormalCitation False)
840+ , ("Footcite", inNote <$> citation "Footcite" NormalCitation False)
841+ , ("Parencite", citation "Parencite" NormalCitation False)
842+ , ("Supercite", citation "Supercite" NormalCitation False)
843+ , ("Footcitetext", inNote <$> citation "Footcitetext" NormalCitation False)
844+ , ("Citeyearpar", citation "Citeyearpar" SuppressAuthor False)
845+ , ("Citeyear", citation "Citeyear" SuppressAuthor False)
846+ , ("Autocite*", citation "Autocite*" SuppressAuthor False)
847+ , ("Cite*", citation "Cite*" SuppressAuthor False)
848+ , ("Parencite*", citation "Parencite*" SuppressAuthor False)
849+ , ("Textcite", citation "Textcite" AuthorInText False)
850+ , ("Textcites", citation "Textcites" AuthorInText True)
851+ , ("Cites", citation "Cites" NormalCitation True)
852+ , ("Autocites", citation "Autocites" NormalCitation True)
853+ , ("Footcites", inNote <$> citation "Footcites" NormalCitation True)
854+ , ("Parencites", citation "Parencites" NormalCitation True)
855+ , ("Supercites", citation "Supercites" NormalCitation True)
856+ , ("Footcitetexts", inNote <$> citation "Footcitetexts" NormalCitation True)
857+ , ("citetext", complexNatbibCitation NormalCitation)
858+ , ("citeauthor", (try (tok *> sp *> controlSeq "citetext") *>
859+ complexNatbibCitation AuthorInText)
860+ <|> citation "citeauthor" AuthorInText False)
861+ , ("nocite", mempty <$ (citation "nocite" NormalCitation False >>=
862+ addMeta "nocite"))
863+ , ("hyperlink", hyperlink)
864+ , ("hypertarget", hypertargetInline)
865+ -- glossaries package
866+ , ("gls", doAcronym "short")
867+ , ("Gls", doAcronym "short")
868+ , ("glsdesc", doAcronym "long")
869+ , ("Glsdesc", doAcronym "long")
870+ , ("GLSdesc", doAcronym "long")
871+ , ("acrlong", doAcronym "long")
872+ , ("Acrlong", doAcronym "long")
873+ , ("acrfull", doAcronym "full")
874+ , ("Acrfull", doAcronym "full")
875+ , ("acrshort", doAcronym "abbrv")
876+ , ("Acrshort", doAcronym "abbrv")
877+ , ("glspl", doAcronymPlural "short")
878+ , ("Glspl", doAcronymPlural "short")
879+ , ("glsdescplural", doAcronymPlural "long")
880+ , ("Glsdescplural", doAcronymPlural "long")
881+ , ("GLSdescplural", doAcronymPlural "long")
882+ -- acronyms package
883+ , ("ac", doAcronym "short")
884+ , ("acf", doAcronym "full")
885+ , ("acs", doAcronym "abbrv")
886+ , ("acl", doAcronym "long")
887+ , ("acp", doAcronymPlural "short")
888+ , ("acfp", doAcronymPlural "full")
889+ , ("acsp", doAcronymPlural "abbrv")
890+ , ("aclp", doAcronymPlural "long")
891+ , ("Ac", doAcronym "short")
892+ , ("Acf", doAcronym "full")
893+ , ("Acs", doAcronym "abbrv")
894+ , ("Acl", doAcronym "long")
895+ , ("Acp", doAcronymPlural "short")
896+ , ("Acfp", doAcronymPlural "full")
897+ , ("Acsp", doAcronymPlural "abbrv")
898+ , ("Aclp", doAcronymPlural "long")
899+ -- siuntix
900+ , ("si", skipopts *> dosi tok)
901+ , ("SI", doSI tok)
902+ , ("SIrange", doSIrange True tok)
903+ , ("numrange", doSIrange False tok)
904+ , ("numlist", doSInumlist)
905+ , ("num", doSInum)
906+ , ("ang", doSIang)
907+ -- hyphenat
908+ , ("bshyp", lit "\\\173")
909+ , ("fshyp", lit "/\173")
910+ , ("dothyp", lit ".\173")
911+ , ("colonhyp", lit ":\173")
912+ , ("hyp", lit "-")
913+ , ("nohyphens", tok)
914+ , ("textnhtt", ttfamily)
915+ , ("nhttfamily", ttfamily)
916+ -- LaTeX colors
917+ , ("textcolor", coloredInline "color")
918+ , ("colorbox", coloredInline "background-color")
919+ -- fontawesome
920+ , ("faCheck", lit "\10003")
921+ , ("faClose", lit "\10007")
922+ -- xspace
923+ , ("xspace", doxspace)
924+ -- etoolbox
925+ , ("ifstrequal", ifstrequal)
926+ , ("newtoggle", braced >>= newToggle)
927+ , ("toggletrue", braced >>= setToggle True)
928+ , ("togglefalse", braced >>= setToggle False)
929+ , ("iftoggle", try $ ifToggle >> inline)
930+ -- biblatex misc
931+ , ("RN", romanNumeralUpper)
932+ , ("Rn", romanNumeralLower)
933+ -- babel
934+ , ("foreignlanguage", foreignlanguage)
935+ -- include
936+ , ("input", rawInlineOr "input" $ include "input")
937+ -- soul package
938+ , ("ul", underline <$> tok)
939+ -- ulem package
940+ , ("uline", underline <$> tok)
941+ -- plain tex stuff that should just be passed through as raw tex
942+ , ("ifdim", ifdim)
943+ ]
944+
945+accent :: PandocMonad m => Char -> Maybe Char -> LP m Inlines
946+accent combiningAccent fallBack = try $ do
947+ ils <- tok
948+ case toList ils of
949+ (Str (T.uncons -> Just (x, xs)) : ys) -> return $ fromList $
950+ -- try to normalize to the combined character:
951+ Str (Normalize.normalize Normalize.NFC
952+ (T.pack [x, combiningAccent]) <> xs) : ys
953+ [Space] -> return $ str $ T.singleton $ fromMaybe combiningAccent fallBack
954+ [] -> return $ str $ T.singleton $ fromMaybe combiningAccent fallBack
955+ _ -> return ils
956+
427957
428958lettrine :: PandocMonad m => LP m Inlines
429959lettrine = do
430− optional rawopt
960+ optional opt
431961 x <- tok
432962 y <- tok
433963 return $ extractSpaces (spanWith ("",["lettrine"],[])) x <> smallcaps y
@@ -447,18 +977,32 @@ alterStr :: (Text -> Text) -> Inline -> Inline
447977alterStr f (Str xs) = Str (f xs)
448978alterStr _ x = x
449979
980+foreignlanguage :: PandocMonad m => LP m Inlines
981+foreignlanguage = do
982+ babelLang <- untokenize <$> braced
983+ case babelLangToBCP47 babelLang of
984+ Just lang -> spanWith ("", [], [("lang", renderLang lang)]) <$> tok
985+ _ -> tok
986+
987+inlineLanguageCommands :: PandocMonad m => M.Map Text (LP m Inlines)
988+inlineLanguageCommands = M.fromList $ mk <$> M.toList polyglossiaLangToBCP47
989+ where
990+ mk (polyglossia, bcp47Func) =
991+ ("text" <> polyglossia, inlineLanguage bcp47Func)
992+
993+inlineLanguage :: PandocMonad m => (Text -> Lang) -> LP m Inlines
994+inlineLanguage bcp47Func = do
995+ o <- option "" $ T.filter (\c -> c /= '[' && c /= ']')
996+ <$> rawopt
997+ let lang = renderLang $ bcp47Func o
998+ extractSpaces (spanWith ("", [], [("lang", lang)])) <$> tok
999+
4501000hyperlink :: PandocMonad m => LP m Inlines
4511001hyperlink = try $ do
4521002 src <- untokenize <$> braced
4531003 lab <- tok
4541004 return $ link ("#" <> src) "" lab
4551005
456−hyperref :: PandocMonad m => LP m Inlines
457−hyperref = try $ do
458− url <- (("#" <>) . untokenize <$> try (sp *> bracketedToks <* sp))
459− <|> untokenize <$> (bracedUrl <* bracedUrl <* bracedUrl)
460− link url "" <$> tok
461−
4621006hypertargetBlock :: PandocMonad m => LP m Blocks
4631007hypertargetBlock = try $ do
4641008 ref <- untokenize <$> braced
@@ -473,6 +1017,31 @@ hypertargetInline = try $ do
4731017 ils <- grouped inline
4741018 return $ spanWith (ref, [], []) ils
4751019
1020+romanNumeralUpper :: (PandocMonad m) => LP m Inlines
1021+romanNumeralUpper =
1022+ str . toRomanNumeral <$> romanNumeralArg
1023+
1024+romanNumeralLower :: (PandocMonad m) => LP m Inlines
1025+romanNumeralLower =
1026+ str . T.toLower . toRomanNumeral <$> romanNumeralArg
1027+
1028+romanNumeralArg :: (PandocMonad m) => LP m Int
1029+romanNumeralArg = spaces *> (parser <|> inBraces)
1030+ where
1031+ inBraces = do
1032+ symbol '{'
1033+ spaces
1034+ res <- parser
1035+ spaces
1036+ symbol '}'
1037+ return res
1038+ parser = do
1039+ Tok _ Word s <- satisfyTok isWordTok
1040+ let (digits, rest) = T.span isDigit s
1041+ unless (T.null rest) $
1042+ Prelude.fail "Non-digits in argument to \\Rn or \\RN"
1043+ safeRead digits
1044+
4761045newToggle :: (Monoid a, PandocMonad m) => [Tok] -> LP m a
4771046newToggle name = do
4781047 updateState $ \st ->
@@ -503,6 +1072,9 @@ ifToggle = do
5031072 report $ UndefinedToggle name' pos
5041073 return ()
5051074
1075+doTerm :: PandocMonad m => Translations.Term -> LP m Inlines
1076+doTerm term = str <$> translateTerm term
1077+
5061078ifstrequal :: (PandocMonad m, Monoid a) => LP m a
5071079ifstrequal = do
5081080 str1 <- tok
@@ -523,6 +1095,13 @@ coloredInline stylename = do
5231095ttfamily :: PandocMonad m => LP m Inlines
5241096ttfamily = code . stringify . toList <$> tok
5251097
1098+rawInlineOr :: PandocMonad m => Text -> LP m Inlines -> LP m Inlines
1099+rawInlineOr name' fallback = do
1100+ parseRaw <- extensionEnabled Ext_raw_tex <$> getOption readerExtensions
1101+ if parseRaw
1102+ then rawInline "latex" <$> getRawCommand name' ("\\" <> name')
1103+ else fallback
1104+
5261105processHBox :: Inlines -> Inlines
5271106processHBox = walk convert
5281107 where
@@ -573,90 +1152,79 @@ treatAsInline = Set.fromList
5731152 , "pagebreak"
5741153 ]
5751154
1155+label :: PandocMonad m => LP m ()
1156+label = do
1157+ controlSeq "label"
1158+ t <- braced
1159+ updateState $ \st -> st{ sLastLabel = Just $ untokenize t }
1160+
1161+dolabel :: PandocMonad m => LP m Inlines
1162+dolabel = do
1163+ v <- braced
1164+ let refstr = untokenize v
1165+ updateState $ \st ->
1166+ st{ sLastLabel = Just refstr }
1167+ return $ spanWith (refstr,[],[("label", refstr)])
1168+ $ inBrackets $ str $ untokenize v
1169+
1170+doref :: PandocMonad m => Text -> LP m Inlines
1171+doref cls = do
1172+ v <- braced
1173+ let refstr = untokenize v
1174+ return $ linkWith ("",[],[ ("reference-type", cls)
1175+ , ("reference", refstr)])
1176+ ("#" <> refstr)
1177+ ""
1178+ (inBrackets $ str refstr)
1179+
5761180lookupListDefault :: (Ord k) => v -> [k] -> M.Map k v -> v
5771181lookupListDefault d = (fromMaybe d .) . lookupList
5781182 where lookupList l m = msum $ map (`M.lookup` m) l
5791183
5801184inline :: PandocMonad m => LP m Inlines
581−inline = do
582− Tok pos toktype t <- lookAhead anyTok
583− let symbolAsString = str . untoken <$> anySymbol
584− let unescapedSymbolAsString =
585− do s <- untoken <$> anySymbol
586− report $ ParsingUnescaped s pos
587− return $ str s
588− case toktype of
589− Comment -> mempty <$ comment
590− Spaces -> space <$ whitespace
591− Newline -> softbreak <$ endline
592− Word -> word
593− Esc1 -> str . T.singleton <$> primEscape
594− Esc2 -> str . T.singleton <$> primEscape
595− Symbol ->
596− case t of
597− "-" -> symbol '-' *>
598− option (str "-") (symbol '-' *>
599− option (str "–") (str "—" <$ symbol '-'))
600− "'" -> symbol '\'' *>
601− option (str "’") (str "”" <$ symbol '\'')
602− "~" -> str "\160" <$ symbol '~'
603− "`" -> doubleQuote <|> singleQuote <|> symbolAsString
604− "\"" -> doubleQuote <|> singleQuote <|> symbolAsString
605− "“" -> doubleQuote <|> symbolAsString
606− "‘" -> singleQuote <|> symbolAsString
607− "$" -> dollarsMath <|> unescapedSymbolAsString
608− "|" -> (guardEnabled Ext_literate_haskell *>
609− symbol '|' *> doLHSverb) <|> symbolAsString
610− "{" -> inlineGroup
611− "#" -> unescapedSymbolAsString
612− "&" -> unescapedSymbolAsString
613− "_" -> unescapedSymbolAsString
614− "^" -> unescapedSymbolAsString
615− "\\" -> mzero
616− "}" -> mzero
617− _ -> symbolAsString
618− CtrlSeq _ -> macroDef (rawInline "latex")
619− <|> inlineCommand'
620− <|> inlineEnvironment
621− <|> inlineGroup
622− _ -> mzero
1185+inline = (mempty <$ comment)
1186+ <|> (space <$ whitespace)
1187+ <|> (softbreak <$ endline)
1188+ <|> word
1189+ <|> macroDef (rawInline "latex")
1190+ <|> inlineCommand'
1191+ <|> inlineEnvironment
1192+ <|> inlineGroup
1193+ <|> (symbol '-' *>
1194+ option (str "-") (symbol '-' *>
1195+ option (str "–") (str "—" <$ symbol '-')))
1196+ <|> doubleQuote
1197+ <|> singleQuote
1198+ <|> (str "”" <$ try (symbol '\'' >> symbol '\''))
1199+ <|> (str "”" <$ symbol '”')
1200+ <|> (str "’" <$ symbol '\'')
1201+ <|> (str "’" <$ symbol '’')
1202+ <|> (str "\160" <$ symbol '~')
1203+ <|> dollarsMath
1204+ <|> (guardEnabled Ext_literate_haskell *> symbol '|' *> doLHSverb)
1205+ <|> (str . T.singleton <$> primEscape)
1206+ <|> regularSymbol
1207+ <|> (do res <- symbolIn "#^'`\"[]&"
1208+ pos <- getPosition
1209+ let s = untoken res
1210+ report $ ParsingUnescaped s pos
1211+ return $ str s)
6231212
6241213inlines :: PandocMonad m => LP m Inlines
6251214inlines = mconcat <$> many inline
6261215
627−opt :: PandocMonad m => LP m Inlines
628−opt = do
629− toks <- try (sp *> bracketedToks <* sp)
630− -- now parse the toks as inlines
631− st <- getState
632− parsed <- runParserT (mconcat <$> many inline) st "bracketed option" toks
633− case parsed of
634− Right result -> return result
635− Left e -> throwError $ PandocParsecError (toSources toks) e
636−
6371216-- block elements:
6381217
6391218preamble :: PandocMonad m => LP m Blocks
6401219preamble = mconcat <$> many preambleBlock
6411220 where preambleBlock = (mempty <$ spaces1)
6421221 <|> macroDef (rawBlock "latex")
643− <|> filecontents
6441222 <|> (mempty <$ blockCommand)
6451223 <|> (mempty <$ braced)
6461224 <|> (do notFollowedBy (begin_ "document")
6471225 anyTok
6481226 return mempty)
6491227
650−rule :: PandocMonad m => LP m Blocks
651−rule = do
652− skipopts
653− width <- T.takeWhile (\c -> isDigit c || c == '.') . stringify <$> tok
654− _thickness <- tok
655− -- 0-width rules are used to fix spacing issues:
656− case safeRead width of
657− Just (0 :: Double) -> return mempty
658− _ -> return horizontalRule
659−
6601228paragraph :: PandocMonad m => LP m Blocks
6611229paragraph = do
6621230 x <- trimInlines . mconcat <$> many1 inline
@@ -694,16 +1262,6 @@ include name = do
6941262 mapM_ (insertIncluded defaultExt) fs
6951263 return mempty
6961264
697−readFileFromTexinputs :: PandocMonad m => FilePath -> LP m (Maybe Text)
698−readFileFromTexinputs fp = do
699− fileContentsMap <- sFileContents <$> getState
700− case M.lookup (T.pack fp) fileContentsMap of
701− Just t -> return (Just t)
702− Nothing -> do
703− dirs <- map T.unpack . splitTextBy (==':') . fromMaybe "."
704− <$> lookupEnv "TEXINPUTS"
705− readFileFromDirs dirs fp
706−
7071265insertIncluded :: PandocMonad m
7081266 => FilePath
7091267 -> FilePath
@@ -713,12 +1271,13 @@ insertIncluded defaultExtension f' = do
7131271 ".tex" -> f'
7141272 ".sty" -> f'
7151273 _ -> addExtension f' defaultExtension
1274+ dirs <- map T.unpack . splitTextBy (==':') . fromMaybe "." <$> lookupEnv "TEXINPUTS"
7161275 pos <- getPosition
7171276 containers <- getIncludeFiles <$> getState
7181277 when (T.pack f `elem` containers) $
7191278 throwError $ PandocParseError $ T.pack $ "Include file loop at " ++ show pos
7201279 updateState $ addIncludeFile $ T.pack f
721− mbcontents <- readFileFromTexinputs f
1280+ mbcontents <- readFileFromDirs dirs f
7221281 contents <- case mbcontents of
7231282 Just s -> return s
7241283 Nothing -> do
@@ -727,6 +1286,10 @@ insertIncluded defaultExtension f' = do
7271286 getInput >>= setInput . (tokenize f contents ++)
7281287 updateState dropLatestIncludeFile
7291288
1289+addMeta :: PandocMonad m => ToMetaValue a => Text -> a -> LP m ()
1290+addMeta field val = updateState $ \st ->
1291+ st{ sMeta = addMetaField field val $ sMeta st }
1292+
7301293authors :: PandocMonad m => LP m ()
7311294authors = try $ do
7321295 bgroup
@@ -735,6 +1298,150 @@ authors = try $ do
7351298 egroup
7361299 addMeta "author" (map trimInlines auths)
7371300
1301+macroDef :: (PandocMonad m, Monoid a) => (Text -> a) -> LP m a
1302+macroDef constructor = do
1303+ (_, s) <- withRaw (commandDef <|> environmentDef)
1304+ (constructor (untokenize s) <$
1305+ guardDisabled Ext_latex_macros)
1306+ <|> return mempty
1307+ where commandDef = do
1308+ (name, macro') <- newcommand <|> letmacro <|> defmacro
1309+ guardDisabled Ext_latex_macros <|>
1310+ updateState (\s -> s{ sMacros = M.insert name macro' (sMacros s) })
1311+ environmentDef = do
1312+ mbenv <- newenvironment
1313+ case mbenv of
1314+ Nothing -> return ()
1315+ Just (name, macro1, macro2) ->
1316+ guardDisabled Ext_latex_macros <|>
1317+ do updateState $ \s -> s{ sMacros =
1318+ M.insert name macro1 (sMacros s) }
1319+ updateState $ \s -> s{ sMacros =
1320+ M.insert ("end" <> name) macro2 (sMacros s) }
1321+ -- @\newenvironment{envname}[n-args][default]{begin}{end}@
1322+ -- is equivalent to
1323+ -- @\newcommand{\envname}[n-args][default]{begin}@
1324+ -- @\newcommand{\endenvname}@
1325+
1326+letmacro :: PandocMonad m => LP m (Text, Macro)
1327+letmacro = do
1328+ controlSeq "let"
1329+ (name, contents) <- withVerbatimMode $ do
1330+ Tok _ (CtrlSeq name) _ <- anyControlSeq
1331+ optional $ symbol '='
1332+ spaces
1333+ -- we first parse in verbatim mode, and then expand macros,
1334+ -- because we don't want \let\foo\bar to turn into
1335+ -- \let\foo hello if we have previously \def\bar{hello}
1336+ contents <- bracedOrToken
1337+ return (name, contents)
1338+ contents' <- doMacros' 0 contents
1339+ return (name, Macro ExpandWhenDefined [] Nothing contents')
1340+
1341+defmacro :: PandocMonad m => LP m (Text, Macro)
1342+defmacro = try $
1343+ -- we use withVerbatimMode, because macros are to be expanded
1344+ -- at point of use, not point of definition
1345+ withVerbatimMode $ do
1346+ controlSeq "def"
1347+ Tok _ (CtrlSeq name) _ <- anyControlSeq
1348+ argspecs <- many (argspecArg <|> argspecPattern)
1349+ contents <- bracedOrToken
1350+ return (name, Macro ExpandWhenUsed argspecs Nothing contents)
1351+
1352+argspecArg :: PandocMonad m => LP m ArgSpec
1353+argspecArg = do
1354+ Tok _ (Arg i) _ <- satisfyTok isArgTok
1355+ return $ ArgNum i
1356+
1357+argspecPattern :: PandocMonad m => LP m ArgSpec
1358+argspecPattern =
1359+ Pattern <$> many1 (satisfyTok (\(Tok _ toktype' txt) ->
1360+ (toktype' == Symbol || toktype' == Word) &&
1361+ (txt /= "{" && txt /= "\\" && txt /= "}")))
1362+
1363+newcommand :: PandocMonad m => LP m (Text, Macro)
1364+newcommand = do
1365+ pos <- getPosition
1366+ Tok _ (CtrlSeq mtype) _ <- controlSeq "newcommand" <|>
1367+ controlSeq "renewcommand" <|>
1368+ controlSeq "providecommand" <|>
1369+ controlSeq "DeclareMathOperator" <|>
1370+ controlSeq "DeclareRobustCommand"
1371+ withVerbatimMode $ do
1372+ Tok _ (CtrlSeq name) txt <- do
1373+ optional (symbol '*')
1374+ anyControlSeq <|>
1375+ (symbol '{' *> spaces *> anyControlSeq <* spaces <* symbol '}')
1376+ spaces
1377+ numargs <- option 0 $ try bracketedNum
1378+ let argspecs = map ArgNum [1..numargs]
1379+ spaces
1380+ optarg <- option Nothing $ Just <$> try bracketedToks
1381+ spaces
1382+ contents' <- bracedOrToken
1383+ let contents =
1384+ case mtype of
1385+ "DeclareMathOperator" ->
1386+ Tok pos (CtrlSeq "mathop") "\\mathop"
1387+ : Tok pos Symbol "{"
1388+ : Tok pos (CtrlSeq "mathrm") "\\mathrm"
1389+ : Tok pos Symbol "{"
1390+ : (contents' ++
1391+ [ Tok pos Symbol "}", Tok pos Symbol "}" ])
1392+ _ -> contents'
1393+ macros <- sMacros <$> getState
1394+ case M.lookup name macros of
1395+ Just macro
1396+ | mtype == "newcommand" -> do
1397+ report $ MacroAlreadyDefined txt pos
1398+ return (name, macro)
1399+ | mtype == "providecommand" -> return (name, macro)
1400+ _ -> return (name, Macro ExpandWhenUsed argspecs optarg contents)
1401+
1402+newenvironment :: PandocMonad m => LP m (Maybe (Text, Macro, Macro))
1403+newenvironment = do
1404+ pos <- getPosition
1405+ Tok _ (CtrlSeq mtype) _ <- controlSeq "newenvironment" <|>
1406+ controlSeq "renewenvironment" <|>
1407+ controlSeq "provideenvironment"
1408+ withVerbatimMode $ do
1409+ optional $ symbol '*'
1410+ spaces
1411+ name <- untokenize <$> braced
1412+ spaces
1413+ numargs <- option 0 $ try bracketedNum
1414+ spaces
1415+ optarg <- option Nothing $ Just <$> try bracketedToks
1416+ let argspecs = map (\i -> ArgNum i) [1..numargs]
1417+ startcontents <- spaces >> bracedOrToken
1418+ endcontents <- spaces >> bracedOrToken
1419+ macros <- sMacros <$> getState
1420+ case M.lookup name macros of
1421+ Just _
1422+ | mtype == "newenvironment" -> do
1423+ report $ MacroAlreadyDefined name pos
1424+ return Nothing
1425+ | mtype == "provideenvironment" ->
1426+ return Nothing
1427+ _ -> return $ Just (name,
1428+ Macro ExpandWhenUsed argspecs optarg startcontents,
1429+ Macro ExpandWhenUsed [] Nothing endcontents)
1430+
1431+bracketedNum :: PandocMonad m => LP m Int
1432+bracketedNum = do
1433+ ds <- untokenize <$> bracketedToks
1434+ case safeRead ds of
1435+ Just i -> return i
1436+ _ -> return 0
1437+
1438+setCaption :: PandocMonad m => LP m ()
1439+setCaption = try $ do
1440+ skipopts
1441+ ils <- tok
1442+ optional $ try $ spaces *> label
1443+ updateState $ \st -> st{ sCaption = Just ils }
1444+
7381445looseItem :: PandocMonad m => LP m Blocks
7391446looseItem = do
7401447 inListItem <- sInListItem <$> getState
@@ -748,6 +1455,10 @@ epigraph = do
7481455 p2 <- grouped block
7491456 return $ divWith ("", ["epigraph"], []) (p1 <> p2)
7501457
1458+resetCaption :: PandocMonad m => LP m ()
1459+resetCaption = updateState $ \st -> st{ sCaption = Nothing
1460+ , sLastLabel = Nothing }
1461+
7511462section :: PandocMonad m => Attr -> Int -> LP m Blocks
7521463section (ident, classes, kvs) lvl = do
7531464 skipopts
@@ -840,7 +1551,7 @@ blockCommands = M.fromList
8401551 , ("address", mempty <$ (skipopts *> tok >>= addMeta "address"))
8411552 , ("signature", mempty <$ (skipopts *> authors))
8421553 , ("date", mempty <$ (skipopts *> tok >>= addMeta "date"))
843− , ("newtheorem", newtheorem inline)
1554+ , ("newtheorem", newtheorem)
8441555 , ("theoremstyle", theoremstyle)
8451556 -- KOMA-Script metadata commands
8461557 , ("extratitle", mempty <$ (skipopts *> tok >>= addMeta "extratitle"))
@@ -884,11 +1595,11 @@ blockCommands = M.fromList
8841595 --
8851596 , ("hrule", pure horizontalRule)
8861597 , ("strut", pure mempty)
887− , ("rule", rule)
1598+ , ("rule", skipopts *> tok *> tok $> horizontalRule)
8881599 , ("item", looseItem)
8891600 , ("documentclass", skipopts *> braced *> preamble)
8901601 , ("centerline", para . trimInlines <$> (skipopts *> tok))
891− , ("caption", mempty <$ setCaption inline)
1602+ , ("caption", mempty <$ setCaption)
8921603 , ("bibliography", mempty <$ (skipopts *> braced >>=
8931604 addMeta "bibliography" . splitBibs . untokenize))
8941605 , ("addbibresource", mempty <$ (skipopts *> braced >>=
@@ -926,8 +1637,7 @@ blockCommands = M.fromList
9261637
9271638
9281639environments :: PandocMonad m => M.Map Text (LP m Blocks)
929−environments = M.union (tableEnvironments blocks inline) $
930− M.fromList
1640+environments = M.fromList
9311641 [ ("document", env "document" blocks <* skipMany anyTok)
9321642 , ("abstract", mempty <$ (env "abstract" blocks >>= addMeta "abstract"))
9331643 , ("sloppypar", env "sloppypar" blocks)
@@ -936,7 +1646,14 @@ environments = M.union (tableEnvironments blocks inline) $
9361646 skipopts *> spaces *> optional braced *> spaces *> blocks)
9371647 , ("figure", env "figure" $ skipopts *> figure)
9381648 , ("subfigure", env "subfigure" $ skipopts *> tok *> figure)
939− , ("center", divWith ("", ["center"], []) <$> env "center" blocks)
1649+ , ("center", env "center" blocks)
1650+ , ("longtable", env "longtable" $
1651+ resetCaption *> simpTable "longtable" False >>= addTableCaption)
1652+ , ("table", env "table" $
1653+ skipopts *> resetCaption *> blocks >>= addTableCaption)
1654+ , ("tabular*", env "tabular*" $ simpTable "tabular*" True)
1655+ , ("tabularx", env "tabularx" $ simpTable "tabularx" True)
1656+ , ("tabular", env "tabular" $ simpTable "tabular" False)
9401657 , ("quote", blockQuote <$> env "quote" blocks)
9411658 , ("quotation", blockQuote <$> env "quotation" blocks)
9421659 , ("verse", blockQuote <$> env "verse" blocks)
@@ -959,7 +1676,7 @@ environments = M.union (tableEnvironments blocks inline) $
9591676 , ("lilypond", rawVerbEnv "lilypond")
9601677 , ("ly", rawVerbEnv "ly")
9611678 -- amsthm
962− , ("proof", proof blocks opt)
1679+ , ("proof", proof)
9631680 -- etoolbox
9641681 , ("ifstrequal", ifstrequal)
9651682 , ("newtoggle", braced >>= newToggle)
@@ -968,29 +1685,130 @@ environments = M.union (tableEnvironments blocks inline) $
9681685 , ("iftoggle", try $ ifToggle >> block)
9691686 ]
9701687
971−filecontents :: PandocMonad m => LP m Blocks
972−filecontents = try $ do
973− controlSeq "begin"
1688+theoremstyle :: PandocMonad m => LP m Blocks
1689+theoremstyle = do
1690+ stylename <- untokenize <$> braced
1691+ let mbstyle = case stylename of
1692+ "plain" -> Just PlainStyle
1693+ "definition" -> Just DefinitionStyle
1694+ "remark" -> Just RemarkStyle
1695+ _ -> Nothing
1696+ case mbstyle of
1697+ Nothing -> return ()
1698+ Just sty -> updateState $ \s -> s{ sLastTheoremStyle = sty }
1699+ return mempty
1700+
1701+newtheorem :: PandocMonad m => LP m Blocks
1702+newtheorem = do
1703+ number <- option True (False <$ symbol '*' <* sp)
9741704 name <- untokenize <$> braced
975− guard $ name == "filecontents" || name == "filecontents*"
976− skipopts
977− fp <- untokenize <$> braced
978− txt <- verbEnv name
979− updateState $ \st ->
980− st{ sFileContents = M.insert fp txt (sFileContents st) }
1705+ sp
1706+ series <- option Nothing $ Just . untokenize <$> bracketedToks
1707+ sp
1708+ showName <- tok
1709+ sp
1710+ syncTo <- option Nothing $ Just . untokenize <$> bracketedToks
1711+ sty <- sLastTheoremStyle <$> getState
1712+ let spec = TheoremSpec { theoremName = showName
1713+ , theoremStyle = sty
1714+ , theoremSeries = series
1715+ , theoremSyncTo = syncTo
1716+ , theoremNumber = number
1717+ , theoremLastNum = DottedNum [0] }
1718+ tmap <- sTheoremMap <$> getState
1719+ updateState $ \s -> s{ sTheoremMap =
1720+ M.insert name spec tmap }
9811721 return mempty
9821722
1723+proof :: PandocMonad m => LP m Blocks
1724+proof = do
1725+ title <- option (B.text "Proof") opt
1726+ bs <- env "proof" blocks
1727+ return $
1728+ B.divWith ("", ["proof"], []) $
1729+ addQed $ addTitle (B.emph (title <> ".")) bs
1730+
1731+addTitle :: Inlines -> Blocks -> Blocks
1732+addTitle ils bs =
1733+ case B.toList bs of
1734+ (Para xs : rest)
1735+ -> B.fromList (Para (B.toList ils ++ (Space : xs)) : rest)
1736+ _ -> B.para ils <> bs
1737+
1738+addQed :: Blocks -> Blocks
1739+addQed bs =
1740+ case Seq.viewr (B.unMany bs) of
1741+ s Seq.:> Para ils
1742+ -> B.Many (s Seq.|> Para (ils ++ B.toList qedSign))
1743+ _ -> bs <> B.para qedSign
1744+ where
1745+ qedSign = B.str "\xa0\x25FB"
1746+
9831747environment :: PandocMonad m => LP m Blocks
9841748environment = try $ do
9851749 controlSeq "begin"
9861750 name <- untokenize <$> braced
9871751 M.findWithDefault mzero name environments <|>
988− theoremEnvironment blocks opt name <|>
1752+ theoremEnvironment name <|>
9891753 if M.member name (inlineEnvironments
9901754 :: M.Map Text (LP PandocPure Inlines))
9911755 then mzero
9921756 else try (rawEnv name) <|> rawVerbEnv name
9931757
1758+theoremEnvironment :: PandocMonad m => Text -> LP m Blocks
1759+theoremEnvironment name = do
1760+ tmap <- sTheoremMap <$> getState
1761+ case M.lookup name tmap of
1762+ Nothing -> mzero
1763+ Just tspec -> do
1764+ optTitle <- option mempty $ (\x -> space <> "(" <> x <> ")") <$> opt
1765+ mblabel <- option Nothing $ Just . untokenize <$>
1766+ try (spaces >> controlSeq "label" >> spaces >> braced)
1767+ bs <- env name blocks
1768+ number <-
1769+ if theoremNumber tspec
1770+ then do
1771+ let name' = fromMaybe name $ theoremSeries tspec
1772+ num <- getNextNumber
1773+ (maybe (DottedNum [0]) theoremLastNum .
1774+ M.lookup name' . sTheoremMap)
1775+ updateState $ \s ->
1776+ s{ sTheoremMap =
1777+ M.adjust
1778+ (\spec -> spec{ theoremLastNum = num })
1779+ name'
1780+ (sTheoremMap s)
1781+ }
1782+
1783+ case mblabel of
1784+ Just ident ->
1785+ updateState $ \s ->
1786+ s{ sLabels = M.insert ident
1787+ (B.toList $
1788+ theoremName tspec <> "\160" <>
1789+ str (renderDottedNum num)) (sLabels s) }
1790+ Nothing -> return ()
1791+ return $ space <> B.text (renderDottedNum num)
1792+ else return mempty
1793+ let titleEmph = case theoremStyle tspec of
1794+ PlainStyle -> B.strong
1795+ DefinitionStyle -> B.strong
1796+ RemarkStyle -> B.emph
1797+ let title = titleEmph (theoremName tspec <> number)
1798+ <> optTitle <> "." <> space
1799+ return $ divWith (fromMaybe "" mblabel, [name], []) $ addTitle title
1800+ $ case theoremStyle tspec of
1801+ PlainStyle -> walk italicize bs
1802+ _ -> bs
1803+
1804+italicize :: Block -> Block
1805+italicize (Para ils) = Para [Emph ils]
1806+italicize (Plain ils) = Plain [Emph ils]
1807+italicize x = x
1808+
1809+env :: PandocMonad m => Text -> LP m a -> LP m a
1810+env name p = p <* end_ name
1811+
9941812rawEnv :: PandocMonad m => Text -> LP m Blocks
9951813rawEnv name = do
9961814 exts <- getOption readerExtensions
@@ -998,17 +1816,15 @@ rawEnv name = do
9981816 rawOptions <- mconcat <$> many rawopt
9991817 let beginCommand = "\\begin{" <> name <> "}" <> rawOptions
10001818 pos1 <- getPosition
1819+ (bs, raw) <- withRaw $ env name blocks
10011820 if parseRaw
1002− then do
1003− (_, raw) <- withRaw $ env name blocks
1004− return $ rawBlock "latex"
1821+ then return $ rawBlock "latex"
10051822 $ beginCommand <> untokenize raw
10061823 else do
1007− bs <- env name blocks
10081824 report $ SkippedContent beginCommand pos1
10091825 pos2 <- getPosition
10101826 report $ SkippedContent ("\\end{" <> name <> "}") pos2
1011− return $ divWith ("",[name],[]) bs
1827+ return bs
10121828
10131829rawVerbEnv :: PandocMonad m => Text -> LP m Blocks
10141830rawVerbEnv name = do
@@ -1067,7 +1883,8 @@ inputMinted = do
10671883 pos <- getPosition
10681884 attr <- mintedAttr
10691885 f <- T.filter (/='"') . untokenize <$> braced
1070− mbCode <- readFileFromTexinputs (T.unpack f)
1886+ dirs <- map T.unpack . splitTextBy (==':') . fromMaybe "." <$> lookupEnv "TEXINPUTS"
1887+ mbCode <- readFileFromDirs dirs (T.unpack f)
10711888 rawcode <- case mbCode of
10721889 Just s -> return s
10731890 Nothing -> do
@@ -1153,7 +1970,8 @@ inputListing = do
11531970 pos <- getPosition
11541971 options <- option [] keyvals
11551972 f <- T.filter (/='"') . untokenize <$> braced
1156− mbCode <- readFileFromTexinputs (T.unpack f)
1973+ dirs <- map T.unpack . splitTextBy (==':') . fromMaybe "." <$> lookupEnv "TEXINPUTS"
1974+ mbCode <- readFileFromDirs dirs (T.unpack f)
11571975 codeLines <- case mbCode of
11581976 Just s -> return $ T.lines s
11591977 Nothing -> do
@@ -1162,8 +1980,7 @@ inputListing = do
11621980 let (ident,classes,kvs) = parseListingsOptions options
11631981 let classes' =
11641982 (case listingsLanguage options of
1165− Nothing -> (take 1 (languagesByExtension defaultSyntaxMap
1166− (T.pack $ takeExtension $ T.unpack f)) <>)
1983+ Nothing -> (take 1 (languagesByExtension (T.pack $ takeExtension $ T.unpack f)) <>)
11671984 Just _ -> id) classes
11681985 let firstline = fromMaybe 1 $ lookup "firstline" options >>= safeRead
11691986 let lastline = fromMaybe (length codeLines) $
@@ -1229,23 +2046,358 @@ orderedList' = try $ do
12292046 bs <- listenv "enumerate" (many item)
12302047 return $ orderedListWith (start, style, delim) bs
12312048
2049+-- tables
2050+
2051+hline :: PandocMonad m => LP m ()
2052+hline = try $ do
2053+ spaces
2054+ controlSeq "hline" <|>
2055+ -- booktabs rules:
2056+ controlSeq "toprule" <|>
2057+ controlSeq "bottomrule" <|>
2058+ controlSeq "midrule" <|>
2059+ controlSeq "endhead" <|>
2060+ controlSeq "endfirsthead"
2061+ spaces
2062+ optional opt
2063+ return ()
2064+
2065+lbreak :: PandocMonad m => LP m Tok
2066+lbreak = (controlSeq "\\" <|> controlSeq "tabularnewline")
2067+ <* skipopts <* spaces
2068+
2069+amp :: PandocMonad m => LP m Tok
2070+amp = symbol '&'
2071+
2072+-- Split a Word into individual Symbols (for parseAligns)
2073+splitWordTok :: PandocMonad m => LP m ()
2074+splitWordTok = do
2075+ inp <- getInput
2076+ case inp of
2077+ (Tok spos Word t : rest) ->
2078+ setInput $ map (Tok spos Symbol . T.singleton) (T.unpack t) <> rest
2079+ _ -> return ()
2080+
2081+parseAligns :: PandocMonad m => LP m [(Alignment, ColWidth, ([Tok], [Tok]))]
2082+parseAligns = try $ do
2083+ let maybeBar = skipMany
2084+ (try $ sp *> (() <$ symbol '|' <|> () <$ (symbol '@' >> braced)))
2085+ let cAlign = AlignCenter <$ symbol 'c'
2086+ let lAlign = AlignLeft <$ symbol 'l'
2087+ let rAlign = AlignRight <$ symbol 'r'
2088+ let parAlign = AlignLeft <$ symbol 'p'
2089+ -- aligns from tabularx
2090+ let xAlign = AlignLeft <$ symbol 'X'
2091+ let mAlign = AlignLeft <$ symbol 'm'
2092+ let bAlign = AlignLeft <$ symbol 'b'
2093+ let alignChar = splitWordTok *> ( cAlign <|> lAlign <|> rAlign <|> parAlign
2094+ <|> xAlign <|> mAlign <|> bAlign )
2095+ let alignPrefix = symbol '>' >> braced
2096+ let alignSuffix = symbol '<' >> braced
2097+ let colWidth = try $ do
2098+ symbol '{'
2099+ ds <- trim . untokenize <$> manyTill anyTok (controlSeq "linewidth")
2100+ spaces
2101+ symbol '}'
2102+ return $ safeRead ds
2103+ let alignSpec = do
2104+ pref <- option [] alignPrefix
2105+ spaces
2106+ al <- alignChar
2107+ width <- colWidth <|> option Nothing (do s <- untokenize <$> braced
2108+ pos <- getPosition
2109+ report $ SkippedContent s pos
2110+ return Nothing)
2111+ spaces
2112+ suff <- option [] alignSuffix
2113+ return (al, width, (pref, suff))
2114+ let starAlign = do -- '*{2}{r}' == 'rr', we just expand like a macro
2115+ symbol '*'
2116+ spaces
2117+ ds <- trim . untokenize <$> braced
2118+ spaces
2119+ spec <- braced
2120+ case safeRead ds of
2121+ Just n ->
2122+ getInput >>= setInput . (mconcat (replicate n spec) ++)
2123+ Nothing -> Prelude.fail $ "Could not parse " <> T.unpack ds <> " as number"
2124+ bgroup
2125+ spaces
2126+ maybeBar
2127+ aligns' <- many $ try $ spaces >> optional starAlign >>
2128+ (alignSpec <* maybeBar)
2129+ spaces
2130+ egroup
2131+ spaces
2132+ return $ map toSpec aligns'
2133+ where
2134+ toColWidth (Just w) | w > 0 = ColWidth w
2135+ toColWidth _ = ColWidthDefault
2136+ toSpec (x, y, z) = (x, toColWidth y, z)
2137+
2138+-- N.B. this parser returns a Row that may have erroneous empty cells
2139+-- in it. See the note above fixTableHead for details.
2140+parseTableRow :: PandocMonad m
2141+ => Text -- ^ table environment name
2142+ -> [([Tok], [Tok])] -- ^ pref/suffixes
2143+ -> LP m Row
2144+parseTableRow envname prefsufs = do
2145+ notFollowedBy (spaces *> end_ envname)
2146+ -- add prefixes and suffixes in token stream:
2147+ let celltoks (pref, suff) = do
2148+ prefpos <- getPosition
2149+ contents <- mconcat <$>
2150+ many ( snd <$> withRaw (controlSeq "parbox" >> parbox) -- #5711
2151+ <|>
2152+ snd <$> withRaw (inlineEnvironment <|> dollarsMath)
2153+ <|>
2154+ (do notFollowedBy
2155+ (() <$ amp <|> () <$ lbreak <|> end_ envname)
2156+ count 1 anyTok) )
2157+
2158+ suffpos <- getPosition
2159+ option [] (count 1 amp)
2160+ return $ map (setpos prefpos) pref ++ contents ++ map (setpos suffpos) suff
2161+ rawcells <- mapM celltoks prefsufs
2162+ oldInput <- getInput
2163+ cells <- mapM (\ts -> setInput ts >> parseTableCell) rawcells
2164+ setInput oldInput
2165+ spaces
2166+ return $ Row nullAttr cells
2167+
2168+parseTableCell :: PandocMonad m => LP m Cell
2169+parseTableCell = do
2170+ spaces
2171+ updateState $ \st -> st{ sInTableCell = True }
2172+ cell' <- multicolumnCell
2173+ <|> multirowCell
2174+ <|> parseSimpleCell
2175+ <|> parseEmptyCell
2176+ updateState $ \st -> st{ sInTableCell = False }
2177+ spaces
2178+ return cell'
2179+ where
2180+ -- The parsing of empty cells is important in LaTeX, especially when dealing
2181+ -- with multirow/multicolumn. See #6603.
2182+ parseEmptyCell = spaces $> emptyCell
2183+
2184+cellAlignment :: PandocMonad m => LP m Alignment
2185+cellAlignment = skipMany (symbol '|') *> alignment <* skipMany (symbol '|')
2186+ where
2187+ alignment = do
2188+ c <- untoken <$> singleChar
2189+ return $ case c of
2190+ "l" -> AlignLeft
2191+ "r" -> AlignRight
2192+ "c" -> AlignCenter
2193+ "*" -> AlignDefault
2194+ _ -> AlignDefault
2195+
2196+plainify :: Blocks -> Blocks
2197+plainify bs = case toList bs of
2198+ [Para ils] -> plain (fromList ils)
2199+ _ -> bs
2200+
2201+multirowCell :: PandocMonad m => LP m Cell
2202+multirowCell = controlSeq "multirow" >> do
2203+ -- Full prototype for \multirow macro is:
2204+ -- \multirow[vpos]{nrows}[bigstruts]{width}[vmove]{text}
2205+ -- However, everything except `nrows` and `text` make
2206+ -- sense in the context of the Pandoc AST
2207+ _ <- optional $ symbol '[' *> cellAlignment <* symbol ']' -- vertical position
2208+ nrows <- fmap (fromMaybe 1 . safeRead . untokenize) braced
2209+ _ <- optional $ symbol '[' *> manyTill anyTok (symbol ']') -- bigstrut-related
2210+ _ <- symbol '{' *> manyTill anyTok (symbol '}') -- Cell width
2211+ _ <- optional $ symbol '[' *> manyTill anyTok (symbol ']') -- Length used for fine-tuning
2212+ content <- symbol '{' *> (plainify <$> blocks) <* symbol '}'
2213+ return $ cell AlignDefault (RowSpan nrows) (ColSpan 1) content
2214+
2215+multicolumnCell :: PandocMonad m => LP m Cell
2216+multicolumnCell = controlSeq "multicolumn" >> do
2217+ span' <- fmap (fromMaybe 1 . safeRead . untokenize) braced
2218+ alignment <- symbol '{' *> cellAlignment <* symbol '}'
2219+
2220+ let singleCell = do
2221+ content <- plainify <$> blocks
2222+ return $ cell alignment (RowSpan 1) (ColSpan span') content
2223+
2224+ -- Two possible contents: either a \multirow cell, or content.
2225+ -- E.g. \multicol{1}{c}{\multirow{2}{1em}{content}}
2226+ -- Note that a \multirow cell can be nested in a \multicolumn,
2227+ -- but not the other way around. See #6603
2228+ let nestedCell = do
2229+ (Cell _ _ (RowSpan rs) _ bs) <- multirowCell
2230+ return $ cell
2231+ alignment
2232+ (RowSpan rs)
2233+ (ColSpan span')
2234+ (fromList bs)
2235+
2236+ symbol '{' *> (nestedCell <|> singleCell) <* symbol '}'
2237+
2238+-- Parse a simple cell, i.e. not multirow/multicol
2239+parseSimpleCell :: PandocMonad m => LP m Cell
2240+parseSimpleCell = simpleCell <$> (plainify <$> blocks)
2241+
2242+-- LaTeX tables are stored with empty cells underneath multirow cells
2243+-- denoting the grid spaces taken up by them. More specifically, if a
2244+-- cell spans m rows, then it will overwrite all the cells in the
2245+-- columns it spans for (m-1) rows underneath it, requiring padding
2246+-- cells in these places. These padding cells need to be removed for
2247+-- proper table reading. See #6603.
2248+--
2249+-- These fixTable functions do not otherwise fix up malformed
2250+-- input tables: that is left to the table builder.
2251+fixTableHead :: TableHead -> TableHead
2252+fixTableHead (TableHead attr rows) = TableHead attr rows'
2253+ where
2254+ rows' = fixTableRows rows
2255+
2256+fixTableBody :: TableBody -> TableBody
2257+fixTableBody (TableBody attr rhc th tb)
2258+ = TableBody attr rhc th' tb'
2259+ where
2260+ th' = fixTableRows th
2261+ tb' = fixTableRows tb
2262+
2263+fixTableRows :: [Row] -> [Row]
2264+fixTableRows = fixTableRows' $ repeat Nothing
2265+ where
2266+ fixTableRows' oldHang (Row attr cells : rs)
2267+ = let (newHang, cells') = fixTableRow oldHang cells
2268+ rs' = fixTableRows' newHang rs
2269+ in Row attr cells' : rs'
2270+ fixTableRows' _ [] = []
2271+
2272+-- The overhang is represented as Just (relative cell dimensions) or
2273+-- Nothing for an empty grid space.
2274+fixTableRow :: [Maybe (ColSpan, RowSpan)] -> [Cell] -> ([Maybe (ColSpan, RowSpan)], [Cell])
2275+fixTableRow oldHang cells
2276+ -- If there's overhang, drop cells until their total width meets the
2277+ -- width of the occupied grid spaces (or we run out)
2278+ | (n, prefHang, restHang) <- splitHang oldHang
2279+ , n > 0
2280+ = let cells' = dropToWidth getCellW n cells
2281+ (restHang', cells'') = fixTableRow restHang cells'
2282+ in (prefHang restHang', cells'')
2283+ -- Otherwise record the overhang of a pending cell and fix the rest
2284+ -- of the row
2285+ | c@(Cell _ _ h w _):cells' <- cells
2286+ = let h' = max 1 h
2287+ w' = max 1 w
2288+ oldHang' = dropToWidth getHangW w' oldHang
2289+ (newHang, cells'') = fixTableRow oldHang' cells'
2290+ in (toHang w' h' <> newHang, c : cells'')
2291+ | otherwise
2292+ = (oldHang, [])
2293+ where
2294+ getCellW (Cell _ _ _ w _) = w
2295+ getHangW = maybe 1 fst
2296+ getCS (ColSpan n) = n
2297+
2298+ toHang c r
2299+ | r > 1 = [Just (c, r)]
2300+ | otherwise = replicate (getCS c) Nothing
2301+
2302+ -- Take the prefix of the overhang list representing filled grid
2303+ -- spaces. Also return the remainder and the length of this prefix.
2304+ splitHang = splitHang' 0 id
2305+
2306+ splitHang' !n l (Just (c, r):xs)
2307+ = splitHang' (n + c) (l . (toHang c (r-1) ++)) xs
2308+ splitHang' n l xs = (n, l, xs)
2309+
2310+ -- Drop list items until the total width of the dropped items
2311+ -- exceeds the passed width.
2312+ dropToWidth _ n l | n < 1 = l
2313+ dropToWidth wproj n (c:cs) = dropToWidth wproj (n - wproj c) cs
2314+ dropToWidth _ _ [] = []
2315+
2316+simpTable :: PandocMonad m => Text -> Bool -> LP m Blocks
2317+simpTable envname hasWidthParameter = try $ do
2318+ when hasWidthParameter $ () <$ (spaces >> tok)
2319+ skipopts
2320+ colspecs <- parseAligns
2321+ let (aligns, widths, prefsufs) = unzip3 colspecs
2322+ optional $ controlSeq "caption" *> setCaption
2323+ spaces
2324+ optional label
2325+ spaces
2326+ optional lbreak
2327+ spaces
2328+ skipMany hline
2329+ spaces
2330+ header' <- option [] . try . fmap (:[]) $
2331+ parseTableRow envname prefsufs <* lbreak <* many1 hline
2332+ spaces
2333+ rows <- sepEndBy (parseTableRow envname prefsufs)
2334+ (lbreak <* optional (skipMany hline))
2335+ spaces
2336+ optional $ controlSeq "caption" *> setCaption
2337+ spaces
2338+ optional label
2339+ spaces
2340+ optional lbreak
2341+ spaces
2342+ lookAhead $ controlSeq "end" -- make sure we're at end
2343+ let th = fixTableHead $ TableHead nullAttr header'
2344+ let tbs = [fixTableBody $ TableBody nullAttr 0 [] rows]
2345+ let tf = TableFoot nullAttr []
2346+ return $ table emptyCaption (zip aligns widths) th tbs tf
2347+
2348+addTableCaption :: PandocMonad m => Blocks -> LP m Blocks
2349+addTableCaption = walkM go
2350+ where go (Table attr c spec th tb tf) = do
2351+ st <- getState
2352+ let mblabel = sLastLabel st
2353+ capt <- case (sCaption st, mblabel) of
2354+ (Just ils, Nothing) -> return $ caption Nothing (plain ils)
2355+ (Just ils, Just lab) -> do
2356+ num <- getNextNumber sLastTableNum
2357+ setState
2358+ st{ sLastTableNum = num
2359+ , sLabels = M.insert lab
2360+ [Str (renderDottedNum num)]
2361+ (sLabels st) }
2362+ return $ caption Nothing (plain ils) -- add number??
2363+ (Nothing, _) -> return c
2364+ let attr' = case (attr, mblabel) of
2365+ ((_,classes,kvs), Just ident) ->
2366+ (ident,classes,kvs)
2367+ _ -> attr
2368+ return $ addAttrDiv attr' $ Table nullAttr capt spec th tb tf
2369+ go x = return x
2370+
2371+-- TODO: For now we add a Div to contain table attributes, since
2372+-- most writers don't do anything yet with attributes on Table.
2373+-- This can be removed when that changes.
2374+addAttrDiv :: Attr -> Block -> Block
2375+addAttrDiv ("",[],[]) b = b
2376+addAttrDiv attr b = Div attr [b]
2377+
12322378block :: PandocMonad m => LP m Blocks
12332379block = do
1234− Tok _ toktype _ <- lookAhead anyTok
1235− res <- (case toktype of
1236− Newline -> mempty <$ spaces1
1237− Spaces -> mempty <$ spaces1
1238− Comment -> mempty <$ spaces1
1239− Word -> paragraph
1240− CtrlSeq "begin" -> environment
1241− CtrlSeq _ -> macroDef (rawBlock "latex")
1242− <|> blockCommand
1243− _ -> mzero)
1244− <|> paragraph
1245− <|> grouped block
2380+ res <- (mempty <$ spaces1)
2381+ <|> environment
2382+ <|> macroDef (rawBlock "latex")
2383+ <|> blockCommand
2384+ <|> paragraph
2385+ <|> grouped block
12462386 trace (T.take 60 $ tshow $ B.toList res)
12472387 return res
12482388
12492389blocks :: PandocMonad m => LP m Blocks
12502390blocks = mconcat <$> many block
12512391
2392+setDefaultLanguage :: PandocMonad m => LP m Blocks
2393+setDefaultLanguage = do
2394+ o <- option "" $ T.filter (\c -> c /= '[' && c /= ']')
2395+ <$> rawopt
2396+ polylang <- untokenize <$> braced
2397+ case M.lookup polylang polyglossiaLangToBCP47 of
2398+ Nothing -> return mempty -- TODO mzero? warning?
2399+ Just langFunc -> do
2400+ let l = langFunc o
2401+ setTranslations l
2402+ updateState $ setMeta "lang" $ str (renderLang l)
2403+ return mempty
src/Text/Pandoc/Readers/LaTeX/Citation.hsdeleted−210

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Inline.hsdeleted−397

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Lang.hs+121−200

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Macro.hsdeleted−184

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Math.hsdeleted−221

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Parsing.hs+54−182

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/SIunitx.hs+71−152

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Table.hsdeleted−378

This diff is too big to show here.

src/Text/Pandoc/Readers/LaTeX/Types.hs+2−15

This diff is too big to show here.

src/Text/Pandoc/Readers/Man.hs+11−18

This diff is too big to show here.

src/Text/Pandoc/Readers/Markdown.hs+239−287

This diff is too big to show here.

src/Text/Pandoc/Readers/MediaWiki.hs+14−21

This diff is too big to show here.

src/Text/Pandoc/Readers/Metadata.hs+18−45

This diff is too big to show here.

src/Text/Pandoc/Readers/Muse.hs+9−10

This diff is too big to show here.

src/Text/Pandoc/Readers/Native.hs+6−8

This diff is too big to show here.

src/Text/Pandoc/Readers/OPML.hs+36−22

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt.hs+25−28

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/Arrows/State.hs+2−2

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/ContentReader.hs+13−19

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/Generic/Namespaces.hs+1−2

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/Generic/Utils.hs+24−9

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/Generic/XMLConverter.hs+14−13

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/Namespaces.hs+5−6

This diff is too big to show here.

src/Text/Pandoc/Readers/Odt/StyleReader.hs+17−21

This diff is too big to show here.

src/Text/Pandoc/Readers/Org.hs+8−5

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/BlockStarts.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/Blocks.hs+21−59

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/DocumentTree.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/ExportSettings.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/Inlines.hs+13−13

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/Meta.hs+5−11

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/ParserState.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/Parsing.hs+6−6

This diff is too big to show here.

src/Text/Pandoc/Readers/Org/Shared.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Readers/RST.hs+116−85

This diff is too big to show here.

src/Text/Pandoc/Readers/Roff.hs+18−16

This diff is too big to show here.

src/Text/Pandoc/Readers/TWiki.hs+27−7

This diff is too big to show here.

src/Text/Pandoc/Readers/Textile.hs+90−96

This diff is too big to show here.

src/Text/Pandoc/Readers/TikiWiki.hs+6−6

This diff is too big to show here.

src/Text/Pandoc/Readers/Txt2Tags.hs+22−48

This diff is too big to show here.

src/Text/Pandoc/Readers/Vimwiki.hs+7−12

This diff is too big to show here.

src/Text/Pandoc/RoffChar.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/SelfContained.hs+8−5

This diff is too big to show here.

src/Text/Pandoc/Shared.hs+101−52

This diff is too big to show here.

src/Text/Pandoc/Slides.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Sources.hsdeleted−195

This diff is too big to show here.

src/Text/Pandoc/Templates.hs+1−5

This diff is too big to show here.

src/Text/Pandoc/Translations.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/UTF8.hs+24−21

This diff is too big to show here.

src/Text/Pandoc/UUID.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers.hs+4−9

This diff is too big to show here.

src/Text/Pandoc/Writers/AnnotatedTable.hs−23

This diff is too big to show here.

src/Text/Pandoc/Writers/AsciiDoc.hs+12−43

This diff is too big to show here.

src/Text/Pandoc/Writers/BibTeX.hsdeleted−61

This diff is too big to show here.

src/Text/Pandoc/Writers/CommonMark.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/ConTeXt.hs+39−45

This diff is too big to show here.

src/Text/Pandoc/Writers/CslJson.hs+9−10

This diff is too big to show here.

src/Text/Pandoc/Writers/Custom.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/Docbook.hs+33−114

This diff is too big to show here.

src/Text/Pandoc/Writers/Docx.hs+395−238

This diff is too big to show here.

src/Text/Pandoc/Writers/Docx/StyleMap.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/Docx/Table.hsdeleted−227

This diff is too big to show here.

src/Text/Pandoc/Writers/Docx/Types.hsdeleted−185

This diff is too big to show here.

src/Text/Pandoc/Writers/DokuWiki.hs+2−4

This diff is too big to show here.

src/Text/Pandoc/Writers/EPUB.hs+212−269

This diff is too big to show here.

src/Text/Pandoc/Writers/FB2.hs+77−61

This diff is too big to show here.

src/Text/Pandoc/Writers/GridTable.hsdeleted−157

This diff is too big to show here.

src/Text/Pandoc/Writers/HTML.hs+59−136

This diff is too big to show here.

src/Text/Pandoc/Writers/Haddock.hs+2−13

This diff is too big to show here.

src/Text/Pandoc/Writers/ICML.hs+1−6

This diff is too big to show here.

src/Text/Pandoc/Writers/Ipynb.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/JATS.hs+56−99

This diff is too big to show here.

src/Text/Pandoc/Writers/JATS/References.hsdeleted−164

This diff is too big to show here.

src/Text/Pandoc/Writers/JATS/Table.hs+6−26

This diff is too big to show here.

src/Text/Pandoc/Writers/JATS/Types.hs+4−15

This diff is too big to show here.

src/Text/Pandoc/Writers/Jira.hs+17−42

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX.hs+835−96

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Caption.hsdeleted−47

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Citation.hsdeleted−181

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Lang.hsdeleted−192

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Notes.hsdeleted−34

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Table.hsdeleted−307

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Types.hsdeleted−83

This diff is too big to show here.

src/Text/Pandoc/Writers/LaTeX/Util.hsdeleted−275

This diff is too big to show here.

src/Text/Pandoc/Writers/Man.hs+3−3

This diff is too big to show here.

src/Text/Pandoc/Writers/Markdown.hs+657−85

This diff is too big to show here.

src/Text/Pandoc/Writers/Markdown/Inline.hsdeleted−616

This diff is too big to show here.

src/Text/Pandoc/Writers/Markdown/Types.hsdeleted−81

This diff is too big to show here.

src/Text/Pandoc/Writers/MediaWiki.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/Ms.hs+10−28

This diff is too big to show here.

src/Text/Pandoc/Writers/Muse.hs+3−6

This diff is too big to show here.

src/Text/Pandoc/Writers/Native.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/ODT.hs+22−28

This diff is too big to show here.

src/Text/Pandoc/Writers/OOXML.hs+22−22

This diff is too big to show here.

src/Text/Pandoc/Writers/OPML.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/OpenDocument.hs+72−123

This diff is too big to show here.

src/Text/Pandoc/Writers/Org.hs+19−39

This diff is too big to show here.

src/Text/Pandoc/Writers/Powerpoint.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/Powerpoint/Output.hs+148−172

This diff is too big to show here.

src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+3−2

This diff is too big to show here.

src/Text/Pandoc/Writers/RST.hs+22−38

This diff is too big to show here.

src/Text/Pandoc/Writers/RTF.hs+11−17

This diff is too big to show here.

src/Text/Pandoc/Writers/Roff.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/Shared.hs+6−20

This diff is too big to show here.

src/Text/Pandoc/Writers/TEI.hs+4−5

This diff is too big to show here.

src/Text/Pandoc/Writers/Texinfo.hs+16−23

This diff is too big to show here.

src/Text/Pandoc/Writers/Textile.hs+1−1

This diff is too big to show here.

src/Text/Pandoc/Writers/ZimWiki.hs+3−5

This diff is too big to show here.

src/Text/Pandoc/XML.hs+20−42

This diff is too big to show here.

src/Text/Pandoc/XML/Light.hsdeleted−89

This diff is too big to show here.

src/Text/Pandoc/XML/Light/Output.hsdeleted−234

This diff is too big to show here.

src/Text/Pandoc/XML/Light/Proc.hsdeleted−139

This diff is too big to show here.

src/Text/Pandoc/XML/Light/Types.hsdeleted−193

This diff is too big to show here.

stack.yaml+22−11

This diff is too big to show here.

test/Tests/Command.hs+39−34

This diff is too big to show here.

test/Tests/Helpers.hs+36−25

This diff is too big to show here.

test/Tests/Lua.hs+3−1

This diff is too big to show here.

test/Tests/Lua/Module.hs+1−3

This diff is too big to show here.

test/Tests/Old.hs+27−13

This diff is too big to show here.

test/Tests/Readers/Creole.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Docx.hs+7−25

This diff is too big to show here.

test/Tests/Readers/DokuWiki.hs+2

This diff is too big to show here.

test/Tests/Readers/EPUB.hs+4−2

This diff is too big to show here.

test/Tests/Readers/FB2.hs+2

This diff is too big to show here.

test/Tests/Readers/HTML.hs+8−21

This diff is too big to show here.

test/Tests/Readers/JATS.hs+2−1

This diff is too big to show here.

test/Tests/Readers/Jira.hs+2−22

This diff is too big to show here.

test/Tests/Readers/LaTeX.hs+21−2

This diff is too big to show here.

test/Tests/Readers/Man.hs+2−1

This diff is too big to show here.

test/Tests/Readers/Markdown.hs+6−4

This diff is too big to show here.

test/Tests/Readers/Muse.hs+2

This diff is too big to show here.

test/Tests/Readers/Odt.hs+3−2

This diff is too big to show here.

test/Tests/Readers/Org.hs+1−1

This diff is too big to show here.

test/Tests/Readers/Org/Block.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Block/CodeBlock.hs+6−4

This diff is too big to show here.

test/Tests/Readers/Org/Block/Figure.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Block/Header.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Block/List.hs+3−14

This diff is too big to show here.

test/Tests/Readers/Org/Block/Table.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Directive.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Inline.hs+5−3

This diff is too big to show here.

test/Tests/Readers/Org/Inline/Citation.hs+3−41

This diff is too big to show here.

test/Tests/Readers/Org/Inline/Note.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Inline/Smart.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Org/Meta.hs+5−15

This diff is too big to show here.

test/Tests/Readers/Org/Shared.hs+3−1

This diff is too big to show here.

test/Tests/Readers/RST.hs+3−1

This diff is too big to show here.

test/Tests/Readers/Txt2Tags.hs+3−1

This diff is too big to show here.

test/Tests/Shared.hs+3−1

This diff is too big to show here.

test/Tests/Writers/AnnotatedTable.hs+1

This diff is too big to show here.

test/Tests/Writers/AsciiDoc.hs+2

This diff is too big to show here.

test/Tests/Writers/ConTeXt.hs+115−122

This diff is too big to show here.

test/Tests/Writers/Docbook.hs+2−103

This diff is too big to show here.

test/Tests/Writers/Docx.hs+2−15

This diff is too big to show here.

test/Tests/Writers/FB2.hs+2

This diff is too big to show here.

test/Tests/Writers/HTML.hs+54−57

This diff is too big to show here.

test/Tests/Writers/JATS.hs+93−135

This diff is too big to show here.

test/Tests/Writers/Jira.hs+1−60

This diff is too big to show here.

test/Tests/Writers/LaTeX.hs+2

This diff is too big to show here.

test/Tests/Writers/Markdown.hs+2

This diff is too big to show here.

test/Tests/Writers/Ms.hs+2

This diff is too big to show here.

test/Tests/Writers/Muse.hs+1

This diff is too big to show here.

test/Tests/Writers/Native.hs+2

This diff is too big to show here.

test/Tests/Writers/OOXML.hs+4−5

This diff is too big to show here.

test/Tests/Writers/Org.hs+13−46

This diff is too big to show here.

test/Tests/Writers/Plain.hs+2

This diff is too big to show here.

test/Tests/Writers/Powerpoint.hs+2

This diff is too big to show here.

test/Tests/Writers/RST.hs+2

This diff is too big to show here.

test/Tests/Writers/TEI.hs+2

This diff is too big to show here.

test/command/1279.md+1−1

This diff is too big to show here.

test/command/1881.md+3−3

This diff is too big to show here.

test/command/2103.mddeleted−8

This diff is too big to show here.

test/command/2378.md+3−3

This diff is too big to show here.

test/command/2874.md+2−2

This diff is too big to show here.

test/command/3516.md+5−1

This diff is too big to show here.

test/command/3752.mddeleted−60

This diff is too big to show here.

test/command/3792.md+1−1

This diff is too big to show here.

test/command/4063.mddeleted−29

This diff is too big to show here.

test/command/4164.md+1−1

This diff is too big to show here.

test/command/4420.md+11

This diff is too big to show here.

test/command/4442.md+1−1

This diff is too big to show here.

test/command/4465.mddeleted−15

This diff is too big to show here.

test/command/5304.md+1−1

This diff is too big to show here.

test/command/5321.md+2−2

This diff is too big to show here.

test/command/5367.md+3−7

This diff is too big to show here.

test/command/5519.md+1−1

This diff is too big to show here.

test/command/5541-nesting.md−1

This diff is too big to show here.

test/command/5686.md+1−1

This diff is too big to show here.

test/command/5708.md+5−1

This diff is too big to show here.

test/command/5986.md+2−3

This diff is too big to show here.

test/command/6026.mddeleted−19

This diff is too big to show here.

test/command/6388.mddeleted−16

This diff is too big to show here.

test/command/6541.mddeleted−27

This diff is too big to show here.

test/command/6620.md−3

This diff is too big to show here.

test/command/6658.mddeleted−75

This diff is too big to show here.

test/command/6709.mddeleted−11

This diff is too big to show here.

test/command/6774.mddeleted−63

This diff is too big to show here.

test/command/6890.md+2−2

This diff is too big to show here.

test/command/6925.mddeleted−34

This diff is too big to show here.

test/command/6948.mddeleted−31

This diff is too big to show here.

test/command/6951.mddeleted−18

This diff is too big to show here.

test/command/6958.mddeleted−10

This diff is too big to show here.

test/command/6993.mddeleted−21

This diff is too big to show here.

test/command/7003.mddeleted−37

This diff is too big to show here.

test/command/7006.mddeleted−20

This diff is too big to show here.

test/command/7009.mddeleted−8

This diff is too big to show here.

test/command/7016.mddeleted−48

This diff is too big to show here.

test/command/7041.mddeleted−23

This diff is too big to show here.

test/command/7042.mddeleted−146

This diff is too big to show here.

test/command/7058.mddeleted−6

This diff is too big to show here.

test/command/7064.mddeleted−32

This diff is too big to show here.

test/command/7067.mddeleted−90

This diff is too big to show here.

test/command/7080.mddeleted−8

This diff is too big to show here.

test/command/7092.mddeleted−8

This diff is too big to show here.

test/command/7099.mddeleted−18

This diff is too big to show here.

test/command/7112.mddeleted−15

This diff is too big to show here.

test/command/7129.mddeleted−27

This diff is too big to show here.

test/command/7132.mddeleted−10

This diff is too big to show here.

test/command/7134.mddeleted−16

This diff is too big to show here.

test/command/7145.mddeleted−12

This diff is too big to show here.

test/command/7155.mddeleted−15

This diff is too big to show here.

test/command/7173.mddeleted−137

This diff is too big to show here.

test/command/7208.mddeleted−6

This diff is too big to show here.

test/command/7214.mddeleted−28

This diff is too big to show here.

test/command/7216.mddeleted−19

This diff is too big to show here.

test/command/7266.mddeleted−10

This diff is too big to show here.

test/command/7272.mddeleted−27

This diff is too big to show here.

test/command/7278.mddeleted−38

This diff is too big to show here.

test/command/7282.mddeleted−7

This diff is too big to show here.

test/command/7288.mddeleted−40

This diff is too big to show here.

test/command/7299.mddeleted−23

This diff is too big to show here.

test/command/7321.mddeleted−24

This diff is too big to show here.

test/command/7323.mddeleted−29

This diff is too big to show here.

test/command/7324.mddeleted−25

This diff is too big to show here.

test/command/7339.mddeleted−11

This diff is too big to show here.

test/command/7340.mddeleted−6

This diff is too big to show here.

test/command/7376.mddeleted−16

This diff is too big to show here.

test/command/7394.mddeleted−85

This diff is too big to show here.

test/command/7397.mddeleted−14

This diff is too big to show here.

test/command/7400.mddeleted−9

This diff is too big to show here.

test/command/7416.mddeleted−19

This diff is too big to show here.

test/command/averroes.bibdeleted−95

This diff is too big to show here.

test/command/biblatex-baez-article.md+1−1

This diff is too big to show here.

test/command/biblatex-baez-online.md+1−1

This diff is too big to show here.

test/command/biblatex-basic.md+3−5

This diff is too big to show here.

test/command/biblatex-cotton.md+3−3

This diff is too big to show here.

test/command/biblatex-examples.bibnew file+1674

This diff is too big to show here.

test/command/biblatex-inproceedings.md+1−1

This diff is too big to show here.

test/command/biblatex-itzhaki.md+1−1

This diff is too big to show here.

test/command/biblatex-jaffe.md+2−2

This diff is too big to show here.

test/command/biblatex-moraux.md+1−1

This diff is too big to show here.

test/command/biblatex-murray.md+2−3

This diff is too big to show here.

test/command/biblatex-nussbaum.mdnew file+63

This diff is too big to show here.

test/command/biblatex-quotes.md+1−1

This diff is too big to show here.

test/command/biblatex-sigfridsson.md+1−1

This diff is too big to show here.

test/command/biblatex-spiegelberg.md+2−2

This diff is too big to show here.

test/command/biblatex-test-case-conversion.md+2−2

This diff is too big to show here.

test/command/biblatex-textnormal.md+1−1

This diff is too big to show here.

test/command/biblatex-wassenberg.md+1−1

This diff is too big to show here.

test/command/biblatex-wilde.md+1−1

This diff is too big to show here.

test/command/chap1/spider.pngdeleted

Binary file not shown.

test/command/chap1/text.mddeleted−17

This diff is too big to show here.

test/command/chap2/spider.pngdeleted

Binary file not shown.

test/command/chap2/text.mddeleted−5

This diff is too big to show here.

test/command/custom-attributes.htmlnew file+16

This diff is too big to show here.

test/command/defaults-inheritance-1.mddeleted−6

This diff is too big to show here.

test/command/defaults-inheritance-2.mddeleted−5

This diff is too big to show here.

test/command/defaults-inheritance-3.mddeleted−6

This diff is too big to show here.

test/command/defaults3.yamldeleted−4

This diff is too big to show here.

test/command/defaults4.yamldeleted−3

This diff is too big to show here.

test/command/defaults5.yamldeleted−2

This diff is too big to show here.

test/command/defaults6.yamldeleted−2

This diff is too big to show here.

test/command/defaults7.yamldeleted−2

This diff is too big to show here.

test/command/defaults8.yamldeleted−2

This diff is too big to show here.

test/command/defaults9.yamldeleted−1

This diff is too big to show here.

test/command/duplicate_attributes.mddeleted−7

This diff is too big to show here.

test/command/empty-inline-code.txtnew file+6

This diff is too big to show here.

test/command/latex-center.mddeleted−13

This diff is too big to show here.

test/command/mmd-metadata.mddeleted−20

This diff is too big to show here.

test/command/nested-table-to-asciidoc-6942.mddeleted−82

This diff is too big to show here.

test/command/newif.mddeleted−55

This diff is too big to show here.

test/command/pandoc-citeproc-119.md+1−1

This diff is too big to show here.

test/command/pandoc-citeproc-320a.md+3−3

This diff is too big to show here.

test/command/pandoc-citeproc-322.md+1−1

This diff is too big to show here.

test/command/pandoc-citeproc-356.md+7−2

This diff is too big to show here.

test/command/pandoc-citeproc-53.md+2−4

This diff is too big to show here.

test/command/pandoc-citeproc-65.md+1−1

This diff is too big to show here.

test/command/pandoc-citeproc-70.md+2−2

This diff is too big to show here.

test/command/pandoc-citeproc-chicago-author-date.md+7−7

This diff is too big to show here.

test/command/pandoc-citeproc-chicago-fullnote-bibliography.md+18−15

This diff is too big to show here.

test/command/svg.md+15−17

This diff is too big to show here.

test/command/toc.md+2−2

This diff is too big to show here.

test/command/unicode-collation.mddeleted−130

This diff is too big to show here.

test/docbook-reader.docbook+2−9

This diff is too big to show here.

test/docx/0_level_headers.native+2−2

This diff is too big to show here.

test/docx/golden/block_quotes.docx

Binary file not shown.

test/docx/golden/codeblock.docx

Binary file not shown.

test/docx/golden/comments.docx

Binary file not shown.

test/docx/golden/custom_style_no_reference.docx

Binary file not shown.

test/docx/golden/custom_style_preserve.docx

Binary file not shown.

test/docx/golden/custom_style_reference.docx

Binary file not shown.

test/docx/golden/definition_list.docx

Binary file not shown.

test/docx/golden/document-properties-short-desc.docx

Binary file not shown.

test/docx/golden/document-properties.docx

Binary file not shown.

test/docx/golden/headers.docx

Binary file not shown.

test/docx/golden/image.docx

Binary file not shown.

test/docx/golden/inline_code.docx

Binary file not shown.

test/docx/golden/inline_formatting.docx

Binary file not shown.

test/docx/golden/inline_images.docx

Binary file not shown.

test/docx/golden/link_in_notes.docx

Binary file not shown.

test/docx/golden/links.docx

Binary file not shown.

test/docx/golden/lists.docx

Binary file not shown.

test/docx/golden/lists_continuing.docx

Binary file not shown.

test/docx/golden/lists_multiple_initial.docx

Binary file not shown.

test/docx/golden/lists_restarting.docx

Binary file not shown.

test/docx/golden/nested_anchors_in_header.docx

Binary file not shown.

test/docx/golden/notes.docx

Binary file not shown.

test/docx/golden/raw-blocks.docxdeleted

Binary file not shown.

test/docx/golden/raw-bookmarks.docxdeleted

Binary file not shown.

test/docx/golden/table_one_row.docx

Binary file not shown.

test/docx/golden/table_with_list_cell.docx

Binary file not shown.

test/docx/golden/tables-default-widths.docxdeleted

Binary file not shown.

test/docx/golden/tables.docx

Binary file not shown.

test/docx/golden/track_changes_deletion.docx

Binary file not shown.

test/docx/golden/track_changes_insertion.docx

Binary file not shown.

test/docx/golden/track_changes_move.docx

Binary file not shown.

test/docx/golden/track_changes_scrubbed_metadata.docx

Binary file not shown.

test/docx/golden/unicode.docx

Binary file not shown.

test/docx/golden/verbatim_subsuper.docx

Binary file not shown.

test/docx/image_vml_as_object.docxdeleted

Binary file not shown.

test/docx/image_vml_as_object.nativedeleted−2

This diff is too big to show here.

test/docx/raw-blocks.nativedeleted−6

This diff is too big to show here.

test/docx/raw-bookmarks.nativedeleted−3

This diff is too big to show here.

test/docx/sdt_elements.native+10−9

This diff is too big to show here.

test/docx/table_captions_no_field.docxdeleted

Binary file not shown.

test/docx/table_captions_no_field.nativedeleted−34

This diff is too big to show here.

test/docx/table_captions_with_field.docxdeleted

Binary file not shown.

test/docx/table_captions_with_field.nativedeleted−54

This diff is too big to show here.

test/docx/table_header_rowspan.docxdeleted

Binary file not shown.

test/docx/table_header_rowspan.nativedeleted−189

This diff is too big to show here.

test/docx/table_one_header_row.docxdeleted

Binary file not shown.

test/docx/table_one_header_row.nativedeleted−18

This diff is too big to show here.

test/docx/table_one_row.docx

Binary file not shown.

test/docx/table_one_row.native+4−4

This diff is too big to show here.

test/docx/table_variable_width.native+17−12

This diff is too big to show here.

test/docx/table_with_list_cell.native+3−3

This diff is too big to show here.

test/docx/tables-default-widths.nativedeleted−92

This diff is too big to show here.

test/docx/tables.native+9−9

This diff is too big to show here.

test/epub/features.native+2−2

This diff is too big to show here.

test/epub/wasteland.epub

Binary file not shown.

test/epub/wasteland.native+4−4

This diff is too big to show here.

test/fb2/images.fb2+1−1

This diff is too big to show here.

test/fb2/test.jpg

Binary file not shown.

test/grofftest.shnew file+26

This diff is too big to show here.

test/html-reader.html−76

This diff is too big to show here.

test/html-reader.native+37−148

This diff is too big to show here.

test/jats-reader.native+1−1

This diff is too big to show here.

test/jats-reader.xml−1

This diff is too big to show here.

test/jira-reader.native+1−1

This diff is too big to show here.

test/latex-reader.native+13−26

This diff is too big to show here.

test/lhs-test.html+1−3

This diff is too big to show here.

test/lhs-test.html+lhs+1−3

This diff is too big to show here.

test/lhs-test.latex+3−3

This diff is too big to show here.

test/lhs-test.latex+lhs+3−3

This diff is too big to show here.

test/lua/module/pandoc-path.luadeleted−44

This diff is too big to show here.

test/markdown-reader-more.native+33−5

This diff is too big to show here.

test/odt/native/tab.nativedeleted−1

This diff is too big to show here.

test/odt/odt/tab.odtdeleted

Binary file not shown.

test/pipe-tables.native+10−2

This diff is too big to show here.

test/pptx/code-custom.pptx

Binary file not shown.

test/pptx/code-custom_templated.pptx

Binary file not shown.

test/pptx/code.pptx

Binary file not shown.

test/pptx/code_templated.pptx

Binary file not shown.

test/pptx/document-properties-short-desc.pptx

Binary file not shown.

test/pptx/document-properties-short-desc_templated.pptx

Binary file not shown.

test/pptx/document-properties.pptx

Binary file not shown.

test/pptx/document-properties_templated.pptx

Binary file not shown.

test/pptx/endnotes.pptx

Binary file not shown.

test/pptx/endnotes_templated.pptx

Binary file not shown.

test/pptx/endnotes_toc.pptx

Binary file not shown.

test/pptx/endnotes_toc_templated.pptx

Binary file not shown.

test/pptx/images.pptx

Binary file not shown.

test/pptx/images_templated.pptx

Binary file not shown.

test/pptx/inline_formatting.pptx

Binary file not shown.

test/pptx/inline_formatting_templated.pptx

Binary file not shown.

test/pptx/lists.pptx

Binary file not shown.

test/pptx/lists_templated.pptx

Binary file not shown.

test/pptx/raw_ooxml.native+1−1

This diff is too big to show here.

test/pptx/raw_ooxml.pptx

Binary file not shown.

test/pptx/raw_ooxml_templated.pptx

Binary file not shown.

test/pptx/reference_depth.pptx

Binary file not shown.

test/pptx/remove_empty_slides.pptx

Binary file not shown.

test/pptx/remove_empty_slides_templated.pptx

Binary file not shown.

test/pptx/slide_breaks.pptx

Binary file not shown.

test/pptx/slide_breaks_slide_level_1.pptx

Binary file not shown.

test/pptx/slide_breaks_slide_level_1_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_templated.pptx

Binary file not shown.

test/pptx/slide_breaks_toc.pptx

Binary file not shown.

test/pptx/slide_breaks_toc_templated.pptx

Binary file not shown.

test/pptx/speaker_notes.pptx

Binary file not shown.

test/pptx/speaker_notes_after_metadata.pptx

Binary file not shown.

test/pptx/speaker_notes_after_metadata_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterheader.pptx

Binary file not shown.

test/pptx/speaker_notes_afterheader_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_afterseps.pptx

Binary file not shown.

test/pptx/speaker_notes_afterseps_templated.pptx

Binary file not shown.

test/pptx/speaker_notes_templated.pptx

Binary file not shown.

test/pptx/start_numbering_at.pptx

Binary file not shown.

test/pptx/start_numbering_at_templated.pptx

Binary file not shown.

test/pptx/tables.pptx

Binary file not shown.

test/pptx/tables_templated.pptx

Binary file not shown.

test/pptx/two_column.pptx

Binary file not shown.

test/pptx/two_column_templated.pptx

Binary file not shown.

test/rst-reader.native+21−3

This diff is too big to show here.

test/tables-rstsubset.native+9−1

This diff is too big to show here.

test/tables.latex+35−50

This diff is too big to show here.

test/tables.ms−44

This diff is too big to show here.

test/tables/nordics.latexdeleted−42

This diff is too big to show here.

test/tables/planets.latexdeleted−30

This diff is too big to show here.

test/tables/students.latexdeleted−31

This diff is too big to show here.

test/test-pandoc.hs+8−18

This diff is too big to show here.

test/txt2tags.native+1−1

This diff is too big to show here.

test/txt2tags.t2t+1−1

This diff is too big to show here.

test/writer.context+40−42

This diff is too big to show here.

test/writer.html4+1−2

This diff is too big to show here.

test/writer.html5+1−2

This diff is too big to show here.

test/writer.icml−2

This diff is too big to show here.

test/writer.jats_archiving+8−10

This diff is too big to show here.

test/writer.jats_articleauthoring+1−1

This diff is too big to show here.

test/writer.jats_publishing+8−10

This diff is too big to show here.

test/writer.jira+35−23

This diff is too big to show here.

test/writer.latex+3−3

This diff is too big to show here.

test/writer.ms+1−1

This diff is too big to show here.

test/writer.opml+1−1

This diff is too big to show here.

test/writer.org+7−7

This diff is too big to show here.

test/writer.zimwiki+3−3

This diff is too big to show here.

test/writers-lang-and-dir.context+2−4

This diff is too big to show here.

test/writers-lang-and-dir.latex+6−6

This diff is too big to show here.

tools/build-arm.shdeleted−110

This diff is too big to show here.

tools/changelog-helper.sh+4−11

This diff is too big to show here.

tools/parseTimings.pldeleted−33

This diff is too big to show here.

trypandoc/index.html+1−1

This diff is too big to show here.

trypandoc/trypandoc.hs+8−3

This diff is too big to show here.

windows/Makefile+1−1

This diff is too big to show here.

windows/pandoc.wxs+2−2

This diff is too big to show here.