@@ -1,5 +1,360 @@
11# Revision history for pandoc
22
3+## pandoc 2.12 (UNRELEASED -- PROVISIONAL)
4+
5+ * Add new unexported module Text.Pandoc.XML.Light, as well
6+ as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc,
7+ Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).
8+
9+ This module exports definitions of `Element` and `Content`
10+ that are isomorphic to xml-light's, but with Text
11+ instead of String. This allows us to keep most of the code in existing
12+ readers that use xml-light, but avoid lots of unnecessary allocation.
13+
14+ We also add versions of the functions from xml-light's
15+ Text.XML.Light.Output and Text.XML.Light.Proc that operate on our
16+ modified XML types, and functions that convert xml-light types to our
17+ types (since some of our dependencies, like texmath, use xml-light).
18+
19+ We export functions that use xml-conduit's parser to produce an
20+ `Element` or `[Content]`. This allows existing pandoc code to use
21+ a better parser without much modification.
22+
23+ The new parser is used in all places where xml-light's parser was
24+ previously used. Benchmarks show a significant performance improvement
25+ in parsing XML-based formats (with docbook, opml, jats, and docx
26+ almost twice as fast, odt and fb2 more than twice as fast).
27+
28+ In addition, the new parser gives us better error reporting than
29+ xml-light. We report XML errors, when possible, using the new
30+ `PandocXMLError` constructor in `PandocError`.
31+
32+ These changes revealed the need for some changes in the tests. The
33+ docbook-reader.docbook test lacked definitions for the entities it used;
34+ these have been added. And the docx golden tests have been updated,
35+ because the new parser does not preserve the order of attributes.
36+
37+ * Text.Pandoc.App
38+
39+ + Add `parseOptionsFromArgs` [API change, new exported function].
40+
41+ * Text.Pandoc.Citeproc.BibTeX
42+
43+ + `Text.Pandoc.Citeproc.writeBibTeXString` now returns
44+ `Doc Text` instead of `Text` (#7068).
45+ + Correctly handle `pages` (= `page` in CSL) (#7067).
46+ + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067).
47+ + In BibTeX output, protect foreign titles since there's no language
48+ field (#7067).
49+ + Clean up BibTeX parsing (#7049). Previously there was a messy code
50+ path that gave strange results in some cases, not passing through raw
51+ tex but trying to extract a string content. This was an artefact of
52+ trying to handle some special bibtex-specific commands in the BibTeX
53+ reader. Now we just handle these in the LaTeX reader and simplify
54+ parsing in the BibTeX reader. This does mean that more raw tex will
55+ be passed through (and currently this is not sensitive to the
56+ `raw_tex` extension; this should be fixed).
57+
58+ * Text.Pandoc.Citeproc.MetaValue
59+
60+ + Correctly parse "raw" date value in markdown references metadata.
61+ (See jgm/citeproc#53.)
62+
63+ * Text.Pandoc.Class
64+
65+ + Add `getTimestamp` [API change]. This attempts to read the
66+ `SOURCE_DATE_EPOCH` environment variable and parse a UTC time
67+ from it (treating it as a unix date stamp, see
68+ https://reproducible-builds.org/specs/source-date-epoch/). If the
69+ variable is not set or can't be parsed as a unix date stamp, then the
70+ function returns the current date.
71+
72+ * Text.Pandoc.Error
73+
74+ + Remove unused variables (Albert Krewinkel)
75+ + Export `renderError` [API change].
76+ + Refactor `handleError` to use `renderError`. This allows us render
77+ error messages without exiting.
78+
79+ * Text.Pandoc.Extensions
80+
81+ + `Ext_task_lists` is now supported by org (and turned
82+ on by default) (Albert Krewinkel, #6336).
83+ + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes
84+ (#7097). This attribute was listed as allowed, but it didn't actually
85+ do anything. Use `attributes` for code attributes and more.
86+
87+ * Lua subsystem:
88+
89+ + Always load built-in Lua scripts from default data-dir (Albert
90+ Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always
91+ loaded from the system's default data directory. Loading from a
92+ different directory by overriding the default path, e.g. via
93+ `--data-dir`, is no longer supported to avoid unexpected behavior
94+ and to address security concerns.
95+ + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565).
96+ The module allows to work with file paths in a convenient and
97+ platform-independent manner.
98+
99+ * Text.Pandoc.PDF
100+
101+ + Disable `smart` extension when building PDF via LaTeX.
102+ This is to prevent accidental creation of ligatures like
103+ `` ?` `` and `` !` `` (especially in languages with quotations like
104+ German), and similar ligature issues. (See jgm/citeproc#54.)
105+
106+ * DocBook reader:
107+
108+ + Avoid expensive tree normalization step, as it is not necessary
109+ with the new XML parser.
110+ + Support `informalfigure` (#7079) (Nils Carlson).
111+
112+ * Docx reader:
113+
114+ + Use Map instead of list for Namespaces. This gives a speedup of
115+ about 5-10%. With this and the XML parsing changes, the docx reader
116+ is now about twice as fast as in the previous release.
117+
118+ * HTML reader:
119+
120+ + Small performance tweaks.
121+ + Also, remove exported class `NamedTag(..)` [API change]. This was just
122+ intended to smooth over the transition from String to Text and is no
123+ longer needed.
124+ + As a result, the functions `isInlineTag` and `isBlockTag`
125+ are no longer polymorphic; they apply to a `Tag Text` [API change].
126+ + Do a lookahead to find the right parser to use. This takes
127+ benchmarks from 34ms to 23ms, with less allocation.
128+ + Fix bad handling of empty `src` attribute in `iframe` (#7099).
129+ If `src` is empty, we simply skip the `iframe`.
130+ If `src` is invalid or cannot be fetched, we issue a warning
131+ nd skip instead of failing with an error.
132+
133+ * JATS reader:
134+
135+ + Avoid tree normalization, which is no longer necessary given the
136+ new XML parser.
137+
138+ * LaTeX reader:
139+
140+ + Code cleanup, removing some unnecessary things.
141+ + Rewrite `withRaw` so it doesn't rely on fragile assumptions
142+ about token positions (which break when macros are expanded)
143+ (#7092). This requires the addition of `sEnableWithRaw` and
144+ `sRawTokens` in `LaTeXState`, and a new combinator `disablingWithRaw`
145+ to disable collecting of raw tokens in certain contexts.
146+ Add `parseFromToks` to Text.Pandoc.Readers.LaTeX.Parsing.
147+ Fix parsing of single character tokens so it doesn't mess
148+ up the new raw token collecting. These changes slightly increase
149+ allocations and have a small performance impact.
150+ + Handle some bibtex/biblatex-specific commands that used to be
151+ dealt with in pandoc-citeproc (#7049).
152+ + Optimize `satisfyTok`, avoiding unnecessary macro expansion steps.
153+ Benchmarks after this change show 2/3 of the run time and 2/3 of the
154+ allocation of the Feb. 10 benchmarks.
155+ + Removed `sExpanded` in state. This isn't actually needed and checking
156+ it doesn't change anything.
157+ + Improve `braced'`. Remove the parameter, have it parse the
158+ opening brace, and make it more efficient.
159+
160+ * Markdown reader:
161+
162+ + Improved handling of mmd link attributes in references (#7080).
163+ Previously they only worked for links that had titles.
164+
165+ * OPML reader:
166+
167+ + Avoid tree normalization, which is no longer necessary with the
168+ new XML parser.
169+
170+ * ODT reader:
171+
172+ + Finer-grained errors on parse failure (#7091).
173+ + Give more information if the zip container can't be unpacked.
174+
175+ * Org reader:
176+
177+ + Support `task_lists` extension (Albert Krewinkel, #6336).
178+ + Fix bug in org-ref citation parsing (Albert Krewinkel, #7101).
179+ The org-ref syntax allows to list multiple citations separated by
180+ comma. Previously commas were accepted as part of the citation id,
181+ so all citation lists were parsed as one single citation.
182+
183+ * RST reader:
184+
185+ + Use `getTimestamp` instead of `getCurrentTime` to fetch timestamp.
186+ Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
187+ + RST reader: fix handling of header in CSV tables (#7064).
188+ The interpretation of this line is not affected by the delim option.
189+
190+ * Jira reader:
191+
192+ + Modified the Doc parser to skip leading blank lines. This fixes
193+ parsing of documents which start with multiple blank lines (#7095).
194+ + Prevent URLs within link aliases to be treated as autolinks
195+ (#6944).
196+
197+ * Text.Pandoc.Shared
198+
199+ + Remove formerly exported functions that are no longer used in the
200+ code base: `splitByIndices`, `splitStringByIndicies`, `substitute`,
201+ and `underlineSpan` (which had been deprecated in April 2020)
202+ [API change].
203+ + Export `handleTaskListItem` (Albert Krewinkel) [API change].
204+
205+ * BibTeX writer:
206+
207+ + BibTeX writer: use doclayout and doctemplate. This change allows
208+ bibtex/biblatex output to wrap as other formats do,
209+ depending on the settings of `--wrap` and `--columns` (#7068).
210+
211+ * CSL JSON writer:
212+
213+ + Output `[]` if no references in input, instead of raising a
214+ PandocAppError as before.
215+
216+ * Docx writer:
217+
218+ + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
219+ Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
220+
221+ * Text.Pandoc.Writers.EPUB
222+
223+ + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
224+ Setting `SOURCE_DATE_EPOCH` will allow reproducible builds (#7093).
225+ This does not suffice to fully enable reproducible in EPUB, since
226+ a unique id is still being generated for each build.
227+ + Support `belongs-to-collection` metadata (#7063) (Nick Berendsen).
228+
229+ * JATS writer:
230+
231+ + Escape special chars in reference elements (Albert Krewinkel).
232+ Prevents the generation of invalid markup if a citation element
233+ contains an ampersand or another character with a special meaning
234+ in XML.
235+
236+ * LaTeX writer:
237+
238+ + Adjust hypertargets to beginnings of paragraphs (#7078).
239+ Use `\vadjust pre` so that the hypertarget takes you to the beginning
240+ of the paragraph rather than one line down.
241+ This makes a particular difference for links to citations using
242+ `--citeproc` and `link-citations: true`.
243+ + Change BCP47 lang tag from `jp` to `ja` (Mauro Bieg, #7047).
244+
245+ * Markdown writer:
246+
247+ + Handle math right before digit. We insert an HTML comment to
248+ avoid a `$` right before a digit, which pandoc will not recognize
249+ as a math delimiter.
250+
251+ * ODT writer:
252+
253+ + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
254+ Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
255+ + Update default ODT style (Lorenzo). Previously, the "First paragraph"
256+ style inherited from "Standard" but not from "Text body." Now
257+ it is adjusted to inherit from "Text body", to avoid some ugly
258+ spacing issues. It may be necessary to update a custom `reference.odt`
259+ in light of this change.
260+
261+ * Org writer:
262+
263+ + Support `task_lists` extension (Albert Krewinkel, #6336).
264+
265+ * Pptx writer:
266+
267+ + Use `getTimestamp` instead of `getCurrentTime` for timestamp.
268+ Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
269+
270+ * JATS templates: tag `author.name` as `string-name` (Albert Krewinkel).
271+ The partitioning the components of a name into surname, given names,
272+ etc. is not always possible or not available. Using `author.name`
273+ allows to give the full name as a fallback to be used when
274+ `author.surname` is not available.
275+
276+ * Add default templates for bibtex and biblatex, so that
277+ the variables `header-include`, `include-before`, `include-after`
278+ (or alternatively the command line options
279+ `--include-in-header`, `--include-before-body`, `--include-after-body`)
280+ may be used.
281+
282+ * LaTeX template: Update to iftex package (#7073) (Andrew Dunning)
283+
284+ * revealjs template: Add 'center' option for vertical slide centering.
285+ (maurerle, #7104).
286+
287+ * Text.Pandoc.XML: Improve efficiency of `fromEntities`.
288+
289+ * Test suite: a more robust way of testing the executable.
290+ Many of our tests require running the pandoc executable. This is
291+ problematic for a few different reasons. First, cabal-install will
292+ sometimes run the test suite after building the library but before
293+ building the executable, which means the executable isn't in place for
294+ the tests. One can work around that by first building, then building and
295+ running the tests, but that's fragile. Second, we have to find the
296+ executable. So far, we've done that using a function `findPandoc` that
297+ attempts to locate it relative to the test executable (which can be
298+ located using findExecutablePath). But the logic here is delicate and
299+ work with every combination of options. To solve both problems, we add
300+ an `--emulate` option to the `test-pandoc` executable. When `--emulate`
301+ occurs as the first argument passed to `test-pandoc`, the program simply
302+ emulates the regular pandoc executable, using the rest of the arguments
303+ (after `--emulate`). Thus, `test-pandoc --emulate -f markdown -t latex`
304+ is just like `pandoc -f markdown -t latex`.
305+ Since all the work is done by library functions, implementing this
306+ emulation just takes a couple lines of code and should be entirely
307+ reliable. With this change, we can test the pandoc executable by running
308+ the test program itself (locatable using `findExecutablePath`) with the
309+ `--emulate` option. This removes the need for the fragile `findPandoc`
310+ step, and it means we can run our integration tests even when we're just
311+ building the library, not the executable. [Note: part of this change
312+ involved simplifying some complex handling to set environment variables
313+ for dynamic library paths. I have tested a build with
314+ `--enable-dynamic-executable`, and it works, but further testing may be
315+ needed.]
316+
317+ * MANUAL.txt
318+
319+ + MANUAL: block-level formatting is not allowed in line blocks (#7107).
320+ + Clarify `tex_math_dollars` extension. Note that no blank lines
321+ are allowed between the delimiters in display math.
322+ + Add MANUAL section on reproducible builds.
323+ + Document no template fallback for absolute path (#7077, Nixon
324+ Enraght-Moony.)
325+ + Improve docs for cite-method.
326+ + Update README and man page.
327+
328+ * Makefile: in `make bench`, create CSV files for comparison and compare
329+ against previous benchmark run. Add timestamp to CSV filenames.
330+
331+ * doc/lua-filters.md: improve documentation for
332+ `pandoc.mediabag.insert`, `pandoc.mediabag.fetch`,
333+ `directory`, `normalize` (Albert Krewinkel).
334+
335+ * Allow base64-bytestring-1.2.* (Dmitrii Kovanikov)
336+
337+ * Require jira-wiki-markup 1.3.3 (Albert Krewinkel)
338+
339+ * Require citeproc 0.3.0.7, which correctly titlecases when titles
340+ contain non-ASCII characters.
341+
342+ * Avoid unnecessary use of NoImplicitPrelude pragma (#7089) (Albert
343+ Krewinkel)
344+
345+ * Benchmarks
346+
347+ + Use the lighter-weight tasty-bench instead of criterion.
348+ + Run writer benchmarks for binary formats too.
349+ + Alphabetize benchmarks.
350+ + Don't run benchmarks for bibliography formats
351+ (yet; we need a special input for them).
352+ + Show allocation data
353+ + Clean up benchmark code.
354+ + Allow specifying patterns using `-p blah'.
355+
356+
357+
3358## pandoc 2.11.4 (2021-01-22)
4359
5360 * Add `biblatex`, `bibtex` as output formats (closes #7040).