git.iko.soy/typeable/pandoc
on master help sign in
Markdown reader: revised definition list syntax (closes #1429).

* This change brings pandoc's definition list syntax into alignment with that used in PHP markdown extra and multimarkdown (with the exception that pandoc is more flexible about the definition markers, allowing tildes as well as colons). * Lazily wrapped definitions are now allowed; blank space is required between list items; and the space before definition is used to determine whether it is a paragraph or a "plain" element. * For backwards compatibility, a new extension, `compact_definition_lists`, has been added that restores the behavior of pandoc 1.12.x, allowing tight definition lists with no blank space between items, and disallowing lazy wrapping.

commit4af8eed764e4b5f84377cf946b03203d01078102 authorJohn MacFarlane <jgm@berkeley.edu> dateSun 20 Jul 2014, 16:33 (12 years ago) parentcdc4ecb
5 files changed+148−36
compare with master
unifiedsplit
README+55−13 src/Text/Pandoc/Options.hs+2 src/Text/Pandoc/Readers/Markdown.hs+38−23 tests/Tests/Readers/Markdown.hs+41 tests/testsuite.txt+12
README+55−13
@@ -1437,8 +1437,8 @@ If default list markers are desired, use `#.`:
14371437
14381438**Extension: `definition_lists`**
14391439
1440−Pandoc supports definition lists, using a syntax inspired by
1441−[PHP Markdown Extra] and [reStructuredText]:[^3]
1440+Pandoc supports definition lists, using the syntax of
1441+[PHP Markdown Extra] with some extensions.[^3]
14421442
14431443 Term 1
14441444
@@ -1455,25 +1455,41 @@ Pandoc supports definition lists, using a syntax inspired by
14551455Each term must fit on one line, which may optionally be followed by
14561456a blank line, and must be followed by one or more definitions.
14571457A definition begins with a colon or tilde, which may be indented one
1458−or two spaces. The body of the definition (including the first line,
1459−aside from the colon or tilde) should be indented four spaces. A term may have
1460−multiple definitions, and each definition may consist of one or more block
1461−elements (paragraph, code block, list, etc.), each indented four spaces or one
1462−tab stop.
1463−
1464−If you leave space after the definition (as in the example above),
1465−the blocks of the definitions will be considered paragraphs. In some
1458+or two spaces.
1459+
1460+A term may have multiple definitions, and each definition may consist of one or
1461+more block elements (paragraph, code block, list, etc.), each indented four
1462+spaces or one tab stop. The body of the definition (including the first line,
1463+aside from the colon or tilde) should be indented four spaces. However,
1464+as with other markdown lists, you can "lazily" omit indentation except
1465+at the beginning of a paragraph or other block element:
1466+
1467+ Term 1
1468+
1469+ : Definition
1470+ with lazy continuation.
1471+
1472+ Second paragraph of the definition.
1473+
1474+If you leave space before the definition (as in the example above),
1475+the text of the definition will be treated as a paragraph. In some
14661476output formats, this will mean greater spacing between term/definition
1467−pairs. For a compact definition list, do not leave space between the
1468−definition and the next term:
1477+pairs. For a more compact definition list, omit the space before the
1478+definition:
14691479
14701480 Term 1
14711481 ~ Definition 1
1482+
14721483 Term 2
14731484 ~ Definition 2a
14741485 ~ Definition 2b
14751486
1476−[^3]: I have also been influenced by the suggestions of [David Wheeler](http://www.justatheory.com/computers/markup/modest-markdown-proposal.html).
1487+Note that space between items in a definition list is required.
1488+(A variant that loosens this requirement, but disallows "lazy"
1489+hard wrapping, can be activated with `compact_definition_lists`: see
1490+[Non-pandoc extensions](#non-pandoc-extensions), below.)
1491+
1492+[^3]: I have been influenced by the suggestions of [David Wheeler](http://www.justatheory.com/computers/markup/modest-markdown-proposal.html).
14771493
14781494[PHP Markdown Extra]: http://www.michelf.com/projects/php-markdown/extra/
14791495
@@ -2629,6 +2645,32 @@ these, so they are presently just ignored.
26292645Parses multimarkdown style header identifiers (in square brackets,
26302646after the header but before any trailing `#`s in an ATX header).
26312647
2648+**Extension: `compact_definition_lists`**\
2649+Activates the definition list syntax of pandoc 1.12.x and earlier.
2650+This syntax differs from the one described [above](#definition-lists)
2651+in several respects:
2652+
2653+ - No blank line is required between consecutive items of the
2654+ definition list.
2655+ - To get a "tight" or "compact" list, omit space between consecutive
2656+ items; the space between a term and its definition does not affect
2657+ anything.
2658+ - Lazy wrapping of paragraphs is not allowed: the entire definition must
2659+ be indented four spaces.[^6]
2660+
2661+[^6]: To see why laziness is incompatible with relaxing the requirement
2662+ of a blank line between items, consider the following example:
2663+
2664+ bar
2665+ : definition
2666+ foo
2667+ : definition
2668+
2669+ Is this a single list item with two definitions of "bar," the first of
2670+ which is lazily wrapped, or two list items? To remove the ambiguity
2671+ we must either disallow lazy wrapping or require a blank line between
2672+ list items.
2673+
26322674Markdown variants
26332675-----------------
26342676
src/Text/Pandoc/Options.hs+2
@@ -84,6 +84,8 @@ data Extension =
8484 | Ext_lists_without_preceding_blankline -- ^ Allow lists without preceding blank
8585 | Ext_startnum -- ^ Make start number of ordered list significant
8686 | Ext_definition_lists -- ^ Definition lists as in pandoc, mmd, php
87+ | Ext_compact_definition_lists -- ^ Definition lists without
88+ -- space between items, and disallow laziness
8789 | Ext_example_lists -- ^ Markdown-style numbered examples
8890 | Ext_all_symbols_escapable -- ^ Make all non-alphanumerics escapable
8991 | Ext_intraword_underscores -- ^ Treat underscore inside word as literal
src/Text/Pandoc/Readers/Markdown.hs+38−23
@@ -846,38 +846,53 @@ defListMarker = do
846846 else mzero
847847 return ()
848848
849−definitionListItem :: MarkdownParser (F (Inlines, [Blocks]))
850−definitionListItem = try $ do
851− -- first, see if this has any chance of being a definition list:
852− lookAhead (anyLine >> optional blankline >> defListMarker)
853− term <- trimInlinesF . mconcat <$> manyTill inline newline
854− optional blankline
855− raw <- many1 defRawBlock
856− state <- getState
857− let oldContext = stateParserContext state
858− -- parse the extracted block, which may contain various block elements:
849+definitionListItem :: Bool -> MarkdownParser (F (Inlines, [Blocks]))
850+definitionListItem compact = try $ do
851+ rawLine' <- anyLine
852+ raw <- many1 $ defRawBlock compact
853+ term <- parseFromString (trimInlinesF . mconcat <$> many inline) rawLine'
859854 contents <- mapM (parseFromString parseBlocks) raw
860− updateState (\st -> st {stateParserContext = oldContext})
855+ optional blanklines
861856 return $ liftM2 (,) term (sequence contents)
862857
863−defRawBlock :: MarkdownParser String
864−defRawBlock = try $ do
858+defRawBlock :: Bool -> MarkdownParser String
859+defRawBlock compact = try $ do
860+ hasBlank <- option False $ blankline >> return True
865861 defListMarker
866862 firstline <- anyLine
867− rawlines <- many (notFollowedBy blankline >> indentSpaces >> anyLine)
868− trailing <- option "" blanklines
869− cont <- liftM concat $ many $ do
870− lns <- many1 $ notFollowedBy blankline >> indentSpaces >> anyLine
871− trl <- option "" blanklines
872− return $ unlines lns ++ trl
873− return $ firstline ++ "\n" ++ unlines rawlines ++ trailing ++ cont
863+ let dline = try
864+ ( do notFollowedBy blankline
865+ if compact -- laziness not compatible with compact
866+ then () <$ indentSpaces
867+ else (() <$ indentSpaces)
868+ <|> notFollowedBy defListMarker
869+ anyLine )
870+ rawlines <- many dline
871+ cont <- liftM concat $ many $ try $ do
872+ trailing <- option "" blanklines
873+ ln <- indentSpaces >> notFollowedBy blankline >> anyLine
874+ lns <- many dline
875+ return $ trailing ++ unlines (ln:lns)
876+ return $ trimr (firstline ++ "\n" ++ unlines rawlines ++ cont) ++
877+ if hasBlank || not (null cont) then "\n\n" else ""
874878
875879definitionList :: MarkdownParser (F Blocks)
876−definitionList = do
877− guardEnabled Ext_definition_lists
878− items <- fmap sequence $ many1 definitionListItem
880+definitionList = try $ do
881+ lookAhead (anyLine >> optional blankline >> defListMarker)
882+ compactDefinitionList <|> normalDefinitionList
883+
884+compactDefinitionList :: MarkdownParser (F Blocks)
885+compactDefinitionList = do
886+ guardEnabled Ext_compact_definition_lists
887+ items <- fmap sequence $ many1 $ definitionListItem True
879888 return $ B.definitionList <$> fmap compactify'DL items
880889
890+normalDefinitionList :: MarkdownParser (F Blocks)
891+normalDefinitionList = do
892+ guardEnabled Ext_definition_lists
893+ items <- fmap sequence $ many1 $ definitionListItem False
894+ return $ B.definitionList <$> items
895+
881896--
882897-- paragraph block
883898--
tests/Tests/Readers/Markdown.hs+41
@@ -16,6 +16,10 @@ markdown = readMarkdown def
1616markdownSmart :: String -> Pandoc
1717markdownSmart = readMarkdown def { readerSmart = True }
1818
19+markdownCDL :: String -> Pandoc
20+markdownCDL = readMarkdown def { readerExtensions = Set.insert
21+ Ext_compact_definition_lists $ readerExtensions def }
22+
1923infix 4 =:
2024(=:) :: ToString c
2125 => String -> (String, c) -> Test
@@ -222,6 +226,43 @@ tests = [ testGroup "inline code"
222226-- , testGroup "round trip"
223227-- [ property "p_markdown_round_trip" p_markdown_round_trip
224228-- ]
229+ , testGroup "definition lists"
230+ [ "no blank space" =:
231+ "foo1\n : bar\n\nfoo2\n : bar2\n : bar3\n" =?>
232+ definitionList [ (text "foo1", [plain (text "bar")])
233+ , (text "foo2", [plain (text "bar2"),
234+ plain (text "bar3")])
235+ ]
236+ , "blank space before first def" =:
237+ "foo1\n\n : bar\n\nfoo2\n\n : bar2\n : bar3\n" =?>
238+ definitionList [ (text "foo1", [para (text "bar")])
239+ , (text "foo2", [para (text "bar2"),
240+ plain (text "bar3")])
241+ ]
242+ , "blank space before second def" =:
243+ "foo1\n : bar\n\nfoo2\n : bar2\n\n : bar3\n" =?>
244+ definitionList [ (text "foo1", [plain (text "bar")])
245+ , (text "foo2", [plain (text "bar2"),
246+ para (text "bar3")])
247+ ]
248+ , "laziness" =:
249+ "foo1\n : bar\nbaz\n : bar2\n" =?>
250+ definitionList [ (text "foo1", [plain (text "bar baz"),
251+ plain (text "bar2")])
252+ ]
253+ , "no blank space before first of two paragraphs" =:
254+ "foo1\n : bar\n\n baz\n" =?>
255+ definitionList [ (text "foo1", [para (text "bar") <>
256+ para (text "baz")])
257+ ]
258+ ]
259+ , testGroup "+compact_definition_lists"
260+ [ test markdownCDL "basic compact list" $
261+ "foo1\n: bar\n baz\nfoo2\n: bar2\n" =?>
262+ definitionList [ (text "foo1", [plain (text "bar baz")])
263+ , (text "foo2", [plain (text "bar2")])
264+ ]
265+ ]
225266 , testGroup "lists"
226267 [ "issue #1154" =:
227268 " - <div>\n first div breaks\n </div>\n\n <button>if this button exists</button>\n\n <div>\n with this div too.\n </div>\n"
tests/testsuite.txt+12
@@ -270,8 +270,10 @@ Tight using spaces:
270270
271271apple
272272: red fruit
273+
273274orange
274275: orange fruit
276+
275277banana
276278: yellow fruit
277279
@@ -279,31 +281,38 @@ Tight using tabs:
279281
280282apple
281283: red fruit
284+
282285orange
283286: orange fruit
287+
284288banana
285289: yellow fruit
286290
287291Loose:
288292
289293apple
294+
290295: red fruit
291296
292297orange
298+
293299: orange fruit
294300
295301banana
302+
296303: yellow fruit
297304
298305Multiple blocks with italics:
299306
300307*apple*
308+
301309: red fruit
302310
303311 contains seeds,
304312 crisp, pleasant to taste
305313
306314*orange*
315+
307316: orange fruit
308317
309318 { orange code block }
@@ -315,6 +324,7 @@ Multiple definitions, tight:
315324apple
316325: red fruit
317326: computer
327+
318328orange
319329: orange fruit
320330: bank
@@ -322,11 +332,13 @@ orange
322332Multiple definitions, loose:
323333
324334apple
335+
325336: red fruit
326337
327338: computer
328339
329340orange
341+
330342: orange fruit
331343
332344: bank