4af8eed · typeable/pandoc

git.iko.soy/typeable/pandoc

on master help sign in

Markdown reader: revised definition list syntax (closes #1429).

4af8eedJohn MacFarlane12 years ago

README+55−13

compare with master

unifiedsplit

@@ -1437,8 +1437,8 @@ If default list markers are desired, use `#.`:

1437

1438**Extension: `definition_lists`**

1439

1440−Pandoc supports definition lists, using a syntax inspired by

1440+Pandoc supports definition lists, using the syntax of

1441−[PHP Markdown Extra] and [reStructuredText]:[^3]

1441+[PHP Markdown Extra] with some extensions.[^3]

1442

1443 Term 1

1444

@@ -1455,25 +1455,41 @@ Pandoc supports definition lists, using a syntax inspired by

1455Each term must fit on one line, which may optionally be followed by

1456a blank line, and must be followed by one or more definitions.

1457A definition begins with a colon or tilde, which may be indented one

1458−or two spaces. The body of the definition (including the first line,

1458+or two spaces.

1459−aside from the colon or tilde) should be indented four spaces. A term may have

1459+

1460−multiple definitions, and each definition may consist of one or more block

1460+A term may have multiple definitions, and each definition may consist of one or

1461−elements (paragraph, code block, list, etc.), each indented four spaces or one

1461+more block elements (paragraph, code block, list, etc.), each indented four

1462−tab stop.

1462+spaces or one tab stop. The body of the definition (including the first line,

1463−

1463+aside from the colon or tilde) should be indented four spaces. However,

1464+as with other markdown lists, you can "lazily" omit indentation except

1465+at the beginning of a paragraph or other block element:

1466+

1467+ Term 1

1468+

1469+ : Definition

1470+ with lazy continuation.

1471+

1472+ Second paragraph of the definition.

1473+

1464−If you leave space after the definition (as in the example above),

1474+If you leave space before the definition (as in the example above),

1465−the blocks of the definitions will be considered paragraphs. In some

1475+the text of the definition will be treated as a paragraph. In some

1466output formats, this will mean greater spacing between term/definition

1476output formats, this will mean greater spacing between term/definition

1467−pairs. For a compact definition list, do not leave space between the

1477+pairs. For a more compact definition list, omit the space before the

1468−definition and the next term:

1478+definition:

1469

1479

1470 Term 1

1480 Term 1

1471 ~ Definition 1

1481 ~ Definition 1

1482+

1472 Term 2

1483 Term 2

1473 ~ Definition 2a

1484 ~ Definition 2a

1474 ~ Definition 2b

1485 ~ Definition 2b

1475

1486

1487+Note that space between items in a definition list is required.

1488+(A variant that loosens this requirement, but disallows "lazy"

1489+hard wrapping, can be activated with `compact_definition_lists`: see

1490+[Non-pandoc extensions](#non-pandoc-extensions), below.)

1491+

1476−[^3]: I have also been influenced by the suggestions of [David Wheeler](http://www.justatheory.com/computers/markup/modest-markdown-proposal.html).

1492+[^3]: I have been influenced by the suggestions of [David Wheeler](http://www.justatheory.com/computers/markup/modest-markdown-proposal.html).

1477

1493

1478[PHP Markdown Extra]: http://www.michelf.com/projects/php-markdown/extra/

1494[PHP Markdown Extra]: http://www.michelf.com/projects/php-markdown/extra/

1479

1495

@@ -2629,6 +2645,32 @@ these, so they are presently just ignored.

2629Parses multimarkdown style header identifiers (in square brackets,

2645Parses multimarkdown style header identifiers (in square brackets,

2630after the header but before any trailing `#`s in an ATX header).

2646after the header but before any trailing `#`s in an ATX header).

2631

2647

2648+**Extension: `compact_definition_lists`**\

2649+Activates the definition list syntax of pandoc 1.12.x and earlier.

2650+This syntax differs from the one described [above](#definition-lists)

2651+in several respects:

2652+

2653+ - No blank line is required between consecutive items of the

2654+ definition list.

2655+ - To get a "tight" or "compact" list, omit space between consecutive

2656+ items; the space between a term and its definition does not affect

2657+ anything.

2658+ - Lazy wrapping of paragraphs is not allowed: the entire definition must

2659+ be indented four spaces.[^6]

2660+

2661+[^6]: To see why laziness is incompatible with relaxing the requirement

2662+ of a blank line between items, consider the following example:

2663+

2664+ bar

2665+ : definition

2666+ foo

2667+ : definition

2668+

2669+ Is this a single list item with two definitions of "bar," the first of

2670+ which is lazily wrapped, or two list items? To remove the ambiguity

2671+ we must either disallow lazy wrapping or require a blank line between

2672+ list items.

2673+

2632Markdown variants

2674Markdown variants

2633-----------------

2675-----------------

2634

2676

src/Text/Pandoc/Options.hs+2

@@ -84,6 +84,8 @@ data Extension =

84 | Ext_lists_without_preceding_blankline -- ^ Allow lists without preceding blank

85 | Ext_startnum -- ^ Make start number of ordered list significant

86 | Ext_definition_lists -- ^ Definition lists as in pandoc, mmd, php

87+ | Ext_compact_definition_lists -- ^ Definition lists without

88+ -- space between items, and disallow laziness

87 | Ext_example_lists -- ^ Markdown-style numbered examples

89 | Ext_example_lists -- ^ Markdown-style numbered examples

88 | Ext_all_symbols_escapable -- ^ Make all non-alphanumerics escapable

90 | Ext_all_symbols_escapable -- ^ Make all non-alphanumerics escapable

89 | Ext_intraword_underscores -- ^ Treat underscore inside word as literal

91 | Ext_intraword_underscores -- ^ Treat underscore inside word as literal

src/Text/Pandoc/Readers/Markdown.hs+38−23

@@ -846,38 +846,53 @@ defListMarker = do

846 else mzero

847 return ()

848

849−definitionListItem :: MarkdownParser (F (Inlines, [Blocks]))

849+definitionListItem :: Bool -> MarkdownParser (F (Inlines, [Blocks]))

850−definitionListItem = try $ do

850+definitionListItem compact = try $ do

851− -- first, see if this has any chance of being a definition list:

851+ rawLine' <- anyLine

852− lookAhead (anyLine >> optional blankline >> defListMarker)

852+ raw <- many1 $ defRawBlock compact

853− term <- trimInlinesF . mconcat <$> manyTill inline newline

853+ term <- parseFromString (trimInlinesF . mconcat <$> many inline) rawLine'

854− optional blankline

855− raw <- many1 defRawBlock

856− state <- getState

857− let oldContext = stateParserContext state

858− -- parse the extracted block, which may contain various block elements:

859 contents <- mapM (parseFromString parseBlocks) raw

854 contents <- mapM (parseFromString parseBlocks) raw

860− updateState (\st -> st {stateParserContext = oldContext})

855+ optional blanklines

861 return $ liftM2 (,) term (sequence contents)

856 return $ liftM2 (,) term (sequence contents)

862

857

863−defRawBlock :: MarkdownParser String

858+defRawBlock :: Bool -> MarkdownParser String

864−defRawBlock = try $ do

859+defRawBlock compact = try $ do

860+ hasBlank <- option False $ blankline >> return True

865 defListMarker

861 defListMarker

866 firstline <- anyLine

862 firstline <- anyLine

863+ let dline = try

867− rawlines <- many (notFollowedBy blankline >> indentSpaces >> anyLine)

864+ ( do notFollowedBy blankline

868− trailing <- option "" blanklines

865+ if compact -- laziness not compatible with compact

866+ then () <$ indentSpaces

867+ else (() <$ indentSpaces)

868+ <|> notFollowedBy defListMarker

869+ anyLine )

870+ rawlines <- many dline

869− cont <- liftM concat $ many $ do

871+ cont <- liftM concat $ many $ try $ do

870− lns <- many1 $ notFollowedBy blankline >> indentSpaces >> anyLine

872+ trailing <- option "" blanklines

871− trl <- option "" blanklines

873+ ln <- indentSpaces >> notFollowedBy blankline >> anyLine

874+ lns <- many dline

872− return $ unlines lns ++ trl

875+ return $ trailing ++ unlines (ln:lns)

873− return $ firstline ++ "\n" ++ unlines rawlines ++ trailing ++ cont

876+ return $ trimr (firstline ++ "\n" ++ unlines rawlines ++ cont) ++

877+ if hasBlank || not (null cont) then "\n\n" else ""

874

878

875definitionList :: MarkdownParser (F Blocks)

879definitionList :: MarkdownParser (F Blocks)

876−definitionList = do

880+definitionList = try $ do

877− guardEnabled Ext_definition_lists

881+ lookAhead (anyLine >> optional blankline >> defListMarker)

882+ compactDefinitionList <|> normalDefinitionList

883+

884+compactDefinitionList :: MarkdownParser (F Blocks)

885+compactDefinitionList = do

886+ guardEnabled Ext_compact_definition_lists

878− items <- fmap sequence $ many1 definitionListItem

887+ items <- fmap sequence $ many1 $ definitionListItem True

879 return $ B.definitionList <$> fmap compactify'DL items

888 return $ B.definitionList <$> fmap compactify'DL items

880

889

890+normalDefinitionList :: MarkdownParser (F Blocks)

891+normalDefinitionList = do

892+ guardEnabled Ext_definition_lists

893+ items <- fmap sequence $ many1 $ definitionListItem False

894+ return $ B.definitionList <$> items

895+

881--

896--

882-- paragraph block

897-- paragraph block

883--

898--

tests/Tests/Readers/Markdown.hs+41

@@ -16,6 +16,10 @@ markdown = readMarkdown def

16markdownSmart :: String -> Pandoc

17markdownSmart = readMarkdown def { readerSmart = True }

19+markdownCDL :: String -> Pandoc

20+markdownCDL = readMarkdown def { readerExtensions = Set.insert

21+ Ext_compact_definition_lists $ readerExtensions def }

22+

19infix 4 =:

23infix 4 =:

20(=:) :: ToString c

24(=:) :: ToString c

21 => String -> (String, c) -> Test

25 => String -> (String, c) -> Test

@@ -222,6 +226,43 @@ tests = [ testGroup "inline code"

222-- , testGroup "round trip"

226-- , testGroup "round trip"

223-- [ property "p_markdown_round_trip" p_markdown_round_trip

227-- [ property "p_markdown_round_trip" p_markdown_round_trip

224-- ]

228-- ]

229+ , testGroup "definition lists"

230+ [ "no blank space" =:

231+ "foo1\n : bar\n\nfoo2\n : bar2\n : bar3\n" =?>

232+ definitionList [ (text "foo1", [plain (text "bar")])

233+ , (text "foo2", [plain (text "bar2"),

234+ plain (text "bar3")])

235+ ]

236+ , "blank space before first def" =:

237+ "foo1\n\n : bar\n\nfoo2\n\n : bar2\n : bar3\n" =?>

238+ definitionList [ (text "foo1", [para (text "bar")])

239+ , (text "foo2", [para (text "bar2"),

240+ plain (text "bar3")])

241+ ]

242+ , "blank space before second def" =:

243+ "foo1\n : bar\n\nfoo2\n : bar2\n\n : bar3\n" =?>

244+ definitionList [ (text "foo1", [plain (text "bar")])

245+ , (text "foo2", [plain (text "bar2"),

246+ para (text "bar3")])

247+ ]

248+ , "laziness" =:

249+ "foo1\n : bar\nbaz\n : bar2\n" =?>

250+ definitionList [ (text "foo1", [plain (text "bar baz"),

251+ plain (text "bar2")])

252+ ]

253+ , "no blank space before first of two paragraphs" =:

254+ "foo1\n : bar\n\n baz\n" =?>

255+ definitionList [ (text "foo1", [para (text "bar") <>

256+ para (text "baz")])

257+ ]

258+ ]

259+ , testGroup "+compact_definition_lists"

260+ [ test markdownCDL "basic compact list" $

261+ "foo1\n: bar\n baz\nfoo2\n: bar2\n" =?>

262+ definitionList [ (text "foo1", [plain (text "bar baz")])

263+ , (text "foo2", [plain (text "bar2")])

264+ ]

265+ ]

225 , testGroup "lists"

266 , testGroup "lists"

226 [ "issue #1154" =:

267 [ "issue #1154" =:

227 " - <div>\n first div breaks\n </div>\n\n <button>if this button exists</button>\n\n <div>\n with this div too.\n </div>\n"

268 " - <div>\n first div breaks\n </div>\n\n <button>if this button exists</button>\n\n <div>\n with this div too.\n </div>\n"

tests/testsuite.txt+12

@@ -270,8 +270,10 @@ Tight using spaces:

270

271apple

272: red fruit

273+

273orange

274orange

274: orange fruit

275: orange fruit

276+

275banana

277banana

276: yellow fruit

278: yellow fruit

277

279

@@ -279,31 +281,38 @@ Tight using tabs:

279

281

280apple

282apple

281: red fruit

283: red fruit

284+

282orange

285orange

283: orange fruit

286: orange fruit

287+

284banana

288banana

285: yellow fruit

289: yellow fruit

286

290

287Loose:

291Loose:

288

292

289apple

293apple

294+

290: red fruit

295: red fruit

291

296

292orange

297orange

298+

293: orange fruit

299: orange fruit

294

300

295banana

301banana

302+

296: yellow fruit

303: yellow fruit

297

304

298Multiple blocks with italics:

305Multiple blocks with italics:

299

306

300*apple*

307*apple*

308+

301: red fruit

309: red fruit

302

310

303 contains seeds,

311 contains seeds,

304 crisp, pleasant to taste

312 crisp, pleasant to taste

305

313

306*orange*

314*orange*

315+

307: orange fruit

316: orange fruit

308

317

309 { orange code block }

318 { orange code block }

@@ -315,6 +324,7 @@ Multiple definitions, tight:

315apple

324apple

316: red fruit

325: red fruit

317: computer

326: computer

327+

318orange

328orange

319: orange fruit

329: orange fruit

320: bank

330: bank

@@ -322,11 +332,13 @@ orange

322Multiple definitions, loose:

332Multiple definitions, loose:

323

333

324apple

334apple

335+

325: red fruit

336: red fruit

326

337

327: computer

338: computer

328

339

329orange

340orange

341+

330: orange fruit

342: orange fruit

331

343

332: bank

344: bank