master...d463ef0 · typeable/pandoc

git.iko.soy/typeable/pandoc

on d463ef0 help sign in

changelog.md+57−1

since splitsnapshots

unifiedsplit

@@ -31,6 +31,12 @@

31 `--no-check-certificate`, which disables certificate checking when

32 resources are fetched by HTTP.

34+ * Unify defaults and markdown metadata parsers (#6328, Nikolay

35+ Yakimov). Clean up code in Text.Pandoc.Readers.Metadata and

36+ properly handle errors in `yamlToMeta`. This fixes parsing

37+ of Boolean fields in metadata withinin defaults files and reduces

38+ code duplication.

39+

34 * Docbook reader:

40 * Docbook reader:

36 + Implement `<procedure>` (#6442, Mathieu Boespflug).

42 + Implement `<procedure>` (#6442, Mathieu Boespflug).

@@ -39,7 +45,11 @@

39 + Implement `<replaceable>` (#6437, Mathieu Boespflug)

45 + Implement `<replaceable>` (#6437, Mathieu Boespflug)

40 + Map `<simplesect>` to unnumbered section (#6436, Mathieu Boespflug).

46 + Map `<simplesect>` to unnumbered section (#6436, Mathieu Boespflug).

48+ * JATS reader:

49+

42− * JATS reader: handle "label" element in section title (#6288).

50+ + Handle "label" element in section title (#6288).

51+ + Parse abstract element into metadata field of same name

52+ (#6480, Albert Krewinkel).

44 * Jira reader (Albert Krewinkel):

54 * Jira reader (Albert Krewinkel):

@@ -80,6 +90,9 @@

80 + Fix inline code in lists (#6284, Nikolay Yakimov).

90 + Fix inline code in lists (#6284, Nikolay Yakimov).

81 Previously inline code containing list markers was sometimes

91 Previously inline code containing list markers was sometimes

82 parsed incorrectly.

92 parsed incorrectly.

93+ + Don't require blank line after grid table (#6481).

94+ This allows grid tables to be enclosed in fenced divs with no

95+ intervening blank lines.

84 * LaTeX reader:

97 * LaTeX reader:

@@ -93,6 +106,41 @@

93 * Org reader (Albert Krewinkel):

106 * Org reader (Albert Krewinkel):

107

95 + Recognize images with uppercase extensions (#6472).

108 + Recognize images with uppercase extensions (#6472).

109+ + Keep unknown keyword lines as raw org. The lines of unknown

110+ keywords, like `#+SOMEWORD: value` are no longer read as metadata,

111+ but kept as raw `org` blocks. This ensures that more information

112+ is retained when round-tripping org-mode files; additionally,

113+ this change makes it possible to support non-standard org

114+ extensions via filters.

115+ + Unify keyword handling. Handling of export settings and other

116+ keywords (like `#+LINK`) has been combined and unified.

117+ + Support `LATEX_HEADER_EXTRA` and `HTML_HEAD_EXTRA`

118+ settings. These export settings are treated like their non-extra

119+ counterparts, i.e., the values are added to the `header-includes`

120+ metadata list.

121+ + Allow multiple `#+SUBTITLE` export settings. The values of all

122+ lines are read as inlines and collected in the `subtitle`

123+ metadata field.

124+ + Read `#+INSTITUTE` values as text with markup. The value is

125+ stored in the `institute` metadata field and used in the

126+ default beamer presentation template.

127+ + The behavior of the `#+AUTHOR` and `#+KEYWORD` export

128+ settings has changed: Org now allows multiple such lines

129+ and adds a space between the contents of each line. Pandoc

130+ now always parses these settings as meta inlines; setting

131+ values are no longer treated as comma-separated lists.

132+ Note that a Lua filter can be used to restore the previous

133+ behavior.

134+ + Read description lines as inlines (#6485). `#+DESCRIPTION` lines

135+ are now treated as text with markup. If multiple such

136+ lines are given, then all lines are read and separated by soft

137+ linebreaks.

138+ + Honor tex export option (#4070). The `tex` export option can be set

139+ with `#+OPTION: tex:nil` and allows three settings:

140+ `t` (the default) causes LaTeX fragments to be parsed as TeX or added

141+ as raw TeX. `nil` removes all LaTeX fragments from the document.

142+ `verbatim` treats LaTeX as text.

143+

144

97 * RST reader:

145 * RST reader:

146

@@ -244,6 +292,7 @@

244 + Use `--enable-local-file-access` in invoking `wkhtmltopdf` (#6474).

292 + Use `--enable-local-file-access` in invoking `wkhtmltopdf` (#6474).

245 `wkhtmltopdf` changed in recent versions to require this for

293 `wkhtmltopdf` changed in recent versions to require this for

246 access to local files. This fixes PDF via HTML5 with `--css`.

294 access to local files. This fixes PDF via HTML5 with `--css`.

295+ + Send verbose output to stderr, not stdout (#6483).

247

296

248 * Text.Pandoc.MIME: Fix MIME type for TrueType fonts in EPUBs

297 * Text.Pandoc.MIME: Fix MIME type for TrueType fonts in EPUBs

249 (#6464, Michael Reed).

298 (#6464, Michael Reed).

@@ -317,6 +366,8 @@

317

366

318 * Remove unnecessary hlint ignores (#6341, Joseph C. Sible).

367 * Remove unnecessary hlint ignores (#6341, Joseph C. Sible).

319

368

369+ * Remove obsolete RelaxedPolyRec extension (#6487, Nikolay Yakimov).

370+

320 * trypandoc improvements (Mike Tzou):

371 * trypandoc improvements (Mike Tzou):

321

372

322 + Add standalone option to the command text (#6210).

373 + Add standalone option to the command text (#6210).

@@ -335,6 +386,11 @@

335 + Update links to reveal.js documentation (#6386, Salim B).

386 + Update links to reveal.js documentation (#6386, Salim B).

336 + Separate adjacent verbatim code blocks (#6307, tom-audm).

387 + Separate adjacent verbatim code blocks (#6307, tom-audm).

337

388

389+ * org.md:

390+

391+ + Document behavior of `smart` extension (#4387, Albert Krewinkel).

392+ + Describe all supported export options in detail.

393+

338 * lua-filters.md:

394 * lua-filters.md:

339

395

340 + Fix description of BulletList Lua type (Levi Gruspe).

396 + Fix description of BulletList Lua type (Levi Gruspe).

doc/org.md+15−11

@@ -14,13 +14,13 @@ The following export keywords are supported:

15- AUTHOR: comma-separated list of author(s); fully supported.

17−- CREATOR: output generator; passed as metadata entry, but

17+- CREATOR: output generator; passed as plain-text metadata entry

18− ignored by most output formats.

18+ `creator`, but not used by any default templates.

20- DATE: creation or publication date; well supported by pandoc.

22−- EMAIL: author email address; passed as metadata entry, but not

22+- EMAIL: author email address; passed as plain-text metadata

23− included in most output formats.

23+ field `email`, but not used by any default templates.

25- LANGUAGE: currently unsupported; use `#+LANG:` instead.

@@ -51,11 +51,12 @@ occur.

52 Pandoc follows the LaTeX exporter in that it allows markup in

53 the description. In contrast, the Org-mode HTML exporter treats

54− reads the description as plain text.

54+ the description as plain text.

56−- LATEX_HEADER: arbitrary lines to add to the document's preamble.

56+- LATEX\_HEADER and LATEX_HEADER_EXTRA: arbitrary lines to add to

57− Contrary to Org-mode, these lines are not inserted before the

57+ the document's preamble. Contrary to Org-mode, these lines are

58− hyperref settings, but close to the end of the preamble.

58+ not inserted before the hyperref settings, but close to the end

59+ of the preamble.

60 The contents of this option are stored as a list of raw LaTeX

61 The contents of this option are stored as a list of raw LaTeX

61 lines in the `header-includes` metadata field.

62 lines in the `header-includes` metadata field.

@@ -74,10 +75,13 @@ occur.

75- SUBTITLE: the document's subtitle; fully supported.

76- SUBTITLE: the document's subtitle; fully supported.

78+ The content of this option is stored as inlines in the

79+ `subtitle` metadata field.

80+

77−- HTML_HEAD: arbitrary lines to add to the HTML document's head;

81+- HTML\_HEAD and HTML\_HEAD\_EXTRA: arbitrary lines to add to the

78− fully supported.

82+ HTML document's head; fully supported.

80− The contents of this option are stored as a list of raw HTML

84+ The contents of these options are stored as a list of raw HTML

81 lines in the `header-includes` metadata field.

85 lines in the `header-includes` metadata field.

83Pandoc-specific options

87Pandoc-specific options

src/Text/Pandoc/Readers/Markdown.hs+2−5

@@ -67,16 +67,13 @@ yamlToMeta :: PandocMonad m

67 -> m Meta

68yamlToMeta opts bstr = do

69 let parser = do

70− meta <- yamlBsToMeta (asBlocks <$> parseBlocks) bstr

70+ meta <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks) bstr

71 return $ runF meta defaultParserState

72 parsed <- readWithM parser def{ stateOptions = opts } ""

73 case parsed of

74 Right result -> return result

75 Left e -> throwError e

77−asBlocks :: Functor f => f (B.Many Block) -> f MetaValue

78−asBlocks p = MetaBlocks . B.toList <$> p

79−

80--

77--

81-- Constants and data structure definitions

78-- Constants and data structure definitions

82--

79--

@@ -241,7 +238,7 @@ yamlMetaBlock = try $ do

241 -- by including --- and ..., we allow yaml blocks with just comments:

238 -- by including --- and ..., we allow yaml blocks with just comments:

242 let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))

239 let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))

243 optional blanklines

240 optional blanklines

244− newMetaF <- yamlBsToMeta (asBlocks <$> parseBlocks)

241+ newMetaF <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks)

245 $ UTF8.fromTextLazy $ TL.fromStrict rawYaml

242 $ UTF8.fromTextLazy $ TL.fromStrict rawYaml

246 -- Since `<>` is left-biased, existing values are not touched:

243 -- Since `<>` is left-biased, existing values are not touched:

247 updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }

244 updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }

src/Text/Pandoc/Readers/Metadata.hs+23−36

@@ -1,4 +1,3 @@

1−{-# LANGUAGE ScopedTypeVariables #-}

2{-# LANGUAGE OverloadedStrings #-}

1{-# LANGUAGE OverloadedStrings #-}

3{- |

2{- |

4 Module : Text.Pandoc.Readers.Metadata

3 Module : Text.Pandoc.Readers.Metadata

@@ -21,7 +20,6 @@ import Data.Text (Text)

21import qualified Data.Text as T

20import qualified Data.Text as T

22import qualified Data.YAML as YAML

21import qualified Data.YAML as YAML

23import qualified Data.YAML.Event as YE

22import qualified Data.YAML.Event as YE

24−import qualified Text.Pandoc.Builder as B

25import Text.Pandoc.Class.PandocMonad (PandocMonad (..))

23import Text.Pandoc.Class.PandocMonad (PandocMonad (..))

26import Text.Pandoc.Definition

24import Text.Pandoc.Definition

27import Text.Pandoc.Error

25import Text.Pandoc.Error

@@ -33,11 +31,11 @@ yamlBsToMeta :: PandocMonad m

33 => ParserT Text ParserState m (F MetaValue)

31 => ParserT Text ParserState m (F MetaValue)

34 -> BL.ByteString

32 -> BL.ByteString

35 -> ParserT Text ParserState m (F Meta)

33 -> ParserT Text ParserState m (F Meta)

36−yamlBsToMeta pBlocks bstr = do

34+yamlBsToMeta pMetaValue bstr = do

37 pos <- getPosition

35 pos <- getPosition

38 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of

36 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of

39 Right (YAML.Doc (YAML.Mapping _ _ o):_)

37 Right (YAML.Doc (YAML.Mapping _ _ o):_)

40− -> fmap Meta <$> yamlMap pBlocks o

38+ -> fmap Meta <$> yamlMap pMetaValue o

41 Right [] -> return . return $ mempty

39 Right [] -> return . return $ mempty

42 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]

40 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]

43 -> return . return $ mempty

41 -> return . return $ mempty

@@ -57,30 +55,21 @@ nodeToKey (YAML.Scalar _ (YAML.SUnknown _ t)) = return t

57nodeToKey _ = throwError $ PandocParseError

55nodeToKey _ = throwError $ PandocParseError

58 "Non-string key in YAML mapping"

56 "Non-string key in YAML mapping"

60−toMetaValue :: PandocMonad m

58+normalizeMetaValue :: PandocMonad m

61− => ParserT Text ParserState m (F MetaValue)

59+ => ParserT Text ParserState m (F MetaValue)

62− -> Text

60+ -> Text

63− -> ParserT Text ParserState m (F MetaValue)

61+ -> ParserT Text ParserState m (F MetaValue)

64−toMetaValue pBlocks x =

62+normalizeMetaValue pMetaValue x =

65 -- Note: a standard quoted or unquoted YAML value will

63 -- Note: a standard quoted or unquoted YAML value will

66 -- not end in a newline, but a "block" set off with

64 -- not end in a newline, but a "block" set off with

67 -- `|` or `>` will.

65 -- `|` or `>` will.

68 if "\n" `T.isSuffixOf` x

66 if "\n" `T.isSuffixOf` x

69− then parseFromString' pBlocks (x <> "\n")

67+ then parseFromString' pMetaValue (x <> "\n")

70− else parseFromString' pInlines x

68+ else parseFromString' asInlines x

71− where pInlines = do

69+ where asInlines = fmap b2i <$> pMetaValue

72− bs <- pBlocks

73− return $ do

74− bs' <- bs

75− return $

76− case bs' of

77− MetaBlocks bs'' ->

78− case bs'' of

79− [Plain ils] -> MetaInlines ils

70+ b2i (MetaBlocks [Plain ils]) = MetaInlines ils

80− [Para ils] -> MetaInlines ils

71+ b2i (MetaBlocks [Para ils]) = MetaInlines ils

81− xs -> MetaBlocks xs

82− _ -> bs'

72+ b2i bs = bs

83−

85checkBoolean :: Text -> Maybe Bool

74checkBoolean :: Text -> Maybe Bool

86checkBoolean t

75checkBoolean t

@@ -92,32 +81,30 @@ yamlToMetaValue :: PandocMonad m

92 => ParserT Text ParserState m (F MetaValue)

81 => ParserT Text ParserState m (F MetaValue)

93 -> YAML.Node YE.Pos

82 -> YAML.Node YE.Pos

94 -> ParserT Text ParserState m (F MetaValue)

83 -> ParserT Text ParserState m (F MetaValue)

95−yamlToMetaValue pBlocks (YAML.Scalar _ x) =

84+yamlToMetaValue pMetaValue (YAML.Scalar _ x) =

96 case x of

85 case x of

97− YAML.SStr t -> toMetaValue pBlocks t

86+ YAML.SStr t -> normalizeMetaValue pMetaValue t

98 YAML.SBool b -> return $ return $ MetaBool b

87 YAML.SBool b -> return $ return $ MetaBool b

99 YAML.SFloat d -> return $ return $ MetaString $ tshow d

88 YAML.SFloat d -> return $ return $ MetaString $ tshow d

100 YAML.SInt i -> return $ return $ MetaString $ tshow i

89 YAML.SInt i -> return $ return $ MetaString $ tshow i

101 YAML.SUnknown _ t ->

90 YAML.SUnknown _ t ->

102 case checkBoolean t of

91 case checkBoolean t of

103 Just b -> return $ return $ MetaBool b

92 Just b -> return $ return $ MetaBool b

104− Nothing -> toMetaValue pBlocks t

93+ Nothing -> normalizeMetaValue pMetaValue t

105 YAML.SNull -> return $ return $ MetaString ""

94 YAML.SNull -> return $ return $ MetaString ""

106

107−yamlToMetaValue pBlocks (YAML.Sequence _ _ xs) = do

96+yamlToMetaValue pMetaValue (YAML.Sequence _ _ xs) =

97+ fmap MetaList . sequence

108− xs' <- mapM (yamlToMetaValue pBlocks) xs

98+ <$> mapM (yamlToMetaValue pMetaValue) xs

109− return $ do

110− xs'' <- sequence xs'

111− return $ B.toMetaValue xs''

112−yamlToMetaValue pBlocks (YAML.Mapping _ _ o) =

99+yamlToMetaValue pMetaValue (YAML.Mapping _ _ o) =

113− fmap B.toMetaValue <$> yamlMap pBlocks o

100+ fmap MetaMap <$> yamlMap pMetaValue o

114yamlToMetaValue _ _ = return $ return $ MetaString ""

101yamlToMetaValue _ _ = return $ return $ MetaString ""

115

102

116yamlMap :: PandocMonad m

103yamlMap :: PandocMonad m

117 => ParserT Text ParserState m (F MetaValue)

104 => ParserT Text ParserState m (F MetaValue)

118 -> M.Map (YAML.Node YE.Pos) (YAML.Node YE.Pos)

105 -> M.Map (YAML.Node YE.Pos) (YAML.Node YE.Pos)

119 -> ParserT Text ParserState m (F (M.Map Text MetaValue))

106 -> ParserT Text ParserState m (F (M.Map Text MetaValue))

120−yamlMap pBlocks o = do

107+yamlMap pMetaValue o = do

121 kvs <- forM (M.toList o) $ \(key, v) -> do

108 kvs <- forM (M.toList o) $ \(key, v) -> do

122 k <- nodeToKey key

109 k <- nodeToKey key

123 return (k, v)

110 return (k, v)

@@ -126,7 +113,7 @@ yamlMap pBlocks o = do

126 where

113 where

127 ignorable t = "_" `T.isSuffixOf` t

114 ignorable t = "_" `T.isSuffixOf` t

128 toMeta (k, v) = do

115 toMeta (k, v) = do

129− fv <- yamlToMetaValue pBlocks v

116+ fv <- yamlToMetaValue pMetaValue v

130 return $ do

117 return $ do

131 v' <- fv

118 v' <- fv

132 return (k, v')

119 return (k, v')

src/Text/Pandoc/Readers/Org/Blocks.hs+11

@@ -76,6 +76,7 @@ block = choice [ mempty <$ blanklines

76 , list

77 , latexFragment

78 , noteBlock

79+ , rawOrgLine

79 , paraOrPlain

80 , paraOrPlain

80 ] <?> "block"

81 ] <?> "block"

@@ -559,6 +560,8 @@ include = try $ do

559 | otherwise -> Para content

560 | otherwise -> Para content

560 _ -> blk

561 _ -> blk

561

562

563+-- | Parses a meta line which defines a raw block. Currently recognized:

564+-- @#+LATEX:@, @#+HTML:@, @#+TEXINFO:@, and @#+BEAMER@.

562rawExportLine :: PandocMonad m => OrgParser m Blocks

565rawExportLine :: PandocMonad m => OrgParser m Blocks

563rawExportLine = try $ do

566rawExportLine = try $ do

564 metaLineStart

567 metaLineStart

@@ -567,6 +570,14 @@ rawExportLine = try $ do

567 then B.rawBlock key <$> anyLine

570 then B.rawBlock key <$> anyLine

568 else mzero

571 else mzero

569

572

573+-- | Parses any meta line, i.e., a line starting with @#+@, into a raw

574+-- org block. This should be the last resort when trying to parse

575+-- keywords. Leading spaces are discarded.

576+rawOrgLine :: PandocMonad m => OrgParser m (F Blocks)

577+rawOrgLine = do

578+ line <- metaLineStart *> anyLine

579+ returnF $ B.rawBlock "org" $ ("#+" <> line)

580+

570commentLine :: Monad m => OrgParser m Blocks

581commentLine :: Monad m => OrgParser m Blocks

571commentLine = commentLineStart *> anyLine *> pure mempty

582commentLine = commentLineStart *> anyLine *> pure mempty

572

583

src/Text/Pandoc/Readers/Org/Meta.hs+67−70

@@ -22,7 +22,7 @@ import Text.Pandoc.Readers.Org.Inlines

22import Text.Pandoc.Readers.Org.ParserState

23import Text.Pandoc.Readers.Org.Parsing

25−import Text.Pandoc.Builder (Blocks, Inlines, ToMetaValue)

25+import Text.Pandoc.Builder (Blocks, Inlines)

26import qualified Text.Pandoc.Builder as B

27import Text.Pandoc.Class.PandocMonad (PandocMonad)

28import Text.Pandoc.Definition

@@ -57,13 +57,13 @@ removeMeta key meta' =

57-- The order, in which blocks are tried, makes sure that we're not looking at

58-- the beginning of a block, so we don't need to check for it

59metaLine :: PandocMonad m => OrgParser m Blocks

60−metaLine = mempty <$ metaLineStart <* (optionLine <|> declarationLine)

60+metaLine = try $ mempty <$ metaLineStart <* keywordLine

62−declarationLine :: PandocMonad m => OrgParser m ()

62+keywordLine :: PandocMonad m => OrgParser m ()

63−declarationLine = try $ do

63+keywordLine = try $ do

64 key <- T.toLower <$> metaKey

65− case Map.lookup key exportSettingHandlers of

65+ case Map.lookup key keywordHandlers of

66− Nothing -> () <$ anyLine

66+ Nothing -> fail $ "Unknown keyword: " ++ T.unpack key

67 Just hd -> hd

69metaKey :: Monad m => OrgParser m Text

@@ -71,28 +71,55 @@ metaKey = T.toLower <$> many1Char (noneOf ": \n\r")

71 <* char ':'

72 <* skipSpaces

74+infix 0 ~~>

75+(~~>) :: a -> b -> (a, b)

76+a ~~> b = (a, b)

77+

74−exportSettingHandlers :: PandocMonad m => Map Text (OrgParser m ())

78+keywordHandlers :: PandocMonad m => Map Text (OrgParser m ())

75−exportSettingHandlers = Map.fromList

79+keywordHandlers = Map.fromList

80+ [ "author" ~~> lineOfInlines `parseThen` collectLines "author"

76− [ ("result" , fmap pure anyLine `parseThen` discard)

81+ , "creator" ~~> fmap pure anyLine `parseThen` B.setMeta "creator"

77− -- Common settings

82+ , "date" ~~> lineOfInlines `parseThen` B.setMeta "date"

78− , ("author" , lineOfInlines `parseThen` collectLines "author")

83+ , "description" ~~> lineOfInlines `parseThen` collectLines "description"

79− , ("date" , lineOfInlines `parseThen` setField "date")

84+ , "email" ~~> fmap pure anyLine `parseThen` B.setMeta "email"

85+ , "exclude_tags" ~~> tagList >>= updateState . setExcludedTags

86+ , "header-includes" ~~>

80− , ("description", lineOfInlines `parseThen` collectLines "description")

87+ lineOfInlines `parseThen` collectLines "header-includes"

88+ -- HTML-specifix export settings

89+ , "html_head" ~~>

90+ metaExportSnippet "html" `parseThen` collectAsList "header-includes"

91+ , "html_head_extra" ~~>

92+ metaExportSnippet "html" `parseThen` collectAsList "header-includes"

81− , ("keywords" , lineOfInlines `parseThen` collectLines "keywords")

93+ , "institute" ~~> lineOfInlines `parseThen` collectLines "institute"

94+ -- topic keywords

82− , ("title" , lineOfInlines `parseThen` collectLines "title")

95+ , "keywords" ~~> lineOfInlines `parseThen` collectLines "keywords"

83− -- LaTeX

96+ -- LaTeX-specific export settings

84− , ("latex_class", fmap pure anyLine `parseThen` setField "documentclass")

97+ , "latex_class" ~~> fmap pure anyLine `parseThen` B.setMeta "documentclass"

98+ , "latex_class_options" ~~>

85− , ("latex_class_options", (pure . T.filter (`notElem` ("[]" :: String)) <$> anyLine)

99+ (pure . T.filter (`notElem` ("[]" :: String)) <$> anyLine)

86− `parseThen` setField "classoption")

100+ `parseThen` B.setMeta "classoption"

87− , ("latex_header", metaExportSnippet "latex" `parseThen`

101+ , "latex_header" ~~>

88− collectAsList "header-includes")

102+ metaExportSnippet "latex" `parseThen` collectAsList "header-includes"

89− -- HTML

103+ , "latex_header_extra" ~~>

90− , ("html_head" , metaExportSnippet "html" `parseThen`

91− collectAsList "header-includes")

104+ metaExportSnippet "latex" `parseThen` collectAsList "header-includes"

105+ -- link and macro

106+ , "link" ~~> addLinkFormatter

107+ , "macro" ~~> macroDefinition >>= updateState . registerMacro

92− -- pandoc-specific

108+ -- pandoc-specific way to include references in the bibliography

93− , ("nocite" , lineOfInlines `parseThen` collectLines "nocite")

109+ , "nocite" ~~> lineOfInlines `parseThen` collectLines "nocite"

110+ -- compact way to set export settings

111+ , "options" ~~> exportSettings

112+ -- pandoc-specific way to configure emphasis recognition

113+ , "pandoc-emphasis-post" ~~> emphChars >>= updateState . setEmphasisPostChar

114+ , "pandoc-emphasis-pre" ~~> emphChars >>= updateState . setEmphasisPreChar

115+ -- result markers (ignored)

116+ , "result" ~~> void anyLine

117+ , "select_tags" ~~> tagList >>= updateState . setSelectedTags

118+ , "seq_todo" ~~> todoSequence >>= updateState . registerTodoSequence

94− , ("header-includes", lineOfInlines `parseThen` collectLines "header-includes")

119+ , "subtitle" ~~> lineOfInlines `parseThen` collectLines "subtitle"

95− , ("institute" , lineOfInlines `parseThen` collectLines "institute")

120+ , "title" ~~> lineOfInlines `parseThen` collectLines "title"

121+ , "todo" ~~> todoSequence >>= updateState . registerTodoSequence

122+ , "typ_todo" ~~> todoSequence >>= updateState . registerTodoSequence

96 ]

123 ]

124

98parseThen :: PandocMonad m

125parseThen :: PandocMonad m

@@ -104,9 +131,6 @@ parseThen p modMeta = do

104 meta <- orgStateMeta <$> getState

131 meta <- orgStateMeta <$> getState

105 updateState (\st -> st { orgStateMeta = modMeta <$> value <*> meta })

132 updateState (\st -> st { orgStateMeta = modMeta <$> value <*> meta })

106

133

107−discard :: a -> Meta -> Meta

108−discard = const id

109−

110collectLines :: Text -> Inlines -> Meta -> Meta

134collectLines :: Text -> Inlines -> Meta -> Meta

111collectLines key value meta =

135collectLines key value meta =

112 let value' = appendValue meta (B.toList value)

136 let value' = appendValue meta (B.toList value)

@@ -141,51 +165,25 @@ collectAsList key value meta =

141 Just x -> [x]

165 Just x -> [x]

142 _ -> []

166 _ -> []

143

167

144−setField :: ToMetaValue a => Text -> a -> Meta -> Meta

145−setField field value meta = B.setMeta field (B.toMetaValue value) meta

146−

147-- | Read an format specific meta definition

168-- | Read an format specific meta definition

148metaExportSnippet :: Monad m => Text -> OrgParser m (F Inlines)

169metaExportSnippet :: Monad m => Text -> OrgParser m (F Inlines)

149metaExportSnippet format = pure . B.rawInline format <$> anyLine

170metaExportSnippet format = pure . B.rawInline format <$> anyLine

150

171

151−--

172+-- | Parse a link type definition (like @wp https://en.wikipedia.org/wiki/@).

152−-- export options

153−--

154−optionLine :: PandocMonad m => OrgParser m ()

155−optionLine = try $ do

156− key <- metaKey

157− case key of

158− "link" -> parseLinkFormat >>= uncurry addLinkFormat

159− "options" -> exportSettings

160− "todo" -> todoSequence >>= updateState . registerTodoSequence

161− "seq_todo" -> todoSequence >>= updateState . registerTodoSequence

162− "typ_todo" -> todoSequence >>= updateState . registerTodoSequence

163− "macro" -> macroDefinition >>= updateState . registerMacro

164− "exclude_tags" -> tagList >>= updateState . setExcludedTags

165− "select_tags" -> tagList >>= updateState . setSelectedTags

166− "pandoc-emphasis-pre" -> emphChars >>= updateState . setEmphasisPreChar

167− "pandoc-emphasis-post" -> emphChars >>= updateState . setEmphasisPostChar

168− _ -> mzero

169−

170−addLinkFormat :: Monad m => Text

171− -> (Text -> Text)

172− -> OrgParser m ()

173−addLinkFormat key formatter = updateState $ \s ->

174− let fs = orgStateLinkFormatters s

175− in s{ orgStateLinkFormatters = Map.insert key formatter fs }

176−

177−parseLinkFormat :: Monad m => OrgParser m (Text, Text -> Text)

173+addLinkFormatter :: Monad m => OrgParser m ()

178−parseLinkFormat = try $ do

174+addLinkFormatter = try $ do

179 linkType <- T.cons <$> letter <*> manyChar (alphaNum <|> oneOf "-_") <* skipSpaces

175 linkType <- T.cons <$> letter <*> manyChar (alphaNum <|> oneOf "-_") <* skipSpaces

180− linkSubst <- parseFormat

176+ formatter <- parseFormat

181− return (linkType, linkSubst)

177+ updateState $ \s ->

178+ let fs = orgStateLinkFormatters s

179+ in s{ orgStateLinkFormatters = Map.insert linkType formatter fs }

182

180

183-- | An ad-hoc, single-argument-only implementation of a printf-style format

181-- | An ad-hoc, single-argument-only implementation of a printf-style format

184-- parser.

182-- parser.

185parseFormat :: Monad m => OrgParser m (Text -> Text)

183parseFormat :: Monad m => OrgParser m (Text -> Text)

186parseFormat = try $ replacePlain <|> replaceUrl <|> justAppend

184parseFormat = try $ replacePlain <|> replaceUrl <|> justAppend

187 where

185 where

188− -- inefficient, but who cares

186+ -- inefficient

189 replacePlain = try $ (\x -> T.concat . flip intersperse x)

187 replacePlain = try $ (\x -> T.concat . flip intersperse x)

190 <$> sequence [tillSpecifier 's', rest]

188 <$> sequence [tillSpecifier 's', rest]

191 replaceUrl = try $ (\x -> T.concat . flip intersperse x . T.pack . urlEncode . T.unpack)

189 replaceUrl = try $ (\x -> T.concat . flip intersperse x . T.pack . urlEncode . T.unpack)

@@ -224,6 +222,7 @@ setEmphasisPostChar csMb st =

224 let postChars = fromMaybe (orgStateEmphasisPostChars defaultOrgParserState) csMb

222 let postChars = fromMaybe (orgStateEmphasisPostChars defaultOrgParserState) csMb

225 in st { orgStateEmphasisPostChars = postChars }

223 in st { orgStateEmphasisPostChars = postChars }

226

224

225+-- | Parses emphasis border character like @".,?!"@

227emphChars :: Monad m => OrgParser m (Maybe [Char])

226emphChars :: Monad m => OrgParser m (Maybe [Char])

228emphChars = do

227emphChars = do

229 skipSpaces

228 skipSpaces

@@ -234,16 +233,14 @@ lineOfInlines = do

234 updateLastPreCharPos

233 updateLastPreCharPos

235 trimInlinesF . mconcat <$> manyTill inline newline

234 trimInlinesF . mconcat <$> manyTill inline newline

236

235

237−--

236+-- | Parses ToDo sequences / keywords like @TODO DOING | DONE@.

238−-- ToDo Sequences and Keywords

239−--

240todoSequence :: Monad m => OrgParser m TodoSequence

237todoSequence :: Monad m => OrgParser m TodoSequence

241todoSequence = try $ do

238todoSequence = try $ do

242 todoKws <- todoKeywords

239 todoKws <- todoKeywords

243 doneKws <- optionMaybe $ todoDoneSep *> todoKeywords

240 doneKws <- optionMaybe $ todoDoneSep *> todoKeywords

244 newline

241 newline

245− -- There must be at least one DONE keyword. The last TODO keyword is taken if

242+ -- There must be at least one DONE keyword. The last TODO keyword is

246− -- necessary.

243+ -- taken if necessary.

247 case doneKws of

244 case doneKws of

248 Just done -> return $ keywordsToSequence todoKws done

245 Just done -> return $ keywordsToSequence todoKws done

249 Nothing -> case reverse todoKws of

246 Nothing -> case reverse todoKws of

test/Tests/Readers/Org/Meta.hs+110−72

@@ -93,6 +93,13 @@ tests =

93 in Pandoc meta mempty

94 ]

96+ , "Subtitle" =:

97+ T.unlines [ "#+SUBTITLE: Your Life in"

98+ , "#+SUBTITLE: /Plain/ Text"

99+ ] =?>

100+ let subtitle = "Your Life in" <> softbreak <> emph "Plain" <> " Text"

101+ in Pandoc (setMeta "subtitle" (toMetaValue subtitle) nullMeta) mempty

102+

96 , "Keywords" =:

103 , "Keywords" =:

97 T.unlines [ "#+KEYWORDS: pandoc, testing,"

104 T.unlines [ "#+KEYWORDS: pandoc, testing,"

98 , "#+KEYWORDS: Org"

105 , "#+KEYWORDS: Org"

@@ -101,42 +108,117 @@ tests =

101 meta = setMeta "keywords" (MetaInlines keywords) nullMeta

108 meta = setMeta "keywords" (MetaInlines keywords) nullMeta

102 in Pandoc meta mempty

109 in Pandoc meta mempty

103

110

104− , "LaTeX_headers options are translated to header-includes" =:

111+ , "Institute" =:

105− "#+LaTeX_header: \\usepackage{tikz}" =?>

112+ "#+INSTITUTE: ACME Inc." =?>

106− let latexInlines = rawInline "latex" "\\usepackage{tikz}"

113+ Pandoc (setMeta "institute" ("ACME Inc." :: Inlines) nullMeta) mempty

107− inclList = MetaList [MetaInlines (toList latexInlines)]

108− meta = setMeta "header-includes" inclList nullMeta

109− in Pandoc meta mempty

110

114

111− , testGroup "LaTeX_CLASS"

115+ , testGroup "LaTeX"

112− [ "LaTeX_class option is translated to documentclass" =:

116+ [ "LATEX_HEADER" =:

113− "#+LATEX_CLASS: article" =?>

117+ "#+LaTeX_header: \\usepackage{tikz}" =?>

114− let meta = setMeta "documentclass" (MetaString "article") nullMeta

118+ let latexInlines = rawInline "latex" "\\usepackage{tikz}"

119+ inclList = MetaList [MetaInlines (toList latexInlines)]

120+ meta = setMeta "header-includes" inclList nullMeta

115 in Pandoc meta mempty

121 in Pandoc meta mempty

116

122

123+ , "LATEX_HEADER_EXTRA" =:

124+ "#+LATEX_HEADER_EXTRA: \\usepackage{calc}" =?>

125+ let latexInlines = rawInline "latex" "\\usepackage{calc}"

126+ inclList = toMetaValue [latexInlines]

127+ in Pandoc (setMeta "header-includes" inclList nullMeta) mempty

128+

129+ , testGroup "LaTeX_CLASS"

130+ [ "stored as documentclass" =:

131+ "#+LATEX_CLASS: article" =?>

132+ let meta = setMeta "documentclass" (MetaString "article") nullMeta

133+ in Pandoc meta mempty

134+

117− , "last definition takes precedence" =:

135+ , "last definition takes precedence" =:

118− T.unlines [ "#+LATEX_CLASS: this will not be used"

136+ T.unlines [ "#+LATEX_CLASS: this will not be used"

119− , "#+LATEX_CLASS: report"

137+ , "#+LATEX_CLASS: report"

120− ] =?>

138+ ] =?>

121− let meta = setMeta "documentclass" (MetaString "report") nullMeta

139+ let meta = setMeta "documentclass" (MetaString "report") nullMeta

140+ in Pandoc meta mempty

141+ ]

142+

143+ , "LATEX_CLASS_OPTIONS as classoption" =:

144+ "#+LATEX_CLASS_OPTIONS: [a4paper]" =?>

145+ let meta = setMeta "classoption" (MetaString "a4paper") nullMeta

122 in Pandoc meta mempty

146 in Pandoc meta mempty

123 ]

147 ]

124

148

125− , "LaTeX_class_options is translated to classoption" =:

149+ , testGroup "HTML"

126− "#+LATEX_CLASS_OPTIONS: [a4paper]" =?>

150+ [ "HTML_HEAD values are added to header-includes" =:

127− let meta = setMeta "classoption" (MetaString "a4paper") nullMeta

151+ "#+html_head: <meta/>" =?>

152+ let html = rawInline "html" "<meta/>"

153+ inclList = MetaList [MetaInlines (toList html)]

154+ meta = setMeta "header-includes" inclList nullMeta

128− in Pandoc meta mempty

155+ in Pandoc meta mempty

129

156

130− , "LaTeX_class_options is translated to classoption" =:

157+ , "HTML_HEAD_EXTRA behaves like HTML_HEAD" =:

131− "#+html_head: <meta/>" =?>

158+ T.unlines [ "#+HTML_HEAD: <meta name=\"generator\" content=\"pandoc\">"

159+ , "#+HTML_HEAD_EXTRA: <meta charset=\"utf-8\">"

160+ ] =?>

161+ let generator = rawInline "html"

162+ "<meta name=\"generator\" content=\"pandoc\">"

132− let html = rawInline "html" "<meta/>"

163+ charset = rawInline "html" "<meta charset=\"utf-8\">"

133− inclList = MetaList [MetaInlines (toList html)]

164+ inclList = toMetaValue [generator, charset]

134− meta = setMeta "header-includes" inclList nullMeta

165+ in Pandoc (setMeta "header-includes" inclList nullMeta) mempty

135− in Pandoc meta mempty

166+ ]

167+ ]

136

168

137− , "Institute" =:

169+ , testGroup "Non-export keywords"

138− "#+INSTITUTE: ACME Inc." =?>

170+ [ testGroup "#+LINK"

139− Pandoc (setMeta "institute" ("ACME Inc." :: Inlines) nullMeta) mempty

171+ [ "Link abbreviation" =:

172+ T.unlines [ "#+LINK: wp https://en.wikipedia.org/wiki/%s"

173+ , "[[wp:Org_mode][Wikipedia on Org-mode]]"

174+ ] =?>

175+ para (link "https://en.wikipedia.org/wiki/Org_mode" ""

176+ ("Wikipedia" <> space <> "on" <> space <> "Org-mode"))

177+

178+ , "Link abbreviation, defined after first use" =:

179+ T.unlines [ "[[zl:non-sense][Non-sense articles]]"

180+ , "#+LINK: zl http://zeitlens.com/tags/%s.html"

181+ ] =?>

182+ para (link "http://zeitlens.com/tags/non-sense.html" ""

183+ ("Non-sense" <> space <> "articles"))

184+

185+ , "Link abbreviation, URL encoded arguments" =:

186+ T.unlines [ "#+link: expl http://example.com/%h/foo"

187+ , "[[expl:Hello, World!][Moin!]]"

188+ ] =?>

189+ para (link "http://example.com/Hello%2C%20World%21/foo" "" "Moin!")

190+

191+ , "Link abbreviation, append arguments" =:

192+ T.unlines [ "#+link: expl http://example.com/"

193+ , "[[expl:foo][bar]]"

194+ ] =?>

195+ para (link "http://example.com/foo" "" "bar")

196+ ]

197+

198+ , testGroup "emphasis config"

199+ [ "Changing pre and post chars for emphasis" =:

200+ T.unlines [ "#+pandoc-emphasis-pre: \"[)\""

201+ , "#+pandoc-emphasis-post: \"]\\n\""

202+ , "([/emph/])*foo*"

203+ ] =?>

204+ para ("([" <> emph "emph" <> "])" <> strong "foo")

205+

206+ , "setting an invalid value restores the default" =:

207+ T.unlines [ "#+pandoc-emphasis-pre: \"[\""

208+ , "#+pandoc-emphasis-post: \"]\""

209+ , "#+pandoc-emphasis-pre:"

210+ , "#+pandoc-emphasis-post:"

211+ , "[/noemph/]"

212+ ] =?>

213+ para "[/noemph/]"

214+ ]

215+

216+ , "Unknown keyword" =:

217+ T.unlines [ "#+UNKNOWN_KEYWORD: Chumbawamba"

218+ , "#+ANOTHER_UNKNOWN: Blur"

219+ ] =?>

220+ rawBlock "org" "#+UNKNOWN_KEYWORD: Chumbawamba" <>

221+ rawBlock "org" "#+ANOTHER_UNKNOWN: Blur"

140 ]

222 ]

141

223

142 , "Properties drawer" =:

224 , "Properties drawer" =:

@@ -193,48 +275,4 @@ tests =

193 ] =?>

275 ] =?>

194 (para (spanWith ("link-here", [], []) mempty <> "Target.") <>

276 (para (spanWith ("link-here", [], []) mempty <> "Target.") <>

195 para (emph ("See" <> space <> "here!")))

277 para (emph ("See" <> space <> "here!")))

196−

197− , "Link abbreviation" =:

198− T.unlines [ "#+LINK: wp https://en.wikipedia.org/wiki/%s"

199− , "[[wp:Org_mode][Wikipedia on Org-mode]]"

200− ] =?>

201− para (link "https://en.wikipedia.org/wiki/Org_mode" ""

202− ("Wikipedia" <> space <> "on" <> space <> "Org-mode"))

203−

204− , "Link abbreviation, defined after first use" =:

205− T.unlines [ "[[zl:non-sense][Non-sense articles]]"

206− , "#+LINK: zl http://zeitlens.com/tags/%s.html"

207− ] =?>

208− para (link "http://zeitlens.com/tags/non-sense.html" ""

209− ("Non-sense" <> space <> "articles"))

210−

211− , "Link abbreviation, URL encoded arguments" =:

212− T.unlines [ "#+link: expl http://example.com/%h/foo"

213− , "[[expl:Hello, World!][Moin!]]"

214− ] =?>

215− para (link "http://example.com/Hello%2C%20World%21/foo" "" "Moin!")

216−

217− , "Link abbreviation, append arguments" =:

218− T.unlines [ "#+link: expl http://example.com/"

219− , "[[expl:foo][bar]]"

220− ] =?>

221− para (link "http://example.com/foo" "" "bar")

222−

223− , testGroup "emphasis config"

224− [ "Changing pre and post chars for emphasis" =:

225− T.unlines [ "#+pandoc-emphasis-pre: \"[)\""

226− , "#+pandoc-emphasis-post: \"]\\n\""

227− , "([/emph/])*foo*"

228− ] =?>

229− para ("([" <> emph "emph" <> "])" <> strong "foo")

230−

231− , "setting an invalid value restores the default" =:

232− T.unlines [ "#+pandoc-emphasis-pre: \"[\""

233− , "#+pandoc-emphasis-post: \"]\""

234− , "#+pandoc-emphasis-pre:"

235− , "#+pandoc-emphasis-post:"

236− , "[/noemph/]"

237− ] =?>

238− para "[/noemph/]"

239− ]

240 ]

278 ]