master...d463ef0 · typeable/pandoc

git.iko.soy/typeable/pandoc

7 files changed+285−195

since splitsnapshots

unifiedsplit

changelog.md+57−1 doc/org.md+15−11 src/Text/Pandoc/Readers/Markdown.hs+2−5 src/Text/Pandoc/Readers/Metadata.hs+23−36 src/Text/Pandoc/Readers/Org/Blocks.hs+11 src/Text/Pandoc/Readers/Org/Meta.hs+67−70 test/Tests/Readers/Org/Meta.hs+110−72

changelog.md+57−1

@@ -31,6 +31,12 @@

3131 `--no-check-certificate`, which disables certificate checking when

3232 resources are fetched by HTTP.

3333

34+ * Unify defaults and markdown metadata parsers (#6328, Nikolay

35+ Yakimov). Clean up code in Text.Pandoc.Readers.Metadata and

36+ properly handle errors in `yamlToMeta`. This fixes parsing

37+ of Boolean fields in metadata withinin defaults files and reduces

38+ code duplication.

39+

3440 * Docbook reader:

3541

3642 + Implement `<procedure>` (#6442, Mathieu Boespflug).

@@ -39,7 +45,11 @@

3945 + Implement `<replaceable>` (#6437, Mathieu Boespflug)

4046 + Map `<simplesect>` to unnumbered section (#6436, Mathieu Boespflug).

4147

42− * JATS reader: handle "label" element in section title (#6288).

48+ * JATS reader:

49+

50+ + Handle "label" element in section title (#6288).

51+ + Parse abstract element into metadata field of same name

52+ (#6480, Albert Krewinkel).

4353

4454 * Jira reader (Albert Krewinkel):

4555

@@ -80,6 +90,9 @@

8090 + Fix inline code in lists (#6284, Nikolay Yakimov).

8191 Previously inline code containing list markers was sometimes

8292 parsed incorrectly.

93+ + Don't require blank line after grid table (#6481).

94+ This allows grid tables to be enclosed in fenced divs with no

95+ intervening blank lines.

8396

8497 * LaTeX reader:

8598

@@ -93,6 +106,41 @@

93106 * Org reader (Albert Krewinkel):

94107

95108 + Recognize images with uppercase extensions (#6472).

109+ + Keep unknown keyword lines as raw org. The lines of unknown

110+ keywords, like `#+SOMEWORD: value` are no longer read as metadata,

111+ but kept as raw `org` blocks. This ensures that more information

112+ is retained when round-tripping org-mode files; additionally,

113+ this change makes it possible to support non-standard org

114+ extensions via filters.

115+ + Unify keyword handling. Handling of export settings and other

116+ keywords (like `#+LINK`) has been combined and unified.

117+ + Support `LATEX_HEADER_EXTRA` and `HTML_HEAD_EXTRA`

118+ settings. These export settings are treated like their non-extra

119+ counterparts, i.e., the values are added to the `header-includes`

120+ metadata list.

121+ + Allow multiple `#+SUBTITLE` export settings. The values of all

122+ lines are read as inlines and collected in the `subtitle`

123+ metadata field.

124+ + Read `#+INSTITUTE` values as text with markup. The value is

125+ stored in the `institute` metadata field and used in the

126+ default beamer presentation template.

127+ + The behavior of the `#+AUTHOR` and `#+KEYWORD` export

128+ settings has changed: Org now allows multiple such lines

129+ and adds a space between the contents of each line. Pandoc

130+ now always parses these settings as meta inlines; setting

131+ values are no longer treated as comma-separated lists.

132+ Note that a Lua filter can be used to restore the previous

133+ behavior.

134+ + Read description lines as inlines (#6485). `#+DESCRIPTION` lines

135+ are now treated as text with markup. If multiple such

136+ lines are given, then all lines are read and separated by soft

137+ linebreaks.

138+ + Honor tex export option (#4070). The `tex` export option can be set

139+ with `#+OPTION: tex:nil` and allows three settings:

140+ `t` (the default) causes LaTeX fragments to be parsed as TeX or added

141+ as raw TeX. `nil` removes all LaTeX fragments from the document.

142+ `verbatim` treats LaTeX as text.

143+

96144

97145 * RST reader:

98146

@@ -244,6 +292,7 @@

244292 + Use `--enable-local-file-access` in invoking `wkhtmltopdf` (#6474).

245293 `wkhtmltopdf` changed in recent versions to require this for

246294 access to local files. This fixes PDF via HTML5 with `--css`.

295+ + Send verbose output to stderr, not stdout (#6483).

247296

248297 * Text.Pandoc.MIME: Fix MIME type for TrueType fonts in EPUBs

249298 (#6464, Michael Reed).

@@ -317,6 +366,8 @@

317366

318367 * Remove unnecessary hlint ignores (#6341, Joseph C. Sible).

319368

369+ * Remove obsolete RelaxedPolyRec extension (#6487, Nikolay Yakimov).

370+

320371 * trypandoc improvements (Mike Tzou):

321372

322373 + Add standalone option to the command text (#6210).

@@ -335,6 +386,11 @@

335386 + Update links to reveal.js documentation (#6386, Salim B).

336387 + Separate adjacent verbatim code blocks (#6307, tom-audm).

337388

389+ * org.md:

390+

391+ + Document behavior of `smart` extension (#4387, Albert Krewinkel).

392+ + Describe all supported export options in detail.

393+

338394 * lua-filters.md:

339395

340396 + Fix description of BulletList Lua type (Levi Gruspe).

doc/org.md+15−11

@@ -14,13 +14,13 @@ The following export keywords are supported:

1414

1515- AUTHOR: comma-separated list of author(s); fully supported.

1616

17−- CREATOR: output generator; passed as metadata entry, but

18− ignored by most output formats.

17+- CREATOR: output generator; passed as plain-text metadata entry

18+ `creator`, but not used by any default templates.

1919

2020- DATE: creation or publication date; well supported by pandoc.

2121

22−- EMAIL: author email address; passed as metadata entry, but not

23− included in most output formats.

22+- EMAIL: author email address; passed as plain-text metadata

23+ field `email`, but not used by any default templates.

2424

2525- LANGUAGE: currently unsupported; use `#+LANG:` instead.

2626

@@ -51,11 +51,12 @@ occur.

5151

5252 Pandoc follows the LaTeX exporter in that it allows markup in

5353 the description. In contrast, the Org-mode HTML exporter treats

54− reads the description as plain text.

54+ the description as plain text.

5555

56−- LATEX_HEADER: arbitrary lines to add to the document's preamble.

57− Contrary to Org-mode, these lines are not inserted before the

58− hyperref settings, but close to the end of the preamble.

56+- LATEX\_HEADER and LATEX_HEADER_EXTRA: arbitrary lines to add to

57+ the document's preamble. Contrary to Org-mode, these lines are

58+ not inserted before the hyperref settings, but close to the end

59+ of the preamble.

5960

6061 The contents of this option are stored as a list of raw LaTeX

6162 lines in the `header-includes` metadata field.

@@ -74,10 +75,13 @@ occur.

7475

7576- SUBTITLE: the document's subtitle; fully supported.

7677

77−- HTML_HEAD: arbitrary lines to add to the HTML document's head;

78− fully supported.

78+ The content of this option is stored as inlines in the

79+ `subtitle` metadata field.

80+

81+- HTML\_HEAD and HTML\_HEAD\_EXTRA: arbitrary lines to add to the

82+ HTML document's head; fully supported.

7983

80− The contents of this option are stored as a list of raw HTML

84+ The contents of these options are stored as a list of raw HTML

8185 lines in the `header-includes` metadata field.

8286

8387Pandoc-specific options

src/Text/Pandoc/Readers/Markdown.hs+2−5

@@ -67,16 +67,13 @@ yamlToMeta :: PandocMonad m

6767 -> m Meta

6868yamlToMeta opts bstr = do

6969 let parser = do

70− meta <- yamlBsToMeta (asBlocks <$> parseBlocks) bstr

70+ meta <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks) bstr

7171 return $ runF meta defaultParserState

7272 parsed <- readWithM parser def{ stateOptions = opts } ""

7373 case parsed of

7474 Right result -> return result

7575 Left e -> throwError e

7676

77−asBlocks :: Functor f => f (B.Many Block) -> f MetaValue

78−asBlocks p = MetaBlocks . B.toList <$> p

79−

8077--

8178-- Constants and data structure definitions

8279--

@@ -241,7 +238,7 @@ yamlMetaBlock = try $ do

241238 -- by including --- and ..., we allow yaml blocks with just comments:

242239 let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."]))

243240 optional blanklines

244− newMetaF <- yamlBsToMeta (asBlocks <$> parseBlocks)

241+ newMetaF <- yamlBsToMeta (fmap B.toMetaValue <$> parseBlocks)

245242 $ UTF8.fromTextLazy $ TL.fromStrict rawYaml

246243 -- Since `<>` is left-biased, existing values are not touched:

247244 updateState $ \st -> st{ stateMeta' = stateMeta' st <> newMetaF }

src/Text/Pandoc/Readers/Metadata.hs+23−36

@@ -1,4 +1,3 @@

1−{-# LANGUAGE ScopedTypeVariables #-}

21{-# LANGUAGE OverloadedStrings #-}

32{- |

43 Module : Text.Pandoc.Readers.Metadata

@@ -21,7 +20,6 @@ import Data.Text (Text)

2120import qualified Data.Text as T

2221import qualified Data.YAML as YAML

2322import qualified Data.YAML.Event as YE

24−import qualified Text.Pandoc.Builder as B

2523import Text.Pandoc.Class.PandocMonad (PandocMonad (..))

2624import Text.Pandoc.Definition

2725import Text.Pandoc.Error

@@ -33,11 +31,11 @@ yamlBsToMeta :: PandocMonad m

3331 => ParserT Text ParserState m (F MetaValue)

3432 -> BL.ByteString

3533 -> ParserT Text ParserState m (F Meta)

36−yamlBsToMeta pBlocks bstr = do

34+yamlBsToMeta pMetaValue bstr = do

3735 pos <- getPosition

3836 case YAML.decodeNode' YAML.failsafeSchemaResolver False False bstr of

3937 Right (YAML.Doc (YAML.Mapping _ _ o):_)

40− -> fmap Meta <$> yamlMap pBlocks o

38+ -> fmap Meta <$> yamlMap pMetaValue o

4139 Right [] -> return . return $ mempty

4240 Right [YAML.Doc (YAML.Scalar _ YAML.SNull)]

4341 -> return . return $ mempty

@@ -57,30 +55,21 @@ nodeToKey (YAML.Scalar _ (YAML.SUnknown _ t)) = return t

5755nodeToKey _ = throwError $ PandocParseError

5856 "Non-string key in YAML mapping"

5957

60−toMetaValue :: PandocMonad m

61− => ParserT Text ParserState m (F MetaValue)

62− -> Text

63− -> ParserT Text ParserState m (F MetaValue)

64−toMetaValue pBlocks x =

58+normalizeMetaValue :: PandocMonad m

59+ => ParserT Text ParserState m (F MetaValue)

60+ -> Text

61+ -> ParserT Text ParserState m (F MetaValue)

62+normalizeMetaValue pMetaValue x =

6563 -- Note: a standard quoted or unquoted YAML value will

6664 -- not end in a newline, but a "block" set off with

6765 -- `|` or `>` will.

6866 if "\n" `T.isSuffixOf` x

69− then parseFromString' pBlocks (x <> "\n")

70− else parseFromString' pInlines x

71− where pInlines = do

72− bs <- pBlocks

73− return $ do

74− bs' <- bs

75− return $

76− case bs' of

77− MetaBlocks bs'' ->

78− case bs'' of

79− [Plain ils] -> MetaInlines ils

80− [Para ils] -> MetaInlines ils

81− xs -> MetaBlocks xs

82− _ -> bs'

83−

67+ then parseFromString' pMetaValue (x <> "\n")

68+ else parseFromString' asInlines x

69+ where asInlines = fmap b2i <$> pMetaValue

70+ b2i (MetaBlocks [Plain ils]) = MetaInlines ils

71+ b2i (MetaBlocks [Para ils]) = MetaInlines ils

72+ b2i bs = bs

8473

8574checkBoolean :: Text -> Maybe Bool

8675checkBoolean t

@@ -92,32 +81,30 @@ yamlToMetaValue :: PandocMonad m

9281 => ParserT Text ParserState m (F MetaValue)

9382 -> YAML.Node YE.Pos

9483 -> ParserT Text ParserState m (F MetaValue)

95−yamlToMetaValue pBlocks (YAML.Scalar _ x) =

84+yamlToMetaValue pMetaValue (YAML.Scalar _ x) =

9685 case x of

97− YAML.SStr t -> toMetaValue pBlocks t

86+ YAML.SStr t -> normalizeMetaValue pMetaValue t

9887 YAML.SBool b -> return $ return $ MetaBool b

9988 YAML.SFloat d -> return $ return $ MetaString $ tshow d

10089 YAML.SInt i -> return $ return $ MetaString $ tshow i

10190 YAML.SUnknown _ t ->

10291 case checkBoolean t of

10392 Just b -> return $ return $ MetaBool b

104− Nothing -> toMetaValue pBlocks t

93+ Nothing -> normalizeMetaValue pMetaValue t

10594 YAML.SNull -> return $ return $ MetaString ""

10695

107−yamlToMetaValue pBlocks (YAML.Sequence _ _ xs) = do

108− xs' <- mapM (yamlToMetaValue pBlocks) xs

109− return $ do

110− xs'' <- sequence xs'

111− return $ B.toMetaValue xs''

112−yamlToMetaValue pBlocks (YAML.Mapping _ _ o) =

113− fmap B.toMetaValue <$> yamlMap pBlocks o

96+yamlToMetaValue pMetaValue (YAML.Sequence _ _ xs) =

97+ fmap MetaList . sequence

98+ <$> mapM (yamlToMetaValue pMetaValue) xs

99+yamlToMetaValue pMetaValue (YAML.Mapping _ _ o) =

100+ fmap MetaMap <$> yamlMap pMetaValue o

114101yamlToMetaValue _ _ = return $ return $ MetaString ""

115102

116103yamlMap :: PandocMonad m

117104 => ParserT Text ParserState m (F MetaValue)

118105 -> M.Map (YAML.Node YE.Pos) (YAML.Node YE.Pos)

119106 -> ParserT Text ParserState m (F (M.Map Text MetaValue))

120−yamlMap pBlocks o = do

107+yamlMap pMetaValue o = do

121108 kvs <- forM (M.toList o) $ \(key, v) -> do

122109 k <- nodeToKey key

123110 return (k, v)

@@ -126,7 +113,7 @@ yamlMap pBlocks o = do

126113 where

127114 ignorable t = "_" `T.isSuffixOf` t

128115 toMeta (k, v) = do

129− fv <- yamlToMetaValue pBlocks v

116+ fv <- yamlToMetaValue pMetaValue v

130117 return $ do

131118 v' <- fv

132119 return (k, v')

src/Text/Pandoc/Readers/Org/Blocks.hs+11

@@ -76,6 +76,7 @@ block = choice [ mempty <$ blanklines

7676 , list

7777 , latexFragment

7878 , noteBlock

79+ , rawOrgLine

7980 , paraOrPlain

8081 ] <?> "block"

8182

@@ -559,6 +560,8 @@ include = try $ do

559560 | otherwise -> Para content

560561 _ -> blk

561562

563+-- | Parses a meta line which defines a raw block. Currently recognized:

564+-- @#+LATEX:@, @#+HTML:@, @#+TEXINFO:@, and @#+BEAMER@.

562565rawExportLine :: PandocMonad m => OrgParser m Blocks

563566rawExportLine = try $ do

564567 metaLineStart

@@ -567,6 +570,14 @@ rawExportLine = try $ do

567570 then B.rawBlock key <$> anyLine

568571 else mzero

569572

573+-- | Parses any meta line, i.e., a line starting with @#+@, into a raw

574+-- org block. This should be the last resort when trying to parse

575+-- keywords. Leading spaces are discarded.

576+rawOrgLine :: PandocMonad m => OrgParser m (F Blocks)

577+rawOrgLine = do

578+ line <- metaLineStart *> anyLine

579+ returnF $ B.rawBlock "org" $ ("#+" <> line)

580+

570581commentLine :: Monad m => OrgParser m Blocks

571582commentLine = commentLineStart *> anyLine *> pure mempty

572583

src/Text/Pandoc/Readers/Org/Meta.hs+67−70

@@ -22,7 +22,7 @@ import Text.Pandoc.Readers.Org.Inlines

2222import Text.Pandoc.Readers.Org.ParserState

2323import Text.Pandoc.Readers.Org.Parsing

2424

25−import Text.Pandoc.Builder (Blocks, Inlines, ToMetaValue)

25+import Text.Pandoc.Builder (Blocks, Inlines)

2626import qualified Text.Pandoc.Builder as B

2727import Text.Pandoc.Class.PandocMonad (PandocMonad)

2828import Text.Pandoc.Definition

@@ -57,13 +57,13 @@ removeMeta key meta' =

5757-- The order, in which blocks are tried, makes sure that we're not looking at

5858-- the beginning of a block, so we don't need to check for it

5959metaLine :: PandocMonad m => OrgParser m Blocks

60−metaLine = mempty <$ metaLineStart <* (optionLine <|> declarationLine)

60+metaLine = try $ mempty <$ metaLineStart <* keywordLine

6161

62−declarationLine :: PandocMonad m => OrgParser m ()

63−declarationLine = try $ do

62+keywordLine :: PandocMonad m => OrgParser m ()

63+keywordLine = try $ do

6464 key <- T.toLower <$> metaKey

65− case Map.lookup key exportSettingHandlers of

66− Nothing -> () <$ anyLine

65+ case Map.lookup key keywordHandlers of

66+ Nothing -> fail $ "Unknown keyword: " ++ T.unpack key

6767 Just hd -> hd

6868

6969metaKey :: Monad m => OrgParser m Text

@@ -71,28 +71,55 @@ metaKey = T.toLower <$> many1Char (noneOf ": \n\r")

7171 <* char ':'

7272 <* skipSpaces

7373

74−exportSettingHandlers :: PandocMonad m => Map Text (OrgParser m ())

75−exportSettingHandlers = Map.fromList

76− [ ("result" , fmap pure anyLine `parseThen` discard)

77− -- Common settings

78− , ("author" , lineOfInlines `parseThen` collectLines "author")

79− , ("date" , lineOfInlines `parseThen` setField "date")

80− , ("description", lineOfInlines `parseThen` collectLines "description")

81− , ("keywords" , lineOfInlines `parseThen` collectLines "keywords")

82− , ("title" , lineOfInlines `parseThen` collectLines "title")

83− -- LaTeX

84− , ("latex_class", fmap pure anyLine `parseThen` setField "documentclass")

85− , ("latex_class_options", (pure . T.filter (`notElem` ("[]" :: String)) <$> anyLine)

86− `parseThen` setField "classoption")

87− , ("latex_header", metaExportSnippet "latex" `parseThen`

88− collectAsList "header-includes")

89− -- HTML

90− , ("html_head" , metaExportSnippet "html" `parseThen`

91− collectAsList "header-includes")

92− -- pandoc-specific

93− , ("nocite" , lineOfInlines `parseThen` collectLines "nocite")

94− , ("header-includes", lineOfInlines `parseThen` collectLines "header-includes")

95− , ("institute" , lineOfInlines `parseThen` collectLines "institute")

74+infix 0 ~~>

75+(~~>) :: a -> b -> (a, b)

76+a ~~> b = (a, b)

77+

78+keywordHandlers :: PandocMonad m => Map Text (OrgParser m ())

79+keywordHandlers = Map.fromList

80+ [ "author" ~~> lineOfInlines `parseThen` collectLines "author"

81+ , "creator" ~~> fmap pure anyLine `parseThen` B.setMeta "creator"

82+ , "date" ~~> lineOfInlines `parseThen` B.setMeta "date"

83+ , "description" ~~> lineOfInlines `parseThen` collectLines "description"

84+ , "email" ~~> fmap pure anyLine `parseThen` B.setMeta "email"

85+ , "exclude_tags" ~~> tagList >>= updateState . setExcludedTags

86+ , "header-includes" ~~>

87+ lineOfInlines `parseThen` collectLines "header-includes"

88+ -- HTML-specifix export settings

89+ , "html_head" ~~>

90+ metaExportSnippet "html" `parseThen` collectAsList "header-includes"

91+ , "html_head_extra" ~~>

92+ metaExportSnippet "html" `parseThen` collectAsList "header-includes"

93+ , "institute" ~~> lineOfInlines `parseThen` collectLines "institute"

94+ -- topic keywords

95+ , "keywords" ~~> lineOfInlines `parseThen` collectLines "keywords"

96+ -- LaTeX-specific export settings

97+ , "latex_class" ~~> fmap pure anyLine `parseThen` B.setMeta "documentclass"

98+ , "latex_class_options" ~~>

99+ (pure . T.filter (`notElem` ("[]" :: String)) <$> anyLine)

100+ `parseThen` B.setMeta "classoption"

101+ , "latex_header" ~~>

102+ metaExportSnippet "latex" `parseThen` collectAsList "header-includes"

103+ , "latex_header_extra" ~~>

104+ metaExportSnippet "latex" `parseThen` collectAsList "header-includes"

105+ -- link and macro

106+ , "link" ~~> addLinkFormatter

107+ , "macro" ~~> macroDefinition >>= updateState . registerMacro

108+ -- pandoc-specific way to include references in the bibliography

109+ , "nocite" ~~> lineOfInlines `parseThen` collectLines "nocite"

110+ -- compact way to set export settings

111+ , "options" ~~> exportSettings

112+ -- pandoc-specific way to configure emphasis recognition

113+ , "pandoc-emphasis-post" ~~> emphChars >>= updateState . setEmphasisPostChar

114+ , "pandoc-emphasis-pre" ~~> emphChars >>= updateState . setEmphasisPreChar

115+ -- result markers (ignored)

116+ , "result" ~~> void anyLine

117+ , "select_tags" ~~> tagList >>= updateState . setSelectedTags

118+ , "seq_todo" ~~> todoSequence >>= updateState . registerTodoSequence

119+ , "subtitle" ~~> lineOfInlines `parseThen` collectLines "subtitle"

120+ , "title" ~~> lineOfInlines `parseThen` collectLines "title"

121+ , "todo" ~~> todoSequence >>= updateState . registerTodoSequence

122+ , "typ_todo" ~~> todoSequence >>= updateState . registerTodoSequence

96123 ]

97124

98125parseThen :: PandocMonad m

@@ -104,9 +131,6 @@ parseThen p modMeta = do

104131 meta <- orgStateMeta <$> getState

105132 updateState (\st -> st { orgStateMeta = modMeta <$> value <*> meta })

106133

107−discard :: a -> Meta -> Meta

108−discard = const id

109−

110134collectLines :: Text -> Inlines -> Meta -> Meta

111135collectLines key value meta =

112136 let value' = appendValue meta (B.toList value)

@@ -141,51 +165,25 @@ collectAsList key value meta =

141165 Just x -> [x]

142166 _ -> []

143167

144−setField :: ToMetaValue a => Text -> a -> Meta -> Meta

145−setField field value meta = B.setMeta field (B.toMetaValue value) meta

146−

147168-- | Read an format specific meta definition

148169metaExportSnippet :: Monad m => Text -> OrgParser m (F Inlines)

149170metaExportSnippet format = pure . B.rawInline format <$> anyLine

150171

151−--

152−-- export options

153−--

154−optionLine :: PandocMonad m => OrgParser m ()

155−optionLine = try $ do

156− key <- metaKey

157− case key of

158− "link" -> parseLinkFormat >>= uncurry addLinkFormat

159− "options" -> exportSettings

160− "todo" -> todoSequence >>= updateState . registerTodoSequence

161− "seq_todo" -> todoSequence >>= updateState . registerTodoSequence

162− "typ_todo" -> todoSequence >>= updateState . registerTodoSequence

163− "macro" -> macroDefinition >>= updateState . registerMacro

164− "exclude_tags" -> tagList >>= updateState . setExcludedTags

165− "select_tags" -> tagList >>= updateState . setSelectedTags

166− "pandoc-emphasis-pre" -> emphChars >>= updateState . setEmphasisPreChar

167− "pandoc-emphasis-post" -> emphChars >>= updateState . setEmphasisPostChar

168− _ -> mzero

169−

170−addLinkFormat :: Monad m => Text

171− -> (Text -> Text)

172− -> OrgParser m ()

173−addLinkFormat key formatter = updateState $ \s ->

174− let fs = orgStateLinkFormatters s

175− in s{ orgStateLinkFormatters = Map.insert key formatter fs }

176−

177−parseLinkFormat :: Monad m => OrgParser m (Text, Text -> Text)

178−parseLinkFormat = try $ do

172+-- | Parse a link type definition (like @wp https://en.wikipedia.org/wiki/@).

173+addLinkFormatter :: Monad m => OrgParser m ()

174+addLinkFormatter = try $ do

179175 linkType <- T.cons <$> letter <*> manyChar (alphaNum <|> oneOf "-_") <* skipSpaces

180− linkSubst <- parseFormat

181− return (linkType, linkSubst)

176+ formatter <- parseFormat

177+ updateState $ \s ->

178+ let fs = orgStateLinkFormatters s

179+ in s{ orgStateLinkFormatters = Map.insert linkType formatter fs }

182180

183181-- | An ad-hoc, single-argument-only implementation of a printf-style format

184182-- parser.

185183parseFormat :: Monad m => OrgParser m (Text -> Text)

186184parseFormat = try $ replacePlain <|> replaceUrl <|> justAppend

187185 where

188− -- inefficient, but who cares

186+ -- inefficient

189187 replacePlain = try $ (\x -> T.concat . flip intersperse x)

190188 <$> sequence [tillSpecifier 's', rest]

191189 replaceUrl = try $ (\x -> T.concat . flip intersperse x . T.pack . urlEncode . T.unpack)

@@ -224,6 +222,7 @@ setEmphasisPostChar csMb st =

224222 let postChars = fromMaybe (orgStateEmphasisPostChars defaultOrgParserState) csMb

225223 in st { orgStateEmphasisPostChars = postChars }

226224

225+-- | Parses emphasis border character like @".,?!"@

227226emphChars :: Monad m => OrgParser m (Maybe [Char])

228227emphChars = do

229228 skipSpaces

@@ -234,16 +233,14 @@ lineOfInlines = do

234233 updateLastPreCharPos

235234 trimInlinesF . mconcat <$> manyTill inline newline

236235

237−--

238−-- ToDo Sequences and Keywords

239−--

236+-- | Parses ToDo sequences / keywords like @TODO DOING | DONE@.

240237todoSequence :: Monad m => OrgParser m TodoSequence

241238todoSequence = try $ do

242239 todoKws <- todoKeywords

243240 doneKws <- optionMaybe $ todoDoneSep *> todoKeywords

244241 newline

245− -- There must be at least one DONE keyword. The last TODO keyword is taken if

246− -- necessary.

242+ -- There must be at least one DONE keyword. The last TODO keyword is

243+ -- taken if necessary.

247244 case doneKws of

248245 Just done -> return $ keywordsToSequence todoKws done

249246 Nothing -> case reverse todoKws of

test/Tests/Readers/Org/Meta.hs+110−72

@@ -93,6 +93,13 @@ tests =

9393 in Pandoc meta mempty

9494 ]

9595

96+ , "Subtitle" =:

97+ T.unlines [ "#+SUBTITLE: Your Life in"

98+ , "#+SUBTITLE: /Plain/ Text"

99+ ] =?>

100+ let subtitle = "Your Life in" <> softbreak <> emph "Plain" <> " Text"

101+ in Pandoc (setMeta "subtitle" (toMetaValue subtitle) nullMeta) mempty

102+

96103 , "Keywords" =:

97104 T.unlines [ "#+KEYWORDS: pandoc, testing,"

98105 , "#+KEYWORDS: Org"

@@ -101,42 +108,117 @@ tests =

101108 meta = setMeta "keywords" (MetaInlines keywords) nullMeta

102109 in Pandoc meta mempty

103110

104− , "LaTeX_headers options are translated to header-includes" =:

105− "#+LaTeX_header: \\usepackage{tikz}" =?>

106− let latexInlines = rawInline "latex" "\\usepackage{tikz}"

107− inclList = MetaList [MetaInlines (toList latexInlines)]

108− meta = setMeta "header-includes" inclList nullMeta

109− in Pandoc meta mempty

111+ , "Institute" =:

112+ "#+INSTITUTE: ACME Inc." =?>

113+ Pandoc (setMeta "institute" ("ACME Inc." :: Inlines) nullMeta) mempty

110114

111− , testGroup "LaTeX_CLASS"

112− [ "LaTeX_class option is translated to documentclass" =:

113− "#+LATEX_CLASS: article" =?>

114− let meta = setMeta "documentclass" (MetaString "article") nullMeta

115+ , testGroup "LaTeX"

116+ [ "LATEX_HEADER" =:

117+ "#+LaTeX_header: \\usepackage{tikz}" =?>

118+ let latexInlines = rawInline "latex" "\\usepackage{tikz}"

119+ inclList = MetaList [MetaInlines (toList latexInlines)]

120+ meta = setMeta "header-includes" inclList nullMeta

115121 in Pandoc meta mempty

116122

117− , "last definition takes precedence" =:

118− T.unlines [ "#+LATEX_CLASS: this will not be used"

119− , "#+LATEX_CLASS: report"

120− ] =?>

121− let meta = setMeta "documentclass" (MetaString "report") nullMeta

123+ , "LATEX_HEADER_EXTRA" =:

124+ "#+LATEX_HEADER_EXTRA: \\usepackage{calc}" =?>

125+ let latexInlines = rawInline "latex" "\\usepackage{calc}"

126+ inclList = toMetaValue [latexInlines]

127+ in Pandoc (setMeta "header-includes" inclList nullMeta) mempty

128+

129+ , testGroup "LaTeX_CLASS"

130+ [ "stored as documentclass" =:

131+ "#+LATEX_CLASS: article" =?>

132+ let meta = setMeta "documentclass" (MetaString "article") nullMeta

133+ in Pandoc meta mempty

134+

135+ , "last definition takes precedence" =:

136+ T.unlines [ "#+LATEX_CLASS: this will not be used"

137+ , "#+LATEX_CLASS: report"

138+ ] =?>

139+ let meta = setMeta "documentclass" (MetaString "report") nullMeta

140+ in Pandoc meta mempty

141+ ]

142+

143+ , "LATEX_CLASS_OPTIONS as classoption" =:

144+ "#+LATEX_CLASS_OPTIONS: [a4paper]" =?>

145+ let meta = setMeta "classoption" (MetaString "a4paper") nullMeta

122146 in Pandoc meta mempty

123147 ]

124148

125− , "LaTeX_class_options is translated to classoption" =:

126− "#+LATEX_CLASS_OPTIONS: [a4paper]" =?>

127− let meta = setMeta "classoption" (MetaString "a4paper") nullMeta

128− in Pandoc meta mempty

149+ , testGroup "HTML"

150+ [ "HTML_HEAD values are added to header-includes" =:

151+ "#+html_head: <meta/>" =?>

152+ let html = rawInline "html" "<meta/>"

153+ inclList = MetaList [MetaInlines (toList html)]

154+ meta = setMeta "header-includes" inclList nullMeta

155+ in Pandoc meta mempty

129156

130− , "LaTeX_class_options is translated to classoption" =:

131− "#+html_head: <meta/>" =?>

132− let html = rawInline "html" "<meta/>"

133− inclList = MetaList [MetaInlines (toList html)]

134− meta = setMeta "header-includes" inclList nullMeta

135− in Pandoc meta mempty

157+ , "HTML_HEAD_EXTRA behaves like HTML_HEAD" =:

158+ T.unlines [ "#+HTML_HEAD: <meta name=\"generator\" content=\"pandoc\">"

159+ , "#+HTML_HEAD_EXTRA: <meta charset=\"utf-8\">"

160+ ] =?>

161+ let generator = rawInline "html"

162+ "<meta name=\"generator\" content=\"pandoc\">"

163+ charset = rawInline "html" "<meta charset=\"utf-8\">"

164+ inclList = toMetaValue [generator, charset]

165+ in Pandoc (setMeta "header-includes" inclList nullMeta) mempty

166+ ]

167+ ]

136168

137− , "Institute" =:

138− "#+INSTITUTE: ACME Inc." =?>

139− Pandoc (setMeta "institute" ("ACME Inc." :: Inlines) nullMeta) mempty

169+ , testGroup "Non-export keywords"

170+ [ testGroup "#+LINK"

171+ [ "Link abbreviation" =:

172+ T.unlines [ "#+LINK: wp https://en.wikipedia.org/wiki/%s"

173+ , "[[wp:Org_mode][Wikipedia on Org-mode]]"

174+ ] =?>

175+ para (link "https://en.wikipedia.org/wiki/Org_mode" ""

176+ ("Wikipedia" <> space <> "on" <> space <> "Org-mode"))

177+

178+ , "Link abbreviation, defined after first use" =:

179+ T.unlines [ "[[zl:non-sense][Non-sense articles]]"

180+ , "#+LINK: zl http://zeitlens.com/tags/%s.html"

181+ ] =?>

182+ para (link "http://zeitlens.com/tags/non-sense.html" ""

183+ ("Non-sense" <> space <> "articles"))

184+

185+ , "Link abbreviation, URL encoded arguments" =:

186+ T.unlines [ "#+link: expl http://example.com/%h/foo"

187+ , "[[expl:Hello, World!][Moin!]]"

188+ ] =?>

189+ para (link "http://example.com/Hello%2C%20World%21/foo" "" "Moin!")

190+

191+ , "Link abbreviation, append arguments" =:

192+ T.unlines [ "#+link: expl http://example.com/"

193+ , "[[expl:foo][bar]]"

194+ ] =?>

195+ para (link "http://example.com/foo" "" "bar")

196+ ]

197+

198+ , testGroup "emphasis config"

199+ [ "Changing pre and post chars for emphasis" =:

200+ T.unlines [ "#+pandoc-emphasis-pre: \"[)\""

201+ , "#+pandoc-emphasis-post: \"]\\n\""

202+ , "([/emph/])*foo*"

203+ ] =?>

204+ para ("([" <> emph "emph" <> "])" <> strong "foo")

205+

206+ , "setting an invalid value restores the default" =:

207+ T.unlines [ "#+pandoc-emphasis-pre: \"[\""

208+ , "#+pandoc-emphasis-post: \"]\""

209+ , "#+pandoc-emphasis-pre:"

210+ , "#+pandoc-emphasis-post:"

211+ , "[/noemph/]"

212+ ] =?>

213+ para "[/noemph/]"

214+ ]

215+

216+ , "Unknown keyword" =:

217+ T.unlines [ "#+UNKNOWN_KEYWORD: Chumbawamba"

218+ , "#+ANOTHER_UNKNOWN: Blur"

219+ ] =?>

220+ rawBlock "org" "#+UNKNOWN_KEYWORD: Chumbawamba" <>

221+ rawBlock "org" "#+ANOTHER_UNKNOWN: Blur"

140222 ]

141223

142224 , "Properties drawer" =:

@@ -193,48 +275,4 @@ tests =

193275 ] =?>

194276 (para (spanWith ("link-here", [], []) mempty <> "Target.") <>

195277 para (emph ("See" <> space <> "here!")))

196−

197− , "Link abbreviation" =:

198− T.unlines [ "#+LINK: wp https://en.wikipedia.org/wiki/%s"

199− , "[[wp:Org_mode][Wikipedia on Org-mode]]"

200− ] =?>

201− para (link "https://en.wikipedia.org/wiki/Org_mode" ""

202− ("Wikipedia" <> space <> "on" <> space <> "Org-mode"))

203−

204− , "Link abbreviation, defined after first use" =:

205− T.unlines [ "[[zl:non-sense][Non-sense articles]]"

206− , "#+LINK: zl http://zeitlens.com/tags/%s.html"

207− ] =?>

208− para (link "http://zeitlens.com/tags/non-sense.html" ""

209− ("Non-sense" <> space <> "articles"))

210−

211− , "Link abbreviation, URL encoded arguments" =:

212− T.unlines [ "#+link: expl http://example.com/%h/foo"

213− , "[[expl:Hello, World!][Moin!]]"

214− ] =?>

215− para (link "http://example.com/Hello%2C%20World%21/foo" "" "Moin!")

216−

217− , "Link abbreviation, append arguments" =:

218− T.unlines [ "#+link: expl http://example.com/"

219− , "[[expl:foo][bar]]"

220− ] =?>

221− para (link "http://example.com/foo" "" "bar")

222−

223− , testGroup "emphasis config"

224− [ "Changing pre and post chars for emphasis" =:

225− T.unlines [ "#+pandoc-emphasis-pre: \"[)\""

226− , "#+pandoc-emphasis-post: \"]\\n\""

227− , "([/emph/])*foo*"

228− ] =?>

229− para ("([" <> emph "emph" <> "])" <> strong "foo")

230−

231− , "setting an invalid value restores the default" =:

232− T.unlines [ "#+pandoc-emphasis-pre: \"[\""

233− , "#+pandoc-emphasis-post: \"]\""

234− , "#+pandoc-emphasis-pre:"

235− , "#+pandoc-emphasis-post:"

236− , "[/noemph/]"

237− ] =?>

238− para "[/noemph/]"

239− ]

240278 ]