0cb7362 · typeable/pandoc

git.iko.soy/typeable/pandoc

Removed `--strict`, added extensions to writer/reader names.

* The `--strict` option has been removed. * Instead of using `--strict`, one can now use `strict` instead of `markdown` as an input or output format name. * The `--enable` and `--disable` optinos have been removed. * It is now possible to enable or disable specific extensions by appending them (with '+' or '-') to the writer or reader name. For example `pandoc -f markdown-footnotes+hard_line_breaks`. * The lhs extensions are now implemented this way, too; you can use either `+lhs` or `+literate_haskell`.

commit0cb7362f62410f58e2356381bbf2c1fe85abe2a5 authorJohn MacFarlane <jgm@berkeley.edu> dateThu 9 Aug 2012, 20:19 (14 years ago) parent71e0c20

5 files changed+73−78

compare with master

unifiedsplit

README+15−22 src/Text/Pandoc.hs+14−5 src/Text/Pandoc/Shared.hs+3−2 src/Text/Pandoc/Templates.hs+9−8 src/pandoc.hs+32−41

README+15−22

@@ -26,8 +26,8 @@ tables, flexible ordered lists, definition lists, delimited code blocks,

2626superscript, subscript, strikeout, title blocks, automatic tables of

2727contents, embedded LaTeX math, citations, and markdown inside HTML block

2828elements. (These enhancements, described below under

29−[Pandoc's markdown](#pandocs-markdown), can be disabled using the `--strict`

30−option.)

29+[Pandoc's markdown](#pandocs-markdown), can be disabled using the `strict`

30+input or output format.)

3131

3232In contrast to most existing tools for converting markdown to HTML, which

3333use regex substitutions, Pandoc has a modular design: it consists of a

@@ -117,10 +117,11 @@ and `xunicode` (if `xelatex` is used).

117117

118118A user who wants a drop-in replacement for `Markdown.pl` may create

119119a symbolic link to the `pandoc` executable called `hsmarkdown`. When

120−invoked under the name `hsmarkdown`, `pandoc` will behave as if the

121−`--strict` flag had been selected, and no command-line options will be

122−recognized. However, this approach does not work under Cygwin, due to

123−problems with its simulation of symbolic links.

120+invoked under the name `hsmarkdown`, `pandoc` will behave as if

121+invoked with `-f strict --email-obfuscation=references`,

122+and all command-line options will be treated as regular arguments.

123+However, this approach does not work under Cygwin, due to problems with

124+its simulation of symbolic links.

124125

125126[Cygwin]: http://www.cygwin.com/

126127[`iconv`]: http://www.gnu.org/software/libiconv/

@@ -192,12 +193,6 @@ General options

192193Reader options

193194--------------

194195

195−`--strict`

196−: Use strict markdown syntax, with no pandoc extensions or variants.

197− When the input format is HTML, this means that constructs that have no

198− equivalents in standard markdown (e.g. definition lists or strikeout

199− text) will be parsed as raw HTML.

200−

201196`-R`, `--parse-raw`

202197: Parse untranslatable HTML codes and LaTeX environments as raw HTML

203198 or LaTeX, instead of ignoring them. Affects only HTML and LaTeX

@@ -403,8 +398,6 @@ Options affecting specific writers

403398 *none* leaves `mailto:` links as they are. *javascript* obfuscates

404399 them using javascript. *references* obfuscates them by printing their

405400 letters as decimal or hexadecimal character references.

406− If `--strict` is specified, *references* is used regardless of the

407− presence of this option.

408401

409402`--id-prefix`=*STRING*

410403: Specify a prefix to be added to all automatically generated identifiers

@@ -764,8 +757,8 @@ Pandoc's markdown

764757Pandoc understands an extended and slightly revised version of

765758John Gruber's [markdown] syntax. This document explains the syntax,

766759noting differences from standard markdown. Except where noted, these

767−differences can be suppressed by specifying the `--strict` command-line

768−option.

760+differences can be suppressed by using the `strict` format instead

761+of `markdown`.

769762

770763Philosophy

771764----------

@@ -934,8 +927,8 @@ Standard markdown syntax does not require a blank line before a block

934927quote. Pandoc does require this (except, of course, at the beginning of the

935928document). The reason for the requirement is that it is all too easy for a

936929`>` to end up at the beginning of a line by accident (perhaps through line

937−wrapping). So, unless `--strict` is used, the following does not produce

938−a nested block quote in pandoc:

930+wrapping). So, unless the `strict` format is used, the following does

931+not produce a nested block quote in pandoc:

939932

940933 > This is a block quote.

941934 >> Nested.

@@ -1291,7 +1284,7 @@ around "Third". Pandoc follows a simple rule: if the text is followed by

12911284a blank line, it is treated as a paragraph. Since "Second" is followed

12921285by a list, and not a blank line, it isn't treated as a paragraph. The

12931286fact that the list is followed by a blank line is irrelevant. (Note:

1294−Pandoc works this way even when the `--strict` option is specified. This

1287+Pandoc works this way even when the `strict` format is specified. This

12951288behavior is consistent with the official markdown syntax description,

12961289even though it is different from that of `Markdown.pl`.)

12971290

@@ -1612,8 +1605,8 @@ which allows only the following characters to be backslash-escaped:

16121605

16131606 \`*_{}[]()>#+-.!

16141607

1615−(However, if the `--strict` option is supplied, the standard

1616−markdown rule will be used.)

1608+(However, if the `strict` format is used, the standard markdown rule

1609+will be used.)

16171610

16181611A backslash-escaped space is parsed as a nonbreaking space. It will

16191612appear in TeX output as `~` and in HTML and XML as `\ ` or

@@ -1847,7 +1840,7 @@ with blank lines, and start and end at the left margin. Within

18471840these blocks, everything is interpreted as HTML, not markdown;

18481841so (for example), `*` does not signify emphasis.

18491842

1850−Pandoc behaves this way when `--strict` is specified; but by default,

1843+Pandoc behaves this way when the `strict` format is used; but by default,

18511844pandoc interprets material between HTML block tags as markdown.

18521845Thus, for example, Pandoc will turn

18531846

src/Text/Pandoc.hs+14−5

@@ -139,11 +139,13 @@ import Text.Pandoc.Templates

139139import Text.Pandoc.Options

140140import Text.Pandoc.Shared (safeRead)

141141import Data.ByteString.Lazy (ByteString)

142+import Data.List (intercalate)

142143import Data.Version (showVersion)

143144import Text.JSON.Generic

144145import Data.Set (Set)

145146import qualified Data.Set as Set

146147import Text.Parsec

148+import Text.Parsec.Error

147149import Paths_pandoc (version)

148150

149151-- | Version number of pandoc library.

@@ -160,10 +162,12 @@ parseFormatSpec = parse formatSpec ""

160162 formatName = many1 $ noneOf "-+"

161163 extMod = do

162164 polarity <- oneOf "-+"

163− name <- many1 $ noneOf "-+"

164− ext <- case safeRead name of

165+ name <- many $ noneOf "-+"

166+ ext <- case safeRead ("Ext_" ++ name) of

165167 Just n -> return n

166− Nothing -> unexpected $ "Unknown extension: " ++ name

168+ Nothing

169+ | name == "lhs" -> return Ext_literate_haskell

170+ | otherwise -> fail $ "Unknown extension: " ++ name

167171 return $ case polarity of

168172 '-' -> Set.delete ext

169173 _ -> Set.insert ext

@@ -172,6 +176,8 @@ parseFormatSpec = parse formatSpec ""

172176readers :: [(String, ReaderOptions -> String -> Pandoc)]

173177readers = [("native" , \_ -> readNative)

174178 ,("json" , \_ -> decodeJSON)

179+ ,("strict" , \o -> readMarkdown

180+ o{ readerExtensions = strictExtensions } )

175181 ,("markdown" , readMarkdown)

176182 ,("rst" , readRST)

177183 ,("docbook" , readDocBook)

@@ -215,6 +221,8 @@ writers = [

215221 ,("texinfo" , PureStringWriter writeTexinfo)

216222 ,("man" , PureStringWriter writeMan)

217223 ,("markdown" , PureStringWriter writeMarkdown)

224+ ,("strict" , PureStringWriter $ \o ->

225+ writeMarkdown o{ writerExtensions = strictExtensions } )

218226 ,("plain" , PureStringWriter writePlain)

219227 ,("rst" , PureStringWriter writeRST)

220228 ,("mediawiki" , PureStringWriter writeMediaWiki)

@@ -224,10 +232,11 @@ writers = [

224232 ,("asciidoc" , PureStringWriter writeAsciiDoc)

225233 ]

226234

235+-- | Retrieve reader based on formatSpec (format+extensions).

227236getReader :: String -> Either String (ReaderOptions -> String -> Pandoc)

228237getReader s =

229238 case parseFormatSpec s of

230− Left e -> Left $ show e

239+ Left e -> Left $ intercalate "\n" $ [m | Message m <- errorMessages e]

231240 Right (readerName, setExts) ->

232241 case lookup readerName readers of

233242 Nothing -> Left $ "Unknown reader: " ++ readerName

@@ -239,7 +248,7 @@ getReader s =

239248getWriter :: String -> Either String Writer

240249getWriter s =

241250 case parseFormatSpec s of

242− Left e -> Left $ show e

251+ Left e -> Left $ intercalate "\n" $ [m | Message m <- errorMessages e]

243252 Right (writerName, setExts) ->

244253 case lookup writerName writers of

245254 Nothing -> Left $ "Unknown writer: " ++ writerName

src/Text/Pandoc/Shared.hs+3−2

@@ -498,5 +498,6 @@ warn msg = do

498498

499499safeRead :: (Monad m, Read a) => String -> m a

500500safeRead s = case reads s of

501− (d,[]):_ -> return d

502− _ -> fail $ "Could not read `" ++ s ++ "'"

501+ (d,x):_

502+ | all isSpace x -> return d

503+ _ -> fail $ "Could not read `" ++ s ++ "'"

src/Text/Pandoc/Templates.hs+9−8

@@ -86,15 +86,16 @@ import qualified Control.Exception.Extensible as E (try, IOException)

8686getDefaultTemplate :: (Maybe FilePath) -- ^ User data directory to search first

8787 -> String -- ^ Name of writer

8888 -> IO (Either E.IOException String)

89−getDefaultTemplate _ "native" = return $ Right ""

90−getDefaultTemplate _ "json" = return $ Right ""

91−getDefaultTemplate _ "docx" = return $ Right ""

92−getDefaultTemplate user "odt" = getDefaultTemplate user "opendocument"

93−getDefaultTemplate user "epub" = getDefaultTemplate user "html"

9489getDefaultTemplate user writer = do

95− let format = takeWhile (/='+') writer -- strip off "+lhs" if present

96− let fname = "templates" </> "default" <.> format

97− E.try $ readDataFile user fname

90+ let format = takeWhile (`notElem` "+-") writer -- strip off extensions

91+ case format of

92+ "native" -> return $ Right ""

93+ "json" -> return $ Right ""

94+ "docx" -> return $ Right ""

95+ "odt" -> getDefaultTemplate user "opendocument"

96+ "epub" -> return $ Right ""

97+ _ -> let fname = "templates" </> "default" <.> format

98+ in E.try $ readDataFile user fname

9899

99100data TemplateState = TemplateState Int [(String,String)]

100101

src/pandoc.hs+32−41

@@ -44,7 +44,7 @@ import System.Exit ( exitWith, ExitCode (..) )

4444import System.FilePath

4545import System.Console.GetOpt

4646import Data.Char ( toLower )

47−import Data.List ( intercalate, isSuffixOf, isPrefixOf )

47+import Data.List ( intercalate, isPrefixOf )

4848import System.Directory ( getAppUserDataDirectory, doesFileExist, findExecutable )

4949import System.IO ( stdout )

5050import System.IO.Error ( isDoesNotExistError )

@@ -58,7 +58,6 @@ import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..))

5858import Network.URI (parseURI, isURI, URI(..))

5959import qualified Data.ByteString.Lazy as B

6060import Data.ByteString.Lazy.UTF8 (toString)

61−import qualified Data.Set as Set

6261import Text.CSL.Reference (Reference(..))

6362#if MIN_VERSION_base(4,4,0)

6463#else

@@ -99,8 +98,8 @@ wrapWords indent c = wrap' (c - indent) (c - indent)

9998 then ",\n" ++ replicate indent ' ' ++ x ++ wrap' cols (cols - length x) xs

10099 else ", " ++ x ++ wrap' cols (remaining - (length x + 2)) xs

101100

102−nonTextFormats :: [String]

103−nonTextFormats = ["odt","docx","epub"]

101+isTextFormat :: String -> Bool

102+isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub"]

104103

105104-- | Data structure for command line options.

106105data Opt = Opt

@@ -133,7 +132,6 @@ data Opt = Opt

133132 , optEPUBFonts :: [FilePath] -- ^ EPUB fonts to embed

134133 , optDumpArgs :: Bool -- ^ Output command-line arguments

135134 , optIgnoreArgs :: Bool -- ^ Ignore command-line arguments

136− , optStrict :: Bool -- ^ Use strict markdown syntax

137135 , optReferenceLinks :: Bool -- ^ Use reference links in writing markdown, rst

138136 , optWrapText :: Bool -- ^ Wrap text

139137 , optColumns :: Int -- ^ Line length in characters

@@ -186,7 +184,6 @@ defaultOpts = Opt

186184 , optEPUBFonts = []

187185 , optDumpArgs = False

188186 , optIgnoreArgs = False

189− , optStrict = False

190187 , optReferenceLinks = False

191188 , optWrapText = True

192189 , optColumns = 72

@@ -237,7 +234,10 @@ options =

237234

238235 , Option "" ["strict"]

239236 (NoArg

240− (\opt -> return opt { optStrict = True } ))

237+ (\opt -> do

238+ err 59 $ "The --strict option has been removed.\n" ++

239+ "Use `strict' input or output format instead."

240+ return opt ))

241241 "" -- "Disable markdown syntax extensions"

242242

243243 , Option "R" ["parse-raw"]

@@ -702,7 +702,7 @@ usageMessage :: String -> [OptDescr (Opt -> IO Opt)] -> String

702702usageMessage programName = usageInfo

703703 (programName ++ " [OPTIONS] [FILES]" ++ "\nInput formats: " ++

704704 (wrapWords 16 78 $ readers'names) ++ "\nOutput formats: " ++

705− (wrapWords 16 78 $ writers'names ++ nonTextFormats) ++ "\nOptions:")

705+ (wrapWords 16 78 $ writers'names) ++ "\nOptions:")

706706 where

707707 writers'names = map fst writers

708708 readers'names = map fst readers

@@ -782,9 +782,10 @@ main = do

782782 ["Try " ++ prg ++ " --help for more information."]

783783

784784 let defaultOpts' = if compatMode

785− then defaultOpts { optReader = "markdown"

785+ then defaultOpts { optReader = "strict"

786786 , optWriter = "html"

787− , optStrict = True }

787+ , optEmailObfuscation =

788+ ReferenceObfuscation }

788789 else defaultOpts

789790

790791 -- thread option data structure through all supplied option actions

@@ -819,7 +820,6 @@ main = do

819820 , optEPUBFonts = epubFonts

820821 , optDumpArgs = dumpArgs

821822 , optIgnoreArgs = ignoreArgs

822− , optStrict = strict

823823 , optReferenceLinks = referenceLinks

824824 , optWrapText = wrap

825825 , optColumns = columns

@@ -867,8 +867,8 @@ main = do

867867

868868 let pdfOutput = map toLower (takeExtension outputFile) == ".pdf"

869869

870− let laTeXOutput = writerName' == "latex" || writerName' == "beamer" ||

871− writerName' == "latex+lhs" || writerName' == "beamer+lhs"

870+ let laTeXOutput = "latex" `isPrefixOf` writerName' ||

871+ "beamer" `isPrefixOf` writerName'

872872

873873 when pdfOutput $ do

874874 -- make sure writer is latex or beamer

@@ -882,11 +882,11 @@ main = do

882882 latexEngine ++ " is needed for pdf output."

883883 Just _ -> return ()

884884

885− reader <- case (lookup readerName' readers) of

886− Just r -> return r

887− Nothing -> err 7 ("Unknown reader: " ++ readerName')

885+ reader <- case getReader readerName' of

886+ Right r -> return r

887+ Left e -> err 7 e

888888

889− let standalone' = standalone || writerName' `elem` nonTextFormats || pdfOutput

889+ let standalone' = standalone || not (isTextFormat writerName') || pdfOutput

890890

891891 templ <- case templatePath of

892892 _ | not standalone' -> return ""

@@ -896,8 +896,8 @@ main = do

896896 Left e -> throwIO e

897897 Right t -> return t

898898 Just tp -> do

899− -- strip off "+lhs" if present

900− let format = takeWhile (/='+') writerName'

899+ -- strip off extensions

900+ let format = takeWhile (`notElem` "+-") writerName'

901901 let tp' = case takeExtension tp of

902902 "" -> tp <.> format

903903 _ -> tp

@@ -919,13 +919,13 @@ main = do

919919 return $ ("mathml-script", s) : variables

920920 _ -> return variables

921921

922− variables'' <- case writerName' of

923− "dzslides" -> do

922+ variables'' <- if "dzslides" `isPrefixOf` writerName'

923+ then do

924924 dztempl <- readDataFile datadir $ "dzslides" </> "template.html"

925925 let dzcore = unlines $ dropWhile (not . isPrefixOf "<!-- {{{{ dzslides core")

926926 $ lines dztempl

927927 return $ ("dzslides-core", dzcore) : variables'

928− _ -> return variables'

928+ else return variables'

929929

930930 -- unescape reference ids, which may contain XML entities, so

931931 -- that we can do lookups with regular string equality

@@ -942,13 +942,8 @@ main = do

942942 then "."

943943 else takeDirectory (head sources)

944944

945− let defaultExts = if strict

946− then strictExtensions

947− else pandocExtensions

948−

949− let readerOpts = def{ readerExtensions = defaultExts

950− , readerSmart = smart || (texLigatures &&

951− (laTeXOutput || writerName' == "context"))

945+ let readerOpts = def{ readerSmart = smart || (texLigatures &&

946+ (laTeXOutput || "context" `isPrefixOf` writerName'))

952947 , readerStandalone = standalone'

953948 , readerParseRaw = parseRaw

954949 , readerColumns = columns

@@ -972,13 +967,10 @@ main = do

972967 writerIgnoreNotes = False,

973968 writerNumberSections = numberSections,

974969 writerSectionDivs = sectionDivs,

975− writerExtensions = defaultExts,

976970 writerReferenceLinks = referenceLinks,

977971 writerWrapText = wrap,

978972 writerColumns = columns,

979− writerEmailObfuscation = if strict

980− then ReferenceObfuscation

981− else obfuscationMethod,

973+ writerEmailObfuscation = obfuscationMethod,

982974 writerIdentifierPrefix = idPrefix,

983975 writerSourceDirectory = sourceDir,

984976 writerUserDataDir = datadir,

@@ -997,7 +989,7 @@ main = do

997989 writerReferenceDocx = referenceDocx

998990 }

999991

1000− when (writerName' `elem` nonTextFormats&& outputFile == "-") $

992+ when (not (isTextFormat writerName') && outputFile == "-") $

1001993 err 5 $ "Cannot write " ++ writerName' ++ " output to stdout.\n" ++

1002994 "Specify an output file using the -o option."

1003995

@@ -1022,7 +1014,7 @@ main = do

10221014

10231015 let doc0 = foldr ($) doc transforms

10241016

1025− doc1 <- if writerName' == "rtf"

1017+ doc1 <- if "rtf" `isPrefixOf` writerName'

10261018 then bottomUpM rtfEmbedImage doc0

10271019 else return doc0

10281020

@@ -1050,12 +1042,11 @@ main = do

10501042 writerFn "-" = UTF8.putStr

10511043 writerFn f = UTF8.writeFile f

10521044

1053− let mbwriter = lookup writerName' writers

1054− case mbwriter of

1055− Nothing -> err 9 ("Unknown writer: " ++ writerName')

1056− Just (IOStringWriter f) -> f writerOptions doc2 >>= writerFn outputFile

1057− Just (IOByteStringWriter f) -> f writerOptions doc2 >>= writeBinary

1058− Just (PureStringWriter f)

1045+ case getWriter writerName' of

1046+ Left e -> err 9 e

1047+ Right (IOStringWriter f) -> f writerOptions doc2 >>= writerFn outputFile

1048+ Right (IOByteStringWriter f) -> f writerOptions doc2 >>= writeBinary

1049+ Right (PureStringWriter f)

10591050 | pdfOutput -> do

10601051 res <- tex2pdf latexEngine $ f writerOptions doc2

10611052 case res of