43448d7 · typeable/pandoc

git.iko.soy/typeable/pandoc

on master help sign in

Preliminary changes for epub3 format.

43448d7John MacFarlane14 years ago

README+23−25

compare with master

unifiedsplit

@@ -17,7 +17,7 @@ another, and a command-line tool that uses this library. It can read

17text, [markdown], [reStructuredText], [XHTML], [HTML 5], [LaTeX]

18(including [beamer] slide shows), [ConTeXt], [RTF], [DocBook XML],

19[OpenDocument XML], [ODT], [Word docx], [GNU Texinfo], [MediaWiki

20−markup], [EPUB], [FictionBook2], [Textile], [groff man] pages, [Emacs

20+markup], [EPUB] (v2 or v3), [FictionBook2], [Textile], [groff man] pages, [Emacs

21Org-Mode], [AsciiDoc], and [Slidy], [Slideous], [DZSlides], or [S5] HTML

22slide shows. It can also produce [PDF] output on systems where LaTeX is

23installed.

@@ -44,7 +44,7 @@ If no *input-file* is specified, input is read from *stdin*.

44Otherwise, the *input-files* are concatenated (with a blank

45line between each) and used as input. Output goes to *stdout* by

46default (though output to *stdout* is disabled for the `odt`, `docx`,

47−and `epub` output formats). For output to a file, use the

47+`epub`, and `epub3` output formats). For output to a file, use the

48`-o` option:

50 pandoc -o output.html input.txt

@@ -159,29 +159,27 @@ General options

159 `json` (JSON version of native AST), `plain` (plain text),

160 `markdown` (pandoc's extended markdown), `markdown_strict` (original

161 unextended markdown), `rst` (reStructuredText), `html` (XHTML

162− 1), `html5` (HTML 5), `latex` (LaTeX), `beamer` (LaTeX beamer

162+ 1), `html5` (HTML 5), `latex` (LaTeX), `beamer` (LaTeX beamer slide show),

163− slide show), `context` (ConTeXt), `man` (groff man), `mediawiki`

163+ `context` (ConTeXt), `man` (groff man), `mediawiki` (MediaWiki markup),

164− (MediaWiki markup), `textile` (Textile), `org` (Emacs Org-Mode),

164+ `textile` (Textile), `org` (Emacs Org-Mode), `texinfo` (GNU Texinfo),

165− `texinfo` (GNU Texinfo), `docbook` (DocBook XML), `opendocument`

165+ `docbook` (DocBook XML), `opendocument` (OpenDocument XML), `odt`

166− (OpenDocument XML), `odt` (OpenOffice text document), `docx`

166+ (OpenOffice text document), `docx` (Word docx), `epub` (EPUB book), `epub3`

167− (Word docx), `epub` (EPUB book), `fb2` (FictionBook2 e-book),

167+ (EPUB v3), `fb2` (FictionBook2 e-book), `asciidoc` (AsciiDoc), `slidy`

168− `asciidoc` (AsciiDoc), `slidy` (Slidy HTML and javascript slide

169− show), `slideous` (Slideous HTML and javascript slide show),

168+ (Slidy HTML and javascript slide show), `slideous` (Slideous HTML and

170− `dzslides` (HTML5 + javascript slide show), `s5` (S5 HTML and

169+ javascript slide show), `dzslides` (HTML5 + javascript slide show), `s5`

171− javascript slide show), or `rtf` (rich text format). Note that

170+ (S5 HTML and javascript slide show), or `rtf` (rich text format). Note that

172− `odt` and `epub` output will not be directed to *stdout*; an output

171+ `odt`, `epub`, and `epub3` output will not be directed to *stdout*; an output

173− filename must be specified using the `-o/--output` option. If `+lhs`

172+ filename must be specified using the `-o/--output` option. If `+lhs` is

174− is appended to `markdown`, `rst`, `latex`, `beamer`, `html`, or

173+ appended to `markdown`, `rst`, `latex`, `beamer`, `html`, or `html5`, the

175− `html5`, the output will be rendered as literate Haskell source:

174+ output will be rendered as literate Haskell source: see [Literate Haskell

176− see [Literate Haskell support](#literate-haskell-support), below.

175+ support](#literate-haskell-support), below. Markdown syntax extensions can

177− Markdown syntax extensions can be individually enabled or disabled

176+ be individually enabled or disabled by appending `+EXTENSION` or

178− by appending `+EXTENSION` or `-EXTENSION` to the format name, as

179− described above under `-f`.

177+ `-EXTENSION` to the format name, as described above under `-f`.

180

178

181`-o` *FILE*, `--output=`*FILE*

179`-o` *FILE*, `--output=`*FILE*

182: Write output to *FILE* instead of *stdout*. If *FILE* is

180: Write output to *FILE* instead of *stdout*. If *FILE* is

183 `-`, output will go to *stdout*. (Exception: if the output

181 `-`, output will go to *stdout*. (Exception: if the output

184− format is `odt`, `docx`, or `epub`, output to stdout is disabled.)

182+ format is `odt`, `docx`, `epub`, or `epub3`, output to stdout is disabled.)

185

183

186`--data-dir=`*DIRECTORY*

184`--data-dir=`*DIRECTORY*

187: Specify the user data directory to search for pandoc data files.

185: Specify the user data directory to search for pandoc data files.

@@ -259,7 +257,7 @@ General writer options

259`-s`, `--standalone`

257`-s`, `--standalone`

260: Produce output with an appropriate header and footer (e.g. a

258: Produce output with an appropriate header and footer (e.g. a

261 standalone HTML, LaTeX, or RTF file, not a fragment). This option

259 standalone HTML, LaTeX, or RTF file, not a fragment). This option

262− is set automatically for `pdf`, `epub`, `fb2`, `docx`, and `odt`

260+ is set automatically for `pdf`, `epub`, `epub3`, `fb2`, `docx`, and `odt`

263 output.

261 output.

264

262

265`--template=`*FILE*

263`--template=`*FILE*

@@ -662,9 +660,9 @@ the system default templates for a given output format `FORMAT`

662by putting a file `templates/default.FORMAT` in the user data

660by putting a file `templates/default.FORMAT` in the user data

663directory (see `--data-dir`, above). *Exceptions:* For `odt` output,

661directory (see `--data-dir`, above). *Exceptions:* For `odt` output,

664customize the `default.opendocument` template. For `pdf` output,

662customize the `default.opendocument` template. For `pdf` output,

665−customize the `default.latex` template. For `epub` output, customize

663+customize the `default.latex` template. For `epub` and `epub3` output,

666−the `epub-page.html`, `epub-coverimage.html`, and `epub-titlepage.html`

664+customize the `epub-page.html`, `epub-coverimage.html`, and

667−templates.

665+`epub-titlepage.html` templates.

668

666

669Templates may contain *variables*. Variable names are sequences of

667Templates may contain *variables*. Variable names are sequences of

670alphanumerics, `-`, and `_`, starting with a letter. A variable name

668alphanumerics, `-`, and `_`, starting with a letter. A variable name

src/Text/Pandoc.hs+4−2

@@ -92,7 +92,8 @@ module Text.Pandoc

92 , writeRTF

93 , writeODT

94 , writeDocx

95− , writeEPUB

95+ , writeEPUB2

96+ , writeEPUB3

96 , writeFB2

97 , writeFB2

97 , writeOrg

98 , writeOrg

98 , writeAsciiDoc

99 , writeAsciiDoc

@@ -199,7 +200,8 @@ writers = [

199 ,("json" , PureStringWriter $ \_ -> encodeJSON)

200 ,("json" , PureStringWriter $ \_ -> encodeJSON)

200 ,("docx" , IOByteStringWriter writeDocx)

201 ,("docx" , IOByteStringWriter writeDocx)

201 ,("odt" , IOByteStringWriter writeODT)

202 ,("odt" , IOByteStringWriter writeODT)

202− ,("epub" , IOByteStringWriter writeEPUB)

203+ ,("epub" , IOByteStringWriter writeEPUB2)

204+ ,("epub3" , IOByteStringWriter writeEPUB3)

203 ,("fb2" , IOStringWriter writeFB2)

205 ,("fb2" , IOStringWriter writeFB2)

204 ,("html" , PureStringWriter writeHtmlString)

206 ,("html" , PureStringWriter writeHtmlString)

205 ,("html5" , PureStringWriter $ \o ->

207 ,("html5" , PureStringWriter $ \o ->

src/Text/Pandoc/Templates.hs+1

@@ -94,6 +94,7 @@ getDefaultTemplate user writer = do

94 "json" -> return $ Right ""

95 "docx" -> return $ Right ""

96 "epub" -> return $ Right ""

97+ "epub3" -> return $ Right ""

97 "odt" -> getDefaultTemplate user "opendocument"

98 "odt" -> getDefaultTemplate user "opendocument"

98 "markdown_strict" -> getDefaultTemplate user "markdown"

99 "markdown_strict" -> getDefaultTemplate user "markdown"

99 "multimarkdown" -> getDefaultTemplate user "markdown"

100 "multimarkdown" -> getDefaultTemplate user "markdown"

src/Text/Pandoc/Writers/EPUB.hs+64−15

@@ -27,7 +27,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

28Conversion of 'Pandoc' documents to EPUB.

29-}

30−module Text.Pandoc.Writers.EPUB ( writeEPUB ) where

30+module Text.Pandoc.Writers.EPUB ( writeEPUB2, writeEPUB3 ) where

31import Data.IORef

32import Data.Maybe ( fromMaybe, isNothing )

33import Data.List ( isPrefixOf, intercalate )

@@ -38,6 +38,8 @@ import qualified Data.ByteString.Lazy as B

38import Text.Pandoc.UTF8 ( fromStringLazy )

39import Codec.Archive.Zip

40import Data.Time.Clock.POSIX

41+import Data.Time

42+import System.Locale

41import Text.Pandoc.Shared hiding ( Element )

43import Text.Pandoc.Shared hiding ( Element )

42import qualified Text.Pandoc.Shared as Shared

44import qualified Text.Pandoc.Shared as Shared

43import Text.Pandoc.Options

45import Text.Pandoc.Options

@@ -56,11 +58,20 @@ import Prelude hiding (catch)

56import Control.Exception (catch, SomeException)

58import Control.Exception (catch, SomeException)

57import Text.HTML.TagSoup

59import Text.HTML.TagSoup

61+data EPUBVersion = EPUB2 | EPUB3 deriving Eq

62+

63+writeEPUB2, writeEPUB3 :: WriterOptions -- ^ Writer options

64+ -> Pandoc -- ^ Document to convert

65+ -> IO B.ByteString

66+writeEPUB2 = writeEPUB EPUB2

67+writeEPUB3 = writeEPUB EPUB3

68+

59-- | Produce an EPUB file from a Pandoc document.

69-- | Produce an EPUB file from a Pandoc document.

70+writeEPUB :: EPUBVersion

60−writeEPUB :: WriterOptions -- ^ Writer options

71+ -> WriterOptions -- ^ Writer options

61 -> Pandoc -- ^ Document to convert

72 -> Pandoc -- ^ Document to convert

62 -> IO B.ByteString

73 -> IO B.ByteString

63−writeEPUB opts doc@(Pandoc meta _) = do

74+writeEPUB version opts doc@(Pandoc meta _) = do

64 epochtime <- floor `fmap` getPOSIXTime

75 epochtime <- floor `fmap` getPOSIXTime

65 let mkEntry path content = toEntry path epochtime content

76 let mkEntry path content = toEntry path epochtime content

66 let opts' = opts{ writerEmailObfuscation = NoObfuscation

77 let opts' = opts{ writerEmailObfuscation = NoObfuscation

@@ -163,17 +174,23 @@ writeEPUB opts doc@(Pandoc meta _) = do

163 let plainTitle = plainify $ docTitle meta

174 let plainTitle = plainify $ docTitle meta

164 let plainAuthors = map plainify $ docAuthors meta

175 let plainAuthors = map plainify $ docAuthors meta

165 let plainDate = maybe "" id $ normalizeDate $ stringify $ docDate meta

176 let plainDate = maybe "" id $ normalizeDate $ stringify $ docDate meta

177+ currentTime <- getCurrentTime

166 let contentsData = fromStringLazy $ ppTopElement $

178 let contentsData = fromStringLazy $ ppTopElement $

167− unode "package" ! [("version","2.0")

179+ unode "package" ! [("version", case version of

180+ EPUB2 -> "2.0"

181+ EPUB3 -> "3.0")

168 ,("xmlns","http://www.idpf.org/2007/opf")

182 ,("xmlns","http://www.idpf.org/2007/opf")

169 ,("unique-identifier","BookId")] $

183 ,("unique-identifier","BookId")] $

170− [ metadataElement (writerEPUBMetadata opts')

184+ [ metadataElement version (writerEPUBMetadata opts')

171− uuid lang plainTitle plainAuthors plainDate mbCoverImage

185+ uuid lang plainTitle plainAuthors plainDate currentTime mbCoverImage

172 , unode "manifest" $

186 , unode "manifest" $

173 [ unode "item" ! [("id","ncx"), ("href","toc.ncx")

187 [ unode "item" ! [("id","ncx"), ("href","toc.ncx")

174 ,("media-type","application/x-dtbncx+xml")] $ ()

188 ,("media-type","application/x-dtbncx+xml")] $ ()

175 , unode "item" ! [("id","style"), ("href","stylesheet.css")

189 , unode "item" ! [("id","style"), ("href","stylesheet.css")

176 ,("media-type","text/css")] $ ()

190 ,("media-type","text/css")] $ ()

191+ , unode "item" ! [("id","nav"), ("href","nav.xhtml")

192+ ,("properties","nav")

193+ ,("media-type","application/xhtml+xml")] $ ()

177 ] ++

194 ] ++

178 map chapterNode (cpgEntry ++ (tpEntry : chapterEntries)) ++

195 map chapterNode (cpgEntry ++ (tpEntry : chapterEntries)) ++

179 map pictureNode (cpicEntry ++ picEntries) ++

196 map pictureNode (cpicEntry ++ picEntries) ++

@@ -190,8 +207,9 @@ writeEPUB opts doc@(Pandoc meta _) = do

190 -- toc.ncx

207 -- toc.ncx

191 let secs = hierarchicalize blocks''

208 let secs = hierarchicalize blocks''

192

209

210+ let navPointNode :: (Int -> String -> String -> [Element] -> Element)

193− let navPointNode :: Shared.Element -> State Int Element

211+ -> Shared.Element -> State Int Element

194− navPointNode (Sec _ nums ident ils children) = do

212+ navPointNode formatter (Sec _ nums ident ils children) = do

195 n <- get

213 n <- get

196 modify (+1)

214 modify (+1)

197 let showNums :: [Int] -> String

215 let showNums :: [Int] -> String

@@ -206,14 +224,17 @@ writeEPUB opts doc@(Pandoc meta _) = do

206 let isSec (Sec lev _ _ _ _) = lev <= 3 -- only includes levels 1-3

224 let isSec (Sec lev _ _ _ _) = lev <= 3 -- only includes levels 1-3

207 isSec _ = False

225 isSec _ = False

208 let subsecs = filter isSec children

226 let subsecs = filter isSec children

209− subs <- mapM navPointNode subsecs

227+ subs <- mapM (navPointNode formatter) subsecs

228+ return $ formatter n tit src subs

229+ navPointNode _ (Blk _) = error "navPointNode encountered Blk"

230+

231+ let navMapFormatter :: Int -> String -> String -> [Element] -> Element

210− return $ unode "navPoint" !

232+ navMapFormatter n tit src subs = unode "navPoint" !

211 [("id", "navPoint-" ++ show n)

233 [("id", "navPoint-" ++ show n)

212 ,("playOrder", show n)] $

234 ,("playOrder", show n)] $

213 [ unode "navLabel" $ unode "text" tit

235 [ unode "navLabel" $ unode "text" tit

214 , unode "content" ! [("src", src)] $ ()

236 , unode "content" ! [("src", src)] $ ()

215 ] ++ subs

237 ] ++ subs

216− navPointNode (Blk _) = error "navPointNode encountered Blk"

217

238

218 let tpNode = unode "navPoint" ! [("id", "navPoint-0")] $

239 let tpNode = unode "navPoint" ! [("id", "navPoint-0")] $

219 [ unode "navLabel" $ unode "text" (plainify $ docTitle meta)

240 [ unode "navLabel" $ unode "text" (plainify $ docTitle meta)

@@ -236,10 +257,31 @@ writeEPUB opts doc@(Pandoc meta _) = do

236 Just _ -> [unode "meta" ! [("name","cover"),

257 Just _ -> [unode "meta" ! [("name","cover"),

237 ("content","cover-image")] $ ()]

258 ("content","cover-image")] $ ()]

238 , unode "docTitle" $ unode "text" $ plainTitle

259 , unode "docTitle" $ unode "text" $ plainTitle

260+ , unode "navMap" $

239− , unode "navMap" $ tpNode : evalState (mapM navPointNode secs) 1

261+ tpNode : evalState (mapM (navPointNode navMapFormatter) secs) 1

240 ]

262 ]

241 let tocEntry = mkEntry "toc.ncx" tocData

263 let tocEntry = mkEntry "toc.ncx" tocData

242

264

265+ let navXhtmlFormatter :: Int -> String -> String -> [Element] -> Element

266+ navXhtmlFormatter n tit src subs = unode "li" !

267+ [("id", "toc-li-" ++ show n)] $

268+ unode "a" ! [("href",src)] $

269+ unode "span" tit :

270+ case subs of

271+ [] -> []

272+ (_:_) -> [unode "ol" subs]

273+

274+ let navData = fromStringLazy $ ppTopElement $

275+ unode "html" ! [("xmlns","http://www.w3.org/1999/xhtml")

276+ ,("xmlns:epub","http://www.idpf.org/2007/ops")] $

277+ [ unode "head" $ unode "title" plainTitle

278+ , unode "body" $

279+ unode "nav" ! [("epub:type","toc")] $

280+ [ unode "h1" plainTitle

281+ , unode "ol" $ evalState (mapM (navPointNode navXhtmlFormatter) secs) 1]

282+ ]

283+ let navEntry = mkEntry "nav.xhtml" navData

284+

243 -- mimetype

285 -- mimetype

244 let mimetypeEntry = mkEntry "mimetype" $ fromStringLazy "application/epub+zip"

286 let mimetypeEntry = mkEntry "mimetype" $ fromStringLazy "application/epub+zip"

245

287

@@ -269,11 +311,13 @@ writeEPUB opts doc@(Pandoc meta _) = do

269 let archive = foldr addEntryToArchive emptyArchive

311 let archive = foldr addEntryToArchive emptyArchive

270 (mimetypeEntry : containerEntry : appleEntry : stylesheetEntry : tpEntry :

312 (mimetypeEntry : containerEntry : appleEntry : stylesheetEntry : tpEntry :

271 contentsEntry : tocEntry :

313 contentsEntry : tocEntry :

314+ ([navEntry | version == EPUB3] ++ picEntries ++ cpicEntry ++ cpgEntry ++

272− (picEntries ++ cpicEntry ++ cpgEntry ++ chapterEntries ++ fontEntries) )

315+ chapterEntries ++ fontEntries) )

273 return $ fromArchive archive

316 return $ fromArchive archive

274

317

275−metadataElement :: String -> UUID -> String -> String -> [String] -> String -> Maybe a -> Element

318+metadataElement :: EPUBVersion -> String -> UUID -> String -> String -> [String]

319+ -> String -> UTCTime -> Maybe a -> Element

276−metadataElement metadataXML uuid lang title authors date mbCoverImage =

320+metadataElement version metadataXML uuid lang title authors date currentTime mbCoverImage =

277 let userNodes = parseXML metadataXML

321 let userNodes = parseXML metadataXML

278 elt = unode "metadata" ! [("xmlns:dc","http://purl.org/dc/elements/1.1/")

322 elt = unode "metadata" ! [("xmlns:dc","http://purl.org/dc/elements/1.1/")

279 ,("xmlns:opf","http://www.idpf.org/2007/opf")] $

323 ,("xmlns:opf","http://www.idpf.org/2007/opf")] $

@@ -292,10 +336,15 @@ metadataElement metadataXML uuid lang title authors date mbCoverImage =

292 not (elt `contains` "identifier") ] ++

336 not (elt `contains` "identifier") ] ++

293 [ unode "dc:creator" ! [("opf:role","aut")] $ a | a <- authors ] ++

337 [ unode "dc:creator" ! [("opf:role","aut")] $ a | a <- authors ] ++

294 [ unode "dc:date" date | not (elt `contains` "date") ] ++

338 [ unode "dc:date" date | not (elt `contains` "date") ] ++

339+ [ unode "meta" ! [("property", "dcterms:modified")] $

340+ (showDateTimeISO8601 currentTime) | version == EPUB3 ] ++

295 [ unode "meta" ! [("name","cover"), ("content","cover-image")] $ () |

341 [ unode "meta" ! [("name","cover"), ("content","cover-image")] $ () |

296 not (isNothing mbCoverImage) ]

342 not (isNothing mbCoverImage) ]

297 in elt{ elContent = elContent elt ++ map Elem newNodes }

343 in elt{ elContent = elContent elt ++ map Elem newNodes }

298

344

345+showDateTimeISO8601 :: UTCTime -> String

346+showDateTimeISO8601 = formatTime defaultTimeLocale "%FT%TZ"

347+

299transformInlines :: HTMLMathMethod

348transformInlines :: HTMLMathMethod

300 -> FilePath

349 -> FilePath

301 -> IORef [(FilePath, FilePath)] -- ^ (oldpath, newpath) images

350 -> IORef [(FilePath, FilePath)] -- ^ (oldpath, newpath) images

src/pandoc.hs+1−1

@@ -83,7 +83,7 @@ wrapWords indent c = wrap' (c - indent) (c - indent)

83 else ", " ++ x ++ wrap' cols (remaining - (length x + 2)) xs

85isTextFormat :: String -> Bool

86−isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub"]

86+isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub","epub3"]

88-- | Data structure for command line options.

89data Opt = Opt