a36d202 · typeable/pandoc

git.iko.soy/typeable/pandoc

Text.Pandoc.Shared: add parameter to uniqueIdent, inlineListToIdentifier.

The parameter is Extensions. This allows these functions to be sensitive to the settings of `Ext_gfm_auto_identifiers` and `Ext_ascii_identifiers`. This allows us to use `uniqueIdent` in the CommonMark reader, replacing some custom code. It also means that `gfm_auto_identifiers` can now be used in all formats. Semantically, `gfm_auto_identifiers` is now a modifier of `auto_identifiers`; for identifiers to be set, `auto_identifiers` must be turned on, and then the type of identifier produced depends on `gfm_auto_identifiers` and `ascii_identifiers` are set. Closes #5057.

commita36d202e862f3fe0652e4f46cf7713120f50be28 authorJohn MacFarlane <jgm@berkeley.edu> dateSun 11 Nov 2018, 13:27 (8 years ago) parentca17ae5

15 files changed+109−83

compare with master

unifiedsplit

MANUAL.txt+24−9 src/Text/Pandoc/Extensions.hs+8−5 src/Text/Pandoc/Parsing.hs+1−1 src/Text/Pandoc/Readers/CommonMark.hs+8−27 src/Text/Pandoc/Readers/Docx.hs+7−3 src/Text/Pandoc/Readers/Odt/ContentReader.hs+7−2 src/Text/Pandoc/Shared.hs+37−22 src/Text/Pandoc/Writers/AsciiDoc.hs+1−1 src/Text/Pandoc/Writers/EPUB.hs+4−4 src/Text/Pandoc/Writers/Markdown.hs+1−1 src/Text/Pandoc/Writers/Muse.hs+1−1 src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+2−1 src/Text/Pandoc/Writers/RST.hs+2−1 src/Text/Pandoc/Writers/Texinfo.hs+2−1 test/command/4742.md+4−4

MANUAL.txt+24−9

@@ -1852,7 +1852,8 @@ output formats

18521852enabled by default in

18531853: `markdown`, `muse`

18541854

1855−The algorithm used to derive the identifier from the header text is:

1855+The default algorithm used to derive the identifier from the

1856+header text is:

18561857

18571858 - Remove all formatting, links, etc.

18581859 - Remove all footnotes.

@@ -1879,6 +1880,9 @@ same text; in this case, the first will get an identifier as described

18791880above; the second will get the same identifier with `-1` appended; the

18801881third with `-2`; and so on.

18811882

1883+(However, a different algorithm is used if

1884+`gfm_auto_identifiers` is enabled; see below.)

1885+

18821886These identifiers are used to provide link targets in the table of

18831887contents generated by the `--toc|--table-of-contents` option. They

18841888also make it easy to provide links from one section of a document to

@@ -1903,6 +1907,13 @@ Causes the identifiers produced by `auto_identifiers` to be pure ASCII.

19031907Accents are stripped off of accented Latin letters, and non-Latin

19041908letters are omitted.

19051909

1910+#### Extension: `gfm_auto_identifiers` ####

1911+

1912+Changes the algorithm used by `auto_identifiers` to conform to

1913+GitHub's method. Spaces are converted to dashes (`-`),

1914+uppercase characters to lowercase characters, and punctuation

1915+characters other than `-` and `_` are removed.

1916+

19061917Math Input

19071918----------

19081919

@@ -4189,10 +4200,11 @@ variants are supported:

41894200 `shortcut_reference_links`, `spaced_reference_links`.

41904201

41914202`markdown_github` (deprecated GitHub-Flavored Markdown)

4192−: `pipe_tables`, `raw_html`, `fenced_code_blocks`, `gfm_auto_identifiers`,

4193− `ascii_identifiers`, `backtick_code_blocks`, `autolink_bare_uris`,

4194− `space_in_atx_header`, `intraword_underscores`, `strikeout`,

4195− `emoji`, `shortcut_reference_links`, `angle_brackets_escapable`,

4203+: `pipe_tables`, `raw_html`, `fenced_code_blocks`, `auto_identifiers`,

4204+ `gfm_auto_identifiers`, `backtick_code_blocks`,

4205+ `autolink_bare_uris`, `space_in_atx_header`,

4206+ `intraword_underscores`, `strikeout`, `emoji`,

4207+ `shortcut_reference_links`, `angle_brackets_escapable`,

41964208 `lists_without_preceding_blankline`.

41974209

41984210`markdown_mmd` (MultiMarkdown)

@@ -4219,10 +4231,13 @@ individually disabled.

42194231Also, `raw_tex` only affects `gfm` output, not input.

42204232

42214233`gfm` (GitHub-Flavored Markdown)

4222−: `pipe_tables`, `raw_html`, `fenced_code_blocks`, `gfm_auto_identifiers`,

4223− `backtick_code_blocks`, `autolink_bare_uris`,

4224− `intraword_underscores`, `strikeout`, `hard_line_breaks`, `emoji`,

4225− `shortcut_reference_links`, `angle_brackets_escapable`.

4234+: `pipe_tables`, `raw_html`, `fenced_code_blocks`, `auto_identifiers`,

4235+ `gfm_auto_identifiers`, `backtick_code_blocks`,

4236+ `autolink_bare_uris`, `space_in_atx_header`,

4237+ `intraword_underscores`, `strikeout`, `emoji`,

4238+ `shortcut_reference_links`, `angle_brackets_escapable`,

4239+ `lists_without_preceding_blankline`.

4240+

42264241

42274242Producing slide shows with pandoc

42284243=================================

src/Text/Pandoc/Extensions.hs+8−5

@@ -56,7 +56,7 @@ import Data.Bits (clearBit, setBit, testBit, (.|.))

5656import Data.Data (Data)

5757import Data.Typeable (Typeable)

5858import GHC.Generics (Generic)

59−import Text.Pandoc.Shared (safeRead)

59+import Safe (readMay)

6060import Text.Parsec

6161

6262#ifdef DERIVE_JSON_VIA_TH

@@ -96,7 +96,8 @@ data Extension =

9696 | Ext_all_symbols_escapable -- ^ Make all non-alphanumerics escapable

9797 | Ext_amuse -- ^ Enable Text::Amuse extensions to Emacs Muse markup

9898 | Ext_angle_brackets_escapable -- ^ Make < and > escapable

99− | Ext_ascii_identifiers -- ^ ascii-only identifiers for headers

99+ | Ext_ascii_identifiers -- ^ ascii-only identifiers for headers;

100+ -- presupposes Ext_auto_identifiers

100101 | Ext_auto_identifiers -- ^ Automatic identifiers for headers

101102 | Ext_autolink_bare_uris -- ^ Make all absolute URIs into links

102103 | Ext_backtick_code_blocks -- ^ GitHub style ``` code blocks

@@ -123,8 +124,9 @@ data Extension =

123124 | Ext_fenced_divs -- ^ Allow fenced div syntax :::

124125 | Ext_footnotes -- ^ Pandoc\/PHP\/MMD style footnotes

125126 | Ext_four_space_rule -- ^ Require 4-space indent for list contents

126− | Ext_gfm_auto_identifiers -- ^ Automatic identifiers for headers, using

127− -- GitHub's method for generating identifiers

127+ | Ext_gfm_auto_identifiers -- ^ Use GitHub's method for generating

128+ -- header identifiers; presupposes

129+ -- Ext_auto_identifiers

128130 | Ext_grid_tables -- ^ Grid tables (pandoc, reST)

129131 | Ext_hard_line_breaks -- ^ All newlines become hard line breaks

130132 | Ext_header_attributes -- ^ Explicit header attributes {#id .class k=v}

@@ -265,6 +267,7 @@ githubMarkdownExtensions = extensionsFromList

265267 , Ext_pipe_tables

266268 , Ext_raw_html

267269 , Ext_fenced_code_blocks

270+ , Ext_auto_identifiers

268271 , Ext_gfm_auto_identifiers

269272 , Ext_backtick_code_blocks

270273 , Ext_autolink_bare_uris

@@ -384,7 +387,7 @@ parseFormatSpec = parse formatSpec ""

384387 extMod = do

385388 polarity <- oneOf "-+"

386389 name <- many $ noneOf "-+"

387− ext <- case safeRead ("Ext_" ++ name) of

390+ ext <- case readMay ("Ext_" ++ name) of

388391 Just n -> return n

389392 Nothing

390393 | name == "lhs" -> return Ext_literate_haskell

src/Text/Pandoc/Parsing.hs+1−1

@@ -1291,7 +1291,7 @@ registerHeader (ident,classes,kvs) header' = do

12911291 let insert' = M.insertWith (\_new old -> old)

12921292 if null ident && Ext_auto_identifiers `extensionEnabled` exts

12931293 then do

1294− let id' = uniqueIdent (B.toList header') ids

1294+ let id' = uniqueIdent exts (B.toList header') ids

12951295 let id'' = if Ext_ascii_identifiers `extensionEnabled` exts

12961296 then mapMaybe toAsciiChar id'

12971297 else id'

src/Text/Pandoc/Readers/CommonMark.hs+8−27

@@ -36,23 +36,20 @@ where

3636import Prelude

3737import CMarkGFM

3838import Control.Monad.State

39−import Data.Char (isAlphaNum, isSpace, toLower)

4039import Data.List (groupBy)

41−import qualified Data.Map as Map

42−import Data.Maybe (mapMaybe)

40+import qualified Data.Set as Set

4341import Data.Text (Text, unpack)

44−import Text.Pandoc.Asciify (toAsciiChar)

4542import Text.Pandoc.Class (PandocMonad)

4643import Text.Pandoc.Definition

4744import Text.Pandoc.Emoji (emojiToInline)

4845import Text.Pandoc.Options

49−import Text.Pandoc.Shared (stringify)

46+import Text.Pandoc.Shared (uniqueIdent)

5047import Text.Pandoc.Walk (walkM)

5148

5249-- | Parse a CommonMark formatted string into a 'Pandoc' structure.

5350readCommonMark :: PandocMonad m => ReaderOptions -> Text -> m Pandoc

5451readCommonMark opts s = return $

55− (if isEnabled Ext_gfm_auto_identifiers opts

52+ (if isEnabled Ext_auto_identifiers opts

5653 then addHeaderIdentifiers opts

5754 else id) $

5855 nodeToPandoc opts $ commonmarkToNode opts' exts s

@@ -78,30 +75,14 @@ convertEmojis s =

7875addHeaderIdentifiers :: ReaderOptions -> Pandoc -> Pandoc

7976addHeaderIdentifiers opts doc = evalState (walkM (addHeaderId opts) doc) mempty

8077

81−addHeaderId :: ReaderOptions -> Block -> State (Map.Map String Int) Block

78+addHeaderId :: ReaderOptions -> Block -> State (Set.Set String) Block

8279addHeaderId opts (Header lev (_,classes,kvs) ils) = do

83− idmap <- get

84− let ident = toIdent opts ils

85− ident' <- case Map.lookup ident idmap of

86− Nothing -> do

87− put (Map.insert ident 1 idmap)

88− return ident

89− Just i -> do

90− put (Map.adjust (+ 1) ident idmap)

91− return (ident ++ "-" ++ show i)

92− return $ Header lev (ident',classes,kvs) ils

80+ ids <- get

81+ let ident = uniqueIdent (readerExtensions opts) ils ids

82+ modify (Set.insert ident)

83+ return $ Header lev (ident,classes,kvs) ils

9384addHeaderId _ x = return x

9485

95−toIdent :: ReaderOptions -> [Inline] -> String

96−toIdent opts =

97− filterAscii . filterPunct . spaceToDash . map toLower. stringify

98− where

99− filterAscii = if isEnabled Ext_ascii_identifiers opts

100− then mapMaybe toAsciiChar

101− else id

102− filterPunct = filter (\c -> isAlphaNum c || c == '_' || c == '-')

103− spaceToDash = map (\c -> if isSpace c then '-' else c)

104−

10586nodeToPandoc :: ReaderOptions -> Node -> Pandoc

10687nodeToPandoc opts (Node _ DOCUMENT nodes) =

10788 Pandoc nullMeta $ foldr (addBlock opts) [] nodes

src/Text/Pandoc/Readers/Docx.hs+7−3

@@ -442,9 +442,11 @@ parPartToInlines' (BookMark _ anchor) =

442442 (modify $ \s -> s { docxAnchorMap = M.insert anchor prevAnchor anchorMap})

443443 return mempty

444444 Nothing -> do

445+ exts <- readerExtensions <$> asks docxOptions

445446 let newAnchor =

446447 if not inHdrBool && anchor `elem` M.elems anchorMap

447− then uniqueIdent [Str anchor] (Set.fromList $ M.elems anchorMap)

448+ then uniqueIdent exts [Str anchor]

449+ (Set.fromList $ M.elems anchorMap)

448450 else anchor

449451 unless inHdrBool

450452 (modify $ \s -> s { docxAnchorMap = M.insert anchor newAnchor anchorMap})

@@ -487,8 +489,9 @@ makeHeaderAnchor' (Header n (ident, classes, kvs) ils)

487489 | (c:_) <- filter isAnchorSpan ils

488490 , (Span (anchIdent, ["anchor"], _) cIls) <- c = do

489491 hdrIDMap <- gets docxAnchorMap

492+ exts <- readerExtensions <$> asks docxOptions

490493 let newIdent = if null ident

491− then uniqueIdent ils (Set.fromList $ M.elems hdrIDMap)

494+ then uniqueIdent exts ils (Set.fromList $ M.elems hdrIDMap)

492495 else ident

493496 newIls = concatMap f ils where f il | il == c = cIls

494497 | otherwise = [il]

@@ -499,8 +502,9 @@ makeHeaderAnchor' (Header n (ident, classes, kvs) ils)

499502makeHeaderAnchor' (Header n (ident, classes, kvs) ils) =

500503 do

501504 hdrIDMap <- gets docxAnchorMap

505+ exts <- readerExtensions <$> asks docxOptions

502506 let newIdent = if null ident

503− then uniqueIdent ils (Set.fromList $ M.elems hdrIDMap)

507+ then uniqueIdent exts ils (Set.fromList $ M.elems hdrIDMap)

504508 else ident

505509 modify $ \s -> s {docxAnchorMap = M.insert newIdent newIdent hdrIDMap}

506510 return $ Header n (newIdent, classes, kvs) ils

src/Text/Pandoc/Readers/Odt/ContentReader.hs+7−2

@@ -54,6 +54,7 @@ import qualified Text.XML.Light as XML

5454import Text.Pandoc.Builder

5555import Text.Pandoc.MediaBag (MediaBag, insertMedia)

5656import Text.Pandoc.Shared

57+import Text.Pandoc.Extensions (extensionsFromList, Extension(..))

5758

5859import Text.Pandoc.Readers.Odt.Base

5960import Text.Pandoc.Readers.Odt.Namespaces

@@ -253,7 +254,9 @@ getPrettyAnchor = proc (baseIdent, uglyAnchor) -> do

253254getHeaderAnchor :: OdtReaderSafe Inlines Anchor

254255getHeaderAnchor = proc title -> do

255256 state <- getExtraState -< ()

256− let anchor = uniqueIdent (toList title) (Set.fromList $ usedAnchors state)

257+ let exts = extensionsFromList [Ext_auto_identifiers]

258+ let anchor = uniqueIdent exts (toList title)

259+ (Set.fromList $ usedAnchors state)

257260 modifyExtraState (putPrettyAnchor anchor anchor) -<< anchor

258261

259262

@@ -768,6 +771,7 @@ read_maybe_nested_img_frame = matchingElement NsDraw "frame"

768771read_frame :: OdtReaderSafe Inlines Inlines

769772read_frame =

770773 proc blocks -> do

774+ let exts = extensionsFromList [Ext_auto_identifiers]

771775 w <- ( findAttr' NsSVG "width" ) -< ()

772776 h <- ( findAttr' NsSVG "height" ) -< ()

773777 titleNodes <- ( matchChildContent' [ read_frame_title ] ) -< blocks

@@ -776,7 +780,8 @@ read_frame =

776780 _ <- updateMediaWithResource -< resource

777781 alt <- (matchChildContent [] read_plain_text) -< blocks

778782 arr (uncurry4 imageWith ) -<

779− (image_attributes w h, src, inlineListToIdentifier (toList titleNodes), alt)

783+ (image_attributes w h, src,

784+ inlineListToIdentifier exts (toList titleNodes), alt)

780785

781786image_attributes :: Maybe String -> Maybe String -> Attr

782787image_attributes x y =

src/Text/Pandoc/Shared.hs+37−22

@@ -116,8 +116,7 @@ import Control.Monad (MonadPlus (..), msum, unless)

116116import qualified Control.Monad.State.Strict as S

117117import qualified Data.ByteString.Lazy as BL

118118import qualified Data.Bifunctor as Bifunctor

119−import Data.Char (isAlpha, isDigit, isLetter, isLower, isSpace, isUpper,

120− toLower)

119+import Data.Char (isAlpha, isLower, isSpace, isUpper, toLower, isAlphaNum)

121120import Data.Data (Data, Typeable)

122121import Data.List (find, intercalate, intersperse, stripPrefix)

123122import qualified Data.Map as M

@@ -137,7 +136,9 @@ import Text.HTML.TagSoup (RenderOptions (..), Tag (..), renderOptions,

137136import Text.Pandoc.Builder (Blocks, Inlines, ToMetaValue (..))

138137import qualified Text.Pandoc.Builder as B

139138import Data.Time

139+import Text.Pandoc.Asciify (toAsciiChar)

140140import Text.Pandoc.Definition

141+import Text.Pandoc.Extensions (Extensions, Extension(..), extensionEnabled)

141142import Text.Pandoc.Generic (bottomUp)

142143import Text.Pandoc.Pretty (charWidth)

143144import Text.Pandoc.Walk

@@ -483,18 +484,29 @@ instance Walkable Block Element where

483484 query f (Blk x) = query f x

484485 query f (Sec _ _ _ ils elts) = query f ils `mappend` query f elts

485486

486−

487487-- | Convert Pandoc inline list to plain text identifier. HTML

488488-- identifiers must start with a letter, and may contain only

489489-- letters, digits, and the characters _-.

490−inlineListToIdentifier :: [Inline] -> String

491−inlineListToIdentifier =

492− dropWhile (not . isAlpha) . intercalate "-" . words .

493− map (nbspToSp . toLower) .

494− filter (\c -> isLetter c || isDigit c || c `elem` "_-. ") .

495− stringify

496− where nbspToSp '\160' = ' '

497− nbspToSp x = x

490+inlineListToIdentifier :: Extensions -> [Inline] -> String

491+inlineListToIdentifier exts =

492+ dropNonLetter . filterAscii . toIdent . stringify

493+ where

494+ dropNonLetter

495+ | extensionEnabled Ext_gfm_auto_identifiers exts = id

496+ | otherwise = dropWhile (not . isAlpha)

497+ filterAscii

498+ | extensionEnabled Ext_ascii_identifiers exts

499+ = mapMaybe toAsciiChar

500+ | otherwise = id

501+ toIdent

502+ | extensionEnabled Ext_gfm_auto_identifiers exts =

503+ filterPunct . spaceToDash . map toLower

504+ | otherwise = intercalate "-" . words . filterPunct . map toLower

505+ filterPunct = filter (\c -> isSpace c || isAlphaNum c || isAllowedPunct c)

506+ isAllowedPunct c

507+ | extensionEnabled Ext_gfm_auto_identifiers exts = c == '_' || c == '-'

508+ | otherwise = c == '_' || c == '-' || c == '.'

509+ spaceToDash = map (\c -> if isSpace c then '-' else c)

498510

499511-- | Convert list of Pandoc blocks into (hierarchical) list of Elements

500512hierarchicalize :: [Block] -> [Element]

@@ -530,17 +542,20 @@ headerLtEq _ _ = False

530542

531543-- | Generate a unique identifier from a list of inlines.

532544-- Second argument is a list of already used identifiers.

533−uniqueIdent :: [Inline] -> Set.Set String -> String

534−uniqueIdent title' usedIdents

535− = let baseIdent = case inlineListToIdentifier title' of

536− "" -> "section"

537− x -> x

538− numIdent n = baseIdent ++ "-" ++ show n

539− in if baseIdent `Set.member` usedIdents

540− then case find (\x -> not $ numIdent x `Set.member` usedIdents) ([1..60000] :: [Int]) of

541− Just x -> numIdent x

542− Nothing -> baseIdent -- if we have more than 60,000, allow repeats

543− else baseIdent

545+uniqueIdent :: Extensions -> [Inline] -> Set.Set String -> String

546+uniqueIdent exts title' usedIdents =

547+ if baseIdent `Set.member` usedIdents

548+ then case find (\x -> not $ numIdent x `Set.member` usedIdents)

549+ ([1..60000] :: [Int]) of

550+ Just x -> numIdent x

551+ Nothing -> baseIdent

552+ -- if we have more than 60,000, allow repeats

553+ else baseIdent

554+ where

555+ baseIdent = case inlineListToIdentifier exts title' of

556+ "" -> "section"

557+ x -> x

558+ numIdent n = baseIdent ++ "-" ++ show n

544559

545560-- | True if block is a Header block.

546561isHeaderBlock :: Block -> Bool

src/Text/Pandoc/Writers/AsciiDoc.hs+1−1

@@ -170,7 +170,7 @@ blockToAsciiDoc _ HorizontalRule =

170170blockToAsciiDoc opts (Header level (ident,_,_) inlines) = do

171171 contents <- inlineListToAsciiDoc opts inlines

172172 ids <- gets autoIds

173− let autoId = uniqueIdent inlines ids

173+ let autoId = uniqueIdent (writerExtensions opts) inlines ids

174174 modify $ \st -> st{ autoIds = Set.insert autoId ids }

175175 let identifier = if null ident ||

176176 (isEnabled Ext_auto_identifiers opts && ident == autoId)

src/Text/Pandoc/Writers/EPUB.hs+4−4

@@ -495,7 +495,7 @@ pandocToEPUB version opts doc@(Pandoc meta _) = do

495495 -- body pages

496496

497497 -- add level 1 header to beginning if none there

498− let blocks' = addIdentifiers

498+ let blocks' = addIdentifiers opts

499499 $ case blocks of

500500 (Header 1 _ _ : _) -> blocks

501501 _ -> Header 1 ("",["unnumbered"],[])

@@ -1056,12 +1056,12 @@ showChapter :: Int -> String

10561056showChapter = printf "ch%03d.xhtml"

10571057

10581058-- Add identifiers to any headers without them.

1059−addIdentifiers :: [Block] -> [Block]

1060−addIdentifiers bs = evalState (mapM go bs) Set.empty

1059+addIdentifiers :: WriterOptions -> [Block] -> [Block]

1060+addIdentifiers opts bs = evalState (mapM go bs) Set.empty

10611061 where go (Header n (ident,classes,kvs) ils) = do

10621062 ids <- get

10631063 let ident' = if null ident

1064− then uniqueIdent ils ids

1064+ then uniqueIdent (writerExtensions opts) ils ids

10651065 else ident

10661066 modify $ Set.insert ident'

10671067 return $ Header n (ident',classes,kvs) ils

src/Text/Pandoc/Writers/Markdown.hs+1−1

@@ -508,7 +508,7 @@ blockToMarkdown' opts (Header level attr inlines) = do

508508 -- we calculate the id that would be used by auto_identifiers

509509 -- so we know whether to print an explicit identifier

510510 ids <- gets stIds

511− let autoId = uniqueIdent inlines ids

511+ let autoId = uniqueIdent (writerExtensions opts) inlines ids

512512 modify $ \st -> st{ stIds = Set.insert autoId ids }

513513 let attr' = case attr of

514514 ("",[],[]) -> empty

src/Text/Pandoc/Writers/Muse.hs+1−1

@@ -270,7 +270,7 @@ blockToMuse (Header level (ident,_,_) inlines) = do

270270 topLevel <- asks envTopLevel

271271 contents <- local (\env -> env { envOneLine = True }) $ inlineListToMuse' inlines

272272 ids <- gets stIds

273− let autoId = uniqueIdent inlines ids

273+ let autoId = uniqueIdent (writerExtensions opts) inlines ids

274274 modify $ \st -> st{ stIds = Set.insert autoId ids }

275275

276276 let attr' = if null ident || (isEnabled Ext_auto_identifiers opts && ident == autoId)

src/Text/Pandoc/Writers/Powerpoint/Presentation.hs+2−1

@@ -735,6 +735,7 @@ makeEndNotesSlideBlocks :: Pres [Block]

735735makeEndNotesSlideBlocks = do

736736 noteIds <- gets stNoteIds

737737 slideLevel <- asks envSlideLevel

738+ exts <- writerExtensions <$> asks envOpts

738739 meta <- asks envMetadata

739740 -- Get identifiers so we can give the notes section a unique ident.

740741 anchorSet <- M.keysSet <$> gets stAnchorMap

@@ -743,7 +744,7 @@ makeEndNotesSlideBlocks = do

743744 else let title = case lookupMetaInlines "notes-title" meta of

744745 [] -> [Str "Notes"]

745746 ls -> ls

746− ident = Shared.uniqueIdent title anchorSet

747+ ident = Shared.uniqueIdent exts title anchorSet

747748 hdr = Header slideLevel (ident, [], []) title

748749 blks = concatMap (\(n, bs) -> makeNoteEntry n bs) $

749750 M.toList noteIds

src/Text/Pandoc/Writers/RST.hs+2−1

@@ -260,7 +260,8 @@ blockToRST (Header level (name,classes,_) inlines) = do

260260 contents <- inlineListToRST inlines

261261 -- we calculate the id that would be used by auto_identifiers

262262 -- so we know whether to print an explicit identifier

263− let autoId = uniqueIdent inlines mempty

263+ opts <- gets stOptions

264+ let autoId = uniqueIdent (writerExtensions opts) inlines mempty

264265 isTopLevel <- gets stTopLevel

265266 if isTopLevel

266267 then do

src/Text/Pandoc/Writers/Texinfo.hs+2−1

@@ -231,7 +231,8 @@ blockToTexinfo (Header level _ lst)

231231 node <- inlineListForNode lst

232232 txt <- inlineListToTexinfo lst

233233 idsUsed <- gets stIdentifiers

234− let id' = uniqueIdent lst idsUsed

234+ opts <- gets stOptions

235+ let id' = uniqueIdent (writerExtensions opts) lst idsUsed

235236 modify $ \st -> st{ stIdentifiers = Set.insert id' idsUsed }

236237 sec <- seccmd level

237238 return $ if (level > 0) && (level <= 4)

test/command/4742.md+4−4

@@ -2,24 +2,24 @@ Check that the commonmark reader handles the `ascii_identifiers`

22extension properly.

44```

5−% pandoc -f commonmark+gfm_auto_identifiers+ascii_identifiers -t native

5+% pandoc -f commonmark+auto_identifiers+gfm_auto_identifiers+ascii_identifiers -t native

66# non ascii ⚠️ räksmörgås

77^D

88[Header 1 ("non-ascii--raksmorgas",[],[]) [Str "non",Space,Str "ascii",Space,Str "\9888\65039",Space,Str "r\228ksm\246rg\229s"]]

99```

1010

1111```

12−% pandoc -f commonmark+gfm_auto_identifiers-ascii_identifiers -t native

12+% pandoc -f commonmark+auto_identifiers+gfm_auto_identifiers-ascii_identifiers -t native

1313# non ascii ⚠️ räksmörgås

1414^D

1515[Header 1 ("non-ascii-\65039-r\228ksm\246rg\229s",[],[]) [Str "non",Space,Str "ascii",Space,Str "\9888\65039",Space,Str "r\228ksm\246rg\229s"]]

1616```

1717

18−`gfm` should have `ascii_identifiers` enabled by default.

18+`gfm` should have `ascii_identifiers` disabled by default.

1919

2020```

2121% pandoc -f gfm -t native

2222# non ascii ⚠️ räksmörgås

2323^D

24−[Header 1 ("non-ascii--raksmorgas",[],[]) [Str "non",Space,Str "ascii",Space,Str "\9888\65039",Space,Str "r\228ksm\246rg\229s"]]

24+[Header 1 ("non-ascii-\65039-r\228ksm\246rg\229s",[],[]) [Str "non",Space,Str "ascii",Space,Str "\9888\65039",Space,Str "r\228ksm\246rg\229s"]]

2525```