-{- $intro
- This package provides @pipes@ utilities for /text streams/ or /character streams/,
- realized as streams of 'Text' chunks. The individual chunks are uniformly /strict/,
- and thus you will generally want @Data.Text@ in scope. But the type
- @Producer Text m r@ ,as we are using it, is a sort of /pipes/ equivalent of the lazy @Text@ type.
-
- This particular module provides many functions equivalent in one way or another to
- the pure functions in
- <https://hackage.haskell.org/package/text-1.1.0.0/docs/Data-Text-Lazy.html Data.Text.Lazy>.
- They transform, divide, group and fold text streams. Though @Producer Text m r@
- is the type of \'effectful Text\', the functions in this module are \'pure\'
- in the sense that they are uniformly monad-independent.
- Simple /IO/ operations are defined in @Pipes.Text.IO@ -- as lazy IO @Text@
- operations are in @Data.Text.Lazy.IO@. Inter-operation with @ByteString@
- is provided in @Pipes.Text.Encoding@, which parallels @Data.Text.Lazy.Encoding@.
-
- The Text type exported by @Data.Text.Lazy@ is basically that of a lazy list of
- strict Text: the implementation is arranged so that the individual strict 'Text'
- chunks are kept to a reasonable size; the user is not aware of the divisions
- between the connected 'Text' chunks.
- So also here: the functions in this module are designed to operate on streams that
- are insensitive to text boundaries. This means that they may freely split
- text into smaller texts and /discard empty texts/. The objective, though, is
- that they should /never concatenate texts/ in order to provide strict upper
- bounds on memory usage.
-
- For example, to stream only the first three lines of 'stdin' to 'stdout' you
- might write:
-
-> import Pipes
-> import qualified Pipes.Text as Text
-> import qualified Pipes.Text.IO as Text
-> import Pipes.Group (takes')
-> import Lens.Family
->
-> main = runEffect $ takeLines 3 Text.stdin >-> Text.stdout
-> where
-> takeLines n = Text.unlines . takes' n . view Text.lines
-
- The above program will never bring more than one chunk of text (~ 32 KB) into
- memory, no matter how long the lines are.
-
--}
-{- $lenses
- As this example shows, one superficial difference from @Data.Text.Lazy@
- is that many of the operations, like 'lines', are \'lensified\'; this has a
- number of advantages (where it is possible); in particular it facilitates their
- use with 'Parser's of Text (in the general <http://hackage.haskell.org/package/pipes-parse-3.0.1/docs/Pipes-Parse-Tutorial.html pipes-parse>
- sense.) The disadvantage, famously, is that the messages you get for type errors can be
- a little alarming. The remarks that follow in this section are for non-lens adepts.
-
- Each lens exported here, e.g. 'lines', 'chunksOf' or 'splitAt', reduces to the
- intuitively corresponding function when used with @view@ or @(^.)@. Instead of
- writing:
-
- > splitAt 17 producer
-
- as we would with the Prelude or Text functions, we write
-
- > view (splitAt 17) producer
-
- or equivalently
-
- > producer ^. splitAt 17
-
- This may seem a little indirect, but note that many equivalents of
- @Text -> Text@ functions are exported here as 'Pipe's. Here too we recover the intuitively
- corresponding functions by prefixing them with @(>->)@. Thus something like
-
-> stripLines = Text.unlines . Group.maps (>-> Text.stripStart) . view Text.lines
-
- would drop the leading white space from each line.
-
- The lenses in this library are marked as /improper/; this just means that
- they don't admit all the operations of an ideal lens, but only /getting/ and /focusing/.
- Just for this reason, though, the magnificent complexities of the lens libraries
- are a distraction. The lens combinators to keep in mind, the ones that make sense for
- our lenses, are @view@ \/ @(^.)@), @over@ \/ @(%~)@ , and @zoom@.
-
- One need only keep in mind that if @l@ is a @Lens'_ a b@, then:
-
--}
-{- $view
- @view l@ is a function @a -> b@ . Thus @view l a@ (also written @a ^. l@ )
- is the corresponding @b@; as was said above, this function will be exactly the
- function you think it is, given its name. Thus to uppercase the first n characters
- of a Producer, leaving the rest the same, we could write:
-
-
- > upper n p = do p' <- p ^. Text.splitAt n >-> Text.toUpper
- > p'
--}
-{- $over
- @over l@ is a function @(b -> b) -> a -> a@. Thus, given a function that modifies
- @b@s, the lens lets us modify an @a@ by applying @f :: b -> b@ to
- the @b@ that we can \"see\" through the lens. So @over l f :: a -> a@
- (it can also be written @l %~ f@).
- For any particular @a@, then, @over l f a@ or @(l %~ f) a@ is a revised @a@.
- So above we might have written things like these:
-
- > stripLines = Text.lines %~ maps (>-> Text.stripStart)
- > stripLines = over Text.lines (maps (>-> Text.stripStart))
- > upper n = Text.splitAt n %~ (>-> Text.toUpper)
-
--}
-{- $zoom
- @zoom l@, finally, is a function from a @Parser b m r@
- to a @Parser a m r@ (or more generally a @StateT (Producer b m x) m r@).
- Its use is easiest to see with an decoding lens like 'utf8', which
- \"sees\" a Text producer hidden inside a ByteString producer:
- @drawChar@ is a Text parser, returning a @Maybe Char@, @zoom utf8 drawChar@ is
- a /ByteString/ parser, returning a @Maybe Char@. @drawAll@ is a Parser that returns
- a list of everything produced from a Producer, leaving only the return value; it would
- usually be unreasonable to use it. But @zoom (splitAt 17) drawAll@
- returns a list of Text chunks containing the first seventeen Chars, and returns the rest of
- the Text Producer for further parsing. Suppose that we want, inexplicably, to
- modify the casing of a Text Producer according to any instruction it might
- contain at the start. Then we might write something like this:
-
-> obey :: Monad m => Producer Text m b -> Producer Text m b
-> obey p = do (ts, p') <- lift $ runStateT (zoom (Text.splitAt 7) drawAll) p
-> let seven = T.concat ts
-> case T.toUpper seven of
-> "TOUPPER" -> p' >-> Text.toUpper
-> "TOLOWER" -> p' >-> Text.toLower
-> _ -> do yield seven
-> p'
-
-
-> >>> let doc = each ["toU","pperTh","is document.\n"]
-> >>> runEffect $ obey doc >-> Text.stdout
-> THIS DOCUMENT.
-
- The purpose of exporting lenses is the mental economy achieved with this three-way
- applicability. That one expression, e.g. @lines@ or @splitAt 17@ can have these
- three uses is no more surprising than that a pipe can act as a function modifying
- the output of a producer, namely by using @>->@ to its left: @producer >-> pipe@
- -- but can /also/ modify the inputs to a consumer by using @>->@ to its right:
- @pipe >-> consumer@
-
- The three functions, @view@ \/ @(^.)@, @over@ \/ @(%~)@ and @zoom@ are supplied by
- both <http://hackage.haskell.org/package/lens lens> and
- <http://hackage.haskell.org/package/lens-family lens-family> The use of 'zoom' is explained
- in <http://hackage.haskell.org/package/pipes-parse-3.0.1/docs/Pipes-Parse-Tutorial.html Pipes.Parse.Tutorial>
- and to some extent in the @Pipes.Text.Encoding@ module here.
projects / github / fretlink / text-pipes.git / blobdiff