1 {-#LANGUAGE RankNTypes#-}
4 module Pipes.Prelude.Text
6 -- * Simple line-based Text IO
18 import qualified System.IO as IO
19 import Control.Exception (throwIO, try)
20 import Foreign.C.Error (Errno(Errno), ePIPE)
21 import qualified GHC.IO.Exception as G
22 import Data.Text (Text)
23 import qualified Data.Text as T
24 import qualified Data.Text.IO as T
26 import qualified Pipes.Safe.Prelude as Safe
27 import Pipes.Safe (MonadSafe(..), runSafeT, runSafeP)
28 import Prelude hiding (readFile, writeFile)
31 Line-based operations are marked with a final \-@Ln@, like 'stdinLn', 'readFileLn', etc.
32 They are drop-in 'Text' replacements for the corresponding 'String' operations in
33 @Pipes.Prelude@ and @Pipes.Safe.Prelude@ - a final \-@Ln@ being added where necessary.
34 In using them, one is producing and consuming semantically significant individual texts,
35 understood as lines, just as one would produce or pipe 'Int's or 'Char's or anything else.
36 Thus, the standard materials from @Pipes@ and @Pipes.Prelude@ and
37 @Data.Text@ are all you need to work with them, and
38 you can use these operations without using any of the other modules in this package.
40 Thus, to take a trivial case, here we upper-case three lines from standard input and write
44 >>> import qualified Pipes.Prelude as P
45 >>> import qualified Pipes.Prelude.Text as Text
46 >>> import qualified Data.Text as T
47 >>> Text.runSafeT $ runEffect $ Text.stdinLn >-> P.take 3 >-> P.map T.toUpper >-> Text.writeFileLn "threelines.txt"
51 >>> :! cat "threelines.txt"
56 Here @runSafeT@ from @Pipes.Safe@ just makes sure to close any handles opened in its scope.
57 Otherwise the point of view is very much that of @Pipes.Prelude@, substituting @Text@ for @String@.
58 It would still be the same even if
59 we did something a bit more sophisticated, like run an ordinary attoparsec 'Text' parser on
60 each line, as is frequently desirable. Here we use
61 a minimal attoparsec number parser, @scientific@, on separate lines of standard input,
62 dropping bad parses with @P.concat@:
64 >>> import Data.Attoparsec.Text (parseOnly, scientific)
65 >>> P.toListM $ Text.stdinLn >-> P.takeWhile (/= "quit") >-> P.map (parseOnly scientific) >-> P.concat
73 The line-based operations are, however, subject to a number of caveats.
74 First, where they read from a handle, they will of course happily
75 accumulate indefinitely long lines. This is likely to be legitimate for input
76 typed in by a user, and for locally produced files of known characteristics, but
77 otherwise not. See the post on
78 <http://www.haskellforall.com/2013/09/perfect-streaming-using-pipes-bytestring.html perfect streaming>
79 to see why @pipes-bytestring@ and this package, outside this module, take a different approach.
80 Furthermore, the line-based operations,
81 like those in @Data.Text.IO@, use the system encoding (and @T.hGetLine@)
82 and thus are slower than the \'official\' route, which would use the very fast
83 bytestring IO operations from @Pipes.ByteString@ and
84 encoding and decoding functions in @Pipes.Text.Encoding@. Finally, the line-based
85 operations will generate text exceptions after the fashion of
86 @Data.Text.Encoding@, rather than returning the undigested bytes in the
87 style of @Pipes.Text.Encoding@.
92 {-| Read separate lines of 'Text' from 'IO.stdin' using 'T.getLine'
93 This function will accumulate indefinitely long strict 'Text's. See the caveats above.
95 Terminates on end of input
97 stdinLn :: MonadIO m => Producer' T.Text m ()
98 stdinLn = fromHandleLn IO.stdin
99 {-# INLINABLE stdinLn #-}
102 {-| Write 'Text' lines to 'IO.stdout' using 'putStrLn'
104 Unlike 'toHandle', 'stdoutLn' gracefully terminates on a broken output pipe
106 stdoutLn :: MonadIO m => Consumer' T.Text m ()
111 x <- liftIO $ try (T.putStrLn str)
113 Left (G.IOError { G.ioe_type = G.ResourceVanished
114 , G.ioe_errno = Just ioe })
117 Left e -> liftIO (throwIO e)
119 {-# INLINABLE stdoutLn #-}
121 {-| Write lines of 'Text' to 'IO.stdout'.
123 This does not handle a broken output pipe, but has a polymorphic return
126 stdoutLn' :: MonadIO m => Consumer' T.Text m r
127 stdoutLn' = for cat (\str -> liftIO (T.putStrLn str))
128 {-# INLINABLE stdoutLn' #-}
131 "p >-> stdoutLn'" forall p .
132 p >-> stdoutLn' = for p (\str -> liftIO (T.putStrLn str))
135 {-| Read separate lines of 'Text' from a 'IO.Handle' using 'T.hGetLine'.
136 This operation will accumulate indefinitely large strict texts. See the caveats above.
138 Terminates on end of input
140 fromHandleLn :: MonadIO m => IO.Handle -> Producer' Text m ()
141 fromHandleLn h = go where
142 getLine :: IO (Either G.IOException Text)
143 getLine = try (T.hGetLine h)
145 go = do txt <- liftIO getLine
148 Right y -> do yield y
150 {-# INLINABLE fromHandleLn #-}
152 -- to do: investigate differences from the above:
153 -- fromHandleLn :: MonadIO m => IO.Handle -> Producer' T.Text m ()
154 -- fromHandleLn h = go
157 -- eof <- liftIO $ IO.hIsEOF h
159 -- str <- liftIO $ T.hGetLine h
162 -- {-# INLINABLE fromHandleLn #-}
165 -- | Write separate lines of 'Text' to a 'IO.Handle' using 'T.hPutStrLn'
166 toHandleLn :: MonadIO m => IO.Handle -> Consumer' T.Text m r
167 toHandleLn handle = for cat (\str -> liftIO (T.hPutStrLn handle str))
168 {-# INLINABLE toHandleLn #-}
171 "p >-> toHandleLn handle" forall p handle .
172 p >-> toHandleLn handle = for p (\str -> liftIO (T.hPutStrLn handle str))
176 {-| Stream separate lines of text from a file. This operation will accumulate
177 indefinitely long strict text chunks. See the caveats above.
179 readFileLn :: MonadSafe m => FilePath -> Producer Text m ()
180 readFileLn file = Safe.withFile file IO.ReadMode fromHandleLn
181 {-# INLINE readFileLn #-}
185 {-| Write lines to a file, automatically opening and closing the file as
188 writeFileLn :: (MonadSafe m) => FilePath -> Consumer' Text m r
189 writeFileLn file = Safe.withFile file IO.WriteMode toHandleLn
190 {-# INLINABLE writeFileLn #-}