typed-process
Run external processes, with strong typing of streams
https://github.com/fpco/typed-process
Version on this page: | 0.2.11.1 |
LTS Haskell 22.33: | 0.2.11.1 |
Stackage Nightly 2024-09-07: | 0.2.12.0 |
Latest on Hackage: | 0.2.12.0 |
typed-process-0.2.11.1@sha256:79e8555666f27a3e4e67fedf430000631d95c937147190b3f10817daca1874b0,2198
Module documentation for 0.2.11.1
- System
- System.Process
typed-process
API level documentation (Haddocks) may be found on Stackage.
This library provides the ability to launch and interact with external processes. It wraps around the process library, and intends to improve upon it by:
- Using type variables to represent the standard streams, making them easier to manipulate
- Use proper concurrency (e.g., the async library) in place of the weird lazy I/O tricks for such things as consuming output streams
- Allow for more complex concurrency by providing STM-based functions
- Using binary I/O correctly
- Providing a more composable API, designed to be easy to use for both simple and complex use cases
NOTE It’s highly recommended that you compile any program using this
library with the multi-threaded runtime, usually by adding ghc-options: -threaded
to your executable stanza in your cabal or package.yaml
file. The
single-threaded runtime necessitates some inefficient polling to be used under
the surface.
Synopsis
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.IO (hPutStr, hClose)
import System.Process.Typed
import qualified Data.ByteString.Lazy as L
import qualified Data.ByteString.Lazy.Char8 as L8
import Control.Concurrent.STM (atomically)
import Control.Exception (throwIO)
main :: IO ()
main = do
-- Run a process, print its exit code
runProcess "true" >>= print
runProcess "false" >>= print
-- Check that the exit code is a success
runProcess_ "true"
-- This will throw an exception: runProcess_ "false"
-- Capture output and error
(dateOut, dateErr) <- readProcess_ "date"
print (dateOut, dateErr)
-- Use shell commands
(dateOut2, dateErr2) <- readProcess_ "date >&2"
print (dateOut2, dateErr2)
-- Interact with a process
let catConfig = setStdin createPipe
$ setStdout byteStringOutput
$ proc "cat" ["/etc/hosts", "-", "/etc/group"]
withProcessWait_ catConfig $ \p -> do
hPutStr (getStdin p) "\n\nHELLO\n"
hPutStr (getStdin p) "WORLD\n\n\n"
hClose (getStdin p)
atomically (getStdout p) >>= L8.putStr
Types
The two primary types in this package are ProcessConfig
and
Process
. ProcessConfig
gives a specification for how to run a
process (e.g., the command to run, working directory, environment
variables) and how to deal with the three standard streams: input,
output, and error. You use one of the functions in this package for
launching a process to turn a ProcessConfig
into a Process
, which
represents an actual running system process.
The easiest way to create a ProcessConfig
is using the IsString
instance and OverloadedStrings
. For example, to run the date
command, we can do the following. (NOTE: The type signatures used here
are simply to spell things out, they are not needed.)
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = do
let dateConfig :: ProcessConfig () () ()
dateConfig = proc "date" []
-- alternatively: `shell "date"` or just "date"
process <- startProcess dateConfig
exitCode <- waitExitCode (process :: Process () () ())
print exitCode
stopProcess process
This shows the general workflow: use startProcess
to launch a
Process
from a ProcessConfig
, interact with it (such as
waitExitCode
to wait for the process to exit), and then clean up
resources with stopProcess
. (We’ll get to those () () ()
type
parameters in the next section.)
Instead of explicitly dealing with startProcess
and stopProcess
,
it’s recommended to instead use withProcessWait
, which uses the bracket
pattern and is exception safe:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = withProcessWait "date" $ \process -> do
exitCode <- waitExitCode (process :: Process () () ())
print exitCode
But this pattern of running a process, waiting for it to exit, and getting its exit code is very common, so it has a helper function of its own:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = do
exitCode <- runProcess "date"
print exitCode
We’ll discuss some functions which automatically check the exit code below.
Type parameters
Both ProcessConfig
and Process
take three type parameters:
the types of the standard input, output, and error streams for the
process. As you saw above, our default is ()
for each, and our
default behavior is to inherit the streams from the parent
process. This is why, when you run the previous programs, the date
program’s output goes directly to your console.
We can override these defaults in a number of ways. Perhaps the easiest is to simply close the stream for the child so it cannot use it at all.
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = do
let dateConfig :: ProcessConfig () () ()
dateConfig = setStdin closed
$ setStdout closed
$ setStderr closed
"date"
exitCode <- runProcess dateConfig
print exitCode
A few things to note:
- The type parameter is still
()
, since there’s no data to return. We’ll see some more interesting cases later. - This process now returns an
ExitFailure 1
, since it tries to write to a closedstdout
file descriptor.
Using proc
and shell
Using the OverloadedStrings
approach works nicely for some cases,
but we’ll often want more control over things. There are two smart
constructors available: proc
takes a command and list of arguments,
and shell
takes a single string which will be passed directly to the
system’s shell.
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = do
-- Command and arguments
runProcess (proc "cat" ["/etc/hosts"]) >>= print
-- Shell
runProcess (shell "cat /etc/hosts >&2 && false") >>= print
The behavior of the OverloadedStrings
approach we’ve used until now
is actually based on these two smart constructors. If you provide it a
string without any spaces (like "date"
), it will use proc
without
any arguments, e.g. fromString "date" = proc "date" []
. If there are
any spaces in the string, it will use shell
.
EXERCISE: Rewrite the previous example to not use the shell
constructor.
Checking the exit code
We’ve done a lot of printing of exit codes. In many cases, we don’t actually want to look at the exit code, but instead just throw an exception if the process failed. Fortunately, we have such an exit-code-checking function.
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = runProcess_ "date"
By adding the _
at the end of runProcess
, we’re now automatically
checking the exit code and throwing an exception if it returns
anything but success. Want to see it in action?
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = runProcess_ "false"
Under the surface, this function is using the checkExitCode
function. We can do this more explicitly if desired:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = withProcessWait "false" checkExitCode
Reading from a process
Sending all output to the parent process’s handles is sometimes
desired, but often we’d rather just capture that output. The easiest
way to do that is to capture it in memory as a lazy
ByteString
. Fortunately, we have a helper readProcess
function for
that:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import System.Exit (ExitCode)
import Data.ByteString.Lazy (ByteString)
main :: IO ()
main = do
(exitCode, out, err) <- readProcess "date"
print (exitCode :: ExitCode)
print (out :: ByteString)
print (err :: ByteString)
One thing to point out is that, even though this is a lazy
ByteString
, it is not using any lazy I/O. When readProcess
exits,
the output has been fully generated, and is resident in memory. We
only use a lazy ByteString
instead of a strict one for better memory
configuration (chunking into multiple smaller bits instead of one
massive chunk of data).
Like runProcess
, there’s an exit-code-checking variant of
readProcess
:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import Data.ByteString.Lazy (ByteString)
main :: IO ()
main = do
(out, err) <- readProcess_ "date"
print (out :: ByteString)
print (err :: ByteString)
EXERCISE: Use shell redirection to move the output from standard output to standard error.
Redirecting to a file
Another technique we’ll commonly want to employ is to redirect output
from a process to a file. This is superior to the memory approach as
it does not have the risk of using large amounts of memory, though it
is more inconvenient. Together with the
UnliftIO.Temporary
, we
can do some nice things:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import UnliftIO.Temporary (withSystemTempFile)
main :: IO ()
main = withSystemTempFile "date" $ \fp h -> do
let dateConfig = setStdin closed
$ setStdout (useHandleClose h)
$ setStderr closed
"date"
runProcess_ dateConfig
readFile fp >>= print
The useHandleClose
function lets us provide an already existing
Handle
, and will close it when done. If you want to write the output
of multiple processes to a single file, you can instead use
useHandleOpen
:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import System.IO (hClose)
import UnliftIO.Temporary (withSystemTempFile)
import Control.Monad (replicateM_)
main :: IO ()
main = withSystemTempFile "date" $ \fp h -> do
let dateConfig = setStdin closed
$ setStdout (useHandleOpen h)
$ setStderr closed
"date"
replicateM_ 10 $ runProcess_ dateConfig
hClose h
readFile fp >>= putStrLn
EXERCISE Create a separate file for error output and capture that as well.
Providing input
Using OverloadedStrings
, it’s trivial to provide some input to a
process:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = runProcess_ $ setStdin "Hello World!\n" "cat"
This is just a shortcut for using the byteStringInput
function:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = runProcess_ $ setStdin (byteStringInput "Hello World!\n") "cat"
But like output and error, we can also use a Handle
or a temporary
file:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import System.IO
import UnliftIO.Temporary (withSystemTempFile)
main :: IO ()
main = withSystemTempFile "input" $ \fp h -> do
hPutStrLn h "Hello World!"
hClose h
withBinaryFile fp ReadMode $ \h' ->
runProcess_ $ setStdin (useHandleClose h') "cat"
Interacting with a process
So far, everything we’ve done has been running processes: spawning a child with some settings, then waiting for it to exit. We will often want to interact with a process: spawn it, and then send it input or receive output from it while it is still running.
For this, using createPipe
makes a lot of sense:
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
import System.IO
main :: IO ()
main = do
let catConfig = setStdin createPipe
$ setStdout createPipe
$ setStderr closed
"cat"
withProcess_ catConfig $ \p -> do
hPutStrLn (getStdin p) "Hello!"
hFlush (getStdin p)
hGetLine (getStdout p) >>= print
hClose (getStdin p)
EXERCISE: What happens if you remove the hClose
line, and why?
Hint: what happens if you both remove hClose
and replace
withProcess_
with withProcess
?
Other settings
We’ve so far only played with modifying streams, but there are a number of other settings you can tweak. It’s best to just look at the API docs for all available functions. We’ll give examples of the two most common settings: the working directory and environment variables.
#!/usr/bin/env stack
-- stack --resolver lts-16.27 script
{-# LANGUAGE OverloadedStrings #-}
import System.Process.Typed
main :: IO ()
main = do
putStrLn "1:"
runProcess_ "pwd"
putStrLn "\n2:"
runProcess_ $ setWorkingDir "/tmp" "pwd"
putStrLn "\n3:"
runProcess_ "env"
putStrLn "\n4:"
runProcess_ $ setEnv [("HELLO", "WORLD")] "env"
Async and STM
When interacting with a process on multiple streams, you’ll often want to use some kind of concurrency. The strong recommendation is to use the async library. Additionally, this library provides a number of functions that use STM, which also plays very nicely with concurrency and the async package. For some examples, check out:
waitExitCodeSTM
getExitCodeSTM
checkExitCodeSTM
byteStringOutput
EXERCISE Reimplement the readProcess
function using
byteStringOutput
and waitExitCodeSTM
.
EXERCISE Reimplement the readProcess_
function using
byteStringOutput
and checkExitCodeSTM
.
Changes
ChangeLog for typed-process
0.2.11.1
- No user-visible changes
0.2.11.0
- Expose more from
System.Process.Typed.Internal
0.2.10.0
- Add
mkPipeStreamSpec
0.2.9.0
- Re-export
StdStream
0.2.8.0
- Re-export
ExitCode
,ExitSuccess
andExitFailure
.
0.2.7.0
- Include empty argument in the show instance.
0.2.6.3
- Doc improvements
0.2.6.2
- Doc improvements
0.2.6.1
- Doc improvements
0.2.6.0
- The cleanup thread applies an
unmask
to the actions which wait for a process to exit, allowing the action to be interruptible.
0.2.5.0
- Add a
nullStream
#24 - Add
withProcessWait
,withProcessWait_
,withProcessTerm
, andwithProcessTerm_
#25
0.2.4.1
- Fix a
Handle
leak inwithProcessInterleave
and its derivatives.
0.2.4.0
- Add
readProcessInterleaved
andreadProcessInterleaved_
to support capturing output from stdout and stderr in a single ByteString value.
0.2.3.0
- Add support for the single-threaded runtime via polling
0.2.2.0
- Add inherit versions of setter functions
0.2.1.0
- Add
readProcessStdout
,readProcessStdout_
,readProcessStderr
, andreadProcessStderr_
- Do not show modified environment information in exceptions
0.2.0.0
- Remove dependency on
conduit
andconduit-extra
. Relevant code added toData.Conduit.Process.Typed
inconduit-extra-1.2.1
.
0.1.1
- Introduce ‘unsafeProcessHandle’ function
0.1.0.1
- Fix bug in
waitForProcess
that caused exit code to be lost - Minor doc improvements
0.1.0.0
- Initial commit