req
Easy-to-use, type-safe, expandable, high-level HTTP client library
Version on this page: | 2.1.0 |
LTS Haskell 22.33: | 3.13.2@rev:4 |
Stackage Nightly 2024-09-07: | 3.13.3 |
Latest on Hackage: | 3.13.3 |
req-2.1.0@sha256:ca64dfb2076e56f7a79e54e5e4a8ea36217769c7aa0b2d89cfc327cb4aa77abc,4751
Module documentation for 2.1.0
- Network
- Network.HTTP
Req
- Motivation and Req vs other libraries
- Unsolved problems
- Related packages
- Blog posts
- Contribution
- License
{-# LANGUAGE OverloadedStrings #-}
module Main (main) where
import Control.Monad.IO.Class
import Data.Aeson
import Network.HTTP.Req
main :: IO ()
-- You can either make your monad an instance of 'MonadHttp', or use
-- 'runReq' in any IO-enabled monad without defining new instances.
main = runReq defaultHttpConfig $ do
let payload = object
[ "foo" .= (10 :: Int)
, "bar" .= (20 :: Int) ]
-- One function—full power and flexibility, automatic retrying on timeouts
-- and such, automatic connection sharing.
r <- req POST -- method
(https "httpbin.org" /: "post") -- safe by construction URL
(ReqBodyJson payload) -- use built-in options or add your own
jsonResponse -- specify how to interpret response
mempty -- query params, headers, explicit port number, etc.
liftIO $ print (responseBody r :: Value)
Req is an easy-to-use, type-safe, expandable, high-level HTTP client library that just works without any fooling around.
What does the phrase “easy-to-use” mean? It means that the library is
designed to be beginner-friendly so it’s simple to add to your monad stack,
intuitive to work with, well-documented, and does not get in your way. Doing
HTTP requests is a common task and Haskell library for this should be very
approachable and clear to beginners, thus certain compromises were made. For
example, one cannot currently modify ManagerSettings
of the default
manager because the library always uses the same implicit global manager for
simplicity and maximal connection sharing. There is a way to use your own
manager with different settings, but it requires a bit more typing.
“Type-safe” means that the library is protective and eliminates certain
classes of errors. For example, we have correct-by-construction URLs, it’s
guaranteed that the user does not send the request body when using methods
like GET or OPTIONS, and the amount of implicit assumptions is minimized by
making the user specify his/her intentions in an explicit form (for example,
it’s not possible to avoid specifying the body or method of request).
Authentication methods that assume HTTPS force the user to use HTTPS at the
type level. The library also carefully hides underlying types from the
lower-level http-client
package because those types are not safe enough
(for example Request
is an instance of IsString
and, if it’s malformed,
it will blow up at run-time).
“Expandable” refers to the ability to create new components for dealing with HTTP without having to resort to ugly hacking. For example, it’s possible to define your own HTTP methods, create new ways to construct the body of a request, create new authorization options, perform a request in a different way, and create your own methods to parse and represent a response. As the user extends the library to satisfy his/her special needs, the new solutions will work just like the built-ins. However, all of the common cases are also covered by the library out-of-the-box.
“High-level” means that there are less details to worry about. The library
is a result of my experiences as a Haskell consultant. Working for several
clients, who had very different projects, showed me that the library should
adapt easily to any particular style of writing Haskell applications. For
example, some people prefer throwing exceptions, while others are concerned
with purity. Just define handleHttpException
accordingly when making your
monad instance of MonadHttp
and it will play together seamlessly. Finally,
the library cuts down boilerplate considerably, and helps you write concise,
easy to read, and maintainable code.
The library uses the following mature packages under the hood to guarantee you the best experience:
http-client
—low level HTTP client used everywhere in Haskell.http-client-tls
—TLS (HTTPS) support forhttp-client
.
It’s important to note that since we leverage well-known libraries that the
whole Haskell ecosystem uses, there is no risk in using Req. The machinery
for performing requests is the same as with http-conduit
and Wreq. The
only difference is the API.
Motivation and Req vs other libraries
This section is my opinion and it contains criticisms of other well-known libraries. If you’re user/fan of one of these libraries, please remember not to react aggressively and respect the fact that I may have different views on API design from yours.
I have spent time to write the library because sending HTTP requests is such a common thing and still there is no high-level library for that in Haskell that I could use with pleasure. I’ll explain why.
First of all, there is http-client
and http-client-tls
. They just work.
I have no issues with the libraries except that they are too low-level for
my taste. Indeed, even the docs say that they are low-level and “intended as
a base layer for more user-friendly packages”. This is exactly how I use
them in Req, as base level. Req is nothing but a different API to
http-client
, so it only works because of the hard work put into
http-client
.
http-conduit
definitely has its place. For one thing it allows you to
stream request and response bodies in constant memory, what other library
allows you to do that? On the other hand if you take a look at
Network.HTTP.Simple
, then although it’s said that it’s a “higher level
API”, it’s mostly the same as vanilla http-client
in spirit/approach and
just adds conduit
-powered functions to perform requests and allows to use
global implicit Manager
(Req does the same). If I tried to frame what
exactly I don’t like about http-conduit
in words, then it would be “the
way requests are constructed”. You set, set, set instead of being forced
to declare necessary bits and being allowed to declare optional bits in a
way that their combination is certainly valid. Also, with http-conduit
you
parse request from a string without the protection of TH that otherwise
saves the day as in Yesod.
Then there is Wreq. wreq
doesn’t see much development
lately. wreq
is by itself a weird
library, IMO. You have functions per method—not very good, as there may be
new methods, like PATCH which is not new but still missing (well, you have
customMethod
, but what is the point of having per-method functions if you
have a more general way to use any method? you should be able to just insert
methods in the “argument slot” of customMethod
and end up with a more
general solution). Now, every method function has a companion that takes
Options
(like you have get
and getWith
). Why the duplication? Where is
generality and flexibility? This is not all though, because you cannot
really use get
you see in the main module, because you want to have
connection sharing. Wreq’s author does not take the gift of automatic
connection re-use Manager
from http-client
provides, he invents the
whole new thing of “sessions”. Only inside a session your connections will
be shared and re-used. However with the session stuff you have yet another
set of per-method functions like get
and getWith
—these are different
ones, to be used with sessions! Now if you have a multi-threaded app, here
is a surprise for you: you can’t share connections between threads as
connections are shared only inside of the withSession
friend and “session
will no longer be valid after that function returns”. There are valid uses
for sessions, but the point is that they are just too inconvenient for
common tasks.
It’s funny that one client I worked for had to invent his own little wrapper
around http-client
just because he could not possibly use wreq
and
http-client
and friends were too low-level. The previous paragraph is
extracted from a talk with a Haskell developer who works for that client. I
thought to myself “something is wrong with HTTP client libraries in Haskell
if they had to make a wrapper”.
What else? I used servant-client
a couple of times but the amount of
boilerplate it requires is frightening. If you have several query
parameters, and you use just one of them, good luck passing lots of
Nothing
s.
Unsolved problems
AWS request signing is problematic because request body can be in the form
of an action to execute (and all that “popper” stuff for streaming), not
just a ByteString
and so getting its digest (hash) is not trivial without
running the action and consuming body in its entirety before the request in
made. In Wreq the author chose to just use error
when body is not a
(strict or lazy) ByteString
. Maybe it’s OK for Wreq, but I don’t consider
this a proper solution for Req as we support full variety of body options.
For example, what if I want to upload 1 Gb file to S3? I want to stream it
in constant memory but at the same time I need to calculate its hash before
I start streaming. One solution to the problem seems to be in taking the
hash explicitly (as an argument of the hypothetical awsAuth
) and making it
a responsibility of the user to calculate the hash correctly. I don’t like
this because it’s not user-friendly. So the question stays open, for now
there is no AWS signing functionality provided out-of-the-box. The best
solution for talking to AWS is the amazonka
package so far.
Related packages
The following packages are designed to be used with Req:
req-conduit
—support for streaming request and response bodies in constant memory.
If you happen to have written a package that adds new features to Req, please submit a PR to include it in this list.
Blog posts
Contribution
Issues, bugs, and questions may be reported in the GitHub issue tracker for this project.
Pull requests are also welcome.
License
Copyright © 2016–2019 Mark Karpov
Distributed under BSD 3 clause license.
Changes
Req 2.1.0
-
Dropped support for GHC 7.10.
-
Added the new
acceptHeader
method to theHttpResponse
type class. Notably, thejsonResponse
method now sets"Accept"
header to"application/json"
.
Req 2.0.1
- Fixed the
httpbin
tests (they changed something on the server again).
Req 2.0.0
- Got rid of
data-default-class
dependency, now we exportdefaultHttpConfig
instead.
Req 1.2.1
- Fixed a typo in the type signature of
parseUrl
.
Req 1.2.0
- Added the
parseUrl
function.
Req 1.1.0
-
Added
customAuth
andattachHeader
to facilitate creation of custom authentication options. -
Added
basicProxyAuth
authentication option.
Req 1.0.0
-
Added the
reqBr
function allowing to consumeResponse BodyReader
without using a pre-defined instance ofHttpResponse
, in a custom way. -
Now streaming of response body does not happen until we’ve checked headers and status code with
httpConfigCheckResponse
. It also doesn’t happen on every retry. Streaming and obtaining of final response value happens only once when we’re happy with everything.Previously we first tried to consume and interpret response body before checking status code and determining whether we should retry the request. This was not good, because we could expect a JSON response but get a response with status code 500, and then still we would try to parse it as JSON first before letting
httpConfigCheckResponse
throw an exception.The corrected behavior should also make retrying more efficient.
-
Changed signatures of several fields of
HttpConfig
:httpConfigCheckResponse
,httpConfigRetryPolicy
, andhttpConfigRetryJudge
in order to eliminate redundantIO
and prevent the possibility that these functions could start consumingBodyReader
. -
Removed the
makeResponsePreview
method from theHttpResponse
type class. Preview business is handled by the library automatically on a lower level now. Users do not need to concern themselves with such stuff. -
Changed the type signature of the
getHttpResponse
method of theHttpResponse
type class. Previously it left too much freedom (and responsibility) to implementers of the method. In fact, we now limit whatgetHttpResponse
does to just consuming and interpretingResponse BodyReader
, so we can properly control details of connection opening/closing etc., for the user. -
Dropped support for GHC 7.8.
-
Minor documentation improvements.
Req 0.5.0
-
Changed the signature of the
makeResponseBodyPreview
fromresponse -> IO ByteString
toresponse -> ByteString
. -
Minor documentation improvements.
Req 0.4.0
- Added the
Req
monad andrunReq
function to run it. This allows to usereq
without defining new (orphan) instances.
Req 0.3.1
- Added
basicAuthUnsafe
.
Req 0.3.0
-
Made URL parsing functions
parseUrlHttp
andparseUrlHttps
recognize port numbers. -
Added
req'
function that allows to perform requests via a callback that receives pre-constructed request and manager. -
Removed the
ReturnRequest
HTTP response implementation as it was not quite safe and was not going to work with retrying. Usereq'
instead for “pure” testing. -
Changed the type of
httpConfigCheckResponse
, so the second argument can be any instance ofHttpResponse
. -
Added built-in automatic retrying. See
httpConfigRetryPolicy
andhttpConfigRetryJudge
inHttpConfig
. The default configuration retries 5 times on request timeouts. -
Added the
makeResponseBodyPreview
method to theHttpResponse
type class that allows to specify how to build a “preview” of response body for inclusion into exceptions. -
Improved wording in the documentation and
README.md
.
Req 0.2.0
-
Added support for multipart form data in the form of
ReqBodyMultipart
body option andreqBodyMultipart
helper function. This also required a change in the type signature ofgetRequestContentType
, which now takesbody
, notProxy body
because we need to extract boundary frombody
and put it intoContent-Type
header. This change, however, shouldn’t be too dangerous for end-users. -
Added support for OAuth 1.0 authentication via
oAuth1
option.
Req 0.1.0
- Initial release.