minio-hs
A MinIO Haskell Library for Amazon S3 compatible cloud storage.
https://github.com/minio/minio-hs#readme
LTS Haskell 21.25: | 1.7.0 |
Stackage Nightly 2023-06-21: | 1.7.0 |
Latest on Hackage: | 1.7.0 |
minio-hs-1.7.0@sha256:b2721568f2cb8d3cb8ff14eb6ee25af15c164f049493c65664bb2a2d1dd9a071,11708
Module documentation for 1.7.0
MinIO Haskell Client SDK for Amazon S3 Compatible Cloud Storage
The MinIO Haskell Client SDK provides simple APIs to access MinIO and any Amazon S3 compatible object storage.
This guide assumes that you have a working Haskell development environment.
Installation
Add to your project
Simply add minio-hs
to your project’s .cabal
dependencies section or if you are using hpack, to your package.yaml
file as usual.
Try it out in a REPL
For a cabal based environment
Download the library source and change to the extracted directory:
$ cabal get minio-hs
$ cd minio-hs-1.6.0/ # directory name could be different
Then load the ghci
REPL environment with the library and browse the available APIs:
$ cabal repl
ghci> :browse Network.Minio
For a stack based environment
From your home folder or any non-haskell project directory, just run:
stack install minio-hs
Then start an interpreter session and browse the available APIs with:
$ stack ghci
> :browse Network.Minio
Examples
The examples folder contains many examples that you can try out and use to learn and to help with developing your own projects.
Quick-Start Example - File Uploader
This example program connects to a MinIO object storage server, makes a bucket on the server and then uploads a file to the bucket.
We will use the MinIO server running at https://play.min.io in this example. Feel free to use this service for testing and development. Access credentials are present in the library and are open to the public.
FileUploader.hs
#!/usr/bin/env stack
-- stack --resolver lts-14.11 runghc --package minio-hs --package optparse-applicative --package filepath
--
-- MinIO Haskell SDK, (C) 2017-2019 MinIO, Inc.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
--
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
import Network.Minio
import Data.Monoid ((<>))
import Data.Text (pack)
import Options.Applicative
import System.FilePath.Posix
import UnliftIO (throwIO, try)
import Prelude
-- | The following example uses minio's play server at
-- https://play.min.io. The endpoint and associated
-- credentials are provided via the libary constant,
--
-- > minioPlayCI :: ConnectInfo
--
-- optparse-applicative package based command-line parsing.
fileNameArgs :: Parser FilePath
fileNameArgs = strArgument
(metavar "FILENAME"
<> help "Name of file to upload to AWS S3 or a MinIO server")
cmdParser = info
(helper <*> fileNameArgs)
(fullDesc
<> progDesc "FileUploader"
<> header
"FileUploader - a simple file-uploader program using minio-hs")
main :: IO ()
main = do
let bucket = "my-bucket"
-- Parse command line argument
filepath <- execParser cmdParser
let object = pack $ takeBaseName filepath
res <- runMinio minioPlayCI $ do
-- Make a bucket; catch bucket already exists exception if thrown.
bErr <- try $ makeBucket bucket Nothing
-- If the bucket already exists, we would get a specific
-- `ServiceErr` exception thrown.
case bErr of
Left BucketAlreadyOwnedByYou -> return ()
Left e -> throwIO e
Right _ -> return ()
-- Upload filepath to bucket; object name is derived from filepath.
fPutObject bucket object filepath defaultPutObjectOptions
case res of
Left e -> putStrLn $ "file upload failed due to " ++ show e
Right () -> putStrLn "file upload succeeded."
Run FileUploader
./FileUploader.hs "path/to/my/file"
Contribute
Development
Download the source
$ git clone https://github.com/minio/minio-hs.git
$ cd minio-hs/
Build the package:
With cabal
:
$ # Configure cabal for development enabling all optional flags defined by the package.
$ cabal configure --enable-tests --test-show-details=direct -fexamples -fdev -flive-test
$ cabal build
With stack
:
$ stack build --test --no-run-tests --flag minio-hs:live-test --flag minio-hs:dev --flag minio-hs:examples
Running tests:
A section of the tests use the remote MinIO Play server at https://play.min.io
by default. For library development, using this remote server maybe slow. To run the tests against a locally running MinIO live server at http://localhost:9000
with the credentials access_key=minio
and secret_key=minio123
, just set the environment MINIO_LOCAL
to any value (and unset it to switch back to Play).
With cabal
:
$ export MINIO_LOCAL=1 # to run live tests against local MinIO server
$ cabal test
With stack
:
$ export MINIO_LOCAL=1 # to run live tests against local MinIO server
stack test --flag minio-hs:live-test --flag minio-hs:dev
This will run all the test suites.
Building documentation:
$ cabal haddock
$ # OR
$ stack haddock
Changes
Changelog
Version 1.7.0 – Unreleased
- Fix data type
EventMessage
to not export partial fields (#179) - Bump up min bound on time dep and fix deprecation warnings (#181)
- Add
dev
flag to cabal for building with warnings as errors (#182) - Fix AWS region map (#185)
- Fix XML generator tests (#187)
- Add support for STS Assume Role API (#188)
Breaking changes in 1.7.0
Credentials
type has been removed. UseCredentialValue
instead.Provider
type has been replaced withCredentialLoader
.EventMessage
data type is updated.
Version 1.6.0
- HLint fixes - some types were changed to newtype (#173)
- Fix XML generation test for S3 SELECT (#161)
- Use region specific endpoints for AWS S3 in presigned Urls (#164)
- Replace protolude with relude and build with GHC 9.0.2 (#168)
- Support aeson 2 (#169)
- CI updates and code formatting changes with ormolu 0.5.0.0
Version 1.5.3
- Fix windows build
- Fix support for Yandex Storage (#147)
- Fix for HEAD requests to S3/Minio (#155)
- Bump up some dependencies, new code formatting, Github CI, example fixes and other minor improvements.
Version 1.5.2
- Fix region
us-west-2
for AWS S3 (#139) - Build examples in CI
- Disable live-server tests by default, but run them in CI
- Drop support for GHC 8.2.x
Version 1.5.1
- Add support for GHC 8.8
Version 1.5.0
- Switch to faster map data type - all previous usage of Data.Map.Strict and Data.Set is replaced with Data.HashMap.Strict and Data.HashSet.
- Add
oiUserMetadata
to parse and return user metadata stored with an object. - Add
GetObjectResponse
data type for the value returned bygetObject
. It now contains parsed ObjectInfo along with the conduit of object bytes.
Version 1.4.0
- Expose runMinioRes and runMinioResWith (#129)
- Improve Haddocks (#127)
- Fix list objects APIs to return directory prefixes when run with recurse set to False (#126)
- Use streaming signature for streaming payloads when on an insecure connection (#123)
Version 1.3.1
- Add TLS helpers to check if server uses TLS, and to disable certificate validation for easier testing (#121)
Version 1.3.0
- Retry requests that timeout using full-jitter backoff (#119)
- Accept GetObjectOptions in statObject (#112)
- Add encryption options to GetObjectOptions and PutObjectOptions (#111)
- Add missing Haddock documentation (#110)
- Add support for S3Select API (#108)
- New travis with support for multiple GHCs (#106)
- Fix region setting in presigned url functions (#107)
Version 1.2.0
- Export Provider and findFirst to look for credentials (#103)
Version 1.1.0
This version brings the following changes:
- Adds experimental Admin APIs (#88, #91, #93, #94, #95, #100)
- Adds support for using Google Compute Storage service when S3 compatibility mode is enabled (#96, #99)
This version also brings some breaking changes (via #101):
- Adds IsString instance to load server address, and updates initialization API to be more user friendly
- Drops usage of data-default package and exposes explicit default values for various types used in the library.
Version 1.0.1
This version brings the following (non-breaking) changes:
- Remove dependency on text-format (#86)
- Remove direct dependency on exceptions (#87)
- Adds lower-bounds on dependencies.
Version 1.0.0
This new release changes the following APIs to add new capabilities:
- Copy Object API now supports more options for source and destination (#73)
- get/put Object functions now support a wider set of options via a separate settings parameter (#71, #72)
- getBucketPolicy and setBucketPolicy APIs are added (#82)
- The library now uses UnliftIO (#83)
Version 0.3.2
This release brings the following changes:
- Add
removeIncompleteUpload
API (#49) - Add presigned operations APIs (#56)
- Add presigned Post Policy API (#58)
- Skip SHA256 checksum header for secure connections (#65)
- Remove resuming capability in PutObject (#67)
- Add ListObjectsV1 API support (#66)
- Add Bucket Notification APIs (#59)
- Reverse #54 - tests fix.
Version 0.3.1
This is a bug-fix release:
- Fix concurrency bug in
limitedMapConcurrently
(#53) - Fix tests related to listing incomplete uploads to accommodate MinIO server’s changed behaviour to not list incomplete uploads. Note that running these tests against AWS S3 are expected to fail. (#54)
Version 0.3.0
This release includes a breaking change:
Users of the library need not call runResourceT
explicitly after
calling runMinio
. This is now done, within the runMinio
call
making usage a bit simpler.
Other changes:
- Export ListUploadsResult and ListObjectsResult (#48)
- Also take max-keys as an argument for listObjects and max-uploads for listIncompleteUploads.
- Add bucket and object name validation (#45)
- Add bucketExists and headBucket APIs (#42)
Version 0.2.1
- Update dependencies, and switch to Stackage LTS 8.5
Version 0.2.0
This is an interim release which brings some new features. However, the library is not complete and APIs may change.
- Remove
listIncompleteParts
API and augmentlistIncompleteUploads
API with information about aggregate size of parts uploaded. - Refactors error types and simpler error throwing/handling behaviour.
- Add
removeObject
API to delete objects from the service. - Rename
Network.Minio.getService
toNetwork.Minio.listBuckets
. - Add
docs/API.md
and examples directory with comprehensive documentation and examples of high-level APIs exported by the library. - Rename types:
- Rename PartInfo -> PartTuple
- Rename ListPartInfo -> ObjectPartInfo
- Add a bucket region cache to avoid locating a bucket’s region for every operation (mainly useful for AWS S3).
- Add new
copyObject
API to perform server side object copying. - Rename
putObjectFromSource
API asputObject
. - Separate out tests into two suites, one with a live-server and the other without any external dependencies.