minio-hs

A MinIO Haskell Library for Amazon S3 compatible cloud storage.

https://github.com/minio/minio-hs#readme

LTS Haskell 21.25:1.7.0
Stackage Nightly 2023-06-21:1.7.0
Latest on Hackage:1.7.0

See all snapshots minio-hs appears in

Apache-2.0 licensed by MinIO Dev Team
Maintained by [email protected]
This version can be pinned in stack with:minio-hs-1.7.0@sha256:b2721568f2cb8d3cb8ff14eb6ee25af15c164f049493c65664bb2a2d1dd9a071,11708

Module documentation for 1.7.0

MinIO Haskell Client SDK for Amazon S3 Compatible Cloud Storage CIHackageSlack

The MinIO Haskell Client SDK provides simple APIs to access MinIO and any Amazon S3 compatible object storage.

This guide assumes that you have a working Haskell development environment.

Installation

Add to your project

Simply add minio-hs to your project’s .cabal dependencies section or if you are using hpack, to your package.yaml file as usual.

Try it out in a REPL

For a cabal based environment

Download the library source and change to the extracted directory:

$ cabal get minio-hs
$ cd minio-hs-1.6.0/ # directory name could be different

Then load the ghci REPL environment with the library and browse the available APIs:

$ cabal repl
ghci> :browse Network.Minio

For a stack based environment

From your home folder or any non-haskell project directory, just run:

stack install minio-hs

Then start an interpreter session and browse the available APIs with:

$ stack ghci
> :browse Network.Minio

Examples

The examples folder contains many examples that you can try out and use to learn and to help with developing your own projects.

Quick-Start Example - File Uploader

This example program connects to a MinIO object storage server, makes a bucket on the server and then uploads a file to the bucket.

We will use the MinIO server running at https://play.min.io in this example. Feel free to use this service for testing and development. Access credentials are present in the library and are open to the public.

FileUploader.hs

#!/usr/bin/env stack
-- stack --resolver lts-14.11 runghc --package minio-hs --package optparse-applicative --package filepath

--
-- MinIO Haskell SDK, (C) 2017-2019 MinIO, Inc.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--     http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
--


{-# LANGUAGE OverloadedStrings   #-}
{-# LANGUAGE ScopedTypeVariables #-}
import           Network.Minio

import           Data.Monoid           ((<>))
import           Data.Text             (pack)
import           Options.Applicative
import           System.FilePath.Posix
import           UnliftIO              (throwIO, try)

import           Prelude

-- | The following example uses minio's play server at
-- https://play.min.io.  The endpoint and associated
-- credentials are provided via the libary constant,
--
-- > minioPlayCI :: ConnectInfo
--

-- optparse-applicative package based command-line parsing.
fileNameArgs :: Parser FilePath
fileNameArgs = strArgument
               (metavar "FILENAME"
                <> help "Name of file to upload to AWS S3 or a MinIO server")

cmdParser = info
            (helper <*> fileNameArgs)
            (fullDesc
             <> progDesc "FileUploader"
             <> header
             "FileUploader - a simple file-uploader program using minio-hs")

main :: IO ()
main = do
  let bucket = "my-bucket"

  -- Parse command line argument
  filepath <- execParser cmdParser
  let object = pack $ takeBaseName filepath

  res <- runMinio minioPlayCI $ do
    -- Make a bucket; catch bucket already exists exception if thrown.
    bErr <- try $ makeBucket bucket Nothing

    -- If the bucket already exists, we would get a specific
    -- `ServiceErr` exception thrown.
    case bErr of
      Left BucketAlreadyOwnedByYou -> return ()
      Left e                       -> throwIO e
      Right _                      -> return ()

    -- Upload filepath to bucket; object name is derived from filepath.
    fPutObject bucket object filepath defaultPutObjectOptions

  case res of
    Left e   -> putStrLn $ "file upload failed due to " ++ show e
    Right () -> putStrLn "file upload succeeded."

Run FileUploader

./FileUploader.hs "path/to/my/file"

Contribute

Contributors Guide

Development

Download the source

$ git clone https://github.com/minio/minio-hs.git
$ cd minio-hs/

Build the package:

With cabal:

$ # Configure cabal for development enabling all optional flags defined by the package.
$ cabal configure --enable-tests --test-show-details=direct -fexamples -fdev -flive-test
$ cabal build

With stack:

$ stack build --test --no-run-tests --flag minio-hs:live-test --flag minio-hs:dev --flag minio-hs:examples

Running tests:

A section of the tests use the remote MinIO Play server at https://play.min.io by default. For library development, using this remote server maybe slow. To run the tests against a locally running MinIO live server at http://localhost:9000 with the credentials access_key=minio and secret_key=minio123, just set the environment MINIO_LOCAL to any value (and unset it to switch back to Play).

With cabal:

$ export MINIO_LOCAL=1 # to run live tests against local MinIO server
$ cabal test

With stack:

$ export MINIO_LOCAL=1 # to run live tests against local MinIO server
stack test --flag minio-hs:live-test --flag minio-hs:dev

This will run all the test suites.

Building documentation:

$ cabal haddock
$ # OR
$ stack haddock

Changes

Changelog

Version 1.7.0 – Unreleased

  • Fix data type EventMessage to not export partial fields (#179)
  • Bump up min bound on time dep and fix deprecation warnings (#181)
  • Add dev flag to cabal for building with warnings as errors (#182)
  • Fix AWS region map (#185)
  • Fix XML generator tests (#187)
  • Add support for STS Assume Role API (#188)

Breaking changes in 1.7.0

  • Credentials type has been removed. Use CredentialValue instead.
  • Provider type has been replaced with CredentialLoader.
  • EventMessage data type is updated.

Version 1.6.0

  • HLint fixes - some types were changed to newtype (#173)
  • Fix XML generation test for S3 SELECT (#161)
  • Use region specific endpoints for AWS S3 in presigned Urls (#164)
  • Replace protolude with relude and build with GHC 9.0.2 (#168)
  • Support aeson 2 (#169)
  • CI updates and code formatting changes with ormolu 0.5.0.0

Version 1.5.3

  • Fix windows build
  • Fix support for Yandex Storage (#147)
  • Fix for HEAD requests to S3/Minio (#155)
  • Bump up some dependencies, new code formatting, Github CI, example fixes and other minor improvements.

Version 1.5.2

  • Fix region us-west-2 for AWS S3 (#139)
  • Build examples in CI
  • Disable live-server tests by default, but run them in CI
  • Drop support for GHC 8.2.x

Version 1.5.1

  • Add support for GHC 8.8

Version 1.5.0

  • Switch to faster map data type - all previous usage of Data.Map.Strict and Data.Set is replaced with Data.HashMap.Strict and Data.HashSet.
  • Add oiUserMetadata to parse and return user metadata stored with an object.
  • Add GetObjectResponse data type for the value returned by getObject. It now contains parsed ObjectInfo along with the conduit of object bytes.

Version 1.4.0

  • Expose runMinioRes and runMinioResWith (#129)
  • Improve Haddocks (#127)
  • Fix list objects APIs to return directory prefixes when run with recurse set to False (#126)
  • Use streaming signature for streaming payloads when on an insecure connection (#123)

Version 1.3.1

  • Add TLS helpers to check if server uses TLS, and to disable certificate validation for easier testing (#121)

Version 1.3.0

  • Retry requests that timeout using full-jitter backoff (#119)
  • Accept GetObjectOptions in statObject (#112)
  • Add encryption options to GetObjectOptions and PutObjectOptions (#111)
  • Add missing Haddock documentation (#110)
  • Add support for S3Select API (#108)
  • New travis with support for multiple GHCs (#106)
  • Fix region setting in presigned url functions (#107)

Version 1.2.0

  • Export Provider and findFirst to look for credentials (#103)

Version 1.1.0

This version brings the following changes:

  • Adds experimental Admin APIs (#88, #91, #93, #94, #95, #100)
  • Adds support for using Google Compute Storage service when S3 compatibility mode is enabled (#96, #99)

This version also brings some breaking changes (via #101):

  • Adds IsString instance to load server address, and updates initialization API to be more user friendly
  • Drops usage of data-default package and exposes explicit default values for various types used in the library.

Version 1.0.1

This version brings the following (non-breaking) changes:

  • Remove dependency on text-format (#86)
  • Remove direct dependency on exceptions (#87)
  • Adds lower-bounds on dependencies.

Version 1.0.0

This new release changes the following APIs to add new capabilities:

  • Copy Object API now supports more options for source and destination (#73)
  • get/put Object functions now support a wider set of options via a separate settings parameter (#71, #72)
  • getBucketPolicy and setBucketPolicy APIs are added (#82)
  • The library now uses UnliftIO (#83)

Version 0.3.2

This release brings the following changes:

  • Add removeIncompleteUpload API (#49)
  • Add presigned operations APIs (#56)
  • Add presigned Post Policy API (#58)
  • Skip SHA256 checksum header for secure connections (#65)
  • Remove resuming capability in PutObject (#67)
  • Add ListObjectsV1 API support (#66)
  • Add Bucket Notification APIs (#59)
  • Reverse #54 - tests fix.

Version 0.3.1

This is a bug-fix release:

  • Fix concurrency bug in limitedMapConcurrently (#53)
  • Fix tests related to listing incomplete uploads to accommodate MinIO server’s changed behaviour to not list incomplete uploads. Note that running these tests against AWS S3 are expected to fail. (#54)

Version 0.3.0

This release includes a breaking change:

Users of the library need not call runResourceT explicitly after calling runMinio. This is now done, within the runMinio call making usage a bit simpler.

Other changes:

  • Export ListUploadsResult and ListObjectsResult (#48)
    • Also take max-keys as an argument for listObjects and max-uploads for listIncompleteUploads.
  • Add bucket and object name validation (#45)
  • Add bucketExists and headBucket APIs (#42)

Version 0.2.1

  • Update dependencies, and switch to Stackage LTS 8.5

Version 0.2.0

This is an interim release which brings some new features. However, the library is not complete and APIs may change.

  • Remove listIncompleteParts API and augment listIncompleteUploads API with information about aggregate size of parts uploaded.
  • Refactors error types and simpler error throwing/handling behaviour.
  • Add removeObject API to delete objects from the service.
  • Rename Network.Minio.getService to Network.Minio.listBuckets.
  • Add docs/API.md and examples directory with comprehensive documentation and examples of high-level APIs exported by the library.
  • Rename types:
    • Rename PartInfo -> PartTuple
    • Rename ListPartInfo -> ObjectPartInfo
  • Add a bucket region cache to avoid locating a bucket’s region for every operation (mainly useful for AWS S3).
  • Add new copyObject API to perform server side object copying.
  • Rename putObjectFromSource API as putObject.
  • Separate out tests into two suites, one with a live-server and the other without any external dependencies.