bloodhound

Elasticsearch client library for Haskell

https://github.com/bitemyapp/bloodhound.git#readme

LTS Haskell 23.1:0.23.0.0
Stackage Nightly 2024-12-21:0.23.0.0
Latest on Hackage:0.23.0.0

See all snapshots bloodhound appears in

BSD-3-Clause licensed by Chris Allen
Maintained by [email protected]
This version can be pinned in stack with:bloodhound-0.23.0.0@sha256:384e4ddfb28bd2650096e91b169ff562d2ab1f21bd9aecfcf08f67a96c69ab70,6802

Module documentation for 0.23.0.0

Bloodhound TravisCI Hackage

Bloodhound (dog)

Elasticsearch client and query DSL for Haskell

Why?

Search doesn’t have to be hard. Let the dog do it.

Endorsements

“Bloodhound makes Elasticsearch almost tolerable!” - Almost-gruntled user

“ES is a nightmare but Bloodhound at least makes it tolerable.” - Same user, later opinion.

Version compatibility

See our TravisCI for a listing of Elasticsearch version we test against.

Stability

Bloodhound is stable for production use. I will strive to avoid breaking API compatibility from here on forward, but dramatic features like a type-safe, fully integrated mapping API may require breaking things in the future.

Testing

The TravisCI tests are run using Stack. You should use Stack instead of cabal to build and test Bloodhound to avoid compatibility problems. You will also need to have an Elasticsearch instance running at localhost:9200 in order to execute some of the tests. See the “Version compatibility” section above for a list of Elasticsearch versions that are officially validated against in TravisCI.

Steps to run the tests locally:

  1. Dig through the [past releases] (https://www.elastic.co/downloads/past-releases) section of the Elasticsearch download page and install the desired Elasticsearch versions.
  2. Install [Stack] (http://docs.haskellstack.org/en/stable/README.html#how-to-install)
  3. In your local Bloodhound directory, run stack setup && stack build
  4. Start the desired version of Elasticsearch at localhost:9200, which should be the default.
  5. Run stack test in your local Bloodhound directory.
  6. The unit tests will pass if you re-execute stack test. If you want to start with a clean slate, stop your Elasticsearch instance, delete the data/ folder in the Elasticsearch installation, restart Elasticsearch, and re-run stack test.

Contributions

Any contribution is welcomed, for consistency reason ormolu is used.

Hackage page and Haddock documentation

http://hackage.haskell.org/package/bloodhound

Elasticsearch Tutorial

It’s not using Bloodhound, but if you need an introduction to or overview of Elasticsearch and how to use it, you can use this screencast.

Examples

See the examples directory for example code.

Index a document

indexDocument testIndex defaultIndexDocumentSettings exampleTweet (DocId "1")
{-
IndexedDocument
  { idxDocIndex = "twitter"
  , idxDocType = "_doc"
  , idxDocId = "1"
  , idxDocVersion = 3
  , idxDocResult = "updated"
  , idxDocShards =
      ShardResult
        { shardTotal = 1
        , shardsSuccessful = 1
        , shardsSkipped = 0
        , shardsFailed = 0
        }
  , idxDocSeqNo = 2
  , idxDocPrimaryTerm = 1
  }
-}

Fetch documents

let query = TermQuery (Term "user" "bitemyapp") boost
let search = mkSearch (Just query) boost
searchByIndex @_ @Tweet testIndex search
{-
SearchResult
    { took = 1
    , timedOut = False
    , shards =
            ShardResult
                { shardTotal = 1
                , shardsSuccessful = 1
                , shardsSkipped = 0
                , shardsFailed = 0
                }
    , searchHits =
            SearchHits
                { hitsTotal = HitsTotal { value = 2 , relation = HTR_EQ }
                , maxScore = Just 0.18232156
                , hits =
                        [ Hit
                                { hitIndex = IndexName "twitter"
                                , hitDocId = DocId "1"
                                , hitScore = Just 0.18232156
                                , hitSource =
                                        Just
                                            Tweet
                                                { user = "bitemyapp"
                                                , postDate = 2009-06-18 00:00:10 UTC
                                                , message = "Use haskell!"
                                                , age = 10000
                                                , location = LatLon { lat = 40.12 , lon = -71.3 }
                                                }
                                , hitSort = Nothing
                                , hitFields = Nothing
                                , hitHighlight = Nothing
                                , hitInnerHits = Nothing
                                }
                        , Hit
                                { hitIndex = IndexName "twitter"
                                , hitDocId = DocId "2"
                                , hitScore = Just 0.18232156
                                , hitSource =
                                        Just
                                            Tweet
                                                { user = "bitemyapp"
                                                , postDate = 2009-06-18 00:00:10 UTC
                                                , message = "Use haskell!"
                                                , age = 10000
                                                , location = LatLon { lat = 40.12 , lon = -71.3 }
                                                }
                                , hitSort = Nothing
                                , hitFields = Nothing
                                , hitHighlight = Nothing
                                , hitInnerHits = Nothing
                                }
                        ]
                }
    , aggregations = Nothing
    , scrollId = Nothing
    , suggest = Nothing
    , pitId = Nothing
    }
-}

Contributors

Possible future functionality

Span Queries

Beginning here: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-span-first-query.html

Function Score Query

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

Node discovery and failover

Might require TCP support.

Support for TCP access to Elasticsearch

Pretend to be a transport client?

Bulk cluster-join merge

Might require making a lucene index on disk with the appropriate format.

GeoShapeQuery

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-query.html

GeoShapeFilter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-filter.html

Geohash cell filter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geohash-cell-filter.html

HasChild Filter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-child-filter.html

HasParent Filter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html

Indices Filter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-indices-filter.html

Query Filter

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html

Script based sorting

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting

Collapsing redundantly nested and/or structures

The Seminearring instance, if deeply nested can possibly produce nested structure that is redundant. Depending on how this affects ES performance, reducing this structure might be valuable.

Runtime checking for cycles in data structures

check for n > 1 occurrences in DFS:

http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic.html

http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic-Map.html

Photo Origin

Photo from HA! Designs: https://www.flickr.com/photos/hadesigns/

Changes

0.23.0.0

  • @blackheaven
    • Rework MonadBH to be function oriented (instead of implementation oriented)
    • Client: parsing is done directly
    • Create and expose Database.Bloodhound.Requests for low-level operations
    • Fix payload parsing issues and types there and there
    • Import @wireapp fork
    • Support OpenSearch 1.3.12 / 2.9.0
    • Use versions instead of semver (VersionNumber wraps Data.Versions.Version)
    • Drop VMVersion in favor of VersionNumber
    • EsError.errorStatus became a Maybe and is deprecated
    • EsResult._type is deprecated
    • IndexedDocumentidxDocType is optional and deprecated
    • Module reorganization (avoid exposing Internal and split ElasticSearch/OpenSearch)
    • Rename ResultsShard’s shards to srShards
    • Upgrade to Cabal 3.0
    • Add optics definitions
# Migration script
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Client/Database.Bloodhound.Client.Cluster/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Aggregation/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Analysis/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Highlight/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Newtypes/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Query/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Reindex/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Sort/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e '/Database.Bloodhound.Internal.StringlyTyped/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Suggest/Database.Bloodhound.Types/g' {}
fd --type file -x sed -i -e 's/Database.Bloodhound.Internal.Task/Database.Bloodhound.Types/g' {}
  • @wireapp
    • Implement updateByQuery
    • Implement reindexing/tasks
    • Add support for ngram and edge_ngram filters
    • Add support for edge_ngram tokenizer
    • Fix CountShared support ES 5.2
  • @robinp
    • Add proper BulkResponse to bulk (#293)

0.22.0.0

  • @blackheaven
    • Fix EsError for OpenSearch

0.21.0.0

  • @blackheaven
    • Fix indexDocument Response type
    • Fix Acknowledged and Accepted FromJSON

0.20.0.0

  • @blackheaven
    • Strong type API, breaking changes
      • Reply is replaced by BHResponse body
      • decode/eitherDecode can’t be used directly (see decodeResponse/eitherDecodeResponse)
      • parseEsResponse return a ParsedEsResponse
      • getStatus returns Either instead of Maybe
      • Many functions have been strong typed from Reply (mainly BHResponse a)
      • respIsTwoHunna has been replaced by isSuccess

0.19.1.0

  • @TristanCacqueray
    • Fix aeson 2 min bound dependency
  • @andrewthad
    • Support edge_ngram tokenizer and token, ngram and edge_ngram token filters
    • Support sum aggregation
  • @mirokuratczyk
    • Point in time (PIT) API support

0.19.0.0

  • @aveltras
    • Add InnerHits
    • Add GHC 9.2.1 support (aeson 2 support)
  • @blackheaven
    • Add Index mappings limits
    • Taking maintainer role
    • Drop GHC 8.0 - 8.8 support
    • Use generic-random instead of quickcheck-arbitrary-template

0.18.0.0

  • @blackheaven
    • Fix deleteDocument endpoint
  • @iivanbakel
    • Support ES7
  • @wireapp
    • Support GHC 8.8
    • Support GHC 8.4
    • Support GHC 8.2.2
    • Support GHC 8.0
    • Implement API Count
  • @bitemyapp
    • Add BulkUpsert
  • @JoseD92
    • Add support for unassigned.node_left.delayed_timeout index setting

0.17.0.0

  • @bitemyapp
    • Removed V1 modules and support.
    • Removed the cabal file, replaced with an Hpack package.yaml.
  • @JoseD92
    • Removed the disused Seminearring class.
    • Added Docker Compose support.
    • Add support for unassigned.node_left.delayed_timeout index setting
  • @Kheldar, @AlexeyRaga, @dsturnbull
    • Added support for BulkUpsert operations that allow for additional metadata.

0.16.0.0

  • @bitemyapp
    • Reorganized V5 modules internally, ripped out Generic, rewrote part of the test suite.
    • Added message field to EsProtocolException
    • Switch from Data.Version to Data.SemVer to support parsing broader version format.
    • Drop support for GHC < 8.0
    • Remove upper bound on aeson.
  • @mxavier
    • Reorganized V1 modules, mirroring the work done on V5.
  • @andrewthad
    • Added support for autogenerated elasticsearch ids in the bulk API
    • Added support for token filters
    • Added action for searching multiple indices
  • @lunaris
    • Added support for scripts fields and function score queries
  • @bermanjosh
    • Added support for direct generators
  • @rvl
  • @ahodgen
    • Add support for wildcard queries
  • @ashutoshrishi
    • Added fuzziness option to a Match Query
    • Added support for “AUTO” Fuzziness alongside a numeric value.

0.15.0.2

  • @michaelxavier
    • Allow http-types 0.12

0.15.0.1

  • @bitemyapp
    • Allow http-types 0.11
    • Removed unused oPath function

0.15.0.0

  • @jagare

    • #204 Fix bug where Location was used instad of LatLon
    • #207 Add instructions on how to run example code

  • @andrewthad

    • #200 Add function for deleting alias
    • #201 Add BulkCreateEncoding, which allows use of aeson’s Encoding for faster bulk creates
    • #203 Add support for total fields limit

  • @chrissound

    • #196 Add missing import to example

  • @tmcgilchrist

    • #186 Fix warnings and drop redundant constraints

  • @23Skidoo

    • #183 Bump dependencies to support GHC 8.2

  • @bermanjosh

    • #182 Support for Suggesters and Template Queries

  • @dawei-dev

    • #181 Fix typo

0.14.0.0

  • @bitemyapp

    • Aeson 1.2

  • @shmish111

    • #177 Added stats aggregation

  • @justinwhite

    • #176 Fixed typo

  • @WraithM

    • #175 Sorted version of getInitialScroll for V5 client

0.13.0.0

  • @michaelxavier

    • #174 doctests are bad
    • #171 Added Semigroup instances
    • #165 Aeson 1.1

  • @bermanjosh

    • #168 Sub-aggregation support
    • #167 Limit QuickCheck to 500 checks

0.12.1.0

  • @michaelxavier
    • #162 Allow Aeson 1.1
  • @bermanjosh
    • #159 Fix bug with tie breaker json instance for MultiMatchQuery

0.12.0.0

This release contains several new APIs, documentation fixes, and some minor type updates, and steps towards ES 2.0 support. Thanks to all of our contributors!

  • @bermanjosh
    • #139 Make nodeOSLoad record a Maybe for Windows compatibility
    • #138 Change the nodePluginVersion record to deal with plugins which report NA as their version.
  • @phadej
    • #155 Shed derive dependency in test.
    • #154 Switch to Simple build type.
    • #153 Loosen deps on aeson, hspec.
  • @andrewthad
    • #151 Add waitForYellowIndex
  • @michaelxavier
    • #150 Fix some failing QuickCheck tests.
    • #147 Drop dependency on deprecated doctest-prop.
    • #137 Add node stats API.
    • #128 Add nodes API.
    • #127 Add haddock sections to break up API.
    • #126 Add snapshot/restore API.
    • #125 Set lower bound for http-client
    • #123 Allow replica count of 0 in smart constructor.
  • @23Skidoo
    • #143 Fix an example.
    • #142 Docs cleanup.
  • @alistair
    • #135 Changes date formatting to have leading zeroes to bring us closer to ES 2.0 support.
    • #133 Fix parent child tests for ES 2.0.
    • #132 Parse status API from ES 2.0.
    • #131 Add cardinality aggregations.
  • @dinnu93
    • #130 Add OverloadedStrings to first example.
  • @MaxGabriel
    • #117 Fix haddocks italicizing forward slashes.

0.11.1.0

  • @bitemyapp
    • Add http-client-0.5.0 support

0.11.0.0

Thanks to the following people, Bloodhound 0.10.0.0 is being released! This one gets a bit messy due to the Aeson 0.11 changeover, but this should be good to go now. Note that Aeson 0.11 returned to Aeson 0.9’s behavior and semantics.

  • @MichaelXavier

    • #112 List indices support
    • #94 Implement index optimization
    • #91 Make respIsTwoHunna more semantic
      • More detail: This is actually the cause of a bug in real code. If you happen to be using parseEsResponse (which uses respIsTwoHunna) to parse the result of certain operations such as creating an index, those operations return a 201 and unjustly are deemed to be a failure.
    • Cleaned up errant Haskell tokens in README
    • #84 Added request auth hooks

  • @dzhus / @MailOnline

    • #85 Add updateDocument

  • @ReadmeCritic

    • #108 Update README URLs based on HTTP redirects

  • @MHova

    • #105 Add helper data types and functions for Missing Aggregations
    • Removed unused server versions from the tests
    • Updated readme to reflect actual ES versions supported and tested
    • Added support for parsing results of Missing Aggregations
    • #104 Export BucketValue
    • #102 Add local testing instructions to the README
    • #89 Support Bool and Numeric keys in TermsResults
    • Added Missing Aggregation support
    • #98 Improve EsProtocolException documentation for human error
    • Updated README to warn about 2.0 compatibility
    • Fix docs specifying an incorrect terminating condition

  • @bitemyapp

    • Merge monkey, puzzled over spurious local doctest failures

0.10.0.0

Thanks to the following people, Bloodhound 0.10.0.0 is being released! This one gets a bit messy due to the Aeson 0.10 upgrade, so you may want to wait for the dust to settle. YMMV.

  • @MichaelXavier

    • #77: Add test for error parsing
    • #76/#78: Support for updating (modifying) index settings
    • #79/#80: Index aliases
    • #81: Low-level scroll API support
    • #82: Date range aggregation

  • @bitemyapp

    • Fucked around with dependencies and broke things in order to upgrade to Aeson 0.10
    • Please forgive me.

0.9.0.0

Thanks to the following people, Bloodhound 0.9.0.0 is being released!

  • @MichaelXavier

    • #75: A more explicit type for errors
    • #74: Add readme and changelog to extra source files

  • @MaxDaten

    • #38/#73 Provide safety by using URL-encoding

  • @centromere

    • #72 Added parent support to documentExists

0.8.0.0

Thanks to the following people, Bloodhound 0.8.0.0 is being released!

  • @MichaelXavier

    • #67: Deriving Monad(Throw|Catch|Mask)
    • #64: Export BH constructor
    • #61: Filter aggregation support
    • #60: Add value_count aggregation support
    • #58: Eliminate partiality in EsResult

  • @centromere

    • #59: Fixed bug with IndexSettings serialization
    • #56: Added fields support to Search
    • #55: Added ability to specify a parent document
    • #54: Fixed IndexTemplate serialization bug
    • #52: Added ability to manipulate templates
    • #51: Fixed mapping API
    • #50: Fixed problem with put sending POST

  • @bermanjosh

    • #63: Url query encoding bug
    • #53: Scan type

  • @sjakobi

    • #69: Replace Control.Monad.Error with CM.Except via mtl-compat
    • #70: Silence redundant import warning with base-4.8.*
    • #71: Use “newManager” instead of deprecated “withManager”

0.7.0.0

  • Added QueryFilter thanks to Bjørn Nordbø!

  • Support for optimistic concurrency control thanks again to @MichaelXavier!

0.6.0.1

  • Allow Aeson 0.9

0.6.0.0

  • Moved to BHMonad, thanks to @MichaelXavier! Now there’s a reader of config information and IO is lifted.

  • SearchHits have a Monoid now, makes combining search results nicer, allows for defaulting when a search cannot be performed.

0.5.0.0

  • Fixed and changed TermsQuery (This caused the major bump)

  • Removed benchmarks from travis.yml

  • Added doctests, examples for Database.Bloodhound.Client. Haddocks should be much nicer.

  • Various fixes, reformatting

0.4.0.0

  • Term and date aggregation - thanks to Christopher Guiney! (@chrisguiney)

Following three thanks to Liam Atkins (@latkins)

0.3.0.0

  • Status “ok” changed from Bool to Maybe Bool thanks to @borisyukd

  • Elasticsearch 1.3.x compatibility fixed with changes to geo bounding boxes - thanks to Curtis Carter! (@ccarter)

  • CI coverage expanded to 1.0.x -> 1.3.x

0.2.0.1

  • Killed off maybeJson/mField/catMaybes in favor of omitNulls

  • Experimenting with RecordWildcards

  • Merged Types and Instances module into Types to prevent possibility of orphans and cull orphan instance warnings.

  • Added note about current supported Elasticsearch version.

0.2.0.0

  • Added TermFilter

  • Renamed createMapping to putMapping

  • Fixed and rebuilt documentation

  • RegexpFlags changed to a sum type instead of Text, thanks to @MichaelXavier!