hw-json-simd

SIMD-based JSON semi-indexer

https://github.com/haskell-works/hw-json-simd#readme

LTS Haskell 23.1:0.1.1.2@rev:1
Stackage Nightly 2024-12-21:0.1.1.2@rev:1
Latest on Hackage:0.1.1.2@rev:1

See all snapshots hw-json-simd appears in

BSD-3-Clause licensed by John Ky
Maintained by [email protected]
This version can be pinned in stack with:hw-json-simd-0.1.1.2@sha256:eb4fde4410b8384b60772aff5571537d2dd8030a4d370c91e8ef512180ac06e8,5863

Module documentation for 0.1.1.2

Used by 1 package in nightly-2024-07-18(full list with versions):
hw-json-simd

hw-json-simd

This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple

Installation

From hackage

cabal new-install hw-json-simd

Changes

Changelog for hw-json-simd

Unreleased changes