zenacy-unicode

Unicode utilities for Haskell

Version on this page:	1.0.0
LTS Haskell 22.43:	1.0.2
Stackage Nightly 2023-12-26:	1.0.2
Latest on Hackage:	1.0.2

See all snapshots zenacy-unicode appears in

MIT licensed and maintained by Michael Williams

This version can be pinned in stack with:zenacy-unicode-1.0.0@sha256:ed4c159e1dd7b9fbdd46c2b74f87a9ae5feace6786d54b1702384c0cf35331db,1655

Module documentation for 1.0.0

Zenacy
- Zenacy.Unicode

Depends on 4 packages(full list with versions):

base, bytestring, vector, word8

Zenacy Unicode

Zenacy Unicode includes tools for checking byte order marks (BOM) and cleaning data to remove invalid bytes. These tools can help ensure that data pulled from the web can be parsed and converted to text.

The following is an example of converting dubious data to a text.

textDecode :: ByteString -> Text
textDecode b =
  case bomStrip b of
    (Nothing, s)           -> T.decodeUtf8 $ unicodeCleanUTF8 s -- Assume UTF8
    (Just BOM_UTF8, s)     -> T.decodeUtf8 $ unicodeCleanUTF8 s
    (Just BOM_UTF16_BE, s) -> T.decodeUtf16BE s
    (Just BOM_UTF16_LE, s) -> T.decodeUtf16LE s
    (Just BOM_UTF32_BE, s) -> T.decodeUtf32BE s
    (Just BOM_UTF32_LE, s) -> T.decodeUtf32LE s

Changes

Change Log

1.0.0

Initial FOSS release