PyF
Quasiquotations for a python like interpolated string formatter
LTS Haskell 22.33: | 0.11.3.0 |
Stackage Nightly 2024-09-07: | 0.11.3.0 |
Latest on Hackage: | 0.11.3.0 |
PyF-0.11.3.0@sha256:33522bc0ae6adc6e1fbd5d54ec8b2d72b1a39d1d3090db0ea809091fabf78f44,2781
Module documentation for 0.11.3.0
PyF
PyF is a Haskell library for string interpolation and formatting.
PyF exposes a quasiquoter fmt
which introduces string interpolation and formatting with a mini language inspired by printf and Python.
Quick Start
>>> :set -XQuasiQuotes
>>> import PyF
>>> name = "Dave"
>>> age = 54
>>> [fmt|Person's name is {name}, age is {age}|]
"Person's name is Dave, age is 54"
The formatting mini language can represent:
- Numbers with different representations (fixed point, general representation, binary, hexadecimal, octal)
- Padding, with the choice of padding char, alignment (left, right, around, between sign and number)
- Sign handling, to display or not the
+
for positive number - Number grouping
- Floating point representation
- The interpolated value can be any Haskell expression.
You will need the extension QuasiQuotes
, enable it with {-# LANGUAGE QuasiQuotes #-}
in top of your source file or with :set -XQuasiQuotes
in your ghci
session. ExtendedDefaultRules
and OverloadedStrings
may be more convenient.
Expression to be formatted are referenced by {expression:formattingOptions}
where formattingOptions
follows the Python format mini-language. It is recommended to read the Python documentation, but the Test file as well as this readme contain many examples.
More Examples
Padding
Left <
/ Right >
/ Around ^
padding:
>>> name = "Guillaume"
>>> [fmt|{name:<11}|]
"Guillaume "
>>> [fmt|{name:>11}|]
" Guillaume"
>>> [fmt|{name:|^13}|]
"||Guillaume||"
Padding inside =
the sign
>>> [fmt|{-3:=6}|]
"- 3"
Float rounding
>>> [fmt|{pi:.2}|]
"3.14"
Binary / Octal / Hex representation (with or without prefix using #
)
>>> v = 31
>>> [fmt|Binary: {v:#b}|]
"Binary: 0b11111"
>>> [fmt|Octal: {v:#o}|]
"Octal: 0o37"
>>> [fmt|Octal (no prefix): {v:o}|]
"Octal (no prefix): 37"
>>> [fmt|Hexa (caps and prefix): {v:#X}|]
"Hexa (caps and prefix): 0X1F"
Grouping
Using ,
or _
.
>>> [fmt|{10 ^ 9 - 1:,}|]
"999,999,999"
>>> [fmt|{2 ^ 32 -1:_b}|]
"1111_1111_1111_1111_1111_1111_1111_1111"
Sign handling
Using +
to display the positive sign (if any) or
to display a space instead:
>>> [fmt|{pi:+.3}|]
"+3.142"
>>> [fmt|{-pi:+.3} (Negative number)|]
"-3.142 (Negative number)"
>>> [fmt|{pi: .3}|]
" 3.142"
>>> [fmt|{-pi: .3} (Negative number)|]
"-3.142 (Negative number)"
0
Preceding the width with a 0
enables sign-aware zero-padding, this is equivalent to inside =
padding with a fill char of 0
.
>>> [fmt|{10:010}|]
0000000010
>>> [fmt|{-10:010}|]
-000000010
Sub-expressions
First argument inside the curly braces can be a valid Haskell expression, for example:
>>> [fmt|2pi = {2* pi:.2}|]
2pi = 6.28
>>> [fmt|tail "hello" = {tail "hello":->6}|]
"tail \"hello\" = --ello"
However the expression must not contain }
or :
characters.
Combined
Most options can be combined. This generally leads to totally unreadable format string ;)
>>> [fmt|{pi:~>5.2}|]
"~~3.14"
Multi-line strings
You can ignore a line break with \
if needed. For example:
[fmt|\
- a
- b\
|]
Will returns "- a\n- b"
. Note how the first and last line breaks are ignored.
Arbitrary value for precision and padding
The precision and padding width fields can be any Haskell expression (including variables) instead of a fixed number:
>>> [fmt|{pi:.{1+2}}|]
3.142
>>> [fmt|{1986:^{2 * 10}d}|]
" 1986 "
Output type
PyF aims at extending the string literal syntax. As such, it default to String
type. However, if the OverloadedString
is enabled, PyF will happilly generate IsString t => t
instead. This means that you can use PyF to generate String
, but also Text
and why not ByteString
, with all the caveats known to this extension.
>>> [fmt|hello {pi.2}|] :: String
"hello 3.14"
Custom types
PyF can format three categories of input types:
- Floating. Using the
f
,g
,e
, … type specifiers. Any type instance ofRealFloat
can be formated as such. - Integral. Using the
d
,b
,x
,o
, … type specifiers. Any type instance ofIntegral
can be formated as such. - String. Using the
s
type specifier. Any type instance ofPyFToString
can be formated as such.
See PyF.Class
if you want to create new instances for the PyFToString
class.
By default, if you do not provide any type specifier, PyF uses the PyFClassify
type class to decide if your type must be formated as a Floating, Integral or String.
Caveats
Type inference
Type inference with numeric literals can be unreliable if your variables are too polymorphic. A type annotation or the extension ExtendedDefaultRules
will help.
>>> v = 10 :: Double
>>> [fmt|A float: {v}|]
A float: 10
Error reporting
Template Haskell is generally known to give developers a lot of frustration when it comes to error messages, dumping an unreadable piece of generated code.
However, in PyF, we took great care to provide clear error reporting, this means that:
- Any parsing error on the mini language results in a clear indication of the error, for example:
foo = [fmt|{age:.3d}|]
File.hs:77:19: error:
|
1 | {age:.3d}
| ^
Type incompatible with precision (.3), use any of {'e', 'E', 'f', 'F', 'g', 'G', 'n', 's', '%'} or remove the precision field.
Note: error reporting uses the native GHC error infrastructure, so they will correctly appear in your editor (using HLS), for example:
- Error in variable name are also readable:
test/SpecUtils.hs:81:33: error:
• Variable not found: chien
• In the quasi-quotation: [fmt|A missing variable: {chien}|]
|
81 | fiz = [fmt|A missing variable: {chien}|]
| ^^^^^
- However, if the interpolated name is not of a compatible type (or too polymorphic), you will get an awful error:
>>*> [fmt|{True:d}|]
<interactive>:80:10: error:
• No instance for (Integral Bool)
arising from a use of ‘PyF.Internal.QQ.formatAnyIntegral’
...
- There is also one class of error related to alignement which can be triggered, when using alignement inside sign (i.e.
=
) with string:
*PyF PyF.Internal.QQ> [fmt|{"hello":=10}|]
<interactive>:89:10: error:
• String Cannot be aligned with the inside `=` mode
...
- Finally, if you make any type error inside the expression field, you are on your own, you’ll get an awful error in the middle of the generated Template Haskell splice.
>>> [fmt|{3 + pi + "hello":10}|]
<interactive>:99:10: error:
• No instance for (Floating [Char]) arising from a use of ‘pi’
...
Custom Delimiters
If {
and }
do not fit your needs, for example if you are formatting a lot of JSON, you can use custom delimiters. All quasi quoters have a parametric form which accepts custom delimiters. Due to the Template Haskell stage restriction, you must define your custom quasi quoter in another module.
For example, in MyCustomDelimiter.hs
:
module MyCustomQQ where
import Language.Haskell.TH.Quote
import PyF
myCustomFormatter :: QuasiQuoter
myCustomFormatter = mkFormatter "fmtWithDelimiters" (fmtConfig {
delimiters = ('@','!')
})
Later, in another module:
import MyCustomQQ
-- ...
[myCustomFormatter|pi = @pi:2.f!|]
Escaping still works by doubling the delimiters, @@!!@@!!
will be formatted as @!@!
.
Have a look at PyF.mkFormatter
for all the details about customization.
Differences with the Python Syntax
The implementation is unit-tested against the reference Python implementation (Python 3.6.4) and should match its result. However some formatters are not supported or some (minor) differences can be observed.
Not supported
- Number
n
formatter is not supported. In Python this formatter can format a number and use current locale information for decimal part and thousand separator. There is no plan to support that because of the impure interface needed to read the locale. - Python supports sub variables in the formatting options in all places, such as
{expression:.{precision}}
. We only support it forprecision
andwidth
. This is more complexe to setup for others fields. - Python literal integers accept binary/octal/hexa/decimal literals, PyF only accept decimal ones, I don’t have a plan to support that, if you really need to format a float with a number of digit provided as a binary constant, open an issue.
- Python supports adding custom formatters for new types, such as date. This may be really cool, for example
[fmt|{today:%Y-%M-%D}
. I don’t know how to support that now.
Differences
- General formatters g and G behaves a bit differently. Precision influence the number of significant digits instead of the number of the magnitude at which the representation changes between fixed and exponential.
- Grouping options allows grouping with an
_
for floating point, Python only allows,
. - Custom delimiters
Build / test
Should work with stack build; stack test
, and with cabal
and (optionally) nix
:
nix-shell # Optional, if you use nix
cabal build
cabal test
There are a few available shells for you.
nix-shell
is the default, current GHC version with language server available.nix-shell ./. -A pyf_xx.shell
is another GHC version (changexx
) without language server.nix-shell ./. -A pyf_xx.shell_hls
is another GHC version (changexx
) with language server.
We also provide a few utility functions:
nix-build ./ -A hlint
will check hlint.nix-shell ./ -A ormolu-fix
will format the codebase.
Library note
PyF.Formatters
exposes two functions to format numbers. They are type-safe (as much as possible) and comes with a combination of formatting options not seen in other formatting libraries:
>>> formatIntegral Binary Plus (Just (20, AlignInside, '~')) (Just (4, ',')) 255
"+~~~~~~~~~~1111,1111"
GHC compatibility
This library is tested in CI with ghc 8.6 to 9.2.
Conclusion
Don’t hesitate to make any suggestion, I’ll be more than happy to work on it.
Hacking
Everything works with nix and flakes. But you can also try with manual cabal / stack if you wish.
nix develop
will open a shell with everything you need to work on PyF, including haskell-language-server. It may be a bit too much, so you can instead:nix develop .#pyf_XY
opens a shell with a specific GHC version and without haskell-language-server. That’s mostly to test compatibility with different GHC version or open a shell without HLS if you are in a hurry. Replacepyf_XY
bypyf_86
,pyf_88
,pyf_810
,pyf_90
orpyf_92
.
Once in the shell, use cabal build
, cabal test
, cabal repl
.
There is a cachix available, used by CI, and already configured in flakes. You can manually run cachix use guibou
if you want.
You can locally build and test everything using:
nix build .#pyf_all
.
Don’t hesitate to submit a PR not tested on all GHC versions.
Formatting
The codebase is formatted with ormolu
. Please run:
nix run .\#run-ormolu
Before submitting.
Treesitter support
Have a look in the ./tree-sitter-pyf/ directory for a parser of PyF which can be integrated in your treesitter compatible editor to get syntax highlighting for PyF.
Changes
Revision history for PyF
0.11.3.0 – 2024-05-15
- Support for GHC 9.10.
0.11.2.1 – 2023-10-25
- Final version for GHC 9.8
0.11.2.0
- Fix for the neovim treesitter syntax highlighter for
fmt
andfmtTrim
quasiquotes - Initial support for GHC 9.8
- Version bump for new MTL
0.11.1.1 – 2023-03-15
- Support for GHC 9.6. Thank you @Kleidukos for initiating the port.
0.11.1.0 – 2022-09-24
- Support for OverloadedRecordsDot syntax in Meta. Thank you @Profpatsch for the report.
- In some context, the error reporting for variable not found in the quasi quote expression was incorrectly reporting existing variables as not found. See https://github.com/guibou/PyF/issues/115 for details. This is now fixed by not abusing GHC api. Thank you @michaelpj for reporting this really weird problem.
0.11.0.0 – 2022-08-10
- Support for GHC 9.4. (Written with a pre-release of GHC 9.4, hopefully it won’t change too much before the release).
- Error reporting now uses the native GHC API. In summary, it means that haskell-language-server will point to the correct location of the error, not the beginning of the quasi quotes.
- PyF will now correctly locate the error for variable not found in expression, even if the expression is complicated. The support for complex expression is limited, and PyF may return a false positive if you try to format a complex lambda / case expression. Please open a ticket if you need that.
- Add support for literal
[]
and()
in haskell expression. - Add support for overloaded labels, thank you Shimuuar.
- Support for
::
in haskell expression. Such as[fmt| 10 :: Int:d}|]
, as a suggestion from julm (close #87). Integral
padding width and precision also for formatter without type specifier.- Extra care was used to catch all
type-defaults
warning message. PyF should not generate code with this kind of warning, unless the embedded Haskell expression are ambiguous (e.g.[fmt|{10}|]
). You can use::
to disambiguate, e.g.[fmt|{10 :: Int}|]
.
0.10.1.0 – 2021-12-05
- Padding width can now be any arbitrary Haskell expression, such as
[fmt|hello pi = {pi:<{5 * 10}}|]
. - Precision (and now padding width) arbitrary expression can now be any
Integral
and it is not limited toInt
anymore. - (Meta): type expression are now parsed and hence allowed inside arbitrary Haskell expression for padding width and precision. For example,
[fmt|Hello {pi:.{3 :: Int}}|]
.
0.10.0.1 – 2021-10-30
-
Due to the dependencies refactor,
PyF
no have no dependencies other than the one packaged with GHC. The direct result is thatPyF
build time is reduced to 6s versus 4 minutes and 20s before. -
Remove the dependency to
megaparsec
and replaces it byparsec
. This should have minor impact on the error messages. -
Huge Change. The parsing of embeded expression does not depend anymore on
haskell-src-ext
andhaskell-src-meta
and instead depends on the built-inghc
lib. -
Added instances for
(Lazy)ByteString
toPyFClassify
andPyFToString
.ByteString
can now be integrated into format string, and will be decoded as ascii. -
Relax the constraint for floating point formatting from
RealFrac
toReal
. As a result, a few new type can be formatted as floating point number. One drawback is that someIntegral
areReal
too and hence it is not an error anymore to format an integral as floating point, but you still need to explicitly select a floating point formatter. -
Added instance for
(Nominal)DiffTime
toPyFClassify
, so you can now format them without conversion. -
Introducing of the new typeclass
PyfFormatIntegral
andPyfFormatFractional
in order to customize the formatting for numbers. An instance is derived for respectively anyIntegral
andReal
types. -
Support for
Char
formatting, as string (showing theChar
value) or as integral, showing theord
. -
Data.Ratio
. -
Introducing
fmtTrim
module. It offers the same behavior asfmt
, but trims common indentation. SePyF.trimIndent
for documentation. -
Introducing
raw
for convenience. It is a multiline string without any escaping, formatting neither leading whitespace handling. -
Introducing
str
andstrTrim
. They are similar tofmt
andfmtTrim
but without formatting. You can see them as multiline haskell string, with special character escaping, but without formatting. For convenience, thestrTrim
version also removes indentation. -
fmtWithDelimiters
is gone and replaced bymkFormatter
inPyF
which is “more” generic.
0.9.0.3 – 2021-02-06
- Test phase do not depend anymore on python (unless cabal flag
python_test
is set). This ease the deployment / packaging process.
0.9.0.2 – 2020-09-11
- Version bump for megaparsec 9.0
0.9.0.1 – 2020-03-25
- Fixs for GHC 8.10
0.9.0.0 – 2019-12-29
- Any type with
Show
instance can be formatted using:s
formatter. For example,[fmt|hello {(True, 10):s}|]
. This breaks compatibility because previous version of PyF was generating an error when try to format to string anything which was not a string, now it accepts roughly anything (with aShow
instance).
0.8.1.2 – 2019-11-08
- Bump megaparsec bounds
0.8.1.1 – 2019-10-13
- Compatibility with GHC 8.8
0.8.1.0 – 2019-09-03
- Precision can now be any arbitrary haskell expression, such as
[fmt|hello pi = {pi:.{1 + 3}}|]
.
0.8.0.2 – 2019-08-27
- (minor bugfix in tests): Use python3 instead of “python” to help build on environment with both python2 and python3
0.8.0.1 – 2019-08-27
- Stack support
0.8.0.0 – 2019-08-06
f
(andfWithDelimiters
) were renamedfmt
(fmtWithDelimiters
).f
was causing too much shadowing in any codebase.- PyF now exposes the typeclass
PyFToString
andPyFClassify
which can be extended to support any type as input for the formatters. - PyF now uses
Data.String.IsString t
as its output type ifOverloadedString
is enabled. It means that it behaves as a real haskell string literal. - A caveat of the previous change is that PyF does not have instances for
IO
anymore.
bugfixes and general improvements
- An important amount of bugfixs
- Error reporting for generic formatting (i.e. formatting without a specified type) is now more robust
- Template haskell splices are simpler. This leads to more efficient / small generated code and in the event of this code appears in a GHC error message, it is more readable.
- PyF now longer emit unnecessary default typing.
0.7.3.0 – 2019-02-28
- Tests: fix non reproducible tests
0.7.2.0 – 2019-02-27
- Fixed: PyF now uses the same haskell extensions as the one used by the current haskell file to parse sub expressions.
0.7.1.0 – 2019-02-11
- Fixed: PyF was wrongly ignoring everything located after a non-doubled closing delimiter.
- New Feature: line break can be escaped with , thus allowing string to start on a new line ignoring the initial backspace
0.7.0.0 – 2019-02-04
- Bump dependencies to megaparsec 7
- Error message are now tested
- Name in template haskell splices are stable. This increases readability of error messages
- Better error message for badly formated expression
Formatting removal
- All monomorphic quasiquoters (
f
,fString
,fText
,fIO
,fLazyText
) are removed - Polymophic quasiquoter
f'
is renamedf
and is the only entry point. Monomorphic users are encouraged to use the polymorphic quasiquoter with type annotation. Formatting
dependency is removed.- Previously named
f
quasiquoters which was exporting toFormatting.Format
is removed. User of this behavior should useFormatting.now
instead.
0.6.1.0 – 2018-08-03
- Custom delimiters, you can use whatever delimiters you want in place of
{
and}
.
0.6.0.0 – 2018-08-02
- Fix the espace parsing of
{{
and}}
as{
and}
0.5.0.0 – 2018-04-16
- Support for negative zero
- Support for 0 modifier
- Exponential formatter now behaves as python
- Support for alternate floatting point represenation
- Lot of documentation
- Test are auto verified with the python reference implementation
0.4.0.0 – 2018-04-13
- Support for grouping option
- Support for inner allignment
- Correct display of NaN and Infinity
- Fix a few cosmetic with python implementation
- Introduce
PyF.Formatters
, type safe generic number formatter solution - Remove dependency to
scientific
0.3.0.0 – 2018-04-01
- Support for haskell subexpression
0.1.1.0 – 2018-01-07
- Add support for the
sign
field.
0.1.0.0 – 2018-01-03
- First version. Released on an unsuspecting world.