spacecookie
Gopher server library and daemon
https://github.com/sternenseemann/spacecookie
LTS Haskell 22.9: | 1.0.0.2 |
Stackage Nightly 2023-12-26: | 1.0.0.2 |
Latest on Hackage: | 1.0.0.2 |
spacecookie-1.0.0.2@sha256:499ebafa27578efd9cd41e75731ed6fdefeec7d5f0b21a1f494d54481f2c5a22,4232
Module documentation for 1.0.0.2
spacecookie
Haskell gopher server daemon and library.
Features
- implements RFC1436
- optionally supports common protocol extensions:
- informational entries via the
i
-type h
-type and URL entries
- informational entries via the
- supports gophermaps (see below)
- supports systemd socket activation
- provides a library for custom gopher applications (see documentation)
Non-Features
spacecookie intentionally does not support:
- HTTP, Gemini: Multi protocol support is a non-goal for spacecookie. For HTTP you can proxy pretty easily, however.
- Search: Gopher supports search transactions, but the spacecookie daemon doesn’t offer the possibility to add a search engine to a gopherspace. It is however entirely possible to implement an index search server using the spacecookie library
Installation
- Nix(OS):
pkgs.haskellPackages.spacecookie
(see also below) - Cabal:
cabal v2-install spacecookie
(see also hackage package)
Documentation
- User Documentation: spacecookie(1)
- Developer Documentation
Configuration
spacecookie is configured via a JSON configuration file.
All available options are documented in
spacecookie.json(5).
This repository also contains an example configuration file in
etc/spacecookie.json
.
Running
After you’ve created your config file just start spacecookie like this:
spacecookie /path/to/spacecookie.json
spacecookie runs as a simple process and doesn’t fork or write a PID file. Therefore any supervisor (systemd, daemontools, …) can be used to run it as a daemon.
With systemd
spacecookie supports systemd socket activation. To set it up you’ll need
to install spacecookie.service
and spacecookie.socket
like so:
cp ./etc/spacecookie.{service,socket} /etc/systemd/system/
systemctl daemon-reload
systemctl enable spacecookie.socket
systemctl start spacecookie.socket
systemctl start spacecookie.service # optional, started by the socket automatically if needed
Of course make sure that all the used paths are correct!
How the systemd integration works is explained in spacecookie(1).
On NixOS
NixOS provides a service module for spacecookie:
services.spacecookie
.
Setting up spacecookie is as simple as adding the following line to your configuration.nix
:
services.spacecookie.enable = true;
For all available options, refer to the NixOS manual:
Adding Content
spacecookie acts as a simple file server, only excluding files or directories that start with a dot. It generates gopher menus automatically, but you can also use custom ones by adding a gophermap file.
spacecookie checks for .gophermap
in every directory it serves and,
if present, uses the menu specified in there.
Such a file looks like this:
You can just start writing text that
will be displayed by the gopher client
without a link to a file. Empty lines are
also possible.
1Menu Entry for a directory full of funny stuff /funny
IFunny Image /funny.jpg
gcat gif /cat.gif
0about me /about.txt
1Floodgap's gopher server / gopher.floodgap.com 70
As you can see, it largely works like the actual gopher menu a server will send to clients, but allows to omit redundant information and to insert lines that are purely informational and not associated with a file. spacecookie.gophermap(5) explains syntax and semantics in more detail.
The format is compatible with the ones supported by Bucktooth and pygopherd. If you notice any incompatibilities, please open an issue.
Portability
spacecookie is regularly tested on GNU/Linux via CI, but should also work on other Unix-like operating systems. Most portability problems arise due to haskell-socket which is for example known not to work on OpenBSD.
Windows support would be possible, but could be tricky as gopher expects Unix-style directory separators in paths. I personally don’t want to invest time into it, but would accept patches adding Windows support.
Changes
Revision history for spacecookie
1.0.0.2
2022-10-03
- Work around cabal#8458,
ensuring that the test suite can be compiled with
cabal-install
3.8.1.0. - Always compile test suite with
-threaded
to avoid randomCurlBadFunctionArg
exceptions when executing integration tests.
1.0.0.1
2021-11-29
This release fixes compilation with aeson >= 2.0
.
1.0.0.0
2021-03-16
TL;DR:
- For server users, new features and configuration options have been added, but old configuration stays compatible. However some gophermap files may need adjusting, especially if they contain absolute paths not starting with a slash.
- For library users there are multiple braking changes to the core API that probably need adjusting in downstream usage as well as some changes to behavior.
Server and Library Users
Gophermap parsing
There have been quite a few, partly breaking changes to gophermap parsing in the library in an effort to fully support the format used in pygopherd and bucktooth. Instances where spacecookie’s parsing deviated from the established format have been resolved and we now ship a test suite which checks compliance against sample files from bucktooth and pygopherd.
We now support relative paths correctly: If a selector in a gophermap doesn’t
start with /
or URL:
it is interpreted as a relative path and prefixed
with the directory the gophermap is located in. This should make writing
gophermaps much more convenient, as it isn’t necessary to enter absolute
selectors anymore. However, absolute selectors not starting with /
are broken by this.
To facilitate these changes, the API of Network.Gopher.Util.Gophermap
changed in the following way:
GophermapEntry
changed to useGophermapFilePath
instead ofFilePath
which may either beGophermapAbsolute
,GophermapRelative
orGophermapUrl
. Additionally,GophermapFilePath
is a wrapper aroundRawFilePath
, contrary to the previous use ofFilePath
.gophermapToDirectoryResponse
takes an additional parameter describing the directory the gophermap is located in to resolve relative to absolute selectors.
Menu lines which only contain a file type and name are now required to be
terminated by a tab before the newline. This also reflects the behavior
of bucktooth and pygopherd (although the latter’s documentation on this
is a bit misleading). Although this breaks entries like 0/file
,
info lines which start with a valid file type character like
1. foo bar baz
no longer get mistaken for normal menu entries.
See #34.
The remaining, less significant changes are:
- Fixed parsing of gophermap files whose last line isn’t terminated by a newline.
- The
gophermaplineWithoutFileTypeChar
line type which mapped menu entries with incompatible file type characters to info lines has been removed. Such lines now result in a parse error. This is a breaking change if you relied on this behavior. parseGophermap
now consumes the end of input.
Changes to Connection Handling
- We now wait up to 1 second for the client to close the connection on
their side after sending all data. This fixes an issue specific to
curl
which would result in it failing with a recv error (exit code 56) randomly. See also #42 and #44. - Requests from clients are now checked more vigorously and limited
in size and time to prevent denial of service attacks.
- Requests may not exceed 1MB in size
- The client is only given 10s to send its request
- After the
\r\n
no additional data may be sent
Server Users
Configuration
- Add new
listen
field to configuration allowing to specify the listening address and port. It expects an object with the fieldsport
andaddr
. The top levelport
option has been deprecated as a result. It is now possible to bind to the link local address::1
only without listening on public addresses. See #13 and #19. - Log output is now configurable via the new
log
field in the configuration. Likelisten
it expects an object which supports the following fields. See #10 and #20.enable
allows to enable and disable logginghide-ips
can be used to hide private information of users from log output. This is now enabled by default.hide-time
allows to hide timestamps if your log setup already takes care of that.level
allows to switch betweenerror
andinfo
log level.
- Make
port
andlisten
→port
settings optional, defaulting to 70.
Config parsing should be backwards compatible. Please open a bug report if you experience any problems with that or any constellation of the new settings.
Other changes
- A not allowed error is now generated if there are any dot directories or
dot files along the path:
/foo/.dot/bar
would now generate an error instead of being processed like before. - GHC RTS options are now enabled and the default option
-I10
is passed to spacecookie. - Exit if dropping privileges fails instead of just logging an error like before. See #45.
- Expand user documentation by adding three man pages
(rendered) on the server daemon:
spacecookie(1)
: daemon invocation and behaviorspacecookie.json(5)
: daemon configurationspacecookie.gophermap(5)
: gophermap format documentation
- Fix the file not found error message erroneously stating that access of that file was not permitted.
- Clarify error message when an URL: selector is sent to spacecookie.
- Print version when
--version
is given - Print simple usage instructions when
--help
is given or the command line can’t be parsed. - A warning is now logged when a gophermap file doesn’t parse and the standard directory response is used as fallback.
Library Users
New Representation of Request and Response
The following changes are the most significant to the library as they break virtually all downstream usage of spacecookie as a library.
The gopher request handler for the runGopher
-variants now receives
a GopherRequest
record representing the request instead of the
selector as a String
. The upsides of this are as follows:
- Handlers now know the IPv6 address of the client in question
- Simple support for search transaction is introduced as the request sent by the client is split into selector and search string.
- Selectors are no longer required to be UTF-8 as
ByteString
is used.
If you want to reuse old handlers with minimal adjustments you can use a snippet like the following. Note though that you might have to make additional adjustments due to the changes to responses.
wrapLegacyHandler :: (String -> GopherResponse)
-> (GopherRequest -> GopherResponse)
wrapLegacyHandler f = f . uDecode . requestSelectorRaw
Corresponding to the switch to ByteString
in GopherRequest
the
whole API now uses ByteString
to represent paths and selectors.
This prompts the following additional, breaking changes:
ErrorResponse
now uses aByteString
instead of aString
.GopherMenuItem
’sItem
now uses aByteString
instead of aFilePath
(you can useencodeFilePath
fromfilepath-bytestring
to fix downstream usage).sanitizePath
andsanitizeIfNotUrl
now operate onRawFilePath
s (which is an alias forByteString
).- As already mentioned, the gophermap API uses
RawFilePath
s instead ofFilePath
s as well.
Logging
The built-in logging support has been removed in favor of a log handler the
user can specify in GopherConfig
. This is a breaking change in two ways:
- The type of
GopherConfig
changed as it has a new field calledcLogHandler
. - By default (
defaultGopherConfig
) the spacecookie library no longer has logging enabled.
The motivation for this was to enable the library user to influence the log output more. More specifically the following abilities were to be made possible for the bundled server daemon:
- It should be possible to hide timestamps in the log output: If you are using systemd for example, the journal will take care of those.
- There should be the ability to hide sensitive information from the log output: Unless necessary client IP addresses shouldn’t be logged to disk.
- The log output should be filterable by log level.
- It should be easy for server implementation to also output log messages
via the same system as the
spacecookie
library.
The best solution to guarantee these properties (and virtually any you could
want) is to let the library user implement logging. This allows any target
output, any kind of logging, any kind of clock interaction to generate
timestamps (or not) etc. This is why the spacecookie library no longer
implements logging. Instead it lets you configure a GopherLogHandler
which may also be used by the user application (it is a simple IO
action). This additionally scales well: In the simplest case this could
be a trivial wrapper around putStrLn
.
The second part to the solution is GopherLogStr
which is the string type
given to the handler. Internally this is currently implemented as a Seq
containing chunks of Builder
s which are coupled with meta data. This
should allow decent performance in building and rendering of GopherLogStr
s.
The latter of which is relatively convenient using FromGopherLogStr
.
The tagged chunks are used to allow a clean implementation of hiding sensitive
data: makeSensitive
can be used to tag all chunks of a GopherLogStr
which
will then be picked up by hideSensitive
which replaces all those chunks
with [redacted]
. This way sensitive information can be contained inline in
strings and users can choose at any given point whether it should remain there
or be hidden.
The new logging mechanism was implemented in #29.
Previously it was attempted to make built-in logging more configurable (see #13 and #19), but this was overly complicated and not as flexible as the new solution. Therefore it was scrapped in favor of the new system.
Other Changes
cRunUserName
has been removed fromGopherConfig
since the functionality doesn’t need special treatment as users can implement it easily via the ready action ofrunGopherManual
. The formerly internaldropPrivileges
function is now available viaNetwork.Gopher.Util
to be used for this purpose. See #45. This is a breaking change and requires adjustment if you used the built in privilege deescalation capabilities.santinizePath
andsantinizeIfNotUrl
have been corrected tosanitizePath
andsanitizeIfNotUrl
respectively. This is a breaking change to the interface ofNetwork.Gopher.Util
.
0.2.1.2 Bump fast-logger
2020-05-23
- Bump fast-logger dependency, fix build
0.2.1.1 Fixed Privilege Dropping
2019-12-10
- Server
- Make
user
parameter in config optional. If it is not given or set tonull
,spacecookie
won’t attempt to change its UID and GID. This is especially useful, if socket activation is used. In that case it is not necessary to start spacecookie asroot
since systemd sets up the socket, sospacecookie
can be already started by the right user and doesn’t need to change UID. - Example Systemd config files
SocketMode
is now660
instead of default666
.- Set
User
andGroup
forspacecookie.service
as well. - Set
"user": null
inspacecookie.json
- Make
- Library
- Fixed issue that led to
runGopher*
trying to change UID even if it wasn’t possible (not running as root). This especially affected thespacecookie
server, sincecRunUserName
would always beJust
. - Made logging related to
dropPrivileges
clearer.
- Fixed issue that led to
0.2.1.0 Systemd Support
2019-10-20
- Improved systemd support.
- Support for the notify service type
- Support for socket activation and socket (fd) storage
- To make use of these new features you’ll have to update your service files
- Added
defaultConfig
value to prevent future breakage in software using the library when theGopherConfig
type is extended. - Pretty print IPv6 addresses in logging
0.2.0.1 Hackage release
2019-05-23
Fixed a problem hindering the hackage release.
0.2.0.0 initial release
2019-05-23
- First version. Released on an unsuspecting world. Includes:
- Library for writing any gopher server / application.
- File system based gopher server with support for gopher maps.
- Supports logging, privilege dropping, the gopher protocol and common extensions.