gophernicus/README.md

Gophernicus

This release: Version DEVEL NOTE: The master branch is rolling Development! DO NOT USE unless you want fiery dragons! (you probably want to git checkout 3.1.1)

Copyright (c) 2009-2018 Kim Holviala

Copyright (c) 2019 Gophernicus Developers

Gophernicus is a modern full-featured (and hopefully) secure gopher daemon. It is licensed under the BSD license.

(If you are looking for installation documentation, please see INSTALL.md).

Support/Contact

Developers can be reached at gophernicus AT gophernicus DOT org.

Our IRC channel is on irc.libera.chat #gophernicus.

You most likely want to subscribe to the gophernicus mailing list at https://lists.tildeverse.org/postorius/lists/gophernicus.lists.tildeverse.org/, especially if you maintain a server. This is where all important announcements are made.

Command line options

-h hostname   Change server hostname (FQDN)      [$HOSTNAME]
-p port       Change server port                 [70]
-T port       Change TLS/SSL port                [0 = disabled]
-r root       Change gopher root                 [/var/gopher]
-t type       Change default gopher filetype     [0]
-g mapfile    Change gophermap file              [gophermap]
-a tagfile    Change gophertag file              [gophertag]
-c cgidir     Change CGI script directory        [/cgi-bin/]
-u userdir    Change users personal gopherspace  [public_gopher]
-l logfile    Log to Apache-compatible combined format logfile

-w width      Change default page width          [67]
-o charset    Change default output charset      [UTF-8]

-s seconds    Session timeout in seconds         [1800]
-i hits       Maximum hits until throttling      [4096]
-k kbytes     Maximum transfer until throttling  [4194304]

-f filterdir  Specify directory for output filters
-e ext=type   Map file extension to gopher filetype
-R old=new    Rewrite the beginning of a selector

-D text|file  Set or load server description for caps.txt
-L text|file  Set or load server location for caps.txt
-A admin      Set admin email for caps.txt

-U paths      Specify a colon-separated list of extra unveil(2) paths
              (OpenBSD only).

-nv           Disable virtual hosting
-nl           Disable parent directory links
-nh           Disable menu header (title)
-nf           Disable menu footer
-nd           Disable dates and filesizes in menus
-nc           Disable file content detection
-no           Disable charset conversion for output
-nq           Disable HTTP-style query strings (?query)
-ns           Disable logging to syslog
-na           Disable autogenerated caps.txt
-nt           Disable /server-status
-nm           Disable shared memory use (for debugging)
-nr           Disable root user checking (for debugging)
-np           Disable HAproxy proxy protocol
-nx           Disable execution of gophermaps and scripts
-nu           Disable personal gopherspaces

-d            Debug output in syslog and /server-status
-v            Display version number and build date
-b            Display the BSD license
-?            Display this help

(end of option list) -- keep this line for automatic extraction!

Setting up a gopher site

After succesfully installing Gophernicus (see INSTALL) you need to set up the gopher root directory. By default Gophernicus serves documents from /var/gopher so start by creating that directory and making sure it's world-readable. Then, simply add files and directories under your root, fire up a gopher browser (Firefox with the OverbiteFF extension, Lynx) and open up gopher://HOSTNAME/ (where HOSTNAME is your server hostname).

That's it, your first gopher site is now up and running. If the links on the root menu don't work make sure you are using the -h HOSTNAME parameter in your configuration (with a valid resolveable hostname instead of HOSTNAME - see INSTALL).

Security

Gophernicus has been written with high security in mind. There should be no buffer overflows or memory allocation problems so it should be safe to run a publicly available gopher server with Gophernicus.

However, the security settings (which are non-changeable) are so strict that you need to keep one thing in mind. Gophernicus will only serve world-readable content. Being readable by the server process is not enough, all files and directories MUST be world-readable or they are simply hidden from all listings and denied if a client asks for them.

The -nx option prevents execution of any script or external file, and the -nu option suppresses scanning for and serving of ~user directories (which are normally at ~/public_html/ for each user).

OpenBSD-specific Security

If you are running Gophernicus on OpenBSD, you may (depending on what features you want to use) be able to take advantage of unveil(2) and pledge(2).

If you run without executable map support (i.e. you run with -nx) then unveil(2) will be enabled and the server root will automatically be unveiled. If run with personal gopherspaces enabled (i.e. you run without -nu), then the password database (/etc/pwd.db) will automatically be unveiled, but you will have to manually unveil the filesystem path(s) from which to serve personal gopherspaces (see -U).

Running with -nm -nu -nx results in the strictest set of pledge(2) promises. If you have executable maps enabled (i.e. you run without -nx), then the promises are relaxed to allow exec. If you have personal gopherspaces enabled (i.e. you run without -nu), then the promises are relaxed to allow getpw. If you have shared memory enabled (i.e. you run without -nm), then pledge(2) support cannot be used at all.

In short, you probably want to run Gophernicus with -nm -nu -nx and then remove the flags that would otherwise disable the features you want.

To see what is going on with regards to pledge(2) and unveil(2), run Gophernicus with -d (to turn on debug logging) and look in your system logs.

Gophermaps

By default all gopher menus are automatically generated from the content of the directory being viewed. If you want to have informational text along with the files, or if you want to completely replace the generated menu with your own you need to take a look at gophermaps. See the README.gophermap for more information.

Gophertags

A gophertag file can be used to virtually rename a directory. Let's assume that you have a directory called "foo" somewhere - it will be listed as "foo" in all automatically generated menus. Now if you create a file foo/gophertag and put the text "bar" into it the menus will show "bar" but the links will still point to "foo". This is useful for creating descriptive names for directories without littering the file system with spaces and weird characters.

Personal gopherspaces

Gophernicus supports users personal gopherspaces. If a user has world-readable directory called public_gopher under his home, a request for gopher://HOSTNAME/1/~user/ will serve documents from that directory.

This is suppressed if the -nu option is given. In this case, any ~ entry which otherwise initiates listing of user directories will be displayed literally.

Virtual hosting

Gophernicus supports virtual hosting, or serving more than one logical domain using the same IP address. Since gopher (RFC1436) doesn't support virtual hosting this requires some hacks.

To enable virtual hosting create one or more directories under your gopher root which are named after your domain names. The primary vhost directory (set with the -h HOSTNAME option) must exist or virtual hosting will be disabled. Then simply add content to the hostname directories and you're (kind of) up and running.

There is a serious issue with virtual hosting.

As stated previously, RFC1436 dosen't support virtual hosting. Clients won't like it.

How the virtual hosting works, is that it loops through the vhosts looking for the selector. As you might think, the root gophermap exists on all of the vhosts, meaning it might not use the correct vhost. There is currently no easy way to fix this.

It is recommended to add '%' on a line by itself to the bottom of your root gophermaps. This will add "special" links of the format example.com/;example.com which forces the correct vhost.

CGI support

Gophernicus supports most parts of the CGI/1.1 standard. Most standard CGI variables are set, and some non-standard ones are added.

By default all scripts and binaries under any directory called /cgi-bin/ are executed as CGI scripts (this includes cgi-bin directories under users personal gopherspaces). Also, if a gophermap is marked executable it is also processed as an CGI script.

As with regular files, CGI scripts must be world-executable (and readable) or they will be ignored. Make sure your CGI script is safe with ANY user input as poorly coded CGI scripts are the number 1 security problem with publicly open Unix/Linux/BSD servers.

The -nx option prevents execution of any script or external file. In this case, they will be simply ignored and no output is given.

Output filtering and PHP support

In addition to CGI scripts Gophernicus supports output filtering scripts. By default output filtering is turned off, but you can turn it on by using the -f FILTERDIR option, creating that directory and creating one or more scripts in there named by either the file suffix, or by the gopher filetype char.

If a file is to be served out which matches either the file suffix script, or the filetype script then instead of simply sending the file to client the output filter script is executed with the original file as the first parameter and the output of the script is then sent to client.

For PHP support install the CLI version of the PHP interpreter and then symlink (or copy) that binary to the directory specified with -f option using the destination name "php".

$ ln -s /usr/bin/php5-cli /usr/lib/gophernicus/filters/php

After that all files with the php suffix will be "filtered" through the PHP command line interpreter. In other words, PHP starts working. And don't use the CGI version of PHP as it outputs HTTP headers the gopher protocol doesn't have.

Charset support and conversions

Gophernicus supports three charsets: US-ASCII, ISO-8859-1 and UTF-8. All textual input is internally upconverted to UTF-8 and then downconverted to whatever charset the client is asking for. The conversion is input autosensing which means that you don't have to specify your filesystem charset, or the charset of your text files - it's all detected automatically.

With standard gopher clients this is a bit of a problem as your text files WILL be converted to 7-bit US-ASCII. This means that all 8-bit charaters WILL BE LOST. This decision was made because no gopher client that I tested was reliably cabable of decoding anything else than pure US-ASCII. If you want to disable the conversion use the -no option, or if you'd like to change the default output charset to something else than US-ASCII just use for example the -o ISO-8859-1 option.

Selector rewriting

Selector rewriting lets you rewrite parts of the selector on the fly. Well, not parts, but really just the start of it. And the rewrite enging here is nothing like Apache's mod_rewrite as I was too lazy to integrate any regex libraries... So, all it does is rewrite a fixed string at the start of the selector to something else. This will let you move your directories around while making sure that existing deeplinks still work.

Examples:

-R "/~user=/~luser"
-R "/old-dir=/new-dir"

Session tracking and statistics

To enable virtual hosting with gopher (RFC1436) clients Gophernicus tracks users and their session. As a side effect of that session tracking, Gophernicus has simple throttling controls to keep nasty users from killing your precious 120MHz PPC 604e server from dying under the load. The throttling defaults are high enough that normal human users will never hit the limits, but it's possible (and mostly preferrable) that a badly behaving crawling agent will be throttled.

The current sessions and other real-time status data can be viewed by opening the URL gopher://HOSTNAME/0/server-status . This status view has been modeled after the Apache server-status which means that it's possible to integrate Gophernicus into existing server monitoring systems. To ease up such integrations, Gophernicus supports HTTP requests of the server-status page using an URL like http://HOSTNAME:70/server-status?auto .

TLS/SSL and proxy support

As of version 2.3 Gophernicus supports the HAproxy proxy protocol version 1. This makes it possible to build a cluster of gopher servers and use HAproxy in front of them all handling client routing to different backend servers.

More useful is putting Gophernicus behind Stunnel4 for TLS/SSL support and use the same proxy protocol to tell Gophernicus the correct remote IP address. The below sample stunnel configuration is all you need to TLS-enable your gopher server. Well, you'll need a certificate too and for that I recommend Let's Encrypt.

In addition to configuring Stunnel for TLS you should add -T TLSPORT to Gophernicus options so that it knows which connetions are coming in encrypted and which are not. Using proper -T also makes it possible for CGI programs to use the $TLS environment variable to know whether the current request was encrypted or not.

Example:

;
; Gophernicus behind Stunnel4 for gopher over TLS
;

; User/group for stunnel daemon
setuid = stunnel4
setgid = stunnel4

; PID file location
pid = /var/run/stunnel4/gophernicus.pid

; Log to file, not syslog
output = /var/log/stunnel4/gophernicus.log
syslog = no

; Certificate in pem format is needed for TLS
cert = /etc/ssl/private/gophernicus.pem

; Enable TCP wrappers
libwrap = yes
service = gophernicus-tls

; Gopher over TLS service
[gophernicus]
accept  = :::7070
connect = 127.0.0.1:70
protocol = proxy