jesus christ my repos are just full of garbage commits to get sphinx and travis to work

— Sam Nicholls (@samstudio8) June 14, 2016

So for my future sanity and possibly yours, here’s a quick guide¹ (and some links to resources) on how to do the things that I don’t do often enough to remember first hand.

To begin…

Install Sphinx²:

pip install Sphinx

From the root of your project, initialise the docs/ directory with sphinx-quickstart:

mkdir docs
cd docs/
sphinx-quickstart

sphinx-quickstart rapidly fires a series of prompts, the defaults are typically sensible, but enable autodoc when prompted. intersphinx might be useful if you have projects whose documentation may cross-reference eachother. viewcode adds links to source code from module listings, which could be helpful to end users. Make good use of the provided Makefile.

doc/ is also an acceptable directory name, by default, ReadTheDocs looks for either, before rummaging around your whole repository looking for something that appears to be documentation.

Generate module API docs

Assuming you enabled the autodoc extension, Sphinx can be set-up to automatically build a nice module index (such as the one found on the Goldilocks documentation) with links to documentation generated from the docstrings of your modules and classes; which is both pretty and a nice excuse to document your code properly too.

However this feature doesn’t work out of the box as you must first kickstart the API documentation with sphinx-apidoc:

cd docs/
sphinx-apidoc -o source/ ../<package>

Be sure to set the -o outputdir that will contain the generated Sphinx source files to source/. This took me a while to figure out, but without it (say just dumping everything into the docs/ directory), the py-modindex.html file would not be generated when built by ReadTheDocs and would thus be missing, causing a 404 on the website³. sourcedir (which is more sensibly called module_path in the Sphinx documentation) should point to your Python package (e.g. ../<package>)⁴.

Continue reading to find out why this still won’t work yet.

Configuration

Sphinx is configured by a conf.py that sits in the docs/ directory. The majority of important configuration options have already been set for you by sphinx-quickstart but here are a couple of things that I typically alter:

Theme

At the time of writing, the default theme is alabaster, rocked by various projects including the glorious requests package. However I actually like the ReadTheDocs default theme (other themes are available) and alter the html_theme accordingly:

html_theme = 'sphinx_rtd_theme'

Make autodoc actually work

sphinx-apidoc merely generates “stubs” for each of your modules. The stubs contain automodule directives which in turn inform sphinx-build to invoke autodoc to do the heavy lifting of actually generating the API documentation from the docstrings of a particular module. I’ve found that out of the box, I just get a screenful of ImportError‘s from autodoc during sphinx-build:

WARNING: autodoc: failed to import module

To ensure that sphinx-build can import your package and generate some lovely API documentation (and that all important module index; py-modindex), simply uncomment this line near the top of conf.py and those warnings should disappear on your next attempt at make html:

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('.'))

For some reason however, this fix doesn’t work in my latest project, so I added the following line beneath for good measure and all was well:

sys.path.insert(0, os.path.abspath('../'))

numpy style documentation with Napoleon

I find reStructuredText kinda grim for docstrings, so I use the sphinx-napoleon extension. This allows you to write numpy or Google style docstrings instead of dense blocks of quite difficult to read RST. As of Sphinx 1.3, the extension no longer needs to be manually installed and can be enabled in the same way as other extensions, like the autodoc:

extensions = [
    ...
    'sphinx.ext.napoleon',
]

napoleon_google_docstring = False
napoleon_use_param = False
napoleon_use_ivar = True

Simply append sphinx.ext.napoleon to the extensions list. Underneath, I’ve added a few configuration options that disable parsing of Google-style docstrings and alter how parameters and return types for functions are formatted in the generated documentation to taste. You can read about the napoleon configuration options on the official documentation and decide for yourself which options you may want to override.

As evident, I personally favour the numpy docstring style, which has a nice and simple guide that I keep misplacing.

Include your README and CHANGELOG

My repository already has a README.rst and CHANGELOG.rst, which can quickly be added to our Sphinx documentation without duplication. First, add we’ll add them to the table of contents in index.rst:

[...]
.. toctree::
   :maxdepth: 2

   readme
   changelog
[...]

Now, we’ll add directives to new readme.rst and changelog.rst files to include the contents from the files in the root of the repository to save duplication. For example:

.. include:: ../README.rst

Simple!

Enable develop branch doc on ReadTheDocs

Assuming you’ve actually set up ReadTheDocs (a very simple point-and-click adventure involving a short form, 2FA with Github and “importing” a project), familiarise yourself with your RTD dashboard. Of immediate use is the Versions tab of a relevant project, from here you can enable documentation to be generated for the various branches of your project. Personally I like to have documentation available for the develop branch, for those who just can’t wait for things to be less broken in master.

Other general Sphinx tips

Talking about classes and methods

Sphinx has a really nifty feature where one can reference classes, functions and the like anywhere in your documentation (even docstrings, too), and it will generate a link to the relevant part of the documentation. However I always forget the syntax, and what this feature is called. Turns out, this is referencing domains in Sphinx terminology and the syntax for each domain is well documented on this page that I just have to keep finding.

Custom CSS

One disagreement I have with the Sphinx RTD theme is how tables appear on smaller screens. Thankfully, it’s quite simple to add overrides to deal with minor design bothers like these. Just define a setup function in your conf.py that adds your stylesheet:

...
def setup(app):
    app.add_stylesheet('theme_overrides.css')
...

I’m not sure if it particularly matters where this setup appears, but for the record, mine precedes the “Options for LaTeX output” comment line. Stylesheets and other “static” content are unsurprisingly expected to appear in the _static directory. If you don’t have a docs/_static directory, now would be a good time to create one and add it to version control. For the sake of completeness, or in case you also dislike the behaviour of tables under the default RTD theme, here is my theme_overrides.css:

/* override table width restrictions as found on https://github.com/getpelican/pelican/issues/1311 */
.wy-table-responsive table td, .wy-table-responsive table th {
    /* !important prevents the common CSS stylesheets from
       overriding this as on RTD they are loaded after this stylesheet */
    white-space: normal !important;
}

.wy-table-responsive {
    overflow: visible !important;
}

Custom requirements

Before Sphinx 1.3, one had to install the sphinx-napoleon extension separately. Although this is no longer the case, you might find yourself wondering how to get custom plugins and the like to work with your ReadTheDocs documentation instance in future:

• Add the name of the extension⁵ to the extensions list in your conf.py as we have done previously.
• Create a newline delimited list of package names to be installed from pip in docs/requirements.txt
• Navigate to the Advanced Settings of your ReadTheDocs project, and tick the Install your project inside a virtualenv option. Specify docs/requirements.txt in the text box underneath.
• Future builds are now automatically built inside a virtualenv and the necessary requirements are fetched with pip.

tl;dr

• sphinx-quickstart to, er, quickstart
• sphinx-apidoc to generate API documentation stubs
• Ensure API doc lives in docs/source/ to avoid ReadTheDocs’ Sphinx builder not generating (missing) py-modindex
• Set html_theme to default in conf.py for “standard” ReadTheDocs theme
• Uncomment system path line in conf.py (may also need to add ../) to avoid Sphinx not finding your package and throwing import errors instead
• Check out the advanced settings (versions) on ReadTheDocs
• Read the numpy docstring styleguide, and about referencing Sphinx domains
• Add a setup function to conf.py to define additional (overriding) stylesheets
• Add a docs/requirements.txt, update your conf.py and tick the virtualenv option on ReadTheDocs Advanced Settings for your project to enable third party plugins via pip.