Download the zip file and unpack it into a directory for the workout. We will create other directories beside it and copy files from the unpacked zip into those directories.
conda update conda
conda install sphinx sphinx_rtd_theme
pip install nbsphinx
If you want to experiment with the integration between Bitbucket and Readthedocs:
Sphinx docs: http://www.sphinx-doc.org/en/stable/contents.html
Readthedocs docs: https://docs.readthedocs.io/en/latest/index.html
reStructuredText primers:
Brandon Rhodes's PyCon 2013 tutorial video
Source repos that you can copy markup from:
Sphinx is very similar in concept to LaTeX. It transforms plain text markdown from a collection of files into a display format.
LaTeX is optimized for producing beautiful typeset output.
Sphinx is optimzed for producing beautiful, highly functional HTML output.
Both can produce other output formats (PDF, ePub, etc.)
Both facilitate:
Both were designed to be extensible
Sphinx was created for the Python language documentation that used to be processed from text to HTML using LaTeX
sphinx-quickstart
¶The Sphinx package provides a program called sphinx-quickstart
that asks you a few questions
and uses the answers to create the files and directories needed to start writing Sphinx docs
for a project.
At a terminal prompt:
mkdir sphinx-demo
cd sphinx-demo
sphinx-quickstart
Welcome to the Sphinx 1.4.8 quickstart utility.
Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).
Enter the root path for documentation.
> Root path for the documentation [.]: docs
You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]:
Inside the root directory, two more directories will be created; "_templates"
for custom HTML templates and "_static" for custom stylesheets and other static
files. You can enter another prefix (such as ".") to replace the underscore.
> Name prefix for templates and static dir [_]:
The project name will occur in several places in the built documentation.
> Project name: Sphinx Demo
> Author name(s): Doug Latornell
Sphinx has the notion of a "version" and a "release" for the
software. Each version can have multiple releases. For example, for
Python the version is something like 2.5 or 3.0, while the release is
something like 2.5.1 or 3.0a1. If you don't need this dual structure,
just set both to the same value.
> Project version: 0.1
> Project release [0.1]:
If the documents are to be written in a language other than English,
you can select a language here by its language code. Sphinx will then
translate text that it generates into that language.
For a list of supported codes, see
http://sphinx-doc.org/config.html#confval-language.
> Project language [en]:
The file name suffix for source files. Commonly, this is either ".txt"
or ".rst". Only files with this suffix are considered documents.
> Source file suffix [.rst]:
One document is special in that it is considered the top node of the
"contents tree", that is, it is the root of the hierarchical structure
of the documents. Normally, this is "index", but if your "index"
document is a custom template, you can also set this to another filename.
> Name of your master document (without suffix) [index]:
Sphinx can also add configuration for epub output:
> Do you want to use the epub builder (y/n) [n]:
Please indicate if you want to use one of the following Sphinx extensions:
> autodoc: automatically insert docstrings from modules (y/n) [n]: y
> doctest: automatically test code snippets in doctest blocks (y/n) [n]:
> intersphinx: link between Sphinx documentation of different projects (y/n) [n]: y
> todo: write "todo" entries that can be shown or hidden on build (y/n) [n]: y
> coverage: checks for documentation coverage (y/n) [n]:
> imgmath: include math, rendered as PNG or SVG images (y/n) [n]:
> mathjax: include math, rendered in the browser by MathJax (y/n) [n]: y
> ifconfig: conditional inclusion of content based on config values (y/n) [n]:
> viewcode: include links to the source code of documented Python objects (y/n) [n]: y
> githubpages: create .nojekyll file to publish the document on GitHub pages (y/n) [n]:
A Makefile and a Windows command file can be generated for you so that you
only have to run e.g. `make html' instead of invoking sphinx-build
directly.
> Create Makefile? (y/n) [y]:
> Create Windows command file? (y/n) [y]:
Creating file docs/conf.py.
Creating file docs/index.rst.
Creating file docs/Makefile.
Creating file docs/make.bat.
Finished: An initial directory structure has been created.
You should now populate your master file docs/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.
As promised,
sphinx-quickstart
created a docs/
directory for us:
ls sphinx-demo
docs/ gsw_py_mat/
and that diectory contains all the files and directories necessary to start producing Sphinx docs for our project:
ls sphinx-demo/docs
api.rst conf.py Makefile _static/ _build/ index.rst notebooks/ _templates/
So, let's build the docs:
cd docs
make html
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v1.4.8
making output directory...
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/2/objects.inv
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.
Build finished. The HTML pages are in _build/html.
To see the HTML docs that we built,
open _build/html/index.html
in a browser.
You can use File > Open File...
and navigate to the index.html
file,
or:
On Linux you can probably do:
firefox _build/html/index.html
On Mac OSX try:
open _build/html/index.html
The source file that produced index.html
is index.rst
:
cat sphinx-demo/docs/index.rst
.. Sphinx Demo documentation master file, created by sphinx-quickstart on Thu Nov 10 12:39:35 2016. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to Sphinx Demo's documentation! ======================================= Contents: .. toctree:: :maxdepth: 2 api notebooks/demo.ipynb Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search`
The conf.py
file contains all of the configuration settings
that told Sphinx the details of how to convert the .rst
files in our
project into an HTML web site.
Most of the settings in conf.py
have sensible default values.
Lets talk a walk through it and change the theme of the HTML output as we do.
Open conf.py
in an editor.
The configuration settings in the conf.py
file that sphinx-quickstart
generated
(and many more)
are documented in the build configuration file
section of the Sphinx docs.
I usually delete the commented out settings that I am happy with the values of (those are the defaults), leaving only the one that I have editted.
The only one that we're going to change right now is at about line 129. Change:
html_theme = 'alabaster'
to
html_theme = 'sphinx_rtd_theme'
We'll explore conf.py
some more when we look at autodoc
.
For now,
run
make html
again and refresh the index.html
page in your browser.
Now our docs look like they will when they are rendered on Readthedocs.
You could copy all the files and directories in _build/html/
to a web server
such as your $HOME/public_html/
space on the EOAS server,
and your docs would be on-line for the world to see.
But you would have to remember to:
.rst
files (or conf.py
), andAn alternative for projects that you are happy to make public is to use version control and the readthedocs.org service to automatically build the docs every time you commit changes. Readthedocs is also a very good place to host docs that you want the world to know about because it has excellent search engine optimization and is well regarded in the Google search rankings.
We'll use Bitbucket,
but you can use Github in the same way.
Let's create a new repository on Bitbucket from the
Repositories > Create Repository
link in the navigation bar:
If you are using sphinx-demo
as the name of the directory we have been working so far,
choose a different name, like workout-sphinx
.
Ensure that This is a private respository
is not checked.
Ensure that Repository type
has Mercurial
selected.
Create the repository.
Expand the I'm starting from scratch
link:
and follow the instructions to clone the (empty) repo on to you local machine:
pwd
/home/doug/Documents/swc/workouts/14nov16-sphinx-rtd-autodoc/sphinx-demo/docs
cd ../..
hg clone ssh://hg@bitbucket.org/douglatornell/workout-sphinx
Make a docs/
directory in the new repo,
and copy the conf.py
, index.rst
, Makefile
, and make.bat
files into it:
mkdir workout-sphinx/docs
cd sphinx-demo/docs
cp conf.py index.rst Makefile make.bat ../../workout-sphinx/docs/
cd ../../workout-sphinx/docs/
Tell Mercurial to track those files:
hg add
adding conf.py
adding index.rst
adding Makefile
adding make.bat
Commit the files:
hg commit -m"Initialize Sphinx docs for the project."
Push the commit to Bitbucket:
hg push
pushing to ssh://hg@bitbucket.org/douglatornell/workout-sphinx
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 2 changes to 2 files
Go to your Readthedocs dashboard
and click the Import a Project
button.
Fill in the form:
Choose a unique name for your project on Readthedocs.
Get the Repository URL
from Bitbucket.
It will be something like https://bitbucket.org/douglatornell/workout-sphinx
.
Ensure that Repository Type
is set to Mercurial
.
Click Next
.
The project page provides all the information about your project's state on Readthedocs:
The Builds
button takes you to a page where you can see build status.
You can drill down into a build to see the build steps in detail.
They are a lot more complicated than our simple make html
above,
but that is in there.
Build errors also show up there.
The View Docs
button takes you to the rendered docs (of course!).
There are details about using Python virtual environments or conda environment
to build your docs in that you have to deal with as things get more complicated,
especially if you use the autodoc extension for Python modules with lots of imports.
The set up pages for that are behind the Admin
button,
and documented in the Readthedocs docs.
Send me an email,
or come and see me if you get stuck.
The last bit of automation work that we need to do is to set up a way for Bitbucket to send a signal to Readthedocs when we push commits so that Readthedocs will pull in those commits and re-buld the docs.
Go to the Webhooks
panel in the Settings
of your Bitbucket repo:
Click the Add Webhook
button,
and fill in the form:
Set the Title
to something exlanatory like: Read the Docs (readthedocs.org)
.
Fill in the URL
with https://readthedocs.org/api/v2/webhook/bitbucket/djl-workout-sphinx/
,
substituting the name of your Readthedocs project for the final section.
Ensure that the Status
is set to Active
.
Ensure that Triggers
is set to Repository push
.
Save the webhook.
NOTE
If you successfully completed the Bitbucket and Readthedocs section above you can continue working in your
workout-sphinx
repo so that you can commit and push the changes we're going to make and see them rendered on Readthedocs.
If you didn't finish the above, or watched it as a demo, keep working in your
sphinx-demo
directory that we set up withsphinx-quickstart
.
autodoc
is an extension that is built into Sphinx.
It is triggered by special markup like:
.. automodule:: foo.bar
:members:
When that autodoc is enabled and Sphinx see that markup it
bar
from the package foo
automodule
directiveSphinx then renders the docs including the docstrings, so if you use Sphinx markup in the docstrings, they get rendered beautifully too.
The documentation for autodoc
is at http://www.sphinx-doc.org/en/stable/ext/autodoc.html.
autodoc
is enabled by including:
'sphinx.ext.autodoc',
in the extensions
list in conf.py
(about line 34).
We made that happen when we answered yes
to the question
> autodoc: automatically insert docstrings from modules (y/n)
back when we ran sphinx-quickstart
,
but you can add it later if you forgot then or change your mind.
We also need to tell Sphinx where to find the Python modules that we want to
extract docstring from.
We do that by uncommenting the os
and sys
imports at lines 20 and 21 in conf.py
.
Also uncomment line 22 and change it to:
sys.path.insert(0, os.path.abspath('..'))
That puts the absolute path of the directory above docs/
on the path that Python
(and thence Sphinx) can import modules from.
We'll put some code there shortly.
One more thing while we are edting conf.py
...
When we ran sphinx-quickstart
we also chose to enable the intersphinx
extension
when we andwered yes
to the question
> intersphinx: link between Sphinx documentation of different projects (y/n)
That added:
'sphinx.ext.intersphinx',
to the extensions
list in conf.py
.
It also added the intersphinx_mapping
dictionary to the end of the file.
We need to change that mapping to use Python 3:
intersphinx_mapping = {'https://docs.python.org/3/': None}
There is a copy of a fully edited conf.py
file in the workout zip file
that you can copy over if you don't want to do the edits.
It is sphinx-rtd-autodoc/docs/conf.py
.
autodoc
for a Python Module¶To save on typing time, let's grab some files from the workout zip archive:
cd sphinx-demo
cp -R ../sphinx-rtd-autodoc/gsw_py_mat ./
cp ../sphinx-rtd-autodoc/docs/api.rst docs/
cd docs/
and build our our docs again. This time we'll use:
make clean html
to force Sphinx to rebuild everything.
rm -rf _build/*
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v1.4.8
making output directory...
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 2 source files that are out of date
updating environment: 2 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... /media/doug/warehouse/swc/workouts/14nov16-sphinx-rtd-autodoc/sphinx-demo/docs/api.rst:: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] index
generating indices... genindex py-modindex
highlighting module code... [100%] gsw_py_mat.py_cmd
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 1 warning.
Build finished. The HTML pages are in _build/html.
That WARNING
is there because we added the api.rst
file to our collection of docs files,
but we didn't tell Sphinx where to put it the table of contents.
We fix that by editing index.rst
to add api
(no extension necessary) to the toctree
directive:
.. Sphinx Demo documentation master file, created by
sphinx-quickstart on Thu Nov 10 12:39:35 2016.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Sphinx Demo's documentation!
=======================================
Contents:
.. toctree::
:maxdepth: 2
api
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
Run make html
again,
then refresh the docs page in your browser and checkout the new section of docs:
gsw_py_mat/py_cmd.py
filegsw_py_mat.py_cmd.make_matlab_cmd()
functionsubprocess.run()
in that function's docstring links to the Python documentationLook at the contents of api.rst
and gsw_py_mat/py_cmd.py
to see how we made all of this awesomeness happen.
sphinx-demo/docs/api.rst
:
********************
gsw_py_mat Functions
********************
This section documents the Python and Matlab functions that are available in the :kbd:`gsw_py_mat`.
Python Function
===============
.. automodule:: gsw_py_mat.py_cmd
:members:
sphinx-demo/gsw_py_mat/py_cmd.py
:
"""Python interface to Matlab GSW functions.
"""
import shlex
def make_matlab_cmd(scriptname, filename, args):
"""Return a command to run the Matlab script :kbd:`scriptname`.
:arg str scriptname: Name of the Matlab script to make the command for.
:arg str filename: Name of file in which Matlab script will store its
results.
:arg list args: Arguments to pass to the Matlab script.
:returns: Command suitable to be passed to :py:func:`subprocess.run`.
:rtype: list
"""
singlequote = "'"
comma = ','
# add quotes to the filename
mfilename = '{0}{1}{0}'.format(singlequote, filename)
firstpart = '{0}('.format(scriptname)
lastpart = ');exit'
functioncall = firstpart + mfilename
for arg in args:
functioncall += comma + str(arg)
functioncall += lastpart
cmd = shlex.split('matlab -nodesktop -nodisplay -r')
cmd.append(functioncall)
return cmd
The markup that is used in docstrings and for cross-referencing code objects in Sphinx is described in the Domains section of the Sphinx docs. The term "domains" is used because Sphinx markup can be applied to languages other than Python.
If you have the Readthedocs integration working you can commit and push the changes we've made and see them on your docs site.
Sadly, the sphinxcontrib-matlabdomain
extension
(https://pypi.python.org/pypi/sphinxcontrib-matlabdomain)
that provides a Sphinx domain for Matlab code objects,
and enables autodoc
to handle them has not been ported to Python 3,
and may contain other bugs.
We can still add documentation for our Matlab functions to api.rst
.
We just have to remember to update it if/when we change the Matlab scripts.
Matlab Functions
================
.. function:: gsw_py_mat.startup()
Matlab startup function that provides access to the Matlab GSW3 library.
.. note:: Only works on machines that have access :file:`/ocean/`.
.. function:: gsw_py_mat.mw_gsw_CT_from_t(filename, SA, t, p)
Calculate conservative temperature from absolute salinity,
in situ temperature,
and pressure.
:arg filename: Name of file in which Matlab script will store its
results.
:arg SA: Absolute salinity.
:arg t: Temperature.
:arg p: Pressure.
.. function:: gsw_py_mat.mw_gsw_SA_from_SP(filename, SP, p, lon, lat)
Calculate absolute salinity from practical salinity and pressure
with corrections based on longitude and latitude.
:arg filename: Name of file in which Matlab script will store its
results.
:arg SP: Practical salinity.
:arg p: Pressure.
:arg lon: Longitude of the location at which the salinity is to be
calculated.
:arg lat: Latitude of the location at which the salinity is to be
calculated.
nbsphinx
Extension¶The nbsphinx
extension
(https://nbsphinx.readthedocs.io/)
enables Sphinx pages to be created from Jupyter Notebooks.
The notebook cells and their output are rendered to HTML.
Note: Any code cells without stored output will be executed during the Sphinx build process. That means that all of the imports must be available in the build environment. That can introduce complications on Readthedocs.
nbsphinx
is enabled by 2 items:
'nbsphinx',
'IPython.sphinxext.ipython_console_highlighting',
in the extensions
list in conf.py
.
With that done,
simply include in a toctree
directive any .ipynb
files that you want rendered as pages in your docs.
The .ipynb
extension is required.
To test it,
copy the demo notebook from the zip archive:
mkdir notebooks
cp ../../sphinx-rts-autodoc/docs/notebooks/demo.ipynb notebooks/
Add notebooks/demo.ipynb
to your index.rst
file:
.. toctree::
:maxdepth: 2
api
notebooks/demo.ipynb
Run make html
,
refresh your docs page in the browser,
and check out the new page.
The notebooks don't have to be in a separate directory,
I just did that to demonstrate that you can have subdirectories in your docs tree.
Typically,
each sub-directory has its own index.rst
with some introductory text,
and a toctree
directive for the sub-directories contents,
but you can organize things as you wish,
so long as every .rst
or .ipynb
file appears in a toctree
somewhere.
Adding the nbsphinx
extension to our docs environment means that we
also need it to be available on Readthedocs.
To do that we have to tell Readthedocs to create a conda environment to
build our docs in,
and to install nbsphinx
and the ipython
into that environment.
Copy 2 files from the zip archive into the top level of your clone of the repo that is connected to Readthedocs:
cd workout-sphinx
cp ../sphinx-rtd-autodoc/readthedocs.yml ./
cp ../sphinx-rtd-autodoc/environment-rtd.yaml ./
readthedocs.yml
tells sphinx to create a conda environment using the information in environment-rtd.yaml
:
conda:
file: environment-rtd.yaml
environment-rtd.yaml
tells Readthe docs how to build the conda environment for our project:
# conda environment description file for docs build envioronment
# on readthedocs.org
name: sphinx-build
dependencies:
- ipython
- python=3
- sphinx
- pip:
- nbsphinx
Commit and push those 2 files along with the other files that we added and changed above, and the docs should build on Readthedocs. Remember to check the Builds page for their status, and to figure out any build errors.
Sphinx is more than a single Python package.
It is built on top of the pre-existing reStructuredText
markup system and the tools
created for working with it.
Syntax highlighting is provided by the Pygments package.
Third party themes and extension provide Sphinx with capabilities beyond what is in its core.