A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system – Gall’s Law.
Co-authored with Arnav Sood
This lecture discusses structuring a project as a Julia module, and testing it with tools from GitHub.
Benefits include
As we’ll see later, Travis is a service that automatically tests your project on the GitHub server.
First, we need to make sure that your GitHub account is set up with Travis CI and Codecov.
As a reminder, make sure you signed up for the GitHub Student Developer Pack or Academic Plan if eligible.
Navigate to the travis-ci.com website and click “sign up with GitHub” – supply your credentials.
If you get stuck, see the Travis tutorial.
Codecov is a service that tells you how comprehensive your tests are (i.e., how much of your code is actually tested).
To sign up, visit the Codecov website, and click “sign up”
Next, click “add a repository” and enable private scope (this allows Codecov to service your private projects).
The result should be
This is all we need for now.
using InstantiateFromURL
github_project("QuantEcon/quantecon-notebooks-julia", version = "0.6.0")
# uncomment to force package installation and precompilation
# github_project("QuantEcon/quantecon-notebooks-julia", version="0.6.0", instantiate=true, precompile = true)
Note: Before these steps, make sure that you’ve either completed the version control lecture or run.
git config --global user.name "Your Name"
Note: Throughout this lecture, important points and sequential workflow steps are listed as bullets.
To set up a project on Julia:
using PkgTemplates
This specifies metadata like the license we’ll be using (MIT by default), the location (~/.julia/dev
by default), etc.
ourTemplate = Template(;user="quanteconuser", plugins = [TravisCI(), Codecov()], manifest = true)
Note: Make sure you replace the quanteconuser
with your GitHub ID.
generate("ExamplePackage.jl", ourTemplate)
If we navigate to the package directory, we should see something like the following.
As a reminder, the location of your .julia
folder can be found by running DEPOT_PATH[1]
in a REPL .
Note: On Mac, this may be hidden; you can either start a terminal, cd ~
and then cd .julia
, or make hidden files visible in the Finder.
The next step is to add this project to Git version control.
We’ll want the following settings
In particular
README.md
, LICENSE
, and .gitignore
, since these are handled by PkgTemplates
.Then,
~/.julia/dev
directory to GitHub Desktop.If you navigate to your git repo (ours is here), you should see something like
Note: Be sure that you don’t separately clone the repo you just added to another location (i.e., to your desktop).
A key note is that you have some set of files on your local machine (here in ~/.julia/dev/ExamplePackage.jl
) and git is plugged into those files.
For convenience, you might want to create a shortcut to that location somewhere accessible.
We also want Julia’s package manager to be aware of the project.
cd(joinpath(DEPOT_PATH[1], "dev", "ExamplePackage"))
Note the lack of .jl
!
] activate
to get into the main Julia environment (more on environments in the second half of this lecture).
] dev .
to add the package.
You can see the change reflected in our default package list by running
] st
For more on the package mode, see the tools and editors lecture.
Now, from any Julia terminal in the future, we can run
using ExamplePackage
To use its exported functions.
We can also get the path to this by running
using ExamplePackage
pathof(ExamplePackage) # returns path to src/ExamplePackage.jl
Let’s unpack the structure of the generated project
.git
, holds the version control information.src
directory contains the project’s source code – it should contain only one file (ExamplePackage.jl
), which readsmodule ExamplePackage
greet() = print("Hello World!")
end # module
test
directory should have only one file (runtests.jl
), which readsusing ExamplePackage
using Test
@testset "ExamplePackage.jl" begin
# Write your own tests here.
end
In particular, the workflow is to export objects we want to test (using ExamplePackage
), and test them using Julia’s Test
module.
The other important text files for now are
Project.toml
and Manifest.toml
, which contain dependency information.In particular, the Project.toml
contains a list of dependencies, and the Manifest.toml
specifies their exact versions and sub-dependencies.
.gitignore
file (which may display as an untitled file), which contains files and paths for git
to ignore.As before, the .toml files define an environment for our project, or a set of files which represent the dependency information.
The actual files are written in the TOML language, which is a lightweight format to specify configuration options.
This information is the name of every package we depend on, along with the exact versions of those packages.
This information (in practice, the result of package operations we execute) will
be reflected in our ExamplePackage.jl
directory’s TOML, once that environment is activated (selected).
This allows us to share the project with others, who can exactly reproduce the state used to build and test it.
See the Pkg3 docs for more information.
For now, let’s just try adding a dependency
v1.1
environment)] activate ExamplePackage
Note that the base environment isn’t special, except that it’s what’s loaded by a freshly-started REPL or Jupyter notebook.
] add Expectations
We can track changes in the TOML, as before.
Here’s the Manifest.toml
We can also run other operations, like ] up
, ] precompile
, etc.
Package operations are listed in detail in the tools and editors lecture.
Recall that, to quit the active environment and return to the base (v1.1)
, simply run
] activate
The basic idea is to work in tests/runtests.jl
, while reproducible functions should go in the src/ExamplePackage.jl
.
For example, let’s say we add Distributions.jl
] activate ExamplePackage
] add Distributions
and edit the source (paste this into the file itself ) to read as follows
module ExamplePackage
greet() = print("Hello World!")
using Expectations, Distributions
function foo(μ = 1., σ = 2.)
d = Normal(μ, σ)
E = expectation(d)
return E(x -> sin(x))
end
export foo
end # module
Let’s try calling this
] activate
using ExamplePackage
ExamplePackage.greet()
foo() # exported, so don't need to qualify the namespace
Note: If you didn’t follow the instructions to add a startup file, you may need to quit your REPL and load the package again.
We can also work with the package from a Jupyter notebook.
Let’s create a new output directory in our project, and run jupyter lab
from it.
Create a new notebook output.ipynb
From here, we can use our package’s functions as we would functions from other packages.
This lets us produce neat output documents, without pasting the whole codebase.
We can also run package operations inside the notebook
The change will be reflected in the Project.toml
file.
Note that, as usual, we had to first activate ExamplePackage
first before making our dependency changes
name = "ExamplePackage"
uuid = "f85830d0-e1f0-11e8-2fad-8762162ab251"
authors = ["QuantEcon User <quanteconuser@gmail.com>"]
version = "0.1.0"
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
Expectations = "2fe49d83-0758-5602-8f54-1f90ad0d522b"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
[targets]
test = ["Test"]
There will also be changes in the Manifest as well .
Be sure to add output/.ipynb_checkpoints
to your .gitignore
file, so that’s not checked in.
Make sure you’ve activated the project environment (] activate ExamplePackage
) before you try to propagate changes.
For someone else to get the package, they simply need to
] dev https://github.com/quanteconuser/ExamplePackage.jl.git
This will place the repository inside their ~/.julia/dev
folder.
Recall that the path to your ~/.julia
folder is
DEPOT_PATH[1]
"/home/ubuntu/.julia"
They can then collaborate as they would on other git repositories.
In particular, they can run
] activate ExamplePackage
to load the list of dependencies, and
] instantiate
to make sure they are installed on the local machine.
It’s important to make sure that your code is well-tested.
There are a few different kinds of test, each with different purposes
In this lecture, we’ll focus on unit testing.
In general, well-written unit tests (which also guard against regression, for example by comparing function output to hardcoded values) are sufficient for most small projects.
Test
Module¶Julia provides testing features through a built-in package called Test
, which we get by using Test
.
The basic object is the macro @test
using Test
@test 1 == 1
@test 1 ≈ 1
Test Passed
Tests will pass if the condition is true
, or fail otherwise.
If a test is failing, we should flag it with @test_broken
as below
@test_broken 1 == 2
Test Broken
Expression: 1 == 2
This way, we still have access to information about the test, instead of just deleting it or commenting it out.
There are other test macros, that check for things like error handling and type-stability.
Advanced users can check the Julia docs.
Let’s add some unit tests for the foo()
function we defined earlier.
Our tests/runtests.jl
file should look like this.
As before, this should be pasted into the file directly
using ExamplePackage
using Test
@test foo() ≈ 0.11388071406436832
@test foo(1, 1.5) ≈ 0.2731856314283442
@test_broken foo(1, 0) # tells us this is broken
And run it by typing ] test
into an activated REPL (i.e., a REPL where you’ve run ] activate ExamplePackage
).
There are a few different ways to run the tests for your package.
runtests.jl
, say by hitting shift-enter
on it in Atom.v1.1
) REPL, run ] test ExamplePackage
.ExamplePackage
) REPL, simply run ] test
(recall that you can activate with ] activate ExamplePackage
).By default, Travis should have access to all your repositories and deploy automatically.
This includes private repos if you’re on a student developer pack or an academic plan (Travis detects this automatically).
To change this, go to “settings” under your GitHub profile
Click “Applications,” then “Travis CI,” then “Configure,” and choose the repos you want to be tracked.
By default, Travis will compile and test your project (i.e., “build” it) for new commits and PRs for every tracked repo with a .travis.yml
file.
We can see ours by opening it in Atom
# Documentation: http://docs.travis-ci.com/user/languages/julia/
language: julia
os:
- linux
- osx
julia:
- 1.1
- nightly
matrix:
allow_failures:
- julia: nightly
fast_finish: true
notifications:
email: false
after_success:
- julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder())'
This is telling Travis to build the project in Julia, on OSX and Linux, using Julia v1.1 and the latest development build (“nightly”).
It also says that if the nightly version doesn’t work, that shouldn’t register as a failure.
Note You won’t need OSX unless you’re building something Mac-specific, like iOS or Swift.
You can delete those lines to speed up the build, likewise for the nightly Julia version.
As above, builds are triggered whenever we push changes or open a pull request.
For example, if we push our changes to the server and then click the Travis badge (the one which says “build”) on the README, we should see something like
Note that you may need to wait a bit and/or refresh your browser.
This gives us an overview of all the builds running for that commit.
To inspect a build more closely (say, if it fails), we can click on it and expand the log options
Note that the build times here aren’t informative, because we can’t generally control the hardware to which our job is allocated.
We can also cancel specific jobs, either from their specific pages or by clicking the grey “x” button on the dashboard.
Lastly, we can trigger builds manually (without a new commit or PR) from the Travis overview
To commit without triggering a build, simply add “[ci skip]” somewhere inside the commit message.
You’ll find that Codecov is automatically enabled for public repos with Travis.
For private ones, you’ll need to first get an access token.
Add private scope in the Codecov website, just like we did for Travis.
Navigate to the repo settings page (i.e., https://codecov.io/gh/quanteconuser/ExamplePackage.jl/settings
for our repo) and copy the token.
Next, go to your Travis settings and add an environment variable as below
Click the Codecov badge to see the build page for your project.
This shows us that our tests cover 50% of our functions in src//
.
Note: To get a more detailed view, we can click the src//
and the resultant filename.
Note: Codecov may take a few minutes to run for the first time
This shows us precisely which methods (and parts of methods) are untested.
As mentioned in version control, sometimes we’ll want to work on external repos that are also Julia projects.
] dev
the git URL (or package name, if the project is a registered Julia package), which will both clone the git repo to ~/.julia/dev
and sync it with the Julia package manager.For example, running
] dev Expectations
will clone the repo https://github.com/quantecon/Expectations.jl
to ~/.julia/dev/Expectations
.
Make sure you do this from the base Julia environment (i.e., after running ] activate
without arguments).
As a reminder, you can find the location of your ~/.julia
folder (called the “user depot”), by running
DEPOT_PATH[1]
"/home/ubuntu/.julia"
The ] dev
command will also add the target to the package manager, so that whenever we run using Expectations
, Julia will load our cloned copy from that location
using Expectations
pathof(Expectations) # points to our git clone
Drag that folder to GitHub Desktop.
The next step is to fork the original (external) package from its website (i.e., https://github.com/quantecon/Expectations.jl
) to your account (https://github.com/quanteconuser/Expectations.jl
in our case).
Edit the settings in GitHub Desktop (from the “Repository” dropdown) to reflect the new URL.
Here, we’d change the highlighted text to read quanteconuser
, or whatever our GitHub ID is.
If you make some changes in a text editor and return to GitHub Desktop, you’ll see something like.
Note: As before, we’re editing the files directly in ~/.julia/dev
, as opposed to cloning the repo again.
Here, for example, we’re revising the README.
To confirm this, we can check the history on our account here; for more on working with git repositories, see the version control lecture.
The green check mark indicates that Travis tests passed for this commit.
For more on PRs, see the relevant section of the version control lecture.
For more on forking, see the docs on GitHub Desktop and the GitHub Website.
If you have write access to the repo, we can skip the preceding steps about forking and changing the URL.
You can use ] dev
on a package name or the URL of the package
] dev Expectations
or ] dev https://github.com/quanteconuser/Expectations.jl.git
as an example for an unreleased package by URL.
Which will again clone the repo to ~/.julia/dev
, and use it as a Julia package.
using Expectations
pathof(Expectations) # points to our git clone
Next, drag that folder to GitHub Desktop as before.
Then, in order to work with the package locally, all we need to do is open the ~/.julia/dev/Expectations
in a text editor (like Atom)
From here, we can edit this package just like we created it ourselves and use GitHub Desktop to track versions of our package files (say, after ] up
, or editing source code, ] add Package
, etc.).
To “un-dev” a Julia package (say, if we want to use our old Expectations.jl
), you can simply run
] free Expectations
To delete it entirely, simply run
] rm Expectations
From a REPL where that package is in the active environment.
Another goal of testing is to make sure that code doesn’t slow down significantly from one version to the next.
We can do this using tools provided by the BenchmarkTools.jl
package.
See the need for speed lecture for more details.
To review the workflow for creating, versioning, and testing a new project end-to-end.
PkgTemplates.jl
.~/.julia/dev/ExamplePackage.jl
, making sure the active environment is the default one (v1.1)
, and hitting ] dev .
.~/.julia/dev/ExamplePackage.jl
) in Atom.src/
directory once they’re stable, and you should export them from that file with export func1, func2
. This will export all methods of func1
, func2
, etc.Following the instructions for a new project, create a new package on your github account called NewtonsMethod.jl
.
In this package, you should create a simple package to do Newton’s Method using the code you did in the Newton’s method exercise in Introductory Examples.
In particular, within your package you should have two functions
newtonroot(f, f′; x₀, tol = 1E-7, maxiter = 1000)
newtonroot(f; x₀, tol = 1E-7, maxiter = 1000)
Where the second function uses Automatic Differentiation to call the first.
The package should include
/src
directoryFor the tests, you should have at the very minimum
nothing
as discussed in error handling@test
for the root of a known function, given the f
and analytical f'
derivativesBigFloat
and not just a Float64
maxiter
is working (e.g. what happens if you call maxiter = 5
tol
is workingAnd anything else you can think of. You should be able to run ] test
for the project to check that the test-suite is running, and then ensure that it is running automatically on Travis CI.
Push a commit to the repository which breaks one of the tests and see what the Travis CI reports after running the build.
Watch the youtube video Developing Julia Packages from Chris Rackauckas. The demonstration goes through many of the same concepts as this lecture, but with more background in test-driven development and providing more details for open-source projects..