Code should execute sequentially if run in a Jupyter notebook
Co-authored with Arnav Sood
This lecture is about structuring your project as a Julia module, and testing it with tools from GitHub
Benefits include
As we’ll see later, Travis is a service that automatically tests your project on the GitHub server
First, we need to make sure that your GitHub account is set up with Travis CI and CodeCov
As a reminder, make sure you signed up for the GitHub Student Developer Pack or Academic Plan https://help.github.com/articles/applying-for-an-academic-research-discount/_ if eligible
Navigate to the Travis website and click “sign up with GitHub.” Supply your credentials
If you get stuck, see the Travis tutorial
NOTE As of May 2018, Travis is deprecating the travis-ci.org
website. All users should use travis-ci.com
CodeCov is a service that tells you how expansive your tests are (i.e., how much of your code is untested)
To sign up, visit the CodeCov website, and click “sign up.” You should see something like this
Next, click “add a repository” and enable private scope (this allows CodeCov to service your private projects)
The result should be
This is all we need for now
using InstantiateFromURL
activate_github("QuantEcon/QuantEconLectureAllPackages", tag = "v0.9.0") # activate the QuantEcon environment
using LinearAlgebra, Statistics, Compat # load common packages
Let’s create a template for our project
This specifies metadata like the license we’ll be using (MIT by default), the location (~/.julia/dev
by default), etc.
using PkgTemplates
ourTemplate = Template(;user="quanteconuser", plugins = [TravisCI(), CodeCov()])
Template: → User: quanteconuser → Host: github.com → License: MIT (Jesse Perla 2018) → Package directory: ~\.julia\dev → Minimum Julia version: v1.0 → SSH remote: No → Plugins: • CodeCov: → Config file: None → 3 gitignore entries: "*.jl.cov", "*.jl.*.cov", "*.jl.mem" • TravisCI: → Config file: Default → 0 gitignore entries
Let’s create a specific project based off this template
generate("ExamplePackage.jl", ourTemplate)
Generating project ExamplePackage:
C:\Users\jlperla\.julia\dev\ExamplePackage\Project.toml
C:\Users\jlperla\.julia\dev\ExamplePackage\src/ExamplePackage.jl
┌ Info: Initialized git repo at C:\Users\jlperla\.julia\dev\ExamplePackage └ @ PkgTemplates C:\Users\jlperla\.julia\packages\PkgTemplates\DyEmW\src\generate.jl:22 ┌ Info: Set remote origin to https://github.com/quanteconuser/ExamplePackage.jl └ @ PkgTemplates C:\Users\jlperla\.julia\packages\PkgTemplates\DyEmW\src\generate.jl:41
Resolving package versions... Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Project.toml` [8dfed614] + Test Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Manifest.toml` [2a0f44e3] + Base64 [8ba89e20] + Distributed [b77e0a4c] + InteractiveUtils [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [9a3f8284] + Random [9e88b42a] + Serialization [6462fe0b] + Sockets [8dfed614] + Test Updating registry at `C:\Users\jlperla\.julia\registries\General` Updating git-repo `https://github.com/JuliaRegistries/General.git` Resolving package versions... Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Project.toml` [no changes] Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Manifest.toml` [2a0f44e3] - Base64 [8ba89e20] - Distributed [b77e0a4c] - InteractiveUtils [8f399da3] - Libdl [37e2e46d] - LinearAlgebra [56ddb016] - Logging [d6f4376e] - Markdown [9a3f8284] - Random [9e88b42a] - Serialization [6462fe0b] - Sockets [8dfed614] - Test
┌ Info: Committed 9 files/directories: src/, Project.toml, Manifest.toml, test/, REQUIRE, README.md, .gitignore, LICENSE, .travis.yml └ @ PkgTemplates C:\Users\jlperla\.julia\packages\PkgTemplates\DyEmW\src\generate.jl:64
If we navigate to the package directory (shown in the output), we should see something like
The next step is to add this project to Git version control
First, open the repository screen in your account as discussed previously. We’ll want the following settings
In particular
README.md
, LICENSE
, and .gitignore
, since these are handled by PkgTemplates
Then, drag and drop your folder from your ~/.julia/dev
directory to GitHub Desktop
Click the “publish branch” button to upload your files to GitHub
If you navigate to your git repo (ours is here), you should see something like
We also want Julia’s package manager to be aware of the project
First, open a REPL in the newly created project directory, either by noting the path printed above, or by running
DEPOT_PATH
3-element Array{String,1}: "C:\\Users\\jlperla\\.julia" "C:\\Users\\jlperla\\AppData\\Local\\Julia-1.0.2\\local\\share\\julia" "C:\\Users\\jlperla\\AppData\\Local\\Julia-1.0.2\\share\\julia"
And navigating to the first element, then the subdirectory /dev/ExamplePackage
Note the lack of .jl
!
You can change the path of a Julia REPL by running
cd(joinpath(DEPOT_PATH[1], "dev", "ExamplePackage"))
Then, run
] activate
To get into the main Julia environment, and
] dev .
Resolving package versions... Updating `C:\Users\jlperla\.julia\environments\v1.0\Project.toml` [e4ee45a0] + ExamplePackage v0.1.0 [`..\..\dev\ExamplePackage`] Updating `C:\Users\jlperla\.julia\environments\v1.0\Manifest.toml` [e4ee45a0] + ExamplePackage v0.1.0 [`..\..\dev\ExamplePackage`]
We see the change reflected in our default package list
] st
Status `C:\Users\jlperla\.julia\environments\v1.0\Project.toml` [c52e3926] Atom v0.7.9 [a93c6f00] DataFrames v0.14.1 [5721bf48] DataVoyager v0.3.1 [0c46a032] DifferentialEquations v5.3.1 [31c24e10] Distributions v0.16.4 [e4ee45a0] ExamplePackage v0.1.0 [`..\..\dev\ExamplePackage`] [2fe49d83] Expectations v1.0.2 [7073ff75] IJulia v1.14.0 [43edad99] InstantiateFromURL v0.1.0 [c601a237] Interact v0.9.0 [54eb57ff] InteractiveCodeSearch v0.2.1 [a98d9a8b] Interpolations v0.10.5 [e5e0dc1b] Juno v0.5.3 [ee78f7c6] Makie v0.9.0 [429524aa] Optim v0.17.1 [d96e819e] Parameters v0.10.1 [d2cacc76] PerlaTonettiWaugh v0.1.0 [`C:\Users\jlperla\.julia\dev\PerlaTonettiWaugh`] [14b8a8f1] PkgTemplates v0.3.0 [91a5bcdd] Plots v0.21.0 [fcd29c91] QuantEcon v0.15.0 [612083be] Queryverse v0.1.0 [ce6b1742] RDatasets v0.6.1 [295af30f] Revise v0.7.12 [112f6efa] VegaLite v0.5.0
Now, from any Julia terminal in the future, we can run
none
using ExamplePackage
To use its exported functions
We can also get the path to this by running
none
using ExamplePackage
pathof(ExamplePackage) # returns path to src/ExamplePackage.jl
Let’s unpack the structure of the generated project
.git
, holds the version control informationsrc
directory contains the project’s source code. Currently, it should contain only one file (ExamplePackage.jl
), which readsnone
module ExamplePackage
greet() = print("Hello World!")
end # module
test
directory should have only one file (runtests.jl
), which reads:none
using ExamplePackage
using Test
@testset "ExamplePackage.jl" begin
# Write your own tests here.
end
In particular, the workflow is to export objects we want to test (using ExamplePackage
), and test them using Julia’s Test
module
The other important text files for now are
Project.toml
and Manifest.toml
, which contain dependency information.gitignore
file (which may display as an untitled file), which contains files and paths for git
to ignoreAs before, the TOML files define an environment for our project
Any package operations we execute will be reflected in our ExamplePackage.jl
directory’s TOML
Likewise, the only packages Julia knows about are those in the ExamplePackage.jl
TOML. For example
] activate ExamplePackage
using QuantEcon # fails, even though QuantEcon is on the machine
This allows us to share the project with others, who can exactly reproduce the state used to build and test it
See the Pkg3 docs for more information
For now, let’s just try adding a dependency
] add Expectations
Resolving package versions... Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Project.toml` [2fe49d83] + Expectations v1.0.2 Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Manifest.toml` [7d9fca2a] + Arpack v0.3.0 [9e28174c] + BinDeps v0.8.10 [b99e7846] + BinaryProvider v0.5.2 [34da2185] + Compat v1.3.0 [864edb3b] + DataStructures v0.14.0 [31c24e10] + Distributions v0.16.4 [2fe49d83] + Expectations v1.0.2 [442a2c76] + FastGaussQuadrature v0.3.2 [e1d29d7a] + Missings v0.3.1 [bac558e1] + OrderedCollections v1.0.2 [90014a1f] + PDMats v0.9.5 [1fd47b50] + QuadGK v2.0.2 [79098fc4] + Rmath v0.5.0 [a2af1166] + SortingAlgorithms v0.3.1 [276daf66] + SpecialFunctions v0.7.2 [2913bbd2] + StatsBase v0.25.0 [4c63d2b9] + StatsFuns v0.7.0 [30578b45] + URIParser v0.4.0 [2a0f44e3] + Base64 [ade2ca70] + Dates [8bb1440f] + DelimitedFiles [8ba89e20] + Distributed [b77e0a4c] + InteractiveUtils [76f85450] + LibGit2 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [a63ad114] + Mmap [44cfe95a] + Pkg [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA [9e88b42a] + Serialization [1a1011a3] + SharedArrays [6462fe0b] + Sockets [2f01184e] + SparseArrays [10745b16] + Statistics [4607b0f0] + SuiteSparse [8dfed614] + Test [cf7118a7] + UUIDs [4ec0a83e] + Unicode
We can track changes in the TOML, as before
Here’s the Manifest
We can also run other operations, like ] up
, ] precompile
, etc.
Recall that, to quit the active environment and return to the base (v1.0)
, simply run
] activate
Without any arguments
The basic idea is to work in tests/runtests.jl
, while reproducible functions should go in the src/ExamplePackage.jl
For example, let’s say we add Distributions.jl
] activate ExamplePackage
] add Distributions
Resolving package versions... Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Project.toml` [31c24e10] + Distributions v0.16.4 Updating `C:\Users\jlperla\.julia\dev\ExamplePackage\Manifest.toml` [no changes]
and edit the source to read as follows:
module ExamplePackage
greet() = print("Hello World!")
using Expectations, Distributions
function foo(μ = 1., σ = 2.)
d = Normal(μ, σ)
E = expectation(d)
return E(x -> sin(x))
end
export foo
end # module
Main.ExamplePackage
Let’s try calling this from the top environment (to stand in for a fresh REPL)
] activate
using ExamplePackage
ExamplePackage.greet()
┌ Info: Precompiling ExamplePackage [e4ee45a0-e7ce-11e8-2a29-0577d03cebf1] └ @ Base loading.jl:1192 WARNING: using ExamplePackage.ExamplePackage in module Main conflicts with an existing identifier.
importing ExamplePackage into Main conflicts with an existing identifier Stacktrace: [1] top-level scope at In[19]:1
foo() # exported, so don't need to qualify the namespace
UndefVarError: foo not defined Stacktrace: [1] top-level scope at In[20]:1
We can also call this function from a Jupyter notebook
Let’s create a new output directory in our project, and run jupyter lab
from it. Call a new notebook output.ipynb
From here, we can use our package’s functions in the usual way. This lets us produce neat output examples, without re-defining everything
We can also edit it interactively inside the notebook
The change will be reflected in the Project.toml
file:
name = "ExamplePackage"
uuid = "f85830d0-e1f0-11e8-2fad-8762162ab251"
authors = ["QuantEcon User <quanteconuser@gmail.com>"]
version = "0.1.0"
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
Expectations = "2fe49d83-0758-5602-8f54-1f90ad0d522b"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
[targets]
test = ["Test"]
UndefVarError: deps not defined Stacktrace: [1] top-level scope at In[21]:5
And the Manifest as well
Be sure to add output/.ipynb_checkpoints
to your .gitignore
file, so that’s not checked in
Make sure you’ve activated the project environment (] activate ExamplePackage
) before you try to propagate changes
For someone else to get the package, they simply need to run
] dev https://github.com/quanteconuser/ExamplePackage.jl.git
Cloning git-repo `https://github.com/quanteconuser/ExamplePackage.jl.git` Updating git-repo `https://github.com/quanteconuser/ExamplePackage.jl.git`
┌ Info: Path `C:\Users\jlperla\.julia\dev\ExamplePackage` exists and looks like the correct package, using existing path instead of cloning └ @ Pkg.Types C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Types.jl:586
Resolving package versions... Updating `C:\Users\jlperla\.julia\environments\v1.0\Project.toml` [e4ee45a0] - ExamplePackage v0.1.0 [`..\..\dev\ExamplePackage`] [f85830d0] + ExamplePackage v0.1.0 [`C:\Users\jlperla\.julia\dev\ExamplePackage`] Updating `C:\Users\jlperla\.julia\environments\v1.0\Manifest.toml` [e4ee45a0] - ExamplePackage v0.1.0 [`..\..\dev\ExamplePackage`] [f85830d0] + ExamplePackage v0.1.0 [`C:\Users\jlperla\.julia\dev\ExamplePackage`]
This will place the repository inside their ~/.julia/dev
folder, and they can drag-and-drop it to GitHub desktop in the usual way
They can then collaborate as they would on other git repositories
In particular, they can run
] activate ExamplePackage
] instantiate
Updating registry at `C:\Users\jlperla\.julia\registries\General` Updating git-repo `https://github.com/JuliaRegistries/General.git`
To make sure the right dependencies are installed on their machine
It’s important to make sure that your code is well-tested
There are a few different kinds of test, each with different purposes
In this lecture, we’ll focus on unit testing
Test
Module¶Julia provides testing features through a built-in package called Test
, which we get by using Test
The basic object is the macro @test
using Test
@test 1 == 1
@test 1 ≈ 1
Test Passed
Tests will pass if the condition is true
, or fail otherwise
If a test is failing, we should flag it and move on
@test_broken 1 == 2
Test Broken
Expression: 1 == 2
This way, we still have access to information about the test, instead of just deleting it or commenting it out
There are other test macros, that check for things like error handling and type-stability. Advanced users can check the Julia docs
Let’s add some unit tests for the foo()
function we defined earlier. Our tests/runtests.jl
file should look like this
using ExamplePackage
using Test
@test foo() == 0.11388071406436832
@test foo(1, 1.5) == 0.2731856314283442
@test_broken foo(1, 0) # tells us this is broken
importing ExamplePackage into Main conflicts with an existing identifier Stacktrace: [1] top-level scope at In[27]:1
And run it by running
] test
Testing ExamplePackage Resolving package versions...
┌ Warning: Could not load Revise. └ @ Main C:\Users\jlperla\.julia\config\startup.jl:6
Testing ExamplePackage tests passed
from an activated REPL
There are a few different ways to run the tests for your package
pkg> test ExamplePackage
pkg> test
(recall that you can activate with pkg> activate ExamplePackage
)runtests.jl
file (as below). Recall that we can get the path of the package by running using ExamplePackage; pathof(ExamplePackage)
from an unactivated REPLBy default, Travis should have access to all your repositories and deploy automatically
This includes private repos if you’re on a student developer pack or an academic plan (Travis detects this automatically)
To change this, go to “settings” under your GitHub profile
Click “Applications,” then “Travis CI,” then “Configure,” and choose the repos you want to be tracked
By default, Travis will compile and test your project (i.e., “build” it) for new commits and PRs for every tracked repo with a .travis.yml
file
We can see ours by opening it in Atom
# Documentation: http://docs.travis-ci.com/user/languages/julia/
language: julia
os:
- linux
- osx
julia:
- 1.0
- nightly
matrix:
allow_failures:
- julia: nightly
fast_finish: true
notifications:
email: false
after_success:
- julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder())'
UndefVarError: language not defined Stacktrace: [1] top-level scope at In[29]:1
This is telling Travis to build the project in Julia, on OSX and Linux, using Julia v1.0 and the latest (“nightly”)
It also says that if the nightly version doesn’t work, that shouldn’t register as a failure
Note You won’t need OSX unless you’re building something Mac-specific, like iOS or Swift. You can delete those lines to speed up the build. Likewise for nightly Julia
As above, builds are triggered whenever we push changes or open a pull request
For example, if we push our changes to the server and then click the Travis badge on the README, we should see something like
This gives us an overview of all the builds running for that commit
To inspect a build more closely (say, if it fails), we can click on it and expand the log options
Note that the build times here aren’t informative, because we can’t generally control the hardware to which our job is allocated
We can also cancel specific jobs, either from their specific pages or by clicking the grey “x” button on the dashboard
Lastly, we can trigger builds manually (without a new commit or PR) from the Travis overview
To commit without triggering a build, simply add [ci skip] somewhere inside the commit message
You’ll find that codecov is automatically enabled for public repos with Travis
For private ones, you’ll need to first get an access token
Add private scope in the CodeCov website, just like we did for Travis
Navigate to the repo settings page (i.e., https://codecov.io/gh/quanteconuser/ExamplePackage.jl/settings
for our repo) and copy the token
Next, go to your travis settings and add an environment variable as below
Click the CodeCov badge to see the build page for your project
This shows us that our tests cover 50 % of our functions in src//
To get a more granular view, we can click the src//
and the resultant filename
This shows us precisely which methods (and parts of methods) are untested
As mentioned in version control, sometimes we’ll want to work on external repos that are also Julia projects
The first thing is to ] dev
the git URL (or package name, if the project is a registered Julia package), which will both clone the git repo and sync it with the Julia package manager
For example, running
] dev Expectations
will clone the repo https://github.com/quantecon/Expectations.jl
to ~/.julia/dev
Make sure you do this from a fresh REPL
As a reminder, you can find the location of your ~/.julia
folder (called the “user depot”), by running
DEPOT_PATH[1]
"C:\\Users\\jlperla\\.julia"
The ] dev
command will also add the target to the package manager, so that whenever we run using Expectations
, Julia will load our cloned copy from that location
using Expectations
pathof(Expectations) # points to our git clone
Next, drag that folder to GitHub Desktop
The next step is to fork the original (external) package from its website (i.e., https://github.com/quantecon/Expectations.jl
) to your account (https://github.com/quanteconuser/Expectations.jl
in our case)
Lastly, edit the settings in GitHub Desktop (from the “Repository” dropdown) to reflect the new URL
Here, we’d change the highlighted text to read quanteconuser
, or whatever our GitHub ID is
If you make some changes in a text editor and return to GitHub Desktop, you’ll see something like
Here, for example, we’re revising the README
Clicking “commit to master” (recall that the checkboxes next to each file indicate whether it’s to be committed) and then pushing (e.g., hitting “push” under the “Repository” dropdown) will add the committed changes to your account
To confirm this, we can check the history on our account here; for more on working with git repositories, see the version control lecture
The green check mark indicates that Travis tests passed for this commit
Clicking “new pull request” from the pull requests tab will show us a snapshot of the changes, and let us create a pull request for project maintainers to review and approve
For more on PRs, see the relevant section of the version control lecture
If you have write access to the repo, we can skip the preceding steps about forking and changing the URL
You can use ] dev
on a package name or the URL of the package.
] dev Expectations
or ] dev https://github.com/quanteconuser/Expectations.jl.git
as an example for an unreleased package by URL
Which will again clone the repo to ~/.julia/dev
, and use it as a Julia package
using Expectations
pathof(Expectations) # points to our git clone
Next, drag that folder to GitHub Desktop
Then, in order to work with a project locally, all we need to do is open it in a text editor (like Atom)
From here, we can edit this package just like we created it ourselves and use GitHub Desktop to track versions of our package files (say, after ] up
, or editing source code, ] add Package
, etc.)
To “un-dev” a Julia package (say, if we want to use our old Expectations.jl
), you can simply run
] free Expectations
To delete it entirely, simply run
] rm Expectations
Both from a fresh REPL
Another goal of testing is to make sure that code doesn’t slow down significantly from one version to the next
We can do this using tools provided by the BenchmarkTools.jl
package
See the need for speed
lecture for more details
Following the instructions for a new project, create a new package on your github account called NewtonsMethod.jl
In this package, you should create a simple package to do Newton’s Method using the code you did in the Newton’s method exercise in Julia by example
In particular, within your package you should have two functions
newtonroot(f, f′; x₀, tol = 1E-7, maxiter = 1000)
newtonroot(f; x₀, tol = 1E-7, maxiter = 1000)
Where the second function one uses Automatic Differentiation to call the first.
The package should include
/src
directoryFor the tests, you should have at the very minimum
nothing
as discussed in error handling@test
for the root of a known function, given the f
and analytical f'
derivativesBigFloat
and not just a Float64
maxiter
is working (e.g. what happens if you call maxiter = 5
tol
is workingAnd anything else you can think of. You should be able to run ] test
for the project to check that the test-suite is running, and then ensure that it is running automatically on Travis CI
Push a commit to the repository which breaks one of the tests and see what the Travis CI reports after running the build