Co-authored with Arnav Sood
While Jupyter notebooks are a great way to get started with the language, eventually you will want to use more powerful tools
We’ll discuss a few of them here, such as
Note that we assume you’ve already completed the getting started and interacting with Julia lectures
Whenever the Julia compiler or REPL starts, it will look for a file called startup.jl
(see Julia Manual)
We provide a file here which does two things
;
for new commandsRevise.jl
package on startup, which lets you see changes you make to a package in real-time (i.e., no need to quit the REPL, open again, and load again)The location for the file is relative to your default Julia environment (e.g. ~/.julia/config/startup.jl
or C:\Users\USERNAME\.julia\config\startup.jl
on Windows)
Recall that you can find the location of the ~/.julia
directory by running
DEPOT_PATH[1]
"/home/qebuild/.julia"
Note: On Mac, this won’t be visible in the Finder unless you specifically enable that option, but you can get to it by running cd .julia; open .
from a new terminal
To add the file:
julia
terminal, type the following] add Revise REPL; precompile
~/.julia/config/
directory if necessary in the terminal or file explorerOn Windows, if you have a shortcut on your desktop or on the taskbar, you could: (1) right-click on the icon; (2) right click on the “julia” text; (3) choose “Properties”, and (4) change the “Start In” to be something such as C:\Users\YOURUSERNAME\Documents
Previously, we discussed basic use of the Julia REPL (“Read-Evaluate-Print Loop”)
Here, we’ll consider some more advanced features
Hitting ;
brings you into shell mode, which lets you run bash commands (PowerShell on Windows)
; pwd
/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed
You can also use Julia variables from shell mode
x = 2
2
; echo $x
2
Hitting ]
brings you into package mode
] add Expectations
will add a package (here, Expectations.jl
)] rm Expectations
will remove that package] st
will show you a snapshot of what you have installed] up
will (intelligently) upgrade versions of your packages] precompile
will precompile everything possibleYou can get a full list of package mode commands by running
] ?
Welcome to the Pkg REPL-mode. To return to the julia> prompt, either press backspace when the input line is empty or press Ctrl+C. Synopsis pkg> cmd [opts] [args] Multiple commands can be given on the same line by interleaving a ; between the commands. Commands activate: set the primary environment the package manager manipulates add: add packages to project build: run the build script for packages develop: clone the full package repo locally for development free: undoes a pin, develop, or stops tracking a repo gc: garbage collect packages not used for a significant time generate: generate files for a new project help: show this message instantiate: downloads all the dependencies for the project pin: pins the version of packages precompile: precompile all the project dependencies preview: previews a subsequent command without affecting the current state remove: remove packages from project or manifest resolve: resolves to update the manifest from changes in dependencies of developed packages status: summarize contents of and changes to environment test: run tests for packages update: update packages in manifest registry add: add package registries registry remove: remove package registries registry status: information about installed registries registry update: update package registries
On some operating systems (such as OSX) REPL pasting may not work for package mode, and you will need to access it in the standard way (i.e., hit ]
first and then run your commands)
Hitting ?
will bring you into help mode
The key use case is to find docstrings for functions and macros, e.g.
? print
Note that objects must be loaded for Julia to return their documentation, e.g.
? @test
will fail, but
using Test
? @test
will succeed
As discussed previously, eventually you will want to use a fully fledged text editor
The most feature-rich one for Julia development is Atom, with the Juno package
There are several reasons to use a text editor like Atom, including
Use Ctrl-,
to get the Settings
pane
Choose the Packages
tab
Type line-ending-selector
into the Filter and then click “Settings” for that package
LF
(only necessary on Windows)Choose the Editor tab
Soft Wrap
Tab Length
default to 4
Ctrl-,
to get the Settings paneInstall
tabuber-juno
into the search box and then click Install on the package that appearsyes
At that point, you should see a built-in REPL at the bottom of the screen and be able to start using Julia and Atom
Sometimes, Juno will fail to find the Julia executable (say, if it’s installed somewhere nonstandard, or you have multiple)
To do this
Ctrl-,
to get Settings pane, and select the Packages tabjulia-client
and choose Settings
- To find the binary, you could run
Sys.BINDIR
in the REPL, then add in an additional/julia
to the end of the screen
C:\Users\YOURUSERNAME\AppData\Local\Julia-1.0.1\bin\julia.exe
on Windows as /Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia
on OSXSee the setup instructions for Juno if you have further issues
If you follow the instructions, you should see something like this when you open a new file
If you don’t, simply go to the command palette and type “Julia standard layout”
The bottom pane is a standard REPL, which supports the different modes above
The “workspace” pane is a snapshot of currently-defined objects
For example, if we define an object in the REPL
x = 2
2
Our workspace should read
The ans
variable simply captures the result of the last computation
The Documentation
pane simply lets us query Julia documentation
The Plots
pane captures Julia plots output (the code is as follows)
using Plots
gr(fmt = :png);
data = rand(10, 10)
h = heatmap(data)
Note: The plots feature is not perfectly reliable across all plotting backends, see the Basic Usage page
Shift + Enter
will evaluate a highlighted selection or line (as above)Ctrl+Shift+Enter
) will run the whole fileSee basic usage for an exploration of features, and the FAQ for more advanced steps (e.g. using with Docker
)
Julia’s package manager lets you set up Python-style “virtualenvs,” or subsets of packages that draw from an underlying pool of assets on the machine
This way, you can work with (and specify) the dependencies (i.e., required packages) for one project without worrying about impacts on other projects
environment
is a set of packages specified by a Project.toml
(and optionally, a Manifest.toml
)registry
is a git repository corresponding to a list of (typically) registered packages, from which Julia can pull (for more on git repositories, see version control)depot
is a directory, like ~/.julia
, which contains assets (compile caches, registries, package source directories, etc.)Essentially, an environment is a dependency tree for a project, or a “frame of mind” for Julia’s package manager
v1.1
) environment as such] st
Project quantecon-notebooks-julia v0.1.0 Status `~/repos/lecture-source-jl/_build/website/jupyter/Project.toml` [6e4b80f9] BenchmarkTools v0.4.2 [a134a8b2] BlackBoxOptim v0.5.0 [324d7699] CategoricalArrays v0.5.5 [34da2185] Compat v2.1.0 [a93c6f00] DataFrames v0.19.1 [1313f7d8] DataFramesMeta v0.5.0 [5721bf48] DataVoyager v0.3.1 [39dd38d3] Dierckx v0.4.1 [31c24e10] Distributions v0.21.0 [2fe49d83] Expectations v1.1.0 [442a2c76] FastGaussQuadrature v0.3.3 [1a297f60] FillArrays v0.6.4 [9d5cd8c9] FixedEffectModels v0.8.1 [587475ba] Flux v0.8.3 [f6369f11] ForwardDiff v0.10.3 [38e38edf] GLM v1.3.1 [28b8d3ca] GR v0.40.0 [a98d9a8b] Interpolations v0.12.2 [b6b21f68] Ipopt v0.6.0 [4076af6c] JuMP v0.20.0 [5ab0869b] KernelDensity v0.5.1 [b964fa9f] LaTeXStrings v1.0.3 [0fc2ff8b] LeastSquaresOptim v0.7.2 [76087f3c] NLopt v0.5.1 [2774e3e8] NLsolve v4.0.0 [429524aa] Optim v0.19.1 [d96e819e] Parameters v0.10.3 [14b8a8f1] PkgTemplates v0.6.2 [91a5bcdd] Plots v0.25.3 [f27b6e38] Polynomials v0.5.2 [92933f4c] ProgressMeter v1.0.0 [1fd47b50] QuadGK v2.1.0 [fcd29c91] QuantEcon v0.16.1 [1a8c2f83] Query v0.12.1 [612083be] Queryverse v0.3.1 [ce6b1742] RDatasets v0.6.3 [295af30f] Revise v2.1.6 [f2b01f46] Roots v0.8.1 [90137ffa] StaticArrays v0.11.0 [2913bbd2] StatsBase v0.31.0 [3eaba693] StatsModels v0.6.3 [f3b207a7] StatsPlots v0.11.0 [112f6efa] VegaLite v0.7.0 [37e2e46d] LinearAlgebra [9a3f8284] Random [2f01184e] SparseArrays [10745b16] Statistics [8dfed614] Test
] generate ExampleEnvironment
Generating project ExampleEnvironment:
ExampleEnvironment/Project.toml
ExampleEnvironment/src/ExampleEnvironment.jl
; cd ExampleEnvironment
/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment
] activate .
Activating environment at `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Project.toml`
where “.” stands in for the “present working directory”
] add Expectations Parameters
Updating registry at `~/.julia/registries/General` Updating git-repo `https://github.com/JuliaRegistries/General.git` Resolving package versions... Updating `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Project.toml` [2fe49d83] + Expectations v1.1.1 [d96e819e] + Parameters v0.11.0 Updating `~/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed/ExampleEnvironment/Manifest.toml` [7d9fca2a] + Arpack v0.3.1 [9e28174c] + BinDeps v0.8.10 [b99e7846] + BinaryProvider v0.5.6 [34da2185] + Compat v2.1.0 [9a962f9c] + DataAPI v1.0.1 [864edb3b] + DataStructures v0.17.0 [31c24e10] + Distributions v0.21.1 [2fe49d83] + Expectations v1.1.1 [442a2c76] + FastGaussQuadrature v0.4.0 [e1d29d7a] + Missings v0.4.1 [bac558e1] + OrderedCollections v1.1.0 [90014a1f] + PDMats v0.9.9 [d96e819e] + Parameters v0.11.0 [1fd47b50] + QuadGK v2.0.3 [79098fc4] + Rmath v0.5.0 [a2af1166] + SortingAlgorithms v0.3.1 [276daf66] + SpecialFunctions v0.7.2 [2913bbd2] + StatsBase v0.32.0 [4c63d2b9] + StatsFuns v0.8.0 [30578b45] + URIParser v0.4.0 [2a0f44e3] + Base64 [ade2ca70] + Dates [8bb1440f] + DelimitedFiles [8ba89e20] + Distributed [b77e0a4c] + InteractiveUtils [76f85450] + LibGit2 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [a63ad114] + Mmap [44cfe95a] + Pkg [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA [9e88b42a] + Serialization [1a1011a3] + SharedArrays [6462fe0b] + Sockets [2f01184e] + SparseArrays [10745b16] + Statistics [4607b0f0] + SuiteSparse [8dfed614] + Test [cf7118a7] + UUIDs [4ec0a83e] + Unicode
Note the lack of commas
ExampleEnvironment
directory in an editor like AtomThe Project TOML should look something like this
name = "ExampleEnvironment"
uuid = "14d3e79e-e2e5-11e8-28b9-19823016c34c"
authors = ["QuantEcon User <quanteconuser@gmail.com>"]
version = "0.1.0"
[deps]
Expectations = "2fe49d83-0758-5602-8f54-1f90ad0d522b"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
We can also
] precompile
Precompiling project... Precompiling Expectations
┌ Info: Recompiling stale cache file /home/qebuild/.julia/compiled/v1.2/Expectations/r1e2K.ji for Expectations [2fe49d83-0758-5602-8f54-1f90ad0d522b] └ @ Base loading.jl:1240
Precompiling Parameters
┌ Info: Recompiling stale cache file /home/qebuild/.julia/compiled/v1.2/Parameters/ycYNs.ji for Parameters [d96e819e-fc66-5662-9728-84c9c7592b0a] └ @ Base loading.jl:1240
Precompiling ExampleEnvironment
┌ Info: Precompiling ExampleEnvironment [a9fe0181-583a-484a-be3c-5186ec7d44fa] └ @ Base loading.jl:1242
Note The TOML files are independent of the actual assets (which live in ~/.julia/packages
, ~/.julia/dev
, and ~/.julia/compiled
)
You can think of the TOML as specifying demands for resources, which are supplied by the ~/.julia
user depot
] activate
Activating environment at `~/repos/lecture-source-jl/_build/website/jupyter/Project.toml`
without any arguments
; cd ..
/home/qebuild/repos/lecture-source-jl/_build/website/jupyter/_downloads/executed
; rm -rf ExampleEnvironment
With this knowledge, we can explain the operation of the setup block
using InstantiateFromURL
github_project("QuantEcon/quantecon-notebooks-julia", version = "0.2.0")
Activated /home/qebuild/repos/lecture-source-jl/_build/website/jupyter/Project.toml Info quantecon-notebooks-julia 0.1.0 activated, 0.2.0 requested
What this activate_github
function does is
.projects
] activate
that environment, and] instantiate
and ] precompile
, if necessary_docker_main:
Docker is a tool that lets you run preconfigured, lightweight environments as applications on your computer or in a computational cloud
The advantage of a Docker-based workflow is that it’s perfectly reproducible, and that setup (of Julia versions and dependencies, etc.) is handled upstream by the image maintainer
Here, we’ll walk through the setup and installation steps, along with the main features of the quantecon/base
Docker image
First, create an account for Docker Hub and create a docker id
Next, download and install Docker for
Note: For Windows
- Hyper-V support should be enabled. For Windows 10 users, this means you must use a Pro, Enterprise, or Education version (not Home or Mobile)
If you don’t meet these requirements, the Docker Toolbox for Windows may help
Next, to verify that there are no obvious errors in the installation, open a terminal (macOS/Linux) or Powershell (Windows) and run
docker pull hello-world
docker run hello-world
You should see something like
docker pull quantecon/base
docker volume rm quantecon
docker volume create quantecon
The first line will delete any existing volume we had with that name
The basic command is (Linux, OS/X)
docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v "$(pwd)":/home/jovyan/local quantecon/base
Or on Powershell on Windows
docker run --rm -p 8888:8888 -v quantecon:/home/jovyan/.julia -v ${PWD}:/home/jovyan/local quantecon/base
rm
instructs Docker to delete the container on exit,PWD
statement will mount the local directory (i.e., where the terminal is) to the Docker for exchanging filesp
flag is for browser accessquantecon:/home/jovyan/.julia
mount is for persisting changes we make to the Julia user depotYou will see something like
In the output, you should see some text near that bottom that looks like
127.0.0.1):8888/?token=7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b
Copy the text after ?token=
(e.g. 7c8f37bf32b1d7f0b633596204ee7361c1213926a6f0a44b
)
In a browser, go to a URL like the following
http://127.0.0.1:8888/lab
To see something like
Note:
Ctrl+C
is also the keyboard shortcut you use to kill the container, so be sure to copy using the mousePaste the text into Password or token:
and choose Log in
to get the full window
We can see that some packages are already pre-installed for our use
A few reminders
Ctrl-C
or type docker stop \$(docker ps -aq)
in a different terminaldocker volume rm quantecon
and docker volume create quantecon
stepsdocker system prune
jupyter notebook list
and (if need be) jupyter notebook stop 8888
. If you have difficulties, see this git issue