Motivation

This IJulia notebook is a list of nice tweaks of the Julia language. Features should roughly be listed with ascending importance.

How does your favorite technical computing language compare to this?

Unicode characters

When using variable names, you have the full range of special characters at your disposal.

In [1]:
α = 3.4
print(α)
pi
3.4
Out[1]:
π = 3.1415926535897...

Scalar multiplication

When multiplying variables with scalar values, computer code allows the same abbreviation that is used in common mathematical notation.

In [2]:
f = x -> 3x^3 + 8x^2 - 3x
f(3)
Out[2]:
144

Chaining comparisons

Multiple comparisons can be simultaneously checked in one line.

In [3]:
vals = rand(40)

intermediate = 0.2 .<= vals .<= 0.6
vals[intermediate]
Out[3]:
13-element Array{Float64,1}:
 0.201635
 0.527303
 0.328445
 0.525737
 0.446058
 0.54903 
 0.375686
 0.518639
 0.37518 
 0.379145
 0.505113
 0.585748
 0.238779

Pipes

To avoid cluttering your workspace, successive operations on an input can also be written as a pipe similar to linux shell commands.

In [4]:
a = rand(400)

# manually compute standard deviation of b
b = exp(a)
mu = mean(b)
centralized = b - mu
std = mean(centralized.^2)
Out[4]:
0.2829726154196466
In [5]:
# written with pipe
std = a |>
    exp |>
    x -> x - mean(x) |>
    x -> x.^2 |>
    mean
Out[5]:
0.2829726154196466

String interpolation

Variable values can easily be incorporated into a string.

In [6]:
fileName = "data.csv"
println("The values are stored in $fileName")
The values are stored in data.csv
In [7]:
a = [1; 2; 3]
println("The values of a are: $a")
The values of a are: [1,2,3]

Ternary operators

Ternary operators are an abbreviation for if ... else ... end expressions. The expression before "?" is the condition expression, and the ternary operation evaluates the expression before the ":" if the condition is true, or the expression after the ":" if it is false.

In [8]:
kk = 4
if kk > 3
    println("greater than 3")
else
    println("smaller than 3")
end
greater than 3
In [9]:
kk > 3 ? println("greater than 3") : println("smaller than 3")
greater than 3

Iterators

Iteration over all entries of a variable can be done without manual indexing.

In [10]:
a = [1; 2; 3]
for ii=1:length(a)
    print(a[ii], ", ")
end
1, 2, 3, 
In [11]:
for entry in a
    print(entry, ", ")
end
1, 2, 3, 

Multiple simultaneous assignments

Values of a tuple or array can be simultaneously be assigned to individual variables.

In [12]:
a = rand(10, 2)
(nRows, nCol) = size(a)
nRows
Out[12]:
10
In [13]:
(mu1, mu2) = mean(a, 1)
mu1
Out[13]:
0.5271161360696317
In [14]:
mu2
Out[14]:
0.426256324570898

Comprehensions

Comprehension is an easy way to create arrays where individual entries follow some structure.

In [15]:
a = [1:10]
Out[15]:
10-element Array{Int64,1}:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
In [16]:
a = [ii for ii=1:10]
Out[16]:
10-element Array{Int64,1}:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
In [17]:
a = [exp(ii)+2 for ii=1:10]
Out[17]:
10-element Array{Float64,1}:
     4.71828
     9.38906
    22.0855 
    56.5982 
   150.413  
   405.429  
  1098.63   
  2982.96   
  8105.08   
 22028.5    
In [18]:
a = [ii for ii=1:10, jj=1:10]
b = [jj for ii=1:10, jj=1:10]
(a, b)
Out[18]:
(
10x10 Array{Int64,2}:
  1   1   1   1   1   1   1   1   1   1
  2   2   2   2   2   2   2   2   2   2
  3   3   3   3   3   3   3   3   3   3
  4   4   4   4   4   4   4   4   4   4
  5   5   5   5   5   5   5   5   5   5
  6   6   6   6   6   6   6   6   6   6
  7   7   7   7   7   7   7   7   7   7
  8   8   8   8   8   8   8   8   8   8
  9   9   9   9   9   9   9   9   9   9
 10  10  10  10  10  10  10  10  10  10,

10x10 Array{Int64,2}:
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10
 1  2  3  4  5  6  7  8  9  10)
In [19]:
using Dates
dats = [Date(2012, 4, ii) for ii=1:10]
Out[19]:
10-element Array{Date,1}:
 2012-04-01
 2012-04-02
 2012-04-03
 2012-04-04
 2012-04-05
 2012-04-06
 2012-04-07
 2012-04-08
 2012-04-09
 2012-04-10
In [20]:
whichYear = [year(dt) for dt in dats]
Out[20]:
10-element Array{Any,1}:
 2012
 2012
 2012
 2012
 2012
 2012
 2012
 2012
 2012
 2012

Square bracket indexing

The syntax for indexing of variables makes use of with square brackets. This way, functions and variables can immediately be distinguished at first sight. Some languages - Matlab, for example - do not share this property.

In [21]:
a[2]
Out[21]:
2

Inline function definitions

Functions can be defined anywhere in the file, and need not reside in a separate file. This allows easy and natural decomposition of large tasks into separate pieces.

In [22]:
function addTwo(x)
    y = x + 2
    return y
end
Out[22]:
addTwo (generic function with 1 method)
In [23]:
addTwo(a[2])
Out[23]:
4

Like in other languages, functions naturally extend to vector inputs.

In [24]:
addTwo(a)
Out[24]:
10x10 Array{Int64,2}:
  3   3   3   3   3   3   3   3   3   3
  4   4   4   4   4   4   4   4   4   4
  5   5   5   5   5   5   5   5   5   5
  6   6   6   6   6   6   6   6   6   6
  7   7   7   7   7   7   7   7   7   7
  8   8   8   8   8   8   8   8   8   8
  9   9   9   9   9   9   9   9   9   9
 10  10  10  10  10  10  10  10  10  10
 11  11  11  11  11  11  11  11  11  11
 12  12  12  12  12  12  12  12  12  12

Splicing

Splicing function arguments allows seamlessly switching between functions that require separate arguments on the one hand, and functions that require individual arguments combined in one array.

In [25]:
function multipleArguments(a, b, c, d, e, f)
    return a + b + c + d + e + f
end
Out[25]:
multipleArguments (generic function with 1 method)
In [26]:
vals = (1, 2, 3, 4, 5, 6)
multipleArguments(vals...)
Out[26]:
21
In [27]:
vals = [1:6]
multipleArguments(vals...)
Out[27]:
21

Multiple dispatch

Function behaviour may vary depending on the type of the arguments. This way, multiple functions with equal name may co-exist.

In [28]:
function reciprocalValue(x::Int)
    return 1/x
end

function reciprocalValue(x::ASCIIString)
    return uppercase(x)
end

function reciprocalValue(x)
    println("Method only makes sense for numbers and strings")
end
Out[28]:
reciprocalValue (generic function with 3 methods)
In [29]:
reciprocalValue(8)
Out[29]:
0.125
In [30]:
reciprocalValue("hello")
Out[30]:
"HELLO"
In [31]:
reciprocalValue(NaN)
Method only makes sense for numbers and strings

Composite types

One can easily define highly customized own types. Through multiple dispatch, behaviour of common mathematical operators can be defined for any new type.

In [32]:
type simplexPoint
    x
    y
end
In [33]:
sp = simplexPoint(0.4, 0.6)
Out[33]:
simplexPoint(0.4,0.6)
In [34]:
function reciprocalValue(sp::simplexPoint)
    return simplexPoint(sp.y, sp.x)
end

reciprocalValue(sp)
Out[34]:
simplexPoint(0.6,0.4)

Macros

Julia comes with quite powerful metaprogramming skills. This allows you to work with both the values that are stored in the variables and the names of variables and functions that were used in the call. This way, you can take some code, manipulate it, and only then you evaluate it.

One example is the @stoptime macro. Before the macro evaluates the input, it starts a stopwatch, and it displays the time that was required after the evalution.

In [35]:
macro stoptime(expr)
    quote
        hhhh = 3 # some line of nonsense to show variable scope
        tic()
        $(esc(expr))
        toc()
    end
end
In [36]:
@stoptime repChar = inv(rand(1000, 1000))
repChar
elapsed time: 0.949734229 seconds
Out[36]:
1000x1000 Array{Float64,2}:
  0.178363   -0.162743   -0.0254373   …  -0.00384031  -0.063538   
 -0.0803985   0.118439    0.01193         0.0903718    0.0675967  
 -0.0284457  -0.0390301  -0.0196053       0.00446068  -0.000667719
  0.150812    0.0126206   0.0236722       0.0308458   -0.0932575  
 -0.123411   -0.0426361  -0.0873105      -0.00503272   0.0157746  
  0.214565   -0.115128   -0.0318926   …   0.0969399   -0.0998336  
 -0.347367   -0.0609739  -0.172953       -0.0747402    0.122522   
 -0.204523   -0.0351727  -0.0692144      -0.0144602   -0.00136389 
  0.13185    -0.0733354  -0.0482174       0.0287405   -0.143989   
  0.0883469  -0.0947949  -0.0697139       0.0313077   -0.105953   
 -0.129635    0.0566574   0.065695    …  -0.0798233    0.0216424  
  0.0717952   0.0307056   0.104658       -0.0263899    0.0151857  
 -0.439015    0.22097     0.166334       -0.00273109   0.0900819  
  ⋮                                   ⋱                           
 -0.0784419  -0.183168   -0.140212       -0.0602018    0.0352725  
 -0.298269   -0.0771679  -0.108061       -0.00613091   0.105007   
  0.170605    0.0200547   0.0724004   …   0.0164545   -0.0615973  
  0.0881894   0.140063    0.0796897       0.0677917    0.0218051  
 -0.0776588  -0.171787   -0.128733        0.0106796   -0.0193732  
 -0.152816   -0.0519178  -0.0670922      -0.00816523   0.0980098  
 -0.31304    -0.0735723  -0.0405657       0.00675744   0.129876   
  0.204485    0.0854393   0.0869174   …   0.0458707    0.00223715 
  0.384736    0.106793    0.210468       -0.079883    -0.250716   
  0.48209     0.0383054   0.0766118       0.0662156   -0.255816   
  0.10385    -0.029067   -0.00391804      0.00494362  -0.0123786  
 -0.0113096  -0.0315841  -0.0529644      -0.0250712    0.0265644  

Macros evaluate in a separate workspace.

In [37]:
try
    hhhh
    catch e
    show(e)
end
UndefVarError(:hhhh)

Using the macroexpand function one can easily look at the complete code that gets evaluated by the macro.

In [38]:
macroexpand(:(@stoptime repChar = inv(rand(1000, 1000))))
Out[38]:
quote  # In[35], line 3:
    #412#hhhh = 3 # line 4:
    tic() # line 5:
    repChar = inv(rand(1000,1000)) # line 6:
    toc()
end

Of course, this stopwatch macro already exists in Julia. It is called @time.

Another example is the @test macro, which allows extremely convenient testing of code.

In [39]:
using Base.Test
@test 3 == (2+1)

As a macro also receives the actual variable names during its call, it can print out the actual call if the test fails.

In [40]:
kk = 4
try
    @test kk == (2+1)
    catch e
    show(e)
end
ErrorException("test failed: kk == 2 + 1")
Example: squaredVariable

Another example, though not the best style, is the following macro that returns the squared value of any given variable. The value will be stored in a variable that matches the name of the original input variable, but with "_squared" appended. Hence, his macro messes with the current workspace, which is generally NOT recommended.

In this example, however, we explicitly want the code to conduct changes to the workspace that are not directly induced through the expression that is handed over to the macro. The macro uses eval to create a new variable with ending "_squared" in the workspace.

In [41]:
macro squaredVariable(a)
    println("Calculating the squared value of $a:")
    newVariableName = symbol(string(a, "_squared"))
    eval(:($newVariableName = $a^2))
end
In [42]:
k = 8
@squaredVariable k
Calculating the squared value of k:
Out[42]:
64
In [43]:
k_squared
Out[43]:
64

Metaprogramming

A crucial feature of the Julia language is that the syntax itself is just implemented in the language just like any other type (Int, Char, ...). Its type is Expr.

In [44]:
cmd = :(x = mean(rand(10)))
typeof(cmd)
Out[44]:
Expr

As with any other type, you can access its fields, which are:

In [45]:
names(cmd)
Out[45]:
3-element Array{Symbol,1}:
 :head
 :args
 :typ 

Hence, you can find the operation in the :head field, and its arguments in the :args field.

In [46]:
cmd.head
Out[46]:
:(=)
In [47]:
cmd.args
Out[47]:
2-element Array{Any,1}:
 :x               
 :(mean(rand(10)))

Example application: bootstrap macro

Using macros and other metaprogramming capabilities, some quite complicated applications can be implemented very concisely. As an example, we now want to implement a macro called bootstrap. For any given Julia function call that evaluates some statistics for some given data sample, the macro shall re-calculate the same statistics for a given number of times with bootstrapped data.

To make the steps a little bit more obvious, let's see step by step, how a given command can be decomposed into the necessary parts.

In [48]:
expr = :(mu = mean(x))
Out[48]:
:(mu = mean(x))

At the top level, the command is an assignment.

In [49]:
expr.head
Out[49]:
:(=)

The left hand of the assignment can be accessed as follows:

In [50]:
expr.args[1]
Out[50]:
:mu

And the right hand is the complete function call:

In [51]:
expr.args[2]
Out[51]:
:(mean(x))

Again, this can be decomposed into the function that is called,

In [52]:
expr.args[2].args[1]
Out[52]:
:mean

and the name of the data variable that needs to be resampled:

In [53]:
expr.args[2].args[2]
Out[53]:
:x

Hence, we now could isolate both the sample data and the function that calculates the required statistics. We can then apply the same function to bootstrapped data samples.

In [54]:
macro bootstrap(nTimes, expr)
    quote
        # get real value
        $(esc(expr))

        # get function to resample
        func = $(expr.args[2].args[1])

        # get data as first argument to function
        data = $(expr.args[2].args[2])
        nObs = length(data)
        bootstrVals = Array(Any, $nTimes)
        for ii=1:$nTimes
            sampInd = rand(1:nObs, nObs)
            samp = data[sampInd]
        
            # apply function to sample
            bootstrVals[ii] = func(samp)
        end
        res = bootstrVals
    end
end

As we can see, the bootstrap macro works:

In [55]:
macroexpand(:(@bootstrap 1500000 mu = mean(x)))
Out[55]:
quote  # In[54], line 4:
    mu = mean(x) # line 7:
    #413#func = mean # line 10:
    #414#data = x # line 11:
    #415#nObs = length(#414#data) # line 12:
    #416#bootstrVals = Array(Any,1500000) # line 13:
    for #417#ii = 1:1500000 # line 14:
        #418#sampInd = rand(1:#415#nObs,#415#nObs) # line 15:
        #419#samp = #414#data[#418#sampInd] # line 18:
        #416#bootstrVals[#417#ii] = #413#func(#419#samp)
    end # line 20:
    #420#res = #416#bootstrVals
end
In [56]:
x = rand(200)
muBstr = @bootstrap 150000 mu = mean(x)
mu
Out[56]:
0.5093330623878153
In [57]:
muBstr
Out[57]:
150000-element Array{Any,1}:
 0.51001 
 0.534658
 0.515049
 0.504732
 0.513977
 0.514036
 0.491998
 0.523921
 0.534615
 0.525906
 0.494712
 0.489036
 0.500321
 ⋮       
 0.528564
 0.545485
 0.532547
 0.520392
 0.525585
 0.530451
 0.518228
 0.533759
 0.529191
 0.530825
 0.504388
 0.532594

Although the bootstrap macro only allows functions with only one argument, its reach can easily be extended through the use of anonymous functions.

In [58]:
varNineFive = x -> quantile(x, 0.95)
VaR_btstr = @bootstrap 150 VaR = varNineFive(x)
Out[58]:
150-element Array{Any,1}:
 0.932387
 0.955608
 0.965272
 0.966714
 0.931518
 0.965272
 0.965344
 0.955608
 0.9551  
 0.937821
 0.966714
 0.937387
 0.955608
 ⋮       
 0.936851
 0.966714
 0.965344
 0.937455
 0.936383
 0.937312
 0.953814
 0.966714
 0.966714
 0.936994
 0.953814
 0.955608

Session info

In [59]:
versioninfo()
Julia Version 0.3.5
Commit a05f87b* (2015-01-08 22:33 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i3-3240 CPU @ 3.40GHz
  WORD_SIZE: 64
  BLAS: libblas.so.3
  LAPACK: liblapack.so.3
  LIBM: libopenlibm
  LLVM: libLLVM-3.3
In [60]:
Pkg.status()
18 required packages:
 - DataArrays                    0.2.9
 - DataFrames                    0.6.0
 - Dates                         0.3.2
 - Debug                         0.0.4
 - Distributions                 0.6.3
 - EconDatasets                  0.0.2
 - GLM                           0.4.2
 - Gadfly                        0.3.10
 - IJulia                        0.1.16
 - JuMP                          0.7.3
 - MAT                           0.2.9
 - NLopt                         0.2.0
 - Quandl                        0.4.0
 - RDatasets                     0.1.1
 - Taro                          0.1.2
 - TimeData                      0.5.1
 - TimeSeries                    0.4.6
 - Winston                       0.11.7
56 additional packages:
 - ArrayViews                    0.4.8
 - BinDeps                       0.3.7
 - Blosc                         0.1.1
 - Cairo                         0.2.22
 - Calculus                      0.1.5
 - Codecs                        0.1.3
 - Color                         0.3.15
 - Compat                        0.2.10
 - Compose                       0.3.10
 - Contour                       0.0.6
 - Copulas                       0.0.0-             master (unregistered, dirty)
 - DataStructures                0.3.5
 - Distances                     0.2.0
 - DualNumbers                   0.1.1
 - Econometrics                  0.0.0-             master (unregistered)
 - FixedPointNumbers             0.0.6
 - GZip                          0.2.13
 - GnuTLS                        0.0.1
 - Graphs                        0.5.0
 - Grid                          0.3.7
 - Gumbo                         0.1.2
 - HDF5                          0.4.10
 - HTTPClient                    0.1.4
 - Hexagons                      0.0.2
 - HttpCommon                    0.0.11
 - HttpParser                    0.0.10
 - ImmutableArrays               0.0.6
 - IniFile                       0.2.4
 - Iterators                     0.1.7
 - JSON                          0.4.0
 - JavaCall                      0.2.0
 - KernelDensity                 0.1.0              b2c9f7d6 (dirty)
 - LibCURL                       0.1.4
 - Loess                         0.0.3
 - MathProgBase                  0.3.8
 - Memoize                       0.0.0
 - NaNMath                       0.0.2
 - Nettle                        0.1.7
 - NumericExtensions             0.6.2
 - NumericFuns                   0.2.3
 - Optim                         0.4.0
 - PDMats                        0.3.1
 - Plotly                        0.0.0-             master (unregistered)
 - REPLCompletions               0.0.3
 - Reexport                      0.0.2
 - Requests                      0.0.6
 - ReverseDiffSparse             0.1.10
 - SHA                           0.0.3
 - Showoff                       0.0.3
 - SortingAlgorithms             0.0.2
 - StatsBase                     0.6.10
 - Tk                            0.2.16
 - URIParser                     0.0.3
 - WorldBankDataTd               0.0.0-             master (unregistered)
 - ZMQ                           0.1.15
 - Zlib                          0.1.7