## Motivation¶

This IJulia notebook is a list of nice tweaks of the Julia language. Features should roughly be listed with ascending importance.

How does your favorite technical computing language compare to this?

## Unicode characters¶

When using variable names, you have the full range of special characters at your disposal.

In [1]:
α = 3.4
print(α)
pi

3.4
Out[1]:
π = 3.1415926535897...

## Scalar multiplication¶

When multiplying variables with scalar values, computer code allows the same abbreviation that is used in common mathematical notation.

In [2]:
f = x -> 3x^3 + 8x^2 - 3x
f(3)

Out[2]:
144

## Chaining comparisons¶

Multiple comparisons can be simultaneously checked in one line.

In [3]:
vals = rand(40)

intermediate = 0.2 .<= vals .<= 0.6
vals[intermediate]

Out[3]:
13-element Array{Float64,1}:
0.201635
0.527303
0.328445
0.525737
0.446058
0.54903
0.375686
0.518639
0.37518
0.379145
0.505113
0.585748
0.238779

## Pipes¶

To avoid cluttering your workspace, successive operations on an input can also be written as a pipe similar to linux shell commands.

In [4]:
a = rand(400)

# manually compute standard deviation of b
b = exp(a)
mu = mean(b)
centralized = b - mu
std = mean(centralized.^2)

Out[4]:
0.2829726154196466
In [5]:
# written with pipe
std = a |>
exp |>
x -> x - mean(x) |>
x -> x.^2 |>
mean

Out[5]:
0.2829726154196466

## String interpolation¶

Variable values can easily be incorporated into a string.

In [6]:
fileName = "data.csv"
println("The values are stored in $fileName")  The values are stored in data.csv  In [7]: a = [1; 2; 3] println("The values of a are:$a")

The values of a are: [1,2,3]


## Ternary operators¶

Ternary operators are an abbreviation for if ... else ... end expressions. The expression before "?" is the condition expression, and the ternary operation evaluates the expression before the ":" if the condition is true, or the expression after the ":" if it is false.

In [8]:
kk = 4
if kk > 3
println("greater than 3")
else
println("smaller than 3")
end

greater than 3

In [9]:
kk > 3 ? println("greater than 3") : println("smaller than 3")

greater than 3


## Iterators¶

Iteration over all entries of a variable can be done without manual indexing.

In [10]:
a = [1; 2; 3]
for ii=1:length(a)
print(a[ii], ", ")
end

1, 2, 3,
In [11]:
for entry in a
print(entry, ", ")
end

1, 2, 3,

## Multiple simultaneous assignments¶

Values of a tuple or array can be simultaneously be assigned to individual variables.

In [12]:
a = rand(10, 2)
(nRows, nCol) = size(a)
nRows

Out[12]:
10
In [13]:
(mu1, mu2) = mean(a, 1)
mu1

Out[13]:
0.5271161360696317
In [14]:
mu2

Out[14]:
0.426256324570898

## Comprehensions¶

Comprehension is an easy way to create arrays where individual entries follow some structure.

In [15]:
a = [1:10]

Out[15]:
10-element Array{Int64,1}:
1
2
3
4
5
6
7
8
9
10
In [16]:
a = [ii for ii=1:10]

Out[16]:
10-element Array{Int64,1}:
1
2
3
4
5
6
7
8
9
10
In [17]:
a = [exp(ii)+2 for ii=1:10]

Out[17]:
10-element Array{Float64,1}:
4.71828
9.38906
22.0855
56.5982
150.413
405.429
1098.63
2982.96
8105.08
22028.5    
In [18]:
a = [ii for ii=1:10, jj=1:10]
b = [jj for ii=1:10, jj=1:10]
(a, b)

Out[18]:
(
10x10 Array{Int64,2}:
1   1   1   1   1   1   1   1   1   1
2   2   2   2   2   2   2   2   2   2
3   3   3   3   3   3   3   3   3   3
4   4   4   4   4   4   4   4   4   4
5   5   5   5   5   5   5   5   5   5
6   6   6   6   6   6   6   6   6   6
7   7   7   7   7   7   7   7   7   7
8   8   8   8   8   8   8   8   8   8
9   9   9   9   9   9   9   9   9   9
10  10  10  10  10  10  10  10  10  10,

10x10 Array{Int64,2}:
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10
1  2  3  4  5  6  7  8  9  10)
In [19]:
using Dates
dats = [Date(2012, 4, ii) for ii=1:10]

Out[19]:
10-element Array{Date,1}:
2012-04-01
2012-04-02
2012-04-03
2012-04-04
2012-04-05
2012-04-06
2012-04-07
2012-04-08
2012-04-09
2012-04-10
In [20]:
whichYear = [year(dt) for dt in dats]

Out[20]:
10-element Array{Any,1}:
2012
2012
2012
2012
2012
2012
2012
2012
2012
2012

## Square bracket indexing¶

The syntax for indexing of variables makes use of with square brackets. This way, functions and variables can immediately be distinguished at first sight. Some languages - Matlab, for example - do not share this property.

In [21]:
a[2]

Out[21]:
2

## Inline function definitions¶

Functions can be defined anywhere in the file, and need not reside in a separate file. This allows easy and natural decomposition of large tasks into separate pieces.

In [22]:
function addTwo(x)
y = x + 2
return y
end

Out[22]:
addTwo (generic function with 1 method)
In [23]:
addTwo(a[2])

Out[23]:
4

Like in other languages, functions naturally extend to vector inputs.

In [24]:
addTwo(a)

Out[24]:
10x10 Array{Int64,2}:
3   3   3   3   3   3   3   3   3   3
4   4   4   4   4   4   4   4   4   4
5   5   5   5   5   5   5   5   5   5
6   6   6   6   6   6   6   6   6   6
7   7   7   7   7   7   7   7   7   7
8   8   8   8   8   8   8   8   8   8
9   9   9   9   9   9   9   9   9   9
10  10  10  10  10  10  10  10  10  10
11  11  11  11  11  11  11  11  11  11
12  12  12  12  12  12  12  12  12  12

## Splicing¶

Splicing function arguments allows seamlessly switching between functions that require separate arguments on the one hand, and functions that require individual arguments combined in one array.

In [25]:
function multipleArguments(a, b, c, d, e, f)
return a + b + c + d + e + f
end

Out[25]:
multipleArguments (generic function with 1 method)
In [26]:
vals = (1, 2, 3, 4, 5, 6)
multipleArguments(vals...)

Out[26]:
21
In [27]:
vals = [1:6]
multipleArguments(vals...)

Out[27]:
21

## Multiple dispatch¶

Function behaviour may vary depending on the type of the arguments. This way, multiple functions with equal name may co-exist.

In [28]:
function reciprocalValue(x::Int)
return 1/x
end

function reciprocalValue(x::ASCIIString)
return uppercase(x)
end

function reciprocalValue(x)
println("Method only makes sense for numbers and strings")
end

Out[28]:
reciprocalValue (generic function with 3 methods)
In [29]:
reciprocalValue(8)

Out[29]:
0.125
In [30]:
reciprocalValue("hello")

Out[30]:
"HELLO"
In [31]:
reciprocalValue(NaN)

Method only makes sense for numbers and strings


## Composite types¶

One can easily define highly customized own types. Through multiple dispatch, behaviour of common mathematical operators can be defined for any new type.

In [32]:
type simplexPoint
x
y
end

In [33]:
sp = simplexPoint(0.4, 0.6)

Out[33]:
simplexPoint(0.4,0.6)
In [34]:
function reciprocalValue(sp::simplexPoint)
return simplexPoint(sp.y, sp.x)
end

reciprocalValue(sp)

Out[34]:
simplexPoint(0.6,0.4)

## Macros¶

Julia comes with quite powerful metaprogramming skills. This allows you to work with both the values that are stored in the variables and the names of variables and functions that were used in the call. This way, you can take some code, manipulate it, and only then you evaluate it.

One example is the @stoptime macro. Before the macro evaluates the input, it starts a stopwatch, and it displays the time that was required after the evalution.

In [35]:
macro stoptime(expr)
quote
hhhh = 3 # some line of nonsense to show variable scope
tic()
$(esc(expr)) toc() end end  In [36]: @stoptime repChar = inv(rand(1000, 1000)) repChar  elapsed time: 0.949734229 seconds  Out[36]: 1000x1000 Array{Float64,2}: 0.178363 -0.162743 -0.0254373 … -0.00384031 -0.063538 -0.0803985 0.118439 0.01193 0.0903718 0.0675967 -0.0284457 -0.0390301 -0.0196053 0.00446068 -0.000667719 0.150812 0.0126206 0.0236722 0.0308458 -0.0932575 -0.123411 -0.0426361 -0.0873105 -0.00503272 0.0157746 0.214565 -0.115128 -0.0318926 … 0.0969399 -0.0998336 -0.347367 -0.0609739 -0.172953 -0.0747402 0.122522 -0.204523 -0.0351727 -0.0692144 -0.0144602 -0.00136389 0.13185 -0.0733354 -0.0482174 0.0287405 -0.143989 0.0883469 -0.0947949 -0.0697139 0.0313077 -0.105953 -0.129635 0.0566574 0.065695 … -0.0798233 0.0216424 0.0717952 0.0307056 0.104658 -0.0263899 0.0151857 -0.439015 0.22097 0.166334 -0.00273109 0.0900819 ⋮ ⋱ -0.0784419 -0.183168 -0.140212 -0.0602018 0.0352725 -0.298269 -0.0771679 -0.108061 -0.00613091 0.105007 0.170605 0.0200547 0.0724004 … 0.0164545 -0.0615973 0.0881894 0.140063 0.0796897 0.0677917 0.0218051 -0.0776588 -0.171787 -0.128733 0.0106796 -0.0193732 -0.152816 -0.0519178 -0.0670922 -0.00816523 0.0980098 -0.31304 -0.0735723 -0.0405657 0.00675744 0.129876 0.204485 0.0854393 0.0869174 … 0.0458707 0.00223715 0.384736 0.106793 0.210468 -0.079883 -0.250716 0.48209 0.0383054 0.0766118 0.0662156 -0.255816 0.10385 -0.029067 -0.00391804 0.00494362 -0.0123786 -0.0113096 -0.0315841 -0.0529644 -0.0250712 0.0265644  Macros evaluate in a separate workspace. In [37]: try hhhh catch e show(e) end  UndefVarError(:hhhh) Using the macroexpand function one can easily look at the complete code that gets evaluated by the macro. In [38]: macroexpand(:(@stoptime repChar = inv(rand(1000, 1000))))  Out[38]: quote # In[35], line 3: #412#hhhh = 3 # line 4: tic() # line 5: repChar = inv(rand(1000,1000)) # line 6: toc() end Of course, this stopwatch macro already exists in Julia. It is called @time. Another example is the @test macro, which allows extremely convenient testing of code. In [39]: using Base.Test @test 3 == (2+1)  As a macro also receives the actual variable names during its call, it can print out the actual call if the test fails. In [40]: kk = 4 try @test kk == (2+1) catch e show(e) end  ErrorException("test failed: kk == 2 + 1") ##### Example: squaredVariable¶ Another example, though not the best style, is the following macro that returns the squared value of any given variable. The value will be stored in a variable that matches the name of the original input variable, but with "_squared" appended. Hence, his macro messes with the current workspace, which is generally NOT recommended. In this example, however, we explicitly want the code to conduct changes to the workspace that are not directly induced through the expression that is handed over to the macro. The macro uses eval to create a new variable with ending "_squared" in the workspace. In [41]: macro squaredVariable(a) println("Calculating the squared value of$a:")
newVariableName = symbol(string(a, "_squared"))
eval(:($newVariableName =$a^2))
end

In [42]:
k = 8
@squaredVariable k

Calculating the squared value of k:

Out[42]:
64
In [43]:
k_squared

Out[43]:
64

## Metaprogramming¶

A crucial feature of the Julia language is that the syntax itself is just implemented in the language just like any other type (Int, Char, ...). Its type is Expr.

In [44]:
cmd = :(x = mean(rand(10)))
typeof(cmd)

Out[44]:
Expr

As with any other type, you can access its fields, which are:

In [45]:
names(cmd)

Out[45]:
3-element Array{Symbol,1}:
:args
:typ 

Hence, you can find the operation in the :head field, and its arguments in the :args field.

In [46]:
cmd.head

Out[46]:
:(=)
In [47]:
cmd.args

Out[47]:
2-element Array{Any,1}:
:x
:(mean(rand(10)))

## Example application: bootstrap macro¶

Using macros and other metaprogramming capabilities, some quite complicated applications can be implemented very concisely. As an example, we now want to implement a macro called bootstrap. For any given Julia function call that evaluates some statistics for some given data sample, the macro shall re-calculate the same statistics for a given number of times with bootstrapped data.

To make the steps a little bit more obvious, let's see step by step, how a given command can be decomposed into the necessary parts.

In [48]:
expr = :(mu = mean(x))

Out[48]:
:(mu = mean(x))

At the top level, the command is an assignment.

In [49]:
expr.head

Out[49]:
:(=)

The left hand of the assignment can be accessed as follows:

In [50]:
expr.args[1]

Out[50]:
:mu

And the right hand is the complete function call:

In [51]:
expr.args[2]

Out[51]:
:(mean(x))

Again, this can be decomposed into the function that is called,

In [52]:
expr.args[2].args[1]

Out[52]:
:mean

and the name of the data variable that needs to be resampled:

In [53]:
expr.args[2].args[2]

Out[53]:
:x

Hence, we now could isolate both the sample data and the function that calculates the required statistics. We can then apply the same function to bootstrapped data samples.

In [54]:
macro bootstrap(nTimes, expr)
quote
# get real value
$(esc(expr)) # get function to resample func =$(expr.args[2].args[1])

# get data as first argument to function
data = $(expr.args[2].args[2]) nObs = length(data) bootstrVals = Array(Any,$nTimes)
for ii=1:\$nTimes
sampInd = rand(1:nObs, nObs)
samp = data[sampInd]

# apply function to sample
bootstrVals[ii] = func(samp)
end
res = bootstrVals
end
end


As we can see, the bootstrap macro works:

In [55]:
macroexpand(:(@bootstrap 1500000 mu = mean(x)))

Out[55]:
quote  # In[54], line 4:
mu = mean(x) # line 7:
#413#func = mean # line 10:
#414#data = x # line 11:
#415#nObs = length(#414#data) # line 12:
#416#bootstrVals = Array(Any,1500000) # line 13:
for #417#ii = 1:1500000 # line 14:
#418#sampInd = rand(1:#415#nObs,#415#nObs) # line 15:
#419#samp = #414#data[#418#sampInd] # line 18:
#416#bootstrVals[#417#ii] = #413#func(#419#samp)
end # line 20:
#420#res = #416#bootstrVals
end
In [56]:
x = rand(200)
muBstr = @bootstrap 150000 mu = mean(x)
mu

Out[56]:
0.5093330623878153
In [57]:
muBstr

Out[57]:
150000-element Array{Any,1}:
0.51001
0.534658
0.515049
0.504732
0.513977
0.514036
0.491998
0.523921
0.534615
0.525906
0.494712
0.489036
0.500321
⋮
0.528564
0.545485
0.532547
0.520392
0.525585
0.530451
0.518228
0.533759
0.529191
0.530825
0.504388
0.532594

Although the bootstrap macro only allows functions with only one argument, its reach can easily be extended through the use of anonymous functions.

In [58]:
varNineFive = x -> quantile(x, 0.95)
VaR_btstr = @bootstrap 150 VaR = varNineFive(x)

Out[58]:
150-element Array{Any,1}:
0.932387
0.955608
0.965272
0.966714
0.931518
0.965272
0.965344
0.955608
0.9551
0.937821
0.966714
0.937387
0.955608
⋮
0.936851
0.966714
0.965344
0.937455
0.936383
0.937312
0.953814
0.966714
0.966714
0.936994
0.953814
0.955608

## Session info¶

In [59]:
versioninfo()

Julia Version 0.3.5
Commit a05f87b* (2015-01-08 22:33 UTC)
Platform Info:
System: Linux (x86_64-linux-gnu)
CPU: Intel(R) Core(TM) i3-3240 CPU @ 3.40GHz
WORD_SIZE: 64
BLAS: libblas.so.3
LAPACK: liblapack.so.3
LIBM: libopenlibm
LLVM: libLLVM-3.3

In [60]:
Pkg.status()

18 required packages:
- DataArrays                    0.2.9
- DataFrames                    0.6.0
- Dates                         0.3.2
- Debug                         0.0.4
- Distributions                 0.6.3
- EconDatasets                  0.0.2
- GLM                           0.4.2
- IJulia                        0.1.16
- JuMP                          0.7.3
- MAT                           0.2.9
- NLopt                         0.2.0
- Quandl                        0.4.0
- RDatasets                     0.1.1
- Taro                          0.1.2
- TimeData                      0.5.1
- TimeSeries                    0.4.6
- Winston                       0.11.7
- ArrayViews                    0.4.8
- BinDeps                       0.3.7
- Blosc                         0.1.1
- Cairo                         0.2.22
- Calculus                      0.1.5
- Codecs                        0.1.3
- Color                         0.3.15
- Compat                        0.2.10
- Compose                       0.3.10
- Contour                       0.0.6
- Copulas                       0.0.0-             master (unregistered, dirty)
- DataStructures                0.3.5
- Distances                     0.2.0
- DualNumbers                   0.1.1
- Econometrics                  0.0.0-             master (unregistered)
- FixedPointNumbers             0.0.6
- GZip                          0.2.13
- GnuTLS                        0.0.1
- Graphs                        0.5.0
- Grid                          0.3.7
- Gumbo                         0.1.2
- HDF5                          0.4.10
- HTTPClient                    0.1.4
- Hexagons                      0.0.2
- HttpCommon                    0.0.11
- HttpParser                    0.0.10
- ImmutableArrays               0.0.6
- IniFile                       0.2.4
- Iterators                     0.1.7
- JSON                          0.4.0
- JavaCall                      0.2.0
- KernelDensity                 0.1.0              b2c9f7d6 (dirty)
- LibCURL                       0.1.4
- Loess                         0.0.3
- MathProgBase                  0.3.8
- Memoize                       0.0.0
- NaNMath                       0.0.2
- Nettle                        0.1.7
- NumericExtensions             0.6.2
- NumericFuns                   0.2.3
- Optim                         0.4.0
- PDMats                        0.3.1
- Plotly                        0.0.0-             master (unregistered)
- REPLCompletions               0.0.3
- Reexport                      0.0.2
- Requests                      0.0.6
- ReverseDiffSparse             0.1.10
- SHA                           0.0.3
- Showoff                       0.0.3
- SortingAlgorithms             0.0.2
- StatsBase                     0.6.10
- Tk                            0.2.16
- URIParser                     0.0.3
- WorldBankDataTd               0.0.0-             master (unregistered)
- ZMQ                           0.1.15
- Zlib                          0.1.7