#!/usr/bin/env python
# coding: utf-8

# # Introduction to Python programming

# J.R. Johansson (jrjohansson at gmail.com)
# 
# The latest version of this [IPython notebook](http://ipython.org/notebook.html) lecture is available at [http://github.com/jrjohansson/scientific-python-lectures](http://github.com/jrjohansson/scientific-python-lectures).
# 
# The other notebooks in this lecture series are indexed at [http://jrjohansson.github.io](http://jrjohansson.github.io).

# ## Python program files

# * Python code is usually stored in text files with the file ending "`.py`":
# 
#         myprogram.py
# 
# * Every line in a Python program file is assumed to be a Python statement, or part thereof. 
# 
#     * The only exception is comment lines, which start with the character `#` (optionally preceded by an arbitrary number of white-space characters, i.e., tabs or spaces). Comment lines are usually ignored by the Python interpreter.
# 
# 
# * To run our Python program from the command line we use:
# 
#         $ python myprogram.py
# 
# * On UNIX systems it is common to define the path to the interpreter on the first line of the program (note that this is a comment line as far as the Python interpreter is concerned):
# 
#         #!/usr/bin/env python
# 
#   If we do, and if we additionally set the file script to be executable, we can run the program like this:
# 
#         $ myprogram.py

# ### Example:

# In[1]:


ls scripts/hello-world*.py


# In[2]:


cat scripts/hello-world.py


# In[3]:


get_ipython().system('python scripts/hello-world.py')


# ### Character encoding

# The standard character encoding is ASCII, but we can use any other encoding, for example UTF-8. To specify that UTF-8 is used we include the special line
# 
#     # -*- coding: UTF-8 -*-
# 
# at the top of the file.

# In[4]:


cat scripts/hello-world-in-swedish.py


# In[5]:


get_ipython().system('python scripts/hello-world-in-swedish.py')


# Other than these two *optional* lines in the beginning of a Python code file, no additional code is required for initializing a program. 

# ## IPython notebooks

# This file - an IPython notebook -  does not follow the standard pattern with Python code in a text file. Instead, an IPython notebook is stored as a file in the [JSON](http://en.wikipedia.org/wiki/JSON) format. The advantage is that we can mix formatted text, Python code and code output. It requires the IPython notebook server to run it though, and therefore isn't a stand-alone Python program as described above. Other than that, there is no difference between the Python code that goes into a program file or an IPython notebook.

# ## Modules

# Most of the functionality in Python is provided by *modules*. The Python Standard Library is a large collection of modules that provides *cross-platform* implementations of common facilities such as access to the operating system, file I/O, string management, network communication, and much more.

# ### References

#  * The Python Language Reference: http://docs.python.org/2/reference/index.html
#  * The Python Standard Library: http://docs.python.org/2/library/
# 
# To use a module in a Python program it first has to be imported. A module can be imported using the `import` statement. For example, to import the module `math`, which contains many standard mathematical functions, we can do:

# In[6]:


import math


# This includes the whole module and makes it available for use later in the program. For example, we can do:

# In[7]:


import math

x = math.cos(2 * math.pi)

print(x)


# Alternatively, we can choose to import all symbols (functions and variables) in a module to the current namespace so that we don't need to use the prefix "`math.`" every time we use something from the `math` module:

# In[8]:


from math import *

x = cos(2 * pi)

print(x)


# This pattern can be very convenient, but in large programs that include many modules it is often a good idea to keep the symbols from each module in their own namespaces, by using the `import math` pattern. This would eliminate potentially confusing problems with name space collisions.
# 
# As a third alternative, we can choose to import only a few selected symbols from a module by explicitly listing which ones we want to import instead of using the wildcard character `*`:

# In[9]:


from math import cos, pi

x = cos(2 * pi)

print(x)


# ### Looking at what a module contains, and its documentation

# Once a module is imported, we can list the symbols it provides using the `dir` function:

# In[10]:


import math

print(dir(math))


# And using the function `help` we can get a description of each function (almost .. not all functions have docstrings, as they are technically called, but the vast majority of functions are documented this way). 

# In[11]:


help(math.log)


# In[12]:


log(10)


# In[13]:


log(10, 2)


# We can also use the `help` function directly on modules: Try
# 
#     help(math) 
# 
# Some very useful modules form the Python standard library are `os`, `sys`, `math`, `shutil`, `re`, `subprocess`, `multiprocessing`, `threading`. 
# 
# A complete lists of standard modules for Python 2 and Python 3 are available at http://docs.python.org/2/library/ and http://docs.python.org/3/library/, respectively.

# ## Variables and types

# ### Symbol names 

# Variable names in Python can contain alphanumerical characters `a-z`, `A-Z`, `0-9` and some special characters such as `_`. Normal variable names must start with a letter. 
# 
# By convention, variable names start with a lower-case letter, and Class names start with a capital letter. 
# 
# In addition, there are a number of Python keywords that cannot be used as variable names. These keywords are:
# 
#     and, as, assert, break, class, continue, def, del, elif, else, except, 
#     exec, finally, for, from, global, if, import, in, is, lambda, not, or,
#     pass, print, raise, return, try, while, with, yield
# 
# Note: Be aware of the keyword `lambda`, which could easily be a natural variable name in a scientific program. But being a keyword, it cannot be used as a variable name.

# ### Assignment

# 
# 
# The assignment operator in Python is `=`. Python is a dynamically typed language, so we do not need to specify the type of a variable when we create one.
# 
# Assigning a value to a new variable creates the variable:

# In[14]:


# variable assignments
x = 1.0
my_variable = 12.2


# Although not explicitly specified, a variable does have a type associated with it. The type is derived from the value that was assigned to it.

# In[15]:


type(x)


# If we assign a new value to a variable, its type can change.

# In[16]:


x = 1


# In[17]:


type(x)


# If we try to use a variable that has not yet been defined we get an `NameError`:

# In[18]:


print(y)


# ### Fundamental types

# In[19]:


# integers
x = 1
type(x)


# In[20]:


# float
x = 1.0
type(x)


# In[21]:


# boolean
b1 = True
b2 = False

type(b1)


# In[22]:


# complex numbers: note the use of `j` to specify the imaginary part
x = 1.0 - 1.0j
type(x)


# In[23]:


print(x)


# In[24]:


print(x.real, x.imag)


# ### Type utility functions

# 
# The module `types` contains a number of type name definitions that can be used to test if variables are of certain types:

# In[25]:


import types

# print all types defined in the `types` module
print(dir(types))


# In[26]:


x = 1.0

# check if the variable x is a float
type(x) is float


# In[27]:


# check if the variable x is an int
type(x) is int


# We can also use the `isinstance` method for testing types of variables:

# In[28]:


isinstance(x, float)


# ### Type casting

# In[29]:


x = 1.5

print(x, type(x))


# In[30]:


x = int(x)

print(x, type(x))


# In[31]:


z = complex(x)

print(z, type(z))


# In[32]:


x = float(z)


# Complex variables cannot be cast to floats or integers. We need to use `z.real` or `z.imag` to extract the part of the complex number we want:

# In[33]:


y = bool(z.real)

print(z.real, " -> ", y, type(y))

y = bool(z.imag)

print(z.imag, " -> ", y, type(y))


# ## Operators and comparisons

# Most operators and comparisons in Python work as one would expect:
# 
# * Arithmetic operators `+`, `-`, `*`, `/`, `//` (integer division), '**' power
# 

# In[34]:


1 + 2, 1 - 2, 1 * 2, 1 / 2


# In[35]:


1.0 + 2.0, 1.0 - 2.0, 1.0 * 2.0, 1.0 / 2.0


# In[36]:


# Integer division of float numbers
3.0 // 2.0


# In[37]:


# Note! The power operators in python isn't ^, but **
2 ** 2


# Note: The `/` operator always performs a floating point division in Python 3.x.
# This is not true in Python 2.x, where the result of `/` is always an integer if the operands are integers.
# to be more specific, `1/2 = 0.5` (`float`) in Python 3.x, and `1/2 = 0` (`int`) in Python 2.x (but `1.0/2 = 0.5` in Python 2.x).

# * The boolean operators are spelled out as the words `and`, `not`, `or`. 

# In[38]:


True and False


# In[39]:


not False


# In[40]:


True or False


# * Comparison operators `>`, `<`, `>=` (greater or equal), `<=` (less or equal), `==` equality, `is` identical.

# In[41]:


2 > 1, 2 < 1


# In[42]:


2 > 2, 2 < 2


# In[43]:


2 >= 2, 2 <= 2


# In[44]:


# equality
[1,2] == [1,2]


# In[45]:


# objects identical?
l1 = l2 = [1,2]

l1 is l2


# ## Compound types: Strings, List and dictionaries

# ### Strings

# Strings are the variable type that is used for storing text messages. 

# In[46]:


s = "Hello world"
type(s)


# In[47]:


# length of the string: the number of characters
len(s)


# In[48]:


# replace a substring in a string with something else
s2 = s.replace("world", "test")
print(s2)


# We can index a character in a string using `[]`:

# In[49]:


s[0]


# **Heads up MATLAB users:** Indexing start at 0!
# 
# We can extract a part of a string using the syntax `[start:stop]`, which extracts characters between index `start` and `stop` -1 (the character at index `stop` is not included):

# In[50]:


s[0:5]


# In[51]:


s[4:5]


# If we omit either (or both) of `start` or `stop` from `[start:stop]`, the default is the beginning and the end of the string, respectively:

# In[52]:


s[:5]


# In[53]:


s[6:]


# In[54]:


s[:]


# We can also define the step size using the syntax `[start:end:step]` (the default value for `step` is 1, as we saw above):

# In[55]:


s[::1]


# In[56]:


s[::2]


# This technique is called *slicing*. Read more about the syntax here: http://docs.python.org/release/2.7.3/library/functions.html?highlight=slice#slice

# Python has a very rich set of functions for text processing. See for example http://docs.python.org/2/library/string.html for more information.

# #### String formatting examples

# In[57]:


print("str1", "str2", "str3")  # The print statement concatenates strings with a space


# In[58]:


print("str1", 1.0, False, -1j)  # The print statements converts all arguments to strings


# In[59]:


print("str1" + "str2" + "str3") # strings added with + are concatenated without space


# In[60]:


print("value = %f" % 1.0)       # we can use C-style string formatting


# In[61]:


# this formatting creates a string
s2 = "value1 = %.2f. value2 = %d" % (3.1415, 1.5)

print(s2)


# In[62]:


# alternative, more intuitive way of formatting a string 
s3 = 'value1 = {0}, value2 = {1}'.format(3.1415, 1.5)

print(s3)


# ### List

# Lists are very similar to strings, except that each element can be of any type.
# 
# The syntax for creating lists in Python is `[...]`:

# In[63]:


l = [1,2,3,4]

print(type(l))
print(l)


# We can use the same slicing techniques to manipulate lists as we could use on strings:

# In[64]:


print(l)

print(l[1:3])

print(l[::2])


# **Heads up MATLAB users:** Indexing starts at 0!

# In[65]:


l[0]


# Elements in a list do not all have to be of the same type:

# In[66]:


l = [1, 'a', 1.0, 1-1j]

print(l)


# Python lists can be inhomogeneous and arbitrarily nested:

# In[67]:


nested_list = [1, [2, [3, [4, [5]]]]]

nested_list


# Lists play a very important role in Python. For example they are used in loops and other flow control structures (discussed below). There are a number of convenient functions for generating lists of various types, for example the `range` function:

# In[68]:


start = 10
stop = 30
step = 2

range(start, stop, step)


# In[69]:


# in python 3 range generates an iterator, which can be converted to a list using 'list(...)'.
# It has no effect in python 2
list(range(start, stop, step))


# In[70]:


list(range(-10, 10))


# In[71]:


s


# In[72]:


# convert a string to a list by type casting:
s2 = list(s)

s2


# In[73]:


# sorting lists
s2.sort()

print(s2)


# #### Adding, inserting, modifying, and removing elements from lists

# In[74]:


# create a new empty list
l = []

# add an elements using `append`
l.append("A")
l.append("d")
l.append("d")

print(l)


# We can modify lists by assigning new values to elements in the list. In technical jargon, lists are *mutable*.

# In[75]:


l[1] = "p"
l[2] = "p"

print(l)


# In[76]:


l[1:3] = ["d", "d"]

print(l)


# Insert an element at an specific index using `insert`

# In[77]:


l.insert(0, "i")
l.insert(1, "n")
l.insert(2, "s")
l.insert(3, "e")
l.insert(4, "r")
l.insert(5, "t")

print(l)


# Remove first element with specific value using 'remove'

# In[78]:


l.remove("A")

print(l)


# Remove an element at a specific location using `del`:

# In[79]:


del l[7]
del l[6]

print(l)


# See `help(list)` for more details, or read the online documentation 

# ### Tuples

# Tuples are like lists, except that they cannot be modified once created, that is they are *immutable*. 
# 
# In Python, tuples are created using the syntax `(..., ..., ...)`, or even `..., ...`:

# In[80]:


point = (10, 20)

print(point, type(point))


# In[81]:


point = 10, 20

print(point, type(point))


# We can unpack a tuple by assigning it to a comma-separated list of variables:

# In[82]:


x, y = point

print("x =", x)
print("y =", y)


# If we try to assign a new value to an element in a tuple we get an error:

# In[83]:


point[0] = 20


# ### Dictionaries

# Dictionaries are also like lists, except that each element is a key-value pair. The syntax for dictionaries is `{key1 : value1, ...}`:

# In[84]:


params = {"parameter1" : 1.0,
          "parameter2" : 2.0,
          "parameter3" : 3.0,}

print(type(params))
print(params)


# In[85]:


print("parameter1 = " + str(params["parameter1"]))
print("parameter2 = " + str(params["parameter2"]))
print("parameter3 = " + str(params["parameter3"]))


# In[86]:


params["parameter1"] = "A"
params["parameter2"] = "B"

# add a new entry
params["parameter4"] = "D"

print("parameter1 = " + str(params["parameter1"]))
print("parameter2 = " + str(params["parameter2"]))
print("parameter3 = " + str(params["parameter3"]))
print("parameter4 = " + str(params["parameter4"]))


# ## Control Flow

# ### Conditional statements: if, elif, else

# The Python syntax for conditional execution of code uses the keywords `if`, `elif` (else if), `else`:

# In[87]:


statement1 = False
statement2 = False

if statement1:
    print("statement1 is True")
    
elif statement2:
    print("statement2 is True")
    
else:
    print("statement1 and statement2 are False")


# For the first time, here we encounter a peculiar and unusual aspect of the Python programming language: Program blocks are defined by their indentation level. 
# 
# Compare to the equivalent C code:
# 
#     if (statement1)
#     {
#         printf("statement1 is True\n");
#     }
#     else if (statement2)
#     {
#         printf("statement2 is True\n");
#     }
#     else
#     {
#         printf("statement1 and statement2 are False\n");
#     }
# 
# In C blocks are defined by the enclosing curly brackets `{` and `}`. And the level of indentation (white space before the code statements) does not matter (completely optional). 
# 
# But in Python, the extent of a code block is defined by the indentation level (usually a tab or say four white spaces). This means that we have to be careful to indent our code correctly, or else we will get syntax errors. 

# #### Examples:

# In[88]:


statement1 = statement2 = True

if statement1:
    if statement2:
        print("both statement1 and statement2 are True")


# In[89]:


# Bad indentation!
if statement1:
    if statement2:
    print("both statement1 and statement2 are True")  # this line is not properly indented


# In[90]:


statement1 = False 

if statement1:
    print("printed if statement1 is True")
    
    print("still inside the if block")


# In[91]:


if statement1:
    print("printed if statement1 is True")
    
print("now outside the if block")


# ## Loops

# In Python, loops can be programmed in a number of different ways. The most common is the `for` loop, which is used together with iterable objects, such as lists. The basic syntax is:

# ### **`for` loops**:

# In[92]:


for x in [1,2,3]:
    print(x)


# The `for` loop iterates over the elements of the supplied list, and executes the containing block once for each element. Any kind of list can be used in the `for` loop. For example:

# In[93]:


for x in range(4): # by default range start at 0
    print(x)


# Note: `range(4)` does not include 4 !

# In[94]:


for x in range(-3,3):
    print(x)


# In[95]:


for word in ["scientific", "computing", "with", "python"]:
    print(word)


# To iterate over key-value pairs of a dictionary:

# In[96]:


for key, value in params.items():
    print(key + " = " + str(value))


# Sometimes it is useful to have access to the indices of the values when iterating over a list. We can use the `enumerate` function for this:

# In[97]:


for idx, x in enumerate(range(-3,3)):
    print(idx, x)


# ### List comprehensions: Creating lists using `for` loops:

# A convenient and compact way to initialize lists:

# In[98]:


l1 = [x**2 for x in range(0,5)]

print(l1)


# ### `while` loops:

# In[99]:


i = 0

while i < 5:
    print(i)
    
    i = i + 1
    
print("done")


# Note that the `print("done")` statement is not part of the `while` loop body because of the difference in indentation.

# ## Functions

# A function in Python is defined using the keyword `def`, followed by a function name, a signature within parentheses `()`, and a colon `:`. The following code, with one additional level of indentation, is the function body.

# In[100]:


def func0():   
    print("test")


# In[101]:


func0()


# Optionally, but highly recommended, we can define a so called "docstring", which is a description of the functions purpose and behavior. The docstring should follow directly after the function definition, before the code in the function body.

# In[102]:


def func1(s):
    """
    Print a string 's' and tell how many characters it has    
    """
    
    print(s + " has " + str(len(s)) + " characters")


# In[103]:


help(func1)


# In[104]:


func1("test")


# Functions that returns a value use the `return` keyword:

# In[105]:


def square(x):
    """
    Return the square of x.
    """
    return x ** 2


# In[106]:


square(4)


# We can return multiple values from a function using tuples (see above):

# In[107]:


def powers(x):
    """
    Return a few powers of x.
    """
    return x ** 2, x ** 3, x ** 4


# In[108]:


powers(3)


# In[109]:


x2, x3, x4 = powers(3)

print(x3)


# ### Default argument and keyword arguments

# In a definition of a function, we can give default values to the arguments the function takes:

# In[110]:


def myfunc(x, p=2, debug=False):
    if debug:
        print("evaluating myfunc for x = " + str(x) + " using exponent p = " + str(p))
    return x**p


# If we don't provide a value of the `debug` argument when calling the the function `myfunc` it defaults to the value provided in the function definition:

# In[111]:


myfunc(5)


# In[112]:


myfunc(5, debug=True)


# If we explicitly list the name of the arguments in the function calls, they do not need to come in the same order as in the function definition. This is called *keyword* arguments, and is often very useful in functions that takes a lot of optional arguments.

# In[113]:


myfunc(p=3, debug=True, x=7)


# ### Unnamed functions (lambda function)

# In Python we can also create unnamed functions, using the `lambda` keyword:

# In[114]:


f1 = lambda x: x**2
    
# is equivalent to 

def f2(x):
    return x**2


# In[115]:


f1(2), f2(2)


# This technique is useful for example when we want to pass a simple function as an argument to another function, like this:

# In[116]:


# map is a built-in python function
map(lambda x: x**2, range(-3,4))


# In[117]:


# in python 3 we can use `list(...)` to convert the iterator to an explicit list
list(map(lambda x: x**2, range(-3,4)))


# ## Classes

# Classes are the key features of object-oriented programming. A class is a structure for representing an object and the operations that can be performed on the object. 
# 
# In Python a class can contain *attributes* (variables) and *methods* (functions).
# 
# A class is defined almost like a function, but using the `class` keyword, and the class definition usually contains a number of class method definitions (a function in a class).
# 
# * Each class method should have an argument `self` as its first argument. This object is a self-reference.
# 
# * Some class method names have special meaning, for example:
# 
#     * `__init__`: The name of the method that is invoked when the object is first created.
#     * `__str__` : A method that is invoked when a simple string representation of the class is needed, as for example when printed.
#     * There are many more, see http://docs.python.org/2/reference/datamodel.html#special-method-names

# In[118]:


class Point:
    """
    Simple class for representing a point in a Cartesian coordinate system.
    """
    
    def __init__(self, x, y):
        """
        Create a new Point at x, y.
        """
        self.x = x
        self.y = y
        
    def translate(self, dx, dy):
        """
        Translate the point by dx and dy in the x and y direction.
        """
        self.x += dx
        self.y += dy
        
    def __str__(self):
        return("Point at [%f, %f]" % (self.x, self.y))


# To create a new instance of a class:

# In[119]:


p1 = Point(0, 0) # this will invoke the __init__ method in the Point class

print(p1)         # this will invoke the __str__ method


# To invoke a class method in the class instance `p`:

# In[120]:


p2 = Point(1, 1)

p1.translate(0.25, 1.5)

print(p1)
print(p2)


# Note that calling class methods can modify the state of that particular class instance, but does not effect other class instances or any global variables.
# 
# That is one of the nice things about object-oriented design: code such as functions and related variables are grouped in separate and independent entities. 

# ## Modules

# One of the most important concepts in good programming is to reuse code and avoid repetitions.
# 
# The idea is to write functions and classes with a well-defined purpose and scope, and reuse these instead of repeating similar code in different part of a program (modular programming). The result is usually that readability and maintainability of a program is greatly improved. What this means in practice is that our programs have fewer bugs, are easier to extend and debug/troubleshoot. 
# 
# Python supports modular programming at different levels. Functions and classes are examples of tools for low-level modular programming. Python modules are a higher-level modular programming construct, where we can collect related variables, functions and classes in a module. A python module is defined in a python file (with file-ending `.py`), and it can be made accessible to other Python modules and programs using the `import` statement. 
# 
# Consider the following example: the file `mymodule.py` contains simple example implementations of a variable, function and a class:

# In[121]:


get_ipython().run_cell_magic('file', 'mymodule.py', '"""\nExample of a python module. Contains a variable called my_variable,\na function called my_function, and a class called MyClass.\n"""\n\nmy_variable = 0\n\ndef my_function():\n    """\n    Example function\n    """\n    return my_variable\n    \nclass MyClass:\n    """\n    Example class.\n    """\n\n    def __init__(self):\n        self.variable = my_variable\n        \n    def set_variable(self, new_value):\n        """\n        Set self.variable to a new value\n        """\n        self.variable = new_value\n        \n    def get_variable(self):\n        return self.variable\n')


# We can import the module `mymodule` into our Python program using `import`:

# In[122]:


import mymodule


# Use `help(module)` to get a summary of what the module provides:

# In[123]:


help(mymodule)


# In[124]:


mymodule.my_variable


# In[125]:


mymodule.my_function() 


# In[126]:


my_class = mymodule.MyClass() 
my_class.set_variable(10)
my_class.get_variable()


# If we make changes to the code in `mymodule.py`, we need to reload it using `reload`:

# In[127]:


reload(mymodule)  # works only in python 2


# ## Exceptions

# In Python errors are managed with a special language construct called "Exceptions". When errors occur exceptions can be raised, which interrupts the normal program flow and fallback to somewhere else in the code where the closest try-except statement is defined.

# To generate an exception we can use the `raise` statement, which takes an argument that must be an instance of the class `BaseException` or a class derived from it. 

# In[128]:


raise Exception("description of the error")


# A typical use of exceptions is to abort functions when some error condition occurs, for example:
# 
#     def my_function(arguments):
#     
#         if not verify(arguments):
#             raise Exception("Invalid arguments")
#         
#         # rest of the code goes here

# To gracefully catch errors that are generated by functions and class methods, or by the Python interpreter itself, use the `try` and  `except` statements:
# 
#     try:
#         # normal code goes here
#     except:
#         # code for error handling goes here
#         # this code is not executed unless the code
#         # above generated an error
# 
# For example:

# In[129]:


try:
    print("test")
    # generate an error: the variable test is not defined
    print(test)
except:
    print("Caught an exception")


# To get information about the error, we can access the `Exception` class instance that describes the exception by using for example:
# 
#     except Exception as e:

# In[130]:


try:
    print("test")
    # generate an error: the variable test is not defined
    print(test)
except Exception as e:
    print("Caught an exception:" + str(e))


# ## Further reading

# * http://www.python.org - The official web page of the Python programming language.
# * http://www.python.org/dev/peps/pep-0008 - Style guide for Python programming. Highly recommended. 
# * http://www.greenteapress.com/thinkpython/ - A free book on Python programming.
# * [Python Essential Reference](http://www.amazon.com/Python-Essential-Reference-4th-Edition/dp/0672329786) - A good reference book on Python programming.

# ## Versions

# In[131]:


get_ipython().run_line_magic('load_ext', 'version_information')

get_ipython().run_line_magic('version_information', '')