Note: Click on "Kernel" > "Restart Kernel and Clear All Outputs" in JupyterLab before reading this notebook to reset its output. If you cannot run this file on your machine, you may want to open it in the cloud .

Chapter 1: Elements of a Program¶

Do you remember how you first learned to speak in your mother tongue? Probably not. No one's memory goes back that far. Your earliest memory as a child should probably be around the age of three or four years old when you could already say simple things and interact with your environment. Although you did not know any grammar rules yet, other people just understood what you said. At least most of the time.

It is intuitively best to take the very mindset of a small child when learning a new language. And a programming language is no different from that. This first chapter introduces simplistic examples and we accept them as they are without knowing any of the "grammar" rules yet. Then, we analyze them in parts and slowly build up our understanding.

Consequently, if parts of this chapter do not make sense right away, let's not worry too much. Besides introducing the basic elements, it also serves as an outlook for what is to come. So, many terms and concepts used here are deconstructed in great detail in the following chapters.

Example: Averaging all even Numbers in a List¶

As our introductory example, we want to calculate the average of all evens in a list of whole numbers: [7, 11, 8, 5, 3, 12, 2, 6, 9, 10, 1, 4].

While we are used to finding an analytical solution in math (i.e., derive some equation with "pen and paper"), we solve this task programmatically instead.

We start by creating a list called numbers that holds all the individual numbers between brackets [ and ].

In [1]:

numbers = [7, 11, 8, 5, 3, 12, 2, 6, 9, 10, 1, 4]

To verify that something happened in our computer's memory, we reference numbers.

In [2]:

numbers

Out[2]:

[7, 11, 8, 5, 3, 12, 2, 6, 9, 10, 1, 4]

So far, so good. Let's see how the desired computation could be expressed as a sequence of instructions in the next code cell.

Intuitively, the line for number in numbers describes a "loop" over all the numbers in the numbers list, one at a time.

The if number % 2 == 0 may look confusing at first sight. Both % and == must have an unintuitive meaning here. Luckily, the comment in the same line after the # symbol has the answer: The program does something only for an even number.

In particular, it increases count by 1 and adds the current number onto the running total. Both count and number are initialized to 0 and the single = symbol reads as "... is set equal to ...". It cannot indicate a mathematical equation as, for example, count is generally not equal to count + 1.

Lastly, the average is calculated as the ratio of the final values of total and count. Overall, we divide the sum of all even numbers by their count: This is nothing but the definition of an average.

The lines of code "within" the for and if statements are indented and aligned with multiples of four spaces: This shows immediately how the lines relate to each other.

In [3]:

count = 0  # initialize variables to keep track of the
total = 0  # running total and the count of even numbers

for number in numbers:
    if number % 2 == 0:  # only work with even numbers
        count = count + 1
        total = total + number

average = total / count

We do not see any output yet but obtain the value of average by referencing it again.

In [4]:

average

Out[4]:

7.0

Output in a Jupyter Notebook¶

Only two of the previous four code cells generate an output while two remained "silent" (i.e., nothing appears below the cell after running it).

By default, Jupyter notebooks only show the value of the expression in the last line of a code cell. And, this output may also be suppressed by ending the last line with a semicolon ;.

In [5]:

"Hello, World!"
"I am feeling great :-)"

Out[5]:

'I am feeling great :-)'

In [6]:

"I am invisible!";

To see any output other than that, we use the built-in print() function. Here, the parentheses () indicate that we call (i.e., "execute") code written somewhere else.

In [7]:

print("Hello, World!")
print("I am feeling great :-)")

Hello, World!
I am feeling great :-)

Outside Jupyter notebooks, the semicolon ; is used as a separator between statements that must otherwise be on a line on their own. However, it is not considered good practice to use it as it makes code less readable.

In [8]:

print("Hello, World!"); print("I am feeling great :-)")

Hello, World!
I am feeling great :-)

(Arithmetic) Operators¶

Python comes with many built-in operators : They are tokens (i.e., "symbols") that have a special meaning to the Python interpreter.

The arithmetic operators either "operate" with the number immediately following them, so-called unary operators (e.g., negation), or "process" the two numbers "around" them, so-called binary operators (e.g., addition).

By definition, operators on their own have no permanent side effects in the computer's memory. Although the code cells in this section do indeed create new numbers in memory (e.g., 77 + 13 creates 90), they are immediately "forgotten" as they are not stored in a variable like numbers or average above. We develop this thought further in the second part of this chapter when we compare expressions with statements.

Let's see some examples of operators. We start with the binary + and the - operators for addition and subtraction. Binary operators mimic what mathematicians call infix notation and have the expected meaning.

In [9]:

77 + 13

Out[9]:

In [10]:

101 - 93

Out[10]:

The - operator may be used as a unary operator as well. Then, it unsurprisingly flips the sign of a number.

In [11]:

-1

Out[11]:

-1

When we compare the output of the * and / operators for multiplication and division, we note the subtle difference between the 42 and the 42.0: They are the same number represented as a different data type.

In [12]:

2 * 21

Out[12]:

In [13]:

84 / 2

Out[13]:

42.0

The so-called floor division operator // always "rounds" to an integer and is thus also called integer division operator. It is an example of an arithmetic operator we commonly do not know from high school mathematics.

In [14]:

84 // 2

Out[14]:

In [15]:

85 // 2

Out[15]:

Even though it appears that the // operator truncates (i.e., "cuts off") the decimals so as to effectively round down (i.e., the 42.5 became 42 in the previous code cell), this is not the case: The result is always "rounded" towards minus infinity!

In [16]:

-85 // 2

Out[16]:

-43

To obtain the remainder of a division, we use the modulo operator %.

In [17]:

85 % 2

Out[17]:

The remainder is 0 only if a number is divisible by another.

A popular convention in both computer science and mathematics is to abbreviate "only if" as "iff", which is short for "if and only if ." The iff means that a remainder of 0 implies that a number is divisible by another but also that a number's being divisible by another implies a remainder of 0. The implication goes in both directions!

So, 49 is divisible by 7.

In [18]:

49 % 7

Out[18]:

Modulo division is also useful if we want to extract the last couple of digits in a large integer.

In [19]:

789 % 10

Out[19]:

In [20]:

789 % 100

Out[20]:

The built-in divmod() function combines the integer and modulo divisions into one step. However, grammatically this is not an operator but a function. Also, divmod() returns a "pair" of integers and not a single one.

In [21]:

divmod(42, 10)

Out[21]:

(4, 2)

Raising a number to a power is performed with the exponentiation operator **. It is different from the ^ operator other programming languages may use and that also exists in Python with a different meaning.

In [22]:

2 ** 3

Out[22]:

The standard order of precedence from mathematics applies (i.e., PEMDAS rule) when several operators are combined.

In [23]:

3 ** 2 * 2 

Out[23]:

Parentheses help avoid confusion and take the role of a delimiter here.

In [24]:

(3 ** 2) * 2

Out[24]:

In [25]:

3 ** (2 * 2)

Out[25]:

Some programmers also use "style" conventions. For example, we might play with the whitespace, which is an umbrella term that refers to any non-printable sign like spaces, tabs, or the like. However, this is not a good practice and parentheses convey a much clearer picture.

In [26]:

3**2 * 2  # bad style; it is better to use parentheses here

Out[26]:

There exist many non-mathematical operators that are introduced throughout this book, together with the concepts they implement. They often come in a form different from the unary and binary ones mentioned above.

Operator Overloading¶

Python overloads certain operators. For example, you may not only "add" numbers but also text: This is called concatenation.

In [27]:

greeting = "Hi "
audience = "class"

In [28]:

greeting + audience

Out[28]:

'Hi class'

Multiplying text with a whole number also works.

In [29]:

10 * greeting

Out[29]:

'Hi Hi Hi Hi Hi Hi Hi Hi Hi Hi '

Objects vs. Types vs. Values¶

Python is a so-called object-oriented language, which is a paradigm of organizing a program's memory.

An object may be viewed as a "bag" of $0$ s and $1$ s in a given memory location. The $0$ s and $1$ s in a bag make up the object's value. There exist different types of bags, and each type comes with its own rules how the $0$ s and $1$ s are interpreted and may be worked with.

So, an object always has three main characteristics. Let's look at the following examples and work them out.

In [30]:

a = 42
b = 42.0
c = "Python rocks"

Identity / "Memory Location"¶

The built-in id() function shows an object's "address" in memory.

In [31]:

id(a)

Out[31]:

94371758832672

In [32]:

id(b)

Out[32]:

140673826512720

In [33]:

id(c)

Out[33]:

140673817995952

These addresses are not meaningful for anything other than checking if two variables reference the same object.

Obviously, a and b have the same value as revealed by the equality operator ==: We say a and b "evaluate equal." The resulting True - and the False further below - is yet another data type, a so-called boolean. We look into them in Chapter 3 .

In [34]:

a == b

Out[34]:

True

On the contrary, a and b are different objects as the identity operator is shows: They are stored at different addresses in the memory.

In [35]:

a is b

Out[35]:

False

If we want to check the opposite case, we use the negated version of the is operator, namely is not.

In [36]:

a is not b

Out[36]:

True

(Data) Type / "Behavior"¶

The type() built-in shows an object's type. For example, a is an integer (i.e., int) while b is a so-called floating-point number (i.e., float).

In [37]:

type(a)

Out[37]:

int

In [38]:

type(b)

Out[38]:

float

Different types imply different behaviors for the objects. The b object, for example, may be "asked" if it is a whole number with the .is_integer() "functionality" that comes with every float object.

Formally, we call such type-specific functionalities methods (i.e., as opposed to functions) and we look at them in detail in Chapter 11 . For now, it suffices to know that we access them with the dot operator . on the object. Of course, b is a whole number, which the boolean object True tells us.

In [39]:

b.is_integer()

Out[39]:

True

For an int object, this .is_integer() check does not make sense as we already know it is an int: We see the AttributeError below as a does not even know what is_integer() means.

In [40]:

a.is_integer()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-40-7db0a38aefcc> in <module>
----> 1 a.is_integer()

AttributeError: 'int' object has no attribute 'is_integer'

The c object is a so-called string type (i.e., str), which is Python's way of representing text. Strings also come with peculiar behaviors, for example, to make a text lower or upper case.

In [41]:

type(c)

Out[41]:

str

In [42]:

c.lower()

Out[42]:

'python rocks'

In [43]:

c.upper()

Out[43]:

'PYTHON ROCKS'

Value / (Semantic) "Meaning"¶

Almost trivially, every object also has a value to which it evaluates when referenced. We think of the value as the conceptual idea of what the $0$ s and $1$ s in the bag mean to humans. In other words, an object's value regards its semantic meaning.

For built-in data types, Python prints out an object's value as a so-called literal : This means that we may copy and paste the value back into a code cell and create a new object with the same value.

In [44]:

Out[44]:

In [45]:

Out[45]:

42.0

In this book, we follow the convention of creating strings with double quotes " instead of the single quotes ' to which Python defaults in its literal notation for str objects. Both types of quotes may be used interchangeably. So, the "Python rocks" from above and 'Python rocks' below create two objects that evaluate equal (i.e., "Python rocks" == 'Python rocks').

In [46]:

Out[46]:

'Python rocks'

Formal vs. Natural Languages¶

Just like the language of mathematics is good at expressing relationships among numbers and symbols, any programming language is just a formal language that is good at expressing computations.

Formal languages come with their own "grammatical rules" called syntax.

Syntax Errors¶

If we do not follow the rules, the code cannot be parsed correctly, i.e., the program does not even start to run but raises a syntax error indicated as SyntaxError in the output. Computers are very dumb in the sense that the slightest syntax error leads to the machine not understanding our code.

If we were to write an accounting program that adds up currencies, we would, for example, have to model dollar prices as float objects as the dollar symbol cannot be understood by Python.

In [47]:

3.99 $ + 10.40 $

  Cell In[47], line 1
    3.99 $ + 10.40 $
         ^
SyntaxError: invalid syntax

Python requires certain symbols at certain places (e.g., a : is missing here).

In [48]:

for number in numbers
    print(number)

  Cell In[48], line 1
    for number in numbers
                         ^
SyntaxError: expected ':'

Furthermore, it relies on whitespace (i.e., indentation), unlike many other programming languages. The IndentationError below is just a particular type of a SyntaxError.

In [49]:

for number in numbers:
print(number)

  Cell In[49], line 2
    print(number)
    ^
IndentationError: expected an indented block after 'for' statement on line 1

Runtime Errors¶

Syntax errors are easy to find as the code does not even run in the first place.

However, there are also so-called runtime errors that occur whenever otherwise (i.e., syntactically) correct code does not run because of invalid input. Runtime errors are also often referred to as exceptions.

This example does not work because just like in the "real" world, Python does not know how to divide by 0. The syntactically correct code leads to a ZeroDivisionError.

In [50]:

1 / 0

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
Cell In[50], line 1
----> 1 1 / 0

ZeroDivisionError: division by zero

Semantic Errors¶

So-called semantic errors, on the contrary, are hard to spot as they do not crash the program. The only way to find such errors is to run a program with test input for which we can predict the output. However, testing software is a whole discipline on its own and often very hard to do in practice.

The cell below copies our first example from above with a "tiny" error. How fast could you have spotted it without the comment?

In [51]:

count = 0
total = 0

for number in numbers:
    if number % 2 == 0:
        count = count + 1
        total = total + count  # count is wrong here, it should be number

average = total / count

In [52]:

average

Out[52]:

3.5

Systematically finding errors is called debugging. For the history of the term, see this article .

Best Practices¶

Thus, adhering to just syntax rules is never enough. Over time, best practices and style guides were created to make it less likely for a developer to mess up a program and also to allow "onboarding" him as a contributor to an established code base, often called legacy code, faster. These rules are not enforced by Python itself: Badly styled code still runs. At the very least, Python programs should be styled according to PEP 8 and documented "inline" (i.e., in the code itself) according to PEP 257 .

An easier to read version of PEP 8 is here. The video below features a well known Pythonista talking about the importance of code style.

In [53]:

from IPython.display import YouTubeVideo
YouTubeVideo("Hwckt4J96dI", width="60%")

Out[53]:

For example, while the above code to calculate the average of the even numbers in [7, 11, 8, 5, 3, 12, 2, 6, 9, 10, 1, 4] is correct, a Pythonista would rewrite it in a more "Pythonic" way and use the built-in sum() and len() functions (cf., Chapter 2 ) as well as a so-called list comprehension (cf., Chapter 8 ). Pythonic code runs faster in many cases and is less error-prone.

In [54]:

numbers = [7, 11, 8, 5, 3, 12, 2, 6, 9, 10, 1, 4]

In [55]:

evens = [n for n in numbers if n % 2 == 0]  # use of a list comprehension

In [56]:

evens

Out[56]:

[8, 12, 2, 6, 10, 4]

In [57]:

average = sum(evens) / len(evens)  # use built-in functions

In [58]:

average

Out[58]:

7.0

To get a rough overview of the mindset of a typical Python programmer, look at these rules, also known as the Zen of Python, that are deemed so important that they are included in every Python installation.

In [59]:

import this

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Jupyter Notebook Aspects¶

The Order of Code Cells is arbitrary¶

We can run the code cells in a Jupyter notebook in any arbitrary order.

That means, for example, that a variable defined towards the bottom could accidentally be referenced at the top of the notebook. This happens quickly when we iteratively built a program and go back and forth between cells.

As a good practice, it is recommended to click on "Kernel" > "Restart Kernel and Run All Cells" in the navigation bar once a notebook is finished. That restarts the Python process forgetting all state (i.e., all variables) and ensures that the notebook runs top to bottom without any errors the next time it is opened.

Notebooks are linear¶

While this book is built with Jupyter notebooks, it is crucial to understand that "real" programs are almost never "linear" (i.e., top to bottom) sequences of instructions but instead may take many different flows of execution.

At the same time, for a beginner's course, it is often easier to code linearly.

In real data science projects, one would probably employ a mixed approach and put reusable code into so-called Python modules (i.e., .py files; cf., Chapter 2 ) and then use Jupyter notebooks to build up a linear report or storyline for an analysis.