# Best practices in development¶

## One virtual environment per project¶

### Why¶

• Isolation
• Different projects have different dependency versions
• You don't want to mess up the system Python

### Tooling¶

pipenv is currently the recommended tool for dependency management but "consider other tools such as pip when pipenv does not meet your use case". If pipenv doesn't fit your use case, I recommend using virtualenvwrapper for managing virtual environments and pip for package management. On the other hand, if you're working on only a couple of projects, built-in venv will do just fine.

#### pyenv¶

• Easily change global / per project Python version
• Also a tool for installing different Python versions (also different runtimes available, e.g. PyPy)
• Useful if you'll need to work with different Python versions

### Why¶

• No surprises (especially in production)
• Make sure that everything works as expected
• Make sure that old stuff works as expected after introducing new features (regression)
• Tests give you confidence while refactoring
• Good tests demonstrate the use cases of application, i.e. they also document the implementation
• ...

### Tooling¶

#### pytest¶

There's unittest module in Python Standard Library but the go-to test runner nowadays is definitely pytest.

Some reasons to use pytest:

#### tox¶

tox makes it simple to test your code against different Python interpreter/runtime and dependency versions. This is important when you're writing software which should support different Python versions, which is usually the case with library-like packages. On the other hand, if you're developing, say, a web application which will be deployed to a known target platform, testing against multiple different versions is usually not necessary. However, tox makes it also possible to configure, for example, linting as part of tox run. Thus, tox may simplify the development workflow significantly by wrapping multiple different operations under a single command.

## Write high quality code¶

### Why¶

• Better maintainability
• Better quality == less bugs
In [ ]:
import this


### Tooling - code formatters¶

PEP8 (see also "for humans version") describes the style guidelines for Python code, you should follow them. Luckily, there are awesome tools that handle this for you while you focus on writing code, not formatting it.

#### black¶

• Note requires Python 3.6+
• Note "This is a beta product" while writing this
• Uncompromising - not as many configuration options as in yapf
• Good integration with editors
• Try it in the playground

### Tooling - linters¶

Automatic code formatting is great but in addition to that, you should use static analyzer (linter) to detect potential pitfalls in your code.

#### pylint¶

• More verbose compared to flake8
• Criticized about the default configuration but is easily configurable
• Gives an overall score

#### bandit¶

• Checks for security vulnerabilities

### Tooling - pre-commit¶

#### pre-commit¶

Ideally, all project contributors should follow the best practices of the project, let it be e.g. respecting PEP8 or making sure there's no linting errors or security vulnerabilities in their change sets. However, as code formatters and linters are (mainly) local tools, this can not be guaranteed. pre-commit let's you configure (.pre-commit-config.yaml file) a set of hooks that will be run as pre actions to a commit/push. After a developer has called once pre-commit install, these hooks will be then automatically ran prior each commit/push.

• Run formatting before commit
• Fail the commit in case linting errors
• Even exercise the test suite before the code ends up to remote (might be time consuming in most scenarios though)
• Easy to configure your own hooks
• And use the existing ones
• There's also pre-push option
• Written in Python but supports also other languages, such as Ruby, Go, and Rust
• Less failed CI builds!

## Structure your code and projects¶

### Why¶

• Package and module structure gives an overview about the project
• Modular design == better reusability

### How¶

Some general guidelines:

• Don't put too much stuff into one module
• Split project into packages
• Be consistent with your naming conventions

A few words about structuring your projects. If you're developing, say, a relative big business application, it makes sense to separate some of the non-core business logic packages into a separate project and publish that as separate package. This way the "main" repository doesn't get bloated and it's more approachable for newcomers. Additionally, there's a change that you (or someone else) can easily reuse this "separated" package in the future, which is often the case e.g. for different kinds of utility functionalities.

Let's take a practical example. If your team has two different applications which interact with the same third party, it's beneficial to implement a separate client library for communication with it. This way a change is needed only in one place (in the client library) if the third party decides to make a backwards incompatible change in their API.

### Tooling¶

#### cookiecutter¶

Cookiecutter is a tool which let's you create projects from predefined templates.

• Rapid set-up of new projects, no need to copy paste from existing ones
• Consistent development practices across all projects (project structure as well as e.g. pre-commit, tox, and CI configuration)
• You can create one yourself or use some of the existing ones
• Written in Python but is applicable for non-Python projects as well, even non-programming related directory and file structures

## Use continuous integration and deployment¶

CI & CD belong to the best practices of software development without controversy, no matter what is the technology stack used for development. From Python point of view, CI is the place where we want to make sure that the other best practices described above are followed. For example, in bigger projects, it may not be even practical/possible to run the full test suite on developer's machine.

### Why¶

• Make sure the tests pass
• CI is the place where it's possible to run also some time consuming tests which the impatient developers prefer to skip on their local machines
• Make sure there's no linting errors
• Ideally, the place to test against all target versions and platforms
• Overall, CI is the last resort for automatically ensuring the quality
• Manual deployments are time consuming and error-prone, CD is automated and deterministic
• You want to automate as much as possible, human time is expensive
• Minimize the time required for code reviews - what could be detected with automatic tools, should be detected by using those tools. Human time is expensive.

### Tooling¶

Tooling depends on which git repository manager option you've chosen and what kind of requirements you have. For example:

## Utilize the capabitilies of your editor¶

### Why¶

• Efficient and fluent development
• There's plenty of tools to make your daily programming easier, why would you not use them

### Tooling¶

As there's a number of different editors and IDEs available, not to mention that everyone has their own preferences, I'll just focus on highlighting some of the features of my favorite IDE, PyCharm, which I highly recommend for Python development.

#### PyCharm¶

• Good integration with pytest, e.g. run single tests / test classes / test modules
• Git integration (in case you don't like command line)
• Easy to configure to use automatic formatting, e.g black
• Intuitive searching capabilities
• Refactoring features
• Debugger
• Jupyter Notebook integration
• Free community edition already contains all you need

## Use existing solutions¶

### Why¶

• Python Standard Library is extensive - and stable!
• There are over 150k packages in PyPI
• Someone has most likely solved the problem you're trying to solve
• Spend 5 minutes doing a google research before starting to solve a new problem, e.g. stackoverflow is a magical place.

## Learn how to debug efficiently¶

### Why¶

• You won't write completely stable code anyway - impossible looking conditions will occur.
• When something is not working as expected, there are plenty of tools out there to help you figure out what's going on.

### Tooling - debuggers¶

#### pdb¶

• Part of the Standard Library
• Sufficient for most use cases

#### ipdb¶

• Feature rich pdb with similar API

#### pdb++¶

• Drop-in replacement for pdb with additional features

### Tooling - profilers¶

#### gprof2dot¶

• Find the performance bottlenecks of your application via illustrative graphs
• If you're using pytest (like you should), see also pytest-profiling which is powered by gprof2dot

#### py-spy¶

• Profile running Python program without the need for modifying the source code or restarting the target process
• Potential tool for identifying problems of e.g. a web application in production

#### objgraph¶

• Graphs which are useful while hunting memory leaks

### Tooling - runtime error tracking¶

These are especially useful with web applications as you'll get reports - and notifications - of runtime exceptions with full tracebacks and variable values. This information is often enough for identifying the root cause of the problem, which is a huge benefit considering the time required for implementing and deploying the fix.

#### Sentry¶

• Complete stack traces with relevant variable (locals()) values
• Browser and OS information of the client
• Support for other languages as well

#### rollbar¶

• Very similar to sentry

While writing this, both sentry and rollbar are free for hobbyist use.

### Tooling - misc¶

#### Use logging instead of prints¶

• logging is part of the Standard Library
• With logging you can redirect the output to a file
• Logs are usually the first place to look at after an end user reports an issue
• You can specify the runtime level - no need to remove the debug prints

### General guidelines¶

• Use Python 3, legacy Python (aka 2.7) will retire soon
• Develop in branches. Even if you're the only person in the project, branching makes it possible to easily switch between different features / bug fixes.
• If you're not developing alone, practice code reviews. It's one of the best ways to learn for both parties.