NEMO User's Guide: an IPython Notebook

Installing a configuration file

Before you can run NEMO, you need a configuration file. The default configuration file (nemo.cfg) is installed with the NEMO package and can be copied into your working directory as a starting point. On Unix systems, this can be found at /usr/local/etc/nemo.cfg. Alternatively, you can set the NEMORC environment variable to point to a configuration file. See the "Configuration file" section below for more details on the format of this file.

A simple example

NEMO can be driven by your own Python code. Some simple examples of how to do this appear below. First, we will create a simulation with a single combined cycle gas turbine (CCGT). The "NSW1:31" notation indicates that the generator is sited in polygon 31 in the NSW1 region.

In [1]:
import nemo
from nemo import scenarios
c = nemo.Context()
scenarios._one_ccgt(c)
print c.generators
[CCGT (NSW1:31), 0 kW]

Then run the simulation:

In [2]:
nemo.run(c)
print c
Timesteps: 8760 h
Demand energy: 204.37 TWh
Unused surplus energy: 0 kWh
Unserved energy: 100.000%
WARNING: reliability standard exceeded
Unserved total hours: 8760
Number of unserved energy events: 1
Shortfalls (min, max): (15.469 GW, 33.645 GW)

The CCGT is configured with a zero capacity. Hence, no electricity is served in the simulation (100% unserved energy) and the largest shortfall was 33,645 MW (33.6 GW). This figure corresponds to the peak demand in the simulated year.

Let's now do a run with two CCGTs (13.2 GW and 20 GW) such that almost of the demand is met except for a few hours of unserved energy:

In [3]:
c = nemo.Context()
c.generators[0].set_capacity(13.2)
nemo.run(c)
print c
Timesteps: 8760 h
Demand energy: 204.37 TWh
Unused surplus energy: 0 kWh
Unserved energy: 0.001%
Unserved total hours: 6
Number of unserved energy events: 2
Shortfalls (min, max): (88.36 MW, 445.14 MW)

If we print the unserved attribute in the context, we can see when the six hours of unserved energy occurred and how large the shortfalls were:

In [4]:
print c.unserved
Date_Time
2010-01-11 13:00:00     88.360
2010-01-11 14:00:00    245.200
2010-01-11 15:00:00    445.140
2010-01-11 16:00:00    113.530
2010-01-12 13:00:00    245.860
2010-01-12 14:00:00    178.365
dtype: float64

Plotting results

NEMO includes a utils.py module that includes a plot function to show the time sequential dispatch. The following example demonstrates its use:

In [5]:
%matplotlib inline
from nemo import utils
utils.plt.rcParams["figure.figsize"] = (12, 6)  # 12" x 6" figure
utils.plot(c)

The previous plot is rather bunched up. Instead, you can also pass a pair of dates to the plot() function to limit the range of dates shown. For example:

In [6]:
%matplotlib inline
from datetime import datetime
utils.plt.rcParams["figure.figsize"] = (12, 6)  # 12" x 6" figure
utils.plot(c, xlim=[datetime(2010, 1, 5), datetime(2010, 1, 12)])

Scripting simulations

Writing NEMO in Python allows the simulation framework to be easily scripted using Python language constructs, such as for loops. Using the previous example, the following small script demonstrates how simulation runs can be automated:

In [7]:
c = nemo.Context()
scenarios._one_ccgt(c)
for i in range(0, 40):
    c.generators[0].set_capacity(i)
    nemo.run(c)
    if c.unserved_energy() == 0:
        break
print c.generators
[CCGT (NSW1:31), 34 GW]

Once the generator capacity reaches 34 GW, there is no unserved energy.

Scenarios

NEMO contains two types of scenarios: supply-side and demand-side scenarios. The supply-side scenario modifies the list of generators. For example:

In [8]:
c = nemo.Context()
scenarios.ccgt(c)
print c.generators
[CCGT (NSW1:31), 0 kW, poly 17 pumped-hydro (QLD1:17), 500 MW, poly 36 pumped-hydro (NSW1:36), 1.74 GW, poly 24 hydro (NSW1:24), 42.5 MW, poly 31 hydro (NSW1:31), 43 MW, poly 35 hydro (NSW1:35), 71 MW, poly 36 hydro (NSW1:36), 2.5139 GW, poly 38 hydro (VIC1:38), 450 MW, poly 39 hydro (VIC1:39), 13.8 MW, poly 40 hydro (TAS1:40), 586.6 MW, poly 41 hydro (TAS1:41), 280 MW, poly 42 hydro (TAS1:42), 590.4 MW, poly 43 hydro (TAS1:43), 462.5 MW, OCGT (NSW1:31), 0 kW]

A list of the current supply-side scenarios (with descriptions) can be obtained by running python evolve.py --list-scenarios from the shell (without the leading !):

In [9]:
!evolve.py --list-scenarios
        __one_ccgt__ 	One CCGT only.
                ccgt 	All gas scenario.
            ccgt-ccs 	CCGT CCS scenario.
            coal-ccs 	Coal CCS scenario.
              re+ccs 	Mostly renewables with fossil and CCS augmentation.
           re+fossil 	Mostly renewables with some fossil augmentation.
               re100 	100% renewable electricity.
     re100+batteries 	Use lots of renewables plus battery storage.
           re100+dsp 	Mostly renewables with demand side participation.
           re100+egs 	100% renewables plus EGS geothermal.
     re100+egs-nocst 	100% renewables with EGS geothermal but no CST.
           re100+geo 	100% renewables plus both HSA and EGS geothermal.
     re100+geo-nocst 	100% renewables plus geothermal, but no CST.
      re100+geo-nopv 	100% renewables plus geothermal, but no CST.
     re100+geo-novre 	100% renewables plus geothermal, but no variable renewable energy (VRE).
    re100+geo-nowind 	100% renewables plus geothermal, but no CST.
           re100+hsa 	100% renewables plus HSA geothermal.
     re100+hsa-nocst 	100% renewables with HSA geothermal, but no CST.
         re100-nocst 	100% renewables, but no CST.
           re100-nsw 	100% renewables in New South Wales only.
           re100-qld 	100% renewables in Queensland only.
            re100-sa 	100% renewables in South Australia only.
         replacement 	The current NEM fleet, more or less.
            theworks 	All technologies.

Demand-side scenarios modify the electricity demand time series before the simulation runs. Demand-side scenarios behave like operators that can be combined in any combination to modify the demand as desired. These are:

  • roll:X rolls the load by x timesteps
  • scale:X scales the load by x percent
  • scaletwh:X scales the load to x TWh
  • shift:N:H1:H2 shifts n megawatts every day from hour h1 to hour h2
  • peaks:N:X adjust demand peaks over n megawatts by x percent
  • npeaks:N:X adjust top n demand peaks by x percent

For example, applying scale:-10 followed by shift:1000:16:12 will reduce the overall demand by 10% and then shift 1 MW of demand from 4pm to noon every day of the year.

Configuration file

NEMO uses a configuration file to give users control over where data such as demand time series are to be found. The location of the configuration file can be specified by setting the NEMORC environment variable. The configuration file format is similar to Windows INI files; it has sections (in brackets) and, within sections, key=value pairs.

The default configuration file is called nemo.cfg. The keys currently recognised are:

  • [costs]
    • co2-price-per-t
    • ccs-storage-costs-per-t
    • coal-price-per-gj
    • discount-rate -- as a fraction (eg 0.05)
    • gas-price-per-gj
    • technology-cost-class -- default cost class
  • [limits]
    • hydro-twh-per-yr
    • bioenergy-twh-per-yr
    • nonsync-penetration -- as a fraction (eg 0.75)
    • minimum-reserves-mw
  • [optimiser]
    • generations -- number of CMA-ES generations to run
    • sigma -- initial step-size
  • [generation]
    • cst-trace -- URL of CST generation traces
    • egs-geothermal-trace -- URL of EGS geothermal generation traces
    • hsa-geothermal-trace -- URL of HSA geothermal generation traces
    • wind-trace -- URL of wind generation traces
    • pv1axis-trace -- URL of 1-axis PV generation traces
    • rooftop-pv-trace -- URL of rooftop PV generation traces
  • [demand]
    • demand-trace -- URL of demand trace data

Running an optimisation

Instead of running a single simulation, it is more interesting to use evolve.py which drives an evolutionary algorithm to find the least cost portfolio that meets demand. There are many options which you can discover by running python evolve.py --help. Here is a simple example to find the least cost portfolio using the default "re100" scenario (100% renewables):

$ python evolve.py -s re100

It is possible to distribute the workload across multiple CPUs and multiple computers. See the SCOOP documentation for more details. To run the same evolution but using all of your locally available CPUs, you need to load the SCOOP module like so:

$ python -m scoop $(which evolve.py) -s re100

At the end of a run, details of the least cost system are printed on the console: the capacity of each generator, the energy supplied, CO2 emissions, costs, and the average cost of generation in dollars per MWh. If you want to see a plot of the system dispatch, you need to use the replay.py script described in the next section.

Many of the optimisation parameters can be controlled from the command line, requiring no changes to the source code. Typically, source code changes are only required to add new supply scenario functions or cost classes. The command line options for evolve.py are documented as follows:

Short option Long option Description Default
-h --help Show help and then exit
-c --carbon-price Carbon price in \$/tonne 25
-d --demand-modifier Demand modifier unchanged
-g --generations Number of generations to run 100
-o --output Filename of results output file (will overwrite) results.json
-r --discount-rate Discount rate 0.05
-s --supply-scenario Generation mix scenario re100
-t --transmission Include transmission costs False
-v --verbose Be verbose False
--bioenergy-limit Limit on annual energy from bioenergy in TWh/year 20
--ccs-storage-costs CCS storage costs in \$/tonne 27
--coal-price Coal price in \$/GJ 1.86
--costs Use different cost scenario AETA2013-in2030-mid
--emissions-limit Limit total emissions to N Mt/year $\infty$
--fossil-limit Limit share of energy from fossil sources 1.0
--gas-price Gas price in \$/GJ 11
--hydro-limit Limit on annual energy from hydro in TWh/year 12
--lambda CMA-ES lambda value None (autodetect)
--list-scenarios Print list of scenarios and exit
--min-regional-generation Minimum share of energy generated intra-region 0.0
--nsp-limit Non-synchronous penetration limit 0.75
--reliability-std Reliability standard (% unserved) 0.002
--seed Seed for random number generator None
--sigma CMA-ES sigma value 2.0
--trace-file Filename for evaluation trace None
--version Print version number and exit

Replaying a simulation

To avoid having to re-run a long optimisation just to examine the resulting system, it is possible to reproduce a single run using the results from an earlier optimisation. The evolve.py script writes an output file at the end of the run (default filename results.json). This file encodes all of the relevant information from the optimisation run so that the solution it found can be replayed easily and accurately by replay.py.

The input file for replay.py may consist of any number of scenarios and configurations to replay, one per line. Blank lines are ignored and comment lines (#) are shown for information. Each non-comment line must contain a JSON record from the results file that evolve.py writes. Typically the input file for replay.py will just be the output file from evolve.py unmodified. However, if you want multiple simulations to be replayed, this is easy to achieve by pasting multiple JSON strings into the file, one per line.

A run is replayed using replay.py like so:

$ replay.py -f results.json -x

The -f switch specifies the name of the input data file (the default is results.json) and the -x option enables a graphical plot of the system dispatch that you can navigate using zoom in, zoom out and pan controls. By including the --spills option, surplus energy in each hour will be plotted above the demand line in a lighter shade than the usual colour of the spilling generator. All command line options can be displayed using:

In [2]:
!replay.py --help
usage: replay.py [-h] -f FILE [--no-legend] [-t] [-v] [-x] [--spills]

Bug reports to: [email protected]

optional arguments:
  -h, --help          show this help message and exit
  -f FILE             filename of results file (default: results.json)
  --no-legend         hide legend
  -t, --transmission  show region exchanges [default: False]
  -v                  verbose mode
  -x                  producing a balancing plot
  --spills            plot surplus generation

Summarising the results

At the end of a simulation run, the results (including score, generator sizes, generated energy, CO2 emissions, etc.) can be fed through an AWK script called summary.awk. You will need a version of AWK installed. This is almost guaranteed to be installed on a Linux or Mac OS X system, but for Windows you can download a version from Sourceforge.

You can feed the output of evolve.py or replay.py into the summary.awk script. For example:

$ replay.py -f results.json -v | awk -f summary.awk

(Note: replace awk with gawk if you are using GNU AWK)

In the next example, the results are written to results.txt (using the Unix tee utility) in addition to being fed through the summary script:

$ evolve.py -s re100 | tee results.txt | awk -f summary.awk