In this notebook you will learn about the storage units in grid2op

Try me out interactively with: Binder

Objectives

Another kind of continuous actions that are modeled in grid2op are the action on storage units. The aim of this notebook is to describe this kind of action, how it works, what it does, and how you can apply them.

Execute the cell below by removing the # character if you use google colab !

Cell will look like:

!pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)

In [ ]:
# !pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)

I) Compatible environments

First, in order to do action on storage units, storage units must be present on the grid. This is not the case for most grid2op environment. So you might want to check if there are storage units, as shown bellow:

In [ ]:
import os
import sys
import grid2op
from tqdm.notebook import tqdm  # for easy progress bar
display_tqdm = False  # this is set to False for ease with the unitt test, feel free to set it to True
import numpy as np
import matplotlib.pyplot as plt

env_name1 = "l2rpn_case14_sandbox"
env_nok = grid2op.make(env_name1, test=True)
print(f"Can I use action on storage units in environment \"{env_name1}\": {env_nok.n_storage > 0}")

env_name2 = "educ_case14_storage"
env = grid2op.make(env_name2, test=True)
print(f"Can I use action on storage units in environment \"{env_name2}\": {env.n_storage > 0}")

II) What are storage units ?

A) Description

Storage units are "elements" of a grid that can behave sometimes as generators, sometimes as load (depending on what they are told to do). They can basically store a certain quantity of energy and release it when aked.

The two main types of storage units we can think of are:

  • "pumped storage": they store electric power by pumping it in an upward reservoir and can produce it again by letting the water through a turbine when going downhill (see this wikipedia article for more information)
  • "batteries": they store energy in a chemical form and can be charge / discharge, quit like the battery of a cellphone, but in (way, way) bigger.

In grid2op a storage unit is defined by different parameters. The main ones are:

  • storage_Emax: the maximum energy (expressed in MWh) the storage unit can contain.
  • storage_Emin: the minimum energy (in MWh) allowed in the unit (for example some batteries should not be "emptied" entirely)
  • storage_loss: the loss (in MW) in the storage unit. This corresponds to the loss of energy that happens continuously. In reality, this can model the "self discharge" of a battery or the evaporation of the upper lake in a pumped storage for example. It should not be mixed with the following two attributes.
  • storage_charging_efficiency: this is the efficiency when the storage units is charged. This efficiency corresponds to the amount of energy that will be stored in the unit if a power of 1MW is taken from the grid to charge it during 1 hour. It has no unit and should be between 0. and 1.
  • storage_discharging_efficiency: this is the efficiency when the storage units is discharged. This efficiency corresponds to the amount of energy that will be substracted from the unit if a power of 1MW is injected in the powergrid during 1 hour. It has no unit and should be between 0. and 1.
  • storage_max_p_prod: the maximum value (still seen from the grid) that a storage unit can inject on the grid. It is expressed in MW.
  • storage_max_p_absorb: the maximum value (still from the grid point of view) that a storage unit can absorb from the grid, expressed also in MW.

The official documentation gives more details about all these attributes, and some others too. In the cell below we give some example on how to access them:

In [ ]:
env.storage_Emax
env.storage_Emin
env.storage_loss
env.storage_charging_efficiency
env.storage_discharging_efficiency

The main usage for storage units in grid2op is to give them setpoint of power they will absorb (or produce) during a time step.

Grid2op handles the conversion of this power (seen from the grid) into the energy stored in the unittaking into account the losses and inefficiencies.

B) Convention

There are different conventions to model power grid elements.

For the storage unit, we adopted the "load convention". In short this means that:

  • if a positive power setpoint is given, then the storage will behave like a load. It will absorb power from the grid. It will recharge.
  • if this same power is negative, then the storage will behave like a generator. It will inject power to the grid. It will discharge.

III) Actions on storage units

Like every grid2op objects, storage units are manipulated through actions. The only thing you can do with storage units is to give a setpoint of how much power you want it to absorb / produce. This is done with the "storage_p" keys in the action.

In the next cell we will ask the storage unit 0 to inject 2.7MW into the grid and the storage unit 1 to absorb 3.14 MW from the grid.

In [ ]:
# create the action described above
# method 1, with the "dictionnary" action comprehension
storage_act1 = env.action_space({"set_storage": [(0, -2.7), (1, 3.14)]})

# method 2, with the "property"
storage_act2 = env.action_space()
storage_act2.storage_p = [(0, -2.7), (1, 3.14)]

# or alternatively, you can pass it a full vector:
storage_setpoint = np.zeros(env.n_storage, dtype=float)
storage_setpoint[0] = -2.7
storage_setpoint[1] = 3.14
storage_act3 = env.action_space({"set_storage": storage_setpoint})

# the same things with the property:
storage_act4 = env.action_space()
storage_act4.storage_p = storage_setpoint

# all the above actions are equivalent. And you can print them:
print(storage_act4)

IV) Storage units in the observation

There are a lot of information given in the observation concerning the storage units, the complete list of attributes you can retrieve is explained in the official documentation.

The most important are:

  • storage_charge: the current "charge" of each storage units (given in MWh)
  • storage_power_target: the setpoint given by the observation to the storage units (in MW)
  • storage_power: the actual power produced / absorbed (still seen from the grid) for every storage units

The following "properties" are met:

  • storage_charge is decreasing (due to storage_loss) if the storage units are not charged
  • storage_power_target corresponds to the storage action given in the previous step by the agent
  • storage_power may be different than the target, for example if the storage units is totally discharged, and you ask it to continue producing power
  • storage_power and storage_power_target are both vectors containing only 0. if no actions are performed on the storage units.

A simple example is given below:

In [ ]:
# I do not do any action, storage power, and storage_power_target are all 0
obs_init = env.reset()
obs1, reward1, done1, info1 = env.step(env.action_space())
print(f"The `storage_power` when no actions on storage units is performed is {obs1.storage_power}")

# I perform the action described above
obs2, reward2, done2, info2 = env.step(storage_act1)
print(f"The `storage_power` after the action described above is {obs2.storage_power}")

# Computing the amount of energy stored in the unit is not trivial. Indeed, each step is (for this environment)
# the equivalent of 5 mins. And if you ask 3.14 MW for 5mins, the charge will not
# be reduced by 3.14 MWh but rather by 3.14 / 60 * 5 MWh as can be seen here:
print(f"The initial charge in the storage unit 1 was: {obs_init.storage_charge[1]:.3f}MWh")
print(f"And after the action on this storage, it is: {obs2.storage_charge[1]:.3f}")
print(f"And we have: 3.14/60*5={3.14/60*5:.3f}")

Oh what is happening here ? We should have a charge of 3.50MWh + 0.26 MWh = 7.76 MWh. Why do we get only 3.74 MWh ?

This is because the storage have some losses: even if you do nothing on it, it will dissipate 0.1MW all the time. See section II) What are storage units ?-What-are-storage-units-?) for more details.

This means that, every 5 minutes, the storage unit will dissipate 0.1 / 60 * 5 = 0.00833333... MWh of energy.

In [ ]:
print(f"The initial energy storage in the unit 0 was: {obs_init.storage_charge[1]:.6f}MWh")
print(f"After doing nothing on this unit, it is: {obs1.storage_charge[1]:.6f}MWh")
print(f"As you see, the energy stored decrease of 0.1 / 60 * 5 = {0.1 / 60 * 5:.6f}MWh / step")
print(f"This explains that after doing nothing, then absorbing 3.14MW, the charge of the storage unit 1 is:\n "
      f"\t\t 3.50 - (0.1/60*5) + ((3.14/60*5) -(0.1/60*5)) = {obs2.storage_charge[1]:.3f} MWh "
     )

NB The formula above is true because, for the storage 1, the charging efficiency is 1.0. This would have been slightly modified (and more complicated) with a charging efficiency different from 1.0 More details about this more complex case are given in the documentation especially the sub section Satisfied equations (of the description of the storage units).

NB As opposed to curtailment or redispatching, storage units action do not "cumulate". An action that you do at a given step will affect only the next step. Battery actions do not last in time. An example is given in the cell below:

In [ ]:
print("I do a storage action")
obs3, reward3, done3, info3 = env.step(storage_act1)
print(f"The setpoint for storage unit 1 is indeed: {obs3.storage_power_target[1]:.2f}MW")
print(f"And the charge of this unit is {obs3.storage_charge[1]:.2f}MWh")
print("Then i do nothing")
obs4, reward4, done4, info4 = env.step(env.action_space())
print(f"The setpoint for storage unit 1 is indeed: {obs4.storage_power_target[1]:.2f}MW")
print(f"And the charge of this unit is {obs4.storage_charge[1]:.2f}MWh")
print("(the difference in the charge is due to the losses in the storage units)")

V) Side effects of using storage units

As always with grid2op, we suppose that the market (or a central authority) already adjusted the production to the consumption: at each steps, if nothing is done, the total demand can be power by the total production, without the intervention of any "agent" in grid2op.

This has some counterparts. For example, if you decide to act on the storage unit, then you will either increase the load (if you charge the units) or the generation (if you discharge them). Basically, you will affect the above equilibrium.

To restore the balance between total production, and total demand, as it was the case for the curtailment, the dispatchable generators are used. In fact, if you ask for a storage action that do not sum to 0, then automatically, the environment will perform some dispatch on the generators.

This behaviour is explained below:

In [ ]:
print(f"I do a storage action that sum in total to {storage_act1.storage_p.sum():.2f} MW")
obs5, reward5, done5, info5 = env.step(storage_act1)
print(f"And the sum of redispatching at this step is {obs5.actual_dispatch.sum():.2f} MW")

print("\nBut if i do nothing the next step, then we will have:")
obs6, reward6, done6, info6 = env.step(env.action_space())
print(f"the sum of redispatching at this step is {obs6.actual_dispatch.sum():.2f} MW")