It is recommended to have a look at the 0_basic_functionalities and 1_Observation_Agents notebooks before getting into this one.
Objectives
This notebook will cover the basics of how to best use the Action class to modify the powergrid efficiently. Indeed, there are multiple concepts behind this class that may not be very clear at first glance.
This notebook will be focused on the manipulation of Actions from an expert system point of view in order to demonstrate how a desired action is fundamentally taken in the Grid2Op environment. We will give a more detailed example later of a more automatic way to handle actions (for example using machine learning, in the notebook 3_TrainingAnAgent).
import os
import sys
import grid2op
res = None
try:
from jyquickhelper import add_notebook_menu
res = add_notebook_menu()
except ModuleNotFoundError:
print("Impossible to automatically add a menu / table of content to this notebook.\nYou can download \"jyquickhelper\" package with: \n\"pip install jyquickhelper\"")
res
Impossible to automatically add a menu / table of content to this notebook. You can download "jyquickhelper" package with: "pip install jyquickhelper"
To manipulate a powergrid, we decided to introduce two distinct (yet close) concepts that will affect the objects differently:
This is another change compared to the previous pypownet implementation, where only the change
concept was implemented. Having these two things helps understand what is really going on in the powergrid and allows to represent better the intention of the Agent, especially in the debugging phase.
Of course, it is perfectly possible to use only the change
capability and thus being closer to the original implementation.
Let's give a "concrete" example to highlight the differences between these two methods. Suppose we have a substation with 5 elements:
Let's also assume the original configuration (before the action is applied, ie the configuration of the observation at time t) is:
Object Name | Original Bus | Original Status |
---|---|---|
$l_1$ (origin) | 1 | connected |
$l_2$ (extremity) | 2 | connected |
$l_3$ (extremity) | NA | disconnected |
$c_1$ | 1 | NA |
$g_1 $ | 2 | NA |
Let's say:
* NB Another breaking change compared to the pypownet implementation is the introduction of "ambiguous" actions. When a powerline is disconneted, it is not connected to any bus (by definition). So if you reconnect it without specifying on which "bus" it's "ambiguous". This action will not be implemented and the episode will terminate. Here, we try to reconnect the powerline $l_3$ without specifying on which bus we want to reconnect it, which will lead to an "ambiguous" action. More details about this will be given later in this notebook.
In this situation, the previous actions are equivalent to:
When some actions are "ambiguous" it means that they cannot be properly and / or univocally interpreted. These actions will be ignored if attempted to be used on the powergrid. This will be equivalent to doing nothing.
For a detailed list of ambiguous actions, the documentation is the only official source. Only some examples are presented here. The documentation is available at _check_for_ambiguity.
An action can be ambiguous in the following cases:
It affects the "injections" in an incorrect way:
It affects the "powerlines" in an incorrect manner:
It has an ambiguous behaviour concerning the topology of some substations
IMPORTANT NOTICE Each Agent has its own action helper
attribute that can be called with self.action_space
. This is the only recommended way to create a valid Action. Using its constructor is strongly NOT recommended, as it requires a deep knowledge of all the elements in the powergrid, as well as their names, their type, the order in which they are used in the backend, etc. For performance reasons, no sanity check are performed to make sure the action that would be created this way is compatible with the backend.
In the next cell, we retrieve the action space used by the Agent.
Two main classes are useful when dealing with Actions in Grid2Op. The Action class is the most basic one. The ActionHelper is a tool that helps create and manipulate some actions.
As in most of our notebooks, we start by creating an Environment. We will use the case14_fromfile
provided as an example. We will then extract the complete action space (action_space
, that is, the actions that can be performed on the power grid) as a dictionary. When a specific action such as change or set is needed to be performed then we can apply this change to the action_space dictionary by accessing it with the relevant key as discussed below.
# import the usefull class
env = grid2op.make(test=True)
action_space = env.action_space
/home/benjamin/Documents/grid2op_dev/getting_started/grid2op/MakeEnv/Make.py:240: UserWarning: You are using a development environment. This environment is not intended for training agents. warnings.warn(_MAKE_DEV_ENV_WARN)
As opposed to the previous plateform, pypownet, there are no restrictions on actions in Grid2Op. Generally speaking, an Action can modify production, loads, topology, etc. By default though, an Action that an Agent can perform is a TopologyAction, which is a specific type of action. A TopologyAction can :
We will focus on this class in this notebook.
Then best way to get an action is to give a dictionnary
to the "action space" of the player. For example, to get the "do nothing" action, you can just pass the empty dictionnary.
do_nothing = action_space({})
If you want to change (or set) the status of most of the powerlines, you can create a vector having the same size as the number of powerlines in the grid, and pass it to the dictionnary with the relevant keys ("set_line_status", "change_line_status") and the proper values (a vector of booleans to set line status, or a vector of integers to change line status). An example is given below. Note that this example only modifies the status of a few powerlines, but this way of defining actions is more adapted when you need to modify the status of many powerlines.
The following code will:
Each powerline / load / generator has an ID and a name.
Keep in mind that the IDs of the powerlines are 0, 1, 2, ..., env.n_line
- 1 (where env.n_line
is the number of lines in the environment). The same goes for the loads and generators.
On the other hand, the names are more human-friendly identifiers for the different objects in the grid.
Therefore, for any vector
containing information about the powerlines, the variable relative to the powerline of id i
can be accessed or modified with vector[i]
since it is the i+1
th powerline in the grid.
This is how we will proceed in the next cell. However, the IDs are easy to use but less meaningful for a human (since it is the names of the powerlines that we see) and sometimes, when inspecting an observation for example, we may want to look at a specific powerline by specifying its name. This will be covered later in the notebook 7_PlottingCapabilities.
change_status = action_space.get_change_line_status_vect()
change_status[0] = True
change_status[1] = True
change_status[2] = True
set_status = action_space.get_set_line_status_vect()
set_status[3] = 1
set_status[4] = 1
set_status[5] = -1
set_status[6] = -1
this_first_act = action_space({"set_line_status": set_status, "change_line_status": change_status})
NB even if it can handle different types, for performance reasons it's better to follow the type of data mentionned below : The dictionnary values should be:
True
means "change"False
means "don't change"0
means "do nothing"1
means "connect it"-1
means "disconnect it"For convenience, an Action object can be inspected easily by using the print
method. It will summarize on which object it has an impact:
print(this_first_act)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - force reconnection of 2 powerlines ([3 4]) - force disconnection of 2 powerlines ([5 6]) - switch status of 3 powerlines ([0 1 2]) - NOT switch anything in the topology - NOT force any particular bus configuration
this_first_act.is_ambiguous()
(False, None)
NB This action is ambiguous so it cannot be implemented on the powergrid. Indeed, powerlines 3 and 4 are reconnected, but we don't specify on which bus! Implementing this action on a grid will be equivalent to doing nothing.
It's not always convenient to manipulate all the status of all the powerlines, or change it. For mor convenience, it's possible to modify only a few of them. The syntax is the following.
the_same_act = action_space({"set_line_status": [(3,1), (4,1), (5,-1), (6,-1)],
"change_line_status": [0,1,2]
})
print(the_same_act)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - force reconnection of 2 powerlines ([3 4]) - force disconnection of 2 powerlines ([5 6]) - switch status of 3 powerlines ([0 1 2]) - NOT switch anything in the topology - NOT force any particular bus configuration
We can check that the two actions are indeed equal:
the_same_act == this_first_act
True
One of the interesting aspects of Grid2Op is to be able to modify the topology of the powergrid. In other words it allows to reconfigure the way the objects (generators, loads, end of powerlines) are interconnected at their substations.
Comparable to the status change, topological change can be interpreted in two disctinct manners, as described above. Topologycal changes include some of the most interesting interactions with the environment.
In this section we study how to modify the topology of the powergrid.
The underlying way to represent the topology is through a integer vector, having the same dimension as the number of objects of the grid. For each object in the grid, this vector tells on which bus it's connected. Manipulating this vector can be done, but is absolutely not handy. We present here the way to change the topology through the helped, which can be done more easily.
To set the bus to which a load is connected, it is recommended to do:
set_bus_load_0 = action_space({"set_bus": {"loads_id": [(0,2)]}})
print(set_bus_load_0)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 2 to load 0 [on substation 1]
To change the bus, a similar interface can be used:
change_bus_load_0 = action_space({"change_bus": {"loads_id": [0]}})
print(change_bus_load_0)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - Change the bus of the following element: - switch bus of load 0 [on substation 1] - NOT force any particular bus configuration
The API is really similar for generator:
change_bus_gen_0_and_1 = action_space({"change_bus": {"generators_id": [0,1]}})
set_bus_gen_3_and_4 = action_space({"set_bus": {"generators_id": [(3,2), (4,2)]}})
print(set_bus_gen_3_and_4)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 2 to generator 4 [on substation 0] - assign bus 2 to generator 3 [on substation 7]
The same goes for each ends of the powerlines:
change_bus_lines_or_0 = action_space({"change_bus": {"lines_or_id": [0]}})
set_bus_lines_or_4 = action_space({"set_bus": {"lines_or_id": [(3,2)]}})
change_bus_lines_ex_15 = action_space({"change_bus": {"lines_ex_id": [15]}})
set_bus_lines_ex_18 = action_space({"set_bus": {"lines_ex_id": [(18,2)]}})
print(set_bus_lines_ex_18)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 2 to line (extremity) 18 [on substation 7]
When reconnecting a powerline, if the bus to which this powerline is reconnected is not specified, the action is ambiguous and thus will not be implemented. It is, in that case, recommended to use the reconnect_powerline
method as followed:
reconnecting_line_1 = action_space.reconnect_powerline(line_id=1,bus_or=1,bus_ex=1)
print(reconnecting_line_1)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - force reconnection of 1 powerlines ([1]) - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 1 to line (origin) 1 [on substation 0] - assign bus 1 to line (extremity) 1 [on substation 4]
For convenience, it might be better sometimes to change the bus of an object from its name instead of its ID in case the ID is not known. Grid2Op allows to do that, but only for changing or setting a bus. These methods take longer than the methods shown above. If they are used at all, it's recommended NOT to use them for training an Agent. Their main goal aims at debugging and / or understanding the behaviour of an Agent.
These methods are:
action_space.change_bus
($\leftarrow$ this is a link)action_space.set_bus
($\leftarrow$ this is a link)Please refer to the official documentation for a complete description of their behaviour. To sum up, we can use them this way:
my_act = action_space.set_bus("gen_1_0", # mandatory name of the element
new_bus=2, # mandatory the new bus to connect it too
type_element="gen", # optional the type of the element, one of "line", "gen" or "load"
previous_action=None # optional: if you want to combine multiple action, you can do it with this
)
print(my_act)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 2 to generator 0 [on substation 1]
action_space.set_bus("1_3_3", # mandatory name of the element
extremity="or", # mandatory, which extrmity to change
new_bus=2, # mandatory the new bus to connect it too
type_element="line", # optional the type of the element, one of "line", "gen" or "load"
previous_action=my_act # optional: if you want to combine multiple action, you can do it with this
)
print(my_act)
This action will: - NOT change anything to the injections - NOT perform any redispatching action - NOT force any line status - NOT switch any line status - NOT switch anything in the topology - Set the bus of the following element: - assign bus 2 to line (origin) 3 [on substation 1] - assign bus 2 to generator 0 [on substation 1]
It is absolutely NOT recommended to use Actions outside of the action space.