#!/usr/bin/env python
# coding: utf-8

# <table style="width:100%">
#   <tr>
#     <td style="background-color:#EBF5FB; border: 1px solid #CFCFCF">
#       <b>National generation capacity: Main notebook</b>
#       <ul>
#           <li>Main notebook (this)</li>
#           <li><a href="processing.ipynb">Processing notebook</a></li>
#           <li><a href="tests.ipynb">Check notebook</a></li>
#       </ul>
#       <br>This Notebook is part of the <a href="http://data.open-power-system-data.org/national_generation_capacity">National Generation Capacity Datapackage</a> of <a href="http://open-power-system-data.org">Open Power System Data</a>.
#     </td>
#   </tr>
# </table>

# # 1. About Open Power System Data 
# This notebook is part of the project [Open Power System Data](http://open-power-system-data.org). Open Power System Data develops a platform for free and open data for electricity system modeling. We collect, check, process, document, and provide data that are publicly available but currently inconvenient to use. 
# More info on Open Power System Data:
# - [Information on the project on our website](http://open-power-system-data.org)
# - [Data and metadata on our data platform](http://data.open-power-system-data.org)
# - [Data processing scripts on our GitHub page](https://github.com/Open-Power-System-Data)
# 
# # 2. About Jupyter Notebooks and GitHub
# This file is a [Jupyter Notebook](http://jupyter.org/). A Jupyter Notebook is a file that combines executable programming code with visualizations and comments in markdown format, allowing for an intuitive documentation of the code. We use Jupyter Notebooks for combined coding and documentation. We use Python 3 as programming language. All Notebooks are stored on [GitHub](https://github.com/), a platform for software development, and are publicly available. More information on our IT-concept can be found [here](http://open-power-system-data.org/it). See also our [step-by-step manual](http://open-power-system-data.org/step-by-step) how to use the dataplatform.
# 
# # 3. About this Data Package
# We provide data in different chunks, or [datapackages](http://frictionlessdata.io/data-packages/). The one you are looking at is on [national generation capacity](http://data.open-power-system-data.org/national_generation_capacity/), 
# 
# This notebook processes data on aggregated generation capacity for European countries. The data includes generation capacities for nuclear, fossil and renewable fuels on an aggregated and, if possible, disaggregated technology levels. The majority of data reflects the years 2013 and 2014.
# The final output of the notebook is a list of national generation capacity differentiated by input fuels and technologies, as well as by different data sources
# 
# # 4. Data sources
# The raw data is based on an extensive research of available national and international statistics. Due to the diverse structure of data sources, a compiled and structured dataset is used as input for further processing. Generally, two cross-national data sources are considered, complemented by national data sources.
# 
# The cross-national data sources are:
# - [ENTSOE "Scenario Outlook & Adequacy Forecasts 2014-2030"](https://www.entsoe.eu/publications/system-development-reports/adequacy-forecasts/soaf-2014-2030/Pages/default.aspx)
# - [ENTSOE "Scenario Outlook & Adequacy Forecasts 2013-2030"](https://www.entsoe.eu/publications/system-development-reports/adequacy-forecasts/soaf-2013-2030/Pages/default.aspx)
# - EUROSTAT "Energy Statistics" compiled in [EU Commission's "Energy datasheets"](http://ec.europa.eu/energy/en/statistics/country)
# 
# Beside this, the data is complemented by national sources, which includes national statistical offices, ministries, regulatory authorities, transmission system operators, as well as other associations. All data sources are listed subsequently and in the `datapackage.json` including their link.
# 
# |	Country	|	Data type	|	Source		|	Type of Institution	|	Description	|	Type of document	|	Coverage	|
# | :--- | :--- | :---:|:---|:---|:---:|---:|
# |	Multiple	|	Aggregated capacities	|	[ENTSOE](https://www.entsoe.eu/publications/system-development-reports/adequacy-forecasts/Pages/default.aspx)		|	Association of European Transmission Network Operators	|	System Outlook and Adequacy Forecasts	|	Report	|	2013, 2014	|
# |	Multiple	|	Aggregated capacities	|	[EUROSTAT](http://ec.europa.eu/energy/en/statistics/country)		|	European Statistical Office	|	Country Datasheets 2016 by the European Commission	|	Statistics	|	1990-2014	|
# |	Multiple	|	Aggregated capacities	|	[UN Statistical Office](http://data.un.org/Data.aspx?d=EDATA&f=cmID%3AEC)		|	Statistical Office of the United Nations	|	Electricity, net installed capacity of electric power plants	|	Statistics	|	1990-2013	|
# |	AT	|	Aggregated capacities	|	[e-control](http://www.e-control.at/statistik/strom/bestandsstatistik)		|	Regulatory Authority	|	Engpassleistung nach Kraftwerkstypen zum 31. Dezember 2014	|	Statistics	|	2013, 2014	|
# |	BE	|	List of generation units	|	[ELIA](http://www.elia.be/en/grid-data/power-generation/generating-facilities)		|	Transmission System Operator	|	Overview of Generating Facilities	|	Statistics	|	actual	|
# |	BG	|	Aggregated capacities	|	[TSO Bulgaria ](http://www.tso.bg/uploads/file/Profile/en/ESO_Annual_Report_2012_en.pdf)		|	Transmission System Operator	|	Generation Capacities	|	Report	|	2012	|
# |	CH	|	Aggregated capacities	|	[BFE](http://www.bfe.admin.ch/themen/00526/00541/00542/00630/index.html?dossier_id=00765)		|	Ministry	|	Schweizerische Elektrizitätsstatistik (Electricity statistics)	|	Report	|	1998-2014	|
# |	CZ	|	Aggregated capacities	|	[ERU](http://www.eru.cz/en/elektrina/statistika-a-sledovani-kvality/rocni-zpravy-o-provozu)		|	Regulatory Authority	|	Yearly Report on Operations	|	Report	|	2010-2014	|
# |	DE	|	Aggregated capacities	|	[BMWi](http://www.bmwi.de/BMWi/Redaktion/Binaer/Energiedaten/energietraeger10-stromerzeugungskapazitaeten-bruttostromerzeugung,property=blob,bereich=bmwi2012,sprache=de,rwb=true.xls)		|	Ministry	|	Energiedaten (Energy Data)	|	Statistics	|	1990-2014	|
# |	DE	|	List of generation units	|	[BNetzA](http://www.bundesnetzagentur.de/EN/Areas/Energy/Companies/SecurityOfSupply/GeneratingCapacity/PowerPlantList/PubliPowerPlantList_node.html)		|	Regulatory Authority	|	List of Power Plants (>10 MW)	|	Statistics	|	actual	|
# |	DE	|	List of generation units	|	[UBA](https://www.umweltbundesamt.de/dokument/datenbank-kraftwerke-in-deutschland)		|	Environmental Agency	|	Kraftwerke in Deutschland (Power Plants in Germany, > 100 MW)	|	Statistics	|	actual	|
# |	DK	|	Aggregated capacities	|	[DEA](http://www.ens.dk/en/info/facts-figures/energy-statistics-indicators-energy-efficiency/annual-energy-statistics)		|	Energy Agency	|	Annual Energy Statistics	|	Statistics	|	2010-2013	|
# |	EE	|	Aggregated capacities	|	[Statistics Estonia](http://pub.stat.ee/px-web.2001/Dialog/varval.asp?ma=FE032&ti=CAPACITY+AND+PRODUCTION+OF+POWER+PLANTS&path=../I_Databas/Economy/07Energy/02Energy_consumption_and_production/01Annual_statistics/&lang=1)		|	National Statistical Office	|	Capacity and Production of Power Plants	|	Statistics	|	1999-2014	|
# |	ES	|	Aggregated capacities	|	[REE](http://www.ree.es/en/publications/statistical-data-of-spanish-electrical-system/national-indicators/)		|	Transmission System Operator	|	National Indicators	|	Statistics	|	1990-2016	|
# |	FI	|	Aggregated capacities	|	[Statistics Finland](http://pxnet2.stat.fi/PXWeb/pxweb/en/StatFin/StatFin__ene__ehk/240_ehk_tau_112_en.px/table/tableViewLayout1/?rxid=31077c25-37e4-480e-81e6-49a66cbe4dc2)		|	National Statistical Office	|	Capacity of Electricity Generation	|	Statistics	|	1970-2015	|
# |	FR	|	Aggregated capacities	|	[RTE](http://www.rte-france.com/en/documents)		|	Transmission System Operator	|	Overview of electrical energy in France	|	Report & Statistics	|	2014-2016	|
# |	FR	|	List of generation units	|	[RTE](http://clients.rte-france.com/lang/an/visiteurs/vie/prod/parc_reference.jsp)		|	Transmission System Operator	|	Installed Capacity of Units	|	Statistics	|	actual	|
# |	GR	|	Aggregated capacities	|	[REA](http://www.rae.gr/site/file/system/docs/ActionReports/national_2012)		|	Regulatory Authority	|	National Report 2012	|	Report	|	2012	|
# |	HR	|	Aggregated capacities	|	[HOPS](http://www.hops.hr/wps/portal/en/web/documents/publications)		|	Transmission System Operator	|	Basic Data	|	Report	|	2009-2013	|
# |	HU	|	List of generation units	|	[Mavir](http://www.mavir.hu/web/mavir-en/92)		|	Transmission System Operator	|	Planned Installed Generation Capacity	|	Report	|	2010-2014	|
# |	IE	|	List of generation units	|	[Eirgrid](http://www.eirgridgroup.com/library/index.xml)		|	Transmission System Operator	|	All-Island Generation Capacity Statement	|	Report	|	2012-2016	|
# |	IT	|	List of generation units	|	[Terna](https://www.terna.it/it-it/sistemaelettrico/transparencyreport/generation/installedgenerationcapacity.aspx)		|	Transmission System Operator	|	Installed Generation Capacity	|	Statistics	|	2010-2014	|
# |	LT	|	List of generation units	|	[Litgrid](http://www.litgrid.eu/index.php/power-system/power-system-information/generation-capacity/546)		|	Transmission System Operator	|	Generation Capacities	|	Statistics	|	actual	|
# |	LU	|	Aggregated capacities	|	[ILR](http://www.ilr.public.lu/electricite/statistiques/index.html)		|	Regulatory Authority	|	Centrales de productions au Luxembourg	|	Statistics	|	2001-2014	|
# |	LV	|	Aggregated capacities	|	[Central Statistical Bureau of Latvia](http://data.csb.gov.lv/pxweb/en/vide/vide__ikgad__energetika/EN0130.px/table/tableViewLayout1/?rxid=a79839fe-11ba-4ecd-8cc3-4035692c5fc8)		|	National Statistical Office	|	Electrical Capacity	|	Statistics	|	2007-2014	|
# |	NL	|	Aggregated capacities	|	[Tennet](http://energieinfo.tennet.org/dataexport/exporteerdatacountry.aspx)		|	Transmission System Operator	|	Installed Generation Capacity	|	Statistics	|	2011-2016	|
# |	NO	|	Aggregated capacities	|	[Ministry of Petroleum and Energy](https://www.regjeringen.no/globalassets/upload/oed/faktaheftet/facts_energy_water.pdf)		|	Ministry	|	Facts: Energy and Water Resources in Norway	|	Report	|	2013	|
# |	NO	|	Aggregated capacities	|	[Ministry of Petroleum and Energy](https://www.regjeringen.no/contentassets/fd89d9e2c39a4ac2b9c9a95bf156089a/facts_2015_energy_and_water_web.pdf)		|	Ministry	|	Facts: Energy and Water Resources in Norway	|	Report	|	2015	|
# |	PL	|	Aggregated capacities	|	[CIRE](http://www.rynek-energii-elektrycznej.cire.pl/st,33,207,tr,75,0,0,0,0,0,podstawowe-dane.html)		|	Energy Market Information Center	|	Moc zainstalowana (Installed Capacity)	|	Statistics	|	2014	|
# |	PT	|	Aggregated capacities	|	[REN](http://www.ren.pt/en-GB/media/publications/)		|	Transmission System Operator	|	Technical Data	|	Report	|	2013, 2014	|
# |	RO	|	Aggregated capacities	|	[Anre ](http://www.anre.ro/download.php?f=ga%2BCig%3D%3D&t=vdeyut7dlcecrLbbvbY%3D)		|	Regulatory Authority	|	Report	|	Report	|	2013, 2014	|
# |	SE	|	Aggregated capacities	|	[Svensk Energi](http://www.svenskenergi.se/Vi-erbjuder/Webbshop/Gratis-publikationer/Statistik/)		|	Energy Agency	|	Statistics Sweden	|	Report	|	2013	|
# |	SI	|	Aggregated capacities	|	[Agencija za energijo](http://www.agen-rs.si/web/en/publications/-/asset_publisher/TCdo6cWN9Shk/content/te-1?_101_INSTANCE_TCdo6cWN9Shk_redirect=%2Fweb%2Fen%2Fpublications)		|	Regulatory Authority	|	Report on the Energy Sector in Slovenia	|	Report	|	2011-2014	|
# |	SK	|	Aggregated capacities	|	[Statistical Office of Slovakia](https://slovak.statistics.sk/wps/portal/ext/themes/multi/energy/publications/!ut/p/b1/jZLBToQwEIafxSdgaIGyxwJLaRYrpbSuXAwHYzDurgfj81tY1BjiYG_NfF87_7RBHxyD_jx8jM_D-3g5D6_Tvk8eXd2kWRZyEKwLQSplrEs1tZJ54MEDXGttaudAOFKCpKEAZS1Al8y-ZvLqp5kjIJlqw4PWghRbfslmHzn-6ueCVxGrAdJaxCB5ZdudphQ4_V__8MfigN5ffvlIwMkvTUP4LhJ50d55v8tJauqEAISLjwBI_7ddtPgI8Hv-hzz35bgrYqdos49W_a8AzPfjwf1lfj_lmIEvGz87Z8DA2l8BSP69Ajz_BGy8_33Qzwj2g2YA-yJbj9yjKcp4A_ApVHU5PQVvJ_u9jjDKl4jffAIuhKL8/dl4/d5/L2dJQSEvUUt3QS80SmtFL1o2X1ZMUDhCQjFBMEc3VDEwSU5OU1VWOFEzVUM0/)		|	National Statistical Office	|	Energy Statistics	|	Report	|	2008-2014	|
# |	UK	|	Aggregated capacities	|	[Department of Energy & Climate Change UK](https://www.gov.uk/government/statistics/electricity-chapter-5-digest-of-united-kingdom-energy-statistics-dukes)		|	Ministry	|	Plant Capacity	|	Statistics	|	2010-2014	|

# # 5. Notation
# As we compile different source for national generation capacity, we define a consolidated classifiation scheme based on energy sources and technologies. The classification scheme is related to four different energy source levels:
# 
# - **Energy source level 0** - Total generation capacity
# - **Energy source level 1** - Generation capacity by energy type (fossil, nuclear, renewable, other)
# - **Energy source level 2** - Generation capacity by fuel or energy source (e.g. coal, lignite, hard coal, natural gas, wind)
# - **Energy source level 3** - Generation capacity by fuel or energy source and detailed fuel for bioenergy
# - **Technology level** - Generation capacity by fuel and technology (e.g. combined cycle natural gas, gas turbine, onshore wind, offshore wind)
# 
# The information on the technology levels is rearranged the following table. It displays the four general technology levels and the corresponding categories:

# In[1]:


import os, pandas as pd;
energylevels = pd.read_excel(os.path.join('input', 'National_Generation_Capacities.xlsx'),
                             sheetname='Summary',
                             header=0,
                             na_values=['-'],
                             parse_cols='A:F',
                             index_col=0,
                             skiprows=5)
energylevels = energylevels[pd.notnull(energylevels.index)]
energylevels = energylevels.replace({0: ' ', 1: True})
energylevels.index.name = 'Technology'
energylevels


# # 6. License
# This notebook as well as all other documents in this repository is published under the [MIT License](LICENSE.md).