#!/usr/bin/env python
# coding: utf-8

# <img style="float: right;" src="./img/ODW_banner_devpost.jpg"></img>

# <hr>

# <a href="./ecmwf_web_mapping_service.ipynb"><< 04-ECMWF Copernicus Climate Change Service (C3S)</a>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space>&nbsp;<space> <a href="./index.ipynb">Index >></a>

# ## <center><h2>Additional Open Data Ressources</h2></center>

# - [Laminar Data Hub API](#laminar-aero)
# - [Reading Buses](#reading-buses)
# - [Natural Earth](#natural-earth) 
# - [European Union Open Data Portal](#eu-open-data) 
# - [European Data Portal](#eu-data-portal)
# - [European Environment Agency (EEA)](#eea)
# - [UK Office for National Statistics](#uk-statistics)
# - [FAOSTAT](#faostat)
# - [World Bank Open Data](#worldbank)
# - [DIVA-GIS](#diva-gis)

# <hr>

# ## <a id='laminar-aero'></a>Laminar Data Hub API

# - [1. Service Description](#service-description)
# - [2. Available datasets](#Available Datasets)
# - [3. How to access the service](#data-access)

# <hr>

# ### <a id='service-description'></a> 1. Service Description

# The Laminar Data Hub enables App Developers to build operational applications using developer-friendly APIs to access harmonised flight, weather, aeronautical, NOTAM and value-added data in a securely hosted cloud platform.  <br> Many of the APIs offer XML and GeoJSON response types, and all are accessed via a RESTful HTTP interface. 

# <b>User key:</b> 79b5a363698c6d5ab3e94c266555c8cb (NOTE: for #OpenDataHack weekend)

# ### <a id='datasets'></a> 2. Available Datasets

# #### Flight Data
# Live flight data is available from four different APIs, each offering a different view of the global commercial airspace.

# | Dataset | Description |
# |-----------------------|----------------------------------|
# |[Flights by Airline](https://developer.laminardata.aero/documentation/flightdata#/)| Retrieves a list of flight summaries for a given airline using an extended version of FIXM 3.0, including scheduled and airborne flights. Summaries include: <br> - Departure and arrival aerodromes <br> - Departure and arrival times (estimated and/or actual) <br> - Positional information (including heading and altitude) <br> - GUFI (Globally Unique Flight Identifier) <br> For a list of possible airlines refer to the Reference Data APIs [here](https://developer.laminardata.aero/documentation/referencedata). <br> The optional flight status parameter allows you to filter flights according to whether they are airborne, cancelled, completed, filed or scheduled.|
# |[Flights by GUFI](https://developer.laminardata.aero/documentation/flightdata#/)| Retrieves the most complete picture of a single flight, available in both GeoJSON and an extended version of FIXM 3.0. Completed flights will remain accessible in the API for 6 hours after the flight has landed. Note that the example GUFI below will not work; you will need a current one from one of the other API calls (e.g. Flights by Airline). Flights by GUFI includes: <br> - Departure and arrival aerodromes <br> - Departure and arrival times (estimated and/or actual) <br> - Positional information (including heading, altitude and speed). <br> To request GeoJSON use the HTTP header "Accept : application/json" using the "Accepts" dropdown or add the query string "format=json" to the request URL. <br> <b>Note:</b> that full representation of the data model (including units of measure) can be found in the schema repository on GitHub |
# |[Flights by Tile](https://developer.laminardata.aero/documentation/flightdata#/)|Retrieves a list of flight summaries for a given tile using an extended version of FIXM 3.0. For more information on tiling see below. The API only returns airborne flights. Summaries include: <br> - Departure and arrival aerodromes <br> - Departure and arrival times (estimated and/or actual) <br> - Positional information (including heading and altitude) <br> - GUFI (Globally Unique Flight Identifier) <br>  The Flights by Tile API has a GeoJSON response option available. To request GeoJSON use the HTTP header "Accept : application/json" using the "Accepts" dropdown or add the query string "format=json" to the request URL. <br> <b>Note:</b> that full representation of the data model (including units of measure) can be found in the schema repository on GitHub. |
# |[Flights by Aerodrom Pair](https://developer.laminardata.aero/documentation/flightdata#/)|Retrieves a list of flight summaries between a pair of aerodromes using an extended version of FIXM 3.0, including scheduled and airborne flights. Summaries include: <br> - Departure and arrival aerodromes, <br> - Departure and arrival times (estimated and/or actual), <br> - Positional information (including heading and altitude) <br> - GUFI (Globally Unique Flight Identifier). <br> The optional flight status parameter allows you to filter flights according to whether they are airborne, cancelled, completed, filed or scheduled. |

# #### Aeronautical Data
# All responses for Aeronautical Data use the [AIXM standard version 5.1](http://www.aixm.aero/).
# APIs providing access to Route Segments (Europe and USA coverage), Aerodromes (Global Coverage) Airspaces (Europe Coverage), Designated Points (USA and Europe Coverage) and Navaids (USA and Europe Coverage) are all available to develop against. Find more detail on the [Aeronautical Data Documentation](https://developer.laminardata.aero/documentation/aeronauticaldata). 

# #### Regulation Data
# Airspace regulations for the European region can be accessed by ICAO prefix or by a specific Regulation ID to retrieve the full Regulation detail. Check out the [Regulation Documentation](https://developer.laminardata.aero/documentation/regulationdata#/) for more information.

# #### Weather Data
# Laminar provides access to up to date aviation specific weather data via it’s METAR (Meteorological Aerodrome Report), TAF (Terminal Aerodrome Forecast) and SIGMET (Significant Meteorological Information) APIs. All responses for Weather Data use the [IWXXM standard version 1.1](http://www.wmo.int/pages/prog/www/WIS/wiswiki/tiki-index.php?page=AvXML-1.1-Release#Online_version).
# Full detail on the Weather APIs can be found on the [Weather Documentation](https://developer.laminardata.aero/documentation/weatherdata#/) pages. 

# #### NOTAM Data
# Laminar provides access to global coverage of NOTAM information via it’s NOTAM by FIR and NOTAM by ID APIs. All responses for NOTAM Data use the [AIXM 5.1 Digital NOTAM Event Specification](http://www.aixm.aero/).
# Full detail on the NOTAM APIs can be found on the [NOTAM Data documentation](https://developer.laminardata.aero/documentation/notamdata) pages.

# ### <a id='data-access'></a> 3. How to access the service?

# All APIs are accessed via a standard RESTful HTTPs interface, with a user key required to gain access to the data.  
# 
# The basic structure of API requests is as follows: https://api.laminardata.aero/{api_version}/some_dataset/{some_parameter}?user_key={access_key}
# 
# * The request structure for each individual API can be found on the [Laminar Hub Documentation](https://developer.laminardata.aero/) page for each respective API.
# 
# * Visit Laminar Data's [GitHub Repository](https://github.com/laminardata) for a complete list of schemas used to serve the XML data found on the Laminar Data APIs.
# 
# * Check out the [Getting Started](https://developer.laminardata.aero/documentation/gettingstarted) guide and [Data Structure overview](https://developer.laminardata.aero/documentation) for hints on how to get started using Laminar Data.

# <hr>

# ## <a id='reading-buses'></a>Reading Buses

# ### <a id='datasets'></a> Available Datasets

# #### Nimbus schedule adherence
# APIs giving: <br> - Bus stop list, <br>   - Live vehicle positions, <br> - live journey details, <br> - bus stop predictions, <br> - list of services, <br> - line patterns, <br> - timetabled journeys
# 
# <b>Access:</b> http://rtl2.ods-live.co.uk/cms (Key generation is on board the CMS interface)  
# <b>User:</b> opendata@reading-buses.co.uk  
# <b>Password:</b> Reading!!

# <b>NOTE:</b> Time frequency - as soon as we receive an update from a vehicle via SIRI or our on board unit, the new position will be reflected in the feed.        
# The service provides urls to each service, users make requests and regular intervals if necessary depending on feed. For example user, might make single request to get list of bus stops, or list of services ( these are things that don’t change often). Then user can make regular requests to get list of vehicles or live journeys. Mashing up these data sets and new datasets can then make compelling apps!
# The API isn’t rate limited at the moment, however we are putting in functionality over the next couple of months to limit services to x calls per user per minute
# 
# In order to access the API, users first register, then admin user approves the registration, then users can login, go to the API page and press “Generate API Key", they get to see their key and they can then use it to access data

# #### Mix Telematics  
# Allows graphic view of live vehicle location, historic trips etc plus export reporting of “events”, trips, mileage, fuel consumption and driver metrics.  
# 
# <b>Access:</b> https://uk.mixtelematics.com/#/login  
# <b>User:</b> jbickerton@reading-buses.co.uk  
# <b>Password:</b> opendata123
# 

# <b>NOTE:</b> Graphic displays are pretty, but reports are accessed under “Measure > Reports”, then select a category from the dropdown. “Events” are heavy braking, heavy acceleration, excess idling, as well as others custom-configured such as low coolant, over speeding in the depot etc. The reporting tool can show detail or will aggregate to eg monthly or totals as required, but will timeout if too much data is requested.

# #### Collision reports
# Export of insurance claims (fault and non-fault) showing crude locations, vehicle details and anonymised driver data. This report has been redacted to remove personal details and is now open data.
# 
# <b>Access:</b> <a href="./reading_buses_data/20170227_reading_claims_summary_report.xlsb"> Reading Claims Summary Report </a>
# 

# #### Vehicle roadcall export  
# 
# List of every vehicle road call, whenever we’ve attended a vehicle at the roadside.
# 
# <b>Access:</b> <a href="./reading_buses_data/201703_vehicle_defect_roadcall_log.xlsx"> 201703 Vehicle Defect Road Call Log</a>

# <hr>

# ## <a id='natural-earth'></a>Natural Earth

# Natural Earth is a public domain map dataset available at 1:10m, 1:50m, and 1:110 million scales. Featuring tightly integrated vector and raster data, with Natural Earth you can make a variety of visually pleasing, well-crafted maps with cartography or GIS software.  
# * http://www.naturalearthdata.com/downloads/  
# 

# <hr>

# ## <a id='eu-open-data'></a>European Union Open Data Portal

# The EU Open Data Portal is your single point of access to a growing range of data produced by the institutions and other bodies of the European Union. Data are free to use, reuse, link and redistribute for commercial or non-commercial purposes.  
# 
# * https://data.europa.eu/euodp/en/data

# <hr>

# ## <a id='eu-data-portal'></a>European Data Portal

# The European Data Portal harvests the metadata of Public Sector Information available on public data portals across European countries. Information regarding the provision of data and the benefits of re-using data is also included.  
# 
# * https://www.europeandataportal.eu/

# <hr>

# ## <a id='eea'></a>European Environment Agency (EEA)

# * http://www.eea.europa.eu/data-and-maps

# <hr>

# ## <a id='uk-statistics'></a> UK Office for National Statistics

# The UK's largest independent producer of official statistics and the recognised national statistical institute of the UK.
# 
# * https://www.ons.gov.uk/

# <hr>

# ## <a id='faostat'></a>Food and Agricultural Organisation (FAO) - FAOSTAT

# FAOSTAT provides free access to food and agriculture data for over 245 countries and territories and covers all FAO regional groupings 
# from 1961 to the most recent year available.
# 
# * http://www.fao.org/faostat/en/#home

# <hr>

# ## <a id='worldbank'></a>World Bank Open Data

# Free and open access to global development data - http://datacatalog.worldbank.org/.  
# * [Data Catalog API](https://datahelpdesk.worldbank.org/knowledgebase/articles/902049-data-catalog-api) 
# * [Climate Data API](https://datahelpdesk.worldbank.org/knowledgebase/articles/902061-climate-data-api)

# ### Data access from Climate Data API

# In[10]:


import requests
import fastkml.kml as kml
import json
import numpy as np

climateApi2 = requests.get("http://climatedataapi.worldbank.org/climateweb/rest/v1/country/kml/AUS")
# Put loaded json kml information to a Python KML object
k = kml.KML()
k.from_string(climateApi2.text)

# Write the KML object to disc
outputFile = file("test.kml","w")
outputFile.write(k.to_string())
outputFile.close()

# Function loadDataFromClimateApi that puts together the url needed for the Worldbank's Climate Data API. It returns a list of python dictionaries.
def loadDataFromClimateApi(url,level,statType,var,start,end,code):
    climateApi = requests.get(url+level+"/"+statType+"/"+var+"/"+start+"/"+end+"/"+code)
    data = json.loads(climateApi.text)
    return data

# Load monthly mean precipitation time series for Australia between 1980 and 1999 --> test is a list of python dictionaries
test = loadDataFromClimateApi(url="http://climatedataapi.worldbank.org/climateweb/rest/v1/",
                             level="country",
                             statType="mavg",
                             var="pr",
                             start="1980",
                             end="1999",
                             code="AUS")

# Data from a list of python dictionaries are retrieved by iterating through the list and then calling the required dictionary entry
testList = []
for item in test:
    testList.append(item[u'monthVals'])

# Convert the list into a numpy array
test2 = np.matrix(testList)
print (test2)
print(test2[:,1])


# <hr>

# ## <a id='diva-gis'></a>DIVA-GIS

# DIVA-GIS provides free spatial data of the whole world.  
# 
# * http://www.diva-gis.org/Data

# <hr>

# &copy; 2017 ECMWF
# 
# This software is licensed under the terms of the Apache Licence Version 2.0 which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. In applying this licence, ECMWF does not waive the privileges and immunities granted to it by virtue of its status as an intergovernmental organisation nor
# does it submit to any jurisdiction.