#!/usr/bin/env python
# coding: utf-8

# # Import data using ArcGIS API for Python
# Completed sample for [ArcGIS DevLab](https://developers.arcgis.com/labs/data/import-data-using-python-api/)
# 
# ## Overview
# **You will learn:** how to import data into ArcGIS Online using ArcGIS API for Python.
# 
# With your [ArcGIS Developer Subscription](/pricing/), you can upload different file formats (CSV, XLS, GPX, GeoJSON, Shapefiles, etc) to ArcGIS Online and then use them in your apps. This process is called [publishing a hosted feature layer](https://doc.arcgis.com/en/arcgis-online/share-maps/publish-features.htm) and behind every feature layer is a [RESTful](https://resources.arcgis.com/en/help/arcgis-rest-api/#/Feature_Service/02r3000000z2000000/) service that supports spatial queries and editing operations.
# 
# With [ArcGIS API for Python](/python/guide/accessing-and-creating-your-content/), you can easily script and automate this data creation process. Once you upload a data item, you can publish that into a web layer. Initially your layers are only accessible to you, but they can be shared more widely using the [sharing permissions](https://doc.arcgis.com/en/arcgis-online/share-maps/share-items.htm) in ArcGIS for Developers and ArcGIS Online. This import process is a fast and easy way to turn static data into live services and build apps to display, filter and edit.
# 
# In this lab you will download and import datasets that were gathered from the [Los Angeles GeoHub](http://geohub.lacity.org/). They include Trailheads (CSV), Trails (GeoJSON), and a Parks and Open Spaces (Shapefile). The data will be stored in your own account in [ArcGIS Online](https://doc.arcgis.com/en/arcgis-online/share-maps/hosted-web-layers.htm) and can be used in other labs.
# 
# You will run this lab in a [Jupyter notebook](/python/guide/using-the-jupyter-notebook-environment/) IDE that is hosted by Esri. Notebooks created here are temporary, hence you must download them if you want to preserve them for later. However, you can [install the ArcGIS API for Python](/python/guide/install-and-set-up/) locally on your computer and run this lab from it, in which case, the notebooks are stored locally on your machine.
# 
# ## Steps
# ### Download data from ArcGIS Online
# You will use Python to download data from ArcGIS, extract it and upload it to your account.

# In[13]:


from arcgis.gis import *
import os
from zipfile import ZipFile


# In[9]:


public_data_item_id = 'a04933c045714492bda6886f355416f2'


# Create an anonymous connection to ArcGIS Online to download this public data

# In[10]:


anon_gis = GIS()


# Access this dataset as an ArcGIS Python API `Item` object

# In[11]:


data_item = anon_gis.content.get(public_data_item_id)
data_item


# Download it locally to your computer

# In[22]:


data_item.download(save_path = r'./') #download to the current directory


# ### Extract the zip file
# Use the `zipfile` library to extract the contents of the downloaded dataset.

# In[23]:


zf = ZipFile(r'./LA_Hub_Datasets.zip')


# Specify a path to extract the contents

# In[24]:


zf.extractall(path=r'./LA_Hub_datasets')


# Use Python `os` module to list the contents of the directory

# In[25]:


file_list = os.listdir(r'./LA_Hub_datasets/')
file_list


# ## Log into ArcGIS Online account
# Make a GIS connection to ArcGIS Online using your developer account

# In[26]:


# replace the username and password with your credentials
dev_gis = GIS("https://www.arcgis.com","username","password")


# ### Import datasets to ArcGIS Online
# The process of importing these datasets consists of two steps - adding the files as a file item and second - publishing them to create feature layers.
# 
# Now add the `Parks and Open Space.zip` `shapefile` as an item and publish it. During this process, we specify a title, a few tags and item type.

# In[30]:


parks_properties = {'title':'Parks and Open Space',
                   'tags':'parks, open data, devlabs',
                   'type':'Shapefile'}
parks_shp = dev_gis.content.add(parks_properties, 
                                data='./LA_Hub_datasets/Parks and Open Space.zip')


# Now query the variable `parks_shp` to visualize it in rich HTML notation in the notebook

# In[31]:


parks_shp


# Next, publish the shape file into a feature layer

# In[32]:


parks_feature_layer = parks_shp.publish()


# Similarly, query the `parks_feature_layer` variable to visualize the feature layer item you just created.

# In[33]:


parks_feature_layer


# ### Identify the URL
# Use the `url` property of the `Item` object to get the **Service URL**.

# In[34]:


parks_feature_layer.url


# ## Congratulations, you are done!
# 
# ### Challenge
# Now publish the `Trailheads.csv` similarly. You will add the csv file as an item then call the `publish()` method on the `Item` object to publish it. You can refer to [help on publishing csv files](https://developers.arcgis.com/python/sample-notebooks/publishing-SDs-shapefiles-and-CSVs/#Publish-a-CSV-file-and-move-it-into-a-folder) if necessary.