Completed sample for ArcGIS DevLab
You will learn: how to import data into ArcGIS Online using ArcGIS API for Python.
With your ArcGIS Developer Subscription, you can upload different file formats (CSV, XLS, GPX, GeoJSON, Shapefiles, etc) to ArcGIS Online and then use them in your apps. This process is called publishing a hosted feature layer and behind every feature layer is a RESTful service that supports spatial queries and editing operations.
With ArcGIS API for Python, you can easily script and automate this data creation process. Once you upload a data item, you can publish that into a web layer. Initially your layers are only accessible to you, but they can be shared more widely using the sharing permissions in ArcGIS for Developers and ArcGIS Online. This import process is a fast and easy way to turn static data into live services and build apps to display, filter and edit.
In this lab you will download and import datasets that were gathered from the Los Angeles GeoHub. They include Trailheads (CSV), Trails (GeoJSON), and a Parks and Open Spaces (Shapefile). The data will be stored in your own account in ArcGIS Online and can be used in other labs.
You will run this lab in a Jupyter notebook IDE that is hosted by Esri. Notebooks created here are temporary, hence you must download them if you want to preserve them for later. However, you can install the ArcGIS API for Python locally on your computer and run this lab from it, in which case, the notebooks are stored locally on your machine.
You will use Python to download data from ArcGIS, extract it and upload it to your account.
from arcgis.gis import *
import os
from zipfile import ZipFile
public_data_item_id = 'a04933c045714492bda6886f355416f2'
Create an anonymous connection to ArcGIS Online to download this public data
anon_gis = GIS()
Access this dataset as an ArcGIS Python API Item
object
data_item = anon_gis.content.get(public_data_item_id)
data_item
Download it locally to your computer
data_item.download(save_path = r'./') #download to the current directory
'./LA_Hub_Datasets.zip'
Use the zipfile
library to extract the contents of the downloaded dataset.
zf = ZipFile(r'./LA_Hub_Datasets.zip')
Specify a path to extract the contents
zf.extractall(path=r'./LA_Hub_datasets')
Use Python os
module to list the contents of the directory
file_list = os.listdir(r'./LA_Hub_datasets/')
file_list
['Parks and Open Space.zip', 'Trailheads.csv', 'Trails.geojson', '__MACOSX']
Make a GIS connection to ArcGIS Online using your developer account
# replace the username and password with your credentials
dev_gis = GIS("https://www.arcgis.com","username","password")
The process of importing these datasets consists of two steps - adding the files as a file item and second - publishing them to create feature layers.
Now add the Parks and Open Space.zip
shapefile
as an item and publish it. During this process, we specify a title, a few tags and item type.
parks_properties = {'title':'Parks and Open Space',
'tags':'parks, open data, devlabs',
'type':'Shapefile'}
parks_shp = dev_gis.content.add(parks_properties,
data='./LA_Hub_datasets/Parks and Open Space.zip')
Now query the variable parks_shp
to visualize it in rich HTML notation in the notebook
parks_shp
Next, publish the shape file into a feature layer
parks_feature_layer = parks_shp.publish()
Similarly, query the parks_feature_layer
variable to visualize the feature layer item you just created.
parks_feature_layer
Use the url
property of the Item
object to get the Service URL.
parks_feature_layer.url
'https://services7.arcgis.com/JEwYeAy2cc8qOe3o/arcgis/rest/services/Parks_and_Open_Space/FeatureServer'
Now publish the Trailheads.csv
similarly. You will add the csv file as an item then call the publish()
method on the Item
object to publish it. You can refer to help on publishing csv files if necessary.