Downloading SmugMug Captions with Python and Jupyter

Prerequistes

This notebook assumes you have set up your environment to use smugpyter.py. Refer to this notebook for details on how to do this.

Getting Ready to use the SmugMug API with Python and Jupyter

Why am I doing this?

My photo captions have evolved into a form of milliblogging. Milliposts (milliblog posts) are terse and tiny; many are single sentences or paragraphs. Taken one-at-a-time milliposts seldom impress but when gathered in hundreds or thousands accidental epics emerge. So, to prevent "epic loss" I want a simple way of downloading and archiving my captions off-line.

If you don't control it you cannot trust it!

When I started blogging I knew that you could not depend on blogging websites to archive and preserve your documents. We had already seen cases of websites mangling content, shutting down without warning, and even worse, censoring bloggers. It was a classic case of, “If you don't control it you cannot trust it." I resolved to maintain complete off-line version controlled copies of my blog posts.

Maintaining off-line copies was made easier by WordPress.com's excellent blog export utility. A simple button push downloads a large XML file that contains all your blog posts with embedded references to images and other inclusions. XML is not my preferred archive format. I am a huge fan of LaTeX and Markdown: two text formats that are directly supported in Jupyter Notebooks. I wrote a little system that parses the WordPress XML file and generates LaTeX and Markdown files. Yet, despite milliblogging long before blogging, I don't have a similar system for downloading and archiving Smugmug metadata. This Jupyter notebook addresses this omission and shows how you can use Python and the Smugmug API to extract gallery and image metadata and store it in version controlled local directories as CSV files.

smugpyter.py runs in the Python 3.6/Jupyter/Win64 environment.

A lot of the code in this notebook was derived from:

  1. https://github.com/marekrei/smuploader/blob/master/smuploader/smugmug.py

  2. https://github.com/kevinlester/smugmug_download

  3. https://github.com/AndrewsOR/MugMatch

The originals did not run in the Python 3.6/Jupyter/Win64 environment and lacked some of the facilities I wanted so I adjusted, tweaked and modified the scripts. The result is incompatible with the originals so I renamed the main class SmugPyter to avoid confusion. Finally, being new to Python and Jupyter, I used the 2to3 tool to help make the changes.

In [1]:
import os
import smugpyter

help(smugpyter)
Help on module smugpyter:

NAME
    smugpyter

DESCRIPTION
    # code from:
    # https://github.com/speedenator/smuploader/blob/master/bin/smregister
    # https://github.com/kevinlester/smugmug_download/blob/master/downloader.py
    # https://github.com/AndrewsOR/MugMatch/blob/master/mugMatch.py
    # modified for python 3.6/jupyter environment - modifications assisted by 2to3 tool

CLASSES
    builtins.object
        SmugPyter
    
    class SmugPyter(builtins.object)
     |  Methods defined here:
     |  
     |  __init__(self, verbose=False)
     |      Constructor. 
     |      Loads the config file and initialises the smugmug service
     |  
     |  case_mask_decode(self, smug_key, case_mask)
     |      Restore letter case to (smug_key).
     |  
     |  case_mask_encode(self, smug_key)
     |      Encode the case mask as an integer.
     |  
     |  create_album(self, album_name, password=None, folder_id=None, template_id=None)
     |      Create a new album.
     |  
     |  create_nice_name(self, name)
     |  
     |  download_image(self, image_info, image_path, retries=5)
     |      Download an image from a url.
     |  
     |  download_smugmug_mirror(self, func_album=None, func_folder=None)
     |      Walk SmugMug folders and albums and apply functions (func_album) and (func_folder).
     |      
     |          smugmug = SmugPyter()
     |          smugmug.download_smugmug_mirror(func_album=smugmug.write_album_manifest)
     |  
     |  get_access_token(self, verifier)
     |      Gets the access token from SmugMug.
     |  
     |  get_album_id(self, album_name)
     |      Get an album id.
     |  
     |  get_album_image_captions(self, album_images)
     |      Get a list of {ImageKey, Caption} dictionaries for (album_images).
     |  
     |  get_album_image_names(self, album_images)
     |      Get a list of {ImageKey, FileName} dictionaries for (album_images).
     |      
     |          smugmug = SmugPyter()
     |          album_images = smugmug.get_album_images('XghWcL')
     |          smugmug.get_album_image_names(album_images)
     |  
     |  get_album_image_real_dates(self, album_images)
     |      Get a list of {ImageKey, AlbumKey, RealDate, FileName} dictionaries 
     |      for (album_images). The performance of this function is mostly appalling
     |      as we must make a web request for every single image date. 
     |      
     |          smugmug = SmugPyter()
     |          album_images = smugmug.get_album_images('XghWcL')
     |          smugmug.get_album_image_real_dates(album_images)
     |  
     |  get_album_images(self, album_id, argument='images')
     |      Get a list of images in an album.
     |      
     |      The (argument) parameter selects various API options. For
     |      example to select all the geotagged images in an album do:
     |      
     |          smugmug = SmugPyter()
     |          smugmug.get_album_images('gLd4hT', "geomedia")
     |  
     |  get_album_info(self, album_id)
     |      Get info for an album.
     |  
     |  get_album_names(self)
     |      Return list of album names.
     |  
     |  get_albums(self)
     |      Get a list of all albums in the account.
     |  
     |  get_authorize_url(self)
     |      Returns the URL for OAuth authorisation.
     |  
     |  get_child_node_uri(self, node_id)
     |      Get child node uri.
     |  
     |  get_folder_id(self, folder_name)
     |      Get category id.
     |  
     |  get_folder_names(self)
     |      Return list of (top-level) folder names.
     |  
     |  get_folders(self)
     |      Get a list of folder names under the user.
     |      Currently supports only top-level folders.
     |  
     |  get_image_date(self, imagemeta_data)
     |      Get an image date.
     |      
     |      There are a number of SmugMug image dates to choose from. The dates
     |      obtained from the (images) or (geomedia) options refer to upload and 
     |      adjustment times. The original image date must be extracted from the image
     |      metadata EXIF/IPTC. Sadly, the metadata tag is nonstandard and there are a 
     |      number of choices. This function attempts to pick the best available 
     |      original image date.
     |  
     |  get_image_download_url(self, image_id)
     |      Get the link for dowloading an image.
     |  
     |  get_latitude_longitude_altitude(self, album_images)
     |      Get a list of {ImageKey, (Latitute,Longitude,Altitude)} dictionaries for (album_images).
     |  
     |  mirror_folders_offline(self, root_uri, root_dir, func_album=None, func_folder=None)
     |      Recursively walk online SmugMug folders and albums and apply
     |      functions (func_album) and (func_folder).
     |  
     |  request(self, method, url, params={}, headers={}, files={}, data=None, header_auth=False, retries=5, sleep=5)
     |      Performs requests, with multiple attempts if needed.
     |  
     |  request_once(self, method, url, params={}, headers={}, files={}, data=None, header_auth=False)
     |      Performs a single request.
     |  
     |  upload_image(self, image_data, image_name, image_type, album_id)
     |      Upload an image.
     |  
     |  write_album_manifest(self, album_id, name, path)
     |      Write TAB delimited file of SmugMug image metadata.
     |  
     |  write_album_real_dates(self, album_id, name, path)
     |      Write TAB delimited file of SmugMug image real dates.
     |      My image dates vary from fairly reliable EXIF dates
     |      to wild guesses about century old prints. SmugMug 
     |      requires a full date and it looks in a number
     |      of places for full dates, see: (get_image_date).
     |  
     |  ----------------------------------------------------------------------
     |  Static methods defined here:
     |  
     |  base36decode(number)
     |      Decode base 36 string and return integer.
     |  
     |  base36encode(number)
     |      Encode positive integer as a base 36 string.
     |  
     |  decode(obj, encoding='utf-8')
     |  
     |  extract_alphanum(in_string)
     |  
     |  load_image(image_path)
     |      Load the image data from a path.
     |  
     |  purify_smugmug_text(in_string)
     |      Convert Smugmug unicode strings to ascii equivalents making non-ascii
     |      characters visible as XML escapes. Also convert embedded control 
     |      character to blanks.
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  smugmug_access_token_uri = 'http://api.smugmug.com/services/oauth/1.0a...
     |  
     |  smugmug_api_base_url = 'https://api.smugmug.com/api/v2'
     |  
     |  smugmug_api_version = 'v2'
     |  
     |  smugmug_authorize_uri = 'http://api.smugmug.com/services/oauth/1.0a/au...
     |  
     |  smugmug_base_uri = 'http://api.smugmug.com'
     |  
     |  smugmug_config = r'C:\Users\john\.smugpyter.cfg'
     |  
     |  smugmug_request_token_uri = 'http://api.smugmug.com/services/oauth/1.0...
     |  
     |  smugmug_upload_uri = 'http://upload.smugmug.com/'

DATA
    ascii_letters = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
    digits = '0123456789'

FILE
    c:\users\john\anacondaprojects\testfolder\smugpyter.py


Create the SmugPyter configuration file.

The SmugPyter class constuctor reads a config file. If this file is missing you cannot create instances of the SmugMug class or connect to your SmugMug account.

In [2]:
# the SmugPyter class constuctor reads a config file in this location.
os.path.join(os.path.expanduser("~"), '.smugpyter.cfg')
Out[2]:
'C:\\Users\\john\\.smugpyter.cfg'

The following prompts for your SmugMug API keys. You can apply for SmugMug keys on your SmugMug account by browsing to the API KEYS section of your account settings.

In [ ]:
# code from https://github.com/speedenator/smuploader/blob/master/bin/smregister
# modified for python 3.6/jupyter environment - modifications assisted by 2to3 tool 

from rauth.service import OAuth1Service
import requests
import http.client
import httplib2
import hashlib
import urllib.request, urllib.parse, urllib.error
import time
import sys
import os
import json
import configparser
import re
import shutil

# depends on previously run cells 
#from smuploader import SmugMug

def write_config(configfile, params):
    config = configparser.ConfigParser()
    config.add_section('SMUGMUG')
    for key, value in params:
        config.set('SMUGMUG', key, value)
    with open(SmugMug.smugmug_config, 'w') as f:
        config.write(f)

if __name__ == '__main__':
    print("\n\n\n#######################################################")
    print("## Welcome! ")
    print("## We are going to go through some steps to set up this SmugMug photo manager and make it connect to the API.")
    print("## Step 0: What is your SmugMug username?")
    username = input("Username: ")
    print('## Step 1: Enter your local directory, e.g. c:/SmugMirror/')
    localdir = input("Directory: ")

    print("## Step 2: Go to https://api.smugmug.com/api/developer/apply and apply for an API key.")
    print("## This gives you unique identifiers for connecting to SmugMug.")
    print("## When done, you can find the API keys in your SmugMug profile.")
    print("## Account Settings -> Me -> API Keys")
    print(("## Enter them here and they will be saved to the config file (" + SmugMug.smugmug_config + ") for later use."))
    consumer_key = input("Key: ")
    consumer_secret = input("Secret: ")

    write_config(SmugMug.smugmug_config, [("username", username), ("consumer_key", consumer_key), 
                                          ("consumer_secret", consumer_secret), ("access_token", ''), 
                                          ("access_token_secret", '')])

    smugmug = SmugMug()
    authorize_url = smugmug.get_authorize_url()
    print(("## Step 2: Visit this address in your browser to authenticate your new keys for access your SmugMug account: \n## " + authorize_url))
    print("## After that, enter the 6-digit key that SmugMug provided")
    verifier = input("6-digit key: ")

    access_token, access_token_secret = smugmug.get_access_token(verifier)

    write_config(SmugMug.smugmug_config, [("username", username), ("consumer_key", consumer_key), 
                                          ("consumer_secret", consumer_secret), ("access_token", access_token),
                                          ("access_token_secret", access_token_secret)])

    print("## Great! All done!")

Try out the SmugPyter class with credentials saved in the previous cell.

In [3]:
smugmug = smugpyter.SmugPyter()
len(smugmug.get_album_names())
Out[3]:
64
In [4]:
help(smugmug)
Help on SmugPyter in module smugpyter object:

class SmugPyter(builtins.object)
 |  Methods defined here:
 |  
 |  __init__(self, verbose=False)
 |      Constructor. 
 |      Loads the config file and initialises the smugmug service
 |  
 |  case_mask_decode(self, smug_key, case_mask)
 |      Restore letter case to (smug_key).
 |  
 |  case_mask_encode(self, smug_key)
 |      Encode the case mask as an integer.
 |  
 |  create_album(self, album_name, password=None, folder_id=None, template_id=None)
 |      Create a new album.
 |  
 |  create_nice_name(self, name)
 |  
 |  download_image(self, image_info, image_path, retries=5)
 |      Download an image from a url.
 |  
 |  download_smugmug_mirror(self, func_album=None, func_folder=None)
 |      Walk SmugMug folders and albums and apply functions (func_album) and (func_folder).
 |      
 |          smugmug = SmugPyter()
 |          smugmug.download_smugmug_mirror(func_album=smugmug.write_album_manifest)
 |  
 |  get_access_token(self, verifier)
 |      Gets the access token from SmugMug.
 |  
 |  get_album_id(self, album_name)
 |      Get an album id.
 |  
 |  get_album_image_captions(self, album_images)
 |      Get a list of {ImageKey, Caption} dictionaries for (album_images).
 |  
 |  get_album_image_names(self, album_images)
 |      Get a list of {ImageKey, FileName} dictionaries for (album_images).
 |      
 |          smugmug = SmugPyter()
 |          album_images = smugmug.get_album_images('XghWcL')
 |          smugmug.get_album_image_names(album_images)
 |  
 |  get_album_image_real_dates(self, album_images)
 |      Get a list of {ImageKey, AlbumKey, RealDate, FileName} dictionaries 
 |      for (album_images). The performance of this function is mostly appalling
 |      as we must make a web request for every single image date. 
 |      
 |          smugmug = SmugPyter()
 |          album_images = smugmug.get_album_images('XghWcL')
 |          smugmug.get_album_image_real_dates(album_images)
 |  
 |  get_album_images(self, album_id, argument='images')
 |      Get a list of images in an album.
 |      
 |      The (argument) parameter selects various API options. For
 |      example to select all the geotagged images in an album do:
 |      
 |          smugmug = SmugPyter()
 |          smugmug.get_album_images('gLd4hT', "geomedia")
 |  
 |  get_album_info(self, album_id)
 |      Get info for an album.
 |  
 |  get_album_names(self)
 |      Return list of album names.
 |  
 |  get_albums(self)
 |      Get a list of all albums in the account.
 |  
 |  get_authorize_url(self)
 |      Returns the URL for OAuth authorisation.
 |  
 |  get_child_node_uri(self, node_id)
 |      Get child node uri.
 |  
 |  get_folder_id(self, folder_name)
 |      Get category id.
 |  
 |  get_folder_names(self)
 |      Return list of (top-level) folder names.
 |  
 |  get_folders(self)
 |      Get a list of folder names under the user.
 |      Currently supports only top-level folders.
 |  
 |  get_image_date(self, imagemeta_data)
 |      Get an image date.
 |      
 |      There are a number of SmugMug image dates to choose from. The dates
 |      obtained from the (images) or (geomedia) options refer to upload and 
 |      adjustment times. The original image date must be extracted from the image
 |      metadata EXIF/IPTC. Sadly, the metadata tag is nonstandard and there are a 
 |      number of choices. This function attempts to pick the best available 
 |      original image date.
 |  
 |  get_image_download_url(self, image_id)
 |      Get the link for dowloading an image.
 |  
 |  get_latitude_longitude_altitude(self, album_images)
 |      Get a list of {ImageKey, (Latitute,Longitude,Altitude)} dictionaries for (album_images).
 |  
 |  mirror_folders_offline(self, root_uri, root_dir, func_album=None, func_folder=None)
 |      Recursively walk online SmugMug folders and albums and apply
 |      functions (func_album) and (func_folder).
 |  
 |  request(self, method, url, params={}, headers={}, files={}, data=None, header_auth=False, retries=5, sleep=5)
 |      Performs requests, with multiple attempts if needed.
 |  
 |  request_once(self, method, url, params={}, headers={}, files={}, data=None, header_auth=False)
 |      Performs a single request.
 |  
 |  upload_image(self, image_data, image_name, image_type, album_id)
 |      Upload an image.
 |  
 |  write_album_manifest(self, album_id, name, path)
 |      Write TAB delimited file of SmugMug image metadata.
 |  
 |  write_album_real_dates(self, album_id, name, path)
 |      Write TAB delimited file of SmugMug image real dates.
 |      My image dates vary from fairly reliable EXIF dates
 |      to wild guesses about century old prints. SmugMug 
 |      requires a full date and it looks in a number
 |      of places for full dates, see: (get_image_date).
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  base36decode(number)
 |      Decode base 36 string and return integer.
 |  
 |  base36encode(number)
 |      Encode positive integer as a base 36 string.
 |  
 |  decode(obj, encoding='utf-8')
 |  
 |  extract_alphanum(in_string)
 |  
 |  load_image(image_path)
 |      Load the image data from a path.
 |  
 |  purify_smugmug_text(in_string)
 |      Convert Smugmug unicode strings to ascii equivalents making non-ascii
 |      characters visible as XML escapes. Also convert embedded control 
 |      character to blanks.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  smugmug_access_token_uri = 'http://api.smugmug.com/services/oauth/1.0a...
 |  
 |  smugmug_api_base_url = 'https://api.smugmug.com/api/v2'
 |  
 |  smugmug_api_version = 'v2'
 |  
 |  smugmug_authorize_uri = 'http://api.smugmug.com/services/oauth/1.0a/au...
 |  
 |  smugmug_base_uri = 'http://api.smugmug.com'
 |  
 |  smugmug_config = r'C:\Users\john\.smugpyter.cfg'
 |  
 |  smugmug_request_token_uri = 'http://api.smugmug.com/services/oauth/1.0...
 |  
 |  smugmug_upload_uri = 'http://upload.smugmug.com/'

In [5]:
smugmug.get_albums()
Out[5]:
[{'AlbumKey': 'ZdrcmM',
  'Title': 'Cover Images',
  'Uri': '/api/v2/album/ZdrcmM'},
 {'AlbumKey': 'rXGggF',
  'Title': 'Profile Images',
  'Uri': '/api/v2/album/rXGggF'},
 {'AlbumKey': 'XghWcL',
  'Title': 'Great and Greater Forebearers',
  'Uri': '/api/v2/album/XghWcL'},
 {'AlbumKey': 'cMf2ft',
  'Title': 'Minnie Raver',
  'Uri': '/api/v2/album/cMf2ft'},
 {'AlbumKey': '5bbtrT',
  'Title': 'Grandparents',
  'Uri': '/api/v2/album/5bbtrT'},
 {'AlbumKey': 'sThkPh', 'Title': 'My Kids', 'Uri': '/api/v2/album/sThkPh'},
 {'AlbumKey': 'X8X9pK',
  'Title': 'The Way We Were',
  'Uri': '/api/v2/album/X8X9pK'},
 {'AlbumKey': 'FZK4j4',
  'Title': "From Hazel's Albums",
  'Uri': '/api/v2/album/FZK4j4'},
 {'AlbumKey': 'ctPGZq',
  'Title': 'Inlaws Outlaws and Friends',
  'Uri': '/api/v2/album/ctPGZq'},
 {'AlbumKey': 'Gq5ZQB',
  'Title': "My Wife's Family",
  'Uri': '/api/v2/album/Gq5ZQB'},
 {'AlbumKey': 'KVcLKD',
  'Title': 'Helen Hamilton',
  'Uri': '/api/v2/album/KVcLKD'},
 {'AlbumKey': '8vcPbH', 'Title': 'Video', 'Uri': '/api/v2/album/8vcPbH'},
 {'AlbumKey': 'gLd4hT',
  'Title': 'Idaho Instants',
  'Uri': '/api/v2/album/gLd4hT'},
 {'AlbumKey': 'mfmZVp',
  'Title': 'In and Around Ottawa',
  'Uri': '/api/v2/album/mfmZVp'},
 {'AlbumKey': 'GKP92h',
  'Title': 'Montana Now and Then',
  'Uri': '/api/v2/album/GKP92h'},
 {'AlbumKey': 'dRSKq6',
  'Title': 'Indiana Images',
  'Uri': '/api/v2/album/dRSKq6'},
 {'AlbumKey': 'Rrzd8K', 'Title': 'Minnesota', 'Uri': '/api/v2/album/Rrzd8K'},
 {'AlbumKey': 'XtSj7f',
  'Title': 'New Mexico Montage',
  'Uri': '/api/v2/album/XtSj7f'},
 {'AlbumKey': '8dvZ6h',
  'Title': 'Missouri Moments',
  'Uri': '/api/v2/album/8dvZ6h'},
 {'AlbumKey': 'SZPrS3',
  'Title': 'Kingston Ontario',
  'Uri': '/api/v2/album/SZPrS3'},
 {'AlbumKey': 'thTWRC',
  'Title': 'California Captures',
  'Uri': '/api/v2/album/thTWRC'},
 {'AlbumKey': 'b9MqQc', 'Title': "Iran 1960's", 'Uri': '/api/v2/album/b9MqQc'},
 {'AlbumKey': 'Kng6tg',
  'Title': "Ghana 1970's",
  'Uri': '/api/v2/album/Kng6tg'},
 {'AlbumKey': 'QPZ5K7',
  'Title': "Beirut Lebanon 1960's",
  'Uri': '/api/v2/album/QPZ5K7'},
 {'AlbumKey': '3JgMKp',
  'Title': 'Diving at Bellairs Barbados BW',
  'Uri': '/api/v2/album/3JgMKp'},
 {'AlbumKey': 'Cwgc79', 'Title': 'Weekenders', 'Uri': '/api/v2/album/Cwgc79'},
 {'AlbumKey': 'nvfkPz',
  'Title': 'Western Road Trip 2015',
  'Uri': '/api/v2/album/nvfkPz'},
 {'AlbumKey': 'J7bGfF',
  'Title': 'North by Northwest',
  'Uri': '/api/v2/album/J7bGfF'},
 {'AlbumKey': 'BFs8q4',
  'Title': 'Tetons Yellowstone 2013',
  'Uri': '/api/v2/album/BFs8q4'},
 {'AlbumKey': 'MrjqMc',
  'Title': 'Arizona Toodling',
  'Uri': '/api/v2/album/MrjqMc'},
 {'AlbumKey': '8SHzxn',
  'Title': 'Along the Yukon River',
  'Uri': '/api/v2/album/8SHzxn'},
 {'AlbumKey': '47CC2p',
  'Title': 'Virginia Fall 2010',
  'Uri': '/api/v2/album/47CC2p'},
 {'AlbumKey': 'r8RHHd',
  'Title': 'Chicago 2007',
  'Uri': '/api/v2/album/r8RHHd'},
 {'AlbumKey': 'BQ3QDf',
  'Title': 'New York 2005',
  'Uri': '/api/v2/album/BQ3QDf'},
 {'AlbumKey': '35K9VD',
  'Title': 'Banff and Jasper 2006',
  'Uri': '/api/v2/album/35K9VD'},
 {'AlbumKey': '9NVXV3',
  'Title': 'Washington DC 2007',
  'Uri': '/api/v2/album/9NVXV3'},
 {'AlbumKey': 'JLs8pq',
  'Title': 'Ottawa to Irvine',
  'Uri': '/api/v2/album/JLs8pq'},
 {'AlbumKey': 'bSbPTr',
  'Title': 'South America 1979',
  'Uri': '/api/v2/album/bSbPTr'},
 {'AlbumKey': 'dccmZ7',
  'Title': "Enewetak Atoll 1980's",
  'Uri': '/api/v2/album/dccmZ7'},
 {'AlbumKey': 'KqMzxx',
  'Title': "ACS School Trips 1960's",
  'Uri': '/api/v2/album/KqMzxx'},
 {'AlbumKey': 'k65QRs',
  'Title': 'Zambia Eclipse Trip',
  'Uri': '/api/v2/album/k65QRs'},
 {'AlbumKey': 'CHK683',
  'Title': 'Briefly Bermuda',
  'Uri': '/api/v2/album/CHK683'},
 {'AlbumKey': 'nSkRJd', 'Title': 'Panoramas', 'Uri': '/api/v2/album/nSkRJd'},
 {'AlbumKey': 'SCSNdg',
  'Title': 'Image Hacking',
  'Uri': '/api/v2/album/SCSNdg'},
 {'AlbumKey': 'pbTtkm',
  'Title': 'Restorations',
  'Uri': '/api/v2/album/pbTtkm'},
 {'AlbumKey': 'cqZRDB',
  'Title': 'Partial Restorations',
  'Uri': '/api/v2/album/cqZRDB'},
 {'AlbumKey': 'fF2rxz',
  'Title': 'Logos Screenshots Covers',
  'Uri': '/api/v2/album/fF2rxz'},
 {'AlbumKey': 'WpcrnD', 'Title': 'Abodes', 'Uri': '/api/v2/album/WpcrnD'},
 {'AlbumKey': '9cxcL6',
  'Title': 'Caught My Eye',
  'Uri': '/api/v2/album/9cxcL6'},
 {'AlbumKey': 'Gb298V',
  'Title': 'Does Not Fit',
  'Uri': '/api/v2/album/Gb298V'},
 {'AlbumKey': 'XP8GpZ', 'Title': 'Flat Things', 'Uri': '/api/v2/album/XP8GpZ'},
 {'AlbumKey': 'PfCsJz',
  'Title': 'Cell Phoning It In',
  'Uri': '/api/v2/album/PfCsJz'},
 {'AlbumKey': 'w5PKZp',
  'Title': 'Been There Done That',
  'Uri': '/api/v2/album/w5PKZp'},
 {'AlbumKey': 'RBxsfC', 'Title': 'Fiscal 2009', 'Uri': '/api/v2/album/RBxsfC'},
 {'AlbumKey': 'fTHTbN',
  'Title': 'To Much Information',
  'Uri': '/api/v2/album/fTHTbN'},
 {'AlbumKey': 'XXJhZx',
  'Title': 'Cripple Chronicles',
  'Uri': '/api/v2/album/XXJhZx'},
 {'AlbumKey': 'RS4ZXJ', 'Title': '63', 'Uri': '/api/v2/album/RS4ZXJ'},
 {'AlbumKey': 'RMWQ6K',
  'Title': 'Direct Cell Uploads',
  'Uri': '/api/v2/album/RMWQ6K'},
 {'AlbumKey': 'GMLn9k', 'Title': 'utilimages', 'Uri': '/api/v2/album/GMLn9k'},
 {'AlbumKey': 'NmMfCc',
  'Title': 'My SmugMug Site Files',
  'Uri': '/api/v2/album/NmMfCc'},
 {'AlbumKey': 'kbnPnG',
  'Title': 'smugsitefiles',
  'Uri': '/api/v2/album/kbnPnG'},
 {'AlbumKey': 'm585cH',
  'Title': 'Camera Awesome Photos',
  'Uri': '/api/v2/album/m585cH'},
 {'AlbumKey': 'jdfVRr',
  'Title': 'Camera Awesome Archive',
  'Uri': '/api/v2/album/jdfVRr'},
 {'AlbumKey': 's4h35q', 'Title': 'Email', 'Uri': '/api/v2/album/s4h35q'}]
In [6]:
smugmug.get_folders()
Out[6]:
[{'Name': 'People', 'NodeID': 'qHr93', 'UrlName': 'People'},
 {'Name': 'Places', 'NodeID': 'J5wzG', 'UrlName': 'Places'},
 {'Name': 'Trips', 'NodeID': 'PgwWL', 'UrlName': 'Trips'},
 {'Name': 'Themes', 'NodeID': 'j7VKf', 'UrlName': 'Themes'},
 {'Name': 'Other', 'NodeID': 'L7kKx', 'UrlName': 'Other'}]
In [7]:
caught_my_eye = smugmug.get_album_id('Caught My Eye')
forebearers = smugmug.get_album_id('Great and Greater Forebearers')
idaho_instants = smugmug.get_album_id("Idaho Instants")
cell_phoning = smugmug.get_album_id("Cell Phoning It In")
[caught_my_eye, forebearers, idaho_instants, cell_phoning]
Out[7]:
['9cxcL6', 'XghWcL', 'gLd4hT', 'PfCsJz']
In [8]:
smugmug.get_album_info(caught_my_eye)
Out[8]:
{'AlbumKey': '9cxcL6',
 'AllowDownloads': False,
 'Backprinting': '',
 'BoutiquePackaging': 'Inherit from User',
 'CanRank': False,
 'CanShare': True,
 'Clean': False,
 'Comments': True,
 'Date': '2009-02-26T23:04:55+00:00',
 'Description': '',
 'EXIF': True,
 'External': True,
 'FamilyEdit': False,
 'Filenames': False,
 'FriendEdit': False,
 'Geography': True,
 'HasDownloadPassword': False,
 'Header': 'Custom',
 'HideOwner': False,
 'HighlightAlbumImageUri': '/api/v2/album/9cxcL6/image/KkkHHKH-0',
 'ImageCount': 105,
 'ImagesLastUpdated': '2017-08-27T01:15:22+00:00',
 'InterceptShipping': 'Inherit from User',
 'Keywords': '',
 'LargestSize': '5K',
 'LastUpdated': '2017-08-27T01:15:10+00:00',
 'Name': 'Caught My Eye',
 'NiceName': 'Caught-My-Eye-1',
 'NodeID': 'pCmh2',
 'OriginalSizes': 279723936,
 'PackagingBranding': True,
 'Password': '',
 'PasswordHint': '',
 'Printable': False,
 'Privacy': 'Public',
 'ProofDays': 0,
 'Protected': True,
 'ResponseLevel': 'Full',
 'SecurityType': 'None',
 'Share': True,
 'Slideshow': True,
 'SmugSearchable': 'Inherit from User',
 'SortDirection': 'Ascending',
 'SortMethod': 'Date Taken',
 'SquareThumbs': False,
 'Title': 'Caught My Eye',
 'TotalSizes': 455677797,
 'UploadKey': '',
 'Uri': '/api/v2/album/9cxcL6',
 'UriDescription': 'Album by key',
 'Uris': {'AlbumComments': {'EndpointType': 'AlbumComments',
   'Locator': 'Comment',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!comments',
   'UriDescription': 'Comments on album'},
  'AlbumDownload': {'EndpointType': 'AlbumDownload',
   'Locator': 'Download',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!download',
   'UriDescription': 'Download album'},
  'AlbumGeoMedia': {'EndpointType': 'AlbumGeoMedia',
   'Locator': 'AlbumImage',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!geomedia',
   'UriDescription': 'Geotagged images from album'},
  'AlbumGrants': {'EndpointType': 'AlbumGrants',
   'Locator': 'Grant',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!grants',
   'UriDescription': 'Grants for the Album'},
  'AlbumHighlightImage': {'EndpointType': 'AlbumHighlightImage',
   'Locator': 'AlbumImage',
   'LocatorType': 'Object',
   'Uri': '/api/v2/album/9cxcL6!highlightimage',
   'UriDescription': 'Highlight image for album'},
  'AlbumImages': {'EndpointType': 'AlbumImages',
   'Locator': 'AlbumImage',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!images',
   'UriDescription': 'Images from album'},
  'AlbumPopularMedia': {'EndpointType': 'AlbumPopularMedia',
   'Locator': 'AlbumImage',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!popularmedia',
   'UriDescription': 'Popular images from album'},
  'AlbumPrices': {'EndpointType': 'AlbumPrices',
   'Locator': 'CatalogSkuPrice',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/album/9cxcL6!prices',
   'UriDescription': 'Purchasable Skus'},
  'AlbumShareUris': {'EndpointType': 'AlbumShareUris',
   'Locator': 'AlbumShareUris',
   'LocatorType': 'Object',
   'Uri': '/api/v2/album/9cxcL6!shareuris',
   'UriDescription': 'URIs that are useful for sharing'},
  'ApplyAlbumTemplate': {'EndpointType': 'ApplyAlbumTemplate',
   'Locator': 'ApplyAlbumTemplate',
   'LocatorType': 'Object',
   'Uri': '/api/v2/album/9cxcL6!applyalbumtemplate',
   'UriDescription': 'Apply an album template'},
  'CollectImages': {'EndpointType': 'CollectImages',
   'Uri': '/api/v2/album/9cxcL6!collectimages',
   'UriDescription': 'Collect images into album'},
  'DeleteAlbumImages': {'EndpointType': 'DeleteAlbumImages',
   'Uri': '/api/v2/album/9cxcL6!deleteimages',
   'UriDescription': 'Delete album images'},
  'Folder': {'EndpointType': 'Folder',
   'Locator': 'Folder',
   'LocatorType': 'Object',
   'Uri': '/api/v2/folder/user/conceptcontrol/Themes/Observations',
   'UriDescription': 'A folder or legacy (sub)category by UrlPath'},
  'HighlightImage': {'EndpointType': 'HighlightImage',
   'Locator': 'Image',
   'LocatorType': 'Object',
   'Uri': '/api/v2/highlight/node/pCmh2',
   'UriDescription': 'Highlight image for a folder, album, or page'},
  'MoveAlbumImages': {'EndpointType': 'MoveAlbumImages',
   'Uri': '/api/v2/album/9cxcL6!moveimages',
   'UriDescription': 'Move images into album'},
  'Node': {'EndpointType': 'Node',
   'Locator': 'Node',
   'LocatorType': 'Object',
   'Uri': '/api/v2/node/pCmh2',
   'UriDescription': 'Node with the given id.'},
  'NodeCoverImage': {'EndpointType': 'NodeCoverImage',
   'Locator': 'Image',
   'LocatorType': 'Object',
   'Uri': '/api/v2/node/pCmh2!cover',
   'UriDescription': 'Cover image for a folder, album, or page'},
  'ParentFolders': {'EndpointType': 'ParentFolders',
   'Locator': 'Folder',
   'LocatorType': 'Objects',
   'Uri': '/api/v2/folder/user/conceptcontrol/Themes/Observations!parents',
   'UriDescription': 'The sequence of parent folders, from the given folder to the root'},
  'UploadFromUri': {'EndpointType': 'UploadFromUri',
   'Uri': '/api/v2/album/9cxcL6!uploadfromuri',
   'UriDescription': 'Upload a photo or video from elsewhere on the web'},
  'User': {'EndpointType': 'User',
   'Locator': 'User',
   'LocatorType': 'Object',
   'Uri': '/api/v2/user/conceptcontrol',
   'UriDescription': 'User By Nickname'}},
 'UrlName': 'Caught-My-Eye-1',
 'UrlPath': '/Themes/Observations/Caught-My-Eye-1',
 'Watermark': False,
 'WebUri': 'https://conceptcontrol.smugmug.com/Themes/Observations/Caught-My-Eye-1',
 'WorldSearchable': True}
In [9]:
album_images = smugmug.get_album_images(forebearers)
len(album_images)
Out[9]:
13
In [10]:
album_captions = smugmug.get_album_image_captions(album_images)
album_latitude_longitude = smugmug.get_latitude_longitude_altitude(album_images)
album_latitude_longitude
Out[10]:
[{'ImageKey': 'qjQxPCS',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'HwwwgDs',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'fjcPpCk',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'jq54JDC',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'hVw9ng9',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'xT2ptsn',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': '4sPSDfZ',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'P2QzsRD',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'QNQGMVg',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'KSMVDvH',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'k6pnSJ4',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'LX8HmDV',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)},
 {'ImageKey': 'bcDxNfx',
  'LatLongAlt': ('0.00000000000000', '0.00000000000000', 0)}]
In [11]:
album_real_dates = smugmug.get_album_image_real_dates(album_images)
album_real_dates
Out[11]:
[{'AlbumKey': 'XghWcL',
  'FileName': 'albert raver aunt may cousin marci daughter.jpg',
  'ImageKey': 'qjQxPCS',
  'RealDate': '1910-03-03T01:01:01'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'augustus a burdick 1864.jpg',
  'ImageKey': 'HwwwgDs',
  'RealDate': '1864-02-25T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'bert raver 1900 [189878796].jpg',
  'ImageKey': 'fjcPpCk',
  'RealDate': '1900-01-01T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'eliza jane gilbert b1840 d1915.jpg',
  'ImageKey': 'jq54JDC',
  'RealDate': '1880-07-04T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'gert parents wedding gilbert paula 1905.jpg',
  'ImageKey': 'hVw9ng9',
  'RealDate': '1906-02-02T01:01:01'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'gilbert tilly eliza leone 1914.jpg',
  'ImageKey': 'xT2ptsn',
  'RealDate': '1914-03-03T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'julia ann weber rock 1894.jpg',
  'ImageKey': '4sPSDfZ',
  'RealDate': '1894-06-01T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'lydia ayres.jpg',
  'ImageKey': 'P2QzsRD',
  'RealDate': '1870-01-01T01:01:01'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'margo man painting 1971.jpg',
  'ImageKey': 'QNQGMVg',
  'RealDate': '1971-08-08T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'mary theresa rock b1861 d1945.jpg',
  'ImageKey': 'KSMVDvH',
  'RealDate': '1930-01-01T01:01:01'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'minnie e raver.jpg',
  'ImageKey': 'k6pnSJ4',
  'RealDate': '1945-01-01T00:00:00'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'william evert baker portrait 1950.jpg',
  'ImageKey': 'LX8HmDV',
  'RealDate': '1950-01-15T01:01:01'},
 {'AlbumKey': 'XghWcL',
  'FileName': 'william great grandfather 1946.jpg',
  'ImageKey': 'bcDxNfx',
  'RealDate': '1946-05-05T14:01:01'}]

Try out other "unsupported" version 2.0 API calls. Documentation for SmugMug Version 2.0 API calls is best obtained by hacking with SmugMug's live API browser tool at:

https://api.smugmug.com/api/v2

The live API tool is far more useful if you log into your SmugMug account and point at your own images.

Walk SmugMug folders and albums and download coveted metadata.

The next cell calls the main function that walks SmugMug folders and writes metadata to local directories. Metadata is saved in TAB delimited CSV manifest files. TAB delimited files are also called TSV files. The function writes one file per album. If local directories do not exist they are created. If manifest files already exist they are are overwritten. The entire SmugMug tree is walked. You might want to adjust where the walk starts if you have hundreds or thousands of albums.

Manifest files follow this naming convention.

manifest-<deblanked-album-name>-<smugmug-album-key>-<key-case-mask>.txt

Here are some examples:

manifest-ZambiaEclipseTrip-k65QRs-6.txt
manifest-FromHazelsAlbums-FZK4j4-1k.txt
In [12]:
smugmug = smugpyter.SmugPyter()

smugmug.download_smugmug_mirror(func_album=smugmug.write_album_manifest)
visiting album GreatandGreaterForebearers
visiting album MinnieRaver
visiting album Grandparents
visiting album MyKids
visiting album TheWayWeWere
visiting album FromHazelsAlbums
visiting album InlawsOutlawsandFriends
visiting album MyWifesFamily
visiting album HelenHamilton
visiting album Video
visiting album IdahoInstants
visiting album InandAroundOttawa
visiting album MontanaNowandThen
visiting album IndianaImages
visiting album Minnesota
visiting album NewMexicoMontage
visiting album MissouriMoments
visiting album KingstonOntario
visiting album CaliforniaCaptures
visiting album Iran1960s
visiting album Ghana1970s
visiting album BeirutLebanon1960s
visiting album DivingatBellairsBarbadosBW
visiting album Weekenders
visiting album WesternRoadTrip2015
visiting album NorthbyNorthwest
visiting album TetonsYellowstone2013
visiting album ArizonaToodling
visiting album AlongtheYukonRiver
visiting album VirginiaFall2010
visiting album Chicago2007
visiting album NewYork2005
visiting album BanffandJasper2006
visiting album SouthAmerica1979
visiting album EnewetakAtoll1980s
visiting album ACSSchoolTrips1960s
visiting album ZambiaEclipseTrip
visiting album BrieflyBermuda
visiting album Panoramas
visiting album ImageHacking
visiting album Restorations
visiting album PartialRestorations
visiting album LogosScreenshotsCovers
visiting album Abodes
visiting album CaughtMyEye
visiting album DoesNotFit
visiting album FlatThings
visiting album CellPhoningItIn
visiting album BeenThereDoneThat
visiting album Fiscal2009
visiting album ToMuchInformation
visiting album CrippleChronicles
visiting album 63
visiting album DirectCellUploads
visiting album utilimages
visiting album MySmugMugSiteFiles
visiting album smugsitefiles
visiting album CameraAwesomePhotos
empty album CameraAwesomePhotos
visiting album CameraAwesomeArchive
empty album CameraAwesomeArchive
visiting album Email
empty album Email
done

Next on the Agenda!

Remember how "no good dead goes unpunished." Well, running code will be "enhanced" whether it's necessary or not. Now that I have a local directories that contain relevant SmugMug metadata in an easily consumed CSV form other notebooks will use these directories to generate what I call "long duration documents." My prefered long duration sources are Markdown and LaTex. Both of these text formats will be readable for centuries if printed on high quality acid free paper and stored in numerous "secure and undisclosed locations."

Remember, always Analyze the Data not the Drivel.

John Baker, Meridian Idaho