This is one of the 100 recipes of the IPython Cookbook, the definitive guide to high-performance scientific computing and data science in Python.
You need the python-apt
package in order to build the package dependency graph. (https://pypi.python.org/pypi/python-apt/)
We also assume that this notebook is executed on a Debian system (like Ubuntu). If you don't have such a system, you can download the data Debian directly on the book's website. Extract it in the current directory, and start this notebook directly at step 7. (http://ipython-books.github.io)
apt
module and we build the list of packages.import json
import apt
cache = apt.Cache()
graph
dictionary will contain the adjacency list of a small portion of the dependency graph.graph = {}
def get_dependencies(package):
if package not in cache:
return []
pack = cache[package]
ver = pack.candidate or pack.versions[0]
# We flatten the list of dependencies,
# and we remove the duplicates.
return sorted(set([item.name
for sublist in ver.dependencies
for item in sublist]))
graph
variable.def get_dep_recursive(package):
if package not in cache:
return []
if package not in graph:
dep = get_dependencies(package)
graph[package] = dep
for dep in graph[package]:
if dep not in graph:
graph[dep] = get_dep_recursive(dep)
return graph[package]
get_dep_recursive('ipython');
with open('data/apt.json', 'w') as f:
json.dump(graph, f, indent=1)
import json
import numpy as np
import networkx as nx
import matplotlib.pyplot as plt
%matplotlib inline
with open('data/apt.json', 'r') as f:
graph = json.load(f)
DiGraph
in NetworkX) from our adjacency list. We reverse the graph to get a more natural ordering.g = nx.DiGraph(graph).reverse()
nx.is_directed_acyclic_graph(g)
simple_cycles
function.set([cycle[0] for cycle in nx.simple_cycles(g)])
g.remove_nodes_from(_)
nx.is_directed_acyclic_graph(g)
ug = g.to_undirected()
deg = ug.degree()
plt.figure(figsize=(4,4));
# The size of the nodes depends on the number of dependencies.
nx.draw(ug, font_size=6,
node_size=[20*deg[k] for k in ug.nodes()]);
nx.topological_sort(g)
You'll find all the explanations, figures, references, and much more in the book (to be released later this summer).
IPython Cookbook, by Cyrille Rossant, Packt Publishing, 2014 (500 pages).