%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from IPython.display import Image
pd.set_option('display.mpl_style', 'default') # Make the graphs a bit prettier
plt.rcParams['figure.figsize'] = (15, 8)
for a fine tutorial, from which I'm showing slides, see http://www.scottbot.net/HIAL/?page_id=41142
Image("https://upload.wikimedia.org/wikipedia/commons/thumb/2/28/6n-graph2.svg/300px-6n-graph2.svg.png")
###h/t wikipedia
1 IS CONNECTED TO 1
1 IS CONNECTED TO 2
1 IS CONNECTED TO 5
2 IS CONNECTED IS 1
. . . .
$\begin{pmatrix} 1 & 1 & 0 & 0 & 1 & 0\\ 1 & 0 & 1 & 0 & 1 & 0\\ 0 & 1 & 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0 & 1 & 1\\ 1 & 1 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0\\ \end{pmatrix}$
A $1$ at column $i$ and row $j$ means that $i$ and $j$ are connected.
$6$ is connected only to $4$; $1$ is connected to itself, $2$, and $5$.
This graph is symmetric: it assumes a connection from $1$ to $2$ is the same as one from $2$ to $1$.
Often this is not the case: think of friending someone on a social network: A friending B does not mean B friends A. Such a graph is called a directed graph.
Create a matrix capturing some of the relationships described in Peter Gabriels' Games without Frontiers (1980):
Jeux sans frontières (4x)
Hans plays with Lotte
Lotte plays with Jane
Jane plays with Willi
Willi is happy again
Suki plays with Leo
Sacha plays with Britt
Adolf builds a bonfire
Enrico plays with it
source: http://petergabriel.com/video/peter-gabriel-games-without-frontiers/
Assume "plays with" is non-directed, so "plays with" could be "plays together."
I'm taking this example from http://kieranhealy.org/blog/archives/2013/06/09/using-metadata-to-find-paul-revere/ and moving it from R
to python
membership_matrix=pd.read_csv("https://raw.githubusercontent.com/kjhealy/revere/master/data/PaulRevereAppD.csv", index_col=[0])
membership_matrix
StAndrewsLodge | LoyalNine | NorthCaucus | LongRoomClub | TeaParty | BostonCommittee | LondonEnemies | |
---|---|---|---|---|---|---|---|
Adams.John | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
Adams.Samuel | 0 | 0 | 1 | 1 | 0 | 1 | 1 |
Allen.Dr | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Appleton.Nathaniel | 0 | 0 | 1 | 0 | 0 | 1 | 0 |
Ash.Gilbert | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Austin.Benjamin | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Austin.Samuel | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Avery.John | 0 | 1 | 0 | 0 | 0 | 0 | 1 |
Baldwin.Cyrus | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Ballard.John | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Barber.Nathaniel | 0 | 0 | 1 | 0 | 1 | 1 | 1 |
Barnard.Samuel | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Barrett.Samuel | 1 | 0 | 0 | 0 | 0 | 0 | 1 |
Bass.Henry | 0 | 1 | 1 | 0 | 1 | 0 | 1 |
Bell.William | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Blake.Increase | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Boit.John | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Bolter.Thomas | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Boyer.Peter | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Boynton.Richard | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
Brackett.Jos | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Bradford.John | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
Bradlee.David | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Bradlee.Josiah | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Bradlee.Nathaniel | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Bradlee.Thomas | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Bray.George | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Breck.William | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Bewer.James | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Brimmer.Herman | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
... | ... | ... | ... | ... | ... | ... | ... |
Swan.James | 0 | 0 | 1 | 0 | 1 | 0 | 0 |
Sweetser.John | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
Symmes.Eben | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Symmes.John | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Tabor.Philip | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Tileston.Thomas | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Trott.George | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
Tyler.Royall | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
Urann.Thomas | 1 | 0 | 1 | 0 | 1 | 0 | 0 |
Vernon.Fortesque | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Waldo.Benjamin | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Warren.Joseph | 1 | 0 | 1 | 1 | 0 | 1 | 1 |
Webb.Joseph | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Webster.Thomas | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Welles.Henry | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
Wendell.Oliver | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
Wheeler.Josiah | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
White.Samuel | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Whitten.John | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Whitwell.Samuel | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Whitwell.William | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Williams.Jeremiah | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Williams.Jonathan | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Williams.Thomas | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Willis.Nathaniel | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Wingfield.William | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
Winslow.John | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
Winthrop.John | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
Wyeth.Joshua | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Young.Thomas | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
254 rows × 7 columns
We have a matrix of 254 rows and 7 columns
We are intested in
who is connected to whom
which organization most connected to which other organization.
Our quarry is something like the similarity matrices we looked at in doing our work with text.
We are going to create an adjacency matrix, another square matrix that indicates who is connected to whom.
(You are not responsible for this but I want you to know the smoke and mirrors.)
We have to do a little of basic arithmetic with matrices to make this work.
First we need to recall the idea of the transpose, which is just flipping the rows and columns of a matrix.
$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}^{\mathrm{T}} = \begin{bmatrix} 1 & 3 \\ 2 & 4 \end{bmatrix}$
The adjacency matrix is the product of a matrix and its transpose.
Say we have a $257 * 7$ matrix; its transpose will be $7 * 257$ matrix.
If we multiply an $M * N$ matrix by an $N * M$ matrix, we get a $M * M$ matrix.
So let's start with the adjacency of every person to every other person.
Our goal, then, is a symmetrical matrix that matches $M$ people to $M$ people.
In python we can use the .dot
method to perform the necessary form of matrix multiplication. We can do this directly on pandas
dataframes.
person_adjacency=membership_matrix.dot(membership_matrix.T)
person_adjacency
Adams.John | Adams.Samuel | Allen.Dr | Appleton.Nathaniel | Ash.Gilbert | Austin.Benjamin | Austin.Samuel | Avery.John | Baldwin.Cyrus | Ballard.John | ... | Whitwell.William | Williams.Jeremiah | Williams.Jonathan | Williams.Thomas | Willis.Nathaniel | Wingfield.William | Winslow.John | Winthrop.John | Wyeth.Joshua | Young.Thomas | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Adams.John | 2 | 2 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 1 |
Adams.Samuel | 2 | 4 | 1 | 2 | 0 | 1 | 1 | 1 | 1 | 1 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 2 | 0 | 2 |
Allen.Dr | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Appleton.Nathaniel | 1 | 2 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 2 |
Ash.Gilbert | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Austin.Benjamin | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Austin.Samuel | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Avery.John | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 2 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Baldwin.Cyrus | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Ballard.John | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Barber.Nathaniel | 1 | 3 | 1 | 2 | 0 | 1 | 1 | 1 | 1 | 1 | ... | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 2 | 1 | 3 |
Barnard.Samuel | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Barrett.Samuel | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 |
Bass.Henry | 1 | 2 | 1 | 1 | 0 | 1 | 1 | 2 | 1 | 1 | ... | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 2 | 1 | 2 |
Bell.William | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Blake.Increase | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Boit.John | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Bolter.Thomas | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Boyer.Peter | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Boynton.Richard | 0 | 2 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Brackett.Jos | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Bradford.John | 0 | 2 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Bradlee.David | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Bradlee.Josiah | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Bradlee.Nathaniel | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Bradlee.Thomas | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Bray.George | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Breck.William | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Bewer.James | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Brimmer.Herman | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
Swan.James | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 2 |
Sweetser.John | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
Symmes.Eben | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Symmes.John | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Tabor.Philip | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Tileston.Thomas | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Trott.George | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Tyler.Royall | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
Urann.Thomas | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 2 |
Vernon.Fortesque | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Waldo.Benjamin | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Warren.Joseph | 2 | 4 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | ... | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 |
Webb.Joseph | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Webster.Thomas | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Welles.Henry | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Wendell.Oliver | 0 | 2 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Wheeler.Josiah | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
White.Samuel | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
Whitten.John | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Whitwell.Samuel | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Whitwell.William | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Williams.Jeremiah | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Williams.Jonathan | 0 | 1 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Williams.Thomas | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Willis.Nathaniel | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Wingfield.William | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 |
Winslow.John | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
Winthrop.John | 1 | 2 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | ... | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 2 | 0 | 1 |
Wyeth.Joshua | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 |
Young.Thomas | 1 | 2 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 1 | ... | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 3 |
254 rows × 254 columns
Just as easily we can get the adjacency among the clubs.
This time we multipy the transpose by the original matrix.
(Remember: matrix multiplication is not commutative.)
person_names=person_adjacency.index
club_adjacency=membership_matrix.T.dot(membership_matrix)
club_adjacency
StAndrewsLodge | LoyalNine | NorthCaucus | LongRoomClub | TeaParty | BostonCommittee | LondonEnemies | |
---|---|---|---|---|---|---|---|
StAndrewsLodge | 53 | 2 | 3 | 2 | 3 | 1 | 3 |
LoyalNine | 2 | 10 | 3 | 0 | 2 | 0 | 3 |
NorthCaucus | 3 | 3 | 59 | 5 | 13 | 9 | 16 |
LongRoomClub | 2 | 0 | 5 | 17 | 2 | 5 | 5 |
TeaParty | 3 | 2 | 13 | 2 | 97 | 3 | 8 |
BostonCommittee | 1 | 0 | 9 | 5 | 3 | 21 | 11 |
LondonEnemies | 3 | 3 | 16 | 5 | 8 | 11 | 62 |
networkx
¶Installed by default in condas If problem, run in shell
conda install networkx
import networkx as nx
introduces new data type, the graph, with lots of operations to create, modify, analyze, graph, import and export graphs
find the full documentation at https://networkx.github.io/
Begin by initializing a graph, then add edges, nodes or both.
my_first_graph=nx.Graph()
like:
fred={}
my_first_graph.add_edge(1,2)
nx.draw(my_first_graph)
/home/mljones/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:1282: UserWarning: findfont: Font family [u'monospace'] not found. Falling back to Bitstream Vera Sans (prop.get_family(), self.defaultFamily[fontext]))
my_first_graph.add_edges_from([(2,3), (2,4)])
nx.draw(my_first_graph)
nx.number_of_nodes(my_first_graph)
4
my_first_graph.nodes()
[1, 2, 3, 4]
my_first_graph.edges()
[(1, 2), (2, 3), (2, 4)]
most important for us is "weight": how strong a connection between two nodes
my_first_graph.add_edge(4,5, weight=3)
nx.draw(my_first_graph)
Default simply graphing method not show the weighted edge.
We will fix this.
For examples of drawing in networkx
, see https://networkx.github.io/documentation/latest/gallery.html
We can get far finer control by building up out graph matplotlib
style.
Just instruct it to plot each element: edges, nodes, and labels.
pos=nx.spring_layout(my_first_graph,iterations=20)
nx.draw_networkx_edges(my_first_graph, pos)
nx.draw_networkx_nodes(my_first_graph, pos)
nx.draw_networkx_labels(my_first_graph, pos)
plt.show()
Don't want those ugly axes!
pos=nx.spring_layout(my_first_graph,iterations=20)
nx.draw_networkx_edges(my_first_graph, pos)
nx.draw_networkx_nodes(my_first_graph, pos)
nx.draw_networkx_labels(my_first_graph, pos)
plt.axis("off")
plt.show()
Note that the algorithm drawing the graphs is non-deterministic. It won't look the same from time to time.
pos=nx.spring_layout(my_first_graph,iterations=20)
nx.draw_networkx_edges(my_first_graph, pos)
nx.draw_networkx_nodes(my_first_graph, pos)
nx.draw_networkx_labels(my_first_graph, pos)
plt.axis("off")
plt.show()
Networkx
is not the best place to do sophisticated graphing.
In order to draw weighted edges differently, we need to add each kind of edge separately. To do so, we have to cycle through all the edges and pick out the ones that should be bigger.
The edges method will provide a list of all our edges.
only one has a weight!
my_first_graph.edges()
[(1, 2), (2, 3), (2, 4), (4, 5)]
my_first_graph.edges(data=True)
[(1, 2, {}), (2, 3, {}), (2, 4, {}), (4, 5, {'weight': 3})]
#skipping in interest of time
pos=nx.spring_layout(my_first_graph,iterations=20)
nx.draw_networkx_edges(my_first_graph,pos,edgelist=large_edges,
width=6)
nx.draw_networkx_edges(my_first_graph,pos,edgelist=small_edges,
width=2)
nx.draw_networkx_nodes(my_first_graph, pos)
nx.draw_networkx_labels(my_first_graph, pos)
plt.axis("off")
plt.show()
Let's now return to our example.
club_adjacency
StAndrewsLodge | LoyalNine | NorthCaucus | LongRoomClub | TeaParty | BostonCommittee | LondonEnemies | |
---|---|---|---|---|---|---|---|
StAndrewsLodge | 53 | 2 | 3 | 2 | 3 | 1 | 3 |
LoyalNine | 2 | 10 | 3 | 0 | 2 | 0 | 3 |
NorthCaucus | 3 | 3 | 59 | 5 | 13 | 9 | 16 |
LongRoomClub | 2 | 0 | 5 | 17 | 2 | 5 | 5 |
TeaParty | 3 | 2 | 13 | 2 | 97 | 3 | 8 |
BostonCommittee | 1 | 0 | 9 | 5 | 3 | 21 | 11 |
LondonEnemies | 3 | 3 | 16 | 5 | 8 | 11 | 62 |
networkx
includes a translation function from adjacency matrices to graphs from_numpy_matrix()
. There's also one for sparse matrices.
club_adjacency.values
array([[53, 2, 3, 2, 3, 1, 3], [ 2, 10, 3, 0, 2, 0, 3], [ 3, 3, 59, 5, 13, 9, 16], [ 2, 0, 5, 17, 2, 5, 5], [ 3, 2, 13, 2, 97, 3, 8], [ 1, 0, 9, 5, 3, 21, 11], [ 3, 3, 16, 5, 8, 11, 62]])
But we've lost the names of our clubs here.
We can put them back by providing a mapping from node numbers to names as a dictionary.
club_adjacency_graph=nx.from_numpy_matrix(club_adjacency.values)
club_adjacency_graph.edges(data=True)
[(0, 0, {'weight': 53}), (0, 1, {'weight': 2}), (0, 2, {'weight': 3}), (0, 3, {'weight': 2}), (0, 4, {'weight': 3}), (0, 5, {'weight': 1}), (0, 6, {'weight': 3}), (1, 1, {'weight': 10}), (1, 2, {'weight': 3}), (1, 4, {'weight': 2}), (1, 6, {'weight': 3}), (2, 2, {'weight': 59}), (2, 3, {'weight': 5}), (2, 4, {'weight': 13}), (2, 5, {'weight': 9}), (2, 6, {'weight': 16}), (3, 3, {'weight': 17}), (3, 4, {'weight': 2}), (3, 5, {'weight': 5}), (3, 6, {'weight': 5}), (4, 4, {'weight': 97}), (4, 5, {'weight': 3}), (4, 6, {'weight': 8}), (5, 5, {'weight': 21}), (5, 6, {'weight': 11}), (6, 6, {'weight': 62})]
pos=nx.spring_layout(club_adjacency_graph,iterations=20)
nx.draw_networkx_edges(club_adjacency_graph, pos)
nx.draw_networkx_nodes(club_adjacency_graph, pos)
nx.draw_networkx_labels(club_adjacency_graph, pos)
plt.axis("off")
plt.show()
club_adjacency.index
Index([u'StAndrewsLodge', u'LoyalNine', u'NorthCaucus', u'LongRoomClub', u'TeaParty', u'BostonCommittee', u'LondonEnemies'], dtype='object')
club_adjacency_names=club_adjacency.index
club_adjacency_graph.nodes()
[0, 1, 2, 3, 4, 5, 6]
Now let's do it with our adjacency matrix for the club members.
dict(zip(club_adjacency_graph.nodes(), club_adjacency_names))
{0: 'StAndrewsLodge', 1: 'LoyalNine', 2: 'NorthCaucus', 3: 'LongRoomClub', 4: 'TeaParty', 5: 'BostonCommittee', 6: 'LondonEnemies'}
dict = __builtins__.dict
dict
dict
name_mapping=dict((zip(club_adjacency_graph.nodes(), club_adjacency_names)))
club_adjacency_graph=nx.relabel_nodes(club_adjacency_graph, name_mapping)
pos=nx.spring_layout(club_adjacency_graph,iterations=20)
nx.draw_networkx_edges(club_adjacency_graph, pos)
nx.draw_networkx_nodes(club_adjacency_graph, pos)
nx.draw_networkx_labels(club_adjacency_graph, pos)
plt.axis("off")
plt.show()
person_adjacency_graph=nx.from_numpy_matrix(person_adjacency.values)
name_mapping=dict(zip(person_adjacency_graph.nodes(), person_names))
person_adjacency_graph=nx.relabel_nodes(person_adjacency_graph, name_mapping)
nx.draw(person_adjacency_graph)
UNintelligible!
pos=nx.spring_layout(person_adjacency_graph, iterations=30)
plt.figure(figsize=(40,40))
nx.draw_networkx_edges(person_adjacency_graph, pos, width=.02)
nx.draw_networkx_nodes(person_adjacency_graph, pos, node_size=50)
nx.draw_networkx_labels(person_adjacency_graph, pos)
plt.axis('off')
plt.show()
networkx
, allows a wide range of analysis, including sophisticated supervised and unsupervised learning.¶Google
's Page-Rank
--the core of its search algorithm--is a graph analysis algorithm.
Intuition is that websites (nodes) referred to by other high-ranking websites (nodes) should be more highly ranked
person_adjacency_graph_centrality=nx.eigenvector_centrality(person_adjacency_graph, max_iter=30)
person_adjacency_graph_centrality
{'Adams.John': 0.04301343128171425, 'Adams.Samuel': 0.0862709288337976, 'Allen.Dr': 0.036550282437213824, 'Appleton.Nathaniel': 0.04782108610845363, 'Ash.Gilbert': 0.008977228114293228, 'Austin.Benjamin': 0.0319866938808436, 'Austin.Samuel': 0.0319866938808436, 'Avery.John': 0.035986843459806635, 'Baldwin.Cyrus': 0.0319866938808436, 'Ballard.John': 0.036550282437213824, 'Barber.Nathaniel': 0.1618414997261579, 'Barnard.Samuel': 0.0820337197368607, 'Barrett.Samuel': 0.04096392199513686, 'Bass.Henry': 0.1545708456338811, 'Bell.William': 0.008977228114293228, 'Bewer.James': 0.0820337197368607, 'Blake.Increase': 0.008977228114293228, 'Boit.John': 0.036550282437213824, 'Bolter.Thomas': 0.0820337197368607, 'Boyer.Peter': 0.0319866938808436, 'Boynton.Richard': 0.04325749755208339, 'Brackett.Jos': 0.0319866938808436, 'Bradford.John': 0.04325749755208339, 'Bradlee.David': 0.0820337197368607, 'Bradlee.Josiah': 0.0820337197368607, 'Bradlee.Nathaniel': 0.0820337197368607, 'Bradlee.Thomas': 0.0820337197368607, 'Bray.George': 0.008977228114293228, 'Breck.William': 0.036550282437213824, 'Brimmer.Herman': 0.0319866938808436, 'Brimmer.Martin': 0.0319866938808436, 'Broomfield.Henry': 0.0319866938808436, 'Brown.Enoch': 0.0319866938808436, 'Brown.Hugh': 0.008977228114293228, 'Brown.John': 0.0319866938808436, 'Bruce.Stephen': 0.0820337197368607, 'Burbeck.Edward': 0.008977228114293228, 'Burbeck.William': 0.008977228114293228, 'Burt.Benjamin': 0.036550282437213824, 'Burton.Benjamin': 0.0820337197368607, 'Cailleteau.Edward': 0.008977228114293228, 'Callendar.Elisha': 0.008977228114293228, 'Campbell.Nicholas': 0.0820337197368607, 'Cazneau.Capt': 0.036550282437213824, 'Chadwell.Mr': 0.036550282437213824, 'Champney.Caleb': 0.036550282437213824, 'Chase.Thomas': 0.1545708456338811, 'Cheever.Ezekiel': 0.06853697631805744, 'Chipman.Seth': 0.008977228114293228, 'Chrysty.Thomas': 0.036550282437213824, 'Church.Benjamin': 0.0862709288337976, 'Clarke.Benjamin': 0.0820337197368607, 'Cleverly.Stephen': 0.004000149578963027, 'Cochran.John': 0.0820337197368607, 'Colesworthy.Gilbert': 0.0820337197368607, 'Collier.Gershom': 0.0820337197368607, 'Collins.Ezra': 0.008977228114293228, 'Collson.Adam': 0.11858400217407457, 'Condy.JamesFoster': 0.1505706960549181, 'Cooper.Samuel': 0.08849686858136113, 'Cooper.William': 0.0064631488445004125, 'Crafts.Thomas': 0.012977377693256263, 'Crane.John': 0.0820337197368607, 'Davis.Caleb': 0.04325749755208339, 'Davis.Edward': 0.0319866938808436, 'Davis.Robert': 0.0820337197368607, 'Davis.William': 0.0319866938808436, 'Dawes.Thomas': 0.0064631488445004125, 'Dennie.William': 0.04782108610845363, 'Deshon.Moses': 0.008977228114293228, 'Dexter.Samuel': 0.0064631488445004125, 'Dolbear.Edward': 0.0820337197368607, 'Doyle.Peter': 0.008977228114293228, 'Eaton.Joseph': 0.0820337197368607, 'Eayres.Joseph': 0.11402041361770426, 'Eckley.Unknown': 0.0820337197368607, 'Edes.Benjamin': 0.04055043201617686, 'Emmes.Samuel': 0.036550282437213824, 'Etheridge.William': 0.0820337197368607, 'Fenno.Samuel': 0.0820337197368607, 'Ferrell.Ambrose': 0.008977228114293228, 'Field.Joseph': 0.004000149578963027, 'Flagg.Josiah': 0.008977228114293228, 'Fleet.Thomas': 0.0064631488445004125, 'Foster.Bos': 0.0319866938808436, 'Foster.Samuel': 0.0820337197368607, 'Frothingham.Nathaniel': 0.0820337197368607, 'Gammell.John': 0.0820337197368607, 'Gill.Moses': 0.0319866938808436, 'Gore.Samuel': 0.0820337197368607, 'Gould.William': 0.008977228114293228, 'Graham.James': 0.008977228114293228, 'Grant.Moses': 0.1505706960549181, 'Gray.Wait': 0.008977228114293228, 'Greene.Nathaniel': 0.0820337197368607, 'Greenleaf.Joseph': 0.0798077799892972, 'Greenleaf.William': 0.04325749755208339, 'Greenough.Newn': 0.0319866938808436, 'Ham.William': 0.008977228114293228, 'Hammond.Samuel': 0.0820337197368607, 'Hancock.Eben': 0.0319866938808436, 'Hancock.John': 0.03844984272534402, 'Hendley.William': 0.0820337197368607, 'Hewes.George': 0.0820337197368607, 'Hickling.William': 0.036550282437213824, 'Hicks.John': 0.0820337197368607, 'Hill.Alexander': 0.011270803671239782, 'Hitchborn.Nathaniel': 0.008977228114293228, 'Hitchborn.Thomas': 0.036550282437213824, 'Hobbs.Samuel': 0.0820337197368607, 'Hoffins.John': 0.008977228114293228, 'Holmes.Nathaniel': 0.036550282437213824, 'Hooton.John': 0.0820337197368607, 'Hopkins.Caleb': 0.0319866938808436, 'Hoskins.William': 0.036550282437213824, 'Howard.Samuel': 0.0820337197368607, 'Howe.Edward': 0.0820337197368607, 'Hunnewell.Jonathan': 0.0820337197368607, 'Hunnewell.Richard': 0.0820337197368607, 'Hunstable.Thomas': 0.0820337197368607, 'Hunt.Abraham': 0.0820337197368607, 'Ingersoll.Daniel': 0.0820337197368607, 'Inglish.Alexander': 0.008977228114293228, 'Isaac.Pierce': 0.0319866938808436, 'Ivers.James': 0.0319866938808436, 'Jarvis.Charles': 0.0319866938808436, 'Jarvis.Edward': 0.008977228114293228, 'Jefferds.Unknown': 0.008977228114293228, 'Jenkins.John': 0.008977228114293228, 'Johnston.Eben': 0.0319866938808436, 'Johonnott.Gabriel': 0.036550282437213824, 'Kent.Benjamin': 0.036550282437213824, 'Kerr.Walter': 0.008977228114293228, 'Kimball.Thomas': 0.036550282437213824, 'Kinnison.David': 0.0820337197368607, 'Lambert.John': 0.0064631488445004125, 'Lee.Joseph': 0.0820337197368607, 'Lewis.Phillip': 0.008977228114293228, 'Lincoln.Amos': 0.0820337197368607, 'Loring.Matthew': 0.0820337197368607, 'Lowell.John': 0.036550282437213824, 'MacKintosh.Capt': 0.0820337197368607, 'MacNeil.Archibald': 0.0820337197368607, 'Machin.Thomas': 0.0820337197368607, 'Mackay.William': 0.011270803671239782, 'Marett.Phillip': 0.008977228114293228, 'Marlton.John': 0.008977228114293228, 'Marshall.Thomas': 0.0319866938808436, 'Marson.John': 0.0319866938808436, 'Mason.Jonathan': 0.0319866938808436, 'Matchett.John': 0.036550282437213824, 'May.John': 0.0820337197368607, 'McAlpine.William': 0.008977228114293228, 'Melville.Thomas': 0.0820337197368607, 'Merrit.John': 0.036550282437213824, 'Milliken.Thomas': 0.008977228114293228, 'Molineux.William': 0.1298548058453144, 'Moody.Samuel': 0.008977228114293228, 'Moore.Thomas': 0.0820337197368607, 'Morse.Anthony': 0.0820337197368607, 'Morton.Perez': 0.036550282437213824, 'Mountford.Joseph': 0.0820337197368607, 'Newell.Eliphelet': 0.0820337197368607, 'Nicholls.Unknown': 0.008977228114293228, 'Noyces.Nat': 0.0319866938808436, 'Obear.Israel': 0.008977228114293228, 'Otis.James': 0.017733952515740194, 'Palfrey.William': 0.008977228114293228, 'Palmer.Joseph': 0.0820337197368607, 'Palms.Richard': 0.036550282437213824, 'Parker.Jonathan': 0.0820337197368607, 'Parkman.Elias': 0.06853697631805744, 'Partridge.Sam': 0.0319866938808436, 'Payson.Joseph': 0.0820337197368607, 'Pearce.Isaac': 0.036550282437213824, 'Pearce.IsaacJun': 0.036550282437213824, 'Peck.Samuel': 0.09101094785115389, 'Peck.Thomas': 0.036550282437213824, 'Peters.John': 0.0820337197368607, 'Phillips.John': 0.008977228114293228, 'Phillips.Samuel': 0.0064631488445004125, 'Phillips.William': 0.0319866938808436, 'Pierce.William': 0.0820337197368607, 'Pierpont.Robert': 0.011270803671239782, 'Pitts.John': 0.0319866938808436, 'Pitts.Lendall': 0.0820337197368607, 'Pitts.Samuel': 0.0319866938808436, 'Porter.Thomas': 0.0820337197368607, 'Potter.Edward': 0.008977228114293228, 'Powell.William': 0.04325749755208339, 'Prentiss.Henry': 0.0820337197368607, 'Prince.Job': 0.0319866938808436, 'Prince.John': 0.0820337197368607, 'Proctor.Edward': 0.1505706960549181, 'Pulling.John': 0.06853697631805744, 'Pulling.Richard': 0.008977228114293228, 'Purkitt.Henry': 0.0820337197368607, 'Quincy.Josiah': 0.017733952515740194, 'Randall.John': 0.0820337197368607, 'Revere.Paul': 0.16601107301371168, 'Roby.Joseph': 0.0820337197368607, 'Roylson.Thomas': 0.0319866938808436, 'Ruddock.Abiel': 0.06853697631805744, 'Russell.John': 0.0820337197368607, 'Russell.William': 0.0820337197368607, 'Sessions.Robert': 0.0820337197368607, 'Seward.James': 0.008977228114293228, 'Sharp.Gibbens': 0.036550282437213824, 'Shed.Joseph': 0.0820337197368607, 'Sigourney.John': 0.036550282437213824, 'Simpson.Benjamin': 0.0820337197368607, 'Slater.Peter': 0.0820337197368607, 'Sloper.Ambrose': 0.008977228114293228, 'Smith.John': 0.004000149578963027, 'Spear.Thomas': 0.0820337197368607, 'Sprague.Samuel': 0.0820337197368607, 'Spurr.John': 0.0820337197368607, 'Stanbridge.Henry': 0.008977228114293228, 'Starr.James': 0.0820337197368607, 'Stearns.Phineas': 0.0820337197368607, 'Stevens.Ebenezer': 0.0820337197368607, 'Stoddard.Asa': 0.036550282437213824, 'Stoddard.Jonathan': 0.036550282437213824, 'Story.Elisha': 0.11858400217407457, 'Swan.James': 0.11858400217407457, 'Sweetser.John': 0.011270803671239782, 'Symmes.Eben': 0.036550282437213824, 'Symmes.John': 0.036550282437213824, 'Tabor.Philip': 0.008977228114293228, 'Tileston.Thomas': 0.036550282437213824, 'Trott.George': 0.004000149578963027, 'Tyler.Royall': 0.0064631488445004125, 'Urann.Thomas': 0.12756123028836766, 'Vernon.Fortesque': 0.0319866938808436, 'Waldo.Benjamin': 0.0319866938808436, 'Warren.Joseph': 0.09524815694809075, 'Webb.Joseph': 0.008977228114293228, 'Webster.Thomas': 0.008977228114293228, 'Welles.Henry': 0.012977377693256263, 'Wendell.Oliver': 0.04325749755208339, 'Wheeler.Josiah': 0.0820337197368607, 'White.Samuel': 0.036550282437213824, 'Whitten.John': 0.008977228114293228, 'Whitwell.Samuel': 0.0319866938808436, 'Whitwell.William': 0.0319866938808436, 'Williams.Jeremiah': 0.0820337197368607, 'Williams.Jonathan': 0.0319866938808436, 'Williams.Thomas': 0.0820337197368607, 'Willis.Nathaniel': 0.0820337197368607, 'Wingfield.William': 0.008977228114293228, 'Winslow.John': 0.0064631488445004125, 'Winthrop.John': 0.06853697631805744, 'Wyeth.Joshua': 0.0820337197368607, 'Young.Thomas': 0.1298548058453144}
argh, need to sort that
sorted(person_adjacency_graph_centrality, key=person_adjacency_graph_centrality.get, reverse=True)[:10] #useful syntax to know for sorting dictionaries!
['Revere.Paul', 'Barber.Nathaniel', 'Chase.Thomas', 'Bass.Henry', 'Condy.JamesFoster', 'Proctor.Edward', 'Grant.Moses', 'Young.Thomas', 'Molineux.William', 'Urann.Thomas']
for name in sorted(person_adjacency_graph_centrality, key=person_adjacency_graph_centrality.get, reverse=True)[0:10]:
print(name, person_adjacency_graph_centrality[name])
('Revere.Paul', 0.16601107301371168) ('Barber.Nathaniel', 0.1618414997261579) ('Chase.Thomas', 0.1545708456338811) ('Bass.Henry', 0.1545708456338811) ('Condy.JamesFoster', 0.1505706960549181) ('Proctor.Edward', 0.1505706960549181) ('Grant.Moses', 0.1505706960549181) ('Young.Thomas', 0.1298548058453144) ('Molineux.William', 0.1298548058453144) ('Urann.Thomas', 0.12756123028836766)
Point isn't that such techniques are always right:
they do uncover relationship
they have tons of false positives
Transparency Toolkit https://transparencytoolkit.org/project/icwatch/
slurp up 27k linked-in profiles for US intelligence agency current or former personel
import requests
r=requests.get('https://raw.githubusercontent.com/TransparencyToolkit/ICWATCH-Data/master/data/original_run/xkeyscore.json')
results=r.json()
len(results)
135
results[0]
{u'address': u'200 Stovall St. Alexandria, VA 22332 United States', u'area': u'Georgia Area', u'certifications': [], u'company': u'Department of the Army', u'company_size': u'10,001+ employees', u'current': u'Yes', u'degree': 0, u'description': None, u'education': [{u'description': u'', u'name': u'American Public University System', u'period': u'2009 \u2013 2011'}, {u'description': u'AS, Intelligence Operations with Honors', u'name': u'Cochise College', u'period': u'2007 \u2013 2008'}], u'end_date': u'2015-04-13', u'founded': u'1775', u'groups': [{u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=37736&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'United States Army Airborne School'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=74308&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Iraq War Veterans'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=154521&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'DOD CLEARANCE GROUP'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=95059&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'US Government Security Clearance'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=5082195&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'SOT-A Association'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=756367&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Bronze Star Medal Recipients Association'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=1271127&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'The Intelligence Community'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=80797&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Top Secret Candidates'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=3264671&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Celestar Military Intelligence & Operations Jobs'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=80088&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'U.S. GOVERNMENT CONNECTIONS'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=2201592&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'AMU & APU Student Veterans of America'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=6400282&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Global B2B Information and Technology'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=81616&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'US Army Intelligence Alumni'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=72244&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Intelligence Professionals'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=2488181&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'AMU & APU Intelligence Studies Program'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=3131037&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'INTELLIGENCE & SECURITY'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=4117560&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Afghanistan War Veterans'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=1751357&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'ClearedJobs.Net - Active Security Clearance required - for Cleared Professionals'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=41210&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'US Army'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=4523766&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'3rd Special Forces Group (Past and Present)'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=3898097&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'AMU & APU Alumni'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=2092950&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Intelligence Analysts'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=945597&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Augusta Professional Networking'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=156213&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Federal Government Contractor Network (TFCN)'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=85499&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'American Public University System'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=100364&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Department of Defense'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=5167129&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Global B2B Networks'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=47803&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Army Veterans'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=81773&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'DEFENSE AND AEROSPACE CONNECTIONS'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=4717381&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'LearnSmart: User Network'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=76373&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Google Android'}, {u'link': u'http://www.linkedin.comhttp://www.linkedin.com/groups?gid=1765567&goback=%2Enpp_robert*5baker%2F11%2F388%2F303&trk=fulpro_grplogo', u'name': u'Cyber Intelligence Network'}], u'headquarters': u'200 Stovall St.\n\nAlexandria,\nVA\n22332\nUnited States', u'industry': u'Computer Networking', u'languages': [], u'linkedin_company_url': u'http://www.linkedin.com/company/1223?trk=ppro_cprof', u'location': u'Augusta', u'name': u'Robert Baker', u'organizations': [], u'pic_path': u'public/uploads/pictures/ghost_person_150x150_v1.png', u'picture': u'https://static.licdn.com/scds/common/u/images/themes/katy/ghosts/person/ghost_person_150x150_v1.png', u'profile_url': u'http://www.linkedin.com/pub/robert-baker/11/388/303', u'related_people': [{u'name': u'Marion Carlton', u'url': u'https://www.linkedin.com/pub/marion-carlton/22/190/454?trk=pub-pbmap'}, {u'name': u'John Doran, PMP', u'url': u'https://www.linkedin.com/pub/john-doran-pmp/9/709/3ba?trk=pub-pbmap'}, {u'name': u'Justin Thompson', u'url': u'https://www.linkedin.com/pub/justin-thompson/26/42a/b91?trk=pub-pbmap'}, {u'name': u'Victoria Bouldin, MBA', u'url': u'https://www.linkedin.com/pub/victoria-bouldin-mba/55/171/b22?trk=pub-pbmap'}, {u'name': u'Kenneth Caligiuri', u'url': u'https://www.linkedin.com/pub/kenneth-caligiuri/42/431/6b4?trk=pub-pbmap'}, {u'name': u'Karah Christiansen', u'url': u'https://www.linkedin.com/pub/karah-christiansen/53/990/873?trk=pub-pbmap'}, {u'name': u'Tracey Moriarty', u'url': u'https://www.linkedin.com/pub/tracey-moriarty/1/190/333?trk=pub-pbmap'}, {u'name': u'Jeff Kissler', u'url': u'https://www.linkedin.com/pub/jeff-kissler/3b/783/ab0?trk=pub-pbmap'}, {u'name': u'Chip London', u'url': u'https://www.linkedin.com/pub/chip-london/7/293/727?trk=pub-pbmap'}, {u'name': u'Kate Hutson', u'url': u'https://www.linkedin.com/pub/kate-hutson/7/414/11b?trk=pub-pbmap'}], u'score': 0, u'skills': [u'Intelligence Analysis', u'Collections Management', u'Imagery', u'Analysis', u'Geospatial', u'SIGINT', u'Counterintelligence', u'GIS', u'Remote Sensing', u'Strategic Intelligence', u'Geospatial Technologies', u'ArcGIS', u'Microsoft Office', u'Word', u'Security Clearance', u'Geomatics', u'Microsoft Word', u'Museum Collections', u'Intelligence', u'DoD', u'Defense', u'Information Assurance', u'National Security', u'Army', u'Special Operations', u'Force Protection', u'Counterterrorism', u'Top Secret', u'Briefing', u'Intelligence Community', u'C4ISR', u'Government Contracting', u'Military Experience', u'Military Operations'], u'start_date': u'2013-02-01', u'timestamp': u'2015-04-13 14:49:32 +0200', u'title': u'DoD Analyst', u'type': u'Government Agency', u'url': u'http://www.goarmy.com', u'website': u'http://www.goarmy.com', u'websites': []}
xkeyscore_friends=nx.Graph()
[name["name"] for name in results[0]['related_people']]
[u'Marion Carlton', u'John Doran, PMP', u'Justin Thompson', u'Victoria Bouldin, MBA', u'Kenneth Caligiuri', u'Karah Christiansen', u'Tracey Moriarty', u'Jeff Kissler', u'Chip London', u'Kate Hutson']
connections=[name["name"] for name in results[0]['related_people']]
len(connections)
10
results[0]["name"]
u'Robert Baker'
[results[0]["name"]]*len(connections)
[u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker', u'Robert Baker']
original=[results[0]['name']]*len(connections)
xkeyscore_friends.add_edges_from(zip(original,connections))
nx.draw(xkeyscore_friends)
xkeyscore_friends.edges()
[(u'Karah Christiansen', u'Robert Baker'), (u'Justin Thompson', u'Robert Baker'), (u'Jeff Kissler', u'Robert Baker'), (u'Marion Carlton', u'Robert Baker'), (u'Tracey Moriarty', u'Robert Baker'), (u'Robert Baker', u'Chip London'), (u'Robert Baker', u'Victoria Bouldin, MBA'), (u'Robert Baker', u'John Doran, PMP'), (u'Robert Baker', u'Kenneth Caligiuri'), (u'Robert Baker', u'Kate Hutson')]
edges_to_add=[]
for entry in results:
connections=[relation["name"] for relation in entry["related_people"]]
number_connections=len(connections)
original_person=[entry["name"]]*number_connections
edges_to_add.extend(zip(original_person,connections))
len(edges_to_add)
1122
edges_to_add[:25]
[(u'Robert Baker', u'Marion Carlton'), (u'Robert Baker', u'John Doran, PMP'), (u'Robert Baker', u'Justin Thompson'), (u'Robert Baker', u'Victoria Bouldin, MBA'), (u'Robert Baker', u'Kenneth Caligiuri'), (u'Robert Baker', u'Karah Christiansen'), (u'Robert Baker', u'Tracey Moriarty'), (u'Robert Baker', u'Jeff Kissler'), (u'Robert Baker', u'Chip London'), (u'Robert Baker', u'Kate Hutson'), (u'Robert Baker', u'Marion Carlton'), (u'Robert Baker', u'John Doran, PMP'), (u'Robert Baker', u'Justin Thompson'), (u'Robert Baker', u'Victoria Bouldin, MBA'), (u'Robert Baker', u'Kenneth Caligiuri'), (u'Robert Baker', u'Karah Christiansen'), (u'Robert Baker', u'Tracey Moriarty'), (u'Robert Baker', u'Jeff Kissler'), (u'Robert Baker', u'Chip London'), (u'Robert Baker', u'Kate Hutson'), (u'Robert Baker', u'Marion Carlton'), (u'Robert Baker', u'John Doran, PMP'), (u'Robert Baker', u'Justin Thompson'), (u'Robert Baker', u'Victoria Bouldin, MBA'), (u'Robert Baker', u'Kenneth Caligiuri')]
xkeyscore_friends_graph=nx.Graph()
xkeyscore_friends_graph.add_edges_from(edges_to_add)
nx.draw(xkeyscore_friends_graph)
pos=nx.spring_layout(xkeyscore_friends_graph, iterations=30)
plt.figure(figsize=(40,40))
nx.draw_networkx_edges(xkeyscore_friends_graph, pos, width=.1)
nx.draw_networkx_nodes(xkeyscore_friends_graph, pos, node_size=50)
nx.draw_networkx_labels(xkeyscore_friends_graph, pos)
plt.axis('off')
plt.show()
Networkx and python good at the manipulation of things into network form, but clunky on visualization and some analysis.
Easy to export in numerous formats, such as json, which is good for use in D3.
import json
from networkx.readwrite import json_graph
Json needs the graph to be converted into a "node-link" format, which comprises
Helper function saves you the trouble of making it.
club_adjacency_graph_node_link = json_graph.node_link_data(club_adjacency_graph)
json.dump(club_adjacency_graph_node_link, open('revere_adjacency.json','w'))
!head revere_adjacency.json
{"directed": false, "graph": [["name", "()"]], "nodes": [{"id": "LondonEnemies"}, {"id": "LoyalNine"}, {"id": "StAndrewsLodge"}, {"id": "LongRoomClub"}, {"id": "BostonCommittee"}, {"id": "TeaParty"}, {"id": "NorthCaucus"}], "links": [{"source": 0, "target": 0, "weight": 62}, {"source": 0, "target": 1, "weight": 3}, {"source": 0, "target": 2, "weight": 3}, {"source": 0, "target": 3, "weight": 5}, {"source": 0, "target": 4, "weight": 11}, {"source": 0, "target": 5, "weight": 8}, {"source": 0, "target": 6, "weight": 16}, {"source": 1, "target": 2, "weight": 2}, {"source": 1, "target": 6, "weight": 3}, {"source": 1, "target": 5, "weight": 2}, {"source": 1, "target": 1, "weight": 10}, {"source": 2, "target": 2, "weight": 53}, {"source": 2, "target": 3, "weight": 2}, {"source": 2, "target": 4, "weight": 1}, {"source": 2, "target": 5, "weight": 3}, {"source": 2, "target": 6, "weight": 3}, {"source": 3, "target": 3, "weight": 17}, {"source": 3, "target": 4, "weight": 5}, {"source": 3, "target": 5, "weight": 2}, {"source": 3, "target": 6, "weight": 5}, {"source": 4, "target": 4, "weight": 21}, {"source": 4, "target": 5, "weight": 3}, {"source": 4, "target": 6, "weight": 9}, {"source": 5, "target": 5, "weight": 97}, {"source": 5, "target": 6, "weight": 13}, {"source": 6, "target": 6, "weight": 59}], "multigraph": false}
Much social network analysis software needs the graphml format.
it's as easy as
nx.write_graphml(G,'so.graphml')
You can import this into the powerful open-source package gephi
. http://gephi.org/
Performative claim:
"Network metaphors and methods are being built in to social practices in ways that at once provide
new, theoretically-informed tools for social actors, and notionally “raw”
data for social network analysts"
Kieran Healy (Duke),
http://kieranhealy.org/files/papers/performativity.pdf