Visualizing of genetic similarity with Lightning + GraphX

Setup lightning

In [1]:
%libraryDependencies += "org.viz.lightning" %% "lightning-scala" % "0.1.6"

In [2]:
%update

In [3]:
import org.viz.lightning._
import org.apache.spark.graphx._

In [4]:
val lgn = Lightning(host="https://lightning-spark-summit.herokuapp.com" )
lgn.enableNotebook()

Load structure similarity data

Public data from http://www.brain-map.org/

In [5]:
val source = "/Users/mathisonian/projects/spark-summit/notebooks/data/allen-connectivity.txt"
val g = GraphLoader.edgeListFile(sc, source)

Show the network (unlabeled)

In [6]:
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))

lgn.force(links)

Show the network colored by degree

In [7]:
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))
val degrees = g.degrees.sortBy(_._1).collect().map(x => Math.log(x._2))

lgn.force(links, value=degrees, colormap="Lightning")

Show the network colored by connected components

In [8]:
val links = g.edges.collect().map(e => Array(e.srcId.toInt, e.dstId.toInt))
val connectedComponents = g.connectedComponents().vertices.sortBy(_._1).map(_._2.toInt).collect()

lgn.force(links, label=connectedComponents)