#!/usr/bin/env python
# coding: utf-8

# # Running Neurokernel on NVIDIA Jetson Embedded Platform

# In this notebook, we show step by step how to run Neurokernel on [Jetson TK1 Embedded Development Kit](http://www.nvidia.com/object/jetson-tk1-embedded-dev-kit.html). It can be applied to the latest [Jetson TX1 platform](http://www.nvidia.com/object/jetson-tx1-dev-kit.html) with minor modification. Before we start, you must have a Jetson TK1 Embedded Development Kit and have enrolled in [NVIDIA Embedded Developer Program](https://developer.nvidia.com/embedded-developer-program).
# To install Jetson Development Pack (JetPack), you also need access to a Ubuntu OS. Follow [these steps](http://docs.nvidia.com/jetpack-l4t/2_0/index.html#developertools/mobile/jetpack/jetpack_l4t/2.0/jetpack_l4t_install.htm) to install JetPack. This notebook assumes that JetPack is already installed on your TK1 Kit.

# <img src='files/files/tk1.png' />

# ## Installing Neurokernel on Jetson TK1

# Please execute the following commands in terminal on the TK1. Note that executing command at a time is preferred to ensure complete installation.

# We first maximize CPU performance to make running the installlation faster. For the 4 main CPU cores to always run at max performance until reboot:

# In[ ]:


sudo su
echo 0 > /sys/devices/system/cpu/cpuquiet/tegra_cpuquiet/enable
echo 1 > /sys/devices/system/cpu/cpu0/online
echo 1 > /sys/devices/system/cpu/cpu1/online
echo 1 > /sys/devices/system/cpu/cpu2/online
echo 1 > /sys/devices/system/cpu/cpu3/online
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
exit


# Update system and install necessary libraries:

# In[ ]:


sudo apt-mark hold xserver-xorg-core
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bash-completion command-not-found
sudo apt-get install emacs24 libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev 
sudo apt-get install libgdbm-dev libc6-dev libbz2-dev liblapack-dev libhdf5-dev libxml2-dev libxslt1-dev
sudo apt-get install python-dev tmux gfortran git


# Install Latest Python:

# In[ ]:


mkdir packages
cd packages
wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz
tar -xvf Python-2.7.11.tgz
cd Python-2.7.11
./configure --prefix=/home/ubuntu/opt
make
make install


# Add path:

# In[ ]:


cd

echo -e "export PATH=/home/ubuntu/opt/bin:\$PATH\n"\
"export LD_LIBRARY_PATH=/home/ubuntu/opt/lib:\$LD_LIBRARY_PATH\n"\
"export PYTHONPATH=/home/ubuntu/opt/lib/python2.7/site-packages:\$PYTHONPATH" | tee -a ~/.bashrc

source .bashrc


# Install Open-mpi:

# In[ ]:


cd packages
wget http://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.1.tar.gz
tar -xvf openmpi-1.10.1.tar.gz
cd openmpi-1.10.1

./configure --with-cuda=/usr/local/cuda --with-threads=posix\
  --disable-mca-dso --prefix=/home/ubuntu/opt

make -j 4
make install


# Install virtualenv:

# In[ ]:


cd ~/packages
wget https://pypi.python.org/packages/source/v/virtualenv/virtualenv-13.1.2.tar.gz
tar -xvf virtualenv-13.1.2.tar.gz
cd virtualenv-13.1.2
python setup.py install

cd 
virtualenv NK
source NK/bin/activate


# Install required Python packages

# In[ ]:


pip install numpy scipy ipython cython numexpr pycuda
pip install bidict pandas networkx mpi4py h5py dill lxml markupsafe shutilwhich ply psutil futures twiggy matplotlib
pip install sphinx sphinx_rtd_theme


# This step will take a lot of time. If some installation fails with error message: No matching distribution found, you can install it manually by replacing link_to_package with the url for the package in the following command:

# In[ ]:


pip install -Iv link_to_package


# Install Neurokernel:

# In[ ]:


cd
git clone https://github.com/neurokernel/neurokernel.git
cd neurokernel
git pull
python setup.py develop


# Test Neurokernel by running one of the examples

# In[ ]:


cd ~/neurokernel/examples/intro/data
python gen_generic_lpu.py -s 0 -l lpu_0 generic_lpu_0.gexf.gz generic_lpu_0_input.h5                
python gen_generic_lpu.py -s 1 -l lpu_1 generic_lpu_1.gexf.gz generic_lpu_1_input.h5
cd ../
python intro_demo.py --gpu_dev 0 0 --log file


# You can ignore warning messages like these:

#     ---------------------------------------------------------
#     The call to cuMemHostRegister failed.
#       Host:  tegra-ubuntu
#       cuMemHostRegister return value:  801
#       Memory Pool:  smcuda
#     ---------------------------------------------------------
#     ---------------------------------------------------------
#     Sorry!  You were supposed to get help about:
#         cuMemHostRegister during init failed
#     from the file:
#         help-mpi-common-cuda.txt
#     But I couldn't find that topic in the file.  Sorry!
#     ---------------------------------------------------------
# 
# or
# 
#     983 more processes have sent help message help-mpi-common-cuda.txt / cuIpcGetMemHandle failed

# This is caused by the fact that cudaHostRegister is not supported in armv7 devices.
# Inspect 'neurokernel.log' after the example finishes execution. If there is no error in the log file, the installation is complete.