Why does it open notebooks in the same tab? How do I get back to the home directory?
Workaround is to right click on notebooks and open in a new tab, but it's very annoying.
When I create a new notebook it seems to open ok in a new tab.
Also, WHY IS IT SO SLOW?
!pip freeze
AGILe==1.4.1
astroid==1.3.6
backports.lzma==0.0.6
backports.shutil-get-terminal-size==1.0.0
backports.ssl-match-hostname==3.4.0.2
bleach==2.0.0
certifi==14.5.14
cmmnbuild-dep-manager==2.1.2
configparser==3.5.0
CouchDB==1.0.1
coverage==4.4.1
cx-Oracle==5.1.1
cycler==0.10.0
Cython==0.23.4
decorator==4.0.9
dill==0.2.5
Django==1.9.12
elasticsearch==2.3.0
entrypoints==0.2.2
funcsigs==1.0.2
future==0.15.2
Genshi==0.7
glite-wms-wmproxy-api-python==3.4.0
gosam===2.0.4-12f4de9
grpcio==1.2.0
h5py==2.6.0
hepdata-converter==0.1.22
hepdata-validator==0.1.8
hspy==1.6.4
html5lib==0.999999999
ipykernel==4.6.1
ipython==5.4.1
ipython-genutils==0.1.0
ipywidgets==5.2.2
Jinja2==2.7.3
joblib==0.10.0
JPype1==0.6.2
jsonschema==2.4.0
jupyter==1.0.0
jupyter-client==4.3.0
jupyter-console==5.0.0
jupyter-contrib-core==0.3.0
jupyter-contrib-nbextensions==0.2.1
jupyter-core==4.2.0
jupyter-nbextensions-configurator==0.2.2
Keras==1.1.0
LHAPDF==6.2.0
logilab-common==1.0.1
lxml==2.3
M2Crypto==0.25.1
MarkupSafe==0.23
matplotlib==1.5.1
messaging==1.0
metakernel==0.13.1
mistune==0.5.1
mock==2.0.0
multiprocess==0.70.4
nbconvert==5.1.1
nbformat==4.0.1
networkx==1.11
nose==1.1.2
notebook==4.2.1
numexpr==2.4.3
numpy==1.11.0
pandas==0.23.4
pandocfilters==1.4.1
pathlib2==2.1.0
pathos==0.2.0
patsy==0.4.1
pbr==2.0.0
pexpect==4.2.0
pickleshare==0.7.2
pox==0.2.2
ppft==1.6.4.6
prettytable==0.7.2
prompt-toolkit==1.0.3
protobuf==3.2.0
psutil==4.3.1
ptyprocess==0.5.1
py==1.4.8
py2neo==1.4.6
py4j==0.10.3
pydot==1.2.3
pydot-ng==1.0.0
Pygments==2.0.2
PyHEADTAIL==1.12.1
pyjapc==1.4.1
pylint==1.4.4
pyparsing==2.1.8
pystan==2.16.0.0
pytest==2.2.4
python-dateutil==2.8.0
python-gitlab==0.20
pytimber==2.4.2
pytz==2015.4
PyYAML==3.11
pyzmq==14.5.0
qmtest==2.4.1
qtconsole==4.2.1
requests==2.10.0
rivet==2.5.4
root-numpy==4.6.0
rootpy==0.8.1
rpy2==2.8.0
scikit-learn==0.18.1
scipy==0.18.1
seaborn==0.7.1
simplegeneric==0.8.1
simplejson==3.8.2
singledispatch==3.4.0.3
six==1.10.0
SQLAlchemy==0.7.7
statsmodels==0.8.0
stomp.py==3.1.3
sympy==0.7.6
tensorflow==1.0.0
terminado==0.5
testpath==0.3.1
Theano==0.8.2
tornado==4.0.2
traitlets==4.2.2
troveharvester==0.1.13
typing==3.5.2.2
unicodecsv==0.14.1
urllib3==1.14
wcwidth==0.1.7
webencodings==0.5.1
widgetsnbextension==1.2.6
yoda==1.6.7
You are using pip version 8.1.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Ok, so it looks like all the SciPy type packages are there as you would expect — including Keras and Tensorflow. But no image processing, no BeautifulSoup (but lxml is there), no Altair/Vega for visualisations...
I can't access a terminal, but pip works. Note the --user
to install into my own account.
Note that these additions persist between sessions, ie. no need to pip
install them more than once. Of course that also means that you can't have different versions, as there's doesn't seem to be a way to have virtual environments.
!pip install --user troveharvester
Collecting troveharvester
Requirement already satisfied (use --upgrade to upgrade): requests in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from troveharvester)
Collecting unicodecsv (from troveharvester)
Installing collected packages: unicodecsv, troveharvester
Successfully installed troveharvester unicodecsv
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
# The two dashes '--' are necessary to identify the arguments
%run -m troveharvester -- -h
usage: troveharvester [-h] {start,restart,report} ... positional arguments: {start,restart,report} start Start a new harvest restart Restart an unfinished harvest report Report on a harvest optional arguments: -h, --help show this help message and exit
Again, no terminal access, but you seem to able to use shell commands ok.
!git
usage: git [--version] [--exec-path[=GIT_EXEC_PATH]] [--html-path] [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] [--git-dir=GIT_DIR] [--work-tree=GIT_WORK_TREE] [--help] COMMAND [ARGS] The most commonly used git commands are: add Add file contents to the index bisect Find by binary search the change that introduced a bug branch List, create, or delete branches checkout Checkout a branch or paths to the working tree clone Clone a repository into a new directory commit Record changes to the repository diff Show changes between commits, commit and working tree, etc fetch Download objects and refs from another repository grep Print lines matching a pattern init Create an empty git repository or reinitialize an existing one log Show commit logs merge Join two or more development histories together mv Move or rename a file, a directory, or a symlink pull Fetch from and merge with another repository or a local branch push Update remote refs along with associated objects rebase Forward-port local commits to the updated upstream head reset Reset current HEAD to the specified state rm Remove files from the working tree and from the index show Show various types of objects status Show the working tree status tag Create, list, delete or verify a tag object signed with GPG See 'git help COMMAND' for more information on a specific command.
Git cloning a GitHub repository seems to work as expected.
!git clone https://github.com/GLAM-Workbench/trove-newspaper-harvester.git
error: could not commit config file /webdav/trove-newspaper-harvester/.git/config Initialized empty Git repository in /webdav/trove-newspaper-harvester/.git/ error: could not commit config file /webdav/trove-newspaper-harvester/.git/config error: could not commit config file /webdav/trove-newspaper-harvester/.git/config remote: Enumerating objects: 22, done. remote: Counting objects: 100% (22/22), done. remote: Compressing objects: 100% (15/15), done. remote: Total 34 (delta 10), reused 17 (delta 7), pack-reused 12 Unpacking objects: 100% (34/34), done.
requirements.txt
¶So a typical thing I'd want to do is clone a repository and then install all the necessary packages from a requirements.txt
file.
cd trove-newspaper-harvester
/webdav/trove-newspaper-harvester
!pip install --user -r requirements.txt
Collecting altair==2.2.2 (from -r requirements.txt (line 1)) Downloading https://files.pythonhosted.org/packages/a4/02/d0ad006918024f8c1509a7f76e94e5e3bdb102ed58da160d3dcfb621fc33/altair-2.2.2-py2.py3-none-any.whl (387kB) 100% |████████████████████████████████| 389kB 2.7MB/s eta 0:00:01 Collecting ipyleaflet==0.9.0 (from -r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/c4/fe/01839b1cdcc4bf8c8424f3f1ddf50f9e4fbee2cb33b52645a4d2be28432c/ipyleaflet-0.9.0-py2.py3-none-any.whl (906kB) 100% |████████████████████████████████| 911kB 1.2MB/s eta 0:00:01 Collecting pandas==0.23.4 (from -r requirements.txt (line 3)) Downloading https://files.pythonhosted.org/packages/5d/d4/6e9c56a561f1d27407bf29318ca43f36ccaa289271b805a30034eb3a8ec4/pandas-0.23.4-cp35-cp35m-manylinux1_x86_64.whl (8.7MB) 100% |████████████████████████████████| 8.7MB 168kB/s eta 0:00:01 Collecting textblob==0.15.1 (from -r requirements.txt (line 4)) Downloading https://files.pythonhosted.org/packages/11/18/7f55c8be6d68ddc4036ffda5382ca51e23a1075987f708b9123712091af1/textblob-0.15.1-py2.py3-none-any.whl (631kB) 100% |████████████████████████████████| 634kB 1.8MB/s eta 0:00:01 Collecting troveharvester==0.2.2 (from -r requirements.txt (line 5)) Downloading https://files.pythonhosted.org/packages/00/12/57284df3a8555ce2e227058017a631b6ea142be698a92e32a58bdb3a299a/troveharvester-0.2.2-py3-none-any.whl Collecting vega==1.4.0 (from -r requirements.txt (line 6)) Downloading https://files.pythonhosted.org/packages/0a/ce/1ab1bf943f9b80f9e87d807a29337daff93cb64c7080b37633f4655b8c21/vega-1.4.0.tar.gz (1.1MB) 100% |████████████████████████████████| 1.1MB 1.1MB/s eta 0:00:01 Collecting wordcloud==1.5.0 (from -r requirements.txt (line 7)) Downloading https://files.pythonhosted.org/packages/5e/b7/c16286efa3d442d6983b3842f982502c00306c1a4c719c41fb00d6017c77/wordcloud-1.5.0-cp35-cp35m-manylinux1_x86_64.whl (357kB) 100% |████████████████████████████████| 358kB 2.9MB/s eta 0:00:01 Collecting scipy==1.1.0 (from -r requirements.txt (line 8)) Downloading https://files.pythonhosted.org/packages/cd/32/5196b64476bd41d596a8aba43506e2403e019c90e1a3dfc21d51b83db5a6/scipy-1.1.0-cp35-cp35m-manylinux1_x86_64.whl (33.1MB) 100% |████████████████████████████████| 33.1MB 48kB/s eta 0:00:01 Collecting scikit-learn==0.19.2 (from -r requirements.txt (line 9)) Downloading https://files.pythonhosted.org/packages/b6/e2/a1e254a4a4598588d4fe88b45ab88a226c289ecfd0f6c90474eb6a9ea6b3/scikit_learn-0.19.2-cp35-cp35m-manylinux1_x86_64.whl (4.9MB) 100% |████████████████████████████████| 4.9MB 323kB/s eta 0:00:01 Collecting matplotlib==2.2.3 (from -r requirements.txt (line 10)) Downloading https://files.pythonhosted.org/packages/de/af/6258db9b26313dd7ad70dba30a60bec62bf030a44208d4cb62966206666f/matplotlib-2.2.3-cp35-cp35m-manylinux1_x86_64.whl (12.6MB) 100% |████████████████████████████████| 12.6MB 129kB/s eta 0:00:01 50% |████████████████▎ | 6.4MB 52.2MB/s eta 0:00:01 Collecting tqdm==4.29.1 (from -r requirements.txt (line 11)) Downloading https://files.pythonhosted.org/packages/ed/d6/3458d39cf4978f4ece846295e83daf5ece710ab0a4106774f7f7b3a68697/tqdm-4.29.1-py2.py3-none-any.whl (46kB) 100% |████████████████████████████████| 51kB 5.0MB/s eta 0:00:01 Collecting arrow==0.13.0 (from -r requirements.txt (line 12)) Downloading https://files.pythonhosted.org/packages/5d/c7/468bb95a10fb8ddb5f3f80e1aef06b78f64d6e5df958c39672f80581381f/arrow-0.13.0.tar.gz (92kB) 100% |████████████████████████████████| 102kB 6.2MB/s ta 0:00:01 Collecting appmode==0.4.0 (from -r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/9d/c2/544a4cd86892b17f00b0006be323026d3b36e28c0eccabc68c3e9e1fb16b/appmode-0.4.0.tar.gz Requirement already satisfied (use --upgrade to upgrade): six in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from altair==2.2.2->-r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): typing in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from altair==2.2.2->-r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): jsonschema in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from altair==2.2.2->-r requirements.txt (line 1)) Collecting toolz (from altair==2.2.2->-r requirements.txt (line 1)) Downloading https://files.pythonhosted.org/packages/14/d0/a73c15bbeda3d2e7b381a36afb0d9cd770a9f4adc5d1532691013ba881db/toolz-0.9.0.tar.gz (45kB) 100% |████████████████████████████████| 51kB 4.8MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): numpy in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from altair==2.2.2->-r requirements.txt (line 1)) Requirement already satisfied (use --upgrade to upgrade): entrypoints in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from altair==2.2.2->-r requirements.txt (line 1)) Collecting xarray<0.11,>=0.10 (from ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/d2/40/e3d9a6b679a7b4ba80e8cdf0b0226809ef454773f7b79ef1b2bd1898bd48/xarray-0.10.9-py2.py3-none-any.whl (473kB) 100% |████████████████████████████████| 481kB 2.3MB/s eta 0:00:01 Collecting ipywidgets<8,>=7.0.0 (from ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/30/9a/a008c7b1183fac9e52066d80a379b3c64eab535bd9d86cdc29a0b766fd82/ipywidgets-7.4.2-py2.py3-none-any.whl (111kB) 100% |████████████████████████████████| 112kB 5.9MB/s eta 0:00:01 Collecting traittypes<3,>=0.2.1 (from ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/9c/d1/8d5bd662703cc1764d986f6908a608777305946fa634d34c470cd4a1e729/traittypes-0.2.1-py2.py3-none-any.whl Collecting python-dateutil>=2.5.0 (from pandas==0.23.4->-r requirements.txt (line 3)) Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 100% |████████████████████████████████| 235kB 2.9MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): pytz>=2011k in /cvmfs/sft.cern.ch/lcg/releases/pytz/2015.4-e9c1e/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages/pytz-2015.4-py3.5.egg (from pandas==0.23.4->-r requirements.txt (line 3)) Collecting nltk>=3.1 (from textblob==0.15.1->-r requirements.txt (line 4)) Downloading https://files.pythonhosted.org/packages/6f/ed/9c755d357d33bc1931e157f537721efb5b88d2c583fe593cc09603076cc3/nltk-3.4.zip (1.4MB) 100% |████████████████████████████████| 1.4MB 823kB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): unicodecsv in /webdav/.local/lib/python3.5/site-packages (from troveharvester==0.2.2->-r requirements.txt (line 5)) Requirement already satisfied (use --upgrade to upgrade): requests in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from troveharvester==0.2.2->-r requirements.txt (line 5)) Collecting pillow (from wordcloud==1.5.0->-r requirements.txt (line 7)) Downloading https://files.pythonhosted.org/packages/8b/e9/5c47710fe383f0582da668302a80a6355fe15c2ce2dde89b50fe34acefa6/Pillow-5.4.1-cp35-cp35m-manylinux1_x86_64.whl (2.0MB) 100% |████████████████████████████████| 2.0MB 576kB/s eta 0:00:01 Collecting kiwisolver>=1.0.1 (from matplotlib==2.2.3->-r requirements.txt (line 10)) Downloading https://files.pythonhosted.org/packages/7e/31/d6fedd4fb2c94755cd101191e581af30e1650ccce7a35bddb7930fed6574/kiwisolver-1.0.1-cp35-cp35m-manylinux1_x86_64.whl (949kB) 100% |████████████████████████████████| 952kB 1.1MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from matplotlib==2.2.3->-r requirements.txt (line 10)) Requirement already satisfied (use --upgrade to upgrade): cycler>=0.10 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from matplotlib==2.2.3->-r requirements.txt (line 10)) Collecting notebook>=5 (from appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/0a/d8/4e9521354ed3d730ba6d8a5af440b66c73245ef46be706e51bead71afc21/notebook-5.7.6-py2.py3-none-any.whl (9.0MB) 100% |████████████████████████████████| 9.0MB 168kB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): ipython>=4.0.0; python_version >= "3.3" in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Requirement already satisfied (use --upgrade to upgrade): ipykernel>=4.5.1 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Collecting nbformat>=4.2.0 (from ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/da/27/9a654d2b6cc1eaa517d1c5a4405166c7f6d72f04f6e7eea41855fe808a46/nbformat-4.4.0-py2.py3-none-any.whl (155kB) 100% |████████████████████████████████| 163kB 5.1MB/s eta 0:00:01 Collecting traitlets>=4.3.1 (from ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/93/d6/abcb22de61d78e2fc3959c964628a5771e47e7cc60d53e9342e21ed6cc9a/traitlets-4.3.2-py2.py3-none-any.whl (74kB) 100% |████████████████████████████████| 81kB 6.1MB/s eta 0:00:01 Collecting widgetsnbextension~=3.4.0 (from ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Downloading https://files.pythonhosted.org/packages/8a/81/35789a3952afb48238289171728072d26d6e76649ddc8b3588657a2d78c1/widgetsnbextension-3.4.2-py2.py3-none-any.whl (2.2MB) 100% |████████████████████████████████| 2.2MB 498kB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): singledispatch in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nltk>=3.1->textblob==0.15.1->-r requirements.txt (line 4)) Requirement already satisfied (use --upgrade to upgrade): setuptools in /cvmfs/sft.cern.ch/lcg/releases/setuptools/36.0.1-8a405/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from kiwisolver>=1.0.1->matplotlib==2.2.3->-r requirements.txt (line 10)) Collecting jupyter-client>=5.2.0 (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/3b/c3/3043fe9ffd140d03c9d091a056794ccdc427c56ec19b8eea74f9ea0a498f/jupyter_client-5.2.4-py2.py3-none-any.whl (89kB) 100% |████████████████████████████████| 92kB 6.5MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): nbconvert in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Collecting terminado>=0.8.1 (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/2e/20/a26211a24425923d46e1213b376a6ee60dc30bcdf1b0c345e2c3769deb1c/terminado-0.8.1-py2.py3-none-any.whl Collecting pyzmq>=17 (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/39/7a/9b98aabbde03fc4c5c38079ddec1331af3846d80c0b73bdb691a21adee9c/pyzmq-18.0.1-cp35-cp35m-manylinux1_x86_64.whl (1.0MB) 100% |████████████████████████████████| 1.1MB 1.1MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): jinja2 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Collecting jupyter-core>=4.4.0 (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/1d/44/065d2d7bae7bebc06f1dd70d23c36da8c50c0f08b4236716743d706762a8/jupyter_core-4.4.0-py2.py3-none-any.whl (126kB) 100% |████████████████████████████████| 133kB 6.0MB/s eta 0:00:01 Collecting Send2Trash (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/49/46/c3dc27481d1cc57b9385aff41c474ceb7714f7935b1247194adae45db714/Send2Trash-1.5.0-py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): ipython-genutils in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Collecting prometheus-client (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/4c/bd/b42db3ec90ffc6be805aad09c1cea4bb13a620d0cd4b21aaa44d13541d71/prometheus_client-0.6.0.tar.gz Collecting tornado<7,>=4.1 (from notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Downloading https://files.pythonhosted.org/packages/b1/7f/8109821ff9df1bf3519169e34646705c32ac13be6a4d51a79ed57f47686e/tornado-6.0.1.tar.gz (481kB) 100% |████████████████████████████████| 491kB 2.3MB/s eta 0:00:01 Requirement already satisfied (use --upgrade to upgrade): decorator in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from traitlets>=4.3.1->ipywidgets<8,>=7.0.0->ipyleaflet==0.9.0->-r requirements.txt (line 2)) Requirement already satisfied (use --upgrade to upgrade): mistune!=0.6 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): pygments in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): bleach in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): pandocfilters>=1.4.1 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): testpath in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): ptyprocess; os_name != "nt" in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from terminado>=0.8.1->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): markupsafe in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from jinja2->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): html5lib>=0.99999999 in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from bleach->nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Requirement already satisfied (use --upgrade to upgrade): webencodings in /cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages (from html5lib>=0.99999999->bleach->nbconvert->notebook>=5->appmode==0.4.0->-r requirements.txt (line 13)) Building wheels for collected packages: vega, arrow, appmode, toolz, nltk, prometheus-client, tornado Running setup.py bdist_wheel for vega ... done Stored in directory: /tmp/user/.cache/pip/wheels/db/57/85/e0285100a71f35c73d7b3975a8899894a73c92068cd4cc1dea Running setup.py bdist_wheel for arrow ... done Stored in directory: /tmp/user/.cache/pip/wheels/1e/d5/30/cace6155de216c74f70b1b866ce6ec2c9bbc60be88073a1fcd Running setup.py bdist_wheel for appmode ... done Stored in directory: /tmp/user/.cache/pip/wheels/c6/d0/7b/f36cfd6c017396336f95f324c147c87ee5b46d9c34fded3f6e Running setup.py bdist_wheel for toolz ... done Stored in directory: /tmp/user/.cache/pip/wheels/f4/0c/f6/ce6b2d1aa459ee97cc3c0f82236302bd62d89c86c700219463 Running setup.py bdist_wheel for nltk ... done Stored in directory: /tmp/user/.cache/pip/wheels/4b/c8/24/b2343664bcceb7147efeb21c0b23703a05b23fcfeaceaa2a1e Running setup.py bdist_wheel for prometheus-client ... done Stored in directory: /tmp/user/.cache/pip/wheels/4b/04/b8/3709c73e7453f311ebd46ad581b89642543213f995e2659b9e Running setup.py bdist_wheel for tornado ... done Stored in directory: /tmp/user/.cache/pip/wheels/09/a9/78/41e23ffcb122e8f4b02537f4480a276ec1ad760dd492373961 Successfully built vega arrow appmode toolz nltk prometheus-client tornado Installing collected packages: python-dateutil, pandas, toolz, altair, xarray, traitlets, jupyter-core, nbformat, tornado, pyzmq, jupyter-client, terminado, Send2Trash, prometheus-client, notebook, widgetsnbextension, ipywidgets, traittypes, ipyleaflet, nltk, textblob, tqdm, arrow, troveharvester, vega, pillow, wordcloud, scipy, scikit-learn, kiwisolver, matplotlib, appmode
Ok, so this seemed to work ok, though it was VERY VERY SLOW.
Note that it updated some already available packages such as Pandas, let's check that I'm using the updated version.
%%time
import pandas as pd
print(pd.__version__)
0.19.0 CPU times: user 418 ms, sys: 136 ms, total: 554 ms Wall time: 22.6 s
!pip freeze | grep pandas
You are using pip version 8.1.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
pandas==0.24.1
That's weird. According to pip
we've installed 24.1, but when we import it's only getting 19.0. How do you upgrade installed packages?
!export PYTHONPATH=$CERNBOX_HOME/.local/lib/python3.5/site-packages:$PYTHONPATH
import os
try:
user_paths = os.environ['PYTHONPATH'].split(os.pathsep)
except KeyError:
user_paths = []
user_paths
['/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/native/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/CMake/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/glib-2.0/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/plugin/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/texinfo/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python3.5/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/visualvm/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/whizard/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/CLHEP-2.3.4.4/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python2.6/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/cmake/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/py/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/archive/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/SHERPA-MC/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/ThePEG/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/JupyROOT/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/graphviz/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/gio/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/libffi-3.2.1/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/accsoft/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/trace_rlog/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/amd64/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/valgrind/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/ruby/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/JsMVA/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/VBFNLO/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/ext/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/__pycache__/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/cairo/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/libxslt-plugins/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/pkgconfig/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/python2.7/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/gle/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/missioncontrol/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/php/site-packages', '/cvmfs/sft.cern.ch/lcg/views/LCG_91python3/x86_64-slc6-gcc62-opt/lib/lib/site-packages', '']
!pwd
/webdav
!pip install --upgrade --user pandas
Requirement already up-to-date: pandas in ./.local/lib/python3.5/site-packages Requirement already up-to-date: numpy>=1.12.0 in ./.local/lib/python3.5/site-packages (from pandas) Requirement already up-to-date: python-dateutil>=2.5.0 in ./.local/lib/python3.5/site-packages (from pandas) Collecting pytz>=2011k (from pandas) Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl Requirement already up-to-date: six>=1.5 in ./.local/lib/python3.5/site-packages (from python-dateutil>=2.5.0->pandas) Installing collected packages: pytz
!pip install --user pillow
Requirement already satisfied (use --upgrade to upgrade): pillow in ./.local/lib/python3.5/site-packages
You are using pip version 8.1.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
%%time
import pandas as pd
print(pd.__version__)
0.19.0 CPU times: user 364 µs, sys: 91 µs, total: 455 µs Wall time: 338 µs
%time !ls
ASIO files Shared datasets trying_out_swan.ipynb welcome.txt lost+found test Untitled1.ipynb Shared trove-newspaper-harvester Untitled.ipynb CPU times: user 10.8 ms, sys: 7.01 ms, total: 17.8 ms Wall time: 566 ms
!pwd
/webdav
import cv2
img = cv2.imread('Shared/ANU-Library/Sydney Stock Exchange 1901-1950/AU NBAC N193-001/N193-001_0001.tif')
img.shape
(6646, 5328, 3)
!pip install --user tqdm
Collecting tqdm Using cached https://files.pythonhosted.org/packages/6c/4b/c38b5144cf167c4f52288517436ccafefe9dc01b8d1c190e18a6b154cd4a/tqdm-4.31.1-py2.py3-none-any.whl Installing collected packages: tqdm