Make sure I run before you embark too far down the rest of this rabbit hole!
pandas
: Data Manipulation Library; basically a Data Science Workhorsexlrd
/xlwt
: Packages for interfacing with Excel filesdask
: Distributed computing library (cloud go brrrrrrrrr)plotly
/plotly-geo
/plotly-kaleido
: interactive graphingipywidgets
: used for interactive displays in JupyterLabjupyterlab_hdf
: Editor/Viewer for HDF filespyarrow
: Advanced Data Processing (think of it like a cross platform pandas
2.0)%conda install -y pandas xlrd xlwt "ipywidgets=7.5" pyarrow
Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... done ## Package Plan ## environment location: C:\Users\me\anaconda3 added / updated specs: - ipywidgets=7.5 - pandas - pyarrow - xlrd - xlwt The following NEW packages will be INSTALLED: abseil-cpp conda-forge/win-64::abseil-cpp-20200225.2-ha925a31_2 arrow-cpp conda-forge/win-64::arrow-cpp-1.0.1-py38hecdc4bb_9_cpu aws-c-common conda-forge/win-64::aws-c-common-0.4.58-ha925a31_0 aws-c-event-stream conda-forge/win-64::aws-c-event-stream-0.1.6-h0fc2d53_4 aws-checksums conda-forge/win-64::aws-checksums-0.1.9-hc1c5f57_1 aws-sdk-cpp conda-forge/win-64::aws-sdk-cpp-1.8.56-hc081641_1 brotli conda-forge/win-64::brotli-1.0.9-ha925a31_1 c-ares conda-forge/win-64::c-ares-1.16.1-he774522_3 gflags conda-forge/win-64::gflags-2.2.2-ha925a31_1004 glog conda-forge/win-64::glog-0.4.0-h0174b99_3 grpc-cpp conda-forge/win-64::grpc-cpp-1.32.0-hc9df816_0 libprotobuf conda-forge/win-64::libprotobuf-3.13.0-h200bbdf_0 libthrift conda-forge/win-64::libthrift-0.13.0-hdfef310_6 libutf8proc conda-forge/win-64::libutf8proc-2.5.0-h9e6e254_2 parquet-cpp conda-forge/noarch::parquet-cpp-1.5.1-2 pyarrow conda-forge/win-64::pyarrow-1.0.1-py38hc564555_9_cpu re2 conda-forge/win-64::re2-2020.08.01-ha925a31_1 Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done Note: you may need to restart the kernel to use updated packages.
%conda install -y -c plotly plotly plotly-geo python-kaleido
Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... done ## Package Plan ## environment location: C:\Users\me\anaconda3 added / updated specs: - plotly-geo - python-kaleido The following NEW packages will be INSTALLED: plotly-geo plotly/noarch::plotly-geo-1.0.0-py_0 python-kaleido plotly/win-64::python-kaleido-0.0.3-py38_0 Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done Note: you may need to restart the kernel to use updated packages.
%pip install jupyterlab_hdf
Collecting jupyterlab_hdf Using cached jupyterlab_hdf-0.4.1-py3-none-any.whl (12 kB) Requirement already satisfied: h5py in c:\users\me\anaconda3\lib\site-packages (from jupyterlab_hdf) (2.10.0) Requirement already satisfied: notebook in c:\users\me\anaconda3\lib\site-packages (from jupyterlab_hdf) (6.0.3) Requirement already satisfied: numpy>=1.7 in c:\users\me\anaconda3\lib\site-packages (from h5py->jupyterlab_hdf) (1.18.5) Requirement already satisfied: six in c:\users\me\anaconda3\lib\site-packages (from h5py->jupyterlab_hdf) (1.15.0) Requirement already satisfied: ipython-genutils in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (0.2.0) Requirement already satisfied: tornado>=5.0 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (6.0.4) Requirement already satisfied: nbformat in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (5.0.7) Requirement already satisfied: traitlets>=4.2.1 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (4.3.3) Requirement already satisfied: terminado>=0.8.1 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (0.8.3) Requirement already satisfied: jupyter-client>=5.3.4 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (6.1.6) Requirement already satisfied: Send2Trash in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (1.5.0) Requirement already satisfied: nbconvert in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (5.6.1) Requirement already satisfied: ipykernel in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (5.3.2) Requirement already satisfied: jinja2 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (2.11.2) Requirement already satisfied: pyzmq>=17 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (19.0.1) Requirement already satisfied: prometheus-client in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (0.8.0) Requirement already satisfied: jupyter-core>=4.6.1 in c:\users\me\anaconda3\lib\site-packages (from notebook->jupyterlab_hdf) (4.6.3) Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in c:\users\me\anaconda3\lib\site-packages (from nbformat->notebook->jupyterlab_hdf) (3.2.0) Requirement already satisfied: decorator in c:\users\me\anaconda3\lib\site-packages (from traitlets>=4.2.1->notebook->jupyterlab_hdf) (4.4.2) Requirement already satisfied: python-dateutil>=2.1 in c:\users\me\anaconda3\lib\site-packages (from jupyter-client>=5.3.4->notebook->jupyterlab_hdf) (2.8.1) Requirement already satisfied: entrypoints>=0.2.2 in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (0.3) Requirement already satisfied: bleach in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (3.1.5) Requirement already satisfied: pygments in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (2.6.1) Requirement already satisfied: mistune<2,>=0.8.1 in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (0.8.4) Requirement already satisfied: defusedxml in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (0.6.0) Requirement already satisfied: pandocfilters>=1.4.1 in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (1.4.2) Requirement already satisfied: testpath in c:\users\me\anaconda3\lib\site-packages (from nbconvert->notebook->jupyterlab_hdf) (0.4.4) Requirement already satisfied: ipython>=5.0.0 in c:\users\me\anaconda3\lib\site-packages (from ipykernel->notebook->jupyterlab_hdf) (7.16.1) Requirement already satisfied: MarkupSafe>=0.23 in c:\users\me\anaconda3\lib\site-packages (from jinja2->notebook->jupyterlab_hdf) (1.1.1) Requirement already satisfied: pywin32>=1.0; sys_platform == "win32" in c:\users\me\anaconda3\lib\site-packages (from jupyter-core>=4.6.1->notebook->jupyterlab_hdf) (227) Requirement already satisfied: setuptools in c:\users\me\anaconda3\lib\site-packages (from jsonschema!=2.5.0,>=2.4->nbformat->notebook->jupyterlab_hdf) (49.2.0.post20200714) Requirement already satisfied: attrs>=17.4.0 in c:\users\me\anaconda3\lib\site-packages (from jsonschema!=2.5.0,>=2.4->nbformat->notebook->jupyterlab_hdf) (19.3.0) Requirement already satisfied: pyrsistent>=0.14.0 in c:\users\me\anaconda3\lib\site-packages (from jsonschema!=2.5.0,>=2.4->nbformat->notebook->jupyterlab_hdf) (0.16.0) Requirement already satisfied: packaging in c:\users\me\anaconda3\lib\site-packages (from bleach->nbconvert->notebook->jupyterlab_hdf) (20.4) Requirement already satisfied: webencodings in c:\users\me\anaconda3\lib\site-packages (from bleach->nbconvert->notebook->jupyterlab_hdf) (0.5.1) Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in c:\users\me\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (3.0.5) Requirement already satisfied: jedi>=0.10 in c:\users\me\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.17.1) Requirement already satisfied: backcall in c:\users\me\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.2.0) Requirement already satisfied: pickleshare in c:\users\me\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.7.5) Requirement already satisfied: colorama; sys_platform == "win32" in c:\users\me\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.4.3) Requirement already satisfied: pyparsing>=2.0.2 in c:\users\me\anaconda3\lib\site-packages (from packaging->bleach->nbconvert->notebook->jupyterlab_hdf) (2.4.7) Requirement already satisfied: wcwidth in c:\users\me\anaconda3\lib\site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.2.5) Requirement already satisfied: parso<0.8.0,>=0.7.0 in c:\users\me\anaconda3\lib\site-packages (from jedi>=0.10->ipython>=5.0.0->ipykernel->notebook->jupyterlab_hdf) (0.7.0) Installing collected packages: jupyterlab-hdf Successfully installed jupyterlab-hdf-0.4.1 Note: you may need to restart the kernel to use updated packages.
!jupyter labextension install @jupyter-widgets/jupyterlab-manager @jupyterlab/hdf5 dask-labextension jupyterlab-plotly@4.11.0 plotlywidget@4.11.0
Building jupyterlab assets (build:prod:minimize)
!jupyter lab build
[LabBuildApp] JupyterLab 2.1.5 [LabBuildApp] Building in C:\Users\me\anaconda3\share\jupyter\lab [LabBuildApp] Building jupyterlab assets (build:prod:minimize)
!jupyter labextension list
JupyterLab v2.1.5 Known labextensions:
app dir: C:\Users\me\anaconda3\share\jupyter\lab @jupyter-widgets/jupyterlab-manager v2.0.0 enabled ok @jupyterlab/hdf5 v0.4.1 enabled ok dask-labextension v3.0.0 enabled ok jupyterlab-plotly v4.11.0 enabled ok plotlywidget v4.11.0 enabled ok
import numpy as np
import pandas as pd
from string import printable
n = 100
df = pd.DataFrame({
'A': np.random.random(n),
'B': np.random.randint(0,100,size=n),
'C': pd.date_range('today',periods=n, freq='h'),
'D': map(''.join,np.random.choice(list(printable), size=(n,12)))
})
df
A | B | C | D | |
---|---|---|---|---|
0 | 0.267533 | 19 | 2020-11-09 13:10:48.343092 | $l<yHb>y*DRk |
1 | 0.596342 | 53 | 2020-11-09 14:10:48.343092 | &'TF~V|\Bqy\n |
2 | 0.589343 | 98 | 2020-11-09 15:10:48.343092 | Cj@(0N|irZ:Z |
3 | 0.098064 | 53 | 2020-11-09 16:10:48.343092 | \nQ\rW0[#E'NN |
4 | 0.643037 | 98 | 2020-11-09 17:10:48.343092 | GIwGkwL\b\=, |
... | ... | ... | ... | ... |
95 | 0.347216 | 45 | 2020-11-13 12:10:48.343092 | d`Kp9Rq?<>-$ |
96 | 0.868712 | 80 | 2020-11-13 13:10:48.343092 | -K?G]aHc#yZs |
97 | 0.830110 | 26 | 2020-11-13 14:10:48.343092 | 7kdoo&0$p//p |
98 | 0.605215 | 6 | 2020-11-13 15:10:48.343092 | XDH!c7\i\t1Mq |
99 | 0.566997 | 80 | 2020-11-13 16:10:48.343092 | 9rSr9w;+_%)e |
100 rows × 4 columns
df.to_csv('sample.csv',index=False)
pd.read_csv('sample.csv')
A | B | C | D | |
---|---|---|---|---|
0 | 0.267533 | 19 | 2020-11-09 13:10:48.343092 | $l<yHb>y*DRk |
1 | 0.596342 | 53 | 2020-11-09 14:10:48.343092 | &'TF~V|\Bqy\n |
2 | 0.589343 | 98 | 2020-11-09 15:10:48.343092 | Cj@(0N|irZ:Z |
3 | 0.098064 | 53 | 2020-11-09 16:10:48.343092 | \nQ\rW0[#E'NN |
4 | 0.643037 | 98 | 2020-11-09 17:10:48.343092 | GIwGkwL\b\=, |
... | ... | ... | ... | ... |
95 | 0.347216 | 45 | 2020-11-13 12:10:48.343092 | d`Kp9Rq?<>-$ |
96 | 0.868712 | 80 | 2020-11-13 13:10:48.343092 | -K?G]aHc#yZs |
97 | 0.830110 | 26 | 2020-11-13 14:10:48.343092 | 7kdoo&0$p//p |
98 | 0.605215 | 6 | 2020-11-13 15:10:48.343092 | XDH!c7\i\t1Mq |
99 | 0.566997 | 80 | 2020-11-13 16:10:48.343092 | 9rSr9w;+_%)e |
100 rows × 4 columns
%conda env export --from-history -f environment.yml
Note: you may need to restart the kernel to use updated packages.