Dataset info
Number of columns | 10 |
Number of rows | 45716 |
Total Missing (%) | 0.49% |
Total size in memory | 64.0 MB |
Column types
String | 1 |
Numeric | 0 |
Date | 0 |
Bool | 0 |
Array | 0 |
Not available | 0 |
%load_ext autoreload
%autoreload 2
import sys
sys.path.append("..")
# Create optimus
from optimus import Optimus
op = Optimus(master="local[*]", app_name = "optimus" , checkpoint= True, verbose=True)
C:\Users\argenisleon\Anaconda3\lib\site-packages\socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Callable You are using PySparkling of version 2.4.10, but your PySpark is of version 2.3.1. Please make sure Spark and PySparkling versions are compatible.
f657036e-90b5-4391-8d48-fb3f16f6b8ae
INFO:optimus:Operative System:Windows INFO:optimus:Just check that Spark and all necessary environments vars are present... INFO:optimus:----- INFO:optimus:SPARK_HOME=C:\opt\spark\spark-2.3.1-bin-hadoop2.7 INFO:optimus:HADOOP_HOME=C:\opt\hadoop-2.7.7 INFO:optimus:PYSPARK_PYTHON=C:\Users\argenisleon\Anaconda3\python.exe INFO:optimus:PYSPARK_DRIVER_PYTHON=jupyter INFO:optimus:PYSPARK_SUBMIT_ARGS=--jars "file:///C:/Users/argenisleon/Documents/Optimus/optimus/jars/RedshiftJDBC42-1.2.16.1027.jar,file:///C:/Users/argenisleon/Documents/Optimus/optimus/jars/mysql-connector-java-8.0.16.jar,file:///C:/Users/argenisleon/Documents/Optimus/optimus/jars/ojdbc8.jar,file:///C:/Users/argenisleon/Documents/Optimus/optimus/jars/postgresql-42.2.5.jar" --driver-class-path "C:/Users/argenisleon/Documents/Optimus/optimus/jars/RedshiftJDBC42-1.2.16.1027.jar;C:/Users/argenisleon/Documents/Optimus/optimus/jars/mysql-connector-java-8.0.16.jar;C:/Users/argenisleon/Documents/Optimus/optimus/jars/ojdbc8.jar;C:/Users/argenisleon/Documents/Optimus/optimus/jars/postgresql-42.2.5.jar" --conf "spark.sql.catalogImplementation=hive" pyspark-shell INFO:optimus:JAVA_HOME=C:\java INFO:optimus:Pyarrow Installed INFO:optimus:----- INFO:optimus:Starting or getting SparkSession and SparkContext... INFO:optimus:Spark Version:2.3.1 INFO:optimus:Setting checkpoint folder local. If you are in a cluster initialize Optimus with master='your_ip' as param INFO:optimus:Deleting previous folder if exists... INFO:optimus:Creating the checkpoint directory... INFO:optimus: ____ __ _ / __ \____ / /_(_)___ ___ __ _______ / / / / __ \/ __/ / __ `__ \/ / / / ___/ / /_/ / /_/ / /_/ / / / / / / /_/ (__ ) \____/ .___/\__/_/_/ /_/ /_/\__,_/____/ /_/ INFO:optimus:Transform and Roll out... INFO:optimus:Optimus successfully imported. Have fun :). INFO:optimus:Config.ini not found
df = op.load.csv("data/Meteorite_Landings.csv").h_repartition()
df.table(10)
name
1 (string)
nullable
|
id
2 (int)
nullable
|
nametype
3 (string)
nullable
|
recclass
4 (string)
nullable
|
mass (g)
5 (double)
nullable
|
fall
6 (string)
nullable
|
year
7 (string)
nullable
|
reclat
8 (double)
nullable
|
reclong
9 (double)
nullable
|
GeoLocation
10 (string)
nullable
|
---|---|---|---|---|---|---|---|---|---|
Acfer⋅232
|
240
|
Valid
|
H5
|
725.0
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
27.73944
|
4.32833
|
(27.739440,⋅4.328330)
|
Elephant⋅Moraine⋅90232
|
8641
|
Valid
|
L6
|
16.9
|
Found
|
01/01/1990⋅12:00:00⋅AM
|
-76.28795
|
156.46841
|
(-76.287950,⋅156.468410)
|
Grove⋅Mountains⋅020090
|
30681
|
Valid
|
Martian⋅(shergottite)
|
7.5
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.99944
|
75.26111
|
(-72.999440,⋅75.261110)
|
Northwest⋅Africa⋅891
|
31912
|
Valid
|
H4
|
70.8
|
Found
|
01/01/2001⋅12:00:00⋅AM
|
None
|
None
|
None
|
Queen⋅Alexandra⋅Range⋅93098
|
19187
|
Valid
|
H6
|
1.2
|
Found
|
01/01/1993⋅12:00:00⋅AM
|
-84.5757
|
162.56524
|
(-84.575700,⋅162.565240)
|
Queen⋅Alexandra⋅Range⋅94691
|
20322
|
Valid
|
H6
|
9.6
|
Found
|
01/01/1994⋅12:00:00⋅AM
|
-84.0
|
168.0
|
(-84.000000,⋅168.000000)
|
Meteorite⋅Hills⋅00977
|
16211
|
Valid
|
H5
|
13.2
|
Found
|
01/01/2000⋅12:00:00⋅AM
|
-79.68333
|
159.75
|
(-79.683330,⋅159.750000)
|
Grove⋅Mountains⋅020114
|
46531
|
Valid
|
L3
|
1.0
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.98194
|
75.25167
|
(-72.981940,⋅75.251670)
|
Pecora⋅Escarpment⋅91483
|
18774
|
Valid
|
H5
|
5.5
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
-85.55819
|
-68.31586
|
(-85.558190,⋅-68.315860)
|
Ramlat⋅as⋅Sahmah⋅390
|
55656
|
Valid
|
H3.8-6
|
0.69
|
Found
|
01/01/2010⋅12:00:00⋅AM
|
20.0949
|
55.69318
|
(20.094900,⋅55.693180)
|
op.profiler.run(df, "name", infer=False, approx_count= True)
{'name': 'string', 'id': 'int', 'nametype': 'string', 'recclass': 'string', 'mass (g)': 'double', 'fall': 'string', 'year': 'string', 'reclat': 'double', 'reclong': 'double', 'GeoLocation': 'string'} Including 'nan' as Null in processing 'name' Including 'nan' as Null in processing 'name' Including 'nan' as Null in processing 'nametype' Including 'nan' as Null in processing 'recclass' Including 'nan' as Null in processing 'fall' Including 'nan' as Null in processing 'year' Including 'nan' as Null in processing 'GeoLocation'
Number of columns | 10 |
Number of rows | 45716 |
Total Missing (%) | 0.49% |
Total size in memory | 64.0 MB |
String | 1 |
Numeric | 0 |
Date | 0 |
Bool | 0 |
Array | 0 |
Not available | 0 |
Unique | |
Unique (%) | 99.56 |
Missing | 0.0 |
Missing (%) | 0 |
String | 45716 |
Integer | 0 |
Float | 0 |
Bool | 0 |
Date | 0 |
Missing | 0 |
Null | 0 |
Value | Count | Frequency (%) |
---|---|---|
Święcany | 1 | 0.002% |
Łowicz | 1 | 0.002% |
Österplana 064 | 1 | 0.002% |
Österplana 063 | 1 | 0.002% |
Österplana 062 | 1 | 0.002% |
Österplana 061 | 1 | 0.002% |
Österplana 060 | 1 | 0.002% |
Österplana 059 | 1 | 0.002% |
Österplana 058 | 1 | 0.002% |
Österplana 057 | 1 | 0.002% |
"Missing" | 0 | 0.0% |
|
|
name
1 (string)
nullable
|
id
2 (int)
nullable
|
nametype
3 (string)
nullable
|
recclass
4 (string)
nullable
|
mass (g)
5 (double)
nullable
|
fall
6 (string)
nullable
|
year
7 (string)
nullable
|
reclat
8 (double)
nullable
|
reclong
9 (double)
nullable
|
GeoLocation
10 (string)
nullable
|
---|---|---|---|---|---|---|---|---|---|
Acfer⋅232
|
240
|
Valid
|
H5
|
725.0
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
27.73944
|
4.32833
|
(27.739440,⋅4.328330)
|
Elephant⋅Moraine⋅90232
|
8641
|
Valid
|
L6
|
16.9
|
Found
|
01/01/1990⋅12:00:00⋅AM
|
-76.28795
|
156.46841
|
(-76.287950,⋅156.468410)
|
Grove⋅Mountains⋅020090
|
30681
|
Valid
|
Martian⋅(shergottite)
|
7.5
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.99944
|
75.26111
|
(-72.999440,⋅75.261110)
|
Northwest⋅Africa⋅891
|
31912
|
Valid
|
H4
|
70.8
|
Found
|
01/01/2001⋅12:00:00⋅AM
|
None
|
None
|
None
|
Queen⋅Alexandra⋅Range⋅93098
|
19187
|
Valid
|
H6
|
1.2
|
Found
|
01/01/1993⋅12:00:00⋅AM
|
-84.5757
|
162.56524
|
(-84.575700,⋅162.565240)
|
Queen⋅Alexandra⋅Range⋅94691
|
20322
|
Valid
|
H6
|
9.6
|
Found
|
01/01/1994⋅12:00:00⋅AM
|
-84.0
|
168.0
|
(-84.000000,⋅168.000000)
|
Meteorite⋅Hills⋅00977
|
16211
|
Valid
|
H5
|
13.2
|
Found
|
01/01/2000⋅12:00:00⋅AM
|
-79.68333
|
159.75
|
(-79.683330,⋅159.750000)
|
Grove⋅Mountains⋅020114
|
46531
|
Valid
|
L3
|
1.0
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.98194
|
75.25167
|
(-72.981940,⋅75.251670)
|
Pecora⋅Escarpment⋅91483
|
18774
|
Valid
|
H5
|
5.5
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
-85.55819
|
-68.31586
|
(-85.558190,⋅-68.315860)
|
Ramlat⋅as⋅Sahmah⋅390
|
55656
|
Valid
|
H3.8-6
|
0.69
|
Found
|
01/01/2010⋅12:00:00⋅AM
|
20.0949
|
55.69318
|
(20.094900,⋅55.693180)
|
op.profiler.run(df, "GeoLocation",infer=True)
INFO:optimus:Processing column 'GeoLocation'... INFO:optimus:_count_data_types() executed in 36.83 sec INFO:optimus:count_data_types() executed in 36.84 sec INFO:optimus:cast_columns() executed in 0.0 sec INFO:optimus:agg_exprs() executed in 4.67 sec INFO:optimus:general_stats() executed in 4.68 sec INFO:optimus:------------------------------ INFO:optimus:Processing column 'GeoLocation'... INFO:optimus:frequency() executed in 6.22 sec INFO:optimus:stats_by_column() executed in 0.0 sec INFO:optimus:Using 'column_exp' to process column 'GeoLocation_len' with function func_col_exp INFO:optimus:Using 'column_exp' to process column 'GeoLocation_len' with function _bucketizer INFO:optimus:hist() executed in 4.79 sec INFO:optimus:hist_string() executed in 8.07 sec
Including 'nan' as Null in processing 'name' Including 'nan' as Null in processing 'nametype' Including 'nan' as Null in processing 'recclass' Including 'nan' as Null in processing 'fall' Including 'nan' as Null in processing 'year' Including 'nan' as Null in processing 'GeoLocation'
INFO:optimus:dataset_info() executed in 3.87 sec
Number of columns | 10 |
Number of rows | 45716 |
Total Missing (%) | 0.49% |
Total size in memory | 80.0 MB |
String | 0 |
Numeric | 0 |
Date | 0 |
Bool | 0 |
Array | 1 |
Not available | 0 |
Unique | |
Unique (%) | 36.499 |
Missing | 0.0 |
Missing (%) | 0 |
String | 0 |
Integer | 0 |
Float | 0 |
Bool | 0 |
Date | 0 |
Missing | 0 |
Null | 7315 |
Value | Count | Frequency (%) |
---|---|---|
None | 7315 | 16.001% |
(0.000000, 0.000000) | 6214 | 13.593% |
(-71.500000, 35.666670) | 4761 | 10.414% |
(-84.000000, 168.000000) | 3040 | 6.65% |
(-72.000000, 26.000000) | 1505 | 3.292% |
(-79.683330, 159.750000) | 657 | 1.437% |
(-76.716670, 159.666670) | 637 | 1.393% |
(-76.183330, 157.166670) | 539 | 1.179% |
(-79.683330, 155.750000) | 473 | 1.035% |
(-84.216670, 160.500000) | 263 | 0.575% |
"Missing" | 0 | 0.0% |
|
|
name
1 (string)
nullable
|
id
2 (int)
nullable
|
nametype
3 (string)
nullable
|
recclass
4 (string)
nullable
|
mass (g)
5 (double)
nullable
|
fall
6 (string)
nullable
|
year
7 (string)
nullable
|
reclat
8 (double)
nullable
|
reclong
9 (double)
nullable
|
GeoLocation
10 (string)
nullable
|
---|---|---|---|---|---|---|---|---|---|
Acfer⋅232
|
240
|
Valid
|
H5
|
725.0
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
27.73944
|
4.32833
|
(27.739440,⋅4.328330)
|
Elephant⋅Moraine⋅90232
|
8641
|
Valid
|
L6
|
16.9
|
Found
|
01/01/1990⋅12:00:00⋅AM
|
-76.28795
|
156.46841
|
(-76.287950,⋅156.468410)
|
Grove⋅Mountains⋅020090
|
30681
|
Valid
|
Martian⋅(shergottite)
|
7.5
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.99944
|
75.26111
|
(-72.999440,⋅75.261110)
|
Northwest⋅Africa⋅891
|
31912
|
Valid
|
H4
|
70.8
|
Found
|
01/01/2001⋅12:00:00⋅AM
|
None
|
None
|
None
|
Queen⋅Alexandra⋅Range⋅93098
|
19187
|
Valid
|
H6
|
1.2
|
Found
|
01/01/1993⋅12:00:00⋅AM
|
-84.5757
|
162.56524
|
(-84.575700,⋅162.565240)
|
Queen⋅Alexandra⋅Range⋅94691
|
20322
|
Valid
|
H6
|
9.6
|
Found
|
01/01/1994⋅12:00:00⋅AM
|
-84.0
|
168.0
|
(-84.000000,⋅168.000000)
|
Meteorite⋅Hills⋅00977
|
16211
|
Valid
|
H5
|
13.2
|
Found
|
01/01/2000⋅12:00:00⋅AM
|
-79.68333
|
159.75
|
(-79.683330,⋅159.750000)
|
Grove⋅Mountains⋅020114
|
46531
|
Valid
|
L3
|
1.0
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.98194
|
75.25167
|
(-72.981940,⋅75.251670)
|
Pecora⋅Escarpment⋅91483
|
18774
|
Valid
|
H5
|
5.5
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
-85.55819
|
-68.31586
|
(-85.558190,⋅-68.315860)
|
Ramlat⋅as⋅Sahmah⋅390
|
55656
|
Valid
|
H3.8-6
|
0.69
|
Found
|
01/01/2010⋅12:00:00⋅AM
|
20.0949
|
55.69318
|
(20.094900,⋅55.693180)
|
INFO:optimus:run() executed in 65.79 sec
op.profiler.run(df, "reclat")
Including 'nan' as Null in processing 'name' Including 'nan' as Null in processing 'nametype' Including 'nan' as Null in processing 'recclass' Including 'nan' as Null in processing 'fall' Including 'nan' as Null in processing 'year' Including 'nan' as Null in processing 'GeoLocation'
Number of columns | 10 |
Number of rows | 45716 |
Total Missing (%) | 0.49% |
Total size in memory | 100.4 MB |
String | 0 |
Numeric | 1 |
Date | 0 |
Bool | 0 |
Array | 0 |
Not available | 0 |
Unique | |
Unique (%) | 28.806 |
Missing | 0.0 |
Missing (%) | 0 |
String | 0 |
Integer | 0 |
Float | 0 |
Bool | 0 |
Date | 0 |
Missing | 0 |
Null | 7315 |
Mean | -39.12258 |
Minimum | -87.36667 |
Maximum | 81.16667 |
Zeros(%) |
Value | Count | Frequency (%) |
---|---|---|
None | 7315 | 16.001% |
0.0 | 6438 | 14.083% |
-71.5 | 4761 | 10.414% |
-84.0 | 3040 | 6.65% |
-72.0 | 1506 | 3.294% |
-79.68333 | 1130 | 2.472% |
-76.71667 | 680 | 1.487% |
-76.18333 | 539 | 1.179% |
-84.21667 | 263 | 0.575% |
-86.36667 | 226 | 0.494% |
"Missing" | 0 | 0.0% |
Minimum | -87.36667 |
5-th percentile | |
Q1 | |
Median | |
Q3 | |
95-th percentile | |
Maximum | 81.16667 |
Range | 168.53334 |
Interquartile range | 0.0 |
Standard deviation | 46.37851 |
Coef of variation | -1.18547 |
Kurtosis | -1.4768 |
Mean | -39.12258 |
MAD | 0.0 |
Skewness | |
Sum | -1502346.20654 |
Variance | 2150.96632 |
|
|
name
1 (string)
nullable
|
id
2 (int)
nullable
|
nametype
3 (string)
nullable
|
recclass
4 (string)
nullable
|
mass (g)
5 (double)
nullable
|
fall
6 (string)
nullable
|
year
7 (string)
nullable
|
reclat
8 (double)
nullable
|
reclong
9 (double)
nullable
|
GeoLocation
10 (string)
nullable
|
---|---|---|---|---|---|---|---|---|---|
Acfer⋅232
|
240
|
Valid
|
H5
|
725.0
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
27.73944
|
4.32833
|
(27.739440,⋅4.328330)
|
Elephant⋅Moraine⋅90232
|
8641
|
Valid
|
L6
|
16.9
|
Found
|
01/01/1990⋅12:00:00⋅AM
|
-76.28795
|
156.46841
|
(-76.287950,⋅156.468410)
|
Grove⋅Mountains⋅020090
|
30681
|
Valid
|
Martian⋅(shergottite)
|
7.5
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.99944
|
75.26111
|
(-72.999440,⋅75.261110)
|
Northwest⋅Africa⋅891
|
31912
|
Valid
|
H4
|
70.8
|
Found
|
01/01/2001⋅12:00:00⋅AM
|
None
|
None
|
None
|
Queen⋅Alexandra⋅Range⋅93098
|
19187
|
Valid
|
H6
|
1.2
|
Found
|
01/01/1993⋅12:00:00⋅AM
|
-84.5757
|
162.56524
|
(-84.575700,⋅162.565240)
|
Queen⋅Alexandra⋅Range⋅94691
|
20322
|
Valid
|
H6
|
9.6
|
Found
|
01/01/1994⋅12:00:00⋅AM
|
-84.0
|
168.0
|
(-84.000000,⋅168.000000)
|
Meteorite⋅Hills⋅00977
|
16211
|
Valid
|
H5
|
13.2
|
Found
|
01/01/2000⋅12:00:00⋅AM
|
-79.68333
|
159.75
|
(-79.683330,⋅159.750000)
|
Grove⋅Mountains⋅020114
|
46531
|
Valid
|
L3
|
1.0
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.98194
|
75.25167
|
(-72.981940,⋅75.251670)
|
Pecora⋅Escarpment⋅91483
|
18774
|
Valid
|
H5
|
5.5
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
-85.55819
|
-68.31586
|
(-85.558190,⋅-68.315860)
|
Ramlat⋅as⋅Sahmah⋅390
|
55656
|
Valid
|
H3.8-6
|
0.69
|
Found
|
01/01/2010⋅12:00:00⋅AM
|
20.0949
|
55.69318
|
(20.094900,⋅55.693180)
|
df.plot.hist(["id", "reclong"], 20)
df.plot.frequency(["id", "reclong"], 10)
df.table()
name
1 (string)
nullable
|
id
2 (int)
nullable
|
nametype
3 (string)
nullable
|
recclass
4 (string)
nullable
|
mass (g)
5 (double)
nullable
|
fall
6 (string)
nullable
|
year
7 (string)
nullable
|
reclat
8 (double)
nullable
|
reclong
9 (double)
nullable
|
GeoLocation
10 (string)
nullable
|
---|---|---|---|---|---|---|---|---|---|
Acfer⋅232
|
240
|
Valid
|
H5
|
725.0
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
27.73944
|
4.32833
|
(27.739440,⋅4.328330)
|
Elephant⋅Moraine⋅90232
|
8641
|
Valid
|
L6
|
16.9
|
Found
|
01/01/1990⋅12:00:00⋅AM
|
-76.28795
|
156.46841
|
(-76.287950,⋅156.468410)
|
Grove⋅Mountains⋅020090
|
30681
|
Valid
|
Martian⋅(shergottite)
|
7.5
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.99944
|
75.26111
|
(-72.999440,⋅75.261110)
|
Northwest⋅Africa⋅891
|
31912
|
Valid
|
H4
|
70.8
|
Found
|
01/01/2001⋅12:00:00⋅AM
|
None
|
None
|
None
|
Queen⋅Alexandra⋅Range⋅93098
|
19187
|
Valid
|
H6
|
1.2
|
Found
|
01/01/1993⋅12:00:00⋅AM
|
-84.5757
|
162.56524
|
(-84.575700,⋅162.565240)
|
Queen⋅Alexandra⋅Range⋅94691
|
20322
|
Valid
|
H6
|
9.6
|
Found
|
01/01/1994⋅12:00:00⋅AM
|
-84.0
|
168.0
|
(-84.000000,⋅168.000000)
|
Meteorite⋅Hills⋅00977
|
16211
|
Valid
|
H5
|
13.2
|
Found
|
01/01/2000⋅12:00:00⋅AM
|
-79.68333
|
159.75
|
(-79.683330,⋅159.750000)
|
Grove⋅Mountains⋅020114
|
46531
|
Valid
|
L3
|
1.0
|
Found
|
01/01/2003⋅12:00:00⋅AM
|
-72.98194
|
75.25167
|
(-72.981940,⋅75.251670)
|
Pecora⋅Escarpment⋅91483
|
18774
|
Valid
|
H5
|
5.5
|
Found
|
01/01/1991⋅12:00:00⋅AM
|
-85.55819
|
-68.31586
|
(-85.558190,⋅-68.315860)
|
Ramlat⋅as⋅Sahmah⋅390
|
55656
|
Valid
|
H3.8-6
|
0.69
|
Found
|
01/01/2010⋅12:00:00⋅AM
|
20.0949
|
55.69318
|
(20.094900,⋅55.693180)
|
df.cols.count_na("*")
Including 'nan' as Null in processing 'name' Including 'nan' as Null in processing 'nametype' Including 'nan' as Null in processing 'recclass' Including 'nan' as Null in processing 'fall' Including 'nan' as Null in processing 'year' Including 'nan' as Null in processing 'GeoLocation'
{'name': 0, 'id': 0, 'nametype': 0, 'recclass': 0, 'mass (g)': 131, 'fall': 0, 'year': 288, 'reclat': 7315, 'reclong': 7315, 'GeoLocation': 7315}
a = {'name': 0,
'id': 0,
'nametype': 0,
'recclass': 0,
'mass (g)': 131,
'fall': 0,
'year': 288,
'reclat': 7315,
'reclong': 7315,
'GeoLocation': 7315}
df.cols.dtypes()
File "<ipython-input-13-1cf015ba60d7>", line 12 df.cols.dtypes() ^ IndentationError: unexpected indent
cols = ["id","mass (g)","reclat"]
# We drops nulls because correlation can not handle them
df_not_nulls = df.rows.drop_na(cols)
df_not_nulls.plot.correlation(cols)
df_not_nulls.cols.correlation(["id","mass (g)", "reclat"], output="array")
{'cols': ['id', 'mass (g)', 'reclat'], 'data': array([[ 1. , -0.01794746, 0.27151272], [-0.01794746, 1. , 0.02908721], [ 0.27151272, 0.02908721, 1. ]])}