In 2020 August, the author attempted a characterization of S&P500 companies based on fundamental metrics during covid19. Here, the author sought to characterize the performance of S&P500 companies 1-month after the first covid19 vaccine success announcement by Pfizer on the Nov 18, 2020. Briefly, the stock return was calculated based on the difference between the end of day (eod) stock prices on the Nov 17 and Dec 18, 2020. The contemporary fundamental metrics of the companies were retrieved to investigate their correlations with the top and worst performer during the month.
You'll need to you install the quanp package (https://quanp.readthedocs.io/en/latest/installation.html) that should install all necessary packages/libraries required to execute the codes in this tutorial. Please create and use virtualenv with python version 3.6 to avoid dependency problem.
# import sys
# !conda install seaborn scikit-learn statsmodels numba pytables
# !conda install -c conda-forge python-igraph leidenalg
# !{sys.executable} -m pip install quanp
# importing library packages
from datetime import datetime, timedelta
import os
import matplotlib.pyplot as pl
import numpy as np
import pandas as pd
import quanp as qp
import re
from IPython.display import display
from matplotlib import rcParams
# setting visualization/logging parameters
pd.set_option('display.max_columns', None)
qp.set_figure_params(dpi=100, color_map = 'viridis_r')
qp.settings.verbosity = 1
qp.logging.print_versions()
Here, we get the 505 S&P500 member companies listed on the wikipedia & get a list of fundamental metrics for each company from the TD Ameritrade API (All functions are available from the quanp tools).
# S&P 500 metadata
df_metadata = qp.datasets.get_wiki_sp500_metadata()
# # S&P 500 fundamentals
# df_fundamental = qp.datasets.download_tickers_fundamental()
# # Download End of Day (eod) price history from the current sp500 list
# qp.datasets.download_tickers_price_history_fromlist(df_metadata.index)
The metadata file has been initialized and saved as /Users/leehongkai/OneDrive/Finance/quanp/notebooks/data/metadata/sp500_metadata.csv
# Optional: The data retried in cell above were saved as csv file. You may execute this cell to avoid
# rerunning the downloading cell above.
df_fundamental = pd.read_csv('data/metadata/sp500_metadata_fundamentalAdded.csv', index_col=0)
print(df_fundamental.columns)
Index(['Security', 'SEC filings', 'GICS Sector', 'GICS Sub-Industry', 'Headquarters Location', 'Date first added', 'CIK', 'Founded', 'beta', 'bookValuePerShare', 'currentRatio', 'divGrowthRate3Year', 'dividendAmount', 'dividendDate', 'dividendPayAmount', 'dividendPayDate', 'dividendYield', 'epsChange', 'epsChangePercentTTM', 'epsChangeYear', 'epsTTM', 'grossMarginMRQ', 'grossMarginTTM', 'high52', 'interestCoverage', 'low52', 'ltDebtToEquity', 'marketCap', 'marketCapFloat', 'netProfitMarginMRQ', 'netProfitMarginTTM', 'operatingMarginMRQ', 'operatingMarginTTM', 'pbRatio', 'pcfRatio', 'peRatio', 'pegRatio', 'prRatio', 'quickRatio', 'returnOnAssets', 'returnOnEquity', 'returnOnInvestment', 'revChangeIn', 'revChangeTTM', 'revChangeYear', 'sharesOutstanding', 'shortIntDayToCover', 'shortIntToFloat', 'symbol', 'totalDebtToCapital', 'totalDebtToEquity', 'vol10DayAvg', 'vol1DayAvg', 'vol3MonthAvg'], dtype='object')
# prepare End of Day (eod) for each company
df_eod = qp.datasets.process_eod_price(df_metadata.index, startdate='2020-11-17', enddate='2020-12-18')
display(df_eod[:2])
display(df_eod[-2:])
BRK.B was excluded as the ticker csv was not found in the /Users/leehongkai/OneDrive/Finance/quanp/notebooks/data/daily BF.B was excluded as the ticker csv was not found in the /Users/leehongkai/OneDrive/Finance/quanp/notebooks/data/daily
MMM | ABT | ABBV | ABMD | ACN | ATVI | ADBE | AMD | AAP | AES | AFL | A | APD | AKAM | ALK | ALB | ARE | ALXN | ALGN | ALLE | LNT | ALL | GOOGL | GOOG | MO | AMZN | AMCR | AEE | AAL | AEP | AXP | AIG | AMT | AWK | AMP | ABC | AME | AMGN | APH | ADI | ANSS | ANTM | AON | AOS | APA | AAPL | AMAT | APTV | ADM | ANET | AJG | AIZ | T | ATO | ADSK | ADP | AZO | AVB | AVY | BKR | BLL | BAC | BK | BAX | BDX | BBY | BIO | BIIB | BLK | BA | BKNG | BWA | BXP | BSX | BMY | AVGO | BR | CHRW | COG | CDNS | CPB | COF | CAH | KMX | CCL | CARR | CTLT | CAT | CBOE | CBRE | CDW | CE | CNC | CNP | CERN | CF | SCHW | CHTR | CVX | CMG | CB | CHD | CI | CINF | CTAS | CSCO | C | CFG | CTXS | CLX | CME | CMS | KO | CTSH | CL | CMCSA | CMA | CAG | CXO | COP | ED | STZ | COO | CPRT | GLW | CTVA | COST | CCI | CSX | CMI | CVS | DHI | DHR | DRI | DVA | DE | DAL | XRAY | DVN | DXCM | FANG | DLR | DFS | DISCA | DISCK | DISH | DG | DLTR | D | DPZ | DOV | DOW | DTE | DUK | DRE | DD | DXC | EMN | ETN | EBAY | ECL | EIX | EW | EA | EMR | ETR | EOG | EFX | EQIX | EQR | ESS | EL | ETSY | EVRG | ES | RE | EXC | EXPE | EXPD | EXR | XOM | FFIV | FB | FAST | FRT | FDX | FIS | FITB | FE | FRC | FISV | FLT | FLIR | FLS | FMC | F | FTNT | FTV | FBHS | FOXA | FOX | BEN | FCX | GPS | GRMN | IT | GD | GE | GIS | GM | GPC | GILD | GL | GPN | GS | GWW | HAL | HBI | HIG | HAS | HCA | PEAK | HSIC | HSY | HES | HPE | HLT | HFC | HOLX | HD | HON | HRL | HST | HWM | HPQ | HUM | HBAN | HII | IEX | IDXX | INFO | ITW | ILMN | INCY | IR | INTC | ICE | IBM | IP | IPG | IFF | INTU | ISRG | IVZ | IPGP | IQV | IRM | JKHY | J | JBHT | SJM | JNJ | JCI | JPM | JNPR | KSU | K | KEY | KEYS | KMB | KIM | KMI | KLAC | KHC | KR | LB | LHX | LH | LRCX | LW | LVS | LEG | LDOS | LEN | LLY | LNC | LIN | LYV | LKQ | LMT | L | LOW | LUMN | LYB | MTB | MRO | MPC | MKTX | MAR | MMC | MLM | MAS | MA | MKC | MXIM | MCD | MCK | MDT | MRK | MET | MTD | MGM | MCHP | MU | MSFT | MAA | MHK | TAP | MDLZ | MNST | MCO | MS | MOS | MSI | MSCI | NDAQ | NOV | NTAP | NFLX | NWL | NEM | NWSA | NWS | NEE | NLSN | NKE | NI | NSC | NTRS | NOC | NLOK | NCLH | NRG | NUE | NVDA | NVR | ORLY | OXY | ODFL | OMC | OKE | ORCL | OTIS | PCAR | PKG | PH | PAYX | PAYC | PYPL | PNR | PBCT | PEP | PKI | PRGO | PFE | PM | PSX | PNW | PXD | PNC | POOL | PPG | PPL | PFG | PG | PGR | PLD | PRU | PEG | PSA | PHM | PVH | QRVO | PWR | QCOM | DGX | RL | RJF | RTX | O | REG | REGN | RF | RSG | RMD | RHI | ROK | ROL | ROP | ROST | RCL | SPGI | CRM | SBAC | SLB | STX | SEE | SRE | NOW | SHW | SPG | SWKS | SLG | SNA | SO | LUV | SWK | SBUX | STT | STE | SYK | SIVB | SYF | SNPS | SYY | TMUS | TROW | TTWO | TPR | TGT | TEL | FTI | TDY | TFX | TER | TSLA | TXN | TXT | TMO | TIF | TJX | TSCO | TT | TDG | TRV | TFC | TWTR | TYL | TSN | UDR | ULTA | USB | UAA | UA | UNP | UAL | UNH | UPS | URI | UHS | UNM | VLO | VAR | VTR | VRSN | VRSK | VZ | VRTX | VFC | VIAC | VTRS | V | VNT | VNO | VMC | WRB | WAB | WMT | WBA | DIS | WM | WAT | WEC | WFC | WELL | WST | WDC | WU | WRK | WY | WHR | WMB | WLTW | WYNN | XEL | XRX | XLNX | XYL | YUM | ZBRA | ZBH | ZION | ZTS | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2020-11-17 00:00:00+00:00 | 173.64 | 113.06 | 99.21 | 269.93 | 243.82 | 77.11 | 467.95 | 83.36 | 149.3 | 21.35 | 43.00 | 109.86 | 266.29 | 102.45 | 47.46 | 127.95 | 165.29 | 127.69 | 451.38 | 115.46 | 54.73 | 99.37 | 1761.66 | 1770.15 | 41.19 | 3135.66 | 11.93 | 80.60 | 12.70 | 86.73 | 116.48 | 38.42 | 239.18 | 159.29 | 184.23 | 104.89 | 118.76 | 232.44 | 127.06 | 135.87 | 319.79 | 329.84 | 207.19 | 56.38 | 11.44 | 119.39 | 74.37 | 114.44 | 50.16 | 277.78 | 117.08 | 138.59 | 29.03 | 100.05 | 252.71 | 176.15 | 1124.21 | 174.12 | 149.44 | 19.26 | 97.14 | 27.55 | 39.44 | 80.78 | 242.41 | 119.02 | 592.57 | 248.81 | 667.49 | 210.05 | 2081.25 | 38.00 | 98.46 | 35.03 | 64.32 | 380.60 | 147.28 | 93.55 | 17.18 | 113.48 | 49.38 | 87.76 | 54.68 | 94.73 | 18.05 | 40.36 | 104.97 | 171.91 | 86.80 | 60.49 | 136.78 | 132.41 | 67.28 | 25.17 | 74.77 | 32.22 | 48.9 | 641.51 | 87.04 | 1268.48 | 150.03 | 87.21 | 214.81 | 79.705 | 361.41 | 41.88 | 50.73 | 33.07 | 119.68 | 209.10 | 168.07 | 63.83 | 53.68 | 77.0 | 85.35 | 49.78 | 52.01 | 35.74 | 56.23 | 38.85 | 79.25 | 205.05 | 332.45 | 116.19 | 36.48 | 37.02 | 371.92 | 166.34 | 92.50 | 231.08 | 67.14 | 73.47 | 229.35 | 108.53 | 113.39 | 258.21 | 37.60 | 48.90 | 13.11 | 334.35 | 37.31 | 142.42 | 77.10 | 24.3 | 22.00 | 34.05 | 208.75 | 93.82 | 84.17 | 384.47 | 125.41 | 54.48 | 132.84 | 92.92 | 39.85 | 63.16 | 22.05 | 97.01 | 116.07 | 48.33 | 212.27 | 65.60 | 85.18 | 118.52 | 76.39 | 111.96 | 45.67 | 171.95 | 751.18 | 62.50 | 260.0 | 256.95 | 128.82 | 56.57 | 91.75 | 238.39 | 43.07 | 123.03 | 89.90 | 117.87 | 38.67 | 162.09 | 275.00 | 47.255 | 91.31 | 285.05 | 146.97 | 26.05 | 28.74 | 134.47 | 109.95 | 267.99 | 37.23 | 33.60 | 115.69 | 8.75 | 118.42 | 71.65 | 83.39 | 28.16 | 27.65 | 21.48 | 21.03 | 24.56 | 117.33 | 158.71 | 151.47 | 9.68 | 61.28 | 41.98 | 98.48 | 60.97 | 93.79 | 189.83 | 224.65 | 410.46 | 15.81 | 13.53 | 46.43 | 91.89 | 153.03 | 30.81 | 64.61 | 152.93 | 47.31 | 10.48 | 103.34 | 23.61 | 72.00 | 272.47 | 205.33 | 51.33 | 14.14 | 23.50 | 20.50 | 422.23 | 12.02 | 162.82 | 197.19 | 455.85 | 92.29 | 209.87 | 297.95 | 82.98 | 44.64 | 45.53 | 100.03 | 117.70 | 50.09 | 21.9 | 114.51 | 358.75 | 749.15 | 16.30 | 199.32 | 172.21 | 25.9 | 158.85 | 105.30 | 133.04 | 118.43 | 149.35 | 44.71 | 116.11 | 22.21 | 191.57 | 66.32 | 15.67 | 117.56 | 143.83 | 15.07 | 14.03 | 241.84 | 31.59 | 31.90 | 35.45 | 198.43 | 204.77 | 431.93 | 74.50 | 57.50 | 42.76 | 100.28 | 75.71 | 141.20 | 43.10 | 252.14 | 68.78 | 36.49 | 375.90 | 43.67 | 159.86 | 10.14 | 81.03 | 124.97 | 5.50 | 40.61 | 514.09 | 122.78 | 114.24 | 269.33 | 55.72 | 336.50 | 91.330 | 80.76 | 216.01 | 173.64 | 110.14 | 81.50 | 46.01 | 1193.36 | 26.42 | 127.37 | 61.99 | 214.46 | 130.91 | 131.15 | 45.13 | 58.54 | 85.54 | 274.69 | 58.81 | 18.96 | 169.57 | 403.64 | 124.33 | 12.20 | 51.40 | 480.63 | 20.17 | 64.53 | 17.685 | 17.81 | 76.61 | 16.65 | 132.21 | 24.75 | 245.64 | 93.79 | 312.63 | 18.89 | 22.06 | 31.89 | 53.86 | 536.89 | 4277.26 | 450.33 | 12.99 | 205.93 | 61.84 | 33.31 | 57.12 | 67.20 | 88.12 | 135.58 | 263.12 | 92.5 | 381.66 | 192.30 | 52.89 | 13.15 | 145.24 | 133.00 | 48.62 | 36.04 | 78.38 | 64.34 | 86.78 | 95.91 | 126.89 | 329.45 | 143.27 | 29.53 | 48.99 | 141.88 | 94.93 | 104.2 | 75.42 | 59.05 | 234.54 | 43.01 | 78.94 | 148.44 | 67.39 | 148.74 | 123.94 | 84.10 | 89.43 | 68.87 | 62.82 | 48.27 | 538.15 | 15.62 | 101.26 | 214.15 | 65.53 | 241.32 | 38.666667 | 405.65 | 110.46 | 75.34 | 336.66 | 256.17 | 298.78 | 20.09 | 55.51 | 44.62 | 134.06 | 513.10 | 731.08 | 78.96 | 140.78 | 60.26 | 172.58 | 62.36 | 45.66 | 184.53 | 98.60 | 70.56 | 195.21 | 231.44 | 348.34 | 30.57 | 221.20 | 74.23 | 129.00 | 141.53 | 162.20 | 28.46 | 163.04 | 111.18 | 8.07 | 394.17 | 375.50 | 105.91 | 441.61 | 155.50 | 45.60 | 479.00 | 131.46 | 61.14 | 131.68 | 148.79 | 577.97 | 135.04 | 48.17 | 42.85 | 413.67 | 63.78 | 40.28 | 276.20 | 44.03 | 16.10 | 14.17 | 205.85 | 40.60 | 352.10 | 168.88 | 214.62 | 133.27 | 21.4 | 54.15 | 173.62 | 50.49 | 194.51 | 205.28 | 60.75 | 222.96 | 85.11 | 32.29 | 16.34 | 210.71 | 30.64 | 41.49 | 140.54 | 70.11 | 71.39 | 149.37 | 39.855 | 144.5 | 121.30 | 231.19 | 100.18 | 25.04 | 65.95 | 286.79 | 42.72 | 21.83 | 43.61 | 29.18 | 191.07 | 20.83 | 210.13 | 94.23 | 72.63 | 22.16 | 130.41 | 95.95 | 104.04 | 340.79 | 150.15 | 39.60 | 165.43 |
2020-11-18 00:00:00+00:00 | 171.40 | 109.77 | 98.72 | 266.21 | 239.73 | 75.50 | 459.47 | 82.54 | 148.1 | 21.37 | 42.86 | 105.95 | 267.70 | 100.59 | 47.72 | 126.54 | 162.53 | 123.47 | 444.61 | 114.56 | 53.06 | 100.07 | 1740.64 | 1746.78 | 40.17 | 3105.46 | 11.84 | 79.06 | 12.74 | 84.13 | 114.24 | 38.55 | 237.70 | 156.19 | 183.33 | 103.44 | 119.24 | 227.76 | 125.80 | 133.23 | 317.99 | 322.38 | 203.36 | 56.49 | 11.04 | 118.03 | 75.71 | 115.05 | 49.27 | 272.03 | 115.45 | 135.62 | 28.62 | 96.17 | 252.22 | 173.16 | 1131.65 | 169.60 | 147.43 | 18.58 | 96.56 | 26.98 | 38.74 | 80.42 | 233.47 | 117.77 | 578.98 | 240.98 | 669.89 | 203.30 | 2044.60 | 38.36 | 96.12 | 34.26 | 62.01 | 380.62 | 145.77 | 92.88 | 16.74 | 111.03 | 49.04 | 85.61 | 53.32 | 92.82 | 17.59 | 39.39 | 103.14 | 172.74 | 88.66 | 58.13 | 135.73 | 133.21 | 65.96 | 23.75 | 72.92 | 32.45 | 47.7 | 642.44 | 84.28 | 1296.00 | 149.28 | 86.00 | 210.93 | 79.500 | 360.49 | 41.32 | 51.05 | 32.51 | 118.99 | 204.98 | 165.78 | 61.50 | 52.61 | 74.7 | 84.61 | 49.14 | 50.85 | 35.48 | 56.66 | 39.18 | 77.69 | 203.75 | 329.59 | 115.97 | 36.09 | 36.69 | 372.23 | 170.65 | 91.33 | 233.46 | 66.35 | 74.54 | 224.70 | 110.16 | 108.32 | 257.33 | 37.91 | 48.91 | 12.76 | 329.21 | 37.74 | 142.40 | 76.08 | 24.5 | 22.03 | 33.50 | 208.57 | 93.65 | 81.21 | 382.46 | 124.36 | 55.30 | 130.20 | 91.53 | 38.81 | 62.15 | 22.28 | 96.73 | 116.75 | 48.53 | 209.44 | 64.54 | 85.24 | 116.84 | 76.89 | 110.19 | 44.93 | 171.95 | 734.32 | 59.96 | 254.2 | 250.32 | 127.01 | 55.01 | 89.75 | 235.13 | 42.70 | 122.52 | 90.15 | 113.12 | 37.21 | 155.43 | 271.97 | 47.210 | 90.65 | 282.50 | 144.97 | 25.70 | 28.55 | 133.11 | 107.97 | 262.62 | 38.31 | 34.68 | 115.27 | 8.82 | 116.68 | 70.99 | 83.32 | 27.66 | 27.13 | 21.32 | 20.71 | 23.67 | 114.30 | 154.02 | 153.50 | 9.73 | 61.13 | 42.72 | 98.65 | 60.90 | 92.46 | 187.92 | 224.30 | 415.01 | 15.39 | 13.41 | 45.88 | 92.08 | 151.24 | 30.35 | 64.24 | 150.64 | 46.03 | 10.50 | 103.35 | 22.72 | 70.01 | 269.83 | 201.11 | 50.01 | 14.15 | 23.19 | 20.45 | 418.44 | 11.87 | 163.02 | 197.46 | 444.06 | 91.18 | 206.10 | 295.23 | 82.74 | 43.91 | 45.06 | 99.46 | 116.77 | 49.56 | 21.4 | 112.39 | 354.74 | 733.38 | 16.01 | 198.27 | 169.07 | 25.9 | 158.71 | 105.24 | 132.21 | 115.77 | 147.37 | 45.96 | 115.25 | 21.85 | 188.55 | 65.02 | 15.51 | 115.14 | 141.06 | 14.55 | 13.76 | 240.34 | 31.63 | 32.04 | 33.61 | 196.35 | 199.03 | 430.33 | 73.24 | 57.33 | 42.56 | 99.53 | 76.48 | 140.17 | 43.92 | 250.30 | 67.09 | 35.66 | 377.56 | 43.07 | 146.74 | 9.94 | 82.20 | 121.89 | 5.49 | 38.54 | 512.26 | 120.34 | 112.00 | 269.36 | 55.03 | 335.94 | 91.095 | 79.26 | 215.52 | 176.53 | 109.47 | 80.25 | 45.71 | 1144.81 | 26.91 | 124.85 | 60.74 | 211.08 | 126.07 | 128.25 | 43.56 | 57.76 | 84.22 | 272.62 | 58.81 | 19.19 | 167.45 | 399.95 | 122.73 | 11.94 | 51.54 | 481.79 | 20.20 | 61.56 | 17.550 | 17.63 | 75.79 | 16.21 | 131.63 | 23.96 | 241.87 | 92.69 | 310.60 | 18.84 | 20.46 | 31.34 | 52.92 | 537.15 | 4290.27 | 454.53 | 12.87 | 204.50 | 59.70 | 32.31 | 56.25 | 66.63 | 89.14 | 131.89 | 265.04 | 91.9 | 380.27 | 190.12 | 52.45 | 12.98 | 143.38 | 128.67 | 48.19 | 36.32 | 77.47 | 60.75 | 84.72 | 94.61 | 129.30 | 322.44 | 142.80 | 29.25 | 48.99 | 139.70 | 94.45 | 101.4 | 75.77 | 57.82 | 229.96 | 43.73 | 78.67 | 144.37 | 67.58 | 147.05 | 122.27 | 81.74 | 89.08 | 69.51 | 61.19 | 47.12 | 519.04 | 15.41 | 99.98 | 211.07 | 65.37 | 245.50 | 38.426667 | 408.37 | 111.30 | 74.13 | 336.48 | 257.16 | 298.52 | 19.97 | 55.24 | 44.17 | 133.04 | 507.12 | 723.61 | 76.61 | 137.56 | 57.22 | 171.21 | 61.78 | 45.60 | 184.27 | 97.91 | 69.18 | 191.94 | 231.76 | 347.50 | 30.08 | 218.19 | 71.68 | 127.85 | 139.79 | 161.56 | 27.98 | 166.85 | 110.31 | 8.28 | 391.87 | 370.17 | 104.82 | 486.64 | 153.55 | 46.01 | 454.18 | 131.37 | 62.28 | 128.42 | 144.44 | 587.87 | 134.37 | 47.45 | 43.33 | 408.94 | 63.45 | 38.62 | 274.21 | 43.26 | 16.27 | 14.37 | 204.03 | 41.05 | 344.51 | 168.45 | 216.25 | 132.61 | 21.4 | 51.25 | 173.26 | 48.21 | 194.26 | 203.90 | 60.32 | 216.35 | 85.56 | 32.70 | 16.19 | 207.83 | 31.19 | 39.79 | 140.81 | 68.91 | 72.63 | 149.09 | 38.230 | 143.9 | 120.53 | 224.69 | 96.75 | 25.06 | 63.83 | 281.29 | 42.49 | 21.61 | 43.78 | 28.78 | 194.17 | 20.00 | 205.70 | 95.64 | 70.50 | 21.87 | 129.08 | 94.98 | 104.17 | 337.58 | 148.19 | 38.94 | 163.50 |
MMM | ABT | ABBV | ABMD | ACN | ATVI | ADBE | AMD | AAP | AES | AFL | A | APD | AKAM | ALK | ALB | ARE | ALXN | ALGN | ALLE | LNT | ALL | GOOGL | GOOG | MO | AMZN | AMCR | AEE | AAL | AEP | AXP | AIG | AMT | AWK | AMP | ABC | AME | AMGN | APH | ADI | ANSS | ANTM | AON | AOS | APA | AAPL | AMAT | APTV | ADM | ANET | AJG | AIZ | T | ATO | ADSK | ADP | AZO | AVB | AVY | BKR | BLL | BAC | BK | BAX | BDX | BBY | BIO | BIIB | BLK | BA | BKNG | BWA | BXP | BSX | BMY | AVGO | BR | CHRW | COG | CDNS | CPB | COF | CAH | KMX | CCL | CARR | CTLT | CAT | CBOE | CBRE | CDW | CE | CNC | CNP | CERN | CF | SCHW | CHTR | CVX | CMG | CB | CHD | CI | CINF | CTAS | CSCO | C | CFG | CTXS | CLX | CME | CMS | KO | CTSH | CL | CMCSA | CMA | CAG | CXO | COP | ED | STZ | COO | CPRT | GLW | CTVA | COST | CCI | CSX | CMI | CVS | DHI | DHR | DRI | DVA | DE | DAL | XRAY | DVN | DXCM | FANG | DLR | DFS | DISCA | DISCK | DISH | DG | DLTR | D | DPZ | DOV | DOW | DTE | DUK | DRE | DD | DXC | EMN | ETN | EBAY | ECL | EIX | EW | EA | EMR | ETR | EOG | EFX | EQIX | EQR | ESS | EL | ETSY | EVRG | ES | RE | EXC | EXPE | EXPD | EXR | XOM | FFIV | FB | FAST | FRT | FDX | FIS | FITB | FE | FRC | FISV | FLT | FLIR | FLS | FMC | F | FTNT | FTV | FBHS | FOXA | FOX | BEN | FCX | GPS | GRMN | IT | GD | GE | GIS | GM | GPC | GILD | GL | GPN | GS | GWW | HAL | HBI | HIG | HAS | HCA | PEAK | HSIC | HSY | HES | HPE | HLT | HFC | HOLX | HD | HON | HRL | HST | HWM | HPQ | HUM | HBAN | HII | IEX | IDXX | INFO | ITW | ILMN | INCY | IR | INTC | ICE | IBM | IP | IPG | IFF | INTU | ISRG | IVZ | IPGP | IQV | IRM | JKHY | J | JBHT | SJM | JNJ | JCI | JPM | JNPR | KSU | K | KEY | KEYS | KMB | KIM | KMI | KLAC | KHC | KR | LB | LHX | LH | LRCX | LW | LVS | LEG | LDOS | LEN | LLY | LNC | LIN | LYV | LKQ | LMT | L | LOW | LUMN | LYB | MTB | MRO | MPC | MKTX | MAR | MMC | MLM | MAS | MA | MKC | MXIM | MCD | MCK | MDT | MRK | MET | MTD | MGM | MCHP | MU | MSFT | MAA | MHK | TAP | MDLZ | MNST | MCO | MS | MOS | MSI | MSCI | NDAQ | NOV | NTAP | NFLX | NWL | NEM | NWSA | NWS | NEE | NLSN | NKE | NI | NSC | NTRS | NOC | NLOK | NCLH | NRG | NUE | NVDA | NVR | ORLY | OXY | ODFL | OMC | OKE | ORCL | OTIS | PCAR | PKG | PH | PAYX | PAYC | PYPL | PNR | PBCT | PEP | PKI | PRGO | PFE | PM | PSX | PNW | PXD | PNC | POOL | PPG | PPL | PFG | PG | PGR | PLD | PRU | PEG | PSA | PHM | PVH | QRVO | PWR | QCOM | DGX | RL | RJF | RTX | O | REG | REGN | RF | RSG | RMD | RHI | ROK | ROL | ROP | ROST | RCL | SPGI | CRM | SBAC | SLB | STX | SEE | SRE | NOW | SHW | SPG | SWKS | SLG | SNA | SO | LUV | SWK | SBUX | STT | STE | SYK | SIVB | SYF | SNPS | SYY | TMUS | TROW | TTWO | TPR | TGT | TEL | FTI | TDY | TFX | TER | TSLA | TXN | TXT | TMO | TIF | TJX | TSCO | TT | TDG | TRV | TFC | TWTR | TYL | TSN | UDR | ULTA | USB | UAA | UA | UNP | UAL | UNH | UPS | URI | UHS | UNM | VLO | VAR | VTR | VRSN | VRSK | VZ | VRTX | VFC | VIAC | VTRS | V | VNT | VNO | VMC | WRB | WAB | WMT | WBA | DIS | WM | WAT | WEC | WFC | WELL | WST | WDC | WU | WRK | WY | WHR | WMB | WLTW | WYNN | XEL | XRX | XLNX | XYL | YUM | ZBRA | ZBH | ZION | ZTS | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2020-12-17 00:00:00+00:00 | 176.65 | 108.78 | 104.89 | 281.99 | 264.47 | 88.46 | 495.36 | 96.84 | 160.34 | 22.09 | 44.81 | 118.97 | 268.60 | 105.61 | 50.17 | 137.48 | 177.70 | 157.93 | 526.72 | 112.82 | 51.73 | 106.56 | 1740.51 | 1747.90 | 43.00 | 3236.08 | 11.56 | 78.74 | 16.80 | 83.86 | 118.98 | 38.23 | 222.71 | 151.86 | 190.05 | 98.74 | 117.63 | 231.24 | 132.17 | 143.78 | 346.63 | 314.08 | 211.30 | 55.26 | 15.75 | 128.700 | 87.50 | 124.89 | 49.38 | 285.37 | 128.19 | 132.37 | 29.62 | 99.09 | 297.35 | 177.34 | 1180.89 | 163.23 | 152.23 | 21.50 | 91.98 | 28.83 | 40.41 | 81.04 | 248.70 | 102.34 | 584.24 | 250.24 | 706.01 | 221.24 | 2113.43 | 39.00 | 100.42 | 35.39 | 62.54 | 426.10 | 150.83 | 92.29 | 17.23 | 127.30 | 47.78 | 94.08 | 54.22 | 99.79 | 21.63 | 38.70 | 103.62 | 179.17 | 89.54 | 64.41 | 135.24 | 130.96 | 62.09 | 21.88 | 78.16 | 37.54 | 51.78 | 653.98 | 88.41 | 1409.68 | 153.41 | 87.21 | 202.63 | 84.29 | 352.00 | 44.82 | 60.05 | 34.78 | 130.74 | 202.34 | 184.94 | 59.51 | 53.27 | 81.73 | 85.66 | 51.36 | 52.77 | 36.17 | 61.07 | 41.97 | 71.62 | 217.80 | 351.52 | 121.68 | 36.59 | 39.68 | 370.29 | 155.75 | 90.55 | 223.43 | 69.59 | 73.47 | 224.49 | 118.62 | 115.73 | 264.53 | 41.11 | 53.27 | 16.10 | 357.91 | 47.385 | 133.93 | 87.48 | 28.74 | 25.31 | 31.41 | 212.55 | 113.05 | 75.93 | 395.16 | 123.76 | 53.83 | 123.55 | 91.52 | 40.45 | 69.49 | 23.39 | 102.50 | 115.29 | 53.05 | 220.61 | 62.90 | 86.52 | 139.85 | 81.16 | 100.56 | 52.15 | 193.75 | 712.74 | 59.64 | 241.85 | 263.83 | 188.10 | 54.04 | 85.78 | 235.16 | 42.65 | 124.48 | 94.03 | 114.17 | 43.48 | 174.50 | 274.48 | 49.57 | 90.73 | 292.26 | 145.97 | 27.13 | 30.50 | 138.10 | 114.70 | 282.01 | 41.75 | 37.61 | 115.62 | 9.08 | 136.42 | 69.33 | 88.89 | 28.54 | 28.16 | 25.16 | 24.72 | 20.75 | 118.63 | 157.05 | 151.59 | 10.88 | 59.7 | 42.03 | 96.95 | 59.07 | 93.43 | 195.42 | 244.43 | 410.04 | 20.08 | 14.17 | 46.78 | 93.21 | 164.75 | 30.49 | 68.89 | 151.61 | 56.67 | 12.18 | 104.70 | 25.64 | 75.10 | 273.97 | 211.94 | 47.77 | 14.29 | 26.96 | 24.22 | 396.34 | 12.34 | 170.47 | 197.88 | 477.80 | 87.68 | 206.11 | 361.30 | 88.28 | 45.44 | 50.65 | 112.79 | 125.55 | 49.75 | 23.80 | 113.58 | 380.32 | 779.78 | 17.48 | 217.02 | 178.95 | 29.59 | 162.76 | 107.00 | 138.71 | 115.32 | 153.62 | 46.45 | 119.67 | 22.23 | 196.62 | 62.48 | 15.58 | 126.86 | 136.32 | 15.08 | 14.30 | 259.68 | 34.62 | 31.00 | 39.19 | 185.71 | 202.69 | 486.24 | 78.64 | 57.34 | 41.81 | 104.83 | 79.95 | 172.63 | 49.46 | 256.63 | 73.98 | 37.12 | 352.60 | 44.41 | 163.54 | 10.25 | 90.10 | 125.83 | 6.91 | 40.58 | 577.35 | 130.29 | 118.69 | 265.73 | 55.28 | 336.90 | 94.60 | 85.84 | 214.25 | 172.91 | 115.34 | 79.82 | 46.49 | 1130.33 | 31.12 | 139.72 | 72.26 | 219.42 | 125.59 | 135.25 | 45.97 | 57.93 | 90.14 | 280.06 | 64.67 | 23.10 | 169.65 | 436.78 | 128.5 | 13.93 | 64.12 | 532.90 | 20.09 | 61.64 | 17.71 | 17.59 | 75.06 | 19.61 | 140.50 | 22.81 | 230.69 | 92.1 | 300.68 | 20.56 | 25.41 | 33.54 | 55.45 | 533.65 | 4219.80 | 451.83 | 19.18 | 196.06 | 63.31 | 40.73 | 63.61 | 65.79 | 85.07 | 135.65 | 273.3 | 95.65 | 441.80 | 235.51 | 52.77 | 12.89 | 145.71 | 143.42 | 46.97 | 38.03 | 85.67 | 67.17 | 79.32 | 114.38 | 145.02 | 356.15 | 144.29 | 27.35 | 48.40 | 138.25 | 96.31 | 100.58 | 77.66 | 57.19 | 232.56 | 44.78 | 96.40 | 159.60 | 68.82 | 149.53 | 120.72 | 100.99 | 94.77 | 70.64 | 61.03 | 47.49 | 496.21 | 15.56 | 96.89 | 213.73 | 62.35 | 247.19 | 40.21 | 425.04 | 116.69 | 73.94 | 321.06 | 225.92 | 282.04 | 22.93 | 66.21 | 44.79 | 131.74 | 554.07 | 726.25 | 89.03 | 145.53 | 60.17 | 171.16 | 61.07 | 46.39 | 182.09 | 103.21 | 70.74 | 182.64 | 238.1 | 364.86 | 33.30 | 250.29 | 74.44 | 130.01 | 151.30 | 197.40 | 29.83 | 171.24 | 116.16 | 9.69 | 380.93 | 402.9 | 119.35 | 655.9 | 162.69 | 46.99 | 464.94 | 131.22 | 67.51 | 145.51 | 140.97 | 598.35 | 138.03 | 47.1 | 54.63 | 455.28 | 65.00 | 38.39 | 272.78 | 45.28 | 17.42 | 15.23 | 202.2 | 45.88 | 341.67 | 174.49 | 229.16 | 139.73 | 22.13 | 56.34 | 174.41 | 51.49 | 217.93 | 201.24 | 60.01 | 236.92 | 86.55 | 36.20 | 17.72 | 211.18 | 33.50 | 38.38 | 139.54 | 66.74 | 73.53 | 146.10 | 40.88 | 173.55 | 117.51 | 250.74 | 92.22 | 29.47 | 65.95 | 273.58 | 53.46 | 23.08 | 44.98 | 34.46 | 190.75 | 21.80 | 212.23 | 114.81 | 67.01 | 23.13 | 151.86 | 99.75 | 108.79 | 375.43 | 151.36 | 41.87 | 162.70 |
2020-12-18 00:00:00+00:00 | 176.42 | 108.97 | 104.45 | 293.39 | 266.25 | 90.37 | 502.95 | 95.92 | 159.99 | 22.54 | 43.89 | 119.30 | 269.55 | 107.08 | 50.35 | 138.90 | 175.26 | 156.75 | 529.92 | 113.75 | 51.05 | 107.40 | 1726.22 | 1731.01 | 43.78 | 3201.65 | 11.66 | 78.86 | 16.51 | 83.29 | 117.51 | 37.48 | 221.52 | 150.24 | 186.94 | 98.61 | 118.10 | 228.49 | 130.65 | 144.56 | 357.81 | 308.64 | 205.49 | 55.64 | 15.33 | 126.655 | 86.09 | 124.30 | 49.68 | 287.72 | 125.32 | 131.29 | 29.40 | 95.11 | 305.00 | 179.24 | 1181.90 | 158.44 | 152.96 | 21.17 | 92.19 | 28.67 | 40.79 | 80.86 | 252.09 | 101.95 | 605.87 | 249.62 | 698.80 | 219.75 | 2099.65 | 38.62 | 96.11 | 35.45 | 63.12 | 434.56 | 153.88 | 92.60 | 16.97 | 132.02 | 47.57 | 91.57 | 54.96 | 98.25 | 21.46 | 38.79 | 105.00 | 180.96 | 90.78 | 63.29 | 134.37 | 131.76 | 61.15 | 21.87 | 77.60 | 37.64 | 51.98 | 652.00 | 87.19 | 1418.57 | 152.36 | 88.15 | 197.08 | 83.04 | 354.35 | 45.44 | 59.06 | 34.21 | 132.65 | 203.17 | 183.98 | 59.23 | 53.74 | 81.02 | 85.93 | 50.90 | 52.63 | 36.12 | 59.56 | 40.98 | 71.15 | 218.68 | 349.63 | 122.36 | 36.24 | 39.41 | 367.00 | 153.81 | 90.66 | 224.31 | 69.55 | 73.23 | 226.11 | 116.16 | 115.36 | 270.70 | 40.68 | 52.60 | 15.63 | 358.57 | 45.840 | 134.02 | 85.99 | 27.80 | 24.52 | 30.49 | 210.48 | 111.88 | 75.37 | 396.71 | 123.86 | 55.66 | 121.70 | 90.32 | 39.24 | 71.33 | 23.10 | 103.68 | 116.08 | 53.06 | 218.58 | 62.54 | 86.82 | 142.61 | 80.90 | 100.02 | 51.77 | 195.17 | 709.73 | 57.65 | 234.37 | 261.31 | 190.76 | 53.35 | 84.71 | 232.53 | 42.57 | 125.54 | 95.40 | 112.49 | 42.73 | 175.23 | 276.40 | 49.34 | 88.31 | 275.57 | 145.22 | 26.63 | 30.34 | 135.78 | 114.81 | 279.47 | 42.02 | 37.28 | 115.65 | 8.95 | 145.85 | 69.83 | 89.63 | 28.16 | 27.79 | 25.01 | 24.63 | 20.21 | 119.49 | 157.94 | 152.67 | 10.81 | 59.4 | 41.01 | 97.53 | 59.08 | 92.81 | 196.33 | 242.13 | 410.46 | 19.68 | 14.43 | 47.65 | 93.81 | 164.34 | 29.83 | 67.78 | 150.88 | 55.95 | 12.12 | 104.61 | 25.17 | 74.77 | 270.45 | 210.04 | 47.94 | 14.09 | 27.27 | 24.46 | 391.62 | 12.39 | 169.62 | 198.55 | 485.52 | 87.98 | 203.88 | 366.06 | 88.34 | 45.22 | 47.46 | 112.83 | 125.85 | 49.98 | 23.94 | 114.29 | 380.11 | 789.09 | 17.45 | 216.82 | 180.39 | 29.48 | 161.63 | 106.58 | 140.46 | 115.88 | 154.51 | 46.24 | 119.08 | 22.68 | 197.62 | 62.08 | 15.42 | 128.26 | 135.08 | 14.62 | 14.21 | 262.15 | 34.77 | 30.87 | 38.63 | 187.21 | 205.11 | 480.97 | 78.66 | 57.43 | 41.75 | 105.67 | 80.84 | 171.40 | 48.61 | 257.67 | 73.43 | 36.15 | 356.03 | 43.80 | 163.20 | 10.16 | 90.93 | 123.73 | 6.73 | 40.35 | 578.88 | 128.82 | 116.27 | 276.35 | 55.59 | 340.33 | 93.37 | 86.45 | 215.08 | 173.55 | 116.71 | 79.53 | 45.65 | 1134.83 | 31.67 | 138.01 | 71.46 | 218.59 | 124.04 | 138.43 | 45.51 | 58.32 | 90.13 | 283.37 | 64.18 | 23.39 | 170.26 | 443.06 | 128.6 | 13.70 | 64.82 | 534.45 | 20.60 | 60.50 | 17.35 | 17.29 | 74.51 | 20.09 | 137.28 | 22.40 | 232.39 | 91.9 | 303.13 | 21.24 | 25.18 | 33.79 | 53.93 | 530.88 | 4226.87 | 451.15 | 18.52 | 199.54 | 61.62 | 39.87 | 65.06 | 65.61 | 86.01 | 134.86 | 272.0 | 96.92 | 439.84 | 236.45 | 52.26 | 12.76 | 146.93 | 142.70 | 46.09 | 37.68 | 86.09 | 66.85 | 78.65 | 114.36 | 144.91 | 364.37 | 145.43 | 26.95 | 47.95 | 139.04 | 96.95 | 98.07 | 76.51 | 56.82 | 228.93 | 45.10 | 95.26 | 162.66 | 68.29 | 147.42 | 120.36 | 101.21 | 93.86 | 70.17 | 59.52 | 45.93 | 498.89 | 15.26 | 96.78 | 218.45 | 63.05 | 247.95 | 40.60 | 426.19 | 117.66 | 72.29 | 322.50 | 227.43 | 276.93 | 22.31 | 64.72 | 45.41 | 130.70 | 565.43 | 728.23 | 85.43 | 148.19 | 58.11 | 169.86 | 60.14 | 46.03 | 181.52 | 103.28 | 70.66 | 184.90 | 240.0 | 357.80 | 32.35 | 255.00 | 72.28 | 131.92 | 150.44 | 203.39 | 28.86 | 170.42 | 117.49 | 9.48 | 380.47 | 408.9 | 120.59 | 695.0 | 164.07 | 47.37 | 466.68 | 131.10 | 67.70 | 147.08 | 141.06 | 608.33 | 138.86 | 45.9 | 55.87 | 448.81 | 64.95 | 37.01 | 268.43 | 44.86 | 17.37 | 15.10 | 203.6 | 44.74 | 338.38 | 175.18 | 227.54 | 137.29 | 21.63 | 54.77 | 174.80 | 50.83 | 217.82 | 204.47 | 60.46 | 236.72 | 84.88 | 35.26 | 17.75 | 211.31 | 33.91 | 36.71 | 145.22 | 66.28 | 73.44 | 145.95 | 40.69 | 172.89 | 117.41 | 250.38 | 91.32 | 29.01 | 64.02 | 277.66 | 53.19 | 22.77 | 44.62 | 33.74 | 188.37 | 21.18 | 204.89 | 115.93 | 66.10 | 22.86 | 149.19 | 100.05 | 109.78 | 373.00 | 151.85 | 41.08 | 163.63 |
# calculate the 1-month log return of the stock price for each company, i.e. price difference
# between Nov 17 and Dec 18, 2020.
df_log_return_1mth = pd.DataFrame(np.log(df_eod.iloc[-1]) - np.log(df_eod.iloc[0]),
columns=['log_return_1mth'])
# merging fundamental data with the log_return data
df_fundamental_logreturn = pd.merge(df_fundamental, df_log_return_1mth, how='inner', left_index=True,
right_index=True)
# describe the 1-month log return of all sp500 companies.
df_fundamental_logreturn['log_return_1mth'].describe()
count 503.000000 mean 0.034189 std 0.078201 min -0.194942 25% -0.015173 50% 0.021695 75% 0.068664 max 0.453485 Name: log_return_1mth, dtype: float64
ls_fundamental_target = ['log_return_1mth'] + ['beta','bookValuePerShare','currentRatio', 'quickRatio',
'dividendYield', 'epsChangePercentTTM', 'epsChangeYear', 'epsTTM', 'grossMarginMRQ', 'grossMarginTTM',
'interestCoverage', 'marketCap', 'marketCapFloat', 'netProfitMarginMRQ','netProfitMarginTTM',
'operatingMarginMRQ', 'operatingMarginTTM', 'peRatio', 'pegRatio', 'pbRatio', 'pcfRatio',
'prRatio', 'returnOnAssets', 'returnOnEquity', 'returnOnInvestment', 'revChangeIn', 'revChangeTTM',
'revChangeYear', 'sharesOutstanding', 'shortIntDayToCover', 'shortIntToFloat', 'totalDebtToCapital',
'totalDebtToEquity', 'ltDebtToEquity', 'vol10DayAvg', 'vol1DayAvg', 'vol3MonthAvg']
from sklearn.preprocessing import MinMaxScaler
# Minmax scaling (0, 1) for all the variables/features
scaler = MinMaxScaler((0, 1))
df_fundamental_logreturn_minmax = pd.DataFrame(scaler.fit_transform(df_fundamental_logreturn[ls_fundamental_target]),
index=df_fundamental_logreturn.index,
columns=ls_fundamental_target)
df_fundamental_logreturn_minmax[ls_fundamental_target].describe()
log_return_1mth | beta | bookValuePerShare | currentRatio | quickRatio | dividendYield | epsChangePercentTTM | epsChangeYear | epsTTM | grossMarginMRQ | grossMarginTTM | interestCoverage | marketCap | marketCapFloat | netProfitMarginMRQ | netProfitMarginTTM | operatingMarginMRQ | operatingMarginTTM | peRatio | pegRatio | pbRatio | pcfRatio | prRatio | returnOnAssets | returnOnEquity | returnOnInvestment | revChangeIn | revChangeTTM | revChangeYear | sharesOutstanding | shortIntDayToCover | shortIntToFloat | totalDebtToCapital | totalDebtToEquity | ltDebtToEquity | vol10DayAvg | vol1DayAvg | vol3MonthAvg | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
count | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 5.030000e+02 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.0 | 503.000000 | 503.0 | 503.0 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 | 503.000000 |
mean | 0.353365 | 0.226347 | 0.056875 | 0.145271 | 0.114280 | 0.184364 | 1.989590e-03 | 0.016070 | 0.024672 | 0.403637 | 0.412475 | 0.006728 | 0.029232 | 0.035193 | 0.067435 | 0.085000 | 0.267448 | 0.239645 | 0.010920 | 0.005163 | 0.011726 | 0.033014 | 0.133036 | 0.121769 | 0.009609 | 0.087929 | 0.039743 | 0.058798 | 0.0 | 0.036025 | 0.0 | 0.0 | 0.106306 | 0.013118 | 0.011910 | 0.047535 | 0.047751 | 0.042469 |
std | 0.120602 | 0.116664 | 0.079418 | 0.119241 | 0.125906 | 0.176910 | 4.458773e-02 | 0.056459 | 0.057981 | 0.259537 | 0.264081 | 0.051631 | 0.079591 | 0.071170 | 0.067395 | 0.086340 | 0.195124 | 0.183059 | 0.047321 | 0.051840 | 0.051304 | 0.055139 | 0.138410 | 0.134736 | 0.059772 | 0.108005 | 0.081471 | 0.126125 | 0.0 | 0.071726 | 0.0 | 0.0 | 0.090536 | 0.051405 | 0.050613 | 0.098145 | 0.098113 | 0.089361 |
min | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.0 | 0.000000 | 0.0 | 0.0 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
25% | 0.277238 | 0.157283 | 0.000000 | 0.078093 | 0.000000 | 0.021329 | 0.000000e+00 | 0.000000 | 0.005598 | 0.208339 | 0.217849 | 0.000187 | 0.005001 | 0.008167 | 0.025067 | 0.028863 | 0.119914 | 0.103099 | 0.003518 | 0.000000 | 0.001694 | 0.013778 | 0.044892 | 0.021036 | 0.000780 | 0.002025 | 0.003512 | 0.000000 | 0.0 | 0.008425 | 0.0 | 0.0 | 0.064701 | 0.002557 | 0.002164 | 0.010087 | 0.010129 | 0.008998 |
50% | 0.334096 | 0.217107 | 0.039281 | 0.127539 | 0.097960 | 0.147817 | 0.000000e+00 | 0.001366 | 0.015117 | 0.405001 | 0.411348 | 0.001212 | 0.010134 | 0.016803 | 0.057756 | 0.069392 | 0.246738 | 0.220733 | 0.006269 | 0.000000 | 0.003511 | 0.022887 | 0.085881 | 0.083621 | 0.002106 | 0.057268 | 0.017471 | 0.000000 | 0.0 | 0.017106 | 0.0 | 0.0 | 0.096443 | 0.004915 | 0.004538 | 0.019402 | 0.019527 | 0.017681 |
75% | 0.406531 | 0.285650 | 0.074949 | 0.195404 | 0.162827 | 0.290675 | 8.397892e-07 | 0.011987 | 0.027749 | 0.607227 | 0.612949 | 0.003111 | 0.024011 | 0.034315 | 0.096650 | 0.118865 | 0.379776 | 0.339604 | 0.009728 | 0.001298 | 0.007707 | 0.035563 | 0.170012 | 0.171502 | 0.004450 | 0.124267 | 0.041234 | 0.070568 | 0.0 | 0.034554 | 0.0 | 0.0 | 0.135109 | 0.009902 | 0.008895 | 0.043851 | 0.043874 | 0.040277 |
max | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000e+00 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.0 | 1.000000 | 0.0 | 0.0 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 |
# Loading pandas dataframe as anndata
adata = qp.AnnData(df_fundamental_logreturn_minmax[ls_fundamental_target])
# Saving raw data for visualization later
adata.raw = adata
# log(x+1) transformation for all data
qp.pp.log1p(adata)
# Standardization scaling per feature
qp.pp.scale(adata)
# add a new `.obs` column for all comapnanies called `GICS_Sector`
adata.obs['GICS_Sector'] = df_fundamental_logreturn['GICS Sector'].values
adata
AnnData object with n_obs × n_vars = 503 × 38 obs: 'GICS_Sector' var: 'mean', 'std' uns: 'log1p'
Reduce the dimensionality of the data by running PCA, which reveals the main axes of variation and denoises the data.
Let us inspect the contribution of single PCs to the total variance in the data. This gives us information about how many PCs we should consider in order to compute the neighborhood relations of cells, e.g. used in the clustering function qp.tl.leiden()
, qp.tl.louvain()
, or tSNE qp.tl.tsne()
. In our experience, often, a rough estimate of the number of PCs does fine. The 'elbow' point seems to suggest at least up to PC5 will be useful to characterize the companies. We are going to do further dimensional reduction based on the first 6 PCs later. We will perform a factor analysis to check for the underlying features that made up these PCs in a separate tutorial notebook later.
rcParams['figure.figsize'] = 5,3
qp.tl.pca(adata, svd_solver='arpack', random_state=42)
qp.pl.pca_variance_ratio(adata, n_pcs=len(adata.var_names))
# Optional: save the anndata in h5ad for reloading of the processed data later
adata.write('data/sp500_1mthlogreturn_afterVaccineSuccess.h5ad')
# adata.read('data/sp500_1mthlogreturn_afterVaccineSuccess.h5ad')
adata
/Users/leehongkai/anaconda/envs/quanp_py36/lib/python3.6/site-packages/anndata/_core/anndata.py:1192: FutureWarning: is_categorical is deprecated and will be removed in a future version. Use is_categorical_dtype instead ... storing 'GICS_Sector' as categorical
AnnData object with n_obs × n_vars = 503 × 38 obs: 'GICS_Sector' var: 'mean', 'std' uns: 'log1p', 'pca' obsm: 'X_pca' varm: 'PCs'
Let us further reduce the dimensionality of the signficant PCs identified above wholly in to 2 dimensions using the tSNE tool implemented as qp.tl.tsne(adata)
.
qp.tl.tsne(adata, n_pcs=5, random_state=42); # only consider the first 8 pcs
WARNING: Consider installing the package MulticoreTSNE (https://github.com/DmitryUlyanov/Multicore-TSNE). Even for n_jobs=1 this speeds up the computation considerably and might yield better converged results.
Before we view the tsne plots with Sector annotations, Let us compute the neighborhood graph of companies using the PCA representation of the data matrix. This will give rise to distances and connectivities in each company. Here, we consider 10 nearest neighbors with 5 PCs derived from the PCA
qp.pp.neighbors(adata, n_neighbors=10, n_pcs=5, random_state=42);
Here, we use Leiden graph-clustering method (community detection based on optimizing modularity) by Traag et al. (2018) to cluster the neighborhood graph of companies, which we already computed in the previous section.
qp.tl.leiden(adata)
We can now map and view the annotations of leiden clustering, GICS_Sector, or any financial metrics/features on the tsne plots. We can see that companies from the Leiden Clusters 8 and 9 seems to perform the best 1 month post-vaccine-success, while the worst performers seem to be ones from the Clusters 7 and 6.
# # Check data distribution of the log_return_1mth to set a appropriate vmin cutoff
df_fundamental_logreturn_minmax['log_return_1mth'].describe()
count 503.000000 mean 0.353365 std 0.120602 min 0.000000 25% 0.277238 50% 0.334096 75% 0.406531 max 1.000000 Name: log_return_1mth, dtype: float64
rcParams['figure.figsize'] = 8,8
qp.pl.tsne(adata, color=['leiden', 'GICS_Sector', 'log_return_1mth'],
vmin=0.277238, legend_loc='on data', use_raw=True, legend_fontsize=20)
We can also embed the neighborhood graph in 2 dimensions using UMAP (McInnes et al., 2018), see below. It is potentially more faithful to the global connectivity of the manifold than tSNE. Before running the UMAP, we compute the correlations between clusters as initiating positions for the UMAP.
rcParams['figure.figsize'] = 5,5
qp.tl.paga(adata)
qp.pl.paga(adata, color=['GICS_Sector'], plot=True)
We can now map and view the annotations of leiden clustering or any financial metrics/features on the umap plots. Again, we can see that companies from the Leiden Clusters 8, 9, and 3 seems to perform the best 1 month post-vaccine-success, while the worst performers seem to be ones from the Clusters 6 and 7.
rcParams['figure.figsize'] = 8,8
qp.tl.umap(adata, init_pos='paga', random_state=42)
qp.pl.umap(adata, color=['leiden', 'log_return_1mth'],
legend_loc='on data', frameon=False, ncols=4,
vmin=0.277238, vmax=1.0, use_raw=True, legend_fontsize=20)
We run qp.tl.dendrogram
to compute hierarchical clustering. Multiple visualizations that can
then include a dendrogram: qp.pl.matrixplot
, qp.pl.heatmap
, qp.pl.dotplot
and qp.pl.stacked_violin
.
qp.tl.dendrogram(adata, 'leiden', var_names=adata.var_names);
qp.pl.matrixplot(adata, var_names=adata.var_names, groupby='leiden', use_raw=True,
cmap='RdBu_r', dendrogram=True, standard_scale='var')
The matrixplot shows the median for each cluster/group of companies. Again, we are seeing Cluster 8 was the top performer in 'log_return_1mth', followed by Cluster 9, Cluster 3, and Cluster 7. Besides, we can also see that the 'beta' was high in both Cluster 8. In order to confirm the results, we are printing out the exact median values of log_return_1mth for each cluster/group of companies in dataframe.
df_leiden = pd.DataFrame(adata.obs['leiden'])
df_all = pd.merge(df_fundamental_logreturn, df_leiden, how='inner', left_index=True, right_index=True)
# sort clusters by median log_return_1mth
stats_perCluster = df_all.groupby(['leiden']).agg({'log_return_1mth': ['median', 'min',
'max', 'mean']}).dropna()
stats_perCluster.columns = ['median', 'min', 'max', 'mean']
stats_perCluster.sort_values(['median'], ascending=False).style.background_gradient(cmap='bwr', axis=0)
median | min | max | mean | |
---|---|---|---|---|
leiden | ||||
8 | 0.103931 | -0.041325 | 0.354671 | 0.101642 |
9 | 0.095243 | -0.031522 | 0.392600 | 0.104538 |
3 | 0.065420 | -0.023377 | 0.453485 | 0.090324 |
7 | 0.042159 | -0.062765 | 0.188069 | 0.042714 |
10 | 0.033966 | -0.036846 | 0.193823 | 0.045277 |
0 | 0.029478 | -0.075752 | 0.205872 | 0.032342 |
2 | 0.018952 | -0.154809 | 0.231976 | 0.032441 |
4 | 0.018178 | -0.194942 | 0.176049 | 0.014633 |
1 | 0.011318 | -0.140537 | 0.152153 | 0.005704 |
11 | 0.000286 | -0.118999 | 0.044832 | -0.012876 |
5 | -0.021796 | -0.112771 | 0.129469 | -0.012337 |
6 | -0.032866 | -0.122403 | 0.087380 | -0.022444 |
Dataframe above shows that the median of 'log_return_1mth' was the highest in Leiden Cluster 8, while the lowest were Cluster 2. To confirm the 'beta', we did the similar in the following cell. We typically dislike high beta as it usually means the company is more risky.
# sort clusters by median beta
stats_perCluster = df_all.groupby(['leiden']).agg({'beta': ['median', 'min',
'max', 'mean']}).dropna()
stats_perCluster.columns = ['median', 'min', 'max', 'mean']
stats_perCluster.sort_values(['median'], ascending=False).style.background_gradient(cmap='bwr', axis=0)
median | min | max | mean | |
---|---|---|---|---|
leiden | ||||
8 | 1.818040 | 1.006990 | 2.895120 | 1.907955 |
3 | 1.360690 | 0.791820 | 4.900250 | 1.575118 |
2 | 1.178360 | 0.000000 | 2.259150 | 1.220481 |
4 | 1.082690 | 0.467950 | 2.253310 | 1.183186 |
0 | 1.060175 | 0.000000 | 1.584150 | 0.997718 |
1 | 1.026930 | 0.119030 | 2.247090 | 1.085153 |
6 | 0.993320 | 0.076360 | 1.737620 | 0.930215 |
7 | 0.974160 | 0.273320 | 1.831970 | 0.944492 |
9 | 0.856785 | 0.000000 | 1.658140 | 0.870808 |
11 | 0.852880 | 0.182590 | 1.539040 | 0.965820 |
10 | 0.819580 | 0.353980 | 1.593090 | 0.857501 |
5 | 0.421335 | 0.000000 | 1.638070 | 0.492284 |
Instead of looking at all features of clusters as previously, we can identify features/metrics that are differentially characterizing each cluster. Here, we can see that the Cluster 8 was significantly positive-correlated with higher beta, volatility 3-month average (vol3MonthAvg), volatility 10-day average (vol10DayAvg), vol1DayAvg, but negatively associated with lower earning per share Trailing-twelve-month (epsTTM), price per earning ration (peRatio), operatingMarginTTM, netProfitMarginTTM, returnOnAssets, etc.
The worst performer, Cluster 6, was associated with higher dividendYield, grossMarginMRQ, grossMarginTTM, TotalDebtToEquity, etc, and lower current, quick ratio, returnOnAssets, etc.
Here, the author is particularly interested in the companies from the Cluster 9, which associated with lower beta, that typically means that the stocks in this cluster are considered less risky. Also, the Cluster 9 were associated with favorable higher Operating Profitability Factors ('netProfitMarginTTM', 'netProfitMarginMRQ', 'grossMarginTTM', 'grossMarginMRQ', 'returnOnInvestment', 'returnOnAssets', 'operatingMarginTTM', 'operatingMarginMRQ') and lower Financial Risk Factors ('ITDebtToEquity' and 'TotalDebtToEquity')
qp.tl.rank_features_groups(adata, 'leiden', groups=['8', '9', '7', '6'], method='wilcoxon')
qp.pl.rank_features_groups(adata, n_features=38, sharey=False, fontsize=10, ncols=2)
We can map and view the annotations of leiden clustering based on the financial metrics/features on the umap plots below. Here, we see that Leiden Cluster 8 was indeed mostly featured by high beta and unfavorable lower Operating Profitabiity Factors; Cluster 9 was featured by lower beta and many favourable higher Operating Profitability Factors; Clusters 6 and 7 were featured by unfavorable lower Solvency Factors (current, quick ratios).
rcParams['figure.figsize'] = 5,5
qp.pl.umap(adata, color=['leiden', 'beta', 'vol3MonthAvg', 'epsTTM', 'netProfitMarginTTM',
'netProfitMarginMRQ', 'grossMarginTTM', 'grossMarginMRQ',
'returnOnInvestment', 'returnOnAssets', 'operatingMarginTTM',
'operatingMarginMRQ', 'currentRatio', 'quickRatio', 'pbRatio'],
legend_loc='on data', frameon=False, ncols=4, cmap='bwr', vmax=0.5, legend_fontsize=25)
stats_perCluster = df_all.groupby(['leiden', 'GICS Sector']).agg({'log_return_1mth': ['median',
'min', 'max', 'mean']}).dropna()
stats_perCluster.columns = ['median', 'min', 'max', 'mean']
stats_perCluster = stats_perCluster.reset_index()
stats_perCluster[stats_perCluster.leiden == '6'].sort_values(['median'], ascending=False).\
style.background_gradient(cmap='bwr', axis=0)
leiden | GICS Sector | median | min | max | mean | |
---|---|---|---|---|---|---|
48 | 6 | Financials | 0.054715 | -0.020357 | 0.087380 | 0.042871 |
49 | 6 | Industrials | 0.036626 | -0.047826 | 0.047238 | 0.012012 |
51 | 6 | Utilities | -0.050636 | -0.050636 | -0.050636 | -0.050636 |
50 | 6 | Real Estate | -0.051799 | -0.122403 | 0.078756 | -0.045433 |
# Printing out Cluster 9 companies
heading_properties = [('font-size', '9px')]
cell_properties = [('font-size', '9.5px')]
dfstyle = [dict(selector="th", props=heading_properties),\
dict(selector="td", props=cell_properties)]
df_all[df_all.leiden == '9'][['Security', 'GICS Sector', 'GICS Sub-Industry']].style.set_table_styles(dfstyle)
Security | GICS Sector | GICS Sub-Industry | |
---|---|---|---|
ATVI | Activision Blizzard | Communication Services | Interactive Home Entertainment |
AKAM | Akamai Technologies Inc | Information Technology | Internet Services & Infrastructure |
ANSS | ANSYS | Information Technology | Application Software |
AJG | Arthur J. Gallagher & Co. | Financials | Insurance Brokers |
ADP | Automatic Data Processing | Information Technology | Internet Services & Infrastructure |
CHD | Church & Dwight | Consumer Staples | Household Products |
COO | The Cooper Companies | Health Care | Health Care Supplies |
EBAY | eBay Inc. | Consumer Discretionary | Internet & Direct Marketing Retail |
EA | Electronic Arts | Communication Services | Interactive Home Entertainment |
EFX | Equifax Inc. | Industrials | Research & Consulting Services |
ETSY | Etsy | Consumer Discretionary | Internet & Direct Marketing Retail |
FFIV | F5 Networks | Information Technology | Communications Equipment |
FTNT | Fortinet | Information Technology | Systems Software |
JKHY | Jack Henry & Associates | Information Technology | Data Processing & Outsourced Services |
MKC | McCormick & Co. | Consumer Staples | Packaged Foods & Meats |
NDAQ | Nasdaq, Inc. | Financials | Financial Exchanges & Data |
NFLX | Netflix Inc. | Communication Services | Movies & Entertainment |
NLOK | NortonLifeLock | Information Technology | Application Software |
ODFL | Old Dominion Freight Line | Industrials | Trucking |
PAYC | Paycom | Information Technology | Application Software |
PYPL | PayPal | Information Technology | Data Processing & Outsourced Services |
PKI | PerkinElmer | Health Care | Health Care Equipment |
ROL | Rollins Inc. | Industrials | Environmental & Facilities Services |
NOW | ServiceNow | Information Technology | Systems Software |
SNPS | Synopsys Inc. | Information Technology | Application Software |
TTWO | Take-Two Interactive | Communication Services | Interactive Home Entertainment |
TYL | Tyler Technologies | Information Technology | Application Software |
VNT | Vontier | Information Technology | Electronic Equipment & Instruments |
# Printing out Cluster 6 companies
heading_properties = [('font-size', '8px')]
cell_properties = [('font-size', ' 8px')]
dfstyle = [dict(selector="th", props=heading_properties),\
dict(selector="td", props=cell_properties)]
df_all[df_all.leiden == '6'][['Security', 'GICS Sector', 'GICS Sub-Industry']].style.set_table_styles(dfstyle)
Security | GICS Sector | GICS Sub-Industry | |
---|---|---|---|
ARE | Alexandria Real Estate Equities | Real Estate | Office REITs |
AXP | American Express Co | Financials | Consumer Finance |
AMT | American Tower Corp. | Real Estate | Specialized REITs |
AMP | Ameriprise Financial | Financials | Asset Management & Custody Banks |
ATO | Atmos Energy | Utilities | Gas Utilities |
AVB | AvalonBay Communities | Real Estate | Residential REITs |
BXP | Boston Properties | Real Estate | Office REITs |
SCHW | Charles Schwab Corporation | Financials | Investment Banking & Brokerage |
CCI | Crown Castle International Corp. | Real Estate | Specialized REITs |
DE | Deere & Co. | Industrials | Agricultural & Farm Machinery |
DLR | Digital Realty Trust Inc | Real Estate | Specialized REITs |
DRE | Duke Realty Corp | Real Estate | Industrial REITs |
EQIX | Equinix | Real Estate | Specialized REITs |
EQR | Equity Residential | Real Estate | Residential REITs |
ESS | Essex Property Trust, Inc. | Real Estate | Residential REITs |
EXR | Extra Space Storage | Real Estate | Specialized REITs |
FRT | Federal Realty Investment Trust | Real Estate | Retail REITs |
GS | Goldman Sachs Group | Financials | Investment Banking & Brokerage |
PEAK | Healthpeak Properties | Real Estate | Health Care REITs |
INFO | IHS Markit Ltd. | Industrials | Research & Consulting Services |
IVZ | Invesco Ltd. | Financials | Asset Management & Custody Banks |
KIM | Kimco Realty | Real Estate | Retail REITs |
MAA | Mid-America Apartments | Real Estate | Residential REITs |
MS | Morgan Stanley | Financials | Investment Banking & Brokerage |
NTRS | Northern Trust Corp. | Financials | Asset Management & Custody Banks |
PLD | Prologis | Real Estate | Industrial REITs |
PSA | Public Storage | Real Estate | Specialized REITs |
RJF | Raymond James Financial Inc. | Financials | Investment Banking & Brokerage |
O | Realty Income Corporation | Real Estate | Retail REITs |
REG | Regency Centers Corporation | Real Estate | Retail REITs |
SBAC | SBA Communications | Real Estate | Specialized REITs |
SPG | Simon Property Group Inc | Real Estate | Retail REITs |
SLG | SL Green Realty | Real Estate | Office REITs |
UDR | UDR, Inc. | Real Estate | Residential REITs |
UPS | United Parcel Service | Industrials | Air Freight & Logistics |
VTR | Ventas Inc | Real Estate | Health Care REITs |
VNO | Vornado Realty Trust | Real Estate | Office REITs |
WELL | Welltower Inc. | Real Estate | Health Care REITs |
The sunburst chart below shows that Cluster 9 mostly consisted of Information Technology Sector, Cluster 10 mostly Energy Sector, Cluster 9 Financial Sector, and Cluster 2 Real Estate Sector.
import plotly.express as px
# grouping statistics
ds = df_all.groupby(['leiden', 'GICS Sector', 'GICS Sub-Industry'])['log_return_1mth'].count().reset_index()
ds.columns = ['leiden', 'GICS Sector', 'GICS Sub-Industry', 'count']
# plotting sunburst
fig = px.sunburst(
ds,
path=[
'leiden', 'GICS Sector', 'GICS Sub-Industry'
],
values='count',
title='Sunburst chart',
width=900,
height=900
)
fig.show()
The sunburst chart below shows that most of the Real Estate companies Fall on the worst performer 1-month post vaccine success, Cluster 6.
import plotly.express as px
from itertools import repeat
Other_clusters = [c for c in df_all.leiden.unique() if c not in ['8', '9', '7', '6']]
dict_regroup = {key: item for key, item in zip(Other_clusters, repeat('Others'))}
dict_regroup['8'] = 'C8 - Top performer'
dict_regroup['9'] = 'C9 - Second top (ideal) performer'
dict_regroup['7'] = 'C7 - Second to Worst performer'
dict_regroup['6'] = 'C6 - Worst performer'
df_regrouped = df_all.replace({'leiden': dict_regroup})
# grouping statistics
ds = df_regrouped.groupby(['GICS Sector', 'leiden'])['log_return_1mth'].count().reset_index()
ds.columns = ['GICS Sector', 'leiden', 'count']
# plotting sunburst
fig = px.sunburst(
ds,
path=[
'GICS Sector','leiden'
],
values='count',
title='Sunburst chart',
width=700,
height=700
)
fig.show()
In this tutorial, we found that although the Cluster 8 was the top performer but it was associated with high beta companies — higher risk in stock market. Their performance were unjustifiable with statistically lower Operating Profit Factor. In comparison, Cluster 9 companies have lower beta — less risky, and its performance is justified by favourable higher Operating Profit Factor and lower Financial Risk Factors.
On the other hand, the worst performer was the Cluster 6, followed by Cluster 7. Both clusters have significantly lower solvency (current and quick ratios). Cluster 6 was also associated with higher financial risk, i.e. higher debt-to-capital/Equity. Of note, the cluster 6 was consisted mainly of the Real Estate companies. The worst performance probably justified by the on-going second wave during the current winter season, where city lockdown is widely adopted.
REFERENCES: