Saeed Amen / Founder of Cuemacro
https://www.cuemacro.com / saeed@cuemacro.com / @saeedamenfx / All material is copyright Cuemacro / 2020
Here we'll quickly introduce tcapy showing you how a few lines of code can quickly generate TCA to understand what you're trading costs are. Whilst tcapy supports the use of databases, caching or distributing computation (which we discuss in other notebooks), we'll focus on the basics here. You can also run this notebook interactively in Binder to play around with the code.
Transaction cost analysis (TCA) involves looking at your trade/order data combined with market data to understand your trading costs. We can use TCA to calculate metrics like slippage and market impact, which we discuss later.
We should be able to understand how trading costs vary depending on upon factors like, asset, liquidity provider and method of execution (manual/algo).
tcapy is one of the first open source Python libraries for doing TCA. Typically, developing a TCA library yourself could cost many hundreds of thousands of dollar. By contrast tcapy is free (of course sponsors for new features are always welcome, and Cuemacro also offers commercial support for tcapy).
We assume that an order has a start and end time, and hence a duration. It will have fields associated with it such as the ticker traded, the execution price, the account, the execution trader, the broker, the algo used etc.
Underneath every order, there might be a number of trades and other events. The difference is that we assume these trade events, are points in time. These trades will tag their respective orders.
What should we consider as our benchmark? We need to have market data which is representative of the market as our reference point to compare our own trades. If we consider the FX spot market there are relatively small number of principal liquidity providers (LPs). At any one time, you might observe different skew in prices from different market makers, because of a number of reasons, such as different forecasts on where spot could go, their own inventory etc. Hence, using the mid of the pricing stream from a single market maker, it may not be representative of the market on aggregate. We are using the mid price as a proxy for the reference price.
We've focused on Dukascopy here, mainly because it's free, to allow readers to run this code (and so it works on Binder). However, you can try different data sources to see how it impacts your results.
At present tcapy has adapters to download market data from two data sources listed below to use as a benchmark. However, if you can feed whatever market data you want to the library and we hope to add more data sources in the future.
Once you've decided which market data to use, you can calculate several benchmarks, which are below. Some benchmarks like TWAP are only appropriate for orders, which have a start and finish time. The abstract class Benchmark
in tcapy can be extended to create your own customised benchmarks.
BenchmarkMarketMid
)BenchmarkTWAP
)BenchmarkVWAP
)Metrics you might wish to calculate the below, and we've indicated the classes which implement these. In tcapy, there is the abstract Metric
class, which can be extended so you can write your own metrics.
MetricSlippage
)MetricTransientMarketImpact
and MetricPersistentMarketImpact
)We haven't implemented this yet, but you might also want to look at some flat fees, which are charged depending on the trading venue (although these are much easier to keep track of, as you don't need any market data to calculate, just your trade data).
As part of TCA, we'd then want to generate statistics based on the metrics like those listed below. An abstract class ResultsForm
aggregates various statistics. You can extend it to create your own aggregations.
BarResultsForm
)DistResultsForm
)TableResultsForm
)Our first step is to add the tcapy library to our PYTHONPATH
. You may need to change these lines. Note, that in our setup, we are running the Jupyter notebook on Windows, and the backend Celery workers on Linux. If you are running everything in the same environment, the paths are likely to be same. We'll also define some paths for the test data we'll use for trade/orders.
import sys
import os
windows_tcapy_path = 'e:/cuemacro/tcapy' # Windows platform
linux_tcapy_path = '/home/tcapyuser/cuemacro/tcapy' # Linux platform
local_test_data_path = '../test/resources/' # Windows platform
remote_test_data_path = '../test/resources/' # WSL drive
# Assuming the front end is on Windows
sys.path.insert(0, windows_tcapy_path)
If the Python path now includes tcapy, we can do all the imports from it, that we'll need later. We'll also import all the other Python libraries we'll use.
from collections import OrderedDict
# This is the entry point for our TCA analysis
from tcapy.analysis.tcaengine import TCAEngineImpl
# To construct the parameters for our TCA calculation
from tcapy.analysis.tcarequest import TCARequest
# To access trade and market data
from tcapy.data.databasesource import DatabaseSourceCSVBinary, \
DatabaseSourceDataFrame, DatabaseSourceDukascopy, DatabaseSourceNCFX
# Import all the metrics and benchmarks we'll use
from tcapy.analysis.algos.benchmark import BenchmarkArrival, BenchmarkMarketSpreadToMid
from tcapy.analysis.algos.metric import MetricSlippage, MetricTransientMarketImpact
# To aggregate the TCA results
from tcapy.analysis.algos.resultsform import TimelineResultsForm, DistResultsForm, BarResultsForm
# To help display the output of a TCA calculation
from tcapy.vis.tcaresults import TCAResults
from tcapy.vis.report.tcareport import TCAReport
from tcapy.vis.displaylisteners import PlotRender
# General classes
from tcapy.conf.constants import Constants
from tcapy.util.mediator import Mediator
# Prevent requests from displaying debug messages for certain libraries
import logging
logging.getLogger("findatapy").setLevel(logging.WARNING)
logging.getLogger("requests").setLevel(logging.WARNING)
logging.getLogger("urllib3").setLevel(logging.WARNING)
# For plotting later
import plotly.io as pio
# For interactive plots (but these disappear when we reopen Jupyter)
pio.renderers.default = "notebook"
# For static plots, we use https://github.com/plotly/Kaleido
# pio.renderers.default = "svg"
pio.renderers.default = "png"
import plotly
from chartpy import Chart, Style
# default size for Plotly charts we use
chart_width = 800
chart_height = 500
chart = Chart()
constants = Constants()
style = Style(width=chart_width, height=chart_height, scale_factor=-1, silent_display=True)
We can also set the market data source for use later, either downloading external data from Dukascopy or we can use some mocked up data in a Parquet file in the test folder.
market_data_store = 'dukascopy'
# market_data_store = os.path.join(local_test_data_path, 'small_test_market_df.parquet')
tcapy has a test dataset of randomised trade/order data in CSV format, which we'll use later. We have generated these by taking market bid/ask data and then randomly perturbing the bid/ask for sells/buys respectively. Random sizes and directions have also been constructed. I'd suggest it's probably better to try using your own real trade data, as the output will be more 'realistic'. A lot of the output we'll get might end up looking artifical because of the simple assumptions we've made in the construction of the test data.
# The test trade/order data is populated between 25 Apr 2017-05 Jun 2017
# with trades/orders for 'EURUSD', 'USDJPY' and 'EURJPY'
csv_trade_order_mapping = {'trade_df' : os.path.join(local_test_data_path, 'small_test_trade_df.csv'),
'order_df' : os.path.join(local_test_data_path, 'small_test_order_df.csv')}
The DatabaseSource
abstract class is extended to provide trade and market data access from internal sources (eg. CSV, Arctic/MongoDB), as well as external sources like Dukascopy. We demonsrate how to use DatabaseSourceDukascopy
to download market data with quoted bid/ask from Dukascopy (we can't download trade data from that). If there are any timeouts in the external download, findatapy
displays a warning, and will also retry the download.
%%time
database_source_dukascopy = DatabaseSourceDukascopy()
market_df = database_source_dukascopy.fetch_market_data(
ticker='EURUSD', start_date='25 Apr 2017', finish_date='28 Apr 2017')
Wall time: 8.25 s
We can look at the market data downloaded to see the format.
market_df.head(10)
bid | ask | mid | ticker | |
---|---|---|---|---|
Date | ||||
2017-04-25 00:00:01.466000+00:00 | 1.08648 | 1.08654 | 1.086510 | EURUSD |
2017-04-25 00:00:01.618000+00:00 | 1.08650 | 1.08654 | 1.086520 | EURUSD |
2017-04-25 00:00:02.276000+00:00 | 1.08650 | 1.08654 | 1.086520 | EURUSD |
2017-04-25 00:00:04.081000+00:00 | 1.08649 | 1.08654 | 1.086515 | EURUSD |
2017-04-25 00:00:05.470000+00:00 | 1.08650 | 1.08654 | 1.086520 | EURUSD |
2017-04-25 00:00:06.039000+00:00 | 1.08650 | 1.08654 | 1.086520 | EURUSD |
2017-04-25 00:00:06.567000+00:00 | 1.08651 | 1.08654 | 1.086525 | EURUSD |
2017-04-25 00:00:07.604000+00:00 | 1.08652 | 1.08655 | 1.086535 | EURUSD |
2017-04-25 00:00:08.503000+00:00 | 1.08654 | 1.08656 | 1.086550 | EURUSD |
2017-04-25 00:00:08.717000+00:00 | 1.08656 | 1.08660 | 1.086580 | EURUSD |
We can use DatabaseSourceCSVBinary
to open up the trade and order data.
trade_df = DatabaseSourceCSVBinary(trade_data_database_csv=csv_trade_order_mapping['trade_df']) \
.fetch_trade_order_data()
order_df = DatabaseSourceCSVBinary(trade_data_database_csv=csv_trade_order_mapping['order_df']) \
.fetch_trade_order_data()
Let's look at the trade data so we can understand the type of format that tcapy needs. We can see that each trade is a point in time. In some cases, these are actual trade
events, other times they are placement
, cancel
or cancel/replace
messages. The times of two different events can be identical, and there are some instances of this in the test datasets of trade/orders. Note, that this can often cause problems with Pandas, so if do create your own Metric
and Benchmark
variations, do check that your implementations can manage it (can require some rewriting of the code).
trade_df.head(5)
account_id | algo_id | algo_settings | ancestor_pointer_id | broker_id | broker_sub_id | event_type | executed_notional | executed_price | id | ... | market_mid | notional_currency | order_notional | portfolio_id | portfolio_manager_id | price_limit | side | ticker | trader_id | venue | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Date | |||||||||||||||||||||
2017-04-25 07:09:31.804000+00:00 | account3 | algo4 | default | order_EURUSD2017-04-25 07:09:31.804000+00:00_2... | broker5 | subbroker1 | placement | 0 | 0.000000 | execution_EURUSD2017-04-25 07:09:31.804000+00:... | ... | 1.087965 | EUR | 18517881 | portfolio4 | pm2 | 1.087965 | -1 | EURUSD | trader1 | venue4 |
2017-04-25 07:09:33.765000+00:00 | account3 | algo4 | default | order_EURUSD2017-04-25 07:09:31.804000+00:00_2... | broker5 | subbroker1 | trade | 18517881 | 1.087965 | execution_EURUSD2017-04-25 07:09:33.765000+00:... | ... | 1.087990 | EUR | 18517881 | portfolio4 | pm2 | 1.087965 | -1 | EURUSD | trader1 | venue3 |
2017-04-25 11:09:52.384000+00:00 | account3 | algo6 | default | order_EURUSD2017-04-25 11:09:52.384000+00:00_2... | broker6 | subbroker6 | placement | 0 | 0.000000 | execution_EURUSD2017-04-25 11:09:52.384000+00:... | ... | 1.088090 | EUR | 8364696 | portfolio4 | pm3 | 1.088090 | -1 | EURUSD | trader5 | venue4 |
2017-04-25 11:09:53.242000+00:00 | account3 | algo6 | default | order_EURUSD2017-04-25 11:09:52.384000+00:00_2... | broker6 | subbroker6 | cancel/replace | 0 | 0.000000 | execution_EURUSD2017-04-25 11:09:53.242000+00:... | ... | 1.088080 | EUR | 0 | portfolio4 | pm3 | 1.088090 | -1 | EURUSD | trader5 | venue3 |
2017-04-25 11:10:20.251000+00:00 | account3 | algo6 | default | order_EURUSD2017-04-25 11:09:52.384000+00:00_2... | broker6 | subbroker6 | trade | 8364696 | 1.088093 | execution_EURUSD2017-04-25 11:10:20.251000+00:... | ... | 1.088125 | EUR | 8364696 | portfolio4 | pm3 | 1.088090 | -1 | EURUSD | trader5 | venue4 |
5 rows × 22 columns
Let's print out all the fields. Some fields are compulsory for use in tcapy, including the index (Date
), ticker
, side
etc. Some fields here are optional such as market_ask
, market_bid
and market_mid
. Similar fields will be autogenerated from the market data and appends to the trade data. DatabaseSource
does minimal changes to the format of the data. Each of these trade message point to the order which they belong to. There can be multiple trade messages within a single order.
trade_df.columns
Index(['account_id', 'algo_id', 'algo_settings', 'ancestor_pointer_id', 'broker_id', 'broker_sub_id', 'event_type', 'executed_notional', 'executed_price', 'id', 'market_ask', 'market_bid', 'market_mid', 'notional_currency', 'order_notional', 'portfolio_id', 'portfolio_manager_id', 'price_limit', 'side', 'ticker', 'trader_id', 'venue'], dtype='object')
If we look at the order data, one the key differences, which we noted is that each order has a benchmark_date_start
and benchmark_date_end
, a start and end time. Note, obviously this randomized data. We also see that it doesn't have columns for executed_notional
data or executed_price
data. tcapy can generate this from the underlying trade fills from the earlier table.
order_df.head(5)
account_id | algo_id | algo_settings | arrival_price | benchmark_date_end | benchmark_date_start | broker_id | broker_sub_id | id | notional | notional_currency | portfolio_id | portfolio_manager_id | price_limit | side | ticker | trader_id | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Date | |||||||||||||||||
2017-04-25 07:09:31.804000+00:00 | account3 | algo4 | default | 1.087965 | 2017-04-25 10:40:16.512000+00:00 | 2017-04-25 07:09:31.804000+00:00 | broker5 | subbroker1 | order_EURUSD2017-04-25 07:09:31.804000+00:00_2... | 18517881 | EUR | portfolio4 | pm2 | 1.087965 | -1 | EURUSD | trader1 |
2017-04-25 11:09:52.384000+00:00 | account3 | algo6 | default | 1.088090 | 2017-04-25 14:33:06.745000+00:00 | 2017-04-25 11:09:52.384000+00:00 | broker6 | subbroker6 | order_EURUSD2017-04-25 11:09:52.384000+00:00_2... | 8364696 | EUR | portfolio4 | pm3 | 1.088090 | -1 | EURUSD | trader5 |
2017-04-25 15:50:41.479000+00:00 | account1 | algo3 | default | 1.093765 | 2017-04-26 06:01:27.935000+00:00 | 2017-04-25 15:50:41.479000+00:00 | broker5 | subbroker5 | order_EURUSD2017-04-25 15:50:41.479000+00:00_2... | 774423 | EUR | portfolio1 | pm6 | 1.093765 | -1 | EURUSD | trader3 |
2017-04-26 06:48:48.562000+00:00 | account4 | algo2 | default | 1.092460 | 2017-04-26 09:57:57.631000+00:00 | 2017-04-26 06:48:48.562000+00:00 | broker3 | subbroker3 | order_EURUSD2017-04-26 06:48:48.562000+00:00_2... | 16465763 | EUR | portfolio3 | pm5 | 1.092460 | -1 | EURUSD | trader5 |
2017-04-26 11:15:50.864000+00:00 | account6 | algo1 | default | 1.089585 | 2017-04-26 14:41:01.382000+00:00 | 2017-04-26 11:15:50.864000+00:00 | broker3 | subbroker6 | order_EURUSD2017-04-26 11:15:50.864000+00:00_2... | 11501881 | EUR | portfolio4 | pm3 | 1.089585 | -1 | EURUSD | trader4 |
tcapy can do TCA on trade data on its own (or trade and order data together). Make sure that your trade/order is in a similar format to use with tcapy.
We earlier created a trade/order mapping, to tell tcapy where to find the trade/order data, which is this case is in CSV files. Note the use of trade_df
and order_df
, which you'll have to use each time. We create a TCAEngineImpl
object which will be used to accept a TCARequest
for doing the TCA computation. We also get reference to the VolatileCache
which uses Redis underneath (if Redis isn't installed, calculations will proceed, it'll just be slower).
tca_version = constants.tcapy_version
tca_engine = TCAEngineImpl(version=tca_version)
2020-09-14 16:28:00,713; INFO:tcapy.analysis.tcaengine: Init TCAEngine version: pro - Env: desktop_laptop_linux (tcaengine.py:53)
We create a TCARequest
, which will be run on our trade/order CSV files. It allows us to specify our TCARequest
in a Pythonic like way, as opposed to using some sort of query language directly (like SQL or q). Obviously, the key advantage of this is because tcapy supports all sorts of databases, we don't need to change our TCARequest
much if we change our underlying data sources.
The market data will be externally downloaded from Dukascopy. In this instance, we have set use_multithreading=False
which avoids the use of Celery, and makes it easier to run on Binder. We can add the below, for calculation:
Metric
to measure the "quality" of each trade/orderMetricSlippage
which by default calcualtes the difference between the mid
price and the executed_price
Benchmark
to capture what the market is around each trade/orderBenchmarkArrival
which will the mid price for the arrival price of every trade/orderResultForm
objects to aggregate the statistics of various fields.TimelineResultsForm
will calculate the average slippage
by day and hour. We use a scalar
of 10000.0
to convert the values into basis points, which are easier to interpretexecuted_notional
although we can change the weighting fieldWhilst we are asking for ticker EURUSD
, tcapy will also search for trades/orders booked the "wrong" way round ie. USDEUR
and normalize them so they are converted to EURUSD
trades/orders.
# We are conducting a TCA computation for trade/order data between 05 May-10 May 2017 for EURUSD
# The trade dataset is from CSV and the market data is downloaded from Dukascopy/or a mocked up file
# For metrics, we are calculating slippage and for benchmarks we are adding a field for the arrival price
# We are creating a timeline of average slippage by date and hour
tca_request = TCARequest(start_date='05 May 2017', finish_date='10 May 2017', ticker=['EURUSD'],
tca_type='detailed',
trade_data_store='csv', market_data_store=market_data_store,
trade_order_mapping=csv_trade_order_mapping,
metric_calcs=[MetricSlippage()],
results_form=[TimelineResultsForm(metric_name='slippage',
by_date='datehour', scalar=10000.0)],
benchmark_calcs=[BenchmarkArrival(),
BenchmarkMarketSpreadToMid()],
use_multithreading=False)
Notice there a lot of the log output can be suppressed if we change the logging level to INFO
in constants.py
.
calculate_tca
will use several other classes. We describe the rough steps of the calculation before:
TCAMarketTradeLoaderImpl
manages the computationTCATickerLoaderImpl
for each ticker will thenDataFactory
, DatabaseSource
etc.)Metric
, Benchmark
etc.)TCAMarketTradeLoaderImpl
will then aggregate the statistics generated (using ResultsForm
)DataFrame
objects returned to the caller by calculate_tca
As we noted before if there are any timeouts in downloading the data, it will attempt to retry a few times. This may take several minutes, if are downloading a lot of data over the web. You might get some warning messages because it is trying to access a cache, which may not exist on Binder, which will also slow down the computation.
# Dictionary of (mostly) dataframes as output from TCA calculation
dict_of_df = tca_engine.calculate_tca(tca_request)
2020-09-14 16:28:00,739; DEBUG:tcapy.analysis.tcamarkettradeloader: Start loading trade/data/computation (tcamarkettradeloader.py:241) 2020-09-14 16:28:00,742; DEBUG:tcapy.analysis.tcatickerloaderimpl: Get market and trade/order data for EURUSD from 2017-05-05 00:00:00+00:00 - 2017-05-10 00:00:00+00:00 (tcatickerloaderimpl.py:80) 2020-09-14 16:28:00,744; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:28:04,830; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:04,832; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:28:08,914; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:08,917; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-10 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:28:17,171; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:28:17,172; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:28:17,266; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:28:17,302; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:28:21,380; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:28:21,382; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:28:21,478; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:28:21,513; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:28:25,595; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:28:25,596; DEBUG:tcapy.analysis.tcatickerloader: Get trade order holder for EURUSD from 2017-05-05 00:00:00+00:00 - 2017-05-10 00:00:00+00:00 (tcatickerloader.py:386) 2020-09-14 16:28:25,634; WARNING:tcapy.data.datafactory: Dataframe empty for ticker USDEUR (datafactory.py:97) 2020-09-14 16:28:25,636; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:28:29,722; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:29,722; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:28:33,802; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:33,804; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-10 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:28:42,074; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:28:42,075; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:28:42,171; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:28:42,204; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:28:46,287; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:28:46,289; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:28:46,382; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:28:46,416; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:28:50,504; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:28:50,541; WARNING:tcapy.data.datafactory: Dataframe empty for ticker USDEUR (datafactory.py:97) 2020-09-14 16:28:50,542; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:28:54,631; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:54,632; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:28:58,713; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:28:58,715; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-10 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:29:07,058; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:29:07,059; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:29:07,153; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:29:07,185; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:29:11,269; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:29:11,270; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:29:11,363; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 17.08 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:29:11,398; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:443)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:29:15,484; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-10 00:00:00+00:00_market_df_None_comp_size_5698367_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:29:15,488; DEBUG:tcapy.analysis.tcatickerloader: Filter the market date by start/finish date (tcatickerloader.py:761) 2020-09-14 16:29:15,495; DEBUG:tcapy.analysis.tcatickerloader: Combine trade/order data (tcatickerloader.py:782) 2020-09-14 16:29:15,503; DEBUG:tcapy.analysis.tcatickerloader: Calculating BenchmarkMarketSpreadToMid for market data (tcatickerloader.py:600) 2020-09-14 16:29:15,527; DEBUG:tcapy.analysis.tcatickerloader: Filter trades by venue (tcatickerloader.py:458) 2020-09-14 16:29:15,530; DEBUG:tcapy.analysis.tcatickerloader: Calculating derived fields and benchmarks (tcatickerloader.py:508) 2020-09-14 16:29:15,531; DEBUG:tcapy.analysis.tcatickerloader: Calculating execution fields (tcatickerloader.py:510) 2020-09-14 16:29:15,543; DEBUG:tcapy.analysis.tcatickerloader: Calculating benchmarks (tcatickerloader.py:536) 2020-09-14 16:29:15,545; DEBUG:tcapy.analysis.tcatickerloader: Calculating BenchmarkArrival for trade_df (tcatickerloader.py:545) 2020-09-14 16:29:15,551; DEBUG:tcapy.analysis.tcatickerloader: Calculating BenchmarkArrival for order_df (tcatickerloader.py:545) 2020-09-14 16:29:15,556; DEBUG:tcapy.analysis.tcatickerloader: Calculating metrics (tcatickerloader.py:554) 2020-09-14 16:29:15,557; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricSlippage for trade_df (tcatickerloader.py:560) 2020-09-14 16:29:15,575; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricSlippage for order_df (tcatickerloader.py:560) 2020-09-14 16:29:15,593; DEBUG:tcapy.analysis.tcatickerloader: Completed derived field calculations for EURUSD (tcatickerloader.py:568) 2020-09-14 16:29:15,594; DEBUG:tcapy.analysis.tcatickerloaderimpl: Generating downsampled market data for potentional display (tcatickerloaderimpl.py:275) 2020-09-14 16:29:15,615; DEBUG:tcapy.analysis.tcatickerloaderimpl: About to join (tcatickerloaderimpl.py:340) 2020-09-14 16:29:15,619; DEBUG:tcapy.analysis.tcatickerloaderimpl: Finished joining (tcatickerloaderimpl.py:356) 2020-09-14 16:29:15,622; DEBUG:tcapy.analysis.tcatickerloaderimpl: About to join (tcatickerloaderimpl.py:340) 2020-09-14 16:29:15,625; DEBUG:tcapy.analysis.tcatickerloaderimpl: Finished joining (tcatickerloaderimpl.py:356)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:29:15,661; DEBUG:tcapy.analysis.tcamarkettradeloader: Finished loading data and calculating metrics on individual _tickers (tcamarkettradeloader.py:248) 2020-09-14 16:29:15,663; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Constructing results form to summarize analysis... (tcamarkettradeloaderimpl.py:74) 2020-09-14 16:29:15,692; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Now join table results... (tcamarkettradeloaderimpl.py:122) 2020-09-14 16:29:15,693; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Finished calculating results form and join table results! (tcamarkettradeloaderimpl.py:135)
Let's take a look at the dictionary returned.
dict_of_df.keys()
odict_keys(['trade_df', 'order_df', 'sparse_market_trade_df', 'sparse_market_order_df', 'market_df_downsampled', 'candlestick_fig', 'timeline_trade_df_slippage_by/mean_datehour/all', 'timeline_order_df_slippage_by/mean_datehour/all', 'market_df'])
We explain what each item in the dictionary is below:
trade_df
- DataFrame of trades (with added calculated fields like slippage, arrival price etc.)order_df
- DataFrame of orders (with added calculated fields like slippage, arrival price etc.)sparse_market_trade_df
- DataFrame with market data and tradessparse_market_order_df
- DataFrame with market data and ordersmarket_df_downsampled
- DataFrame with downsampled market datacandlestick_fig
- Plotly JSON of the market data as candlestickstimeline_trade_df_slippage_by/mean_date/all
- a timeline of the slippage for the tradestimeline_order_df_slippage_by/mean_date/all
- a timeline of the slippage for the ordersWe can plot the timeline of the slippage by day and hour. During this period the slippage has been between 0.3 and 0.05 basis points (bp), which seems reasonable for EUR/USD.
style.title = 'Timeline of slippage by day (bp)'
chart.plot(dict_of_df['timeline_trade_df_slippage_by/mean_datehour/all'], engine='plotly', style=style)
Let's also take a look at the returned trade_df
DataFrame. Most of the fields are the same, but there now additional fields fields calculated by tcapy. Note, these will vary depending on which Metric
and Benchmark
objects you specify in the TCARequest
:
arrival
- arrival price from the benchmark at time of the order/just beforeslippage_benchmark
- difference between mid price from the market data, just before/at tradespread_to_benchmark
- differnce to the bid (for sells) and to the ask (for buys) - if this data is availableslippage
- difference between executed_price
and benchmarkslippage_anomalous
- 1 if the slippage is beyond a certainWe also have fields for the notional in reporting currency (here USD
) which are autogenerated from the market data combined with the trade data. You can specify you own reporting currency in Constants
. It is useful to have this field, if you want to calculate total trading costs across many different currency pairs later.
We can use a higher level TCAResults
object to simplify the output of the TCA calculation. Here we create the Plotly JSON for all the charts which have been requested.
tca_results = TCAResults(dict_of_df, tca_request, chart_width=chart_width, chart_height=chart_height)
tca_results.render_computation_charts()
2020-09-14 16:29:25,576; DEBUG:tcapy.vis.displaylisteners: Plotting main timeline 2583 (displaylisteners.py:988) 2020-09-14 16:29:25,624; DEBUG:tcapy.vis.displaylisteners: Rendered plot 2583 and (displaylisteners.py:998) 2020-09-14 16:29:25,627; DEBUG:tcapy.vis.displaylisteners: Plotting main timeline 2576 (displaylisteners.py:988) 2020-09-14 16:29:25,676; DEBUG:tcapy.vis.displaylisteners: Rendered plot 2576 and (displaylisteners.py:998)
Let's get the trade data from the TCAResults
objects.
tca_results.trade_order['trade_df'].head(5)
account_id | algo_id | algo_settings | ancestor_pointer_id | broker_id | broker_sub_id | event_type | executed_notional | executed_price | id | ... | notional_reporting_currency_mid | reporting_currency | executed_notional_in_reporting_currency | order_notional_in_reporting_currency | notional | arrival | slippage_benchmark | spread_to_benchmark | slippage | slippage_anomalous | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Date | |||||||||||||||||||||
2017-05-05 06:27:26.514000+00:00 | account5 | algo2 | default | order_EURUSD2017-05-05 06:26:47.189000+00:00_2... | broker6 | subbroker1 | trade | 18008368 | 1.097430 | execution_EURUSD2017-05-05 06:27:26.514000+00:... | ... | 1.097445 | USD | 1.976319e+07 | 1.976319e+07 | 18008368 | 1.097445 | 1.097445 | -0.000005 | -0.000013 | 1 |
2017-05-05 11:30:00.791000+00:00 | account5 | algo3 | default | order_EURUSD2017-05-05 11:28:43.816000+00:00_2... | broker3 | subbroker3 | trade | 5370024 | 1.095723 | execution_EURUSD2017-05-05 11:30:00.791000+00:... | ... | 1.095725 | USD | 5.884070e+06 | 5.884070e+06 | 5370024 | 1.095725 | 1.095725 | -0.000014 | -0.000002 | 0 |
2017-05-05 14:56:49.373000+00:00 | account4 | algo2 | default | order_EURUSD2017-05-05 14:56:33.125000+00:00_2... | broker2 | subbroker5 | trade | 6198753 | 1.098892 | execution_EURUSD2017-05-05 14:56:49.373000+00:... | ... | 1.098875 | USD | 6.811655e+06 | 6.811655e+06 | 6198753 | 1.098875 | 1.098875 | -0.000014 | -0.000016 | 1 |
2017-05-05 20:33:21.932000+00:00 | account6 | algo2 | default | order_EURUSD2017-05-05 20:33:14.677000+00:00_2... | broker5 | subbroker6 | trade | 6270424 | 1.099439 | execution_EURUSD2017-05-05 20:33:21.932000+00:... | ... | 1.099425 | USD | 6.893861e+06 | 6.893861e+06 | 6270424 | 1.099425 | 1.099425 | -0.000023 | -0.000012 | 0 |
2017-05-08 10:31:33.011000+00:00 | account1 | algo3 | default | order_EURUSD2017-05-08 10:30:57.526000+00:00_2... | broker4 | subbroker2 | trade | 3276653 | 1.094703 | execution_EURUSD2017-05-08 10:31:33.011000+00:... | ... | 1.094680 | USD | 3.586887e+06 | 3.586887e+06 | 3276653 | 1.094680 | 1.094680 | -0.000009 | -0.000021 | 1 |
5 rows × 32 columns
We can access the timeline of the slippage for trades by accessing the timeline
property. Note, it won't be populated for date/hours where we haven't traded.
tca_results.timeline['trade_df_slippage_by/mean_datehour/all'].head(5)
value | |
---|---|
Date | |
2017-05-05 06:00:00 | -0.133895 |
2017-05-05 11:00:00 | -0.017519 |
2017-05-05 14:00:00 | -0.156906 |
2017-05-05 20:00:00 | -0.122419 |
2017-05-08 10:00:00 | -0.209665 |
Here we create a TCARequest
for the same dataset. This time, however, we also calculate the MetricTransientMarketImpact
, which we've defined as the 5 second move from a trade or start of an order. We also want to calculate the distribution of the slippage by side (ie. buy/sell) with DistResultsForm
and to calculate the average slippage (weighted by the executed notional) with BarResultForm
. Note, we've used scalar
parameters of 10000.0
to convert to basis point to ease understanding.
All these Metric
and ResultsForm
calculations will be calculated on both trades and orders. However, we can specify for it to be calculated only on for example trades, by instantiating the objects using the market_trade_order_list
parameter eg. MetricSlippage(market_trade_order_list=['trade_df'])
tca_request = TCARequest(start_date='05 May 2017', finish_date='20 May 2017', ticker='EURUSD',
tca_type='detailed',
trade_data_store='csv', market_data_store=market_data_store,
trade_order_mapping=csv_trade_order_mapping,
metric_calcs=[MetricSlippage(),
MetricTransientMarketImpact(
transient_market_impact_gap={'5' : 's'})],
results_form=[DistResultsForm(metric_name='slippage',
aggregate_by_field='side', scalar=10000.0),
BarResultsForm(metric_name='slippage',
aggregate_by_field='ticker', scalar=10000.0),
BarResultsForm(metric_name='slippage',
aggregate_by_field='broker_id', scalar=10000.0)],
benchmark_calcs=[BenchmarkMarketSpreadToMid()],
use_multithreading=False)
Let's kick off the TCA calculation! Again this can take several minutes to download the data. You might get some warning messages because it is trying to access a cache.
%%time
# Dictionary of (mostly) dataframes as output from TCA calculation
dict_of_df = tca_engine.calculate_tca(tca_request)
2020-09-14 16:29:25,756; INFO:tcapy.analysis.tcaengine: More than 1 ticker specified for TCA detailed computation. Only working on first (tcaengine.py:122) 2020-09-14 16:29:25,757; DEBUG:tcapy.analysis.tcamarkettradeloader: Start loading trade/data/computation (tcamarkettradeloader.py:241) 2020-09-14 16:29:25,758; DEBUG:tcapy.analysis.tcatickerloaderimpl: Get market and trade/order data for EURUSD from 2017-05-05 00:00:00+00:00 - 2017-05-20 00:00:00+00:00 (tcatickerloaderimpl.py:80) 2020-09-14 16:29:25,758; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:29:29,846; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:29:29,848; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:29:33,928; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:29:33,930; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-20 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:29:58,402; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:29:58,404; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:29:58,744; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:29:58,873; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:30:02,952; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:30:02,953; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:30:03,295; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:30:03,415; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:30:07,502; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:30:07,502; DEBUG:tcapy.analysis.tcatickerloader: Get trade order holder for EURUSD from 2017-05-05 00:00:00+00:00 - 2017-05-20 00:00:00+00:00 (tcatickerloader.py:386) 2020-09-14 16:30:07,545; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:30:11,628; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:30:11,629; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:30:15,710; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:30:15,712; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-20 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:30:40,156; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:30:40,157; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:30:40,531; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:30:40,652; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:30:44,741; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:30:44,758; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:30:45,108; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:30:45,222; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:30:49,303; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:30:49,309; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:30:53,383; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:30:53,384; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:30:57,464; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:30:57,466; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-20 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:31:21,990; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:31:21,991; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:31:22,329; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:31:22,447; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:31:26,523; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:31:26,524; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:31:26,880; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:31:26,996; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:31:31,069; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:31:31,112; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:31:35,197; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:31:35,199; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:31:39,279; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:31:39,282; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-20 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:32:03,729; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:32:03,730; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:32:04,082; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:32:04,201; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:32:08,282; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:32:08,300; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:32:08,640; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:32:08,760; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:32:12,844; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_USDEUR_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:32:12,849; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:32:16,923; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:32:16,924; DEBUG:tcapy.data.volatilecache: Attempting to get list from cache: ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] (volatilecache.py:540) 2020-09-14 16:32:21,004; WARNING:tcapy.data.volatilecache: Couldn't retrieve ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] from cache: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it. (volatilecache.py:500) 2020-09-14 16:32:21,007; DEBUG:tcapy.data.databasesource: Downloading 2017-05-05 00:00:00 - 2017-05-20 00:00:00 for EURUSD (databasesource.py:3816) 2020-09-14 16:32:45,499; WARNING:tcapy.analysis.algos.benchmark: mid not in market data (benchmark.py:90) 2020-09-14 16:32:45,500; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426) 2020-09-14 16:32:45,848; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:32:45,965; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:32:50,042; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:32:50,043; DEBUG:tcapy.data.volatilecache: Attempting to push ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp'] to cache (volatilecache.py:426)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:32:50,384; DEBUG:tcapy.util.deltaizeserialize: Pandas dataframe of size: ----------- 63.639 MB ----------- in 1 chunk(s) (deltaizeserialize.py:221) 2020-09-14 16:32:50,506; DEBUG:tcapy.data.volatilecache: Now pushing ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:443) 2020-09-14 16:32:54,590; DEBUG:tcapy.data.volatilecache: Pushed ['dukascopy_EURUSD_2017-05-05 00:00:00+00:00_2017-05-20 00:00:00+00:00_market_df_None_comp_size_21020283_endsizearrow_'] to cache (volatilecache.py:461) 2020-09-14 16:32:54,595; DEBUG:tcapy.analysis.tcatickerloader: Filter the market date by start/finish date (tcatickerloader.py:761) 2020-09-14 16:32:54,618; DEBUG:tcapy.analysis.tcatickerloader: Combine trade/order data (tcatickerloader.py:782) 2020-09-14 16:32:54,625; DEBUG:tcapy.analysis.tcatickerloader: Calculating BenchmarkMarketSpreadToMid for market data (tcatickerloader.py:600) 2020-09-14 16:32:54,680; DEBUG:tcapy.analysis.tcatickerloader: Filter trades by venue (tcatickerloader.py:458) 2020-09-14 16:32:54,682; DEBUG:tcapy.analysis.tcatickerloader: Calculating derived fields and benchmarks (tcatickerloader.py:508) 2020-09-14 16:32:54,684; DEBUG:tcapy.analysis.tcatickerloader: Calculating execution fields (tcatickerloader.py:510) 2020-09-14 16:32:54,693; DEBUG:tcapy.analysis.tcatickerloader: Calculating benchmarks (tcatickerloader.py:536) 2020-09-14 16:32:54,694; DEBUG:tcapy.analysis.tcatickerloader: Calculating metrics (tcatickerloader.py:554) 2020-09-14 16:32:54,695; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricSlippage for trade_df (tcatickerloader.py:560) 2020-09-14 16:32:54,712; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricTransientMarketImpact for trade_df (tcatickerloader.py:560) 2020-09-14 16:32:54,742; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricSlippage for order_df (tcatickerloader.py:560) 2020-09-14 16:32:54,760; DEBUG:tcapy.analysis.tcatickerloader: Calculating MetricTransientMarketImpact for order_df (tcatickerloader.py:560)
Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.
2020-09-14 16:32:54,784; DEBUG:tcapy.analysis.tcatickerloader: Completed derived field calculations for EURUSD (tcatickerloader.py:568) 2020-09-14 16:32:54,786; DEBUG:tcapy.analysis.tcatickerloaderimpl: Generating downsampled market data for potentional display (tcatickerloaderimpl.py:275) 2020-09-14 16:32:54,831; DEBUG:tcapy.analysis.tcatickerloaderimpl: About to join (tcatickerloaderimpl.py:340) 2020-09-14 16:32:54,835; DEBUG:tcapy.analysis.tcatickerloaderimpl: Finished joining (tcatickerloaderimpl.py:356) 2020-09-14 16:32:54,837; DEBUG:tcapy.analysis.tcatickerloaderimpl: About to join (tcatickerloaderimpl.py:340) 2020-09-14 16:32:54,840; DEBUG:tcapy.analysis.tcatickerloaderimpl: Finished joining (tcatickerloaderimpl.py:356) 2020-09-14 16:32:54,865; DEBUG:tcapy.analysis.tcamarkettradeloader: Finished loading data and calculating metrics on individual _tickers (tcamarkettradeloader.py:248) 2020-09-14 16:32:54,866; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Constructing results form to summarize analysis... (tcamarkettradeloaderimpl.py:74) 2020-09-14 16:32:54,957; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Now join table results... (tcamarkettradeloaderimpl.py:122) 2020-09-14 16:32:54,958; DEBUG:tcapy.analysis.tcamarkettradeloaderimpl: Finished calculating results form and join table results! (tcamarkettradeloaderimpl.py:135)
Wall time: 3min 29s
Let's inspect the output. This time we have some different DataFrames, in particular:
dist_trade_df_slippage_by/pdf/side
- distribution of slippage on trades split up by buy/sellbar_trade_df_slippage_by/mean/ticker
- average of slippage on trades split by tickerbar_trade_df_slippage_by/mean/broker_id
- average of slippage on trades split by brokerdist_order_df_slippage_by/pdf/side
- distribution of slippage on orders split up by buy/sellbar_order_df_slippage_by/mean/ticker
- average of slippage on trades split by tickerbar_order_df_slippage_by/mean/broker_id
- average of slippage on trades split by brokerdict_of_df.keys()
odict_keys(['trade_df', 'order_df', 'sparse_market_trade_df', 'sparse_market_order_df', 'market_df_downsampled', 'candlestick_fig', 'dist_trade_df_slippage_by/pdf/side', 'bar_trade_df_slippage_by/mean/ticker', 'bar_trade_df_slippage_by/mean/broker_id', 'dist_order_df_slippage_by/pdf/side', 'bar_order_df_slippage_by/mean/ticker', 'bar_order_df_slippage_by/mean/broker_id', 'market_df'])
We can use the PlotRender
object (which is used by the GUI) to create different sorts of Ploty charts such as the timelines, distributions etc. In this case we are plotting the distribution of the slippage by buy/sell. Note that if we'd used the TCAResults
object instead it would have simplified this code.
plot_render = PlotRender()
# Generate Plotly JSON Figure
dist_fig = plot_render.plot_dist(dist_df=dict_of_df['dist_order_df_slippage_by/pdf/side'],
metric='slippage', split_by='side', width=chart_width, height=chart_height)
# Render the Plotly JSON Figure to view
chart.plot(dist_fig, engine='plotly', style=style)
2020-09-14 16:32:54,987; DEBUG:tcapy.vis.displaylisteners: Plotting distribution dist (displaylisteners.py:1239) 2020-09-14 16:32:55,005; DEBUG:tcapy.vis.displaylisteners: Rendered distribution dist (displaylisteners.py:1243)
Alternatively, we can push the output into a TCAResults
object, which can simplify the TCA output, in a much easier to use way. We can see that several charts end up getting rendered, such as timeline and distribution plots, together with more descriptive titles. We do not need to get into the low level details of how to do the plotting with Plotly, tcapy does that all for us using chartpy. However, if we want we can of course edit the returned Plotly Figure
object (see here on do this).
tca_results = TCAResults(dict_of_df, tca_request, chart_width=chart_width, chart_height=chart_height)
tca_results.render_computation_charts()
2020-09-14 16:32:55,151; DEBUG:tcapy.vis.displaylisteners: Plotting main timeline 2352 (displaylisteners.py:988) 2020-09-14 16:32:55,198; DEBUG:tcapy.vis.displaylisteners: Rendered plot 2352 and (displaylisteners.py:998) 2020-09-14 16:32:55,201; DEBUG:tcapy.vis.displaylisteners: Plotting main timeline 2346 (displaylisteners.py:988) 2020-09-14 16:32:55,249; DEBUG:tcapy.vis.displaylisteners: Rendered plot 2346 and (displaylisteners.py:998) 2020-09-14 16:32:55,283; DEBUG:tcapy.vis.displaylisteners: Plotting distribution Trades slippage by/pdf/side PDF (displaylisteners.py:1239) 2020-09-14 16:32:55,296; DEBUG:tcapy.vis.displaylisteners: Rendered distribution Trades slippage by/pdf/side PDF (displaylisteners.py:1243) 2020-09-14 16:32:55,300; DEBUG:tcapy.vis.displaylisteners: Plotting distribution Orders slippage by/pdf/side PDF (displaylisteners.py:1239) 2020-09-14 16:32:55,313; DEBUG:tcapy.vis.displaylisteners: Rendered distribution Orders slippage by/pdf/side PDF (displaylisteners.py:1243)
We can pickup the Plotly JSON Figure
object ready made to push to chartpy
for plotting.
chart.plot(tca_results.dist_charts['trade_df_slippage_by/pdf/side'], engine='plotly', style=style)
Depending on what we have specified to be calculated in the TCARequest
the TCAReport
can render the following charts (which are available as attributes
timeline_charts
- timeline aggregatessparse_market_charts
- mixture of market data with points to denote trades/ordersbar_charts
- average aggregates eg. average slippagedist_charts
- distribution charts eg. distribution of slippagescatter_charts
- scatter charts eg. plotting slippage vs. notionalstyled_tables
- tables in HTML formatWe can also get the underlying DataFrames using the following properties
trade_order
- DataFrames of trades/orderstimeline
- DataFrames of timelinessparse_market
- DataFrames with a mixture of market data/trade/order databar
- DataFrames with bar chart style datadist
- DataFrames with distribution style datascatter
- DataFrames with scatter style datatable
- DataFrames intended to be displayed as tablesmarket
- DataFrames with high frequency tick market dataLet's see inside the dist_charts
and bar_charts
to see keys are available.
print(tca_results.dist_charts.keys())
print(tca_results.bar_charts.keys())
dict_keys(['trade_df_slippage_by/pdf/side', 'order_df_slippage_by/pdf/side']) dict_keys(['trade_df_slippage_by/mean/ticker', 'trade_df_slippage_by/mean/broker_id', 'order_df_slippage_by/mean/ticker', 'order_df_slippage_by/mean/broker_id'])
Plot the average slippage by ticker! In this case, we only have one ticker, so probably just easier to report the number itself, which arount 0.5 bp.
print(tca_results.bar['order_df_slippage_by/mean/ticker'])
chart.plot(tca_results.bar_charts['order_df_slippage_by/mean/ticker'],
engine='plotly', style=style)
ticker ticker_index EURUSD -0.217804
Plot the average slippage by broker. Looks like broker4
hasn't been particularly good!
chart.plot(tca_results.bar_charts['order_df_slippage_by/mean/broker_id'], engine='plotly', style=style)
We can plot one of the PDF charts too.
chart.plot(tca_results.dist_charts['trade_df_slippage_by/pdf/side'],
engine='plotly', style=style)
Let's try seeing what the sparse_market_chart
property looks like.
print(tca_results.sparse_market_charts.keys())
dict_keys(['EURUSD_trade_df', 'EURUSD_order_df'])
We plot the market data benchmark alongside the trades as dots, sized by their executed notional and with different colors for buy (green) and sell (red).
chart.plot(tca_results.sparse_market_charts['EURUSD_trade_df'], engine='plotly', style=style)
Because these charts are already Plotly Figure objects in JSON, we can also display them directly, without the Chart
object.
tca_results.sparse_market_charts['EURUSD_order_df']
If we print the charts, we can see inside the Plotly Figure JSON format.
print(tca_results.sparse_market_charts['EURUSD_trade_df'])
Figure({ 'data': [{'connectgaps': True, 'line': {'color': 'rgba(0.20392156862745098, 0.5411764705882353, 0.7411764705882353, 1.0)', 'dash': 'solid', 'width': 3}, 'mode': 'lines', 'name': 'mid', 'text': '', 'type': 'scatter', 'x': [2017-05-05 00:00:00+00:00, 2017-05-05 01:00:00+00:00, 2017-05-05 02:00:00+00:00, ..., 2017-05-19 18:19:18.110000+00:00, 2017-05-19 19:00:00+00:00, 2017-05-19 20:00:00+00:00], 'y': array([1.0979177951812744, 1.0976229906082153, 1.0974935293197632, ..., '', 1.1204367876052856, 1.1205419301986694], dtype=object)}, {'connectgaps': True, 'line': {'color': 'rgba(0.0, 0.0, 0.0, 1.0)', 'dash': 'solid', 'width': 0.5}, 'mode': 'lines', 'name': 'bid', 'text': '', 'type': 'scatter', 'x': [2017-05-05 00:00:00+00:00, 2017-05-05 01:00:00+00:00, 2017-05-05 02:00:00+00:00, ..., 2017-05-19 18:19:18.110000+00:00, 2017-05-19 19:00:00+00:00, 2017-05-19 20:00:00+00:00], 'y': array([1.0978981256484985, 1.0976039171218872, 1.097473382949829, ..., '', 1.1204227209091187, 1.12051522731781], dtype=object)}, {'connectgaps': True, 'line': {'color': 'rgba(0.0, 0.0, 0.0, 1.0)', 'dash': 'solid', 'width': 0.5}, 'mode': 'lines', 'name': 'ask', 'text': '', 'type': 'scatter', 'x': [2017-05-05 00:00:00+00:00, 2017-05-05 01:00:00+00:00, 2017-05-05 02:00:00+00:00, ..., 2017-05-19 18:19:18.110000+00:00, 2017-05-19 19:00:00+00:00, 2017-05-19 20:00:00+00:00], 'y': array([1.0979373455047607, 1.097642183303833, 1.0975137948989868, ..., '', 1.1204509735107422, 1.1205687522888184], dtype=object)}, {'connectgaps': True, 'line': {'color': 'rgba(0.5568627450980392, 0.7294117647058823, 0.25882352941176473, 1.0)', 'dash': 'solid', 'width': 1}, 'marker': {'size': [0.0, 0.0, 0.0, ..., 5.466823811857581, 0.0, 0.0]}, 'mode': 'markers', 'name': 'buy trade', 'text': '', 'type': 'scatter', 'x': [2017-05-05 00:00:00+00:00, 2017-05-05 01:00:00+00:00, 2017-05-05 02:00:00+00:00, ..., 2017-05-19 18:19:18.110000+00:00, 2017-05-19 19:00:00+00:00, 2017-05-19 20:00:00+00:00], 'y': array(['', '', '', ..., 1.120751000477769, '', ''], dtype=object)}, {'connectgaps': True, 'line': {'color': 'rgba(0.8862745098039215, 0.2901960784313726, 0.2, 1.0)', 'dash': 'solid', 'width': 1}, 'marker': {'size': [0.0, 0.0, 0.0, ..., 5.466823811857581, 0.0, 0.0]}, 'mode': 'markers', 'name': 'sell trade', 'text': '', 'type': 'scatter', 'x': [2017-05-05 00:00:00+00:00, 2017-05-05 01:00:00+00:00, 2017-05-05 02:00:00+00:00, ..., 2017-05-19 18:19:18.110000+00:00, 2017-05-19 19:00:00+00:00, 2017-05-19 20:00:00+00:00], 'y': array(['', '', '', ..., '', '', ''], dtype=object)}, {'boxpoints': False, 'fillcolor': '#FFFFFF', 'line': {'color': '#808080'}, 'name': 'Increasing', 'showlegend': False, 'type': 'box', 'whiskerwidth': 0, 'x': array([datetime.datetime(2017, 5, 5, 2, 0), datetime.datetime(2017, 5, 5, 2, 0), datetime.datetime(2017, 5, 5, 2, 0), ..., datetime.datetime(2017, 5, 19, 20, 0), datetime.datetime(2017, 5, 19, 20, 0), datetime.datetime(2017, 5, 19, 20, 0)], dtype=object), 'y': [1.0973000526428223, 1.0975000858306885, 1.097825050354004, ..., 1.1207250356674194, 1.1207250356674194, 1.120845079421997]}, {'boxpoints': False, 'fillcolor': '#348ABD', 'line': {'color': '#348ABD'}, 'name': 'Decreasing', 'showlegend': False, 'type': 'box', 'whiskerwidth': 0, 'x': array([datetime.datetime(2017, 5, 5, 0, 0), datetime.datetime(2017, 5, 5, 0, 0), datetime.datetime(2017, 5, 5, 0, 0), ..., datetime.datetime(2017, 5, 19, 19, 0), datetime.datetime(2017, 5, 19, 19, 0), datetime.datetime(2017, 5, 19, 19, 0)], dtype=object), 'y': [1.097564935684204, 1.098075032234192, 1.0979499816894531, ..., 1.1204099655151367, 1.1204099655151367, 1.1211400032043457]}], 'layout': {'height': 500, 'legend': {'bgcolor': '#F5F6F9', 'font': {'color': '#4D5663'}, 'x': 0.05, 'y': 1}, 'paper_bgcolor': 'rgba(0,0,0,0)', 'plot_bgcolor': 'rgba(0,0,0,0)', 'showlegend': True, 'template': '...', 'title': {'font': {'color': '#4D5663'}, 'text': 'EURUSD trades'}, 'width': 800, 'xaxis': {'gridcolor': '#E1E5ED', 'showgrid': True, 'tickfont': {'color': '#4D5663'}, 'title': {'font': {'color': '#4D5663'}, 'text': ''}, 'zerolinecolor': '#E1E5ED'}, 'yaxis': {'gridcolor': '#E1E5ED', 'showgrid': True, 'tickfont': {'color': '#4D5663'}, 'title': {'font': {'color': '#4D5663'}, 'text': ''}, 'zerolinecolor': '#E1E5ED'}} })
We can update the properties of a Plotly figure too, such as the title, without having to redraw the whole chart.
trade_fig = tca_results.sparse_market_charts['EURUSD_trade_df']
trade_fig.update_layout(title='Look the title is changed!')
trade_fig
We've seen a quick 10 minute view of tcapy, with some examples of how to call the library. We've also done many other notebooks, showing some of the more advanced functionality of tcapy, which you can look at next. If you are interesting supporting this project, please contact saeed@cuemacro.com.