Quickstart

Here is a quick rundown of ffn’s capabilities. For a more complete guide, read the source, or check out the API docs.

import ffn
%matplotlib inline

Data Retrieval

The main method for data retrieval is the get function. The get function uses a data provider to download data from an external service and packs that data into a pandas DataFrame for further manipulation.

Note

You should note that upon import ffn modifies the pandas.core.base.PandasObject to provide added functionality to pandas objects, including DataFrames.

data = ffn.get('agg,hyg,spy,eem,efa', start='2010-01-01', end='2014-01-01')
print(data.head())
                   agg        hyg        spy        eem        efa
 Date
 2010-01-04  74.942818  43.466671  89.225403  33.181232  38.846069
 2010-01-05  75.283783  43.672871  89.461578  33.422070  38.880318
 2010-01-06  75.240242  43.785816  89.524582  33.491989  39.044666
 2010-01-07  75.153183  43.962566  89.902481  33.297764  38.894009
 2010-01-08  75.196701  44.031300  90.201675  33.561905  39.202148

By default, the data is downloaded from Yahoo! Finance and the Adjusted Close is used as the security’s price. Other data sources are also available and you may select other fields as well. Fields are specified by using the following format: {ticker}:{field}. So, if we want to get the Open, High, Low, Close for aapl, we would do the following:

print(ffn.get('aapl:Open,aapl:High,aapl:Low,aapl:Close', start='2010-01-01', end='2014-01-01').head())
             aaplopen  aaplhigh   aapllow  aaplclose
 Date
 2010-01-04  7.622500  7.660714  7.585000   7.643214
 2010-01-05  7.664286  7.699643  7.616071   7.656429
 2010-01-06  7.656429  7.686786  7.526786   7.534643
 2010-01-07  7.562500  7.571429  7.466071   7.520714
 2010-01-08  7.510714  7.571429  7.466429   7.570714

The default data provider is ffn.data.web(). This is basically just a thin wrapper around pandas’ pandas.io.data provider. Please refer to the appropriate docs for more info (data sources, etc.). The ffn.data.csv() provider is also available when we want to load data from a local file. In this case, we can tell ffn.data.get() to use the csv provider. In this case, we also want to merge this new data with the existing data we downloaded earlier. Therefore, we will provide the data object as the existing argument, and the new data will be merged into the existing DataFrame.

data = ffn.get('dbc', provider=ffn.data.csv, path='test_data.csv', existing=data)
print(data.head())
                   agg        hyg        spy        eem        efa    dbc
 Date
 2010-01-04  74.942818  43.466671  89.225403  33.181232  38.846069  25.24
 2010-01-05  75.283783  43.672871  89.461578  33.422070  38.880318  25.27
 2010-01-06  75.240242  43.785816  89.524582  33.491989  39.044666  25.72
 2010-01-07  75.153183  43.962566  89.902481  33.297764  38.894009  25.40
 2010-01-08  75.196701  44.031300  90.201675  33.561905  39.202148  25.38

As we can see above, the dbc column was added to the DataFrame. Internally, get is using the function ffn.merge, which is useful when you want to merge TimeSeries and DataFrames together. We plan on adding many more data sources over time. If you know your way with Python and would like to contribute a data provider, please feel free to submit a pull request - contributions are always welcome!

Data Manipulation

Now that we have some data, let’s start manipulating it. In quantitative finance, we are often interested in the returns of a given time series. Let’s calculate the returns by simply calling the to_returns or to_log_returns extension methods.

returns = data.to_log_returns().dropna()
print(returns.head())
                  agg       hyg       spy       eem       efa       dbc
 Date
 2010-01-05  0.004539  0.004733  0.002643  0.007232  0.000881  0.001188
 2010-01-06 -0.000579  0.002583  0.000704  0.002090  0.004218  0.017651
 2010-01-07 -0.001158  0.004029  0.004212 -0.005816 -0.003866 -0.012520
 2010-01-08  0.000579  0.001562  0.003322  0.007901  0.007891 -0.000788
 2010-01-11 -0.000772 -0.000893  0.001395 -0.002085  0.008176 -0.003157

Let’s look at the different distributions to see how they look.

ax = returns.hist(figsize=(12, 5))
_images/quickstart_11_0.png

We can also use the numerous functions packed into numpy, pandas and the like to further analyze the returns. For example, we can use the corr function to get the pairwise correlations between assets.

returns.corr().as_format('.2f')
agg hyg spy eem efa dbc
agg 1.00 -0.12 -0.33 -0.23 -0.29 -0.18
hyg -0.12 1.00 0.77 0.75 0.76 0.49
spy -0.33 0.77 1.00 0.88 0.92 0.59
eem -0.23 0.75 0.88 1.00 0.90 0.62
efa -0.29 0.76 0.92 0.90 1.00 0.61
dbc -0.18 0.49 0.59 0.62 0.61 1.00

Here we used the convenience method as_format to have a prettier output. We could also plot a heatmap to better visualize the results.

returns.plot_corr_heatmap();
_images/quickstart_15_0.png

We used the ffn.core.plot_corr_heatmap(), which is a convenience method that simply calls ffn’s ffn.core.plot_heatmap() with sane arguments.

Let’s start looking at how all these securities performed over the period. To achieve this, we will plot rebased time series so that we can see how they each performed relative to eachother.

ax = data.rebase().plot(figsize=(12,5))
_images/quickstart_17_0.png

Performance Measurement

For a more complete view of each asset’s performance over the period, we can use the ffn.core.calc_stats() method which will create a ffn.core.GroupStats object. A GroupStats object wraps a bunch of ffn.core.PerformanceStats objects in a dict with some added convenience methods.

perf = data.calc_stats()

Now that we have our GroupStats object, we can analyze the performance in greater detail. For example, the plot method yields a graph similar to the one above.

perf.plot();
_images/quickstart_21_0.png

We can also display a wide array of statistics that are all contained in the PerformanceStats object. This will probably look crappy in the docs, but do try it out in a Notebook. We are also actively trying to improve the way we display this wide array of stats.

print(perf.display())
 Stat                 agg         hyg         spy         eem         efa         dbc
 -------------------  ----------  ----------  ----------  ----------  ----------  ----------
 Start                2010-01-04  2010-01-04  2010-01-04  2010-01-04  2010-01-04  2010-01-04
 End                  2013-12-31  2013-12-31  2013-12-31  2013-12-31  2013-12-31  2013-12-31
 Risk-free rate       0.00%       0.00%       0.00%       0.00%       0.00%       0.00%

 Total Return         16.36%      39.22%      76.92%      5.46%       33.43%      1.66%
 Daily Sharpe         1.11        0.97        0.93        0.18        0.44        0.11
 Daily Sortino        1.84        1.51        1.48        0.29        0.69        0.17
 CAGR                 3.87%       8.65%       15.37%      1.34%       7.50%       0.41%
 Max Drawdown         -5.14%      -10.06%     -18.61%     -30.87%     -25.86%     -24.34%
 Calmar Ratio         0.75        0.86        0.83        0.04        0.29        0.02

 MTD                  -0.56%      0.41%       2.59%       -0.41%      2.18%       0.59%
 3m                   0.02%       3.42%       10.52%      3.48%       6.08%       -0.39%
 6m                   0.57%       5.84%       16.32%      9.55%       18.12%      2.11%
 YTD                  -1.98%      5.75%       32.31%      -3.65%      21.44%      -7.63%
 1Y                   -1.98%      5.75%       32.31%      -3.65%      21.44%      -7.63%
 3Y (ann.)            3.08%       7.83%       16.07%      -2.34%      8.17%       -2.34%
 5Y (ann.)            -           -           -           -           -           -
 10Y (ann.)           -           -           -           -           -           -
 Since Incep. (ann.)  3.87%       8.65%       15.37%      1.34%       7.50%       0.41%

 Daily Sharpe         1.11        0.97        0.93        0.18        0.44        0.11
 Daily Sortino        1.84        1.51        1.48        0.29        0.69        0.17
 Daily Mean (ann.)    3.86%       8.70%       15.73%      4.35%       9.73%       1.83%
 Daily Vol (ann.)     3.48%       8.97%       16.83%      24.56%      22.31%      16.84%
 Daily Skew           -0.40       -0.55       -0.39       -0.12       -0.26       -0.47
 Daily Kurt           2.30        7.50        4.03        3.06        3.64        2.90
 Best Day             0.84%       3.05%       4.65%       7.20%       6.74%       4.34%
 Worst Day            -1.24%      -4.26%      -6.51%      -8.34%      -7.46%      -6.70%

 Monthly Sharpe       1.23        1.11        1.22        0.30        0.60        0.27
 Monthly Sortino      2.49        2.19        2.36        0.53        1.06        0.43
 Monthly Mean (ann.)  3.59%       9.51%       16.99%      6.43%       11.06%      4.61%
 Monthly Vol (ann.)   2.93%       8.56%       13.91%      21.45%      18.41%      17.10%
 Monthly Skew         -0.34       0.14        -0.32       -0.10       -0.37       -0.74
 Monthly Kurt         0.02        1.75        0.24        1.28        0.17        1.16
 Best Month           1.77%       8.49%       10.91%      16.27%      11.61%      9.89%
 Worst Month          -2.00%      -5.30%      -7.95%      -17.89%     -11.19%     -14.62%

 Yearly Sharpe        0.65        2.79        1.10        -0.06       0.50        -0.40
 Yearly Sortino       2.77        inf         inf         -0.11       1.32        -0.58
 Yearly Mean          3.16%       7.85%       16.73%      -1.13%      9.32%       -2.24%
 Yearly Vol           4.86%       2.82%       15.22%      19.05%      18.72%      5.57%
 Yearly Skew          -0.54       1.49        0.22        0.58        -1.69       0.27
 Yearly Kurt          -           -           -           -           -           -
 Best Year            7.70%       11.06%      32.31%      19.05%      21.44%      3.50%
 Worst Year           -1.98%      5.75%       1.89%       -18.79%     -12.23%     -7.63%

 Avg. Drawdown        -0.48%      -1.18%      -1.78%      -5.16%      -4.96%      -5.09%
 Avg. Drawdown Days   16.95       15.70       17.55       78.22       60.04       107.85
 Avg. Up Month        0.83%       1.86%       3.58%       5.87%       4.37%       4.28%
 Avg. Down Month      -0.49%      -2.31%      -3.21%      -3.41%      -4.15%      -3.35%
 Win Year %           66.67%      100.00%     100.00%     33.33%      66.67%      33.33%
 Win 12m %            81.08%      97.30%      94.59%      59.46%      70.27%      45.95%
 None

Lots to look at here. We can also access the underlying PerformanceStats for each series, either by index or name.

# we can also use perf[2] in this case
perf['spy'].display_monthly_returns()
   Year    Jan    Feb    Mar    Apr    May    Jun    Jul    Aug    Sep    Oct    Nov    Dec    YTD
 ------  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----
   2010  -5.24   3.12   6.09   1.55  -7.95  -5.17   6.83  -4.5    8.96   3.82   0      6.69  13.14
   2011   2.33   3.47   0.01   2.9   -1.12  -1.69  -2     -5.5   -6.94  10.91  -0.41   1.04   1.89
   2012   4.64   4.34   3.22  -0.67  -6.01   4.06   1.18   2.51   2.54  -1.82   0.57   0.89  15.99
   2013   5.12   1.28   3.8    1.92   2.36  -1.33   5.17  -3      3.16   4.63   2.96   2.59  32.31
perf[2].plot_histogram();
_images/quickstart_26_0.png

Most of the stats are also available as pandas objects - see the stats, return_table, lookback_returns attributes.

perf['spy'].stats
 start                    2010-01-04 00:00:00
 end                      2013-12-31 00:00:00
 rf                                       0.0
 total_return                        0.769155
 cagr                                 0.15375
 max_drawdown                       -0.186055
 calmar                              0.826367
 mtd                                 0.025926
 three_month                         0.105247
 six_month                           0.163183
 ytd                                 0.323077
 one_year                            0.323077
 three_year                           0.16066
 five_year                                NaN
 ten_year                                 NaN
 incep                                0.15375
 daily_sharpe                         0.93439
 daily_sortino                       1.478916
 daily_mean                          0.157279
 daily_vol                           0.168323
 daily_skew                         -0.388777
 daily_kurt                          4.028481
 best_day                            0.046499
 worst_day                          -0.065123
 monthly_sharpe                      1.221065
 monthly_sortino                     2.362922
 monthly_mean                        0.169906
 monthly_vol                         0.139146
 monthly_skew                       -0.319921
 monthly_kurt                        0.235707
 best_month                          0.109147
 worst_month                        -0.079455
 yearly_sharpe                       1.099284
 yearly_sortino                           inf
 yearly_mean                          0.16731
 yearly_vol                          0.152199
 yearly_skew                          0.21847
 yearly_kurt                              NaN
 best_year                           0.323077
 worst_year                           0.01895
 avg_drawdown                       -0.017845
 avg_drawdown_days                  17.550725
 avg_up_month                        0.035827
 avg_down_month                     -0.032066
 win_year_perc                            1.0
 twelve_month_win_perc               0.945946
 dtype: object

Numerical Routines and Financial Functions

ffn also provides commonly used numerical routines and plans to add many more in the future. One can easily determine the proper weights using a mean-variance approach using the ffn.core.calc_mean_var_weights() function.

returns.calc_mean_var_weights().as_format('.2%')
 agg    79.52%
 hyg     6.47%
 spy    14.01%
 eem     0.00%
 efa     0.00%
 dbc     0.00%
 dtype: object

Some other interesting functions are the clustering routines, such as a Python implementation of David Varadi’s Fast Threshold Clustering Algorithm (FTCA)

returns.calc_ftca(threshold=0.8)
 {1: ['eem', 'spy', 'efa'], 2: ['agg'], 3: ['dbc'], 4: ['hyg']}