SEP loader

[philschrad]

I've used the sep script to ingest the sep data set from a dump and it works but with some warnings:

a total of 8639 securities were loaded into this bundle
/Users/phil/miniconda3/envs/p3zip/lib/python3.5/site-packages/zipline/data/us_equity_pricing.py:931: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  if not issubdtype(actual, expected):
[2018-03-27 04:38:36.511868] WARNING: UsEquityPricing: Couldn't compute ratio for dividend {'ex_date': numpy.datetime64('2008-01-02T00:00:00.000000000'), 'amount': nan, 'sid': 220}
[2018-03-27 04:38:36.513775] WARNING: UsEquityPricing: Couldn't compute ratio for dividend {'ex_date': numpy.datetime64('2008-01-02T00:00:00.000000000'), 'amount': 0.1, 'sid': 257}
[2018-03-27 04:38:36.514156] WARNING: UsEquityPricing: Couldn't compute ratio for dividend {'ex_date': numpy.datetime64('2008-01-02T00:00:00.000000000'), 'amount': nan, 'sid': 303}
[2018-03-27 04:38:36.514422] WARNING: UsEquityPricing: Couldn't compute ratio for dividend {'ex_date': numpy.datetime64('2008-01-02T00:00:00.000000000'), 'amount': 0.18, 'sid': 538}
[2018-03-27 04:38:36.514710] WARNING: UsEquityPricing: Couldn't compute ratio for dividend {'ex_date': numpy.datetime64('2008-01-02T00:00:00.000000000'), 'amount': 0.24, 'sid': 579}
...

and then when I try to use it with the simple pipeline engine I get this:

~/miniconda3/envs/p3zip/lib/python3.5/site-packages/zipline/data/us_equity_pricing.py in load_adjustments(self, columns, dates, assets)
   1302             list(columns),
   1303             dates,
-> 1304             assets,
   1305         )
   1306 

~/miniconda3/envs/p3zip/lib/python3.5/site-packages/zipline/data/_adjustments.pyx in zipline.data._adjustments.load_adjustments_from_sqlite()

~/miniconda3/envs/p3zip/lib/python3.5/site-packages/zipline/data/_adjustments.pyx in zipline.data._adjustments.load_adjustments_from_sqlite()

TypeError: a float is required

any ideas?

[pbharrin]

Thanks for bringing this to my attention. I use Python 2.7 and don’t get the float warning or error. I see you are using Python 3.5, that may be the source of the error. I would like for this to work on both versions.

[GelliFrancesco]

It is not a Python's version issue. It is because the first row for that ticker has a non-zero dividend. When adjusting the prices, the code looks for the previous closing price, but since it's the first date for that stock there is no previous data, hence that case is skipped and the array ratios will have a "hole" and look like [...0.99,0.99, 0.98, nan, 0.99...].

This is probably the reason why the pipeline crashes: it basically expects a float but a nan is found. As a workaround you can either manually remove such cases where a dividend is happening on the first date, or replace such nan with 1.0.

Cheers