python - How should I use the h5py library for storing time series data?

Question

I have some time series data that i previously stored as hdf5 files using pytables. I recently tried storing the same with h5py lib. However, since all elements of numpy array have to be of same dtype, I have to convert the date (which is usually the index) into 'float64' type before storing it using h5py lib. When I use pytables, the index and its dtype are preserved which makes it possible for me to query the time-series without the need of pulling it all in the memory. I guess with h5py that is not possible. am I missing something here? And if not, under what situations should i use h5py lib to store time series data? I ask this question cause, clarity on this could help me design a more efficient (processing & storage wise) project.

below is simple code, where I have to lose index information in order to store it as a single dtype object

dt_range = pd.date_range('2016-12-01','2016-12-10')
data = np.arange(0,20).reshape(-1,2)
df = pd.DataFrame(data,index = dt_range, columns = list('ab'), dtype = 'float')
df.index  = df.index.to_julian_date()
df = df.reset_index()
h = h5py.File(r'pathemp.h5', 'w')
dset = h.create_dataset('temp',data = df.values, shape = (10,3))

Answer 1

I'd use pandas to_hdf

dt_range = pd.date_range('2016-12-01','2016-12-10')
data = np.arange(0,20).reshape(-1,2)
df = pd.DataFrame(data,index = dt_range, columns = list('ab'), dtype = 'float')
df.index  = df.index.to_julian_date()
df = df.reset_index()

with pd.HDFStore('temp.h5', 'w') as h:
    df.to_hdf(h, 'temp')

pd.read_hdf('temp.h5', 'temp')