python - Why cannot numpy arrays convert from datetime to np.datetime64 implicitly?

Question

Say, I have a datetime:

given_time = datetime(2013, 10, 8, 0, 0, 33, 945109,
                      tzinfo=psycopg2.tz.FixedOffsetTimezone(offset=60, 
                                                             name=None))

I would like to transform it into np.datetime64:

np.datetime64(given_time)
> numpy.datetime64('2013-10-08T00:00:33.945109+0100')

It works well. However, if I have an array of given_time:

given_times = np.array([given_time]*3) # dtype is object

Both given_times.astype('datetime64') and given_times = np.array([given_time] * 3, dtype=np.datetime64) would trigger TypeError: Cannot cast datetime.datetime object from metadata [us] to [D] according to the rule 'same_kind'

So, I have to specify the unit:

given_times.astype('datetime64[us]')
# or
given_times = np.array([given_time]*3, dtype='datetime64[us]')

My question is, why do I have to specify the unit here? It doesn't require unit in np.datatime64 constructor.

Answer 1

I know it's an old question, but I'd try to answer in case anyone else comes across this.

1. As of 1.11, numpy doesn't try to automatically convert iterables of date/datetime objects to datetime64 arrays, this is pretty clear from this excerpt in the test suite:

# at the moment, we don't automatically convert these to datetime64

dt = datetime.date(1970, 1, 1)
arr = np.array([dt])
assert_equal(arr.dtype, np.dtype('O'))

dt = datetime.datetime(1970, 1, 1, 12, 30, 40)     
arr = np.array([dt])
assert_equal(arr.dtype, np.dtype('O'))

Ideally, numpy would figure that datetime64 with correct units could be used; see this issue.

2. When constructing datetime64 from a scalar, the unit it set to M8[D] for date objects and to M8[us] for datetime objects (a relevant test).

3. When you specify dtype='datetime64', or, similarly, dtype='M8', the units are set to "generic", which later resolves to M8[D] (although it would be logical to have it resolve to M8[D], see this issue):

>>> np.datetime_data(np.dtype('datetime64'))
('generic', 1)
>>> np.datetime_data(np.dtype('M8'))
('generic', 1)
>>> np.datetime_data(np.dtype('M8[D]'))
('D', 1)
>>> np.datetime_data(np.dtype('M8[us]'))
('us', 1)

4. given_times.astype('datetime64') no longer raises an exception -- this was fixed in 1.11.

5. Starting from 1.11, datetime64 objects are timezone-naive, so passing a datetime object with tzinfo set like in the provided example will trigger a deprecation warning.