Python 2.7: import performance

Question

currently, I am importing bunch of .py files scattered across the file system via:

def do_import(name):
    import imp

    fp, pathname, description = imp.find_module(name)
    with fp:
        return imp.load_module(name, fp, pathname, description)

known_py_files = ['workingDir/import_one.py', 'anotherDir/import_two.py'] # and so forth
for py_file in known_py_files:
    do_import(py_file)

when I've timed the .py files as such below, they are in the magnitude of e-5 and e-6.

import_one.py

import time

import_stime = time.time()
import_dur = time.time() - import_stime
print import_dur

However, the call to do_import() is in the magnitude of e-3. I am guessing because of the overhead of importing it.

This is a problematic for me because im importing lots of files serially and the time to import adds up.

Is there a faster way to import than the approach mentioned above?

Answer 1

If all of the files are under one directory, you can create an __init__.py empty file in each level of the nested directory, and import the name of the root directory, like import root in the following example:

/ root
    - __init__.py
    / workingDir
        - __init__.py
        - import_one.py
    / anotherDir
        - __init__.py
        - import_two.py

That structure is called "package".