xgboost - How to disable GPUs in H2O AutoML

Question

When I run an experiment with H2O AutoML, I got the error: "terminate called after throwing an instance of 'thrust::system::system_error' what(): parallel_for failed: invalid resource handle". This error message comes from XGBoost and it is because of the GPU limit exceed.

While I'm using the regular XGBoost, I set the cuda visible devices parameter to blank to disable GPUs. However, this arguments seems to be ignored in H2O AutoML - XGBoost implementation.

import os
os.environ["CUDA_VISIBLE_DEVICES"] = ""

Currently, the only xgboost can be run on gpu in H2O AutoML.

The question it that anybody knows how to disable GPUs in H2O AutoML?

As a workaround, I excluded XGBoost algorithm to run my experiment for now. The trouble is passed when I exclude XGBoost but I do not want to give up the power of XGBoost.

from h2o.automl import H2OAutoML
model = H2OAutoML(max_runtime_secs = 60*60*2, exclude_algos = ["XGBoost"])

question from:https://stackoverflow.com/questions/65897690/how-to-disable-gpus-in-h2o-automl

Answer 1

That's definitely an oversight and we will need to add the ability to turn on/off and/or specify the GPU. I opened a ticket for this. I wonder if there's a way to temporarily disable the GPU at the system level (outside of H2O/Python) in the meantime? Thanks for the report!