Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.3k views
in Technique[技术] by (71.8m points)

deployment - Use mlflow to serve a custom python model for scoring

I am using Python code generated from an ml software with mlflow to read a dataframe, perform some table operations and output a dataframe. I am able to run the code successfully and save the new dataframe as an artifact. However I am unable to log the model using log_model because it is not a lr or classifier model where we train and fit. I want to log a model for this so that it can be served with new data and deployed with a rest API

df = pd.read_csv(r"/home/xxxx.csv")


with mlflow.start_run():
    def getPrediction(row):
        
        perform_some_python_operaions 

        return [Status_prediction, Status_0_probability, Status_1_probability]
    columnValues = []
    for column in columns:
        columnValues.append([])

    for index, row in df.iterrows():
        results = getPrediction(row)
        for n in range(len(results)):
            columnValues[n].append(results[n])

    for n in range(len(columns)):
        df[columns[n]] = columnValues[n]

    df.to_csv('dataset_statistics.csv')
    mlflow.log_artifact('dataset_statistics.csv')
   
question from:https://stackoverflow.com/questions/65887231/use-mlflow-to-serve-a-custom-python-model-for-scoring

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

MLflow supports custom models of mlflow.pyfunc flavor. You can create a custom class inherited from the mlflow.pyfunc.PythonModel, that needs to provide function predict for performing predictions, and optional load_context to load the necessary artifacts, like this (adopted from the docs):

class MyModel(mlflow.pyfunc.PythonModel):

    def load_context(self, context):
        # load your artifacts

    def predict(self, context, model_input):
        return my_predict(model_input.values)

You can log to MLflow whatever artifacts you need for your models, define Conda environment if necessary, etc.
Then you can use save_model with your class to save your implementation, that could be loaded with load_model and do the predict using your model:

mlflow.pyfunc.save_model(
        path=mlflow_pyfunc_model_path, python_model=MyModel(), artifacts=artifacts)

# Load the model in `python_function` format
loaded_model = mlflow.pyfunc.load_model(mlflow_pyfunc_model_path)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...