Recommended way to serve PyTorch predictions with Google AI Platform

Question

Reading through Google's documentation, it seems there are two different ways of serving predictions form PyTorch models.

1. Using custom prediction routines
2. Creating a custom container and using torchserve

Although option 1 seems more popular from reading through online blogs such as this one, it seems weird to not use option 2 since the documentation explicitly mentions PyTorch.

Can anyone provide why one might choose one option over another?

question from:https://stackoverflow.com/questions/66067587/recommended-way-to-serve-pytorch-predictions-with-google-ai-platform

Answer 1

1 is simpler in that you don't have to worry about creating an http handler, but it's probably harder for you to test with locally.

2 you build your own container and so you can try running it on your local first. It's slightly more work because you have to be able to set up a http handler.