Azure Devops - Create integration test pipeline that runs after all other triggered pipelines

Question

I have a single repo that is segmented into a number of projects, each of which have a build that generates a Docker image. I've configured Azure devops to trigger on the proper paths for each project such that I get the right images build when their subdirectories change. One PR can trigger the build of one or multiple images.

Once all my images are built, I want to trigger an integration test job that starts all of the images that comprise my service. If the PR didn't change one it should run service:latest rather than the one(s) which were built. How can I create a test pipeline that doesn't start until each of the other pipelines finish and know which image version to use? For the image, I think I can just use something like the PR number or commit hash in the image tag and revert to service:latest if that image doesn't exist.

What's not clear is how to have the integration test job wait for all of the dependent (ideally parallelized) builds to finish. I considered the idea of a single pipeline with different build jobs, but this creates another problem of wanting to have independently triggered CDs for each build artifact output. I never want to re-deploy images that haven't changed and I want to be able to track the deployments of those which have changed so a single CD for all builds is a bad idea.

question from:https://stackoverflow.com/questions/66057008/azure-devops-create-integration-test-pipeline-that-runs-after-all-other-trigge

Answer 1

You can create a Release pipeline to run your test job and configure invoke rest api Gate to check if all the pipeline were completed. See below steps:

1, Create a release pipeline.

2, Add all the artifacts from the other pipelines as the release pipeline Artifacts source. So that the pipeline will be triggered when there is any new version of artifact being published.

3,Create a generic service connection to connect to your azure devops organization. See this thread for detailed steps.

4, Configure invoke rest api Gate for the release pipeline. See below screenshot.

5, Configure the invoke rest api gate to call Build-latest rest api to get the latest build of the other dependency pipeline. See below screen:

6, Add multiple invoke rest api gate one for each of your dependency pipelines

7, Configure the evaluation option for the Gate.

After above configurations, the release pipeline will be executed if all the pipeline is evaluated to be completed and succeeded.

To decide which image version to use, you can clone your source code in the release pipeline and run below git command to check which folders are changed. Then you can decide which images are to use. Please see this thread for more information.

git diff HEAD HEAD~ --name-only