jenkins - Jenkinsfile print stdout during docker run

Question

Jenkins declarative job running python script inside docker container:

stage('Run Python script') {
        steps {
            script {
                withAWS(role: 'nrg-jenkins', roleAccount: env.CRISPR_AWS_ACCOUNT_ID, region: env.CRISPR_REGION) {
                    stdout = app.inside('-e "LOCAL_FILE=s3file"' +
                                        ' -e INPUT_TYPE=' + params.INPUT_TYPE +
                                        ' -e "OUTPUT_CSV_PATH=output_csv"' +
                                        ' -e GENOME_ID=' + params.GENOME_ID +
                                        ' -e MYSQL_USER=' + env.MYSQL_USER +
                                        ' -e MYSQL_PASSWORD=' + env.MYSQL_PASSWORD +
                                        ' -e MYSQL_USER=' + env.MYSQL_USER +
                                        ' -e MYSQL_PORT=' + env.MYSQL_PORT +
                                        ' -e MYSQL_HOST=' + env.MYSQL_HOST)
                                {
                                    sh(script:'python /gff2mysql/gff2mysql.py', returnStdout:true)
                                }
                    echo stdout
                }
            }
        }
    }

The problem is that this implementation prints the stdout only when the script is finished its work.

Is there possibility to see stdout from the container during the actual run continuously?

question from:https://stackoverflow.com/questions/66051222/jenkinsfile-print-stdout-during-docker-run

Answer 1

You no need to capture the output and print that separately .docker.image('').inside() {} Jenkins DSL will do it for you. Take a look some of the example from Jenkins

If I tried to phrase a similar example like you to run a python script inside the docker build agent and get the python produces output, it will be like my below example. The only extra part to use with python command is -u because it will enable -u : unbuffered binary stdout and stderr. with this, you can see python ongoing pyton script printed output data along with your Jenkins log without waiting for the script to finish

node {
    docker.image('python:alpine3.13').inside {
        stage('prepare python script') {
            writeFile file: 'my-script.py', text: """
                # This script will just print one number / sec and terminate if count = 10 
                import time
    
                def iterate():
                    for x in range(100):
                        print(x)
                        time.sleep(1)
                        if(x == 10):
                            break
                        else:
                            pass
                
                if __name__ == "__main__":
                    iterate()
            """.stripIndent()
            
        }
        stage('run python script') {
            sh "python -u my-script.py"
        }
    }
}

and for your case, it will be like

stage('Run Python script') {
        steps {
            script {
                withAWS(role: 'nrg-jenkins', roleAccount: env.CRISPR_AWS_ACCOUNT_ID, region: env.CRISPR_REGION) {
                    docker.image('image:tag').inside('-e "LOCAL_FILE=s3file"' +
                                        ' -e INPUT_TYPE=' + params.INPUT_TYPE +
                                        ' -e "OUTPUT_CSV_PATH=output_csv"' +
                                        ' -e GENOME_ID=' + params.GENOME_ID +
                                        ' -e MYSQL_USER=' + env.MYSQL_USER +
                                        ' -e MYSQL_PASSWORD=' + env.MYSQL_PASSWORD +
                                        ' -e MYSQL_USER=' + env.MYSQL_USER +
                                        ' -e MYSQL_PORT=' + env.MYSQL_PORT +
                                        ' -e MYSQL_HOST=' + env.MYSQL_HOST)
                                {
                                    sh "python -u /gff2mysql/gff2mysql.py"
                                }
                 
                }
            }
        }
    }

Here are some more example to achieve your expected build by using the docker DSL

Scripted Pipeline

def toolToInstall = 'curl'
def app = docker.image('alpine')
node {
    stage('update') {
        app.inside("-u 0:0 -e TOOL_TO_INSTALL=${toolToInstall}") {
            sh """
                apk update
                echo $TOOL_TO_INSTALL
                apk add $TOOL_TO_INSTALL
            """
        }
    }
}

Declarative Pipeline

def toolToInstall = 'curl'
pipeline {
    agent {
        docker {
            image 'alpine'
            args  '-e TOOL_TO_INSTALL=${toolToInstall}'
            args  '-u 0:0'
        }
    }
    stages {
        stage('do something') {
            steps {
                sh """
                    apk update
                    echo $TOOL_TO_INSTALL
                    apk add $TOOL_TO_INSTALL
                """
            }
        }
    }
}