How to access Junit test counts in Jenkins Pipeline project

Question

I have just started with Jenkins

My freestyle project used to report JUnit tests results in Slack like this

MyJenkinsFreestyle - #79 Unstable after 4 min 59 sec (Open)
Test Status:
    Passed: 2482, Failed: 13, Skipped: 62

Now I have moved the same to pipeline project, and all is good except that Slack notifications do not have Test Status

done MyPipelineProject #68 UNSTABLE

I understand I have to construct the message to send to Slack, and I have done that above for now.

The only issue is how do I read the test status - the passed count, failed count etc. This is called "test summary" in Jenkins slack-plugin commit, and here is the screenshot

So how do I access Junit tests count/details in Jenkins Pipeline project ? - so that these are reported in notifications.

UPDATE: In the Freestyle project, the Slack notification itself has the "test summary", and there is no option to opt (or not) for the test summary.

In Pipeline project, my "junit" command to "Publish JUnit test results" is before sending Slack notification.

So in code those lines look like this (this are last lines of the last stage):

bat runtests.bat
junit 'junitreport/xml/TEST*.xml'
slackSend channel: '#testschannel', color: 'normal', message: "done ${env.JOB_NAME} ${env.BUILD_NUMBER} (<${env.BUILD_URL}|Open>)";

Solution 1

From this presentation of Cloudbees I found that it should be possible via "build" object. It has code like

def testResult = build.testResultAction
def total = testResult.totalCount

But currentBuild does not provide access to testResultAction.

So kept searching and found this post "react on failed tests in pipeline script". There Robert Sandell has given "pro tip"

Pro tip, requires some "custom whitelisting":

AbstractTestResultAction testResultAction =  currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
if (testResultAction != null) {
    echo "Tests: ${testResultAction.failCount} / ${testResultAction.failureDiffString} failures of ${testResultAction.totalCount}.\n\n" 
}

This worked like a charm - just that I had to deselect "Groovy sandbox" checkbox. Now I have these in the build log

Tests: 11  / ±0 failures of 2624

Now I will use this to prepare string to notify in slack with test results.

---

UPDATE:

Finally, the function I used to get output like the following (Note the "failure diff" after failed tests is very useful)

Test Status:
  Passed: 2628, Failed: 6  / ±0, Skipped: 0

Is the following:

import hudson.tasks.test.AbstractTestResultAction

@NonCPS
def testStatuses() {
    def testStatus = ""
    AbstractTestResultAction testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
    if (testResultAction != null) {
        def total = testResultAction.totalCount
        def failed = testResultAction.failCount
        def skipped = testResultAction.skipCount
        def passed = total - failed - skipped
        testStatus = "Test Status:\n  Passed: ${passed}, Failed: ${failed} ${testResultAction.failureDiffString}, Skipped: ${skipped}"

        if (failed == 0) {
            currentBuild.result = 'SUCCESS'
        }
    }
    return testStatus
}

---

UPDATE 2018-04-19

Note the above require manual "whitelisting" of methods used. Here is how you can whitelist all the methods in one go

Manually update the whitelist...

Exit Jenkins

Create/Update %USERPROFILE%.jenkins\scriptApproval.xml with the following content

<?xml version='1.0' encoding='UTF-8'?>
<scriptApproval plugin="[email protected]">
<approvedScriptHashes>
</approvedScriptHashes>
<approvedSignatures>
<string>method hudson.model.Actionable getAction java.lang.Class</string>
<string>method hudson.model.Cause getShortDescription</string>
<string>method hudson.model.Run getCauses</string>
<string>method hudson.tasks.test.AbstractTestResultAction getFailCount</string>
<string>method hudson.tasks.test.AbstractTestResultAction getFailureDiffString</string>
<string>method hudson.tasks.test.AbstractTestResultAction getSkipCount</string>
<string>method hudson.tasks.test.AbstractTestResultAction getTotalCount</string>
<string>method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild</string>
</approvedSignatures>
<aclApprovedSignatures/>
<approvedClasspathEntries/>
<pendingScripts/>
<pendingSignatures/>
<pendingClasspathEntries/>
</scriptApproval>

• Restart Jenkins
• and then verify that the "In script approval" has the above entries approved
• NOTE: Its the which is important. So if the scriptApproval file is already there, then you will generally need to ensure the contents of tag.

Solution 2

For anyone coming here in 2020, there appears to be a simpler way now. The call to 'junit testResults' returns a TestResultSummary object, which can be assigned to a variable and used later.

As an example to send the summary via slack:

def summary = junit testResults: '/somefolder/*-reports/TEST-*.xml'
slackSend (
   channel: "#mychannel",
   color: '#007D00',
   message: "\n *Test Summary* - ${summary.totalCount}, Failures: ${summary.failCount}, Skipped: ${summary.skipCount}, Passed: ${summary.passCount}"
)

Solution 3

To expand upon @vikramsjn's answer, here is what I use to get the test summary in my Jenkinsfile:

import hudson.tasks.test.AbstractTestResultAction
import hudson.model.Actionable

@NonCPS
def getTestSummary = { ->
    def testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
    def summary = ""

    if (testResultAction != null) {
        def total = testResultAction.getTotalCount()
        def failed = testResultAction.getFailCount()
        def skipped = testResultAction.getSkipCount()

        summary = "Test results:\n\t"
        summary = summary + ("Passed: " + (total - failed - skipped))
        summary = summary + (", Failed: " + failed)
        summary = summary + (", Skipped: " + skipped)
    } else {
        summary = "No tests found"
    }
    return summary
}

I then use this method to instantiate my testSummary variable:

def testSummary = getTestSummary()

This will return something similar to:

"Test results:
     Passed: 123, Failed: 0, Skipped: 0"

Solution 4

First of all, thank you for above answers. They saved me a lot of time, I used proposed solution in my pipeline. I however didn't use "whitelisting" and it works fine. I use shared libraries for Jenkins pipeline and here is a piece of that shared library with pipeline and using of methods to get counts:

import hudson.model.*
import jenkins.model.*
import hudson.tasks.test.AbstractTestResultAction

def call(Closure body) {
    ...

    def emailTestReport = ""

    pipeline {
        ...

        stages{
            stage('Test'){
                ...
                post {
                    always {
                        junit 'tests.xml'

                        script {
                            AbstractTestResultAction testResultAction =  currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
                            if (testResultAction != null) {
                                def totalNumberOfTests = testResultAction.totalCount
                                def failedNumberOfTests = testResultAction.failCount
                                def failedDiff = testResultAction.failureDiffString
                                def skippedNumberOfTests = testResultAction.skipCount
                                def passedNumberOfTests = totalNumberOfTests - failedNumberOfTests - skippedNumberOfTests
                                emailTestReport = "Tests Report:\n Passed: ${passedNumberOfTests}; Failed: ${failedNumberOfTests} ${failedDiff}; Skipped: ${skippedNumberOfTests}  out of ${totalNumberOfTests} "
                            }
                        }

                        mail to: '[email protected]',
                        subject: "Tests are finished: ${currentBuild.fullDisplayName}",
                        body: "Tests are finished  ${env.BUILD_URL}\n  Test Report: ${emailTestReport} "
                    }

                }
            }
        }
    }
}

p.s. If I create emailTestRepot as a local variable inside script "section" I get next exception:

an exception which occurred:
    in field locals
    in field parent
    in field caller
    in field e
    in field program
    in field threads
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@11cd92de
Caused: java.io.NotSerializableException: hudson.tasks.junit.TestResultAction
...

I struggled a lot with trying to fix that java.io.NotSerializableException. As I understood I needed to use "whitelisting" to prevent NotSerializableException. But I really didn't want to do it and when I moved "def emailTestReport" out of pipeline it worked just fine.

View more solutions