How do I run my cucumber-jvm features in parallel? (with maven using cucumber-jvm-parallel-plugin)
I finally got it to work! Here's what I had to do to make my tests run in parallel:
I move some fields in the
maven-surefire-plugin
execution
tag to theconfiguration
tag. I moved this section:<forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/*IT.class</include> </includes>
- So that my
maven-surefire-plugin
now looks like this:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19.1</version> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/*IT.class</include> </includes> </configuration> </plugin>
- So that my
Since
cucumber-jvm-parallel-plugin
is now running the show, I had to remove my cucumber runner class. I commented it out like this:
/*
@RunWith(Cucumber.class)
@CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
*/
public class RunApiTest {
}
And that's it. Since Parallel**IT.class
where getting created, it was maven-surefire-plugin
that was not working right.
Katie
Software Engineer specializing in everything web. These are a few of my favorite things: angular node.js javascript go python java c++ bash jquery katies.io
Updated on June 05, 2022Comments
-
Katie almost 2 years
I am trying to incorporate a maven plugin cucumber-jvm-parallel-plugin into my Cucumber-JVM code and am running into some problems... I think I have configured my
pom.xml
properly, but my cucumber features are still getting ran one by one instead of in parallel.How do I configure my pom.xml to make my cucumber features run in parallel? I have some feature files that block execution and I don't want to have to wait for them to finish in order to start testing the other features.
I followed two tutorials, and can't find where I am going wrong, or if this is expected:
- https://opencredo.com/test-automation-concepts-parallel-test-execution/
- http://automationrhapsody.com/running-cucumber-tests-in-parallel/
Structure
Here's my program's structure:
├── features/ │ └── api/ │ ├── 006-Email.feature │ └── 999-Login.feature ├── results/ │ └── api-json.json └── src/ └── java/ ├── pom.xml ├── src/ │ └── test/ │ └── java/ │ └── com/ │ └── mycompany/ │ └── commonapps/ │ └── queuemanager/ │ ├── Globals.java │ ├── GmailHelper.java │ ├── HttpDeleteWithBody.java │ ├── JsonHelper.java │ ├── RequestHelper.java │ ├── RunApiTest.java │ ├── ThreadedSteps.java │ └── UserHelper.java └── steps/ └── com/ └── mycompany/ └── commonapps/ └── queuemanager/ ├── GivenSteps.java ├── Setup.java ├── ThenSteps.java └── WhenSteps.java
Maven Pom Logic
The way I configured my pom is as follows:
-
I copy
../../features/api/
to my build path using Maven Resources Plugin<build> <resources> <resource> <directory>../../features/api/</directory> </resource> </resources> </build>
-
I tell cucumber-jvm-parallel-plugin that my
features
are now located attarget/classes/active/
, because I think../../features/api/
was copied totarget/
, sotarget/active/
should contain my.feature
files.. I think.<plugin> <groupId>com.github.temyers</groupId> <artifactId>cucumber-jvm-parallel-plugin</artifactId> ... <configuration> ... <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory> ... </configuration> </plugin>
Then, after I run
maven test
, all.feature
files are then compiled intotarget/test-classes/Parallel**IT.class
. One per feature file.-
So I tell maven-surefire-plugin to run all
<include>**/Parallel*IT.class</include>
files in parallel, which are the feature files created by cucumber-jvm-parallel-plugin.<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <executions> <execution> <id>acceptance-test</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/Parallel*IT.class</include> </includes> </configuration> </execution> </executions> </plugin>
Full pom.xml
And my pom.xml is:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mycompany.commonapps</groupId> <artifactId>e2e-server</artifactId> <version>0.0.1-SNAPSHOT</version> <dependencies> ... </dependencies> <build> <!-- COPY RESOURCES TO TARGET/ --> <resources> <!-- copies features to target: "${project.build.directory}/classes/active/" --> <resource> <directory>../../features/api/</directory> </resource> </resources> <!-- RUN EACH CUCUMBER FEATURE FILE AS A FORK --> <plugins> <plugin> <groupId>com.github.temyers</groupId> <artifactId>cucumber-jvm-parallel-plugin</artifactId> <version>1.2.1</version> <executions> <execution> <id>generateRunners</id> <phase>generate-test-sources</phase> <goals> <goal>generateRunners</goal> </goals> <configuration> <glue>com.mycompany.commonapps.queuemanager</glue> <outputDirectory>${project.build.directory}/generated-test-sources</outputDirectory> <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory> <cucumberOutputDir>${project.build.directory}</cucumberOutputDir> <format>json</format> <!--<strict>true</strict> <monochrome>true</monochrome> <useTestNG>false</useTestNG> <namingScheme>simple</namingScheme>--> <parallelScheme>FEATURE</parallelScheme> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <!-- RUN ALL CLASSES CREATED BY cucumber-jvm-parallel-plugin IN PARALLEL --> <executions> <execution> <id>acceptance-test</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/Parallel*IT.class</include> </includes> </configuration> </execution> </executions> </plugin> <!-- ADD ../../STEPS/ TO THE SOURCE --> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>build-helper-maven-plugin</artifactId> <version>1.7</version> <executions> <execution> <id>add-source</id> <phase>generate-sources</phase> <goals> <goal>add-source</goal> </goals> <configuration> <sources> <source>steps/</source> <source>src/test/java/</source> </sources> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>
Cucumber Runner
And my cucumber running class looks like this:
@RunWith(Cucumber.class) @CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"}) public class RunApiTest { }
Output
When I run
mvn test
, I get the following:... [INFO] ------------------------------------------------------------------------ [INFO] Building e2e Server Tests 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2 [INFO] [INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ e2e--server --- [INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/steps added. [INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java added. [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ e2e--server --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 7 resources [INFO] Copying 2 resources [INFO] Copying 7 resources [INFO] [INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ e2e--server --- [INFO] Changes detected - recompiling the module! [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 12 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/classes [INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations. [INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- cucumber-jvm-parallel-plugin:1.2.1:generateRunners (generateRunners) @ e2e--server --- [INFO] Adding /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/generated-test-sources to test-compile source root [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ e2e--server --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ e2e--server --- [INFO] Changes detected - recompiling the module! [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 10 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/test-classes [INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations. [INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-surefire-plugin:2.17:test (default-test) @ e2e--server --- [INFO] Surefire report directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- Running com.mycompany.commonapps.queuemanager.RunApiTest .... *Tests run one by one properly* ... Tests run: 31, Failures: 2, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:40 min [INFO] Finished at: 2016-11-10T15:49:11-08:00 [INFO] Final Memory: 20M/261M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project e2e--server: There are test failures. [ERROR] [ERROR] Please refer to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Does anyone see what I did wrong?
More Debugging Info
- cucumber-jvm-parallel-plugin's
outputDirectory
is getting created correctly and contains a bunch of Parallel**IT.class files - cucumber-jvm-parallel-plugin's
featuresDirectory
is getting accessed correctly. - cucumber-jvm-parallel-plugin's
cucumberOutputDir
is NOT getting created at all after a build.
-
vikramvi over 6 yearscan you please clarify stackoverflow.com/questions/45436476/…
-
paul almost 5 yearsHey Katie, how can I let the runner run the show and still have everything run in parallel? If we can't, which limitation is stopping us to do so.