本文共 7134 字,大约阅读时间需要 23 分钟。
在运行测试用例时,有时候希望本次运行结束后自动运行失败的测试用例,以排除结果由于网络或其他连接原因导致的偶发抖动。通过参考查阅资料,有以下几种方法可以达到目的:
***1、maven的surefire 插件,有rerun功能
参考官方文档:http://maven.apache.org/components/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html
***2、junit自定义注解,针对单个case做重试
参考:http://stackoverflow.com/questions/8295100/how-to-re-run-failed-junit-tests-immediately
***3、junit添加testRule规则,针对单个case做重试
***4、junit多线程,扫描测试日志,查找失败的case,记录到Map文件,运行失败的测试用例后将日志写回到原日志中
参考:http://blog.csdn.net/neven7/article/details/43529569
由于项目需要,本篇文章主要介绍第一种方法!
一、maven surefire plugin
1、我这里用的是Version: 2.18.1,该版本支持Re-run Failing Tests,要求Junit 4.X版本
运行命令如下,rerunFailingTestsCount为重试tests次数,设为0或小于0时,参数会被忽略
mvn -Dsurefire.rerunFailingTestsCount=2 test
2、控制台输出结果:
1)如果测试在第一次就成功,则rerun设置将被忽略
3)如果测试在某次rerun成功,则停止rerun,并在最后一次rerun输出PASS, 如:
Run 1: ... Run 2: PASS Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Flakes: 1 ------ Flakes表示有一次rerun
2)如果测试重试count次后仍失败,则结果没有Flake字段,且结果为:
Run 1: ... Run 2: ... Run 3: ...
2、.xml测试报告结果:
1)Rerun后通过
<testcase name=".." classname=".." time="0.1">
flaky failure stack trace flaky failure success
2)Rerun后仍未通过
<testcase name=".." classname=".." time="0.1">
first failure stack trace first failure rerun failure stack trace rerun failure
二、一段简单的代码示例:
@Test public void unRerunTest() {
System.out.println("这是不需要重试机制测试case");
@Test public void unRerunTest() { System.out.println("这是不需要重试机制测试case"); boolean bool = false; Assert.assertFalse("应该为false", bool); } @Test public void rerunTest1() { boolean first = false; long time = System.currentTimeMillis(); System.out.println("这是第一个需要重试机制测试case,time:" + time + ", first=" + first); Assert.assertTrue("first应该为true", first); } @Test public void rerunTest2() { boolean first = true; long time = System.currentTimeMillis(); System.out.println("这是第二个需要重试机制测试case,time:" + time + ", first=" + first); Assert.assertFalse("first应该为false", first); }
pom.xml配置:org.apache.maven.plugins maven-surefire-plugin 2.18.1 org.apache.maven.surefire surefire-junit47 2.18.1
1、控制台输出结果:
这是不需要重试机制测试case 这是第一个需要重试机制测试case,time:1433940475436, first=false 这是第二个需要重试机制测试case,time:1433940475439, first=true Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec <<< FAILURE! - in com.xiaomi.testteam.rerun_test.MavenRerunTest rerunTest1(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0 sec <<< FAILURE! java.lang.AssertionError: first应该为true at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest1(MavenRerunTest.java:30) rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0.001 sec <<< FAILURE! java.lang.AssertionError: first应该为false at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(MavenRerunTest.java:42) 这是第一个需要重试机制测试case,time:1433940475443, first=false 这是第二个需要重试机制测试case,time:1433940475444, first=true Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec <<< FAILURE! - in com.xiaomi.testteam.rerun_test.MavenRerunTest rerunTest1(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0 sec <<< FAILURE! java.lang.AssertionError: first应该为true at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest1(MavenRerunTest.java:30) rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0 sec <<< FAILURE! java.lang.AssertionError: first应该为false at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(MavenRerunTest.java:42) Results : Failed tests: com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest1(com.xiaomi.testteam.rerun_test.MavenRerunTest) Run 1: MavenRerunTest.rerunTest1:30 first应该为true Run 2: MavenRerunTest.rerunTest1:30 first应该为true com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Run 1: MavenRerunTest.rerunTest2:42 first应该为false Run 2: MavenRerunTest.rerunTest2:42 first应该为false Tests run: 3, Failures: 2, Errors: 0, Skipped: 0
可以看出,测试是在运行整个suite第一次结束之后,检查fail的测试用例并再次运行,直到count次重试之后,测试Case依旧fail,则打印所有重试的结果!
如果把test增加时间设置,结果则变成:
@Test public void rerunTest1() { boolean first = false; long time = System.currentTimeMillis(); if (time % 2 == 0) {// 用于测试在count内rerun通过,打印结果对比 first = true; } System.out.println("这是第一个需要重试机制测试case,time:" + time + ", first=" + first); Assert.assertTrue("first应该为true", first); } @Test public void rerunTest2() { boolean first = true; long time = System.currentTimeMillis(); if (time % 2 == 0) { first = false; } System.out.println("这是第二个需要重试机制测试case,time:" + time + ", first=" + first); Assert.assertFalse("first应该为false", first); }
这是第一个需要重试机制测试case,time:1433928323102, first=true 这是第二个需要重试机制测试case,time:1433928323103, first=true Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec <<< FAILURE! - in com.xiaomi.testteam.rerun_test.MavenRerunTest rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0 sec <<< FAILURE! java.lang.AssertionError: first应该为false at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(MavenRerunTest.java:41) 这是第二个需要重试机制测试case,time:1433928323109, first=true Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec <<< FAILURE! - in com.xiaomi.testteam.rerun_test.MavenRerunTest rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Time elapsed: 0 sec <<< FAILURE! java.lang.AssertionError: first应该为false at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(MavenRerunTest.java:41) 这是第二个需要重试机制测试case,time:1433928323112, first=false Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec - in com.xiaomi.testteam.rerun_test.MavenRerunTest Results : Flaked tests: com.xiaomi.testteam.rerun_test.MavenRerunTest.rerunTest2(com.xiaomi.testteam.rerun_test.MavenRerunTest) Run 1: MavenRerunTest.rerunTest2:41 first应该为false Run 2: MavenRerunTest.rerunTest2:41 first应该为false Run 3: PASS