BDD Kickstart Atlanta

Cucumber Reports

Cucumber can report results in several different formats, using formatters. Formatters are specified either on the command line or in code. Some formatters generate files while others print to STDOUT. You can specify as many formatters as you like.

The available formatters are:

Pretty

Prints the gherkin source with additional colours and stack traces for errors

HTML

This report is handy for publishing

JSON

This report contains all the information from the gherkin source, with additional results for each step, including embedded screenshots. This report is meant to be post-processed into another visual format by 3rd party tools such as Cucumber Jenkins.

Progress

This report prints results to STDOUT, on character at a time. It looks like this:

....F--U.......

Usage

This report prints statistics to STDOUT. Programmers may find it useful to find slow or unused Step Definitions.

JUnit

This report generates XML files just like Apache Ant’s junitreport task. This XML format is understood by most Continuous Integration servers, who will use it to generate visual reports.

Rerun

This report generates a file that lists the location of failed Scenarios. This can be picked up by subsequent Cucumber runs, allowing only previously failed Scenarios to be rerun. Programmers may find this useful while fixing broken scenarios, especially if running all scenarios is time-consuming.

Embedding extra information in reports

Programmers can embed text and images into reports via an API that is available in Step Definitions and Hooks. This can make it easier to diagnose failures. Some reports will ignore embedded data while others will include it.

Embedding Text

Text is embedded in the following way:

When /I do the deed/ do
  puts 'This goes into the report'
  Kernel.puts 'This goes to STDOUT, but not into the report'
end
@Before
public void before(Scenario scenario) {
    this.scenario = scenario;
}

@When("I do the deed")
public void do_the_deed() {
    scenario.write("This goes into the report");
    System.out.println("This goes to STDOUT, but not into the report");
}

Embedding Screenshots

TODO: Add an image of a report with a screenshot. Can be a close-up with a small section.

If you are using Cucumber with a GUI driver such as a Browser Automation tool, it can sometimes be useful to embed screenshots or videos of the user interface in Cucumber’s HTML or JSON report.

Cucumber itself doesn’t provide an API to take screenshots or videos, but it provides an API to embed them into the report. For example, if you are using Selenium WebDriver, you can ask it to give you a screenshot, and then embed them into the report.

After do |scenario|
  if(scenario.failed?)
    page.driver.browser.save_screenshot("html-report/#{scenario.__id__}.png")
    embed("#{scenario.__id__}.png", "image/png", "SCREENSHOT")
  end
end
@After
public void embedScreenshot(Scenario scenario) {
    if(result.isFailed()) {
        byte[] screenshot = webDriver.getScreenshotAs(OutputType.BYTES);
        scenario.embed(screenshot, "image/png");
    }
}