Sitodo (PMPL Variant)
A basic todo app project for teaching basic Web programming, Git workflows, and CI/CD. Heavily inspired by the running example in "Test-Driven Development with Python" book by Harry Percival.
Note: The project has been customised from the upstream in order to be used in PMPL (SQA) course at the Faculty of Computer Science Universitas Indonesia.
Table of Contents
Getting Started
Setting Up Development Environment
The following tools need to be installed in order to build and run the project:
- Java 17 JDK (Java Development Kit)
- PostgreSQL 14
- You can install PostgreSQL system-wide or use container (Docker/Podman)
- A
docker-compose.yml
has been provided to quickly start a PostgreSQL and pgAdmin containers
- Apache Maven 3.8.5
-
Mozilla Firefox
- Required by the functional (Selenium) test suite and BDD test suite
Ensure java
, javac
, and mvn
commands can be invoked from inside the shell:
$ java --version
$ javac --version
$ mvn --version
The output should display the versions of java
, javac
, and mvn
respectively.
We recommend IntelliJ IDEA Community Edition as the IDE for developing the project. Other IDE or text editors, such as Eclipse and Visual Studio Code, might work. However, we may not be able to help troubleshoot any IDE-related issues. In addition, we include IntelliJ-specific run configurations in the codebase that will add shortcuts for running the test suites and coverage reporting from within IntelliJ.
Build & Run
To run the entire test suite, execute:
mvn test
To run a select test suite, e.g. unit or functional test, add
-Dgroups
parameter. For example, to run only the unit test suite, executemvn test -Dgroups=unit
. Similarly, to run only the functional test suite, executemvn test -Dgroups=e2e
.
To build an executable Spring Boot application, execute:
mvn package -DskipTests
The
-DskipTests
option letspackage
task to build the app into executable JAR file without running all test suites. If the option was omitted, then all test suites will run, thus increasing the duration of the building process, especially the functional test suite that runs much longer than the unit test suite.
The JAR file will be generated at ./target
directory. To run it,
execute:
java -jar sitodo.jar
You can customise the configuration by providing an application.properties
file in the same directory as the executable JAR file. See the built-in
configuration in the source code.
Additionally, you can also set the configuration during runtime using environment variables.
Following Spring Boot convention, properties are named in all uppercase and dot separators are replaced with underscores.
For instance, spring.datasource.url
becomes SPRING_DATASOURCE_URL
when configured using an environment variable.
See the example in the GitLab CI/CD configuration, specifically in the job for running tests.
Running Example
See the running example based on the upstream's main
branch at Fly.io.
Tutorial: Writing Behavior-Driven Development (BDD) Test Suite
Previously, the lecture session in the class has explained the motivation and the basics of BDD. This tutorial will cover the practical aspects by outlining the basic steps of writing BDD test suite. We will see the process of writing the executable specifications and the "glue" code, followed by running the test suite and generating the test report. Finally, we will do an exercise to write new executable specifications and the corresponding "glue" code, while maintaining the correctness and the code coverage.
Executable Specifications
The executable specifications in BDD follow a structured outline using Gherkin syntax.
In this example, we follow the Given-When-Then
format. Each section serves a specific purpose:
-
Given
is used to set up the pre-conditions or the initial context for the test scenario -
When
is used to define the event or action that we want to simulate -
Then
is used to verify the expected outcome of the performed action
For example, look at the following executable specification for "Add new todo" feature:
Feature: Add items into the todo list
Scenario: Add single item into the list
Given Alice is looking at the TODO list
When she adds "Touch grass" to the list
Then she sees "Touch grass" as an item in the TODO list
Scenario: Add multiple items into the list
Given Alice is looking at the TODO list
When she adds "Buy bread" to the list
And she adds "Buy candy" to the list
Then she sees "Buy bread" as an item in the TODO list
And she sees "Buy candy" as an item in the TODO list
The executable specification above describes two possible user flows when they are adding items to the todo list. The first user flow is to add a single item into the list, and the second user flow is to add multiple items into the list.
One of the advantages of following this format is that the test scenario can be easily written by non-technical roles. They can focus on writing the test scenario without worrying about the implementation details.
Eventually, there needs some way to "glue" the executable specifications into the actual test code. That is, the executable specifications that is written in Gherkin syntax drives the test procedures in the test code. To achieve that, we will write the "glue" code and apply two patterns relevant to our need: Screenplay & Page Object Model (POM).
Note: There also exists the equivalent Gherkin syntax in other languages such as Bahasa Indonesia. For example,
Given
,When
, andThen
are equals toJika
,Ketika
, andMaka
, respectively. See the Localization package on Cucumber website for more information. It may be useful if you want to write the executable specifications in a natural language other than English.
Glue Code
Remember that while the executable specifications can be written by non-technical roles, the corresponding "glue" code needs to be written by developers or technical QA personnel. We will explore how we can bridge the gap between the executable specifications and the automated test procedures.
First, we need to decide at what level the test procedures shall be done. BDD can be used to drive the unit testing procedures. That is, the test script in the unit test suite is written to satisfy the executable specifications. However, recall that the unit tests, by design, exercise the code at the unit level. There might be cases where interaction between units (e.g., shared states or interdependency between modules) are not covered, which are something that can be done if we run a test suite against the actual deployed and running system. Therefore, we will use BDD at the level of functional (or, end-to-end) test in this project.
Before we can begin in writing the "glue" code in BDD, let us see an example of a functional test written using Selenium and Selenide:
/**
* Simplified, non-refactored version of a test case in AddTodoItemTest.java functional test suite.
*/
@Test
@DisplayName("A user can create a single todo item")
void example_addTodo_singleItem() {
// Open the initial page
open("/");
// Create a new item
WebElement inputField = $(By.tagName("input"));
inputField.sendKeys("Touch grass", Keys.ENTER);
// See the list for the newly inserted item
$(By.tagName("tbody")).findAll(By.tagName("tr"))
.shouldHave(CollectionCondition.size(expectedItems.size()))
.should(CollectionCondition.allMatch("Valid row", (row) -> isRowValid(List.of("Tounch grass"), row)));
// The page can be accessed at the new, unique URL
String currentUrl = webdriver().driver().url();
assertTrue(currentUrl.matches(".+/list/\\d+$"), "The URL was: " + currentUrl);
}
private boolean isRowValid(List<String> expectedItems, WebElement row) {
List<WebElement> columns = row.findElements(By.tagName("td"));
assertEquals(DEFAULT_NUMBER_OF_COLUMNS, columns.size(),
"There were " + columns.size() + " columns in a row");
String id = columns.get(0).getText();
String title = columns.get(1).getText();
return Pattern.matches("\\d+", id) && expectedItems.contains(title);
}
The code example above simulates the actions of a user when adding a todo item. The user opens the web application, then proceed to create a todo item by filling the expected input field. After submitting the new todo item, the user expects it to be stored and visible in the list. As you may have noticed, the test code is highly technical and unlikely to be written by someone without the technical expertise, such as the customer or business analyst.
After looking at the example above, we can see why the "glue" code is needed when writing BDD test suite. We need to write the "glue" code to simulate the actions done by the user, which are described as part of the executable specifications.
To begin writing the "glue" code, we need to create the abstraction of the UI elements and user actions as defined in the executable specifications. Let us start with creating the UI elements abstraction by following the Page Object Model (POM) pattern. See the following page object example that represents the todo list page:
/**
* Page object model of the main todo list page,
* specifically the significant elements of the todo list page.
*
* Reference:
* https://serenity-bdd.github.io/docs/screenplay/screenplay_webdriver#using-the-target-class
*/
public class TodoListPage extends PageObject {
public static Target ITEM_NAME_FIELD = Target
.the("item name field")
.located(By.id("id_new_item"));
public static Target ENTER_BUTTON = Target
.the("enter button")
.located(By.tagName("button"));
public static Target ITEMS_LIST = Target
.the("item list")
.located(By.xpath("//tbody/tr/td[contains(@class, 'todo-item-title')]"));
}
The TodoListPage
class above represents the todo list page as a page object.
It only contains the references to UI elements that are significant in the context of running the test procedures.
For example, the existing BDD test suite currently only requires simulating user interaction with three UI elements:
the input text field, the submit button, and the table that contains the todo items.
Now, we will see how we can abstract the user actions by using Screenplay pattern. Let us start with creating the abstraction for the action of "adding a todo item":
/**
* A custom interaction class to fill a text input field.
*
* Reference: https://serenity-bdd.github.io/docs/screenplay/screenplay_webdriver#interacting-with-elements
*/
public class AddAnItem {
public static Performable withName(String itemName) {
return Task
.where("{0} adds an item with name " + itemName,
Enter.theValue(itemName) // Uses Serenity's `Enter` interaction class
.into(TodoListPage.ITEM_NAME_FIELD)
)
.then(Task.where("{0} clicks the enter button",
Click.on(TodoListPage.ENTER_BUTTON) // Uses Serenity's `CLick` interaction class
));
}
}
The AddAnItem
class above represents the action of adding a todo item performed by an "actor" (representation of user).
The action is modeled as a set of Java statements and written in fluent-style to enhance readability.
In addition, it utilises the interaction classes provided by the Serenity BDD test framework,
which under the hood are actually Selenium code.
Once we have the infrastructure set up, i.e. the abstraction of page and user actions, we can begin write the "glue" code. The "glue" code comprises step definitions that will correspond to Gherkin statements found in the executable specifications. The following code example is the step definitions for "add new todo item" feature:
public class AddItemStepDefinitions {
@Given("{actor} is looking at the TODO list")
public void actor_is_looking_at_her_todo_list(Actor actor) {
actor.wasAbleTo(NavigateTo.theTodoListPage());
}
@When("{actor} adds {string} to the list")
public void she_adds_to_the_list(Actor actor, String itemName) {
actor.attemptsTo(AddAnItem.withName(itemName));
}
@Then("{actor} sees {string} as an item in the TODO list")
public void she_sees_as_an_item_in_the_todo_list(Actor actor, String expectedItemName) {
List<String> todoItems = TodoListPage.ITEMS_LIST.resolveAllFor(actor)
.textContents();
actor.attemptsTo(
Ensure.that(todoItems)
.contains(expectedItemName)
);
}
}
The AddItemStepDefinitions
class above is the "glue" code for the executable specifications of "add new todo item" feature.
Notice that each Gherkin statement, i.e. statements that start with Given
, When
, and Then
, is mapped to a corresponding method.
The Serenity BDD test framework utilises pattern matching to pair the Gherkin statement with the corresponding method during test suite execution.
Now that we have created the "glue" code, it is time to run the BDD test suite.
Execute Test Suite & Reporting
Before we run the BDD test suite, let us see how it is configured first at
CucumberTestSuite.java
:
package com.example.sitodo.bdd;
import io.cucumber.junit.CucumberOptions;
import net.serenitybdd.cucumber.CucumberWithSerenity;
import org.junit.runner.RunWith;
/**
* The test runner class that runs the executable scenarios,
* i.e. the feature files written in Gherkin format.
*/
@RunWith(CucumberWithSerenity.class) // Runs the test suite using the test runner provided by Serenity
@CucumberOptions(
plugin = {"pretty"},
features = "src/test/resources/features"
)
public class CucumberTestSuite { }
The CucumberTestSuite
class represents the BDD test suite.
It specifies the test runner, i.e. CucumberWithSerenity
class,
and the options for the test runner such as the location of the test scenarios.
The options for the test runner provided by Serenity can also be provided via a configuration file named serenity.conf
.
The content of the configuration file is as follows:
serenity {
take.screenshots = FOR_FAILURES
}
headless.mode = false
webdriver {
autodownload = true
base.url = "http://localhost:8080"
driver = firefox
capabilities {
browserName = "firefox"
acceptInsecureCerts = true
}
}
The options in the configuration file are self-explanatory. More information about the configuration can be found in the Serenity documentation.
In the case of the BDD suite, the test runner expects the web application has been running at specified URL, i.e. the value set in base.url
.
The web application will not automatically run when we execute the BDD test suite.
Therefore, we have to build and deploy the web application prior to running the tests.
Before running the BDD test suite, run the web application in a shell:
mvn spring-boot:run
Then, run the BDD test suite in a different shell:
mvn -P bdd verify
The BDD test suite can also be run directly in IntelliJ by running CucumberTestSuite
run configuration.
Before executing the BDD test suite, IntelliJ will automatically run the web application.
To see the test report generated by the test runner,
open the HTML document located at ./target/site/serenity/index.html
.
The report will show information such as the total number of features and their test status.
Note: To see how the test suites are configured and run by Maven build system, you can peruse the build configuration at
pom.xml
. The configuration utilises several plugins such as Maven Surefire and Maven Failsafe to run the test runners. Additionally, to ensure that the BDD test suite can be run independently, the configuration use a custom Maven profile to isolate the BDD test suite from the other test suites.
Tasks
Your tasks are to write new executable specifications for two existing features that yet to be covered by the BDD test suite. The two features are: "see motivation message" and "mark a todo item." At minimum, you need to write two new test scenarios for both features. The test scenarios can be as simple as covering the "positive" and "negative" case that may occur on both features, or to cover other possible outputs.
Mandatory Tasks
- Fork this project into your own namespace (GitLab account)
-
Set up the development environment for the project locally
- Refer to Setting Up Development Environment section
-
Run the project locally and try the existing features
- Try to add new todo items and mark their completion status
- See how the motivation message is updated based on the number of completed todo items
- Run the test suites locally, ensure all test passes
- Write a new executable specification for "see motivation" feature that contains at least two test scenarios
- Write a new executable specification for "mark a todo item" feature that contains at least two test scenarios
- Write the required "glue" code to support the new executable specifications
- Maintain code coverage greater than or equal to 97%
-
Summarise the whole process and write them in a
summary.md
- Write an analysis of what you did and what you learned from this exercise and please include the BDD test report
Note: You are allowed to modify the production code (i.e. the
src/main
directory) to make your test code can obtain the reference to the UI elements in the page. For example, you can add a new HTML attribute, CSS class, or CSS id to an element in the page to make it easier to be located by selector methods.
Additional Tasks
-
Implement the error handling when posting an empty todo item by following TDD approach in BDD (also known as: outside-in TDD)
- Start by writing the executable specification that describes the error handling
- Then, implement the error handling by following the TDD cycle (i.e. red-green-refactor phases)
- Ensure the error handling works
- Finally, demonstrate that both the unit and BDD test suites run successfully
- Increase the code coverage to 100%
- (Not Graded) Deploy the app to a cloud platform and run the BDD test suite by targeting the deployed app
License
This project is licensed under the terms of the MIT license.