It is quite normal for things to move quickly in software development. However, the faster you move, the more things break. As a codebase grows in size, its pieces become more and more complex, with every line adding a potential bug. In Wikimedia Foundation, we keep a handle on this through rigorous amounts of testing. Manual testing requires a lot of effort especially when you have a large core repository with a plethora of plugins and extensions that need to be tested. One of the hot frameworks on the scene is Cypress, a complete end to end testing solution.
In the past, web-app end-to-end testing has been a tricky beast. Good ol’ Selenium has been the main solution for quite some time now and has a huge history. Despite its impressive browser compatibility, having tests that are more consistent and less flakey can be a non-trivial matter mainly because Selenium was not designed for app testing. Which is what makes me super excited about Cypress, which not only promises to fix the old and broken ways of past framework but also make the entire process more developer centric. After spending over two months trying to migrate the tests written in WebdriverIO to Cypress, I wanted to share my thoughts on my journey with Cypress thus far.
End-to-end testing is notoriously known for its fragmented experience. You need to bring a lot of your own tools, for example, you need:
- a test runner
- an assertion library
- some reporters
- mocks, etc
Cypress includes all of these good stuff out of the box, making setup and configuration dead simple. At the same time, Cypress gives developers an immense amount of freedom when it comes to plugging in things of your own preference. In short,
Cypress takes a batteries included but removable approach
A good software should be accompanied by a great documentation for it complete.
Cypress has one of the best documentation that I have ever read, with guides on everything that you are likely to encounter. In fact, I remember when I was just starting out with Cypress, I immediately reached out to Amit Punjabi, my very close friend and mentor, for some good resources regarding Cypress and he immediately pointed me out to their official documentation. Not only do they do a great job telling you how to use the product, but have in-depth explanations on the architecture, flakey tests and best practices.
If you have the chance, before adopting anything of this scale, I always think it’s a good idea to test it on a small project first, just to get a feel. Before advocating for it, I added it to my personal blog, just to see how the experience was.
A very simple scenario:
npm script in
package.json, and running it in
TravisCI. I was blown away with how fast it took me, under an hour. I had toiled hard trying to set up video recording with Puppeteer and Jest. Not only did Cypress perform the assertions but also it was recording videos!
Page Object model is an object design pattern in Selenium, where web pages are represented as classes, and the various elements on the page are defined as variables on the class. All possible user interactions can then be implemented as methods on the class.
Since well-named methods in classes are easy to read, this works as an elegant way to implement test routines that are both readable and easier to maintain or update in the future. Let us take an example of a page model for testing user account creation in WebdriverIO.
All you need to do is now call the
CreateAccountPage.createAccount method with proper parameters in order to create a user account. Let us have a look at the Cypress version for this:
If you have written end-to-end tests before, you may be familiar with Gherkin syntax, used by Cucumber. This is an expressive, English-like way to write test scenarios. It can help with documenting your features and non-developers can contribute to writing test cases. I found a way to integrate this file syntax into Cypress using a plugin.
After writing these commands, the plugin will then go to Cypress to actually run the implementations:
End-to-end testing is all about making sure elements on the page have the right content. When writing Cypress tests, you will likely be spending 70% of your time selecting elements and peering inside them. Cypress has a standard get() command which exposes a jQuery-like selector to you. This should be familiar to those with some experience in Selenium. The problem with this selector is that it can be used incorrectly and there is no way to enforce (with code) it’s usage. Welcome cypress-testing-library, a wonderful tool maintained by a great testing advocate in the community, Kent C. Dodds.
This plugin exposes a myriad of commands prefixed with
find which work similarly to how
get() does in native Cypress. All of these commands make for selectors that are resilient to change. This can have a dramatic effect on how your tests stay consistent as your application progresses.
Debugging end-to-end tests with Selenium can be somewhat of a nightmare. With Cypress, this pain is at an all-time low. One of the biggest focuses of the core product is to be able to debug is one of the more pleasant experiences in your Cypress journey. Like for most things, they have a great guide to get you started.
Unfortunately, if your organization needs to have support for IE11, you are out of luck. The Cypress team has explicitly said they won’t be supporting it. There is an incredible thread on Github that I really hope you read through. It goes into why they are rolling this out slowly and didn’t choose WebDriver from the beginning and wrote their own custom driver. This is the only place where Cypress falls behind Selenium. However, it does support most of the Chromium-based browsers and is starting to roll-out support for Firefox too and this should serve the needs for most organisations.
I highly recommend everyone in the end-to-end testing community to give Cypress a shot. The quick setup, smooth documentation, fluid construction of tests and warm and supportive community will likely make you fall in love with it.