Optimizing Your Test Strategy with Playwright
Playwright is not a silver bullet, just like other tools, there are things it is very good at and others that should be avoided completely.
✅ Recommended
UI Web application testing
This is the bread and butter for playwright. They are doing incredible things with UI testing. In my opinion, this is the easiest framework to understand and learn right now. Specifically, there are a handful of things that set playwright apart from other frameworks:
- It is cross-browser and supports Mobile Web. This means that with some simple configuration you can write a single test and run it against multiple browsers to emulate multiple mobile web devices.
- Cross language support. While it isn't currenly officially possible to write playwright tests in every language, it has APIs and support for a lot of the heavy hitters: .NET, Java, Python, TypeScript, JavaScript. It seems unlikely to find a team that was not comfortable with at least one of those languages.
- It AUTO WAITS! Playwright has built in functionality that checks for an element to be actionable before performing actions on it. You no longer have to place any explicit waits or timeouts in your tests, Playwright does it for you, and it does a good job of it.
- The tooling around this is amazing. The trace viewer allows you to inspect your test at every stage and see all the interactions and logs that happen during the test run. The playwright inspector allows you to set breakpoints in your code, you can highlight checks and it will highlight what that matches to in your browser, making it incredibly easy to triage and find the root cause of any issue. You can even write new locator statements and watch what it resolves to, or use the built in tools to select elements on the page and let Playwright give you the code.
- To stop myself from simply naming all of the features that come with Playwright, I will end with this: their documentation is incredible. If you take the time to read through everything Playwright has to offer, you will come away fully equipped to tackle nearly any obstacle you are facing.
API Testing:
This was not always the case, but playwright has become a contender in the API testing space. The APIReleaseContext allows users to execute API calls without the browser overhead.
Unlike the UI testing variant above, I will say, this is not for everyone. This really shines when you are already working with Playwright. If you already have an established Postman, pytest, gatlin, k6, RestAssured, etc. suite, you likely are not going to get a lot of gains by switching over to Playwright.
With that being said, if you are starting a new project, or you are switching over to Playwright for your UI or end-to-end testing, it might be worth considering this tool for both, especially if the same team is going to be responsible for both tiers.
⚠️ Caution
Not perfect, but definitely will apply to some situations.
Contract Testing:
One variation on API testing I wanted to call out was Contract Testing. Playwright doesn't have anything baked in to do contract testing, but there are libraries that can be added to help with this.
For example, Pact has a lot of support that covers most modern languages. It is a fast, industry standard that you can couple with Playwright to round out your test suite.
Joi is another library that is great for javascript based contract testing. It is lightweight and it largely just concerns itself with validating structure and responses.
Running Too Many End-to-End (e2e) Tests:
One big issue you may run into when implementing your e2e suite is having too many tests. This might sound odd at first, but having too many e2e tests might be an indicator that you are not testing enough at a lower level of the testing pyramid. This can lead to a more test hourglass type of shape.
E2e tests are the slowest and most complex. Pushing more ownership to higher levels of the test pyramid will make your test suite more brittle, leading to more maintenance and will make it harder to iterate quickly because of the amount of time your test suite takes to run.
Focus on critical user journeys and test edge cases at lower levels of the test pyramid.
Testing Design/Visual Details:
Playwright has tooling to do visual comparison testing, but I personally believe that the tooling needs to mature a bit more before it can be viable for a large organization. With that being said, if you have a smaller team, the issues I am calling out may not be a concern.
They functionally have everything you need to do visual testing. You can create baseline images, test against those images, inspect differences that it finds. You can even ignore specific areas of the UI that might be dynamic and you want to ignore.
In my opinion, a tool like this really should have some kind of dashboard or service that allows people to approve the new baseline, but this likely requires the images to live in a different place or with a cloud service. With Playwright, the images live directly with the code, this has it's own benefits, but with large teams, or rapidly changing systems, this could be a frustrating exercise to make sure each team member always has the latest baseline image.
Accessibility Testing:
Playwright also has an optional package to allow for some accessibility testing. This utilizes Axe accessibility testing tools, and can be utilized directly in your tests.
There is nothing necessarily wrong with this offering from Playwright. However, it only will detect some common problems and will never try to do more than that.
Playwright suggests in a disclaimer that if you are trying to ensure accessibility of your application, you need to do more. Automation, in this case, will not be able to catch all of your issues and manual assessments are ultimately required.
So, while you can do accessibility testing with Playwright, that will not prove that your application is accessible.
Performance Testing:
Simulating performance with something like playwright seems like an appealing idea. Playwright simulates user actions, and performance testing aims to determine the performance of your application under load, seems like a good match. Unfortunately, there is one small problem: cost.
The problem comes with the amount of resources required to run these tests. The single tool I know of that supports Playwright as a load test is called Artillery. Luckily, they are very upfront about their resource recommendations and they have a cost breakdown. Normally, it is fairly trivial to generate load from a runner perspective because you have 50-100 virtual users (VUs) per vCPU, but when using Playwright to generate load, you need 1 vCPU per VU. That is a lot of resources. Keep in mind, these are just the costs for the hardware that is running your tests, NOT for the hardware required for your application to scale under load.
Costs might change in the future to make this approach more viable, but as it stands, I wouldn't recommend using Playwright to do performance or load testing on it's own becuase of the prohibitive cost and tooling available.
The only reason this is not the Avoid section is because there is an alternate approach that might suit your needs without being too costly. You can take a hybrid approach to performance testing. This allows you to simulate the majority of the load with the traditional API calls and run a small percentage of browser based tests at the same time. This will keep runner costs relatively low while still getting the feedback you might want with simulated user actions.
❌ Avoid
These are things that you should avoid. You are likely just testing something at the wrong layer.
Unit Testing Business Logic or Static Content:
This might seem obvious, but you should not use Playwright to test functionality that can be tested at a unit testing layer. Think critically about what can be pushed into a lower level of the test pyramid. If an e2e test is the first thing to catch static content rendering incorrectly, you are missing tests.
Testing Component Logic in Isolation:
Using Playwright for testing individual component logic is often overkill. For this, component testing libraries like React Testing Library, Vue Test Utils, or Storybook interaction testing provide faster feedback and more focused tests.
Excessive Mocking:
If your Playwright tests require extensive mocking of backend services or APIs, integration tests might be more appropriate.
Conclusion
Playwright proves itself to be an exceptionally powerful and user-friendly framework, particularly excelling in the realm of UI web application testing where its auto-waits, cross-browser capabilities, and superb tooling offer significant advantages. While it presents a viable option for other types of testing as well, it's crucial to recognize its limitations.
Playwright is not intended to replace foundational testing layers; unit and component tests remain indispensable for specific logic and isolated component validation. Ultimately, leveraging Playwright effectively means understanding its sweet spots and respecting the testing pyramid – using this robust tool strategically for complex E2E scenarios and critical user journeys, while relying on more focused tools for lower-level verification.