Playwright is a free toolkit that controls web browsers automatically, so programmers can check that websites and apps work correctly. It is aimed mainly at software developers and testers, with extra tools for scripts and AI agents.
Has 3 of 7
How many people bookmarked its code on GitHub. It shows how popular it is, not how good it is or how well it replaces the app.
How hard it is to start using it. Easy: install it like a normal app, or use it in your browser. Advanced: you run it on your own server, use the command line, or it is a tool for developers.
Its code changed within the last year, so someone is still looking after it.
Does the whole job.
Ordered by how fully it replaces the app, Verified, setup and, when there are enough, people's votes.
Windows · Mac · Linux · Android · iPhoneApache-2.0
Maestro is a free, open-source tool that automatically tests apps and websites on phones, tablets and browsers by following simple written steps. It is aimed at software testers and developers who want to check that their apps work.
Has 2 of 7
How many people bookmarked its code on GitHub. It shows how popular it is, not how good it is or how well it replaces the app.
How hard it is to start using it. Easy: install it like a normal app, or use it in your browser. Advanced: you run it on your own server, use the command line, or it is a tool for developers.
Its code changed within the last year, so someone is still looking after it.
Does the whole job.
Ordered by how fully it replaces the app, Verified, setup and, when there are enough, people's votes.
pytest is a free tool that helps programmers write and run small checks, called tests, to confirm their code works as intended. It is made for software developers who build apps and libraries.
Has 0 of 7 · Tests in real browsers ✗ +5
How many people bookmarked its code on GitHub. It shows how popular it is, not how good it is or how well it replaces the app.
How hard it is to start using it. Easy: install it like a normal app, or use it in your browser. Advanced: you run it on your own server, use the command line, or it is a tool for developers.
Its code changed within the last year, so someone is still looking after it.
Does part of the job.
Ordered by how fully it replaces the app, Verified, setup and, when there are enough, people's votes.
This is an add-on for k6, a tool that runs performance and load tests, letting those tests create and check resources on Kubernetes, a system for running and managing software on server clusters. It is meant for programmers and IT staff who test software running on Kubernetes.
Has 0 of 7
How many people bookmarked its code on GitHub. It shows how popular it is, not how good it is or how well it replaces the app.
How hard it is to start using it. Easy: install it like a normal app, or use it in your browser. Advanced: you run it on your own server, use the command line, or it is a tool for developers.
Its code changed within the last year, so someone is still looking after it.
Does part of the job.
Ordered by how fully it replaces the app, Verified, setup and, when there are enough, people's votes.
None of these match. Try another device.
Help me choose
I want to pick an open-source replacement for TestRail. Below is a list from Open Source Everything (CC BY-SA 4.0), in the order its website shows it. The order is a starting point, not a verdict.
- Playwright: Playwright is a free toolkit that controls web browsers automatically, so programmers can check that websites and apps work correctly. It is aimed mainly at software developers and testers, with extra tools for scripts and AI agents. [Advanced setup; Windows · Mac · Linux; replaces fully; Apache-2.0; https://github.com/microsoft/playwright]
- Maestro: Maestro is a free, open-source tool that automatically tests apps and websites on phones, tablets and browsers by following simple written steps. It is aimed at software testers and developers who want to check that their apps work. [Advanced setup; Windows · Mac · Linux · Android · iPhone; replaces fully; Apache-2.0; https://github.com/mobile-dev-inc/maestro]
- pytest: pytest is a free tool that helps programmers write and run small checks, called tests, to confirm their code works as intended. It is made for software developers who build apps and libraries. [Advanced setup; devices not listed; replaces partly; MIT; https://github.com/pytest-dev/pytest]
- k6: This is an add-on for k6, a tool that runs performance and load tests, letting those tests create and check resources on Kubernetes, a system for running and managing software on server clusters. It is meant for programmers and IT staff who test software running on Kubernetes. [Advanced setup; devices not listed; replaces partly; Apache-2.0; https://github.com/grafana/xk6-kubernetes]
Setup rating: "Easy setup" means you can install it like a normal app, use it in a browser. "Advanced setup" means you must run it on your own server or use the command line, or it is a tool for developers.
Full details: https://opensourceeverything.org/data/groups/dev-testing.json
Before recommending anything, ask me these three questions (one at a time, in plain words): 1) Which device or devices will I use it on (Windows, Mac, Linux, Android, iPhone)? 2) How comfortable am I with technical setup: only installing apps from a store, happy to follow a step-by-step guide, or comfortable with servers and the command line? 3) What is my budget: free only, or willing to pay for hosting or support? Then recommend one to three projects from this list only, and for each say why, using how fully it replaces the app, its setup rating, and the devices it runs on. If nothing on the list fits me, say so plainly instead of stretching.
Before you build
Read the list of open-source projects at https://opensourceeverything.org/data/groups/dev-testing.json. Compare it with my project or idea below. Tell me which projects overlap, what I could reuse instead of rewriting, and what mine does that none of them do.
My project or idea:
