mirror of
https://github.com/gradio-app/gradio.git
synced 2025-02-11 11:19:58 +08:00
612fa6325c
* document CI * update documentation note about immortality of non-main documentation in S3 * Apply suggestions from code review Co-authored-by: Aarni Koskela <akx@iki.fi> * technical stuff * add more stuff * fix * Update testing-guidelines/ci.md Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * Apply suggestions from code review Co-authored-by: Aarni Koskela <akx@iki.fi> * fix * fix * fix * fix * fix * fix --------- Co-authored-by: Aarni Koskela <akx@iki.fi> Co-authored-by: Abubakar Abid <abubakar@huggingface.co>
104 lines
4.5 KiB
Markdown
104 lines
4.5 KiB
Markdown
## Playwright Tips
|
|
|
|
Gradio uses [playwright](https://playwright.dev/docs/intro) to interact with gradio applications programmatically to ensure that both the frontend and backend function as expected.
|
|
Playwright is very powerful but it can be a little intimidating if you haven't used it before.
|
|
No one on the team is a testing expert so don't be afraid to ask if you're unsure how to do something.
|
|
Likewise, if you learn something new about playwright, please share with the team!
|
|
|
|
### Tip 1 - Retrying Assertions
|
|
|
|
Playwright tests are written imperatively - first type into this textbox, then click this button, then check this textbox has this output.
|
|
This is nice because it matches how users interact with Gradio applications.
|
|
However, playwright carries out these steps much faster than any human can!
|
|
This can cause you to check whether a textbox has the correct output before the server is finished processing the request.
|
|
|
|
For this reason, playwright ships with some [retrying assertions](https://playwright.dev/docs/test-assertions#auto-retrying-assertions).
|
|
These assertions will retry until they pass or a timeout is reached, by default 5 seconds.
|
|
So even if playwright checks a DOM element before the server is done, it gives the server a chance to finish by retrying.
|
|
|
|
An example of a retrying assertion is `toBeChecked`. Note that you can manually increase the timeout as well:
|
|
|
|
```js
|
|
// 5 seconds
|
|
await expect(page.getByTestId('checkbox')).toBeChecked({timeout?: 5000});
|
|
```
|
|
|
|
An example of a non-retrying assertion is `isChecked`:
|
|
|
|
```js
|
|
await expect(page.getByTestId("checkbox").isChecked());
|
|
```
|
|
|
|
Sometimes there may not be a retrying assertion for what you need to check.
|
|
In that case, you can retry any custom async function until it passes using `toPass` ([docs](https://playwright.dev/docs/test-assertions#expecttopass)).
|
|
|
|
```js
|
|
await expect(async () => {
|
|
const response = await page.request.get("https://api.example.com");
|
|
expect(response.status()).toBe(200);
|
|
}).toPass();
|
|
```
|
|
|
|
### Tip 2 - Don't rely on internal network calls to check if something is done
|
|
|
|
Internal network calls are not visible to the user, so they can be refactored whenever.
|
|
If we have tests that rely on a request to a given route finishing before moving on, for example, they will fail if we ever change the route name or some other implementation detail.
|
|
|
|
It's much better to use a retrying assertion that targets a visible DOM element with a larger timeout to check if some work is done.
|
|
|
|
Avoid this:
|
|
|
|
```js
|
|
const uploadButton = page...
|
|
await uploadButton.click();
|
|
await page.waitForRequest("**/upload?*");
|
|
await expect(page.getByTestId("file-component")).toHaveValue(...)
|
|
```
|
|
|
|
Do This:
|
|
|
|
```js
|
|
const uploadButton = page...
|
|
await uploadButton.click();
|
|
await expect(page.getByTestId("file-component")).toHaveValue(..., {timeout?: 5000});
|
|
```
|
|
|
|
### Tip 3 - Use the playwright trace viewer
|
|
|
|
Whenever a test fails locally, playwright will write out some details about the test to the `test-results` directory at the top level of the repo.
|
|
|
|
You can view the trace using the following command:
|
|
|
|
```bash
|
|
npx playwright show-trace test-results/<directory-name>/trace.zip
|
|
```
|
|
|
|
You can see a "video" of the failing test, a screenshot of when it failed, as well as all the network calls and console messages.
|
|
|
|
|
|
|
|
If a test fails on CI, you can obtain the same trace by downloading the artifact from github actions.
|
|
|
|
1. From the failing Github Actions page, go to the `Summary` page
|
|
2. Scroll down to the bottom to where it says `Artifacts`
|
|
3. Click on `playwright-screenshots` to download a zip archive.
|
|
4. Unzip it and use the `show-trace` command.
|
|
|
|
|
|
|
|
### Tip 4 - Playwright can write the test for you
|
|
|
|
You can write the basic skeleton of the test automatically by just interacting with the UI!
|
|
|
|
First, start a gradio demo from the command line. Then use the following command and point it to the URL of the running demo:
|
|
|
|
```bash
|
|
npx playwright codegen <url>
|
|
```
|
|
|
|
This will open up a Chromium session where each interaction with the page will be converted into a playwright accessor.
|
|
|
|
NOTE: Only copy the `test("test-name", ....)` not the imports. For playwright to work when running in the gradio CI, `test` and `expect` need to be imported from `@gradio/tootils`.
|
|
|
|
|