Testing Flutter apps with Cypress

Question

Is it possible to test Flutter applications using the Cypress framework instead of using the built-in testing components that Flutter provides? If so, what are the pros & cons of both for Flutter testing if I know Cypress well?

Solution

Yes, technically, it's possible.

Here's a spec that passes with the basic flutter counter app:

describe('Counter app', () => {
    beforeEach(() => {
      cy.visit('http://localhost:_example_port_/#/')
      cy.get('flt-semantics-placeholder').click({force: true})
    })
    it('Increments on button press', ()=>{
        cy.get(`[aria-label="0"]`)
        cy.get(`[aria-label="Increment"]`).click()
        cy.get(`[aria-label="1"]`)
    })
})

To explain, if you enable semantics by clicking on the hidden 'flt-semantic-placeholder' element, flutter adds a semantics layer used by screen readers. Widgets with tooltips and text are automatically assigned aria-labels, which Cypress can use to find and click on elements. (You can get more control over these labels using a Semantics Widget.)

I found this worked for the canvas renderer, but it crashed when I tried to run multiple test cases in the same run. So, use the html renderer, ie run flutter for the test with something like:

flutter run -d chrome --web-renderer html --web-port 3443 

Okay, so clicking a button is pretty straightforward. What about other interactions?

Inputing text into a field:

Pretty straightforward. See this example.

Simulating a scroll event:

In short, no solution yet. See this markdown.

Simulating drag and drop:

Not likely. See this markdown.

Now for the pros and cons...

Pros:

• Cypress is more user friendly than the integration testing flutter provides. Being able to easily hot reload tests, and being able to click around and inspect a live state of a failing test are nice features.

Cons:

• Can't (yet) simulate scroll or drag.
• Flutter web isn't too performant, and in particular the first load takes a long time. Tests are slow running.
• Nothing indicates either the Flutter team or the Cypress team have any plans to support testing Flutter with Cypress.
• As of this posting, no online guides or articles for testing Flutter with Cypress.

See also:

• What are the tags, <flt-*> generated by Flutter for Web?
• https://www.didierboelens.com/2018/07/semantics/