Learning Elm Part 6 - End-to-End Tests

As always, you can find a deployed version here, with the Elm Debugger here and the source code here.

Adding End-to-End Tests

In the previous post we finally added some persistence to our Elm Application. In this post I wanted to start splitting the code into separate files. But instead I ploughed ahead with adding the features I wanted to use, which will eventually make me ditch my old React webapp¹. What I still need is:

1. Advanced edit mode
2. Auto-complete frequent transactions
3. Import/Export
4. Encryption
5. Replication

So I started with the first functionality and soon run into a bug. The problem is that my AI generated sample has transactions that cannot be created in the app’s current stage: they use accounts that are not in the default account list.

The bug is quite simple: until now, I had assumed transactions were created through the app, so when you edit them, the accounts should be in the dropdowns. But they are not! So if you click on most of the transactions in the sample, they will display the wrong accounts in the dropdowns. For example if we click on the Streaming service subscription renewal, where we should see Expenses:Entertainment and Liabilities:CreditCard:

We don’t see those, but the default Expenses:Groceries and Assets:Cash, because the custom accounts are not in the default list, which is used to render the dropdown.

Not only that, but I hadn’t realized that ChatGPT had switched the destination/source accounts. I should start writing this blog in the daytime, with plenty of coffee!

But as I have a day job, I’ll start with something different: adding end-to-end tests.

Exactly 10 years ago I had used a JavaScript BDD framework yadda with the nascent Selenium Webdriver JS implementation. I had a really nice experience with them.

10 years is like centuries in JavaScript framework time-scale, so I researched a bit to see what the cool kids were using today, and what people in the Elm community use. Many suggested and/or were using cypress at work - so I thought I’d try that.

Cypress

Cypress is a BIG piece of software system, and I really didn’t spend a whole lot of time reading through the docs. I followed the Installing Cypress in their Getting Started guied. Clicked next next next, and it generated all the default files.

I only added the baseUrl in the cypress.config.js to point to my development URL.

const { defineConfig } = require("cypress");

module.exports = defineConfig({
  e2e: {
    baseUrl: "http://localhost:3000",
    setupNodeEvents(on, config) {
      // implement node event listeners here
    },
  },
});

I see cypress is a very opinionated framework, with their own Best Practices. As I read through their documentation I chose to follow them and ask later. For example, they strongly recommend not to have cypress start the development server, which is something that I would have done 10 years ago. So I won’t do that!

This is my cypress/e2e/edit-transaction-with-custom-accounts.cy.js:

describe("Editing transaction with custom accounts", () => {
  it("should render the correct expense/source drop-downs", () => {
    // 1. Open our webapp
    cy.visit("/");

    // 2. Call an internal API to setup
    cy.addTransaction({
      date: "2024-02-29",
      description: "Fill-up tank",
      destination: "Expenses:Auto:Gas",
      source: "Assets:PayPal",
      amount: 1999,
      currency: "USD",
    });

    // 3. Now click the line to edit it
    cy.contains("Fill-up tank").click();

    // 4. Assert destination value and text
    cy.get('[data-cy="destination"]').should("have.value", "Expenses:Auto:Gas");
    cy.get('[data-cy="destination"]')
      .find("option:selected")
      .should("have.text", "Expenses:Auto:Gas");

    // 5. Assert source value and text
    cy.get('[data-cy="source"]').should("have.value", "Assets:PayPal");
    cy.get('[data-cy="source"]')
      .find("option:selected")
      .should("have.text", "Assets:PayPal");
  });
});

I had to do a few things to make this work. Let’s go through them.

Test Isolation

In their Test Isolation documentation of their Core Concepts they state that their default testIsolation for e2e tests is set to true. And this means:

When test isolation is enabled, Cypress resets the browser context before each test by:

• clearing the dom state by visiting about:blank
• clearing cookies in all domains
• clearing localStorage in all domains
• clearing sessionStorage in all domains

My first problem was that PouchDB uses IndexedDB - so no, my tests were not isolated, because it is not in the list above.

So I asked ChatGPT and confirmed with the MDN documentation on how to delete my databases. To ChatGPT’s credit, I thought it had hallucinated the cypress-indexeddb package, but it seems to exist!

I created this small command instead, and placed it in the cypress/support/commands.js file:

Cypress.Commands.add("deleteIndexedDB", () =>
  cy
    .window()
    .its("indexedDB")
    .then((indexedDB) => indexedDB.deleteDatabase("_pouch_elm_expenses_local"))
);

Which basically calls window.indexedDb.deleteDatabase(...) but through cypress.

I added that command in all the end-to-end tests by using beforeEach() in cypress/support/e2e.js:

import "./commands";

beforeEach(() => cy.deleteIndexedDB());

And now I was good to go

Programmatic APIs

In Organizing Tests, Logging In, Controlling State they list this anti-pattern:

❗ Anti-Pattern: Sharing page objects, using your UI to log in, and not taking shortcuts.

✅ Best Practice: Test specs in isolation, programmatically log into your application, and take control of your application’s state.

I thought a bit about how to translate that into my use-case, and if that was really necessary. I finally decided to try it out, exposing a JavaScript API in development mode.

I added ELM_APP_EXPOSE_TEST_JS=true to my .env file, and disabled it in .env.production. And in my src/index.js:

import "./main.css";
import { Elm } from "./Main.elm";
import * as serviceWorker from "./serviceWorker";
import PouchDb from "pouchdb-browser";

const exposeJsApi = process.env.ELM_APP_EXPOSE_TEST_JS == "true";

async function main() {
  let db = new PouchDb("elm_expenses_local");
  // (...)
  if (exposeJsApi) {
    window.ElmExpenses = {
      deleteAllData,
      importSampleData,
      putTransaction,
    };
  }
}
// (...)

With that setup, I added the new cypress command:

Cypress.Commands.add("addTransaction", (txnInput) =>
  cy
    .window()
    .its("ElmExpenses")
    .then((ElmExpenses) => ElmExpenses.putTransaction(txnInput))
);

That is what is being called in cy.addTransaction() in the test above.

Adding selectors

In their Selecting Elements section of their Best Practices, they recommend using data-*, for example data-cy, to find elements in the application:

<button
  id="main"
  class="btn btn-large"
  name="submission"
  role="button"
  data-cy="submit"
>
  Submit
</button>

And use cy.get('[data-cy="submit"]').click() - so… ok, I’ll use that.

I made my small Elm helper function:

cyAttr : String -> Html.Attribute Msg
cyAttr name =
    attribute "data-cy" name

-- (...)
viewEmptyState : () -> Html Msg
viewEmptyState _ =
--(...)
  button [ class "blue ui button", cyAttr "add-transaction", onClick (SetPage Edit) ]
    [ text "Add Transaction" ]

And used it as I created my tests.

HTTPS in Local Network

For some reason, when running with Firefox, it will not use http://localhost:3000 but my address in my local network:

On Your Network: http://192.168.1.13:3000/

The problem with this, is that Firefox disables all the crypto API in that case, so it breaks my app. And my tests.

Anyway, I am using Electron for the tests because of this.

The failing test

Running npm run cypress:open opens the UI (which is great, by the way), and it looks something like this:

You can also run it through their command line:

(...)

  Running:  edit-transaction-with-custom-accounts.cy.js                                     (2 of 3)


  Editing transaction with custom accounts
    1) should render the correct expense/source drop-downs


  0 passing (5s)
  1 failing

  1) Editing transaction with custom accounts
       should render the correct expense/source drop-downs:

      Timed out retrying after 4000ms
      + expected - actual

      -'Expenses:Groceries'
      +'Expenses:Auto:Gas'
      
      at Context.eval (webpack:///./cypress/e2e/edit-transaction-with-custom-accounts.cy.js:17:42)




  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  1                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     5 seconds                                                                        │
  │ Spec Ran:     edit-transaction-with-custom-accounts.cy.js                                      │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


  (Screenshots)

  -  /home/pablo/Code/elm-expenses/cypress/screenshots/edit-transaction-with-custom-a     (1280x720)
     ccounts.cy.js/Editing transaction with custom accounts -- should render the corr               
     ect expensesource drop-downs (failed).png                                                      

And this is the screenshot it generated:

Fixing the bug

I’ll use an intermediate solution for now. I’ll modify my FormInput record to hold to new list of strings:

• extraDestinations for extra destinations
• extraSources for extra sources

I’ll modify my transactionFormInput function to populate them, and modify the two functions which populate the options

transactionFormInput : Transaction -> FormInput
transactionFormInput txn =
    { id = txn.id
    , version = txn.version
    , date = Date.toIsoString txn.date
    , description = txn.description
    , destination = txn.destination.account
    , source = txn.source.account
    , amount = toFloat txn.destination.amount / 100.0 |> String.fromFloat
    , currency = txn.destination.currency
    , extraDestinations = [ txn.destination.account ]
    , extraSources = [ txn.source.account ]
    }

destinationOptions : Model -> List (Html Msg)
destinationOptions model =
    let
        options : List String
        -- Join the defaults from settings with the extra ones
        options =
            (model.settings.destinationAccounts ++ model.formInput.extraDestinations)
                -- keeping only the first one found
                |> List.Extra.unique

        selectedOpt : String
        selectedOpt =
            model.formInput.destination
    in
    options
        |> List.map (\opt -> option [ value opt, selected (selectedOpt == opt) ] [ text opt ])


sourceOptions : Model -> List (Html Msg)
sourceOptions model =
    let
        options : List String
        options =
            (model.settings.sourceAccounts ++ model.formInput.extraSources)
                |> List.Extra.unique

        selectedOpt : String
        selectedOpt =
            model.formInput.source
    in
    options
        |> List.map (\opt -> option [ value opt, selected (selectedOpt == opt) ] [ text opt ])

I’ll also need this for my frequent transactions auto-complete feature later (where you might be adding a transaction frequently that is not in the default list, through the advanced edit mode).

And voilà - the test passes:

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  crud-transaction.cy.js                   00:03        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  edit-transaction-with-custom-accoun      00:01        1        1        -        -        - │
  │    ts.cy.js                                                                                    │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  first-run.cy.js                          00:01        3        3        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:05        7        7        -        -        -  

Final thoughts

Cypress seems like an impressive tool. I was impressed! I see there is a cucumber/gherkin pre-processor for cypress, and I’d like to try it out. Having business-oriented features, with no clutter about [data-cy="xxx"], was a big win for me even though it adds complexity to the setup. Maybe it will crop-up in this never-ending series.

Ok, enough for today. Bye!