Learning Elm Part 11 - Encryption

Published: Mar 22, 2024 - Reading time: 16 minutes

In this iteration we’ll add encryption to our small application. I’ll start by giving a small overview of how I’m handling encryption in my old React application, and go on from there.

As always, you can find a deployed version here, with the Elm Debugger here, and the source code here.

Encryption at rest vs. encryption at replication
New initialization state machine
The encryption status state machine
Implementing encryption
Encrypted database abstraction
Switching between encrypted and plain-text databases
End-to-end tests
Final thoughts

Encryption at rest vs. encryption at replication

In my old React application, all data is stored in plain text in the browser, and encryption is done previous to replication:

+------------+                           +------------------+
| Plain-text |           encrypt         | Remote encrypted |
| IndexedDb  |  <====  & decrypt  ====>  | PouchDB          |
+------------+        transformation     +------------------+

In this application, I want to give users the chance to encrypt data at rest:

+-----------------+                                  +-----------+
| Plain-text      |  <===  encrypt before write ===> | Encrypted |
| Browser memory  |         decrypt after read       | IndexedDB |
+-----------------+                                  +-----------+

Having the data encrypted in the IndexedDB, has its benefits:

Threat	Encryption at replication	Encryption at rest
Browser vulnerability	Attacker reads plain-text	Attacker reads encrypted
Unattended device	Attacker reads plain-text	Attacker reads encrypted
Cross-site scripting	Attacker reads plain-text	?

But it also has its costs:

You need to enter a password each time you open the app
You can’t use indexes - data is encrypted!

Number one is obviously attenuated if you use a password manager.

Number two will be a problem later on, when we try to implement features like searching for a transaction, or listing transactions only of certain accounts. But there are workarounds.

About cross-site scripting, I am not quite sure how easy it would be for an attacker to gain access to the Elm application. With my limited knowledge of how it manipulates the DOM and where data lives, I can’t answer that question.

New initialization state machine

We will be encrypting both the settings database, and the transactions database. This adds several edge cases to our application initialization:

stateDiagram-v2 [*] --> Initializing Initializing --> FirstRun: No settings DB Initializing --> EncryptedDB: Encrypted settings DB EncryptedDB --> DecryptedDB: Password OK EncryptedDB --> DecryptionError: Bad password DecryptionError --> DecryptedDB: Password OK DecryptedDB --> [*]

I’ll also make things worse for myself by allowing unencrypted databases, and allowing switches between them:

stateDiagram-v2 [*] --> Initializing Initializing --> FirstRun: No settings DB Initializing --> PlainTextDB: No encryption Initializing --> EncryptedDB: Encrypted settings DB EncryptedDB --> DecryptedDB: Password OK EncryptedDB -->DecryptionError: Bad password DecryptionError --> DecryptedDB: Password OK FirstRun --> DecryptedDB: Created a password FirstRun --> PlainTextDB: No password PlainTextDB --> DecryptedDB: Set password DecryptedDB --> PlainTextDB: Removed password DecryptedDB --> DecryptedDB: Changed password PlainTextDB --> [*] DecryptedDB --> [*]

Those are a lot of arrows!

You can add/view transactions either in DecryptedDB or in PlainTextDB.

The first change, is the response to our initialize port in our src/ElmPort.js. It will now handle a new kind of response:

// src/ElmPort.js

// Elm calls this code to initialize the databases
app.ports.initialize.subscribe(async () => {
  try {
    // Try to open databases
    const resp = await dbPort.initialize();
    if (resp instanceof InitResponseFirstRun) {
      // This is an empty database
      app.ports.gotFirstRun.send();
    } else if (resp instanceof InitResponseOk) {
      // This is a non-empty database, with no encryption
      app.ports.gotInitOk.send(resp.settings);
    } else if (resp instanceof InitResponseEncrypted) {
      // =============  NEW  ========================== //
      // This is a non-empty database, with encryption  //
      // ============================================== //
      app.ports.gotEncryptedSettings.send();
    } else {
      // Error!
      console.error("Unknown response", resp);
      app.ports.gotInitError.send(`Unknown response ${JSON.stringify(resp)}`);
    }
  } catch (e) {
    // Error!
    console.error(e);
    app.ports.gotInitError.send(e.message);
  }
});

The encryption status state machine

In our Elm code, we will keep track of our settings status with a new EncryptionStatus type:

type EncryptionStatus
    = Unknown
    | Encrypted
    | Decrypted String
    | DecryptionError

The initial state will be Unknown. Our JavaScript “backend” will let us know if the database is encrypted so that our Elm “frontend” can ask for a password. That is done with the gotEncryptedSettings() port we have just seen above. These are the 4 new ports:

Port	Msg	Description
`gotEncryptedSettings()`	`GotEncryptedSettings`	JavaScript tells Elm that we need a password
`decryptSettings(password)`	n/a	Elm tells JavaScript to attempt decryption with the given password
`decryptedSettings(password)`	`GotDecryptionSuccess String`	JavaScript tells Elm we successfully decrypted with the given password
`decryptedSettingsError(msg)`	`GotDecryptionError`	JavaScript tells Elm that decryption failed, with a message

We also need to modify our saveSettings port, now it will send both a Settings model, and a Maybe String:

-port saveSettings : Settings -> Cmd msg
+port saveSettings : ( Settings, Maybe String ) -> Cmd msg

To ask for a password, we have a small change in our viewForm in src/Settings.elm, where we render the input box, and an “Open” button:

viewForm : State -> Html Msg
viewForm model =
    case model.encryption of
        Encrypted ->
            viewFormAskPassword model

        DecryptionError ->
            viewFormAskPassword model

        _ ->
            viewFormDecrypted model


viewFormAskPassword : State -> Html Msg
viewFormAskPassword model =
    let
        errorView =
            if model.encryption == DecryptionError then
                viewFormErrors [ "Invalid password" ]

            else
                div [] []
    in
    div []
        [ form [ class "ui large form", onSubmit DecryptSettings ]
            [ div [ class "field" ]
                [ label [] [ text "Enter password" ]
                , input [ name "currentPassword", type_ "password", cyAttr "current-password", attribute "autocomplete" "current-password", value model.inputs.currentPassword, onInput EditCurrentPassword ] []
                ]
            , div [ class "ui positive button right floated", cyAttr "open", onClick DecryptSettings ]
                [ text "Open" ]
            ]
        , errorView
        ]

If our settings are either Encrypted or DecryptionError, then we render the “Enter password” input.
Otherwise, we render the old form.

The old form will have three new inputs:

Current password (to change or remove password)
New password
Confirm new password

They look something like this:

New Views

These are the new Msg handlers:

update : Msg -> State -> ( State, Cmd Msg, Bool )
update msg model =
    case msg of
-- (...)
        GotEncryptedSettings ->
            ( { model | encryption = Encrypted }, Cmd.none, False )

        GotDecryptionError ->
            ( { model | encryption = DecryptionError }, Cmd.none, False )

        DecryptSettings ->
            ( model, decryptSettings model.inputs.currentPassword, False )

        GotDecryptionSuccess (Ok password) ->
            ( { model | encryption = Decrypted password }, Cmd.none, True )
-- (...)

GotEncryptedSettings transitions us to Encrypted
GotDecryptionError transitions us to DecryptError
DecryptSettings is triggered by the “Open” button, and runs the decryptSettings command
GotDecryptionSuccess transitions us to Decrypted, which holds the current password

Implementing encryption

I’ll use the same small encryption library I used in my React app. It uses the SubtleCrypto from the Web Crypto API. Given a passphrase, I create an encrypt/decrypt helper that uses AES-GCM to encrypt/decrypt text. The text is encrypted and returned with the initialization vector (IV) prepended. This means that our encrypted plain-text looks something like this: <iv>|<ciphertext>. Decrypting splits the text by the | to obtain the iv, and decrypts. Please use a peer-reviewed security system instead of copying and pasting my code! I am no security expert so I will recommend you look for other implementations. You’ll see a big red warning in the MDN docs linked above about rolling your own crypto.

We will need to keep both the _id and _rev fields unencrypted for our replication to work. All other fields are passed through JSON.stringify(), encrypted, and stored in a single field named crypt. The documents all look like this:

{
    "crypt": "fe78eb8739672b291b5f2a5f|a149...d1d3193",
    "_id": "2024-03-01-d80dfeef-4eab-4c01-98b8-13ba714b4966",
    "_rev": "1-b98e3bf9f6224e33e973de12223ef4c1"
}

Encrypted database abstraction

I’ll create two small classes that are basically passthrough to PouchDB. They’ll live in our src/Db.js.

I’ll use a “marker” document to signal that the database is encrypted. I’ll store this mark:

const ENCRYPTED_ID = "encryption-mark";
const mark = {
  _id: ENCRYPTED_ID,
  encryption: true,
  uuid: "2358ac53-feeb-49cf-afcd-84dcd0142a35",
};

If we find it, we know the DB is encrypted
If we can decrypt it, we know the password is correct

Besides the PouchDB methods we are currently using, both database classes will implement these:

initialize() will do some sanity checks
encrypt(password) will create a new encrypted database and return it, overwriting this one
decrypt() will decrypt the DB, overwriting it
isEncrypted() queries the encryption state
getPassphrase() returns the passphrase, if any.

Our first database implementation is our “plain text db”:

// src/Db.js

class Db {
  constructor(name) {
    this.name = name;
    this.db = new PouchDb(name);
  }

  async initialize() {
    try {
      await this.db.get(ENCRYPTED_ID);
      throw new MaybeEncrypted("We might have a encryption mark");
    } catch (e) {
      if (e.name == "not_found") {
        // ignore, happy path!
      } else {
        throw e;
      }
    }
  }

  put(doc, ...params) {
    return this.db.put(doc, ...params);
  }

  get(id) {
    return this.db.get(id);
  }

  remove(...params) {
    return this.db.remove(...params);
  }

  allDocs(...params) {
    return this.db.allDocs(...params);
  }

  bulkDocs(...params) {
    return this.db.bulkDocs(...params);
  }

  destroy() {
    return this.db.destroy();
  }

  isEncrypted() {
    return false;
  }

  getPassphrase() {
    return null;
  }

  async encrypt(passphrase) {
    const result = await this.db.allDocs({ include_docs: true });
    const docs = result.rows.map((row) => {
      delete row.doc._rev;
      return row.doc;
    });
    await this.db.destroy();
    const encrypted = new EncryptedDb(this.name, passphrase);
    await encrypted.initialize();
    await encrypted.bulkDocs(docs);
    return encrypted;
  }

  decrypt() {
    // no-op
    return Promise.resolve();
  }
}

Nothing much to see here!

The encrypted implementation is a bit more interesting:

class EncryptedDb extends Db {
    constructor(db, passphrase) {
        super(db);
        this.passphrase = passphrase;
        this.encryption = buildEncryption(passphrase);
    }

    async initialize() {
        const info = await this.db.info();
        if (info.doc_count === 0) {
            await this.db.put(await this.encryption.encrypt(mark));
        } else {
            try {
                const maybeEncrypted = await this.db.get(ENCRYPTED_ID);
                if (isEncrypted(maybeEncrypted)) {
                    const decryptedMark = await this.encryption.decrypt(maybeEncrypted);
                    if (isDecryptedMarkError(decryptedMark)) {
                        throw new InvalidPassphrase("The encryption passphrase is invalid");
                    }
                    // OK!
                } else {
                    throw new NotEncrypted("The encryption mark is not encrypted");
                }
            } catch (e) {
                if (e instanceof DecryptionError) {
                    console.error(e);
                    throw new InvalidPassphrase("The encryption passphrase is invalid");
                }
                if (e.name == "not_found") {
                    throw new NotEmpty("The DB has documents but no encryption mark");
                }
                throw e;
            }
        }
    }
//...

We have a small helper that encrypts/decrypts documents, built with buildEncryption(passphrase). When initializing a database, we first check to see if it has any documents… and basically implement this flowchart:

flowchart TD A[Open] -->|Get info| B{Has documents?} B --> |no| C(Insert mark) C --> Z[OK] B --> |yes| D{Has encryption mark?} D --> |no| E(Not Empty) E --> Y[ERROR] D --> |yes|F{Can we decrypt?} F --> |no|G[Bad password] G --> Y F --> |yes| H[Good password] H --> Z F --> |other| I[Bad encryption mark] I --> Y

The rest of src/EncryptedDb.js just uses our helper when reading/writing data:

// (...)
put(doc, ...params) {
    return this.encryption.encrypt(doc).then(enc => this.db.put(enc, ...params));
}

get(id) {
    return this.db.get(id).then(enc => this.encryption.decrypt(enc));
}

async allDocs(...params) {
    const result = await this.db.allDocs(...params);
    const rows = result.rows.filter(row => row.id != ENCRYPTED_ID);
    for (const row of rows) {
        if (row.doc) {
            row.doc = await this.encryption.decrypt(row.doc);
        }
    }
    result.rows = rows;
    return result;
}

async bulkDocs(docs, ...params) {
    const encrypted = [];
    for (const doc of docs) {
        encrypted.push(await this.encryption.encrypt(doc));
    }
    return this.db.bulkDocs(encrypted, ...params);
}
// (...)

Switching between encrypted and plain-text databases

I don’t think this feature is really useful, to be honest. I implemented it just to try it out. I will probably remove it later on, or add a big disclaimer to first download a copy of the data. The current implementation works in memory, so if the browser dies, hits a bug in my code or whatever, you could lose data. I could use a new database name to first have a new copy in IndexedDB, then delete the old. But what about new documents that you added in another tab after I called allDocs()? Or if your spouse pushed new documents through replication? Too many edge cases that just disappear when you kill this feature and implement import/export.

End-to-end tests

This is my new encryption.feature covering the implemented features:

Feature: Users should be able encrypt their data

    Scenario: Encrypting on first run
        When I set the new password to "secret password"
        When I set the new password confirmation to "secret password"
        And I save my settings
        When I reload the app
        Then I see "Enter password"

    Scenario: Opening an encrypted app
        Given an encrypted app with password "my cool password"
        And I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        When I reload the app
        Then I see "Enter password"
        When I enter the password "my cool password"
        And I click the "Open" button
        Then I see "Pizza"

    Scenario: Opening an encrypted app with the wrong password
        Given an encrypted app with password "my cool password"
        When I reload the app
        When I enter the password "wrong password"
        And I click the "Open" button
        Then I see an error message "Invalid password"

    Scenario: Reading encrypted data from PouchDB
        Given an encrypted app with password "my cool password"
        And I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        Then there are no unencrypted documents in PouchDB named "elm_expenses_local"
        And there are no unencrypted documents in PouchDB named "elm_expenses_settings"
        And there is an encrypted document in PouchDB named "elm_expenses_local" with ID starting with "2024-03-01"

    Scenario: Reading unencrypted data from PouchDB
        Given I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        Then there are no encrypted documents in PouchDB named "elm_expenses_local"
        And there are no encrypted documents in PouchDB named "elm_expenses_settings"
        And there is an unencrypted document in PouchDB named "elm_expenses_local" with ID starting with "2024-03-01"

    Scenario: Encrypting an unencrypted application
        Given I have saved the default settings
        And I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        When I go to settings
        And I set the new password to "secret password"
        And I set the new password confirmation to "secret password"
        And I save my settings
        # We need to give time for the settings to be persisted - replace later with "Then I am in transaction list"
        Then I see "Pizza"
        When I reload the app
        Then I see "Enter password"
        And there are no unencrypted documents in PouchDB named "elm_expenses_local"
        And there are no unencrypted documents in PouchDB named "elm_expenses_settings"
        And there is an encrypted document in PouchDB named "elm_expenses_local" with ID starting with "2024-03-01"

    Scenario: Decrypting an encrypted application
        Given an encrypted app with password "my cool password"
        And I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        When I reload the app
        And I enter the password "my cool password"
        And I click the "Open" button
        And I go to settings
        And I set the current password to "my cool password"
        And I save my settings
        # We need to give time for the settings to be persisted - replace later with "Then I am in transaction list"
        Then I see "Pizza"
        When I reload the app
        Then I see "Pizza"
        And there are no encrypted documents in PouchDB named "elm_expenses_local"
        And there are no encrypted documents in PouchDB named "elm_expenses_settings"
        And there is an unencrypted document in PouchDB named "elm_expenses_local" with ID starting with "2024-03-01"

    Scenario: Encrypting an encrypted application with a new password
        Given an encrypted app with password "my old password"
        And I have saved the following transactions:
            | date       | description | destination                  | source      | amount | currency |
            | 2024-03-01 | Pizza       | Expenses:Eat Out & Take Away | Assets:Cash | 3999   | USD      |
        When I reload the app
        And I enter the password "my old password"
        And I click the "Open" button
        And I go to settings
        And I set the current password to "my old password"
        And I set the new password to "pizza"
        And I set the new password confirmation to "pizza"
        And I save my settings
        # We need to give time for the settings to be persisted - replace later with "Then I am in transaction list"
        Then I see "Pizza"
        When I reload the app
        And I enter the password "my old password"
        And I click the "Open" button
        Then I see an error message "Invalid password"
        And there are no unencrypted documents in PouchDB named "elm_expenses_local"
        And there are no unencrypted documents in PouchDB named "elm_expenses_settings"
        And there is an encrypted document in PouchDB named "elm_expenses_local" with ID starting with "2024-03-01"

It is the first time in this series I actually verify that data is persisted in PouchDB, breaking encapsulation. This is quite common in end-to-end testing (verifying that data is actually stored where it is supposed to be).

To support this, I added a new method in my DevApi:

// src/DevApi.js
readRawDataFromDb(name) {
    const db = new PouchDb(name);
    return db.allDocs({include_docs: true}).then(results => results.rows.map(row => row.doc));
}


// cypress/e2e/encryption.js
Then('there are no unencrypted documents in PouchDB named {string}', (name) => {
    cy.readRawDataFromDb(name).then(docs => {
        const unencrypted = docs.filter(d => d.crypt === undefined);
        if (unencrypted.length > 0) {
            throw new Error("Found at least one document unencrypted: " + JSON.stringify(unencrypted[0]));
        }
    });
})

Then('there is an encrypted document in PouchDB named {string} with ID starting with {string}', (name, idPrefix) => {
    cy.readRawDataFromDb(name).then(docs => {
        if (docs.some(d => d.crypt !== undefined && d._id.startsWith(idPrefix))) {
            return;
        }
        throw new Error(`No encrypted document with ID starting with "${idPrefix}" found`);
    });
})

Final thoughts

Adding encryption added more edge-cases to handle, but it is of course worth it. Adding replication will add even more sources of error. These would be hard to deal with in a more “real application”. If you don’t want to rollout your own replication algorithm, you’d need to check if you are replicating to a CouchDB with the same encryption. How could you make sure you avoid writing plain-text data to an encrypted DB? All those are problems for future me. I’ll enjoy the non-replication problems for the time being.

One of the reasons I wanted to try RxDB is because they already have both replication and encryption. So be sure to look into it! They do per-field encryption, which can help you get around some of the pesky downsides when it comes to querying. The only downside is that they use crypto-js unless you pay the premium version, which uses the native crypto:

The 👑 premium encryption-web-crypto plugin that is based on the native Web Crypto API which makes it faster and more secure to use. Document inserts are about 10x faster compared to crypto-js and it has a smaller build size because it uses the browsers API instead of bundling an npm module.

Maybe I’ll have time and see if I manage to make it work with create elm app, but I doubt it.

In later iterations, I plan to add an in-memory, plain-text, PouchDB that will allow us to query documents by accounts (for example), with real-time synchronization to our encrypted IndexedDB PouchDB instance. But I’ll need to consider the performance and downsides of this first. Something like this:

[Plain-text, in-memory DB]   <---- Reads
           ⬆
encrypt/decrypt transform
           ⬇
[Encrypted, IndexedDB DB]    <---- Writes

Until next time!

Table of contents