Data Storage

IndexedDB API

# IndexedDB API

Today we are going to learn how to use an IndexedDB to store local data in the browser.

Here is an entire playlist of videos about IndexedDB (opens new window). The first video talks about using Jake Archibald's idb-keyval library (opens new window) as a quick way to get up and running with IndexedDB. The other 8 videos explain how to use IndexedDB with the native built-in browser API.

The idb-keyval library lets you manage a data store in IndexedDB if you are able to store your data in a simple structure like you would with localStorage. Think key-value pairs.

If you need a more robust data solution then the full native IndexedDB API gives us what we need - the ability to create a bunch of separate data collections each filled with a bunch of JSON-like objects.

Later this semester you will be learning how to work with MongoDB in the MAD9124 server-side course. MongoDB is what is known as a document database, as opposed to a relational database like MySQL.

Document databases are comprised of a series of collections. Each collection can have a bunch of documents. Each document is basically a JavaScript Object that has been converted to JSON. Most document databases will convert the JSON string into a binary representation for better compression and performance.

If you want to learn about the idb-keyval library then watch the first video in that playlist. Here, and in class, we will be discussing how to use the native IndexedDB API.

Just like the Cache API, it has a series of methods that work with Promises.

# IndexedDB Setup

The IndexedDB API allows us to work with the data from both a regular web page script as well as from a Service Worker.

The window.indexedDB property is the root property that gives us access to the database. We call the open(dbname, dbversion) method to access a specific database.

The structure for the data in the browser is:

domain -> database(s) -> collections -> documents.

You can have multiple databases.

Each databases can have multiple collections (also called Stores).

Each Collection|Store can hold many, many documents.

Each document is a JSON string representation of a JS Object.

When you open the database you need to pass in the name and version number for the database. If a database with that name and version exist then it will be opened. If it doesn't exist then it will be created and opened.

The open command runs asynchronously. It will trigger various events like success,error, and upgradeneeded. You can use addEventListener or the older on... syntax approach to adding the event listener functions.

let DB = null;
let dbOpenRequest = indexedDB.open('bookDB', 2);
dbOpenRequest.onupgradeneeded = function (ev) {
  DB = ev.target.result;
  //if you want to create collections/stores or modify them do it here
  //You can write code to make specific DB changes depending on the old db version
  if (ev.oldVersion < 1) {
    //remove a Store|collection
    DB.deleteObjectStore('bookList');
  }
  if (ev.oldVersion < 2) {
    //make changes if old DB version number is less than 2
    let options = {
      keyPath: 'bookid',
      //the required unique id property for each of the Store's documents
      autoIncrement: false,
    };
    //create a new store that will hold documents about books.
    let bookStore = DB.createObjectStore('bookList', options);
    //we can create indexes on the various properties expected in the documents.
    var titleIndex = store.createIndex('by_title', 'title', { unique: true });
    //we use indexes for sorting and searching our results.
  }
};
dbOpenRequest.onerror = function (err) {
  //an error has happened during opening
};
dbOpenRequest.onsuccess = function (ev) {
  DB = dbOpenRequest.result;
  //or ev.target.result
  //result will be the reference to the database that you will use
  console.log(DB.name);
  console.log(DB.version);
  console.log(DB.objectStoreNames); //list of all the store names.
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

Each time you want to work with the data inside a Store you need to use a transaction object. We create the transaction object and then use the transaction's objectStore method to access the one or more stores that will be used.

Then we would call one of the ObjectStore methods. Each method will return a Request object. When the method request is complete, it will trigger the complete event on the transaction.

let tx = DB.transaction(storeName, mode);
//storeName is a String or an Array
//mode is 'readonly' or 'readwrite'
let bookStore = tx.objectStore('bookStore');
// bookStore.count(); number of documents in the store
// bookStore.add(); add a new document
// bookStore.put(obj, key); update a document or insert if it doesn't exist
// bookStore.delete(); delete a document
// bookStore.get(key); get a single document whose key matches
// bookStore.getKey(keyRange); get a get from a key range
// bookStore.getAll(query); get all the documents that match
// Each of these methods returns a request object that will have
// a success event and an error event

tx.onerror = function (err) {
  //error happened
};
tx.oncomplete = function (ev) {
  //the transaction has run and given us the result, if any
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

The readonly mode is used for retrieving data from the database. All the other operations need to use readwrite.

We will use transaction -> store -> method as the approach for all our data operations with the database.

We can make as many method requests as we want on a store inside transaction. We would use the success event for each method request as the trigger for making the next method request. As an example, we could get() a record, and then call put() to make a change to it.

# Reading Data (cRud)

To read the data we will use the get(key) or getAll(query) methods. get will fetch a single document whose key matches the provided one. The property that was defined as the keyPath when the store was created is the key that we are matching. Alternatively, you can pass a keyRange to the get method.

The getAll method accepts a KeyRange object as the query to match. A KeyRange Object (opens new window) provides a lower and/or upper range bounding value(s). It will return all the documents that fall within the defined range. If you call the getAll method without a range then all documents in the store will be returned.

Here is an example of fetching a single document from a store.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let bookStore = tx.objectStore(storeName);
let bookid = '9781484257371';
let getRequest = bookStore.get(bookid);
getRequest.onerror = (err) => {
  //error with get request... will trigger the tx.onerror too
};
getRequest.onsuccess = (ev) => {
  let obj = getRequest.result;
  console.log({ obj });
  //will then trigger the tx.oncomplete
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Here is an example of fetching a bunch of documents that fall within a range.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let bookStore = tx.objectStore(storeName);
let range = IDBKeyRange.bound('A', 'F');
let getRequest = bookStore.getAll(range);
//range will be compared to the keyPath as well as any Store indexes
getRequest.onerror = (err) => {
  //error with the getAll
  //this will trigger the tx.onerror too
};
getRequest.onsuccess = (ev) => {
  let result = getRequest.result;
  //in this case it will be an Array
  result.forEach((doc) => {
    console.log({ doc });
  });
  //this will trigger the tx.oncomplete now
  //OR we could call another method here
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

A Key Range is create with a few different methods called on the IDBKeyRange object.

IDBKeyRange.lowerBound('B'); //match things higher in value than this
IDBKeyRange.lowerBound('B', true); //match things higher or equal to this
IDBKeyRange.upperBound('M'); //match things lower in value than this
IDBKeyRange.upperBound('M', true); //match things lower or equal
IDBKeyRange.bound('B', 'M'); //match things between these
IDBKeyRange.bound('B', 'M', true, true); //between or equal to these values

1
2
3
4
5
6

Any of these methods will return a key range object that can be used in the various method requests on the ObjectStore.

# Inserting Data (Crud)

Inserting a new document into the database is done with the add method request. This is the CREATE part of the CRUD operations.

When you are creating a database this is typically the first operation that you will do, since your database is of no value without data in it.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let bookStore = tx.objectStore(storeName);
let newBook = {
  bookid: 9876543210123,
  title: 'Cool Web Stuff',
  author: 'Steve Griffith',
};
let addReq = bookStore.add(newBook);
addReq.onerror = (err) => {
  //failed insert
};
addReq.onsuccess = (ev) => {
  console.log('book document added');
  //tx.oncomplete called now
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Updating Data (crUd)

Updating documents in the database store is done with the put method. You can pass in a document object or a document object and a key. If the key matches or the keyPath value in your object matches another document's key value in the store then an update will be done.

If there is no match for the keyPath value then the put method will do an insert.

The steps that you carry out before the put are the same as the ones that you do before an add or delete or get or getAll method request.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let bookStore = tx.objectStore(storeName);
let book = {
  bookid: 9876543210123,
  title: 'REALLLLY Cool Web Stuff',
  author: 'Steve Griffith',
};
let upsertReq = bookStore.put(book);
//will replace the existing book because it has the same bookid
upsertReq.onerror = (err) => {
  //failed update
  //tx.onerror triggered now
};
upsertReq.onsuccess = (ev) => {
  console.log('Book updated');
  //tx.oncomplete triggered now
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# Deleting Data (cruD)

When we need to delete documents from our database store then we use the delete method and pass it a key or a keyRange. Passing in a keyRange can delete multiple documents at the same time. Always be careful with your delete operations because there is no undo.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let bookStore = tx.objectStore(storeName);
let bookid = '9781484257371';
let delReq = bookStore.delete(bookid);
delReq.onerror = (err) => {
  //error trying to delete
  //tx.onerror triggered now
};
delReq.onsuccess = (ev) => {
  //tx.oncomplete will be triggered now
  console.log('document deleted');
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# Using Cursors

A cursor is a common way to step through all the documents retrieved by a getAll method request. We can create a cursor from one of the indexes that we created in our Store.

let storeName = 'bookStore';
let tx = DB.transaction(storeName, mode);
tx.onerror = (err) => {
  console.log('failed to successfully run the transaction');
};
tx.oncomplete = (ev) => {
  console.log('finished the transaction... wanna do something else');
};
let store = tx.objectStore(storeName);
//access the index from the store
let index = store.index('by_title'); //the one we created above
//create the range that will be applied to the store via the index
let range = IDBKeyRange.bound('G', 'N', true, true);
//create our cursor
//next, nextunique, prev, prevunique
let cursor = index.openCursor(range, 'next');
cursor.onsuccess = (ev) => {
  //called each time a value is retrieved from the cursor
  let cursorObj = ev.target.result; //or cursor.result
  if (cursorObj) {
    console.log(cursorObj.value);
    cursorObj.continue(); //triggers cursor.onsuccess again
  } else {
    console.log('end of cursor');
  }
};
cursor.onerror = (err) => {
  //an error when accessing the cursor or creating the cursor
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# What to do this week

TODO

Things to do before next week

Complete Exercise 1: IndexedDB
Read and Watch all the content from Modules 2.1, 2.2, and 3.1.
Watch the IndexedDB API Playlist (opens new window)
Prepare questions to ask in class.
Finish working on Hybrid one and submit it.

← 2.1: Cookies, LocalStorage, Cache 3.1: ES Modules →