PWA

3.1 Data Storage & Manipulation

Module Still Under Development

# Cache API Summary

# How Do I???

So, for all these examples the assumed set up is a main.js file that imports a cache.js file. All the methods dealing with the cache(s) will be inside the cache.js file and will return a Promise. Here is an example of a function inside the main.js file that calls a function in cache.js and receives a Promise in return. The code in main.js uses a then() to wait for the completion of the code in cache.js.

//main.js
import CACHE from './cache.js';
// from inside a method in main.js we will call a method in cache.js
const APP = {
  cacheRef: null, //a variable to hold a reference to a cache.
  someFunc(){
    CACHE.open('my-cache-name')
      .then(cache=>{
        APP.cacheRef = cache; //save the reference in the variable
      });
  }

1
2
3
4
5
6
7
8
9
10
11

If you need to do multiple things in a row, like open a cache, then delete a file, then get a list of all the files, then build the HTML for that list, then you need to use a chain of .then() methods with a return of the Promise method inside each then().

const APP = {
  someFunc(filename) {
    CACHE.open('my-cache-name')
      .then((cache) => {
        APP.cacheRef = cache; //save the reference to the cache
        return CACHE.delete(filename);
      })
      .then(() => {
        //now the delete is complete,
        //get the list from the cache passed in
        return CACHE.getFiles(APP.cacheRef);
      })
      .then((fileList) => {
        //fileList is the array of files
        //build the HTML
        return APP.buildList(fileList);
      })
      .then(() => {
        //HTML is built... need to update anything else?
      })
      .catch((err) => {
        //handle an error from any step
      });
  },
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Open a Cache and keep a Reference

If all we need to do is open a cache and keep a reference to that cache for later use...

const CACHE = {
  open(cacheName) {
    return caches.open(cacheName);
  },
};

1
2
3
4
5

# Get a List of Caches

If you want a list of all the available caches...

const CACHE = {
  listCaches() {
    return caches.keys();
  },
};

1
2
3
4
5

Then back in the main JS file we get an array that we can loop through.

//main.js
CACHE.listCaches().then((list) => {
  //list is an array that you can loop through
});

1
2
3
4

# Delete Old Caches

If you need to delete one or more caches we can use Promise.all() to delete an array of names. This is a less common thing to do. Typically, we do this in a Service Worker when we want to get rid of old versions of caches.

We pass the name (or names) of cache(s) to keep to our function. It will return a Promise that lets us know when it is complete.

const CACHE = {
  deleteCache(toKeep) {
    return CACHE.listCaches().then((keys) => {
      //keys is the array of all the cache names
      return Promise.all(keys.filter((key) => key != toKeep).map((name) => name.delete()));
    });
  },
};

1
2
3
4
5
6
7
8

# Get a List of Files in a Cache

This function could be written in two ways. First, you can pass a reference to the cache that contains the files. Second, you could pass the name of the cache that holds the files

const CACHE = {
  getFileList(cache) {
    //pass in the cache where the files are located
    //version 1 - by cache reference
    return cache.keys();
  },
  getFileList(cacheName) {
    //if you pass in the name of the cache instead, then you would call CACHE.open() first
    //version 2 - by cache name
    return CACHE.open(cacheName).then((cache) => {
      return cache.keys();
    });
  },
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Back in the main JS file we get an array of all the Request objects from the cache. Each of those Request objects has a url property which can be used to create a URL object. From the URL object you can extract any part of the url that you want - origin, hostname, pathname, port, protocol, query string, hash, etc.

//main.js
const APP = {
  cacheRef: null,
  init() {
    CACHE.open('my-cache-name').then((cache) => {
      APP.cacheRef = cache;
    });
  },
  getFiles() {
    CACHE.getFileList(APP.cacheRef).then((fileList) => {
      //fileList is an array of each Request in the Cache
      fileList.forEach((req) => {
        let url = new URL(req.url);
        console.log(url.pathname);
        console.log(url.origin);
        console.log(url.search);
        console.log(url.port);
        console.log(url.hash);
      });
    });
  },
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Delete a File in a Cache

To delete a file from a cache you need to have the file name or a Request containing the file name, plus a way to reference the cache. To reference the cache you need a cache name or a variable holding the reference to the cache. For this example we will pass in the reference to the cache.

You could alternatively pass in a Request object or URL object instead of a filename.

const CACHE = {
  deleteFile(filename, cache) {
    return cache.delete(filename);
  },
};

1
2
3
4
5

# Add a File to a Cache

Your function to add a file to a cache could accept a filename, a URL, or a Request object. The best approach is to have a Request object. This way you can define things like the HTTP method, whether it is cors or no-cors request, and whether or not credentials should be included.

The file to be saved needs to be wrapped in a Response object that also gets passed to the CACHE function.

//in main.js
let filename = crypto.randomUUID() + '.json';
let request = new Request(filename, {
  method: 'get',
  credentials: omit,
});
let data = JSON.stringify({ id: 123, some: 'info', type: 'object' });
let file = new File([data], filename, { type: 'application/json' });
let response = new Response(file, { status: 200, statusText: 'all good' });
CACHE.saveFile(APP.cacheRef, request, response)
  .then(() => {
    //this function runs after the file is saved.
  })
  .catch((err) => {
    //handles failure to save file in cache.
  });

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

The third thing that gets passed to the function is a reference to the cache.

//in cache.js
const CACHE = {
  saveFile(cache, req, resp) {
    return cache.put(req, resp);
  },
};

1
2
3
4
5
6

# Get the Contents from any File

When a cache saves files, it does so with key value pairs. The Request object is the key and the Response object is the value.

The Response object is a container that holds a File. Depending on the type of file there is a different response method that we have to use to get the contents.

Whether it is a Response object coming from a Cache or a Response object coming from a fetch, it is still just a Response object.

In our cache function we need a filename but we can uses caches instead of a reference to a single cache.

In the main Javascript file we will be given a Promise containing the contents of a file OR an Error.

//main.js
let filename = 'style.css';
CACHE.getText(filename)
  .then((txt) => {
    //txt is the text from the css file
  })
  .catch((err) => {
    //Error about no match
  });

1
2
3
4
5
6
7
8
9

# Get the Contents from a Text File

To get the contents of a text file, which includes .txt, .html, or .css, we use the .text() method.

const CACHE = {
  getText(filename) {
    return caches.match(filename).then((cacheResponse) => {
      //cacheResponse is the Response object or null
      if (cacheResponse) return cacheResponse.text();
      throw new Error('No match for text file in the cache');
    });
  },
};

1
2
3
4
5
6
7
8
9

# Get the Contents from a JSON File

To get an Object built from the contents of a .json text file we would use the .json() method on the Response object that is returned by the match() method.

const CACHE = {
  getJSON(filename) {
    return caches.match(filename).then((cacheResponse) => {
      //cacheResponse is the Response object or null
      if (cacheResponse) return cacheResponse.json();
      throw new Error('No match for json file in the cache');
    });
  },
};

1
2
3
4
5
6
7
8
9

# Get the Contents from an Image File

When you want an image (or any media file) from a cache we will use the blob() method.

The result from the blob() method is actual just a reference to the location in memory where the binary data from inside the image file is saved.

const CACHE = {
  getImage(filename) {
    return caches.match(filename).then((cacheResponse) => {
      //cacheResponse is the Response object or null
      if (cacheResponse) return cacheResponse.blob();
      throw new Error('No match for media file in the cache');
    });
  },
};

1
2
3
4
5
6
7
8
9

Then back in the main JavaScript file when we receive the blob reference variable, we need to use the URL.createObjectURL() method which will create a URL that points to the location in memory. This newly created URL can be used as the src for any <img> element.

//main.js
let filename = 'myavatar.png';

CACHE.getImage(filename)
  .then((blobRef) => {
    //blobRef is the pointer to the location in memory of the blob
    let url = URL.createObjectURL(blobRef);
    let img = document.getElementById('someImage');
    img.src = url;
  })
  .catch((err) => {
    //error about no file match in cache
  });

1
2
3
4
5
6
7
8
9
10
11
12
13

# More Data Storage

When you need to save information for a user in their own browser there are a variety of ways to save that information depending on how much data there is, how it needs to be and how frequently it needs to be accessed.

Ways to save data that can be accessed again later via JavaScript:

Cookies [see module 2.1]
WebStorage (LocalStorage and SessionStorage) [see module 2.1]
Cache API [see module 2.2]
IndexedDB [see module 3.1]
Browser Cache can save files but cannot be accessed via JavaScript [see module 2.1]
Generated files, that the user can save and then re-upload at a later time.

# IndexedDB

The notes here will give a brief introduction to how IndexedDB works. If you want a greater understanding of how to use IndexedDB as part of a project then you should watch this playlist (opens new window).

IndexedDB is a type of document database that you can create in the browser.

It has methods for opening and creating a new database and then creating collections. Then methods for adding new objects, updating objects, and deleting objects. Additionally, you can search through a collection for specific objects based on the values of different properties. Each of these tasks is called a request. And, just like an HTTP request, you won't know how long it will take. However, they don't use Promises, they use events.

The way you can know when your IndexedDB requests are finished running is with event listeners.

let db = null;
let dbOpenRequest = indexedDB.open('DBName', version);
dbOpenRequest.addEventListener('error', (ev) => {
  //database failed to open
});
dbOpenRequest.addEventListener('success', (ev) => {
  //database has opened successfully
  db = ev.target.result;
});
dbOpenRequest.addEventListener('upgradeneeded', (ev) => {
  //the db version has changed
});

1
2
3
4
5
6
7
8
9
10
11
12

When you need to run several requests in a row, as part of a bigger process, then you should use a transaction object to wrap each of the calls to the database.

//db is a reference to the opened IndexedDB
let tx = db.transaction('storeName', 'readwrite');
tx.oncomplete = (ev) => {
  //when the transaction is complete
};
tx.onerror = (ev) => {
  //something in the transaction failed
};

1
2
3
4
5
6
7
8

Once you have a transaction then you can point it at a specific object collection from your IndexedDB and then run whatever command(s) you want.

let store = tx.objectStore('storeName');
let getRequest = store.getAll();

1
2

Each request that you make inside the transaction has a success event and an error event.

getRequest.onsuccess = (ev) => {
  let request = ev.target; //the getRequest
  let result = request.result; //the result of the getAll() call
};
getRequest.onerror = (ev) => {
  //this request failed
};

1
2
3
4
5
6
7

You can make as many requests as you want inside the transaction. Each request has its own success and error event and the transaction has a success and error event too.

If you want, you can wrap your transaction and requests in functions and make those functions use Promises. That way you can chain together all your database requests.

Wrapping Event Listeners with Promises

Here is a tutorial about wrapping event listeners with Promises (opens new window).

# Manipulating Data

When you create objects in JavaScript they can be hard-coded, they can be from an API, or they can be from user created data. Whatever the source, you will often need to manipulate, edit, or rearrange that information.

Maybe you fetch data from an API and it returns an object like this:

let data = {
  created_at: 1701231234320,
  results: [
    {
      id: 92837492387,
      title: 'AI is learning from you',
      isbn: {
        'isbn-10': 1234567890,
        'isbn-13': 1234567890123,
      },
      authors: [
        {id:23746, name:'Yann LeCun', email: 'yann@meta.org'},
        {id:3453, name:'Tristan Harris', email: 'tristan@socialdilema.org'},
        {id:23746, name:'Sebastien Bubeck', email: 'sebastien@ms.org'},
      ],
      numbers: [
        small: [1, 4, 6, 3, 6, 8],
        medium: [1234, 3456, 6775, 86877],
        large: [1709812367228000, 15098234876234000, 4200003876],
      ],
      publisher: 'Sample publishing house',
      pages: 1234,
      other: 'more properties...',
    },
  ]
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

So, there is a lot of information here. There are many properties here and in the results array there is only one object. What if there were 1000 objects inside the results array? What if the only things you wanted from this data were the title and author name values from each object inside each object inside of results?

This is the type of data manipulation that you need to understand and need to be comfortable using regularly. We want to be able to remove objects that lack the properties we need. We want to discard properties that we don't need from each object. We might want to only keep the contents of the object array.

# Basic Object Manipulation

So, here are some common tasks for manipulating JavaScript Objects.

checking for property existence

If you want to check if a property exists in an Object or anywhere in it's prototype chain, then we can use the in operator as part of an if statement or ternary statement.

if ('someProp' in myObj) {
}
if ('map' in myArray) {
}
if ('length' in myStr) {
}
let doesExist = 'someProp' in myObj ? true : false;

1
2
3
4
5
6
7

If you need to know whether an Object has a property connected directly to itself, and not just somewhere on the prototype chain, then you should use the static Object method hasOwn.

let myObj = { someProp: 123 };
let myArr = [1, 2, 3, 4];
console.log(Object.hasOwn(Object, 'toString')); //false
console.log(Object.hasOwn(myObj, 'toString')); //false
console.log(Object.hasOwn(myArr, 'toString')); //false
console.log(Object.hasOwn(myObj, 'someProp')); //true

1
2
3
4
5
6

adding properties

Adding a property to a JavaScript object is the same as editing a property. If the property exists then the new value will replace the old one. If the property does not exist yet, it will be created. You can use either dot notation or the square bracket syntax.

myObj.someProp = 'new value';
myObj['someProp'] = 'another new value';

1
2

removing properties

You can assign null or undefined directly to a property. However, this does not remove the property. If you actually want to remove the property then you need to use the delete keyword.

delete myObj.someProp;
delete myObj.anotherProp;

1
2

# Accessing Properties

When you want to access the properties of an Object then we can use dot notation or square brackets. For an Array we need to use square brackets with integer indexes. You can drill down through each level by using chains of those.

//the isbn-13 number from the first object in the results array
data.results[0].isbn['isbn-13'];
data['results'][0]['isbn']['isbn-13']; //alternate version of previous line

//the last authors name from the first object in the results array
data.results[0].authors[data.results[0].authors.length - 1].name;

1
2
3
4
5
6

# Iterable Manipulation

JavaScript has a protocol called iterable. It is not an actual object type but more a set of rules that object types need to follow. An Array is a type of object that follows the iterable protocol. In some languages, Iterable is actually a type of object. You can say "iterable object" and other developers will understand what you mean.

The iterable object that you get from an API will always be an Array. At least until the new Tuple data type becomes an official part of the ECMA standard. Read about the Records and Tuples proposal (opens new window).

Arrays also have a new at method which is an alternative to the square brackets and index number for accessing an array value.

When you manipulate data held in Arrays, you will usually be doing so with the map, filter, reduce, or flatmap methods. These methods all return a new array

As a demonstration of the problem described above - removing objects from the results array that are missing a property, and removing the properties other than title and author names - this script does these things.

//let data = { results: [{}, {}]... }
let resultsData = data.results
  .filter((item) => {
    //running the filter first to remove objects we don't want to keep
    return Object.hasOwn(item, 'title') && Object.hasOwn(item, 'authors') && Array.isArray(item.authors);
    //only keep objects that have both a title and authors prop and authors is an array
  })
  .map((item) => {
    //create new objects that only have a title property and a copy of the authors array
    return { title, authors: [...item.authors] };
  });

1
2
3
4
5
6
7
8
9
10
11

# Modules, Namespaces, and Organizing Code

When you start building websites, they tend to be rather simple and will use only one JavaScript file. As functionality grows and the amount of code required for your web app grows, so does the importance of organizing your code in a way that is easy to manage and maintain.

Additionally, as part of the organization of our code, we need to avoid naming conflicts between our objects and methods, and the ones that we add to our website from external sources and libraries.

The first simple approach to avoiding naming conflicts is through namespacing objects. In namespacing we simply wrap a collection of variables and functions as properties inside an Object literal.

const APP = {
  apikey: 'unique-string-for-api-calls',
  baseurl: 'https://example.com/api/v2',
  init: function () {
    //function that gets called when the page loads
    APP.addListeners();
    APP.getData();
  },
  getData: function () {
    //function to make a fetch call
    fetch(APP.baseurl, {
      headers: {
        apikey: APP.apikey,
      },
    });
  },
  addListeners: function () {
    //add all the event listeners for the page
  },
};
//when the page loads call the APP.init method
document.addEventListener('DOMContentLoaded', APP.init);

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

We can capitalize the name of the namespace as a convention to make it easier to spot in the code. To use the namespace, all you need to do is prefix any of the properties or methods with the name of the namespace.

Before the addition of ES Modules support to the browsers developers would just create a different file for each namespace and then load them all via script tags in the HTML. Every script would get loaded into the global scope. The namespaces will prevent conflicts between any function or value as long as each of the namespaces have their own unique name.

Now, we also have ES Modules. So, we create the same separate files as we do with the namespaces. We can keep the namespace object or return everything back to individual variables and functions.

Each file needs to be loaded as a module. To do that we just add the type="module" attribute to the first <script> tag to be loaded by the webpage.

<script src="./js/main.js" type="module"></script>

As soon as one script is loaded as a module then it will be able to use the import command. Any script loaded via an import command will automatically be a module. In the module script, any variable or function that you want to be allowed to be imported, will have to be marked with the export keyword.

//./js/somecode.js
export default function first() {
  //the main function for this module
  third(); //calling a private function from this module
}

function second() {
  //some other exportable function
}

function third() {
  //a private function inside this module
}

const abc = 123;
let def = 'hello';

export { second, abc, def };

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

//./js/main.js
import start, { second, abc, def } from './somecode.js';

function init() {
  start(); //the function from the other file called first()
  document.body.addEventListener('click', second); //function from the other file
  console.log(abc);
  console.log(def);
}

document.addEventListener('DOMContentLoaded', init);

1
2
3
4
5
6
7
8
9
10
11

# ToDo This Week

To Do this week

read all the content for modules 3.1 and 3.2
read all the content for Hybrid web component creation
review Hybrid Web Component exercise 2
complete the Cache API exercise

← 2.2: Cache, Files & Responses 3.2: Intro to Workers →