PWA

2.1 Web Storage, Cookies, & Cache

Module Still Under Development

# Client-Side Storage

There are a number of ways that we can store information in the browser (on the client-side). This week we will examine those options.

First we need to remember why we store data on the client-side instead of the server-side.

Client-Side Data	Server-Side Data

Available only from one device	Available to all users potentially
Tied to one domain	Can be used across domains through an API
Can be deleted by the user easily. User controls their own data.	Requires controlled access to delete
No guarantee that the information is kept. The browser is allowed to delete it.	Server-side databases designed to permanently and redundantly store information.
Not available directly by the server	Controlled by the domain owner

All the different types of client-side storage can be seen in Chrome via the Developer Tools Application tab.

# Cookies

Cookies were the first form of storage that was available in the browser. They are just a single string that is saved in the browser, in a storage area that is tied to a specific domain. Originally cookie strings were only allowed to be 255 characters long. Now the string length is over 2000 characters, which is a long string but still fairly limited.

Cookie Security

Cookies from one domain are NOT allowed to be read by code that came from another domain.

Each browser has its own limitations on storage of cookie data. There is a limit for the total storage of cookies as well as a limit per domain.

The cookie strings are made up of name value pairs, just like values in the query string of a URL. Here is an example of what the data part of a cookie could look like.

let cookieData = 'name=Steve;';

In addition to the data part of the cookie string, there are settings that control different limits on the cookie, such as the max age or expiry date for the cookie. Notice the semi-colon at the end of the data portion of the cookie string. AFter the semi-colon we would add the other settings. Each of the settings are separated by a semi-colon.

let cookieSettings = 'Path=/;Domain=127.0.0.1;Max-Age=30000;Secure;SameSite;HttpOnly;';
let cookie = cookieData + cookieSettings;

1
2

We would combine these two parts together to create a whole cookie string. Alternatively you could just put it all in one string to begin with.

Path

By default, cookies apply to the root folder of your website. However we can add to this path value to restrict the cookies to a smaller part of our website.

Domain

By default, the domain value will be the domain of the HTML file. However, we can restrict it further to a specific subdomain if we want.

Max-Age

The max-age part of the cookie String will be the number of seconds that the cookie is to be considered valid.

Expires

As an alternative to the Expires setting you can provide an expiration date for the cookie. This will be compared to the time and date in the browser, not any server time or date.

Secure

If the secure value exists it means the cookie can only be accessed or set over https.

HttpOnly

The HttpOnly setting means that the setting is only used during HTTP requests to the server. The JavaScript cookie API does not have access to the value when reading a cookie.

SameSite

With the SameSite value set the cookie can be restricted to only to be sent with requests for files that are going to the same domain as the original HTML file.

Cookies on the client side can only be read by pages that come from the same domain as the cookie.

However, the cookie string can be sent from the browser as one of the headers in any request to any domain. To prevent this you should add SameSite; to your settings.

SameSite can be set to the possible values Lax, None or Strict. With the value Strict the cookies will only be sent with requests to the matching domain. Lax is similar but the cookies are only sent when the user is navigating to the domain where the cookie originated. The cookie would not be sent with an API request sent to the cookie's domain. None means that cookies will be sent with all requests except that the Secure setting must have also been set.

If you want to set or update a cookie from the browser you only have to overwrite the whole cookie string using the document.cookie property.

document.cookie = 'user=123&name=Steve;path=/;domain=localhost;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';

That one line of code will replace the previously saved cookie with those settings.

Every change to the settings means that you are setting a separate cookie. This snippet is actually trying to set three cookies in the browser. However, since the first and last cookies have the same path, and domain, and default values for the other settings, the third cookie actually overwrites the first one.

document.cookie = 'user=123;Path=/;Domain=127.0.0.1;Max-Age=55000;';
document.cookie = 'name=Tony;Path=/;Domain=127.0.0.1;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';
document.cookie = 'user=456;Path=/;Domain=127.0.0.1;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';

1
2
3

When you want to read the value from the cookies you would put the document.cookie value into a variable, split the string on the ; semi-colons plus a space, and then take each portion.

let cookie = document.cookie;
console.log(cookie); // `user=456; name=Tony` or  `name=Tony; user=456`
let cookies = cookie.split('; ');
//cookies is an array of name=value pairs
cookies.forEach((c) => {
  //use split and array destructuring
  let [name, value] = c.split('=');
  console.log(`The ${name} cookie has a value of ${value}`);
});

1
2
3
4
5
6
7
8
9

All the cookies that have been set for the current Path and Domain will be gathered into a single string that we split apart and then split each of the parts on their = equal signs.

# Passing Data between Pages

When building a multi-page website there will often be times when you want to pass data, as key-value pairs, between your pages.

Each HTML page will load, and then load the script for the page. So, we need to be able to package the data in a way that can then be sent from the first page along with the HTTP Request and then read from the HTTP Response. The approaches from easiest to hardest are:

In the URL querystring.
In a cookie.
Written with some type of storage API that can be read on the second page (localStorage, sessionStorage, IndexedDB, and CacheAPI).

# In the QueryString

To pass data through the querystring, you can hard code a value in an anchor tag.

<a href="/pagetwo.html?name=sam&brother=dean">Go to Page Two</a>

Then in the script on the second page you can extract the value from the querystring.

let url = new URL(document.location.href); //the current page url
let params = url.searchParams; // a URLSearchParams object with all the queryString parameters
console.log(params.has('name')); // true
console.log(params.get('name')); // sam
console.log(params.get('brother')); // dean

1
2
3
4
5

In the first page, if you want to dynamically add elements to a querystring in a URL, you can use a similar script to the second page.

let url = new URL('http://localhost:/pagetwo.html');
let params = new URLSearchParams(); // or url.searchParams
params.set('name', 'sam');
params.set('brother', 'dean');
url.search = params.toString();
console.log(url);
location.assign(url.href); //navigate to the new page with the querystring

1
2
3
4
5
6
7

Another way this can be accomplished is by submitting a form. An HTML <form> element has a method attribute with a default value of GET, and an action attribute which is the URL to use as the HTTP Request URL. When a form is submitted an HTTP GET Request is sent to the server for the URL defined in the action. The elements from the form (input, select, textarea, button) will have their name and value pairs added to the querystring if the method is the default GET.

<form action="pagetwo.html" method="GET">
  <input type="text" name="name" value="sam" />
  <input type="text" name="brother" value="dean" />
  <button name="send">SEND DATA TO PAGE TWO</button>
</form>

1
2
3
4
5

NOTE: the form elements must have name attributes, not just ids.

When the user clicks the button, the form is submitted. This means an HTTP request is made for the url in the action attribute using the method defined in the method attribute.

A cookie is really just a header in an HTTP Request or Response.

An HTTP Request can have a header called cookie, whose value will be the cookies currently saved in the browser.

An HTTP Response can have a header called set-cookie, whose value will be a cookie that needs to be saved by the browser.

If you use JavaScript to set a cookie in the browser:

document.cookie = 'name=sam;maxAge=3600;secure=false;path=/';

then it will be part of the next HTTP Request made to the server. The next page to load will be able to access the document.cookie value too.

Cookies that have the httpOnly=true flag setting will not be cookies that can be set, read, or modified in JavaScript. They will automatically be sent with every navigation request and response, but not with fetch calls. They can only be set in a Response object that is coming from the server.

# Storage APIs

With localStorage and sessionStorage you use a key to label the data and then you can store any value that can be stringified with the JSON.stringify() method.

//on page one
let data = {
  name: 'sam',
  brother: 'dean',
};
localStorage.setItem('mydata', JSON.stringify(data));

//on page two
let json = localStorage.getItem('mydata');
let data = json ? JSON.parse(json) : null;

1
2
3
4
5
6
7
8
9
10

With IndexedDB you can also save information to be read later on from another page. The premise is the same as working with localStorage or sessionStorage, it just requires more code.

With the Cache API you can also save data to be read later. However, you need to turn the data into a file to be opened and read later.

//on page one
let dataTxt = 'name=sam&brother=dean';
let textFile = new File([dataTxt], 'data.txt', { type: 'text/plain' });
let dataObj = {
  name: 'sam',
  brother: 'dean',
};
let json = JSON.stringify(dataObj);
let jsonFile = new File([json], 'data.json', { type: 'application/json' });

let cache = caches.open('myfiles');
let textRequest = new Request('data.txt');
cache.put(textRequest, new Response(textFile, { headers: { 'content-type': 'text/plain' } }));

let jsonRequest = new Request('data.json');
cache.put(jsonRequest, new Response(jsonFile, { headers: { 'content-type': 'application/json' } }));

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

The above code will create a text file and a json file and save them in the cache for the current origin.

To retrieve the data we use the cache.match() method to extract the Response objects from the cache. Then you can do exactly what you would do with a fetch response. Call the .json() or .text() method to read the contents from the files.

# WebStorage (LocalStorage and SessionStorage)

The Web Storage API has two parts - localStorage and sessionStorage. They have the same methods and properties and work the exact same way. The only difference is that the sessionStorage values are automatically deleted at the end of the user's session. This means when they close the browser or the tab that contained the current domain as part of the tab's current history.

From this point on I will be referring to localStorage only but everything also applies to sessionStorage. Both are a global property accessible from the window object in the browser. They do not exist in NodeJS.

All the data stored in localStorage gets stored as a JSON string.

This means that only information that is capable of being a JSON string can be saved in localStorage. No functions, no DOM elements, can be stored in localStorage.

We will need to use the built-in JSON methods to convert the data back and forth between a string and an object.

let myObject = {
  id: 123,
  title: 'The Big Bang Theory',
  seasons: 12,
  firstSeasonYear: 2007,
  stillPlaying: false,
};
//convert the object into a JSON string.
let stringData = JSON.stringify(myObject);
//convert the string back into an object
let objectCopy = JSON.parse(stringData);

1
2
3
4
5
6
7
8
9
10
11

If you ever need to make a deep copy of an object, this is one way of doing it.

If the data that you want to convert to JSON is already a just a string, then you should not use the JSON.stringify() method as it will add an extra set of quotation marks.

Remember

It is important to remember that we have no way of editing the strings inside of localStorage. All we can ever do is replace the string with a new one.

Every time we read or write a value from localStorage we need to use a unique key. The key will be the unique name for that chunk of data. All keys need to be String values. An example key that you could use for an assignment would be abcd0001-com.example-userscores. It has a username, a domain name, and a descriptive term for what the data is.

Unique Key Names

Remember to keep your localStorage key names unique. You don't want to overwrite other data being saved in localStorage or have your data overwritten. Use a name that includes a reference to the domain, the web application, and even your own development team, plus the purpose of the data.

# Single Source of Truth

When working with data sources like localStorage and cookies, which are not directly accessible as a variable, we need to create what is known as a single source of truth.

It is possible to have multiple functions in multiple scripts that are accessing the values in your cookies or localStorage within the context of the same page load. If one script updates localStorage then we might not know that the change was made in another function that is sharing that data.

To avoid problems caused by this the best approach is to create your own globally accessible variable that will hold the data.

The cookie and localStorage values are the CACHED copy of the data. They only exist to give us context when the app is loaded the next time. We do NOT work directly from these sources. We do not keep updating and re-reading these data sources. Only read the value once when the page loads. Work with a local copy of the data from a variable. Update your variable as needed. Overwrite the cookie or localStorage after you update your variable.

When the page first loads, we read the value from the cookie and/or localStorage and put it into the global variable.

Any function that needs to access or change the information will make the change to that global variable.

Each time the global variable is altered you can call on the document.cookie property or localStorage.setItem method to update the cached value. Remember we are only updating the cache for the next time the app loads.

The only time we ever read from the storage is when the app first loads - using the DOMContentLoaded event as our trigger to fetch the cached data.

# Caches

The other place where we can store data is in the local Cache API. To be clear, this is the cache that we can see through the Application tab in the Chrome Web Development tools.

The Cache API is not to be confused with the Cache-Control Header and the browser cache which is part of the HTTP Response specification. The Cache API is fully controlled by the developer. The developer gets to decide which files are saved and for how long.

The browser cache is the cache where the browser saves previously downloaded images, css, js, and html files. The browser does this to save time and bandwidth. The browser also has its own cache, not the one controlled by the Cache API. It will save copies of the files in each website. Each file has an expiry date when a new version will need to be downloaded to replace the cached version.

The Cache-Control Header (opens new window) is a header that is sent from the server or proxy-server to the client along with a requested file. It tells the browser how to handle the caching of the requested file and how long it should be considered valid before requesting a new copy of the file.

If you need to review what headers are and how they work, review the notes in MAD9014 (opens new window).

# Cache-Control Header

The name of the header is cache-control and it can hold a variety of directives. Here are a few examples of the header showing some of the directives.

Cache-Control: max-age=604800
Cache-Control: no-cache
Cache-Control: max-age=604800, must-revalidate
Cache-Control: no-store

1
2
3
4

Here is a video about generating custom Response Objects that contain File Objects and how to save them with the Cache API.

# Chrome Dev Tools

Here is a short video that outlines all the Chrome Dev Tools that you could use to work with all the data storage and file apis.

# Caching, Proxy Servers, and CDN

The primary purpose of caching a file is to save time later. If you already have a file in your cache then you won't have to download it again. Cache-control headers control how a browser handles files that it receives.

The Cache API allows another level of caching which is controlled through JavaScript.

When you request a file from a Server, that server could be located anywhere in the world. There will be a delay between when a file is requested and returned to the browser. This delay is generally be referred to as latency.

There are also CDNs - Content Delivery Networks that can be used to reduce latency. These are made up of proxy servers, which are located all over the world, so that they will be closer to anyone requesting content from a website that is associated with the CDN (pays the CDN). The closer proxy server will have lower latency than the distant servers. Each proxy server can save cached copies of request files.

The length of time that files are cached for will vary depending on the file type, the original source of the file, and the popularity of the files.

An example of a CDN that you will probably use at some point is https://www.jsdelivr.com/ (opens new window). This is a CDN that can be used for free hosting of public JS libraries.

In the coming weeks we will be talking about Service Workers. These Service Workers will be able to act like a Proxy Server except it will be on each users' own computer.

# Serializable

In JavaScript, functions are first class objects. That means you can store a reference to a function inside a variable, and you can pass a function reference as a parameter to a function or as a return value from a function.

However, you cannot store a function inside of a JSON string. There is no way to send a function to a server via an HTTP Request or from a server via an HTTP Response.

Functions are not serializable. This means that they cannot be converted into a JSON representation. HTML Elements are another type of Object that is also not serializable.

# StructuredClone

If you want to make a copy of a JavaScript object and avoid having to recursively copy the nested objects, then you can use the new structuredClone() method.

let obj = {
  prop1: [1, 2, 3, 4, 5, 6],
  prop2: { a: 123, b: 234, c: 456 },
};
let copy = structuredClone(obj);
//make a copy of the nested array and object inside the new copy object

1
2
3
4
5
6

# Passing Functions

So, if you can't pass a function through an HTTP Request or Response it also means that you can't pass a function as a message between a page and a service worker or between tabs. Later on we will be talking about Client API messaging, Broadcast messaging, and Channel messaging. We will need a solution to passing a function.

What we can pass is a String. We can pass the name of a function as a String.

When you declare a function in a simple non-module script, the name of the function is actually added to the window object. You can check if the property is there.

function sam() {
  console.log('this is sam.');
}

function dean(nameOfFunc) {
  if (nameOfFunc in window && typeof window[nameOfFunc] === 'function') {
    //property exists in the window object
    //and the property is a function
    //so you can call the function
    nameOfFunc();
  } else {
    console.log(`function ${nameOfFunc} does not exist.`);
  }
}
dean('sam'); //this is sam
dean('crowley'); //function crowley does not exist
dean('rowena'); //function rowena does not exist

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

The above solution works if you are not dealing with a module. When you load a script with <script type="module"> it means that a module scope is created for the file and the functions are NOT added to the window object. If this is the case, then you can take the name of the function and wrap it in an eval() call. This will try to convert the string into an object with a matching name - the function.

//script loaded as module
function sam() {
  console.log('This is sam.');
}
function dean(nameOfFunction) {
  if (typeof eval(nameOfFunction) === 'function') {
    //it is a function so call it
    eval(nameOfFunction)();
  } else {
    console.log(`${nameOfFuntion} is not a function.`);
  }
}
dean('sam'); // This is sam.
dean('charlie'); // charlie is not a function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

One other way that this can be accomplished with modules is if you place all the functions that you want to call, based on their names, into their own module and you import that file with a namespace.

// list.js
//my functions to call based on name
function sam() {
  console.log('This is sam.');
}
function crowley() {
  console.log('This is crowley.');
}
export { sam, crowley };

1
2
3
4
5
6
7
8
9

Above is the file that we will import and search for function names.

import * as LIST from './list.js';

function dean(nameOfFunction) {
  if (nameOfFunction in LIST && typeof LIST[nameOfFunction] === 'function') {
    //it is something imported in LIST
    //it is a function
    //call the function
    LIST[nameOfFunction]();
  } else {
    console.log('Not a function');
  }
}
dean('sam'); // works
dean('crowley'); // works
dean('rowena'); // fails

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# ToDo This Week

To Do this week

read all the content for modules 2.1 and 2.2
Complete and submit Hybrid exercise 1 - JS Class Quiz.
Review the description for the Cache API Assignment.

← 1.2: Git & Pull Requests 2.2: Cache, Files & Responses →