Data Storage

Cookies, Web Storage API, Cache API

# Client-Side Storage

There are a number of ways that we can store information in the browser (on the client-side). This week we will examine those options.

First we need to remember why we store data on the client-side instead of the server-side.

Client-Side Data	Server-Side Data

Available only from one device	Available to all users potentially
Tied to one domain	Can be used across domains through an API
Can be deleted by the user easily	Requires controlled access to delete
No guarantee that the information is kept. The browser is allowed to delete it.	Server-side databases designed to permanently and redundantly store information.
Not available directly by the server	Controlled by the domain owner

All the different types of client-side storage can be seen in Chrome via the Developer Tools Application tab.

# Starter Code for Class

Start with this code (opens new window) for the work in class today.

Follow the instructions in the README.md file.

# Cookies

Cookies were the first form of storage that was available in the browser. They are just a single string that is saved in the browser, in a storage area that is tied to a specific domain. Originally cookie strings were only allowed to be 255 characters long. Now the string length is over 2000 characters, which is a long string but still fairly limited.

Cookie Security

Cookies from one domain are NOT allowed to be read by code that came from another domain.

Each browser has its own limitations on storage of cookie data. There is a limit for the total storage of cookies as well as a limit per domain.

The cookie strings are made up of name value pairs, just like values in the query string of a URL. Here is an example of what the data part of a cookie could look like.

let cookieData = 'name=Steve;';

In addition to the data part of the cookie string, there are settings that control different limits on the cookie, such as the max age or expiry date for the cookie. Notice the semi-colon at the end of the data portion of the cookie string. AFter the semi-colon we would add the other settings. Each of the settings are separated by a semi-colon.

let cookieSettings =
  'Path=/;Domain=127.0.0.1;Max-Age=30000;Secure;SameSite;HttpOnly;';
let cookie = cookieData + cookieSettings;

1
2
3

We would combine these two parts together to create a whole cookie string. Alternatively you could just put it all in one string to begin with.

Path

By default, cookies apply to the root folder of your website. However we can add to this path value to restrict the cookies to a smaller part of our website.

Domain

By default, the domain value will be the domain of the HTML file. However, we can restrict it further to a specific subdomain if we want.

Max-Age

The max-age part of the cookie String will be the number of seconds that the cookie is to be considered valid.

Expires

As an alternative to the Expires setting you can provide an expiration date for the cookie. This will be compared to the time and date in the browser, not any server time or date.

Secure

If the secure value exists it means the cookie can only be accessed or set over https.

HttpOnly

The HttpOnly setting means that the setting is only used during HTTP requests to the server. The JavaScript cookie API does not have access to the value when reading a cookie.

SameSite

With the SameSite value set the cookie can be restricted to only to be sent with requests for files that are going to the same domain as the original HTML file.

Cookies on the client side can only be read by pages that come from the same domain as the cookie.

However, the cookie string can be sent from the browser as one of the headers in any request to any domain. To prevent this you should add SameSite; to your settings.

SameSite can be set to the possible values Lax, None or Strict. With the value Strict the cookies will only be sent with requests to the matching domain. Lax is similar but the cookies are only sent when the user is navigating to the domain where the cookie originated. The cookie would not be sent with an API request sent to the cookie's domain. None means that cookies will be sent with all requests except that the Secure setting must have also been set.

If you want to set or update a cookie from the browser you only have to overwrite the whole cookie string using the document.cookie property.

document.cookie =
  'user=123&name=Steve;path=/;domain=localhost;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';

1
2

That one line of code will replace the previously saved cookie with those settings.

Every change to the settings means that you are setting a separate cookie. This snippet is actually trying to set three cookies in the browser. However, since the first and last cookies have the same path, and domain, and default values for the other settings, the third cookie actually overwrites the first one.

document.cookie = 'user=123;Path=/;Domain=127.0.0.1;Max-Age=55000;';
document.cookie =
  'name=Tony;Path=/;Domain=127.0.0.1;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';
document.cookie =
  'user=456;Path=/;Domain=127.0.0.1;Expires=Fri, 14 Jan 2022 07:28:00 GMT;';

1
2
3
4
5

When you want to read the value from the cookies you would put the document.cookie value into a variable, split the string on the ; semi-colons plus a space, and then take each portion.

let cookie = document.cookie;
console.log(cookie); // `user=456; name=Tony` or  `name=Tony; user=456`
let cookies = cookie.split('; ');
//cookies is an array of name=value pairs
cookies.forEach((c) => {
  //use split and array destructuring
  let [name, value] = c.split('=');
  console.log(`The ${name} cookie has a value of ${value}`);
});

1
2
3
4
5
6
7
8
9

All the cookies that have been set for the current Path and Domain will be gathered into a single string that we split apart and then split each of the parts on their = equal signs.

# Web Storage API

The Web Storage API has two parts - localStorage and sessionStorage. They have the same methods and properties and work the exact same way. The only difference is that the sessionStorage values are automatically deleted at the end of the user's session. This means when they close the browser or the tab that contained the current domain as part of the tab's current history.

From this point on I will be referring to localStorage only but everything also applies to sessionStorage. Both are a global property accessible from the window object in the browser. They do not exist in NodeJS.

All the data stored in localStorage gets stored as a JSON string.

This means that only information that is capable of being a JSON string can be saved in localStorage. No functions, no DOM elements, can be stored in localStorage.

We will need to use the built-in JSON methods to convert the data back and forth between a string and an object.

let myObject = {
  id: 123,
  title: 'The Big Bang Theory',
  seasons: 12,
  firstSeasonYear: 2007,
  stillPlaying: false,
};
//convert the object into a JSON string.
let stringData = JSON.stringify(myObject);
//convert the string back into an object
let objectCopy = JSON.parse(stringData);

1
2
3
4
5
6
7
8
9
10
11

If you ever need to make a deep copy of an object, this is one way of doing it.

If the data that you want to convert to JSON is already a just a string, then you should not use the JSON.stringify() method as it will add an extra set of quotation marks.

Remember

It is important to remember that we have no way of editing the strings inside of localStorage. All we can ever do is replace the string with a new one.

Every time we read or write a value from localStorage we need to use a unique key. The key will be the unique name for that chunk of data. All keys need to be String values. An example key that you could use for an assignment would be abcd0001-com.example-userscores. It has a username, a domain name, and a descriptive term for what the data is.

Unique Key Names

Remember to keep your localStorage key names unique. You don't want to overwrite other data being saved in localStorage or have your data overwritten. Use a name that includes a reference to the domain, the web application, and even your own development team, plus the purpose of the data.

# LocalStorage Methods

The methods that we have for localStorage are: getItem, setItem, removeItem, clear, key and the property length.

The length property will tell you how many items have been saved in localStorage for the current domain. You can create as many different keys as you want for your domain. Each key will have its own set of data.

JavaScript files use the Domain of the HTML page that loaded them as the domain context for things like cookies and localStorage.

let myData = []; //my global variable to hold the data, with default value.
let storage = localStorage.getItem('abcd0001-com.example-userscores');
//if there is no key matching this then we will get undefined as the value.
if (storage) {
  //we got some data. Use JSON.parse to turn the string into an object
  myData = JSON.parse(storage);
} else {
  //nothing matched. No data under that key yet.
  //Setting a default value is a good idea.
  localStorage.setItem(
    'abcd0001-com.example-userscores',
    JSON.stringify(myData)
  );
  //don't forget the JSON.stringify() part.
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

To remove a key and its value we would use the removeItem method. It needs the key.

localStorage.removeItem('abcd0001-com.example-userscores');

To remove ALL keys and values for the current domain, use the clear method.

localStorage.clear();

If you think that there may be lots of possible keys for the current domain and you want to search through them, you can use the key method in conjunction with the length property and a loop.

let len = localStorage.length;
for(let i=0; i<len; i++){
  console.log( `${localStorage.key(i)} is the name of the `${i}`th key.` );
}

1
2
3
4

# Single Source of Truth

When working with data sources like localStorage and cookies, which are not directly accessible as a variable, we need to create what is known as a single source of truth.

It is possible to have multiple functions in multiple scripts that are accessing the values in your cookies or localStorage within the context of the same page load. If one script updates localStorage then we might not know that the change was made in another function that is sharing that data.

To avoid problems caused by this the best approach is to create your own globally accessible variable that will hold the data.

The cookie and localStorage values are the CACHED copy of the data. They only exist to give us context when the app is loaded the next time. We do NOT work from them.

When the page first loads, we read the value from the cookie and/or localStorage and put it into the global variable.

Any function that needs to access or change the information will make the change to that global variable.

Each time the global variable is altered you can call on the document.cookie property or localStorage.setItem method to update the cached value. Remember we are only updating the cache for the next time the app loads.

The only time we ever read from the storage is when the app first loads - using the DOMContentLoaded event as our trigger to fetch the cached data.

# Cache API

Added as part of HTML5, the Cache API lets javascript developers intentionally save files, including data files, in the browser to improve performance and reduce latency for repeated future visits.

The Cache API is separate from the built-in browser cache that is controlled by the browser and HTTP Request and Response Headers.

When building a cache, the main reason you would want one is to be able to run your website when the network fails or the user turns off their wifi. We want our app to be able to run offline after the initial load.

While the Cache API is primarily used from Service Workers for Progressive Web Apps, we can also use it directly from any webpage. (We will be talking about Service Workers soon).

It is based on Promises (opens new window), just like the Fetch API.

# Loading the Cache

When you want to save files in the cache to use later.

let cacheName = 'myCache-v1'; //key name for our cache
let assets = ['/img/demo.jpg', '/img/other.jpg', '/img/joanne.png']; //the array of files to save in our cache

caches
  .open(cacheName)
  .then((cache) => {
    let urlString = '/img/1011-800x600.jpg?id=one';
    cache.add(urlString); //add = fetch + put

    let url = new URL('http://127.0.0.1:5500/img/1011-800x600.jpg?id=two');
    cache.add(url);

    let req = new Request('/img/1011-800x600.jpg?id=three');
    cache.add(req);

    //cache.addAll(assets).then() is an alternative that lets you save a list
  })
  .catch((err) => {
    //the open method failed.
  });

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

In this example we are saving three copies of the same file (an image called 1011-800x600.jpg) into our cache. This could have been three different files. This code is just to illustrate that changing the querystring value makes the browser view it as 3 different files.

If you want to have an array of file names you could call the cache.addAll method and pass in the array. The current domain name will be prepended to all those strings when the requests are made, unless another protocol and domain are used at the start of the string.

The cache object also has fetch and put methods. The fetch method will download the file. The put method will save the file in the HTTP Response from the fetch in the cache. The add method does both the fetch and put steps together.

If you ever need to remove something from the cache then you can call the cache.delete() method to remove the file. It takes a Request, URL or USVString, which represent the file, as an argument. It returns a Promise that resolves when the file has been deleted.

# Reading the Cache

When you want to read a file from the cache then we use the cache.match() method. It will return a Response object, just like a fetch call would. The difference is that it is looking at the internal browser cache for your domain under the key used to create the cache object.

//the request argument can be a USVString, a URL object, or a Request object
//the options argument is optional. It is an object with 3 properties
let options = {
  ignoreSearch: true, //ignore the queryString
  ignoreMethod: true, //ignore the method - POST, GET, etc
  ignoreVary: false, //ignore if the response has a VARY header
};
cache
  .match(request, options)
  .then((response) => {
    // Do something with the response object... just like you would with fetch
    // if(! response.ok) throw new Error(response.statusText)
    // response.json()
    // response.blob()
    // response.text()
  })
  .catch((err) => {
    console.warn(err.message);
  });

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

The Cache API is not to be confused with the Cache-Control Header that is part of the HTTP Response specification. The Cache-Control Header (opens new window) is a header that is sent from the server or proxy-server to the client along with a requested file. It tells the browser how to handle the caching of the requested file and how long it should be considered valid before requesting a new copy of the file.

# Cache-Control Header

The name of the header is cache-control and it can hold a variety of directives. Here are a few examples of the header showing some of the directives.

Cache-Control: max-age=604800
Cache-Control: no-cache
Cache-Control: max-age=604800, must-revalidate
Cache-Control: no-store

1
2
3
4

We will talk more about fetch calls, headers, request and response objects in week five.

# What to do this week

TODO

Things to do before next week

Read and Watch all the content from Modules 2.1, 2.2, and 3.1.
Prepare questions to ask in class.
Continue working on Hybrid one
Watch the videos about Cookies, LocalStorage, and the Cache API.

LocalStorage API

Cache API

WebStorage Events

← 1.2: Data Structures 2.2: IndexedDB →