PWA

4.1 Service Workers

Module Still Under Development

# Fetch API And Service Workers

By far, the most important thing you will do with a service worker is to make it work as a proxy server. A proxy server is a server that sits between your browser's webpage and the actual webserver that is sending the files. It has the ability to cache files for later use, manage when new versions of the file are needed, cache data in localStorage or an indexedDB, and basically provide an offline experience for the website.

The first step is to add an event listener for the fetch event. This event is triggered EVERY time the browser sends a request for a file. Inside the event will be a Request object, which is the Request being sent from the browser. Now, our service worker gets to decide how to handle that request.

self.addEventListener('fetch', (ev) => {
  //this function runs on every request from the browser.
  console.log(ev.request);
  //the event object contains a request property that is a Request Object.
});

1
2
3
4
5

MDN reference for Request Objects (opens new window)

MDN reference for Response Objects (opens new window)

The Request object will contain a url property with the full URL of the Request, a method property that tells us if it was a GET, POST, PUT, PATCH, DELETE, HEAD, or OPTIONS request, plus a mode property that tells us if the request was cors, no-cors, same-origin, or navigate. The navigate mode can be an important distinction. The user might be trying to leave the website or navigate to a different page within our site. The other modes have to do with downloading files that will be used within our page.

There is also a headers property that contains important meta information about the request. These might be needed to make requests to the actual Server. The headers can include things like the content-type of the file being requested.

Once we have the request we have many options.

Search the cache and return the file found there.
Search the cache and if no matching file, do a fetch request for the file.
Only forward the request on to the actual server and return a file from the server.
Search the cache, return a file from there, and also forward the request to the server to get a new version of the requested file.
Check if the user is online or offline and return different files based on the result.

These options are commonly referred to as caching strategies.

To send anything back to the browser we would use the respondWith() method that can be found on the fetch event. The respondWith() method needs to be passed a Response object or a Promise that resolves to a Response object.

Here are a few examples demonstrating this.

ev.respondWith(resp); // a response object you build

ev.respondWith(caches.match(requestObj)); //a match from a cache

ev.respondWith(fetch(url)); //a response from a server

ev.respondWith(
  new Promise((resolve, reject) => {
    resolve(new Response(/* build a response object */));
  })
);

ev.respondWith(async () => {
  //an async function which returns a Promise
  //just make sure that the function returns a Response Object
  //to be wrapped by the Promise.
});

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

You can also wrap the ev.respondWith calls inside of switch or if statements. The self.addEventListener('fetch', (ev){}) can have a variety of respondWith calls. Only one of them will actually execute.

Here is a simple example showing a couple options.

Only the first ev.respondWith() call will actually run.

self.addEventListener('fetch', (ev) => {
  //1. just forward the request to the actual server and back to the browser
  ev.respondWith(fetch(ev.request));

  //2. check the cache
  // return the file from the cache if it exists
  // if not, do a fetch, add to the cache and return to browser the fetched file
  // remember to handle a failed fetch to the server
  // pay attention to the nested `return` statements
  ev.respondWith(
    caches.match(ev.request).then((cacheResponse) => {
      return (
        cacheResponse ||
        fetch(ev.request)
          .then((fetchResponse) => {
            //if the fetch fails...
            if (!fetchResponse.ok) {
              //return any failed fetch to the browser if you want
              return fetchResponse;
              //or throw an error and handle it in custom ways
              throw new Error(fetchResponse.statusText);
            }
            //we have a file from the server
            return caches.open('mycache').then((cache) => {
              cache.put(ev.request, fetchResponse.clone());
              //we have to make a copy of the response to put it in the cache
              //then return the file to the browser
              return fetchResponse;
            });
          })
          .catch((err) => {
            //had an error fetching the file
            //what do you want to return to the browser request???

            //a custom response
            let json = JSON.stringify({ id: 123, name: 'bob' });
            let file = new File([json], 'data.json', { type: 'application/json' });
            return new Response(file, { status: 200, statusText: 'Ok', headers });

            //a 404 error file from the cache
            return caches.match('/404.html');
          })
      );
    })
  ); //end of ev.respondWith()
});

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# Headers of Requests and Responses

If you are using the ev.request.headers object to figure out what kind of request you are dealing with, then you can use its get() method to read any header.

ev.request.headers.get('content-type');
//would return the value of the `content-type` header in the request object.
cacheMatch.headers.get('content-type');
//would return the value of the Response object that was returned from the Cache.
fetch(url).then((response) => {
  response.headers.get('content-type');
  //would return the value of the Response object that was returned by the fetch
});

1
2
3
4
5
6
7
8

Not all servers will return a Content-Type header. Sometimes this can be a permission issue or a CORS issue. At other times you might have an error that prevents access to the content because the Response object has a status code of 0. Both these issues can be resolved by adding a couple options to the fetch() method.

fetch(url, {
  mode: 'cors',
  credentials: 'omit',
}).then((response) => {
  //this request is attempting to make a request to a different origin
  //it is not sending cookies or other credential information along with the HTTP request
  //this will generally fix errors when you are getting an http status of zero.
});

1
2
3
4
5
6
7
8

# Online and Offline in the Service Worker

When you need to check if the browser is currently offline, you can do it in the Service Worker the same way you would in your main script.

//assume that we are online
//a failed fetch will tell us that we were unable to reach the server...aka offline
let online = 'isOnline' in navigator ? navigator.isOnline : true;

1
2
3

With the navigator.isOnline property we can tell if we are completely offline. It is also possible that the user is online but has a terrible connection. In these cases, a fetch() call will fail and trigger the catch() method. A fetch() call can timeout if it takes too long over the terrible connection.

If we are using the NetworkError inside the first fetch(url).then() method then the Error that happens when the fetch is unable to reach the server will just be a standard Error object. This is a good way to differentiate.

The HTTP status code of zero or the missing content-type header was still the result of making a fetch call that actually communicated with a server.

# Nested Promises and returns

Inside the ev.respondWith() method we need to return a Promise object that contains a Response object. Sometimes this can lead to a series of nested return statements.

ev.respondWith(
  caches
    .match(ev.request)
    .then((cacheResponse) => {
      if (cacheResponse) {
        return cacheResponse;
      } else {
        fetch(ev.request)
          .then((response) => {
            let url = new URL(ev.request.url);
            let type = resp.headers.get('content-type');
            let path = url.pathname;
            if (!response.ok) throw new NetworkError(`Failed to fetch ${path}`, response);
            //at this point you might want to save the fetched response in the cache
            //you need to return the call to caches.open()
            return caches.open(cacheName).then((cache) => {
              //create a copy of the fetched response to save in the cache
              cache.put(ev.request, resp.clone());
              //then return the response
              return response;
            });
          })
          .catch((err) => {
            //handle the failure of the fetch
            //check for Error vs NetworkError
            //we should still return a Response object
            //maybe an alternative item from the cache...
          });
      }
    })
    .catch((err) => {
      //handle the error for the caches.match()
      //we should still return a Response object
    })
);

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

# Cache put() vs add()

A cache object has a put(request, response) and a add(request) method. Both will save a file Response to the Cache. The difference between them is that the put method ONLY saves a Response to the Cache using the Request as the key. The add method will make an HTTP request to the server to get the file and then save the Response in the Cache. If you already have a Response object then use the put method.

# Caches and Service Workers

We talked about the Cache API back in week 3.

Service Workers can access caches the same way that a regular script would. Service Workers typically use caches to save files that will be used to cache data files fetched from the server as well as saving files that can be used if the web app is offline and can't access the server.

With a Service Worker we have the install, activate, and fetch events. We do different things with the Cache from inside each of these events.

install event - This is the time to add an array of files to the cache that will be needed if offline.

activate event - This is the time to delete older versions of the cache that are no longer needed.

fetch event - This is the time to add a single file to the cache, if there is an updated version of the file, or if you want to save a new file that was not previously in the cache.

For any of these to work we need to first get a reference to the cache with caches.open(cacheName). This returns a Promise that contains the reference to the cache with the specified name.

The install and activate events will use the ev.waitUntil() method to wait for the Promise to be resolved, similar to the await keyword. This way we are able to finish working with the Cache before the function stops running.

In the fetch event we will use the ev.respondWith() method to handle asynchronous fetch calls and send a response back to the browser.

let cacheName = 'aUniqueCacheName';
let cacheList = []; //an array of files to save in the cache

self.addEventListener('install', (ev) => {
  //save the list of files in the cache
  ev.waitUntil(
    caches.open(cacheName).then((cache) => {
      cache
        .addAll(cacheList)
        .then(() => {
          //all the files were saved in the cache
        })
        .catch((err) => {
          //at least one file failed to be written to the cache
        });
    })
  );
});

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

Each of the entries in the Cache can be either a URL or a Response object.

In the activate event we need to get a list of all the names of all the caches. We do this with the keys() event which returns a Promise that contains an array of all the cache names.

self.addEventListener('activate', (ev) => {
  ev.waitUntil(
    caches.keys().then((keys) => {
      return Promise.all(keys.filter((nm) => nm != cacheName).map((nm) => caches.delete(nm)));
    })
  );
});

1
2
3
4
5
6
7

In the fetch listener function we are handling files, one at a time. For each file you will have to make decisions by looking at the file name, looking at the headers, checking the the current online|offline status and more.

The fetch event object has a request property. This is a Request object like any other. It has a url property and a headers property.

Inside the listener function we will check in the cache for files by passing a Request object or a url string to a caches.match() method. We can use caches because we are going to search through all current cache objects.

caches.match(ev.request).then((cacheMatch) => {
  //the cacheMatch variable is a Response object that came out of the cache
  //it matched whatever Request object was in the event object.
});

1
2
3
4

If there is no match in the Cache for ev.request then the cacheMatch object will contain null. You can use this to decide if you want to return a match from the cache or if you need to do a fetch() call.

The video below is an examination of how the Cache API can be used in connection with Service Workers.

This second video is how to use the caches from inside a Service Worker to keep things in sync with the Service Worker Life Cycle.

# IndexedDB and Service Workers

If you want to learn about using a local document database in the browser, there is a built-in one called IndexedDB.

IndexedDB is a JSON-like database that you create in your browser and can use to store local information, similar to what you do with LocalStorage. The difference between IndexedDB and LocalStorage is that IndexedDB can hold more data and your script can search through the data and modify it. With IndexedDB, you don't have to download all the data and parse it before searching through the data for values or to modify or delete parts of the data.

Here is a full playlist of videos about how to use IndexedDB (opens new window)

This is the first video in that playlist:

# Caching Strategies for Service Workers

There are a number of basic strategies that are fairly standard for any website. You can also create your own more specific ones for any app that you build.

Regardless of which strategy that you like to use, there will never be a single strategy that you use for every request on any website.

You can create a single function for each strategy for easy reuse.

Then, inside the fetch event listener function you will gather a bunch of values like the request file name, request file type, whether the browser is online, etc. Using those values, you create a series of nested if statements and call the appropriate cache strategy function for each condition.

# Cache Only

This strategy is used when you only want to send a response from a Cache, no network requests involved.

function cacheOnly(ev) {
  //only the response from the cache
  return caches.match(ev.request);
}

1
2
3
4

# Cache First

This strategy is used when you want to check the cache first and then only make a network request if the cache request fails.

function cacheFirst(ev) {
  //try cache then fetch
  return caches.match(ev.request).then((cacheResponse) => {
    return cacheResponse || fetch(ev.request);
  });
}

1
2
3
4
5
6

# Network Only

This strategy is used when you only want to make fetch requests and you want to ignore the cache entirely.

function networkOnly(ev) {
  //only the result of a fetch
  return fetch(ev.request);
}

1
2
3
4

# Network First

This strategy is used when you want to make fetch requests but fall back on the cache if that fails.

function networkFirst(ev) {
  //try fetch then cache
  return fetch(ev.request).then((response) => {
    if (response.status > 0 && !response.ok) return caches.match(ev.request);
    return response;
  });
}

1
2
3
4
5
6
7

# Stale While Revalidate

This strategy involves using both the cache and the network. The user is sent the copy in the cache if it exists. Then, regardless of whether or not the request exists in the cache, a fetch request is made. The new fetch response is saved in the cache to be ready for the next request.

function staleWhileRevalidate(ev) {
  //return cache then fetch and save latest fetch
  return caches.match(ev.request).then((cacheResponse) => {
    let fetchResponse = fetch(ev.request).then((response) => {
      return caches.open(cacheName).then((cache) => {
        cache.put(ev.request, response.clone());
        return response;
      });
    });
    return cacheResponse || fetchResponse;
  });
}

1
2
3
4
5
6
7
8
9
10
11
12

# Network First And Revalidate

This strategy is similar to the Stale While Revalidate strategy except it prioritizes the fetch request over the current value in the cache.

function networkFirstAndRevalidate(ev) {
  //attempt fetch and cache result too
  return fetch(ev.request).then((response) => {
    if (response.status > 0 && !response.ok) return caches.match(ev.request);
    //accept opaque responses with status code 0
    //still save a copy
    return cache.put(ev.request, response.clone()).then(() => {
      return response;
    });
  });
}

1
2
3
4
5
6
7
8
9
10
11

# Request Object Properties

When you want to select a different strategy for each file you need to have information about each Request in order to make those decisions.

The ev.request object that we get from the Fetch Event contains lots of information that we can use for decision making.

let mode = ev.request.mode; // navigate, cors, no-cors
let method = ev.request.method; //get the HTTP method
let url = new URL(ev.request.url); //turn the url string into a URL object
let queryString = new URLSearchParams(url.search); //turn query string into an Object
let isOnline = navigator.onLine; //determine if the browser is currently offline
let isImage =
  url.pathname.includes('.png') ||
  url.pathname.includes('.jpg') ||
  url.pathname.includes('.svg') ||
  url.pathname.includes('.gif') ||
  url.pathname.includes('.webp') ||
  url.pathname.includes('.jpeg') ||
  url.hostname.includes('some.external.image.site'); //check file extension or location
let selfLocation = new URL(self.location);
//determine if the requested file is from the same origin as your website
let isRemote = selfLocation.origin !== url.origin;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Online and Offline

When you want to check if the browser is online or offline, the navigator.onLine property tells you absolutely if you are offline. The online value being true can be deceptive. So, it would be better to also do a test. Pick an endpoint that you know always works efficiently and do a fetch call with the HEAD method. If that returns successfully, then you are definitely online.

async function isConnected() {
  //can only be called from INSIDE an ev.respondWith()
  const maxWait = 2000; //if it takes more than x milliseconds
  if (!navigator.onLine) return false; //immediate response if offline
  //exit if already known to be offline
  let req = new Request('https://jsonplaceholder.typicode.com/users/1', {
    method: 'HEAD',
  });
  let t1 = performance.now();
  return await fetch(req)
    .then((response) => {
      let t2 = performance.now();
      let timeDiff = t2 - t1;
      // console.log({ timeDiff });
      if (!response.ok || timeDiff > maxWait) throw new Error('offline');
      return true; //true if navigator online and the HEAD request worked
    })
    .catch((err) => {
      return false; //the fetch failed or the response was not valid
    });
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

You can also add a timer to that response. If you don't get a HEAD response quickly then you have a poor connection.

REMEMBER to add a URL that you trust to this Request.

You might be able to save a few milliseconds by adding a preconnect in your HTML head with the same domain.

<link rel="preconnect" href="https://jsonplaceholder.typicode.com" crossorigin />

You can optionally restrict the maxWait value to a lower number of milliseconds. Exceeding this value when calculating the difference between two performance.now() calls means that you can also treat slow connections as offline too.

performance.now() gives you an accurate number of milliseconds since the service worker started running. Call this before and after your HEAD fetch and it will tell you how long it took.

If you want to use this method, remember that it can only be called from INSIDE a ev.respondWith( ). You need to make it the first step in a chain of .then( ).then( ) calls. Remember to return at each level of nesting.

In the sample version of sw.js this is being demonstrated inside the networkFirstCred function.

# Fetch Event Handling

Inside the fetch event listener function you will likely have many if statements, switch case statements, nested if statements, and logical short-circuiting.

Treat the respondWith() method calls like function return statements. The first one that is encountered will send back the Response. However, the code in your function will continue to run after it is called. So, always have an else{ } block. Don't make two respondWith calls in the same block.

self.addEventListener('fetch', (ev) => {
  let isOnline = navigator.onLine;
  if (isOnline) {
    ev.respondWith(fetchOnly(ev));
  } else {
    ev.respondWith(cacheOnly(ev));
  }
});

1
2
3
4
5
6
7
8

# Response Objects

There will also be times where you want to look at the Response object. You might want to check its headers and return different things depending on those values.

fetch(ev.request).then((response) => {
  let hasType = response.headers.has('content-type');
  let type = response.headers.get('content-type');
  let size = response.headers.get('content-length');
  console.log(type, size);
});

1
2
3
4
5
6

MDN Headers Object reference (opens new window)

# Opaque Responses

Sometimes, when you are making fetch calls you will get an Opaque response. This is a valid response but one that cannot be read by JavaScript. You cannot use JavaScript to get the values of the headers or call .text() or .json() or .blob() on. This happens when you are making a cross-origin request. Eg: trying to get an image file from a website that is not your own.

The way to avoid some of the security issues is to stop credentials and identifying information from being passed to the external web server. We do this by setting the credentials setting on the request to omit. This will prevent the information being sent and solve some of the errors that you see with external requests.

self.addEventListener('fetch', (ev) => {
  //ev.request is the request coming from the web page
  //we can change its settings when making a fetch( ) call
  ev.respondWith(
    fetch(ev.request, {
      credentials: 'omit',
    })
  );
});

1
2
3
4
5
6
7
8
9

There are times when an Opaque response is not a problem. If the Request has a destination property set to image then we have no issue. The HTML is allowed to load and use items like images or fonts when they are Opaque. It is the JavaScript that is not allowed to use Opaque responses.

To close this potential loop hole, You are not allowed to set the value of the destination header from JavaScript. Only the browser can set it. The destination property of the Request may be audio, audioworklet, document, embed, font, frame, iframe, image, manifest, object, paintworklet, report, script, sharedworker, style, track, video, worker or xslt strings, or the empty string, which is the default value.

If the destination is blank then the request is coming from a fetch() call in your script, Cache request, beacons, WebSockets, or a few other things.

The blank response or a response whose destination is script or *-worklet or *-worker will have tighter CORS restrictions.

You can also change SOME of the other settings like Headers in the Request before it is sent to the server.

# ToDo This Week

To Do this week

read all the content for modules 4.1 and 4.2
Read through the details of the Service Worker assignment.

← 3.2: Intro to Workers 4.2: Messaging & Screen APIs →