My blog has moved!

You should be automatically redirected in 6 seconds. If not, visit
http://www.dullroar.com
and update your bookmarks.

Monday, October 4, 2010

Detecting offline status in HTML 5

Update, December 20, 2012 - Devin Rhode has taken the code, made some changes to support jQuery, and published it on github here: https://github.com/devinrhode2/check-online. Check it out, and thanks to Devin for doing the work!

This post details some of the issues I hit while trying to detect "true" offline status in HTML 5, specifically in Safari on the iPad (iOS 3.2.2), and how I worked around them. Hopefully it will help others who are trying to use HTML 5 in offline mode. It assumes you are familiar with the requirements for taking HTML offline.

To me one of the failings of the current HTML 5 spec as written is that it talks about how to take a Web site offline, but notes in the documentation about the navigator.onLine property and the ononline event that "This attribute is inherently unreliable. A computer can be connected to a network without having Internet access." So detecting you're offline can be a bit difficult.

And in fact, while researching I found that only later versions of IE (8?) truly detect when there's no network connectivity, and that other browsers may only return false from navigator.onLine if the user has chosen that browser's "work in offline mode" option, regardless of their actual network connectivity. And such is iPad's Safari, which simply lies - it blithely returns true whether you're online or offline (airplane mode or wireless manually turned off). So, if you're reading someone's answer to a post on "How do I detect if I am offline" and they say to check navigator.onLine and/or implement the ononline/onoffline events, they don't know what they're talking about.

If you are writing a cross-browser app that may try to "phone home" when it has network connectivity, but otherwise interacts with local storage when offline, you will have to roll your own connectivity detection. That's what we're here for. There are quite a few people who suggest making an AJAX callback to your Web site, but few that actually show it, so even though it's relatively simple, I am going to show you how.

Let's review some basic code. First, here is the manifest file, cache.manifest:


CACHE MANIFEST
# v32 <== change this to force reloading cached resources
CACHE:
favicon.ico
images/toolbar32-home.png
images/toolbar32-pages.png
images/vpwebapp-toolbar.png
index.html
scripts/jquery-1.4.2.min.js
scripts/main.js
scripts/offline.js
styles/branding.css
NETWORK:
scripts/ping.js


Basically, we want to download various files and cache them locally, but we do not want to cache the ping.js file (we'll see why later), so we put it under the NETWORK section.

Here is a bare index.html with code for downloading and swapping cache (and to help troubleshoot):


<!DOCTYPE html>
<html manifest="cache.manifest">
<head>
    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="black" />
    <meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0; minimum-scale=1.0; user-scalable=0;" />
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <link rel="stylesheet" href="styles/branding.css" />
    <title>Offline HTML 5</title>
    <script type="text/ecmascript">
        var cache = window.applicationCache;

        cache.addEventListener("cached", function () {
            console.log("All resources for this web app have now been downloaded. You can run this application while not connected to the internet");
        }, false);
        cache.addEventListener("checking", function () {
            console.log("Checking manifest");
        }, false);
        cache.addEventListener("downloading", function () {
            console.log("Starting download of cached files");
        }, false);
        cache.addEventListener("error", function (e) {
            console.log("There was an error in the manifest, downloading cached files or you're offline: " + e);
        }, false);
        cache.addEventListener("noupdate", function () {
            console.log("There was no update needed");
        }, false);
        cache.addEventListener("progress", function () {
            console.log("Downloading cached files");
        }, false);
        cache.addEventListener("updateready", function () {
            cache.swapCache();
            console.log("Updated cache is ready");
            // Even after swapping the cache the currently loaded page won't use it
            // until it is reloaded, so force a reload so it is current.
            window.location.reload(true);
            console.log("Window reloaded");
        }, false);
    </script>
</head>
<body>
    <p id="online_status" class="p4"></p>
    <script src="scripts/jquery-1.4.2.min.js" type="text/ecmascript"></script>
    <script src="scripts/main.js" type="text/ecmascript"></script>
</body>
</html>


The only applicationCache event handler really needed is updateready, although the others come in handy when debugging, especially on a platform like the iPad. Make sure you turn on the console in iPad’s Safari - that's about the only on-device debugging you’re going to get (along with the web server logs - those can be helpful, too).

Before we go on, here are some notes on things I've learned while pursuing this:

  • The files specified in the manifest will be downloaded asynchronously and will not (necessarily) come down in the order specified in the file.
  • If you are offline the applicationCache error event handler will be called. It seems natural that you could use that for offline detection, except that it gets called for other reasons, too, e.g., a malformed manifest file, unavailable resources pointed to by the manifest file (401s, 404s, 5xx), etc. Thus it seemed cleaner to me to break out offline detection into its own test.
  • Note the comment in the updateready handler. If you don't force a page reload as in the code shown, then you will have to refresh the page manually to get an updated offline cache to be used by the browser in the current session.
  • Changing the manifest file to force a re-download and even clearing cache doesn't always really re-download everything, especially on the iPad. A few times I've had to power cycle the iPad to make sure cache is really cleared. This was especially true of the web page and its embedded script itself.
  • Note my prior blog post on a gotcha specific to manifest files and iPad's Safari.
  • Because your browser's cache also gets involved, it can be helpful during development and debugging to turn off caching directives in your Web server. You may want to do this anyway, otherwise it could be hard to push out changes, even by changing the manifest file, since it could be cached by the browser, too - see the debugging section of Dive Into HTML 5's Let's Take This Offline chapter for details. As that section's title says, "Kill me! Kill me now!"

With that out of the way, let's look into a way to determine whether we're really, really offline. There are quite a few resources on the Web about this, but you have to piece them together, so I thought I'd provide it here as simple example code. Note that this code requires jQuery.


$(document).ready(function () {//debugger;
    $(document.body).bind("online", checkNetworkStatus);
    $(document.body).bind("offline", checkNetworkStatus);
    checkNetworkStatus();
});

function checkNetworkStatus() {
    if (navigator.onLine) {
        // Just because the browser says we're online doesn't mean we're online. The browser lies.
        // Check to see if we are really online by making a call for a static JSON resource on
        // the originating Web site. If we can get to it, we're online. If not, assume we're
        // offline.
        $.ajaxSetup({
            async: true,
            cache: false,
            context: $("#status"),
            dataType: "json",
            error: function (req, status, ex) {
                console.log("Error: " + ex);
                // We might not be technically "offline" if the error is not a timeout, but
                // otherwise we're getting some sort of error when we shouldn't, so we're
                // going to treat it as if we're offline.
                // Note: This might not be totally correct if the error is because the
                // manifest is ill-formed.
                showNetworkStatus(false);
            },
            success: function (data, status, req) {
                showNetworkStatus(true);
            },
            timeout: 5000,
            type: "GET",
            url: "scripts/ping.js"
        });
        $.ajax();
    }
    else {
        showNetworkStatus(false);
    }
}

function showNetworkStatus(online) {
    if (online) {
        $("#online_status").html("Online");
    }
    else {
        $("#online_status").html("Offline");
    }

    console.log("Online status: " + online);
}



Now, the first thing to notice about the code is that we hook the new(er) online and offline browser events, along with checking navigator.onLine's status. I only do this because there may be a browser, somewhere, some day, that actually implements all this correctly. Instead, once navigator has lied to us and told us we're online, we're going to double-check anyway by performing a simple AJAX call back to our site to retrieve a given (non-cached, non-offline) file. In our case this is ping.js:


{ "status": "Online" }


Note this is crafted and retrieved as a JSON result, but it could really be anything because we don't use the contents, we simply look for success in pulling it down. The important thing is that it is not cached. Hence you have to make sure it is specified in the NETWORK section of the manifest and that your server sends HTTP headers down telling the browser not to cache the file, as well as telling the AJAX call not to cache it, either (see the cache parameter on the jQuery ajaxSetup call above).

The AJAX request has a timeout set to five seconds (5,000 milliseconds). You can adjust that up or down based on your typical connectivity speed. Right now the code treats any error as being "offline." That may not be literally true - perhaps the site is down. But in that case, if you can't "phone home," it really doesn't matter why. For all practical purposes you are "offline." Some could argue that there would still be the ability to perhaps access another site (using JSONP or a script tag), or send email or whatever, but for my purpose this is Good Enough.

Some final points:

  • The checkNetworkStatus function is called as soon as the document is ready (using jQuery's ready function). It may very well fire before the manifest checking is finished, even if everything's already cached (and in fact, it does so pretty consistently for me). So when looking at console output don't expect the output from showNetworkStatus to be the last lines.
  • If you want to continuously check whether the online status has changed to offline (at least until the online/offline events are fixed sometime in the future) you're going to have to loop based on something like a JavaScript timer. I leave that as a exercise for the reader.
  • Note the commented out debugger statement as the first line of the ready function. When testing outside of a mobile device (which you should always do first to get it working before finding out how iPad's Safari is broken), uncommenting this can be handy if you can't figure out what's going on via the JavaScript console.
  • When doing any debugging for HTML 5 offline capabilities, especially on mobile devices, the console is your friend. So debug like it's 1979!

Well, I hope this simple example helps and saves you some time. If it does or if you have recommendations on how to do it better, feel free to leave a comment!

30 comments:

Nick Boyce said...

Thanks for this post, it's been really useful.

For some reason the ajax function was returning successful even when there was no connectivity, so I had to change the success function like so:

if (data === null) {
showNetworkStatus(false);
} else {
showNetworkStatus(true);
}

It works fine like this, but it's still quite puzzling...

Jim said...

Nick,

That is odd. But thanks for letting me know, in case it hits me (it hasn't so far).

And thanks for the feedback in general. Glad to know it helped somebody - sometimes it feels like just "posting into the ether."

Boy Interrupted said...

Jim,

Very nice. Seems like a (mostly) foolproof way to identify online/offline status. I am working on a JQT mobile web app that requires such a feature. Just what I was looking for!

Thanks!

Pramod

Jim said...

Pramod,

Glad to have helped, and thanks for the comment!

Kspoosh said...

Thanks a lot! This solved my problems too.

Something that looks so easy (navigator.onLine) is bound to be the contrary.

Is there any evolution in how the 'onLine' setting will be implemented in the future, or will this remain a rather odd property?

Kspoosh said...

Thanks a lot! Your solution solved my problems. I initially had the idea to use an ajax call, but it's indeed a good idea to bind this to existing events.

Something that looks so easy and sounds like pure logic (navigator.onLine), is bound to be complicated.

I suppose there will be updates to this spec? Or will it keep confusing first timers?

Jim said...

Kspoosh, glad to have helped!

I have no idea if this is going to get better in the future. I would hope so. If you read the section of the HTML 5 spec I link to that discusses detecting offline status it is more complicated than you would first think. But for me and I would bet 99% of most programmers, the approach I use here is "good enough."

Fish said...

I like this concept a lot. I wanted to use something like this to show/hide a button that shouldn't be visible when offline.

I've tried to implement it but for some reason ping.js is shown as "canceled" in safari's activity window. But all other files downloaded properly. Any idea why. I'm stumped.

Thanks

Jim said...

Fish,

First, thanks for the feedback. I like knowing that people are finding this helpful.

Second, no idea why, but some general suggestions that I mentioned in the post:

1) Look at the HTTP traffic coming back from the Web server. The debugging tools in most modern browsers are helpful, but external tools sometimes have more capabilities. On Windows I like Fiddler. I've used Charles on my Mac (although it's for-pay, but there's a free trial period).

2) Also look at your Web server logs. There can sometimes be a hint of what's going wrong on that side. Is the HTTP response code being sent for the ping.js file a 200?

3) Check your manifest file. Errors in the manifest file WILL cause problems.

4) Clear cache (obsessively - I've even had different behaviors between just clearing cache and clearing cache, exiting the browser, starting the browser and clearing cache again).

I know that's not specific, but I've found debugging the offline behaviors is a bit of an art. I am sure the tooling will get better over time.

Fish said...

Thanks for the suggestions Jim. I'll give them a shot.

I noticed that when .ajax cache was set to false my other json file (main data file) was also cancelled. But when set to true my other was downloaded properly but my ping.js file (I renamed it ping.json) was still cancelled. Very weird.

I've had everything working great offline and online as it should its just this one hurtle to get past. I'll keep testing. Or just leave the button their offline if I have to.

Thx again.

Jim said...

Fish,

Note that you can set the .ajax caching behavior PER CALL, so you should be able to set it one way for your file and another for the ping file and get the behavior you want. Check the jQuery doco for details.

Giorgio said...

Really useful post, although I had to integrate the last function as for some reason it was continuously called in Firefox.
Do you think I should try to transform the code into a jQuery plugin? It seems an outsourceable task.

Jim said...

Giorgio,

I am glad you found it helpful! And yes, I think it would be cool if it were a jQuery plugin. All I'd request is a bit of credit for "the original idea," but I think you should go for it. :)

Rob said...

I'd like to display my offline page if the connection is spotty. To be more specific, if the connection is good enough to send a text (but slow enough such that downloading my webpage would take a long time) I'd like to show my offline page intend of downloading the page from the server. BUT, the user also needs to use this page to send a small text string to the server. Is something like this possible? Thanks for the great article!

Jim said...

Rob, you could do that pretty easily, I would think, with the small text string going up as either a query parameter or POST request variable on the "ping" AJAX call (for example).

Rob said...

Thanks Jim, that makes sense. Is there a way to tell if the connection is "good" or "spotty"? If it's good, I'd want to fetch a lot of data from the server, but if the connection is poor I'd just want to show the cached page and let the user send a little bit of data to the server. Would timing how long a ping response from the server takes be meaningful? Thanks again. I hope my question makes sense :-)

Jim said...

Rob, timing the "ping" could be part of it, especially if the ping response "payload" were something larger than what I show in my example. Depending on your app's requirements, you could even set up a sort of "COMET" approach and continuously ping in the background and keep track on an ongoing basis (which could be good for a mobile app where the person may be in a vehicle or whatever). The obvious issue would be tuning the frequency and payload size to be (a) meaningful, but (b) without a negative impact on performance (or data transfer limits, if it's a mobile app where they may be paying for their bandwidth on a usage basis).

Rob said...

Thanks Jim. I'll try to post back if and when I have a solution in case it's of interest to others.

Jim said...

Thanks, Rob. I look forward to it!

Joeri Sebrechts said...

I think you need to check the response from the request to ping.js, because of the case where the user has connectivity but is getting redirected for every request to a generic html page (to enter credit card details). That is basically an offline scenario, but it can look like online.

Jim said...

Joeri,

In my app that wouldn't happen, but if that is how your app is designed, then it would be a fine thing to check the response from ping.js. Thanks for the comment!

Anonymous said...

Wonderful post! Thank you so much!

Jim Lehmer said...

Anon, thanks! :) Glad it helped.

Devin Rhode said...

Requesting a .js file could be detected as malicous activity, so I used .css. The most obvious choice would be .json but there could be api collisions, and html5 boilerplate's .htaccess filename based cache busting doesn't list .json, presumably for a good reason.

Also, it's smart to verify the contents of the file. I just put 'success'

Anti-virus programs might think this is fishy because it's not valid css, but it's not totally invalid css, and would pass 99.9999% of the time

Your version isn't compatible with the latest jQuery, so I'm going to post this on github everyone can discuss and work on it together, checkout github.com

Devin Rhode said...

Repo is here: https://github.com/devinrhode2/check-online

Compatible with jQuery 1.8, fixes a few issues mentioned above but I could've missed some mentioned.

Jim Lehmer said...

Devin, thanks, esp. for the shout out on Github, although my name is Jim, not Rob. :)

I will update the original post to point to your Github repo as well.

Chris Russell said...

Thanks this was very useful. Chris

Jim Lehmer said...

Chris, glad it helped!

bsy said...

Jim/Devin, Does this approach (and the available code) assume that the check is being made after the page has been loaded at least once?

How can I use this approach for checking online connectivity when the page is a static file on a device (e.g. bootstrap html file in Android which tries to see whether it should refresh with online url or offline url)?

Thanks for any clarity.

Jim Lehmer said...

bsy,

That's an interesting question. I don't know the answer (Devin?). For it to work at all I would guess the path in the HTML file to the manifest file would have to be absolute, as would all the paths to the various cached and networked entries (obviously).

To answer your first point, yes, the way it was written to work was that you do get successful connectivity at least once up front. But your scenario is an interesting one and has me thinking.