Client side error logging

Everybody uses some analytics service to know how users uses its site. Some programmers even use custom events for this.

Generally it is good idea to know also if your user go out because of js error on your site.

Error logging on client

When something happen on development site programmer usually open developer console and look at errors, but if this happen for remote user we cannot ask each to open console and send us errors.

window.onerror - it is js DOM event handler that catch all programming errors. Some examples of errors:

// syntax error
<script>{</script>

// uncaught exception
<script>throw "some message"</script>

(i think anyone can suggest even more)

window.onerror handler has 3 arguments:

window.onerror = function(message, url, line) {
// message - it is some description of this error
// url - it is url of script that cause error
// line - it is line number in script where error happen
}

Simple example at jsfiddle on clicking button we can see in console messages:

So this is all we need to handle all errors. Second thing it is how to report errors.

On client we have a lot of choices where and how to send requests. For example i will use my tracking example from previous post. I need to include loading code in head as early as possible to get errors in other scripts:

<script>
window.onerror = function(message, url, line) {
 __it("error", message, url, line)
}
</script>

Of course i understand that almost everyone prefer to use something more popular and advanced than just analytics from some blog post. Google analytics has special category for exceptions and someone will write something like this:

window.onerror = function(message, url, line) {
  ga('send', 'exception', {
   'exDescription': 'Error "' + message + '" at '+ url + ' on line ' + line
  });
}

As a basic and free solution this is enough. But also need to say that exists a lot of paid services that solve similar problem but with some nice interface and more advanced features (I do not want publish its url, but if someone need i will add).

Jenkins ♡ node.js

With my colleague (from InGo) we are building new small startup which will help (we hope) to people to solve “problem of 1000 tabs opened” (usually to read them later).

Our server are built with node.js express application and for CI we are using Jenkins. This post is about how we make Jenkins understand node.js test tools results.

First I have installed some plugins:

• AnsiColor - this one to have a nicer console output when it is available
• Jenkins Cobertura Plugin - code coverage
• Checkstyle plugin - better one for jshint or Violations plugin
• xUnit plugin or JUnit plugin - test’s results (they uses similar xml files)

How I manage versions of tools

First jenkins pull all sources from repository. Then first build step, make npm install and I have all my dependencies installed (dev too).

In package.json in devDependencies I have all tools that need to run tests, get metrics and so on.

I think it is required to point: with node.js in current state there is no right way, but there is the most convenient way for me in our project at current state. So if I will found something more convenient I will switch to that way.

To do not spend time each time to edit jenkins job I created Makefile and added tasks to it.

Test results with xUnit plugin

For tests we are using mocha and should.js (and supertest for http requests). This is a job to get xUnit compatible xml:

test-jenkins-xunit: @NODE_ENV=test ./node_modules/.bin/mocha \
    --recursive \
    --check-leaks \
    --reporter xunit \
    $(MOCHA_OPTS) 1> results/xunit.xml

In build steps I add shell script step:

make test-jenkins-xunit 

And in post build steps:

JsHint warnings/erros with Checkstyle plugin

After jshint 2.X.X was released behaviour of default reporters was changes. In previous version –show-non-errors makes in output xml (with checkstyle reporter) show not used variables and globals as warnings. Now it uses own reporter. To return old results I fill simple checkstyle reporter with old and new code.

Makefile task:

jshint-jenkins:
    ./node_modules/.bin/jshint --reporter=checkstyle-reporter.js $(JS) 1> results/checkstyle.xml

Now need to run this task at shell step and point Checkstyle plugin to file with results:

And now we see nice and convenient chart in our job:

JsHint warnings/errors with Violations plugin

This section outdated (I prefer Checkstyle plugin) but if you want to use Violations plugin.

First again Makefile task:

test-jenkins-jslint:
    ./node_modules/.bin/jshint $(JS) --jslint-reporter 1> results/jshint.xml || exit 0

And how to use it in jenkins build step:

make test-jenkins-xunit 

I added ‘exit 0’ because it return 1 if anything found and build step will fail.

Exists one problem with Violations plugin and xml that produced by jshint. Because it uses relative path you cannot see in job report where actual problem is. I saw several posts about it and nobody publish solution. After previous build step i have added new one that fix xml:

sed -E "s?<file name=\"(.*)\?\">?<file name=\"$WORKSPACE/\1\">?" results/jshint.xml > results/jshint-fixed.xml

This will replace relative paths in name attribute to absolute that Violations Plugin can handle.

Now after build step:

Code coverage with Cobertura plugin

Makefile task:

test-jenkins-cov:
    @NODE_ENV=test ./node_modules/.bin/istanbul cover --report cobertura --dir
        ./results ./node_modules/.bin/_mocha -- \
        --recursive \
        --check-leaks \
        --reporter $(REPORTER) \
        $(MOCHA_OPTS)

Build step is very simple:

make test-jenkins-cov

Last thing after build step:

Now I press ‘Save’ and run this job to see reports (I hope You don’t forget to add repository to this job). Click on job name:

When I click on any of this charts i can see more detailed report. This one from Violations Plugin that you can see when click on file in report (and you will not see this if you do not fix problem with relational paths).

That is all! I hope this post will help to somebody. I decide to write it because there is no similar post that describe whole process.

Making web tracking

A few weeks ago, I had a task to track if users at a partner site open our pop-up. There are a lot of solutions, but the main idea is simple. When an event happens, we send a request to our server.

Need to say that I just need to send get request and make it as most universal as possible.

One of the best solutions (that I chose) is to use a transparent image (remember how to make transparent png working in ie6, no? but I remember =). It is more than universal - img tag appeared very long time ago!

When someone include our image in markup of page via img tag, browser will load it - that is what i want. So lets take 1px transparent gif (small and good supported) - I did not have such so I googled it and found… several. I took the smallest one - 35 bytes.

Well, yes this does not prevent you to get request you do not want.

I will use Lift framework as we use it at InGo. First I need bytes of this image:

Files.realAllBytes(Paths.get(”./empty.gif”))

And save it somewhere.

val emptyGif: Array[Byte] = Array(71, 73, 70, 56, 57, 97, 1, 0, 1, 0, -128, -1, 0, -1, -1, -1, 0, 0, 0, 44, 0, 0, 0, 0, 1, 0, 1, 0, 0, 2, 2, 68, 1, 0, 59)

Now i need rest endpoint to return my image:

object TrackApi extends RestHelper with Loggable {

    val emptyGifResponse = InMemoryResponse(emptyGif, headers, Nil, 200)

    serve {
        case "track" :: "empty" :: Nil Get req if req.path.suffix == "gif" =>
            emptyGifResponse
    }
}

That is it! Working and powerful web analytics platform … almost.

Basic working case it is

<img alt=”” src=”//example.com/track/empty.gif” />

This is enough to know that someone open some page. From request headers we can get all required information about client: Referer, User-Agent, etc.

I almost forget to say about how to prevent browser from caching our image - minimal set of headers:

val headers = List(
"Expires"       -> "0",
"Cache-Control" -> "no-cache, no-store, must-revalidate",
"Pragma"        -> "no-cache",
"Content-Type"  -> "image/gif"
)

If we want to limit with server-side solution then we can set response header Set-Cookie and identify user each time. But this can be problematic sometimes because, in general, we cannot trust request headers. It was not my case and add small js library to be included in site of our partner.

First we need some good way to load our js library to do not interfere client site. Remember how each big site load its own scripts, create script tags and insert them somewhere in page and load async, let make similar or maybe in the same way:

// i will expose only 2 public objects to host object,
// this is because i want to be sure that client can 
// change it without problems if it is required
// SuperAnalyticsObject - first exposed object. 
// it contains name of tracking function - second exposed object
window.SuperAnalyticsObject = name;
// first need to check if library was not loaded already in some way
// if it is not then create small mock object 
// that collect tracked events before script loaded
window[name] = window[name] || function () {
    (window[name].q = window[name].q || []).push(arguments)
}
// now construct script tag and insert it at the top
script = document.createElement(scriptTag),
firstScript = document.getElementsByTagName(scriptTag)[0];
script.async = 1;
script.src = src;
firstScript.parentNode.insertBefore(script, firstScript);

Now let make it friendly to js minifier (I am using closure usually - it shows the best results) and save variable values for this small script:

(function (window, document, scriptTag, src, name, script, firstScript) {
    window.SuperAnalyticsObject = name;
    window[name] = window[name] || function () {
        (window[name].q = window[name].q || []).push(arguments)
    };
    script = document.createElement(scriptTag),
    firstScript = document.getElementsByTagName(scriptTag)[0];
    script.async = 1;
    script.src = src;
    firstScript.parentNode.insertBefore(script, firstScript);
})(window, document, 'script', '//example.com/public/tracking.js', '__it');

With this code snippet I will load my js script and set name of exposed function.

Now about tracking script.

First I need some utility functions to read and write cookie (I want to know that it is the same user) and generate user id.

Id generation is the most simple function there (just funky random string good enough to be user id):

var generateId = function () {
    return Math.round(Math.random() * new Date().getTime()) + '.' + new Date().getTime();
};

Reading the cookie is also not hard - all cookies available for this page are stored in document.cookie as string name=value;*. Lets parse it:

var getCookieValue = function (name) {
    var res = [],
        cookies = document.cookie.split(";");
        name = RegExp("^\\s*" + name + "=\\s*(.*?)\\s*$");
    for (var i = 0; i < cookies.length; i++) {
        var m = cookies[i].match(name);
        m && res.push(m[1])
    }
    return res;
};

How to set cookie:

var setCookie = function (cookieName, cookieValue, path, domain, expires) {
    var removeWWW = function (a) {
        0 == a.indexOf("www.") && (a = a.substring(4));
        return a.toLowerCase();
    };
    // i remove www. prefix because i want cookie to be available on all subdomains
    domain = domain || removeWWW(document.domain);
    expires = expires || 63072E6; // 2 years in miliseconds by default
    path = path || '/';
    // cookie is restricred in length
    cookieValue &&
    2E3 < cookieValue.length && (cookieValue = cookieValue.substring(0, 2E3));
    cookieName = cookieName + "=" + cookieValue + "; path=" + path + "; ";
    expires && (cookieName += "expires=" + (new Date((new Date).getTime() + expires)).toGMTString() + "; ");
    domain && (cookieName += "domain=" + domain + ";");
    document.cookie = cookieName;
};

Now lets load our empty gif - it core of tracking. Just load gif:

var loadImage = function (query, callback, src) {
    callback = callback || function () {};
    src = (src || baseUrl) + '?' + query;
    // Image it is DOM representation of <img alt="" /> set it height and width
    var img = new Image(1, 1);
    img.src = src;
    // for me it does not matter if it loaded or it is not, i just should try to load
    img.onload = function () {
        img.onload = null;
        img.onerror = null;
        callback();
    };
    img.onerror = function () {
        img.onload = null;
        img.onerror = null;
        callback();
    };
};

That is more than enough to make it working, finally lets describe default query parameters and load events that were generated before script loaded.

And let generate event:

// this function i will expose it can be called with any number of arguments first one is required it is name of event
var track = function () {
    var values = Array.prototype.slice.call(arguments, 0),
    name = values.shift();
    // first required parameter it is user id that we take from cookie
    var q = 'u=' + encodeURIComponent(track.options.id) +
        // second required parameter it is name of event
        '&e=' + encodeURIComponent(name) +
        // third required parameter it is host where event was generated
        '&h=' + encodeURIComponent(location.host) +
        // fourth parameter it is just random number to be suer browser will not try cache request
        '&r=' + (1 * new Date());
        // other parameters it is arguments of event any relevant information that i will want to send
    for (var i = 0; i < values.length; i++) {
        q += '&a=' + encodeURIComponent(values[i]);
    }

    loadImage(q);
};

track.options = {};
// try to read user id
var id = getCookieValue(cookieName)[0];
if (!id) {
    // and create one if it is empty
    id = 'IT-' + generateId();
    setCookie(cookieName, id);

    track.options.init = true
}
track.options.id = id;

// how to expose my function in host object
var name = window.SuperAnalyticsObject || '__it';

// load all already generated events
if(window[name] && window[name].q) {
    var len = window[name].q.length;
    for(var i = 0; i < len; i++) track.apply(undefined, window[name].q[i]);
}
window[name] = track;

That is all, fully functional code to make web tracking. As an improvement, we can record number to tried and request frequency to do not kill our server. There is a lot of things that can be done and improved, but similar code solve my task.

Maybe someone will be usefull to use XHR and CORS to restrict some request with only user browsers. To send POST you will need to create form with inputs and fire submit.

I took most ideas from stackoverflow and from already working analytics scripts.