Javascript

Quick review: Backbone.js monitoring with Caliper.io

I’ve used Backbone.js and JavaScript quite heavily the last three years, and in this time its become very clear that there are few monitoring / measuring tools that fits well into the JS stack. Luckily this is gradually changing and today I will take a quick peek into one such tool; Caliper.io.

Caliper.io delivers Backbone.js View monitoring and works simply by setting a Caliper.config.apiKey variable and loading the js file after backbone.js and before your views. You should also name your views for easier identification in the reports:

After you’ve done this and pushed it into production you can just sit tight and watch the data flow into Caliper as your users use your app. After a few interactions you will start to see something:

Screen Shot 2013-06-28 at 7.39.25 AM

You see two lines, the first displays a Router.route, hence the signpost! The second is a click event, hence the pointer. The other columns show the number of times this occured, the median time taken and the 90th percentile — what time usage 90% of the users are inside, effectively filtering out the slowest 10% that might not be too representative.

Inspecting a single event

Now lets inspect the media/filmstrip.focus method by clicking on it:

Single event

After opening the event report and making a selection in the Response Time Distribution graph, I get a listing of all actual interactions — this is where things start to get interesting!

As you can see, this route has affected 50% of my customers, highlighting to me that its an important part, and I can what the time was spent on — roughly — between AJAX and Rendering and Other. Averages in each of these columns would be good, as I normally dont care about 1 out of 1000 events time taken on Rendering, but I do care once something takes way too much time all across the line.

Lets delve deeper and look at one specific row — I’ll pick the slowest one:

Screen Shot 2013-06-28 at 7.50.44 AM

No surprises there! The AJAX group were supposed to take 94.3%, and this just shows that I have a 232ms call here.

Inspect a slow route

Lets look at a route that actually takes too long to finish:

Screen Shot 2013-06-28 at 8.01.41 AM

Here you can see what happens when you dont give Views a __name__ property, I suddenly have an UnnamedView spending 190ms rendering! I also get suspicious of the fact that /settings.json is loaded in order to render /album, even though its run async with /tags.json so it probably doesn’t impact rendering time much.

You can also notice that I’ve just selected the one run of this route that was very slow — theres a cluster of them that all finished below some 375ms.

Conclusion

I’ve only just set up and deployed Caliper.io a few days ago, but so far I think this will help my team drive down interaction slowness by identifying slow View methods and Routes on production data. And as time goes by I’m sure the Caliper.io app itself gets more fully functional, and at that I do have some requests / cons:

  1. I really need to filter everything on domain, origin, browser, and preferably custom variables like  user access level etc.
  2. POST requests should record POST-data up to a certain size, I’m mostly thinking about passed flags that might be crucial in recreating the scenario
  3. More averages. For a specific route/event I really want to know how much MyView.fooRender took across all users, without having to open every row and manually guestimate in my head.
  4. How about tracking what happens in the model? Yes this probably results in an AJAX call at some point, but sometimes I just iterate over some stupidly large dataset or whatever — I think showing custom model methods makes sense.
  5. Events? One hidden performance in event driven applications are misfiring events — double firing, event chains that results in double listeners. I don’t know how this could be best monitored, but if this could be solved that would be awesome!
  6. Errors. At some point it probably makes sense to also monitor errors in the same app, especially when the monitoring tool is context aware of the framework library used and thus its common errors.

In sum; I’m liking the looks of Caliper.io and we will probably end up a customer.

About Raymond Julin

Lead developer at Keyteq Labs, a product business in Bergen, Norway with a reputation for modern user friendly solutions.

2 Comments

  1. Hi Raymond,

    Thanks for the review. We’re glad you’re liking Caliper as much as we do.

    We’re already looking at breaking down information against browser so you’ll see some things coming in this area soon. Mis-firing events always catches us out too so getting something in place for this is exciting to us also.

    As for your other suggestions we have been planning them internally, especially errors. If you ever want to talk directly drop me an email (kalv@caliper.io) – we would be very happy to talk further with you.

    Kalv.

Trackbacks / Pings

Leave a Reply