Caliper.io delivers Backbone.js View monitoring and works simply by setting a Caliper.config.apiKey variable and loading the js file after backbone.js and before your views. You should also name your views for easier identification in the reports:
After you’ve done this and pushed it into production you can just sit tight and watch the data flow into Caliper as your users use your app. After a few interactions you will start to see something:
You see two lines, the first displays a Router.route, hence the signpost! The second is a click event, hence the pointer. The other columns show the number of times this occured, the median time taken and the 90th percentile — what time usage 90% of the users are inside, effectively filtering out the slowest 10% that might not be too representative.
Inspecting a single event
Now lets inspect the media/filmstrip.focus method by clicking on it:
After opening the event report and making a selection in the Response Time Distribution graph, I get a listing of all actual interactions — this is where things start to get interesting!
As you can see, this route has affected 50% of my customers, highlighting to me that its an important part, and I can what the time was spent on — roughly — between AJAX and Rendering and Other. Averages in each of these columns would be good, as I normally dont care about 1 out of 1000 events time taken on Rendering, but I do care once something takes way too much time all across the line.
Lets delve deeper and look at one specific row — I’ll pick the slowest one:
No surprises there! The AJAX group were supposed to take 94.3%, and this just shows that I have a 232ms call here.
Inspect a slow route
Lets look at a route that actually takes too long to finish:
Here you can see what happens when you dont give Views a __name__ property, I suddenly have an UnnamedView spending 190ms rendering! I also get suspicious of the fact that /settings.json is loaded in order to render /album, even though its run async with /tags.json so it probably doesn’t impact rendering time much.
You can also notice that I’ve just selected the one run of this route that was very slow — theres a cluster of them that all finished below some 375ms.
I’ve only just set up and deployed Caliper.io a few days ago, but so far I think this will help my team drive down interaction slowness by identifying slow View methods and Routes on production data. And as time goes by I’m sure the Caliper.io app itself gets more fully functional, and at that I do have some requests / cons:
- I really need to filter everything on domain, origin, browser, and preferably custom variables like user access level etc.
- POST requests should record POST-data up to a certain size, I’m mostly thinking about passed flags that might be crucial in recreating the scenario
- More averages. For a specific route/event I really want to know how much MyView.fooRender took across all users, without having to open every row and manually guestimate in my head.
- How about tracking what happens in the model? Yes this probably results in an AJAX call at some point, but sometimes I just iterate over some stupidly large dataset or whatever — I think showing custom model methods makes sense.
- Events? One hidden performance in event driven applications are misfiring events — double firing, event chains that results in double listeners. I don’t know how this could be best monitored, but if this could be solved that would be awesome!
- Errors. At some point it probably makes sense to also monitor errors in the same app, especially when the monitoring tool is context aware of the
frameworklibrary used and thus its common errors.
In sum; I’m liking the looks of Caliper.io and we will probably end up a customer.