CodeProject
I haven't written anything technical for a while, and that's mainly because the past year I changed jobs a few times. After working at Sears Israel for almost 3 years, I thought it's time to find the next adventure. I think I finally found a good match for me, and I'll probably write a whole post about that soon.
For now, I'll just say that at the new startup I work at, we're doing a lot of work on the ELK stack, and I got to do a lot of work on Kibana. With years of experience on various client side applications, I still learned a lot from looking at kibana's code. I think there are many things here written really elegantly, so I wanted to point them out in a concentrated post on the subject. Also, there are some bad notes, mainly minor things (in my opinion) that I will mention as well.
At First Glance
Kibana 4 is a large AngularJS application. The first thing I noticed when looking at the code is that it has a great structure. Many AngularJS tutorials (or any other tutorials for MVC frameworks) and code-bases I've worked on have the messy structure of a 'models' directory, a 'controllers' directory, and a 'views' (or 'templates') directory.
AngularJS did the right thing by organising the code by features/components, and not by code-framework definitions. This makes it much easier to navigate through the code base, and to easily add more features.
Having a code base organised by controllers, models, views, etc, doesn't do much for your code base except become a pile of unrelated features in each directory, violating the
Separation of Concerns principle.
(In the image you can see each component grouped in it's own directory, which includes it's templates, it's code and it's styles all together)
In addition, most AngularJS applications I've seen have all their routes defined in one file (usually app.js or index.js), which goes along with many global definitions, and sometimes logic related to specific pages or models all in a single file with no relation to any feature.
Kibana's code is nicely organised, and each 'plugin' or 'component' (discover/visualize/dashboard/settings/etc) defines it's own routes in it's own controller.
They manage to do this by creating their own 'RouteManager' (
https://github.com/elastic/kibana/blob/master/src/kibana/utils/routes/index.js). This basically defines the same api as angular's RouteManager, but it collects the routes you define, and in the end calls angular's route manager to actually add them (by calling routes.config here :
https://github.com/elastic/kibana/blob/master/src/kibana/index.js#L41).
This custom route manager also adds the ability to resolve certain things before the route is called, which is real useful in many situations.
Javascript Libraries
The creators of kibana did a great job (with a few minor exceptions that I will explain in the end) in choosing many open source javascript libraries to lean on while building kibana. It's usually a good idea to not reinvent the wheel, especially when someone already did a good job before you.
RequireJS
RequireJS is a javascript module loader. It helps you create modular javascript code, and makes it really easy dealing with dependencies between modules. Kibana's code does a great job utilizing RequireJS by defining most javascript modules in the AMD standard.
A really nice trick they did here that is definitely worth mentioning is the
'Private' service they created. This is a wrapper that allows you to define a RequireJS module, with angularJS dependencies. This allows you to use angular's dependency injection abilities side-by-side with RequireJS' DI abilities.
Regularly loading RequireJS modules in the code looks like this :
define(function(require) {
var myService = require('my_service');
// now do something with myService
});
Using the 'Private' service you load modules like this :
define(function(require) {
var myAngularService = Private(require('my_angular_service'));
// now you can use myAngularService
});
And most important is that my_angular_service looks like this :
define(function(require) {
return function($q, $location, $routeParams) {
// all angular providers in the function parameters are available here!
};
});
The Private service uses angular's get() method to retrieve the $injector provider, and uses it to inject the dependencies we need.
(Take a look at the 'Private' service code here :
https://github.com/elastic/kibana/blob/master/src/kibana/services/private.js)
lodash!
If you're not familiar with lodash, you should be. It's the missing javascript utility library that will definitely help you DRY up your javascript code. It has many "LINQ"-like methods (for those familiar with C#), and many other basic methods you would usually write yourself to help iterate over json objects and arrays in javascript. One of the really nice features about lodash is that most methods you can chain to make your code more readable and lodash uses lazy evaluation so performance is amazing!
I don't want to start writing about the features of lodash, but I strongly suggest reading their
docs, and getting familiar with it.
Almost every service, component or controller in the kibana code starts with this line :
var _ = require('lodash');
They also did a really good job extending lodash with some utility methods of their own. Take a look at these files to see for yourself :
https://github.com/elastic/kibana/blob/master/src/kibana/utils/_mixins_chainable.js
https://github.com/elastic/kibana/blob/master/src/kibana/utils/_mixins_notchainable.js
(There's one thing I don't like here, which is the methods 'get' and 'setValue' - They do a 'deepGet' and 'deepSet' which is like saying "hey, i know i have something here in this object, but have no idea where it is". This just doesn't feel right... :/ )
Some HTML5
Throughout the code there has been some good use of html5 features.
The first one I noticed and really liked is the
'Notifier' service (
https://github.com/elastic/kibana/blob/master/src/kibana/components/notify/_notifier.js). I really like the abstraction here over notifying the user of different message types, and the abstraction over the browser's 'console' methods. The
'lifecycle' method (
https://github.com/elastic/kibana/blob/master/src/kibana/components/notify/_notifier.js#L139) is really neat, and uses the
console.group() method to group messages in the browser's console. It also uses
'window.performance.now' which is really nice, and much better than using the older
'Date.now()' method (it's more exact, and it's relative to the navigationStart metric).
Kibana also makes use of the less-common <wbr/> tag. This is new to html5 and is intended to give you a little more control over where the line breaks when text overflows in it's container.
There's also use of 'localStorage' and 'sessionStorage' for saving many local view settings in the different kibana pages. In general, they did a great job in persisting the user's state on the client side. When navigating between tabs, it keeps you on the last view you were in when returning to the tab.
Another nice thing is that there is a lot of use with
aria-* attributes, and recently I see more and more of this in the newer commits. It's nice to see a big open source project dedicating time to these kinds of details.
Object Oriented Programming
There is a great deal of attention to the design of objects in the code.
First, I like the way inheritance is implemented here. A simple lodash 'mixin' allows for object inheritance.
inherits: function (Sub, Super) {
Sub.prototype = Object.create(Super.prototype, {
constructor: {
value: Sub
},
superConstructor: Sub.Super = Super
});
return Sub;
}
(
https://github.com/elastic/kibana/blob/master/src/kibana/utils/_mixins_chainable.js#L23)
Many objects in the code use this to inherit all the properties of some base object. Here's an example from the 'SearchSource' object :
return function SearchSourceFactory(Promise, Private) {
var _ = require('lodash');
var SourceAbstract = Private(require('components/courier/data_source/_abstract'));
var SearchRequest = Private(require('components/courier/fetch/request/search'));
var SegmentedRequest = Private(require('components/courier/fetch/request/segmented'));
_(SearchSource).inherits(SourceAbstract);
function SearchSource(initialState) {
SearchSource.Super.call(this, initialState);
}
// more SearchSource object methods
}
(
https://github.com/elastic/kibana/blob/master/src/kibana/components/courier/data_source/search_source.js#L9)
You can see the SearchSource object inherits all the base properties from the SourceAbstract object.
In addition, all the methods that would've been static are defined on the object prototype. This is great mainly for memory usage. Putting a method on the object's prototype makes sure there's only one instance of the method in memory.
Memory Usage
Since kibana is a big single-page application, there is a need to be careful with memory usage. Many apps like kibana can be left on in a browser for a long time without any refresh, so it's important to make sure there are no memory leaks. AngularJS makes this easy to implement, but many programmers don't bother going the extra mile for this.
In the kibana code, many directives subscribe to the
'$destroy' event and unbind event handlers not to hold references to unused objects.
An example from a piece of kibana code (the css_truncate directive) :
$scope.$on('$destroy', function () {
$elem.unbind('click');
$elem.unbind('mouseenter');
});
(
https://github.com/elastic/kibana/blob/master/src/kibana/directives/css_truncate.js#L41)
Code Conventions
Kibana's code is mostly very organized, and more importantly readable. A small negative point goes here for some inconsistencies with variable naming. There are classes that have public methods that start with '_' and some don't.
For an example of this, look at the DocSource object. This file has even commented 'Public API' and 'Private API' but the naming convention differences between the two aren't clear.
(
https://github.com/elastic/kibana/blob/master/src/kibana/components/courier/data_source/doc_source.js)
Code Comments
I can say the code has enough comments, but I have no idea how much that actually is, since most of the code is readable without comments, which is an amazing thing. There are great comments in most places that should have them.
Just a funny anecdote is that I was surprised to see comments that actually draw in ascii art the function they describe! Kudos!
/**
* Create an exponential sequence of numbers.
*
* Creates a curve resembling:
*
* ;
* /
* /
* .-'
* _.-"
* _.-'"
* _,.-'"
* _,..-'"
* _,..-'""
* _,..-'""
* ____,..--'""
*
* @param {number} min - the min value to produce
* @param {number} max - the max value to produce
* @param {number} length - the number of values to produce
* @return {number[]} - an array containing the sequence
*/
createEaseIn: _.partialRight(create, function (i, length) {
// generates numbers from 1 to +Infinity
return i * Math.pow(i, 1.1111);
})
(
https://github.com/elastic/kibana/blob/master/src/kibana/utils/sequencer.js#L29)
CSS Styling
Another great success here was using the 'less' format for css files. This allows for small and concise 'less' files, and reuse of css components easily (known as 'mixins'). There has been a great job here done with colors especially - All colors are defined in a single file (
https://github.com/elastic/kibana/blob/master/src/kibana/styles/theme/_variables.less). Editing this file, you can easily create your own color scheme.
(There are a few exceptions - mainly a few colors defined in js files or css files, but It's 99% covered in _variables.less).
Build Process
Kibana has a grunt build process setup. It compiles the css files, combines them and js files (without minifying, using r.js), adds parameters to the resource files for cache-busting, and some more small tasks.
I would be happy to see this upgraded to using gulp, which is stream based and has a much nicer api (in my opinion), but grunt still does the job.
Performance
After writing so many good points about kibana's source code, this is where I lack good feedback.
Maybe it's because when building kibana they had in mind that it's not to be served over the internet, and it's just an internal tool, and maybe it's just because I'm overly sensitive after working for quite a while on the performance team at Sears Israel (working on ShopYourWay.com). Either way, if it was an online website, it's performance would be considered under-par.
JS files aren't minified. They are combined, but not minified. Unfortunately, the code isn't even prepared to just minify the files. In order to do this, angularjs dependencies need to be defined with the dependencies declared as strings before the function itself. Otherwise angularjs's dependency injection mechanism won't work.
CSS files aren't minified either, just combined.
JS files are ~5MB !!! Yes, almost 5MB!! That's huge, and it's all downloaded on kibana's initial load. This could've been done in a few separate files, downloading only the ones needed for the initial view first. This would already be a great improvement.
Though there are advantages to not minifying the js, and I think that's what the creators had in mind - It's easier to debug with DevTools (no need for mapping files), and although initial load will take a long time, after that there is no wait on any other pages. If the resources are cached on your machine, then even getting back to kibana the second time should be really fast.
There are also some libraries in the source code which I think are redundant and maybe could've been removed with a little extra work. One example is jquery, which I think is frowned upon using with angularjs. AngularJS comes with jqlite, which is a smaller version of jquery and should suffice.
I hope it doesn't sound like I think they did a bad job - I'm pointing out some areas in the code that maybe could've been done differently. All in all the app is amazing, and works great! :)
In conclusion
I had a great time learning and working (and still working) on kibana's code. I tried to show a lot of good things I like about the code, and point out a few minor bad things in the code. I hope you enjoyed reading this, and Kudos to you if you got to this point! :)
I also hope to write another post about how kibana communicates with elasticsearch and maybe another one on how it renders the visualizations with the help of D3.js
