The beauty of HarpJS and to easily serve a static website

Update: though the below it still valid, it seems is starting to end their services and they closed my subscription already. This doesn’t affect the HarpJS server of course, though, this means you’ll need another platform where you can install NodeJS and extend it with Harp in order to get your site running.

Sometimes, when you are asked to build a website, you get the following set of requirements:

  • They need a reasonable amount of pages, not a one-pager or the home/about/contact website, but more up to 10, 20 or 30 different pages
  • Different pages means they really want every page a little different in layout and text, pictures,… so no page is equal to the other
  • Some elements are however recurring in every page, e.g. footer and header, and some elements are recurring in half of the pages, e.g. a side bar, which you do not show in every page but only when there is some room left to fill
  • The content and pages will seldomly change, and if so, they will contact you to do it
  • There are one or two very specific requests you will have to implement in some pieces of client side javascaript, and you know that in the future the customer can ask any question to change a certain page in a certain way (that might not even make sense to you)

Since I’ve had this kind of request a couple of times, I’ve spent a reasonable amount on thinking and experimenting how this is best handled. I considered a series of options:

  • just HTML, CSS and Javascript: Though this is doable, any reasonable developer will soon encounter the moral and enthical problem that he is sometimes copying and pasting code, which are bad practices. Since you will be the one who has to handle the changes every two months, it will be you who will forget where he copied what and thus get an inconsistent update. There is something as JQuery widgets, but this is more for “repeating” particular parts on a single page. And even though the templates can be separated from the page, it is still not a beautiful solution to insert a part of javascript to attach the widget on a particular location. Plus, you can see the page parts being loaded after the page as loaded.
  • a CMS system: CMS systems are good if you have the same type of content that must be published regularly by many different people. For example, the same type of articles on a blog, the same type of objects you sell in an online shop. Usually the layout is then determined by some sort of theme that defines how each of these objects must be rendered. In my situation, where every page is different, in style and content, it would mean that I have a content type for each page, and, must make a theme page for each content type, of which I will have only item. Apart from the setup of the CMS, this means I still have to make something per page, which seems a bit too much work.
  • Website builders, Squarespace, Webix: Though these services really make it easy to build a responsive website, and I tried most of them, I always run in trouble. Either, their templates cannot be adjusted enough, or the editors do not give you enough freedom to put the last pixel on the right spot. Custom javascripts are also not always that easy to add, and if the customer makes a request for a little change, you’ll have to see whether the tool allows it, not whether it is technically possible. It’s frustrating and hard to explain you cannot do something your customer has seen on another webpage… it affects your credibility (did you choose the wrong platform, or ar you incapable, because the others can).
  • Custom server side code: Use of an MVC framework such as ASP.NET MVC, or SailsJS for this seemed like a reasonable overkill to me. It costs reasonable amount of time for something that you as a developer can use to achieve this, but will cost time and money, and doesn’t add any value to the customer.
  • Client MVC framework: I’m a big fan of AngularJS, but, for this kind of requests, it is a little over the top. Setting up routers, controllers and directives is a little more than I want to do for such a simple task.

So at a certain point in my research I ran into harpjs, “the static web server with built-in preprocessing“, which sounded nice and turned out to be a perfect solution for my problem. Actually, I think they made it because they had the exact same problem. A simple solution for a simple problem. HarpJS is based on NodeJS, though when you are using it, you don’t notice this. But it’s good to know it runs on NodeJS, at least you are sure it is based on a proper engine. HarpJS gives you that HTML web server with an extra twitch on top of it.

The advantages of HarpJS are:

  • You can use partials, so you can split of certain portions of your HTML code in separate files and then insert them very easily in the other pages using simple tags. The Harp server will make sure they are inserted and all is sent to the client as one HTML page. The client doesn’t see where the different parts come from and the page is renderd as one.
  • You can use SASS or LESS, which is really good to avoid the complex CSS pages you anyway always end up with, and, you don’t have to do the SASS or LESS building yourself
  • You can run the harp server locally and test your solution
  • I started with a responsive template and theme I bought online for 5$, which I split up in partials. Now I have nice responsive pages but I’m still in full control of every pixel in the website. Buying such a template is almost always profitable these days, because starting a layout from scratch will most likely cost you more than 5 or 10$. You can still change the parts you are not 100% happy about if that would be really needed, but in all other cases, it saves you hours of time. Also, if you are more a developer than a designer, you are not good at design anyway.
  • I can add, remove or modify any line of client code or framework that I want.
  • There are some other fun tools and scripting languages you can use an that will get automatically processed, such as Markdown and Coffeescript, but I don’t use them.

This way, I was able to create the initial website pages in a matter of hours, and it will cost me in total two or three days to put everything in the correct place, without writing a single line of server side code. I used SASS for the extra styles I needed, but I don’t have to do the preprocessing manually. I can just focus on writing the code and content that is specific for this customer.

The thing is of course that by using HarpJS, you are actually also using NodeJS, and, consequentually, you need a NodeJS hosting to serve your website to the public. So our simple static website suddenly requires something different than a simple HTML hosting you can find for free or for 1$ a month. That is where comes into play, the hosted HarpJS server, that comes with a nice extra: publishing from your Dropbox account. Apart from having a fair price for hosting, publishing from Dropbox makes it really easy to build and maintain this site. Just do everything locally and test with your local HarpJS server that works on your Dropbox folder, and when done, browse to the website and publish with one click.

As a developer, I know we like to have complex challenges and build websites with reasonable server and client side code. But sometimes the customer needs simple things, and then HarpJS is really a nice solution.



Custom bootstrapping an AngularJS application to load configuration settings

Why do we need custom bootstrapping

If you use AngularJS for a larger application, you sometimes need information before AngularJS itself is started (often called boostrapped). In my scenario, my front-end AngularJS application connects to an API on another server, and there are different servers for different purposes (development, QA, …). To which server I must connect is in a JSON file in the root directory of my front-end application. This way I can use $http to get this “local” file, read the API base URL and then call the API functions. Why this is very handy is not the topic of this article, but it helps to move code and to address different servers solely by chaning  one little file and does not require any code changes.

My application however stores if the user was logged in last time. If so, it will immediatly fetch a number of values from the API  using a locally stored user authentication token. This is done using an AngularJS service.

Due to the async nature of $http you now have a “race condition”.

The reason is simple: the call to the local $http configuration file can or cannot be completed while the AngularJS application started to initialize the user service, which will fetch the initial user values from the API. If the configuration file is loaded, the call to the API will succeed, but else the API URL is not configured, thus it will fail.

A simple but bad solution would be to first get the configuration file and only upon completion of that call initialize the user service. It will work, but you created a situation that will bite you in the … later on:

  • If you have other services with other API calls, they will also have to follow the same pattern and wait until the first get of the configuration is done
  • Other code cannot execute as long as the services are not intialized, or at least has no sense to execute, so you will have to “wait”

Custom bootstrapping an AngularJS application

If everything has to wait until the configuration is loaded, you can just as well wait with bootstrapping AngularJS. The major advantage here is that your AngularJS application is not aware of this first required step and can be developed as if the URL is always availble.

The first step in bootstrapping AngularJS yourself is removing your ng-app attribute you have in your HTML file. It is this attribute that will bootstrap AngularJS and if you don’t remove it, your custom bootstrap will fail and return you an error “you cannot bootstrap the same application twice”.

The second step is loading this configuration file and bootstrapping AngularJS. As such we need to execute a small piece of Javascript code that will be executed as soon as the document is loaded. Even though your AngularJS application is not yet bootstrapped, we can already use the angular libraries to attach some code to the document ready event, and load the configuration using $http. Once that is done, we launch or application.

    function ($http) {
        .success(function (data, status, headers, config) {
            angular.module('myApp').constant('StartupConfig', data);
            angular.bootstrap(document, ['myApp']);
        .error(function (data, status, headers, config) {
            var default_config = { api_url: 'http://localhost:1234/api/' };
            angular.module('myApp').constant('StartupConfig', default_config);
            angular.bootstrap(document, ['myApp']);

I usually put this code next to where I define my AngularJS application and where I put all the other application configuration. After fetching the URL, we will make it available via an AngularJS constant that can then be injected in any service or controller that needs it. After all that is done, the AngularJS application can just start.

If you have more initialisation to do, it might be better to introduce extra functions and work with promises to keep things clean, but in my case this is sufficient. Note that when there is no file found, I implemented it in such a way the application would still start but connect to some default URL. Also this behaviour depends on your requirements.

Create REST based applications using SailsJS, Waterlock, AngularJS, Restangular and Material, An introduction

I’ve done reasonable research on different web stacks and combinations of stacks to find out what would be the most useful for me. I ended up using the SWARM stack, which, as far as I know, is a term nobody uses (yet), but is short for Sails + Waterlock + Angular  + Restangular + Material. In all follow up articles I will show more code on the different topics, but below you can find the complete “why” I choose these technologies.

First of all, useful for me can be different than it is for somebody else, but here are my criteria for “useful”:

  • As platform independent as possible, for both development and running the application in production.
  • Front and back separated by a REST interface. You never know whether you will have to make another front-end one day (e.g. an app for iOS or Android), and REST keeps these options open
  • Front and back should work with a well documented, easy to understand framework so I can add functionality very fast without worrying about the technical details, nor the implementation of too much GUI
  • I don’t want to worry about the DB that is used… because I’m not sure on that part yet
  • Open source, preferably MIT License or alike, and well supported by the community. I don’t want to end up with a component somebody wrote “half” or abandoned long time ago.

For the first item on the list, I’m convinced NodeJS is an ideal candidate for the back-end, since it avoids the need of a compiler, heave duty webserver and alike… it’s pretty straightforward, runs on Windows boxes, Macbook,… just have to share the files. Also, all reasonable cloud providers are able to run this, so we’re safe with this choice.

Continuing on the backend, writing a REST application from scratch in NodeJS is still a reasonable amount of work, and reinventing the wheel has helped no one ever, so a web stack that enables this would come in handy. I started with the Express stack, which is pretty basic but already nice to work with, it still required me to do a lot of work. I looked at Loopback framework, where there is a pretty Arc frontend, but this isn’t completely open source (as in free), and for one reason or another I couldn’t find my way in it, so I didn’t stick to it. After checking out a few alternatives (Koa, Hapi, …), I stumbled upon SailsJS, and I’m a fan ever since! Though SailsJS is actually an MVC framework, it creates a REST API by default for all the model/controllers, and you do not need to use the views at all, but can still profit from all the other goodies it offers: policies, nice structure, clear config files,… all very simple to learn. Equally great and a fantastic advantage is the use of the Waterline ORM, which doesn’t force you in the SQL or NoSQL corner, but supports both of them, and even combinations. In development, it works in local files, so you can postpone  your choice if you’re not sure yet, but still develop. For a REST backend, I could not find any better solution that leaves me with so many options, and is so easy to learn.

Note: Waterline ORM can also be used outside of the SailsJS framework, and I would advise it in any way since it make DB development really easy and relieves developers from the actually database dependent stuff.

One item left for the back-end was user authentication, since SailsJS doesn’t provide this out-of-the-box (for good reasons). If your requirements are front and back-end separated by a REST interface to support other apps in the future, you must be consistent and the only way to go is web tokens. Cookies are not done (anymore, for authentication), and session authentication will bring you into trouble if you ever want to scale up, or work with apps that do not support this. SailsJS is based on express and all express modules are reusable, so by default people tend to move to Passport for this task. Truth be told, I’m not a fan of Passport… just couldn’t find my way in it (as you might have noticed, I rely on “feelings” for my selection of components, so this is pretty subjective I guess). As an alternative I found Waterlock, which is specifically designed for JWT authentication on Sails, and it does the job just fine!

So with the back-end covered, time to go to the front-end. I’m a fan of AngularJS since the day I met it, and if I see the community around it, I’m not the only one. It has a clean (MVC) structure that makes it really easy to make single page applications, and though “single page application” was not really one of my requirements, it makes life simpler. With AngularJS however, I have two issues:

  • Absence of nice HTML controls. The nice controls are not provided, since it is no concern of an MVC framework, but, nobody wants the default HTML controls today
  • Calling REST functions is not easy enough, even if there is something like ngResource available

On the HTML controls part, I started with bootstrap, and even though it is nice to work with, it sometimes doesn’t fit into the AngularJS modus operandi. There is an Angular Bootstrap component, but this wasn’t satisfying either. After that, I encountered Angular Material library:

The Angular Material project is an implementation of Material Design in Angular.js. This project provides a set of reusable, well-tested, and accessible UI components based on the Material Design system….  Material Design is a specification for a unified system of visual, motion, and interaction design that adapts across different devices and different screen sizes.

Nothing more to say on that one. Library works great and delivers nice controls. So only part left is call the REST API from within AngularJS. After looking at some generators and alike, I came across Restangular:

AngularJS service to handle Rest API Restful Resources properly and easily

Restangular really delivers what it promises, works nice in AngularJS and calls SailsJS API without any problems, has a good documentation and is well supported.

And with that all in place, I had my full stack ready that meets all my initial requirements for developing web applications. Note that an extra consequence is that all of the pieces in this architecture are written in Javascript, so there is only one language used. Though it isn’t an explicit requirement for me, it could be of interest for some development teams

Why I like a NodeJS web application on Azure using SQL as a backend

I’ve been doing a lot of studying lately on which technology is best for my new project, and with the abundance of frameworks life doesn’t get any easier. I have a number of requirements which most of us have when creating a new web applications:

– A framework that is easy to use and well supported (in terms of modules, libraries, … whatever makes us not invent the wheel twice)
– Must run in the cloud
– Must run for free in the cloud while I’m developing the thing. I only want to pay as soon as people pay me
– Must be able to develop code easy on any machine

After all my research, I think I found my ideal combination: it is a NodeJS application that delivers a REST API and accompanying AngularJS client pages in combination with SQL on the backend. A number of people reading this will put question marks with this choice of technology, but, there is a good reason why this combination makes sense.

Why NodeJS?

I like NodeJS for two reasons, which are not accidentally on my requirements list:

– It’s javascript, and doesn’t require a compile step while developing. Hence, it is easy to develop NodeJS applications on any system. Macbook, not a problem. Pc at home, with windows 8.1. No problem. Editors in abundance, see my previous article on Visual Studio Code fore example, but for Windows there is also Visual Studio Community edition, one of the better Javascript editors available today.
– There is an extensive list of free modules available via NPM, and up until today, I haven’t found one problem I couldn’t tackle with a module that is freely available. Is seems me from reinventing the wheel, just make up the glue to deliver a nice service

Note: I like C# code as well, don’t get me wrong, and I’ve written web applications in MVC as well. I just like the fact that NodeJS has no compiler requirement, which makes it very easy to develop.

Why Azure?

Most of the people I meet associate Azure development with ASP.NET websites (with our without MVC). That is one option, but NodeJS applications are equally good supported, and, web applications are for free on Azure! You can say about MS what you want, but Azure is a nice piece of work, and the fact you can easily run NodeJS applications on a free of charge web tier is fantastic. Deployment via Git is included and super easy, so what else do you need? I think most people don’t know these options, and often turn to Amazon and OpenShift for these kind of things, as they are more commonly associated with NodeJS.

Though not really applicable for a simple website, but, there is also an Azure module for NodeJS available to manage your Azure objects, so don’t underestimate what NodeJS and Azure have to offer.

Why SQL?

If you see NodeJS, you always see MongoDB or another NoSQL backend. I like MongoDB and NoSQL for what they are made, but, my data is simple and relational, so why should I go for a NoSQL DB? It would just end up in objects referring to each other like a relational schema, I would use Mongoose to enforce all the constraints…. so at the end of the day, SQL does the job here better than NoSQL. I don’t believe following a trend or hype is a good thing if it just for the sake of the trend. If you can get profit out of it, do it, else, stick to what you have and works.

NodeJS has no issue with SQL, for those who are in doubt. There is a great library called Sequelize (check npm repository) that makes it really easy to work with traditional SQL databases.

A limited MySQL DB is also available for free on Azure, so that’s good for kicking off my development. When the app grows, it might not be sufficient anymore, but the customers should pay for the growing storage cost, so no worries about that.

Note: it thought it was possible to get a limited free MS SQL instance for development, but, this seems not to be available anymore. However, the Sequelize component should hide most of the backend for us thus this should be doable.

Note: it is possible to get a MongoDB instance as well on Azure, thought I’m not sure whether that is for free to start with (up until a number of MB’s). If you would need a MongoDB in the cloud, you can always turn to MongoLabs for a free MongoDB instance in the cloud (to start with). I takes only 5 minutes to register and have it working.


For these reasons, this combination is my ideal environment. It might be yours as well for the same or even slightly other reasons. It’s not a trendy combination, but I believe it is a solid one, and that is what counts at the end of the day.