The beauty of HarpJS and harp.io to easily serve a static website

Update: though the below it still valid, it seems harp.io is starting to end their services and they closed my subscription already. This doesn’t affect the HarpJS server of course, though, this means you’ll need another platform where you can install NodeJS and extend it with Harp in order to get your site running.

Sometimes, when you are asked to build a website, you get the following set of requirements:

  • They need a reasonable amount of pages, not a one-pager or the home/about/contact website, but more up to 10, 20 or 30 different pages
  • Different pages means they really want every page a little different in layout and text, pictures,… so no page is equal to the other
  • Some elements are however recurring in every page, e.g. footer and header, and some elements are recurring in half of the pages, e.g. a side bar, which you do not show in every page but only when there is some room left to fill
  • The content and pages will seldomly change, and if so, they will contact you to do it
  • There are one or two very specific requests you will have to implement in some pieces of client side javascaript, and you know that in the future the customer can ask any question to change a certain page in a certain way (that might not even make sense to you)

Since I’ve had this kind of request a couple of times, I’ve spent a reasonable amount on thinking and experimenting how this is best handled. I considered a series of options:

  • just HTML, CSS and Javascript: Though this is doable, any reasonable developer will soon encounter the moral and enthical problem that he is sometimes copying and pasting code, which are bad practices. Since you will be the one who has to handle the changes every two months, it will be you who will forget where he copied what and thus get an inconsistent update. There is something as JQuery widgets, but this is more for “repeating” particular parts on a single page. And even though the templates can be separated from the page, it is still not a beautiful solution to insert a part of javascript to attach the widget on a particular location. Plus, you can see the page parts being loaded after the page as loaded.
  • a CMS system: CMS systems are good if you have the same type of content that must be published regularly by many different people. For example, the same type of articles on a blog, the same type of objects you sell in an online shop. Usually the layout is then determined by some sort of theme that defines how each of these objects must be rendered. In my situation, where every page is different, in style and content, it would mean that I have a content type for each page, and, must make a theme page for each content type, of which I will have only item. Apart from the setup of the CMS, this means I still have to make something per page, which seems a bit too much work.
  • Website builders, Squarespace, Webix: Though these services really make it easy to build a responsive website, and I tried most of them, I always run in trouble. Either, their templates cannot be adjusted enough, or the editors do not give you enough freedom to put the last pixel on the right spot. Custom javascripts are also not always that easy to add, and if the customer makes a request for a little change, you’ll have to see whether the tool allows it, not whether it is technically possible. It’s frustrating and hard to explain you cannot do something your customer has seen on another webpage… it affects your credibility (did you choose the wrong platform, or ar you incapable, because the others can).
  • Custom server side code: Use of an MVC framework such as ASP.NET MVC, or SailsJS for this seemed like a reasonable overkill to me. It costs reasonable amount of time for something that you as a developer can use to achieve this, but will cost time and money, and doesn’t add any value to the customer.
  • Client MVC framework: I’m a big fan of AngularJS, but, for this kind of requests, it is a little over the top. Setting up routers, controllers and directives is a little more than I want to do for such a simple task.

So at a certain point in my research I ran into harpjs, “the static web server with built-in preprocessing“, which sounded nice and turned out to be a perfect solution for my problem. Actually, I think they made it because they had the exact same problem. A simple solution for a simple problem. HarpJS is based on NodeJS, though when you are using it, you don’t notice this. But it’s good to know it runs on NodeJS, at least you are sure it is based on a proper engine. HarpJS gives you that HTML web server with an extra twitch on top of it.

The advantages of HarpJS are:

  • You can use partials, so you can split of certain portions of your HTML code in separate files and then insert them very easily in the other pages using simple tags. The Harp server will make sure they are inserted and all is sent to the client as one HTML page. The client doesn’t see where the different parts come from and the page is renderd as one.
  • You can use SASS or LESS, which is really good to avoid the complex CSS pages you anyway always end up with, and, you don’t have to do the SASS or LESS building yourself
  • You can run the harp server locally and test your solution
  • I started with a responsive template and theme I bought online for 5$, which I split up in partials. Now I have nice responsive pages but I’m still in full control of every pixel in the website. Buying such a template is almost always profitable these days, because starting a layout from scratch will most likely cost you more than 5 or 10$. You can still change the parts you are not 100% happy about if that would be really needed, but in all other cases, it saves you hours of time. Also, if you are more a developer than a designer, you are not good at design anyway.
  • I can add, remove or modify any line of client code or framework that I want.
  • There are some other fun tools and scripting languages you can use an that will get automatically processed, such as Markdown and Coffeescript, but I don’t use them.

This way, I was able to create the initial website pages in a matter of hours, and it will cost me in total two or three days to put everything in the correct place, without writing a single line of server side code. I used SASS for the extra styles I needed, but I don’t have to do the preprocessing manually. I can just focus on writing the code and content that is specific for this customer.

The thing is of course that by using HarpJS, you are actually also using NodeJS, and, consequentually, you need a NodeJS hosting to serve your website to the public. So our simple static website suddenly requires something different than a simple HTML hosting you can find for free or for 1$ a month. That is where harp.io comes into play, the hosted HarpJS server, that comes with a nice extra: publishing from your Dropbox account. Apart from having a fair price for hosting, publishing from Dropbox makes it really easy to build and maintain this site. Just do everything locally and test with your local HarpJS server that works on your Dropbox folder, and when done, browse to the Harp.io website and publish with one click.

As a developer, I know we like to have complex challenges and build websites with reasonable server and client side code. But sometimes the customer needs simple things, and then HarpJS is really a nice solution.

 

Advertisements

Create REST based applications using SailsJS, Waterlock, AngularJS, Restangular and Material, An introduction

I’ve done reasonable research on different web stacks and combinations of stacks to find out what would be the most useful for me. I ended up using the SWARM stack, which, as far as I know, is a term nobody uses (yet), but is short for Sails + Waterlock + Angular  + Restangular + Material. In all follow up articles I will show more code on the different topics, but below you can find the complete “why” I choose these technologies.

First of all, useful for me can be different than it is for somebody else, but here are my criteria for “useful”:

  • As platform independent as possible, for both development and running the application in production.
  • Front and back separated by a REST interface. You never know whether you will have to make another front-end one day (e.g. an app for iOS or Android), and REST keeps these options open
  • Front and back should work with a well documented, easy to understand framework so I can add functionality very fast without worrying about the technical details, nor the implementation of too much GUI
  • I don’t want to worry about the DB that is used… because I’m not sure on that part yet
  • Open source, preferably MIT License or alike, and well supported by the community. I don’t want to end up with a component somebody wrote “half” or abandoned long time ago.

For the first item on the list, I’m convinced NodeJS is an ideal candidate for the back-end, since it avoids the need of a compiler, heave duty webserver and alike… it’s pretty straightforward, runs on Windows boxes, Macbook,… just have to share the files. Also, all reasonable cloud providers are able to run this, so we’re safe with this choice.

Continuing on the backend, writing a REST application from scratch in NodeJS is still a reasonable amount of work, and reinventing the wheel has helped no one ever, so a web stack that enables this would come in handy. I started with the Express stack, which is pretty basic but already nice to work with, it still required me to do a lot of work. I looked at Loopback framework, where there is a pretty Arc frontend, but this isn’t completely open source (as in free), and for one reason or another I couldn’t find my way in it, so I didn’t stick to it. After checking out a few alternatives (Koa, Hapi, …), I stumbled upon SailsJS, and I’m a fan ever since! Though SailsJS is actually an MVC framework, it creates a REST API by default for all the model/controllers, and you do not need to use the views at all, but can still profit from all the other goodies it offers: policies, nice structure, clear config files,… all very simple to learn. Equally great and a fantastic advantage is the use of the Waterline ORM, which doesn’t force you in the SQL or NoSQL corner, but supports both of them, and even combinations. In development, it works in local files, so you can postpone  your choice if you’re not sure yet, but still develop. For a REST backend, I could not find any better solution that leaves me with so many options, and is so easy to learn.

Note: Waterline ORM can also be used outside of the SailsJS framework, and I would advise it in any way since it make DB development really easy and relieves developers from the actually database dependent stuff.

One item left for the back-end was user authentication, since SailsJS doesn’t provide this out-of-the-box (for good reasons). If your requirements are front and back-end separated by a REST interface to support other apps in the future, you must be consistent and the only way to go is web tokens. Cookies are not done (anymore, for authentication), and session authentication will bring you into trouble if you ever want to scale up, or work with apps that do not support this. SailsJS is based on express and all express modules are reusable, so by default people tend to move to Passport for this task. Truth be told, I’m not a fan of Passport… just couldn’t find my way in it (as you might have noticed, I rely on “feelings” for my selection of components, so this is pretty subjective I guess). As an alternative I found Waterlock, which is specifically designed for JWT authentication on Sails, and it does the job just fine!

So with the back-end covered, time to go to the front-end. I’m a fan of AngularJS since the day I met it, and if I see the community around it, I’m not the only one. It has a clean (MVC) structure that makes it really easy to make single page applications, and though “single page application” was not really one of my requirements, it makes life simpler. With AngularJS however, I have two issues:

  • Absence of nice HTML controls. The nice controls are not provided, since it is no concern of an MVC framework, but, nobody wants the default HTML controls today
  • Calling REST functions is not easy enough, even if there is something like ngResource available

On the HTML controls part, I started with bootstrap, and even though it is nice to work with, it sometimes doesn’t fit into the AngularJS modus operandi. There is an Angular Bootstrap component, but this wasn’t satisfying either. After that, I encountered Angular Material library:

The Angular Material project is an implementation of Material Design in Angular.js. This project provides a set of reusable, well-tested, and accessible UI components based on the Material Design system….  Material Design is a specification for a unified system of visual, motion, and interaction design that adapts across different devices and different screen sizes.

Nothing more to say on that one. Library works great and delivers nice controls. So only part left is call the REST API from within AngularJS. After looking at some generators and alike, I came across Restangular:

AngularJS service to handle Rest API Restful Resources properly and easily

Restangular really delivers what it promises, works nice in AngularJS and calls SailsJS API without any problems, has a good documentation and is well supported.

And with that all in place, I had my full stack ready that meets all my initial requirements for developing web applications. Note that an extra consequence is that all of the pieces in this architecture are written in Javascript, so there is only one language used. Though it isn’t an explicit requirement for me, it could be of interest for some development teams

Installing IISNode 0.2.2 on Windows 2012

There are a number of articles written on IISNode, but I didn’t really find an up-to-date article that describes how to install the latest version on a clean Windows 2012 using the setup packages provided by IISNode, so I’m just writing down the experience to save another developer’s time, because it wasn’t “all out of the box”.

First of all the basics:
  • We use a Windows Server 2012 installation, 64 bit (of course), simple standard version (it is a test setup from my side)
  • Make sure IIS is installed on this system, so that you have working web server… after all it’s called IISNode so there is the requirement for an IIS
  • Make sure NodeJS is installed on your server. IISNode does not install node itself, it is only the glue between NodeJS and IIS (and that is a good thing!). It however does not check whether NodeJS is installed, compared to IIS where the requirement is checked.

ORDER IS IMPORTANT! I accidentally didn’t install all these prerequisites upfront (which is just a nice way of saying I forgot a part), and had a hard time to get IISNode working on the system. Also a read a bunch of articles on problems when only using x64 packages of NodeJS, so installed both x86 and x64 packages of NodeJS on my machine. Don’t know whether that is still a requirement, but it was at one point.

Installing IISNode

Installation of IISNode is getting a matter of downloading the package from the github repo. I used the iisnode-full-iis7-v0.2.2-x64.msi package, which was the most up-to-date at the time of writing. I just ran default installation, and this creates an installation directory in C:\program files\iisnode (I expected it to be in the Inetpub folder to be honest).

After installation go to the installation directory and run setupsamles.bat AS AN ADMINISTRATOR!

Running hello.js

Just as with any other technology, our first goal is getting the “Hello World!” running, which is a matter of getting hello.js to work. The hello.js is located in C:\Program Files\iisnode\www\helloworld. If you did everything correct up until now, you should be able to browse to http://localhost/node/index.htm that contains a link to helloworld example, which will show the readme file on the sample. Launch hello.js from there and you should see this:
iisnode1

 

Important: if you want to play with the installed samples in the directory they are installed, you will, with default permissions, not be able to change the files since you have no permissions to save files in the Program Files directory! Easiest way out is to change the security permissions on the www folder of the iisnode installation directory !

Running express applications

The samples directory contains a number of sample projects, amongst them one with Express framework, so if you need this, make sure to check those out because there a couple of specific items to be added to the web.config file regarding the URL rewrite module. On the topic, there is perfectly fine article here which makes it useless that I write about it. Important thing to achieve this, is the fact that this configuration is done in IIS web.config file, which is unknown to the node project itself and does not require any changes to your legacy code… which brings me to the next topic.

Transferring legacy NodeJS http server applications to iisnode

IISNode requires little or no changes to your application. The most obvious one is the location where you set the port number for your server to listen. Since this is determined by IIS, this will be passed as an environment variable to your application:

    http.createServer(function (req, res) {
         ...
    }).listen(process.env.PORT);

Second part most of us have in custom applications is some additional npm packages, e.g. I use Winston to do some logging output. Luckily, this doesn’t really change with IISNode: just run npm install –save command to add your package and all will be just fine…. so that is a relief and makes life easy and makes sure I do not have to alter my code too much.

Important: if you create your own applications in another location, make sure that NodeJS has proper access rights to the folder, else execution will fail since NodeJS process launched via IIS runs in lower priveleges than your own account. The error does not really mention this is the problem, but in the description there is a small remark on that a log file was not written, which made me come to this conclusion. 

Dual run

Taking the above into account, these changes are that limited, that when you make sure to add a little line of code in your application (var http_port = process.env.PORT || config.http_port), you can still run the application from the command line using the node.exe executable as before, and that is a really good thing. It allows you to run the same code on servers where IIS is not an option, which is a enormous advantage!

Why I like a NodeJS web application on Azure using SQL as a backend

I’ve been doing a lot of studying lately on which technology is best for my new project, and with the abundance of frameworks life doesn’t get any easier. I have a number of requirements which most of us have when creating a new web applications:

– A framework that is easy to use and well supported (in terms of modules, libraries, … whatever makes us not invent the wheel twice)
– Must run in the cloud
– Must run for free in the cloud while I’m developing the thing. I only want to pay as soon as people pay me
– Must be able to develop code easy on any machine

After all my research, I think I found my ideal combination: it is a NodeJS application that delivers a REST API and accompanying AngularJS client pages in combination with SQL on the backend. A number of people reading this will put question marks with this choice of technology, but, there is a good reason why this combination makes sense.

Why NodeJS?

I like NodeJS for two reasons, which are not accidentally on my requirements list:

– It’s javascript, and doesn’t require a compile step while developing. Hence, it is easy to develop NodeJS applications on any system. Macbook, not a problem. Pc at home, with windows 8.1. No problem. Editors in abundance, see my previous article on Visual Studio Code fore example, but for Windows there is also Visual Studio Community edition, one of the better Javascript editors available today.
– There is an extensive list of free modules available via NPM, and up until today, I haven’t found one problem I couldn’t tackle with a module that is freely available. Is seems me from reinventing the wheel, just make up the glue to deliver a nice service

Note: I like C# code as well, don’t get me wrong, and I’ve written web applications in MVC as well. I just like the fact that NodeJS has no compiler requirement, which makes it very easy to develop.

Why Azure?

Most of the people I meet associate Azure development with ASP.NET websites (with our without MVC). That is one option, but NodeJS applications are equally good supported, and, web applications are for free on Azure! You can say about MS what you want, but Azure is a nice piece of work, and the fact you can easily run NodeJS applications on a free of charge web tier is fantastic. Deployment via Git is included and super easy, so what else do you need? I think most people don’t know these options, and often turn to Amazon and OpenShift for these kind of things, as they are more commonly associated with NodeJS.

Though not really applicable for a simple website, but, there is also an Azure module for NodeJS available to manage your Azure objects, so don’t underestimate what NodeJS and Azure have to offer.

Why SQL?

If you see NodeJS, you always see MongoDB or another NoSQL backend. I like MongoDB and NoSQL for what they are made, but, my data is simple and relational, so why should I go for a NoSQL DB? It would just end up in objects referring to each other like a relational schema, I would use Mongoose to enforce all the constraints…. so at the end of the day, SQL does the job here better than NoSQL. I don’t believe following a trend or hype is a good thing if it just for the sake of the trend. If you can get profit out of it, do it, else, stick to what you have and works.

NodeJS has no issue with SQL, for those who are in doubt. There is a great library called Sequelize (check npm repository) that makes it really easy to work with traditional SQL databases.

A limited MySQL DB is also available for free on Azure, so that’s good for kicking off my development. When the app grows, it might not be sufficient anymore, but the customers should pay for the growing storage cost, so no worries about that.

Note: it thought it was possible to get a limited free MS SQL instance for development, but, this seems not to be available anymore. However, the Sequelize component should hide most of the backend for us thus this should be doable.

Note: it is possible to get a MongoDB instance as well on Azure, thought I’m not sure whether that is for free to start with (up until a number of MB’s). If you would need a MongoDB in the cloud, you can always turn to MongoLabs for a free MongoDB instance in the cloud (to start with). I takes only 5 minutes to register and have it working.

Conclusion

For these reasons, this combination is my ideal environment. It might be yours as well for the same or even slightly other reasons. It’s not a trendy combination, but I believe it is a solid one, and that is what counts at the end of the day.

Running Redis on Docker on Windows

When it comes down to OS’es, I’m not the monogamous kind of guy. I like things that work on Windows, Linux, and Mac, preferably without a lot of troubles (actually, preferably without any troubles, but, I’m a realistic type of guy as well). As such, I was interested to see the power of Redis, but dissapointed it doesn’t really run “out of the box” on Windows. Ok, there is a port avialable, but that’s not always the real thing. So I decided to kill two birds with one stone, and bring in another technology that is still on my bucket list: Docker.

Docker is a software container technology and has some nice advantages for both developers and IT admins, and after 15 minutes it turns out to run real smooth on my Windows machine as well. More information can be found on the Docker site installation documentation. Another nice and very to the point intro can be found here. Note that running Docker on Windows really involves quite some technology, but the installer really hides that nicely. Once docker is running, make sure you know the IP address of the VM host it is running on so you can connect later. It is logged in the console, but will not be visible anymore after running the redis command.

Phase 1 completed, and for those who like open source technology combined with really fast, no nonsense installation and the usual struggles, I really advise Docker and their public repositories. Installing Redis on Docker is a single line command:

docker run -p 6379:6379 redis

If all goes as planned, after the download, your Docker window should look like the below image.

Truth to be told, I did have some issues during installation, where it had an unexpected EOF and the untar command failed. This seem to be common and according to different sources is caused by the “network” and more specifically the “Wifi”. I don’t know the details, but disabling Wifi and continue on the wire solved the issue.

rediscmd

And that is actually all! Redis is running on your machine, ready to use, which means in my case to be used from within NodeJS. Tons of information are available on how to do this, so I will simply show my sample test code I quickly wrote in the Visual Studio Code editor, and this for four reasons:

  • I’m fond of the editor, which I expressed in my previous post
  • You can really easily debug your server side code
  • I really like intellisense that can be added via definitelytyped.org, it is available for most common node modules and thus also for redis package, and can be simply added by adding the references on top of the file
  • Be warned when connecting to the Redis server. First of all, because this is actually running in a VM with a virtual network on your machine, and thus you need to connect using an IP address as the default localhost will not work. Second, you will use hostname/IP address and port combination, and Redis client takes the port number first. It could be me, but in all languages where I connected to a server, host name is first and then port number comes second. Of course the documentation is correct, but it was that trivial for me that it took me a while to notice and I spend to much time wondering why I was always getting these connecting errors.

rediscode

And here it is, a full functioning system with Redis running in Docker on Windows and connecting via NodeJS app written in VSCode.

 

 

Visual Studio Code for writing and debugging NodeJS applications

I’ve always been a fan of Visual Studio. I started using it when it was still version 6 (when my professional life started) and the only reasonable language was C++. Even though I worked on many different platforms in the meantime, none of the tools I encountered ever came close to Visual Studio. You can hate MS as much as you want (which I do not) but you have got to love their development environment.

Last couple of months I really got into the NodeJS world, which I think is a really great environment: you can develop on any platform, and it runs on all of the platforms, cloud or on premise, everybody supports it. You cannot get any “vendor lock-in”, which is great and you have all options open. You want to move to Azure? Fine, no problem. Amazon. Equally ok. Run locally on your Macbook. Done. An extra advantage is the simple type and run principle (which is a well known principle I just invented), which means there is no compilation or building required before running the application.

If you read this intro, you will understand that I was very pleased with the announcement of Visual Studio Code, which fits perfectly in my scheme of things. Of course there is Sublime which everybody is using for this kind of task (just as me), and truth to be told, some of the VSCode features are simply copies, but it works equally good for me, and has suprised me. Biggest reason why it pleases me: it’s simple. As powerfull as Sublime is, I only used 2% of it’s features for developing node applications, so it was too much. VSCode is simple, offering everything I needed, and even more, using the docs on https://code.visualstudio.com/Docs/ I was able to create a “blanco nodejs template application” and configure debugging in under 5 minutes. Combine this with the great support for client side javascript debugging we have today in allmost all browsers, and you have full support for debugging on client and server!

vscode debugging

 

Another great feature is the support for type definitions in combination with the intellisense. All info is in the docs, just mentioning it here because it actually really works, even tough I’m not the biggest fan of Typescript (I don’t like any “extra” steps for converting or compiling things… remember the type and run principle), I don’t even use it my apps. It just great to have intellisense support for the basics of a language.

vscode2

Next job is to experiment with the task features VSCode offers, which also seems to solve some issues for me… but I need a little more time for that one.

Two remarks to close this article:

– It could well be that Sublime is equally capable of doing all of this, I don’t know. And that is exactly my point. In all the power of Sublime I couldn’t easily find these things, while with VSCode I got things running in no time with a very to the point document

– There are still some improvements to be done on VSCode, especially the part on HTML tags feels sometimes wrong. For example, in any modern HTML editor typing <br> will convert your tag to <br />, while VSCode will just complete the tag, so you end up with <br></br> which of course nobody wants. Same goes for a lot of other tags. Also starting a closing tag does not automatically bring up intellisense to suggest the rest of the tag. These features are available in Visual Studio 201x, so they know how to do it…