File bundling with Gulp

As the project is moving forward there are new files appearing. There are few javascript files now, and I know there are going to be more to come. This is bit of an issue – more files mean more requests to server, which directly causes slower loading time. Browsers can only do so many concurrent requests, if you will require more files than browser limit, you will have to wait for previous request to complete. On server it is also more stressing to serve dozen requests for different files than to give one file and be done with it.

With Visual Studio 2015 comes built in support for task running. There are few task systems available, most popular being Grunt and Gulp. They support various javascript written tasks and you can always write your own one, doing what you need. They may be used for anything, but I will get them to do bundling for me.

Bundling is process of merging multiple files into one resource, that can then be used on page. In my case I want to get all my javascript files bundled, but css can also be merged if you want. I’ve picked Gulp as my poison of choice.

First thing that has to be done is to create packages.json file – Visual studio has template file to support that. Second – gulpfile.js – in this file I will be defining my tasks, gulp automatically looks for this file for list of tasks. So does task runner explorer (you can turn its window on in View-Other Windows-Task Runner Explorer, or by pressign Ctrl+Alt+Backspace).

What I need is some task definitions downloaded by npm – node js package manager. To do this I write what I need in packages.json

{
    "version": "1.0.0",
    "name": "ASP.NET",
    "private": true,
  "devDependencies": {
    "gulp": "3.9.1",
    "gulp-concat": "2.6.0",
    "gulp-concat-css": "2.2.0",
    "del": "2.2.0"
  }
}

DevDependencies is the section of interest here. I will be using gulp to run tasks, gulp-concat for javascript files and del for removing unnecessary files. You can right on this file in Solution Explorer and restore packages. They will appear in node_modules directory.

Now it is time to write some tasks in gulp. First – it needs to import required tasks (the ones that were references through packages.json):

var gulp = require('gulp');
var concat = require('gulp-concat');
var concatCss = require('gulp-concat-css');
var del = require('del');

And after that tasks can be defined:

var bundlesOutputDir = "Bundles";

var bundles = [
    { scripts: ["Content/raim.js", "Content/PlayersList.js"], output: "raim_main.js"}
];

gulp.task('clean', function () {
    del(bundlesOutputDir + '/*');
});

gulp.task('bundling', function () {
    bundles.forEach(function (bundle) {
        gulp.src(bundle.scripts)
            .pipe(concat(bundle.output))
            .pipe(gulp.dest(bundlesOutputDir));
    });
});

Those are pretty basic. clean goes to defined output directory for bundles and removes all the files. bundling on the other hand iterates over my array of bundles I want to create (which at the moment only holds one thing), loads those files (or marks them as files to process), pipes (forwards) them to concat task, which handles merging files and creates new file with name passed as parameter. This newly created file is then piped to gulp.dest task, which outputs it to target directory.

When ran from task explorer it sure does create raim_main.js file in Bundles directory!

But bundling by hand is not what I want to do. This should be done automatically, each time I change any file. I want to have them ready for use the second I stop writing them. For this, gulp.watch is perfect:

gulp.task("watch", function () {
    bundles.forEach(function (bundle) {
        gulp.watch(bundle.scripts, ['clean', 'bundling']);
    });
});

gulp.task('default', ['clean', 'bundling', 'watch']);

Again, going through all bundles, I mark those files for watch process and each time any of those files change I want to trigger clean task and later bundling. Additionally, default task is created to run those tasks automatically when gulp is ran as parameter (e.g. from console or during build process on CI machine).

Triggering default task prepares working space for me, old bundles removed, new ones created and watch configured.

But what about newly cloned repository, if someone downloads it? Or if I restart visual studio – I don’t want to have to go to task explorer and trigger those tasks. Thankfully Visual Studio supports biding tasks to few events in VS, one of those being – project opened. It is done simply by providing special structure in comment at top of the file:

/// <binding ProjectOpened='default' />

There can be comma separated list of tasks to run on start, but I just need one at the moment. This way if I unload the project, kill task running in background, and reload main project again – default tasks kicks in automatically, preparing workspace. Neat!

What is left is to reference the bundle from html file:

<a href="http://../Bundles/raim_main.js">http://../Bundles/raim_main.js</a>

And it should be… wait – no. Scripts are in Bundles directory, but Nancy wont be serving those files. What it needs is some configuration to tell it there is static content there. One line in Bootstrap file:

nancyConventions.StaticContentsConventions.AddDirectory("Bundles");

And only now all is running smoothly.

Nancy in Class Library

I’m just starting new project (https://github.com/pako1337/raim) and thought about doing it in different technology stack, not usual ASP.NET MVC I’m used to at work. I chose Nancy FX for the moment, and later it should be joined by SignalR. Nothing fancy for now, but I don’t need much at this stage.

Setting up Nancy is pretty straightforward, all instructions are available on web (https://github.com/NancyFx/Nancy/wiki/Hosting-nancy-with-owin) and it is just few steps to having Nancy module and returning hello world string. What I want to do differently is – I don’t want to use ASP.NET project. I always use it and never bothered to see what’s inside of it. It started bothering me some time ago that I have no idea how is it different to normal class library.

First thing first, I have set up new solution and createn new, empty project. In Visual Studio for some reason empty projects are not empty at all – there are bunch of dependencies already included – basic stuff. And there is useless Class1.cs – straight to bin with it!

Next nuget and installing few packages: Nancy, Nancy.Owin and Microsoft.Owin.Host.SystemWeb (which depends on Microsoft.Owin). Easy-peasy. Since we are doing Owin, Startup.cs is nice to have, important to put it in the same namespace as project’s default one, otherwise it needs assembly attribute to point to it – but why make life harder if there is nice convention.

    public class Startup
    {
        public void Configration(IAppBuilder builder)
        {
            builder.UseNancy();
        }
    }

It doesn’t get much easier than that. Owin set up (at the minimum, but still). Now it is time for Nancy module that will handle our requests.

    public class HomeModule : NancyModule
    {
        public HomeModule()
        {
            Get["/"] = parameters => "Welcome to Raim!";
        }
    }

Also as simple as it gets. When you get request to application with no parameters and nothing at all in request’s path – call this anonymous method to return string. not quite HTML – but will let us know if all is set up. F5 and here we… wait, what? Class library cannot be started directly? Well, yea, Visual Studio won’t run class libraries at all.

At first I was sure that it has to be some simple thing. Setting somewhere to tell it it can be started directly, connected to IIS Express (I want it to be my dev server). Looked through properties and found exactly nothing. My google-fu was useless as well. First few pages were of no use, guess no one is doing this the hard way!

My next idea was – create ASP.NET project, diff csproj files and find differences. Did that. At first I noticed nothing strange – few additional references, some files and directories I didn’t have. Nothing strange. Then I noticed one line that seemed interesting – entry. And two Guids in it. Decided to copy it over to my csproj (need to use editor outside of VS to do that; or you have to unload the project, edit the file manualy and then reload it). And what do you know – works like a charm! Maybe that is some common knowledge, but I happened to never stumble upon it in my few years as software dev.

...
<Project>
    <PropertyGroup>
        <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>
        ...

What is left is double check IIS settings, F5 – yes, please add web.config file, thank you – 403.14 – Forbidden! But why? Shouldn’t it be working? Took me a while to realise what is wrong – and I feel silly now, knowing that – project was building into /bin/Debug directory, while IIS was looking into /bin directly. Of course it does, it always did, fool of a Took! Quick change in settings and finally – “Welcome to Raim!” appears on my screen!

Nothing extra fancy yet, but hey! At least now I know that how VS differentiate between web application, console application and simple class library.

Learn your shortcuts

How much time do you spend in editor working with code? Debugging it? Hopefully a lot. How much time do you spend working with your keyboard? Hopefully most. Mouses are evil. It takes to much time to move hand to mouse, select option, click, move hand back.

But there’s a better way! Learn shortcuts! Visual studio have lots of them. Resharper too. Web browser? Sure does. Total Commander? Windows? Vim? Shortcuts all the way.

I always tried to learn how to do things with keyboard rather than mouse for more efficiency. Some time ago I started writing stuff in Vim. That’s when I learned mouse is mostly useless when writing stuff. Now I’m back to good ol’ Visual studio with Resharper and I usually don’t even think about touching mouse. And I cringe when someone’s searching menu with mouse or (God forbid) clicks right mouse button to do something.

First – shortcuts are simple. So simple that after few usages, your fingers will remember how to do them without you thinking about what exactly needs to be pressed. Ctrl+Shift+B for build is as natural for me, as Ctrl+S ever was. Ctrl+U+R, Ctrl+U+U and Ctrl+U+L are so natural for me that I had to check this shortcut right now because I didn’t remember it. My fingers do – when I’m in VS they know what to do. F9, F10, F11 – you all know them in debugging I hope. What about Ctrl+F10 and Shift+F11?

But I don’t know all the shortcuts. When and how do I learn them? I go with simple rule of thumb – if I’m doing something once – that’s cool, I can click it. Second time – well, ok – I will go with you one more time, you evil mouse! Third time? You won’t trick me, I’ll google the shortcut and will google it again and again until I’ll teach my fingers how to do it properly. No more pointing and clicking.

And my hands are better on keyboard as well, holding mouse is fine only when I’m shooting at those filthy zombies. Left 4 Dead – here I come!

Trying to load incorrect version of dll – bindingRedirect bites

I got an error saying that application tries to load Npgsql, v. 2.0.12.0 but it can’t find it. Whoops, missing dll. Or that’s what I thought. Checked output folder and references in project – reference is there, it’s copied. So what’s the problem? Then I noticed the version does not match. I have referenced 2.0.11.0, cause that was the version other dependencies were build against. So where does this newer version loading code coming from?

Sadly it took me quite some time to figure it out. First I was thinking there is some mismatch in dependencies, that I’ve messed something up when rebuilding Simple.Data.PostgreSql. Checked it, double checked it – all was perfectly fine. Then I used my favorite tool – search. There was just one line in whole solution referencing 2.0.12.0web.config:

<runtime>
  <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
    <dependentAssembly>
      <assemblyIdentity name="Npgsql" publicKeyToken="5d8b90d52f46fda7" culture="neutral" />
      <bindingRedirect oldVersion="0.0.0.0-2.0.11.0" newVersion="2.0.11.0" />
    </dependentAssembly>
  </assemblyBinding>
</runtime>

bindingRedirect – a way to tell that we should load newer library even if strongly named reference requires older version. What a bummer – how did it get there? I didn’t put it there myself, that’s for sure.

Then I remembered – first I installed Npgsql from Nuget and I picked latest version. And installation from Nuget modified my web.config file. But removing this dependency from Nuget did not remove config entry!

So lesson learned – Nuget is great tool, but pay attention to the changes it makes to your code or you may end up with some stupid errors. Easy to fix, but sometimes hard to pin point.

Visual Studio not copying assemblies to output directory

Today’s not a good day for my coding. I planned to do database design and data access logic using Simple.Data and everything was going wrong. Outdated version of Simple.Data.PostgreSql, had to download code from github manually. Thanks to Maciej Aniserowicz this project was recently updated to later version of Simple.Data. But along the way I was stumbling upon one issue after another. I don’t like those days when tools are mostly in the way of getting work done and not helping at all.

My biggest complain today will be Visual Studio. Yes, it is big and slow. But it is also pretty damn good IDE. Add Resharper to it and sky is the limit. Well, on my fun projects I’m using Visual Studio Express mainly because it is for free. So no Resharper magic for me, no plugins, but it’s free – I don’t complain (too much). I have two versions installed: Visual Studio 2013 for Desktop and Visual Studio 2013 for Web. Today for the first time in quite a while I started desktop development version to build quick proof of concept console application. At some point I started getting exception from library. But I could not view it! Apparently exception assistant, this fancy floating window that appears when something went terribly wrong, was removed form VS2013 for Desktop. Why? I have no idea, but I am greatly disappointed. You can read about reported issue (and vote for a fix) here: Enable Exception Assistant option missing from VS2012 Express For Desktop

But then the main issue came and hit me when I least expected it. Imagine situation where you have main project A which references project B. Classic situation. B further references few dll libraries: B1, B2, B3. But it explicitly uses only B1, rest is used through reflection or something. Now – I would expect, and I’m pretty sure that’s how it worked previously, that all three dlls will get copied to A‘s output directory. Since I created reference I know it’s there for some reason and would like VS to respect it. However no, that won’t happen. Does not matter if you have Copy Local set to true for all libraries – only B1 will be deployed since it is the only one used. Took me way too much time to figure it out.

You can look at this issue here:
Visual Studio does not copy referenced assemblies through the reference hierarchy
and vote for this to be fixed. VS team decided not to fix it since it has not enough votes. I don’t get the logic behind this – it seems to me like basic functionality is broken and they decide not to do anything about it.

Of course this issue is simple to work around – you can either reference the same libraries in A project or create after-build task to copy them. It will work, but it is far from perfect – now there are two places you need to remember to modify if you will start messing with references. IDE should be there to help you, not to make it harder.