Thursday, December 17, 2015

Microsoft Certified Solutions Developer!

After 3 years, 3 exams, and an excessive amount of study I have achieved the Microsoft Certified Solutions Developer - Web Applications certification.

I realise that many people don't put a huge amount of emphasis on exams, valuing real world experience instead. But I have found that what I learned studying for these exams has been extremely valuable to my profession, helping me make more informed decisions in my day to day work.

There is no substitute for real world experience, but real world experience leaves a lot of gaps and doesn't always teach the best habits. The exams have enabled me to fill in these gaps and develop a real technical proficiency which I may not have had without them.

Born to Learn just posted this blog talking about a recent study which found 4 advantages certified staff have over non-certified staff, there are some interesting findings!

Tuesday, October 27, 2015

Amazing Things You Can Do With Azure Storage Queues

Load Balancing and Workload distribution

Queues can be used to organise the distribution of a workload across multiple processors.

A common technique is to post messages to a queue which are a key to a task which needs to be performed, and have multiple worker processes pick these tasks from the queue to perform in parallel.

This can be combined with other storage mechanisms to perform complex tasks. Say you were trying to render an animation. The queue could be loaded with keys, which point to an entity in Table storage. This entity could contain the Url of an image stored in blob storage. A worker process would pick up the message from the queue, reference the entity from the table and download the image from blob storage. It could then perform compute intensive tasks on the image while other worker processes picked up other images.

In this way, cloud storage can be used to augment worker roles and perform powerful parallel operations.

Asynchronous Processing and Temporal Decoupling

It might sound like something you do with a sonic screwdriver, but temporal decoupling simply means that senders and receivers don't need to be sending and receiving messages at the same time. With messages stored on the queue, they can be processed when the receiver is ready to do so, and the sender is not held up.

This can be useful for performing background tasks which are not critical to the main operation of a program, for example, a web application, rather than providing a slow response, can just queue up a thumbnail generation process and respond to the user quickly. A game, which requires the utmost UI responsiveness, might store replay data asynchronously using queues.

Monday, October 05, 2015

Calling Long Running Methods Asynchronously

When you are able to call an async method, do it using async and await or by making your method return a Task:

public async Task<string> RequestData(Uri uri)
    WebClient webClient = new WebClient();
    return await webClient.DownloadStringTaskAsync(uri);
} // Rely on underlying Tasks whenever possible.

public Task<string> RequestDataAsync(Uri uri)
    WebClient webClient = new WebClient();
    return webClient.DownloadStringTaskAsync(uri);
} // Rely on underlying Tasks whenever possible.

Dealing with synchronous calls

When the method you are calling is not asynchronous, wrap the method in a task delegate:

public Task<string> RequestDataAsync(Uri uri)
    return Task.Run<string>(
        () =>
            WebClient webClient = new WebClient();
            return webClient.DownloadString(uri);
} // Create Tasks when you have no other choice.

Capturing Exceptions

Getting a little more advanced, what if you might need to capture exceptions from a long running method? Use the following pattern:

public Task<string> RequestDataAsync(Uri uri)
    var tcs = new TaskCompletionSource<string>();
    WebClient webClient = new WebClient();
    webClient.DownloadStringCompleted +=
        (_, args) =>
    return tcs.Task;
} // Example of wrapping Async/Completed.

More info

Async'ing Your Way to a Successful App with .NET

Creating Async Libraries That Are Modular, Reusable and Fast, in Microsoft Visual C# and Visual Basic

Thursday, August 20, 2015

NuGet Like a Boss: Part 1 - Don't Check in Packages

After suffering like many with so many Package Restore woes in my projects I decided to make notes on the best way to deal with Nuget packages.

Ignore the packages folder

Not doing this means you check in the packages which are huge. This is annoying and kind of defeats the purpose of Nuget. When you ignore (and therefore don't check in) your packages folder, anyone getting your source code can run package restore on the solution and Nuget will download the packages automatically.


First, add a file named .tfignore. This may require some Command prompt renaming as some set ups don't like files beginning with a dot. When you get past this annoyance, open the file in notepad and enter the following:


That tells TFS to ignore the packages folder. For some bizarre reason, this doesn't include the respositories.config file. You'll need to add a second line as follows:


You'd think this would be it, but you may notice that your packages folder is already in your Pending changes. To get around this, create a folder called .nuget (command prompt trickery may be required) and in there create a file called NuGet.config. It must go in this folder, even if you have another NuGet.config at solution level. Enter the following text:

<?xml version="1.0" encoding="utf-8"?>
    <add key="disableSourceControlIntegration" value="true" />

This should ensure that your packages stay out of source control.

Finally ensure that .tfignore and NuGet.config are added to source control so that these settings should apply for anyone using the project.


Be aware that the .tfignore file may not work if someone has already checked in the packages folder. Delete this from source control and you should be good.

Monday, August 10, 2015

Developing for the Cloud - An Introduction

When you create a web application for the cloud, there are many things that need to be done differently. It's not just a case of saying "I'm doing cloud" and all you're really doing is putting it on someone else's VM. Doing this, the costs are much higher than if the application is designed with cloud in mind. This might be fine from an infrastructure point of view, but the cloud can have profound impacts on development from the ground up.

Azure allows us, and also forces us, to engineer our applications completely differently.

Forces? Well, this is because when you're hosting in an App Service plan, you're billed based on compute time and resource usage. This forces you to write more efficient code. You can't just write code that performs unnecessary operations and get away with it - the efficiency of code now directly translates to dollars. This demands that you think twice when writing code, espcially loops, to ensure you're being efficient. Caching becomes a major priority so that you're not hitting a database unless you absolutely have to.

With the weight of costs, you need to start thinking about more efficient ways of doing everything. Luckily Azure offers many services to help increase application efficiency. The most basic example is storing images and other static resources in Blob storage. This is fast, lightweight, and extremely cheap.

Scripts and CSS can be offloaded to localised CDNs. This also increases speed and significantly decreases cost.

Some data can shift from SQL to NoSQL models such as Table Storage, DocumentDB, or Graph data with its powerful relational features. An application might use all of these data types simultaneously for different purposes. For example, a shopping cart site might use SQL to store a user's identity information, Table storage to manage their shopping cart, DocumentDB to store product details, and Graph data to build "You might also like" links.

Need to run asynchronous/background tasks such as generating thumbnail images? Use Web jobs to run scripts, executable, and third party libraries completely outside of the typical web application Request/Response lifetime. Use queues to decouple logic from the main workflow, or use multiple worker threads to perform parallel processing.

We can also take advantage of Azure Search (powerful search services), Service bus (host web services locally but expose them via the cloud), Azure machine learning (predictive analytics, data mining, and AI), and Notification hubs (push messages down to clients).

Then, we can host our applications in App Service plans (instead of Virtual Machines) and take advantage of Visual Studio Online's build and Continuous Integration features. We can also leverage elastic scaling and Application Insights.

There are many more features in Azure and more appearing every day.

To summarise, developing for the cloud is very different from building a website that will be isolated on a local server. It's a far more distributed model where separation of concerns is much more pronounced than we're used to. Developers and architects will need to think differently when designing applications.

This is just an intro to cloud features. In the future I will go into cloud architectural design patterns which allow us to design our applications to be resilient in failure-heavy environments.

Tuesday, June 23, 2015

How to Be an Organised Developer (and spend more time coding!)

As a developer your main focus is to write code. But over time, you'll find that there is a lot more to development than this. If you're not aware of this, you might one day wake up and realise that the notepad file you used for passwords and connection strings has gotten out of hand.

Being more organised from the start can help keep you focussed on coding and help you stay more efficient. If you move jobs you'll pick up a lot of information in the first few weeks, you'll want to organise it well from the start. If you stay in the same job for years, the 'other stuff' you accumulate can get messy and cumbersome.

Like good code, making an effort to organise yourself well from the start can pay dividends later on, when it comes to navigating and maintaining all your stuff. Here's a list of some of the "stuff" you'll find yourself accumulating as a developer, and how to keep it well organised.

OneNote (or Equivalent)

An essential tool for all developers.

The main thing this is useful for is storing essential information such as test data, database names, licence numbers for tools such as Resharper and Linqpad, and lists.

You can also use it for debugging information, screen grabs, functional specs, checklists and pretty much anything you can think of. OneNote allows you to organise all this quite effectively with its use of tabs and pages.

Note that by "equivalent", I don't mean Notepad++. You need your notes in one well organised and secure location (save your notebooks remotely), you don't want to have to worry about hitting "Save", and you also need to be able to paste in screen grabs, tables and other rich content. Evernote might come close but OneNote is trusted in enterprise environments which can give it the edge.

Logging Access

Pretty soon you're going to need to access logs for auditing, testing or debugging purposes. Remember to record all details in your note software.

Product Documentation/Wikis

As a developer you will probably be responsible for writing a lot of documentation. Make sure it's in an easily discoverable place and well maintained. Good code should be self-documenting, yes, but other stakeholders need to know what the code is doing from a non-technical perspective.


You'll always have a selection of really important links. This might include:

  • Product documentation/wikis
  • Test Harnesses
  • ALM tools (TFS, Git)
  • Communication tools (Sharepoint, Trello)
  • Administration tools (Timesheets, financial, personnel software)
  • Development learning materials, tutorials, blog posts, communities etc. 
Figure out the best way to organise these based on your needs and make sure they're backed up and accessible everywhere (use Chrome/Firefox Sync or a bookmarks manager).

SQL Files

Most developers will probably have a collection of SQL files of common, useful queries, for logging, basic CRUD, etc. Make sure these are in an easy to access, secure, backed up and preferably remote location.


Specifically in Web Development, tools such as iMacros can be invaluable for automating frequent tasks such as logging in to test sites or running common actions on test harnesses. Remember, keep them well organised and backed up remotely using Dropbox/OneDrive etc.

Powershell Scripts and batch files

You might also have some PC management tasks that need to be automated. Powershell is fantastic for these kind of tasks, and is becoming even more useful with the advent of DSC.

Linqpad Scripts

Similarly, Linqpad allows for frequently used code to be stored and used in a lightweight manner without all the project overhead.

Code Toolbox

Finally, all developers should have a code toolbox, or a collection of libraries and code snippets they use regularly in their projects. This is a lot to go into so I will create a new blog on it, but basically it could consist of
  • Project templates 
  • Emailing library 
  • Cloud storage library 
  • Logging classes 
  • MVC Html helpers etc
The idea is to allow for Rapid Development by having everything you frequently need at your fingertips and not having to worry about finding code for frequently performed tasks such as input forms or membership features. As I said, I'll give this topic a blog of its own soon.


Hopefully this has inspired you to ditch that pile of text files and get your administrative stuff in order. Doing so will make you more efficient and help you focus on what's really important - writing code.

How do you organise yourself? Is there anything I've missed? Leave a comment!

Friday, May 29, 2015

Why Bother With Unit Tests?

I still see a lot of scepticism on WHY we should do Unit Tests and if they're really worth it.

Of course, they usually add time to the development cycle. They can be hard to create. Quite often, they seem pointless; Why do I need to check if my TwoTimesTwo() method returns 4? Of course it's going to!

Well, the benefits of Unit Tests may not always be obvious. Often they're disconnected from the problems caused by not doing them. For example, a manager is not likely to blame a Production issue on the fact that you didn't do Unit tests.

Most of the time, if you release some code that performs what it needs to, nobody cares about the quality. However, the people who will care, are those who want to maintain your code later, especially if it ends up being:

  1. You; or 
  2. A violent psychopath who knows where you live

Unit Tests can also help to document code, showing you what it should and shouldn't be doing. They can help you understand it better.

But maintainability still isn't enough for many people.

Personally I don’t think anyone can really understand the benefits of Unit Testing until they've written some, and felt the satisfaction of those green ticks, and the confidence they give you. Your code is now rock solid, nobody is going to break it without knowing about it (including yourself).

With Unit Tested code you can be sure that every component is doing what it should be. Acceptance Criteria is another step and I'm a strong proponent of automated behaviour driven integrated tests too. But Unit Tests can be more granular, covering a vast array of tiny details that can otherwise be easily overlooked and become the source of fiddly bugs later on. Also, when bugs are found, usually this results in messy, band aid fixes which make the code less flexible. TDD makes the code more maintainable to start with. The necessity to write SOLID code is increased by the need to ensure the code is highly modular and interface driven.

In the end, the extra time spent building unit tests is returned several times by spending less time fixing bugs and maintaining messy code.

The best way to see the benefits of Unit Tests is to write some. Start easy by writing one specifically to test a bug fix. You can then be sure that bug will never surface again.

Getting started is hard, don't get me wrong. It's like lifting weights for the first time, it's going to hurt. But push through the pain and you'll start to see the benefits. It will get easier.

Friday, March 20, 2015

How to Reduce Deployment Risk

Functionality Switches

Feature flags controllable by configuration or an "admin tool", are an excellent way to deploy new features. When going live, you can turn on a feature only for your Production test user, and run some smoke tests, before turning the feature on for the wider world.

Any issues found later can then easily be mitigated by "switching off" the broken features for live users, while at the same time, leaving them on for the test users so that some fault investigation can be done on Production.

Providing there are no breaking regression changes, this will help to avoid rollbacks.

Good logging

Every significant action should be monitored and logged. Obviously this includes calls to external services, but you should also strive to implement logging based on items in the Acceptance criteria. Log results of actions so that you can see if they match expectations. This allows for granular diagnosis so you can see exactly what isn't working and where.

Reduced Functionality Instead of Errors

A good, defensive technique is to fall back to previous functionality when something fails, rather than giving the user an error. Obviously this depends on the scenario and this won't always be possible.

However, combined with functionality switches, this can allow users to continue to use your application while you identify a fault using your production test users. This is also greatly dependent on your logging, of course, as your fault will not be manifesting in the interface.