Amazing Things You Can Do With Azure Storage Queues

Load Balancing and Workload distribution

Queues can be used to organise the distribution of a workload across multiple processors.

A common technique is to post messages to a queue which are a key to a task which needs to be performed, and have multiple worker processes pick these tasks from the queue to perform in parallel.

This can be combined with other storage mechanisms to perform complex tasks. Say you were trying to render an animation. The queue could be loaded with keys, which point to an entity in Table storage. This entity could contain the Url of an image stored in blob storage. A worker process would pick up the message from the queue, reference the entity from the table and download the image from blob storage. It could then perform compute intensive tasks on the image while other worker processes picked up other images.

In this way, cloud storage can be used to augment worker roles and perform powerful parallel operations.

Asynchronous Processing and Temporal Decoupling

It might sound like something you do with a sonic screwdriver, but temporal decoupling simply means that senders and receivers don't need to be sending and receiving messages at the same time. With messages stored on the queue, they can be processed when the receiver is ready to do so, and the sender is not held up.

This can be useful for performing background tasks which are not critical to the main operation of a program, for example, a web application, rather than providing a slow response, can just queue up a thumbnail generation process and respond to the user quickly. A game, which requires the utmost UI responsiveness, might store replay data asynchronously using queues.

Popular posts from this blog

Foundations of Humane Technology - Course completed

Image
Back in January (has it been that long!) I completed the Foundations of Humane Technology course from the Center of Humane Technology . You may have heard of them, they made the Netflix Documentary " The Social Dilemma ". If you haven't watched the movie, it's well worth a watch, delving into the often devastating impacts of Social Media. The course explored how to build technology in a more responsible way to minimise the harmful consequences of the technology you build. It's perfect for technology oriented Product owners and developers like myself, requiring us to think about externalities and unintended consequences, and aligning our values with the products we build. I was made to reflect on how technology has impacted me personally, and realised how my health, cognition and attention were impacted by technology overuse. I was obliged to acknowledge that I am very privileged and lucky, I don’t have any major accessibility issues and my demographic allows me t
Read more

Writing Good Acceptance Criteria

Image
As discussed in my last article , Writing good Acceptance Criteria gives the Agile team clarity and helps them work towards a common goal, saving time and effort while increasing quality. Now I'd like to dig into the details of how to write good Acceptance Criteria. Behaviour Driven vs Procedural Driven Mindsets Many of us may be used to writing/seeing procedural driven tests; step by step instructions with actions and expected results. Procedure-driven tests are often imperative and trace a path through the system that covers multiple behaviors. As a result, they may be unnecessarily long, which can delay failure investigation, increase maintenance costs, and create confusion. Acceptance criteria, and indeed Behaviour Driven tests, should instead be behaviour driven. This means they should be Declarative , specifying how a system should behave in a particular scenario. Procedural/Imperative: Given the user opens a web browser And the user navigate
Read more

Story Points are Utter Nonsense

Image
For over a decade, I've lived a lie. I made myself believe that Agile Story points made sense. I rationalised it like we all did. "They map to complexity, not time". Yet we spend hours on Sprint Planning meetings deciding how many jobs we can fit into a Sprint - a unit of time . Also - doesn't complexity relate to time? "They're a rough guide, they're not supposed to be accurate." How on Earth did we end up with a unit of measure that is "not supposed to be accurate"? "It depends on experience of the person doing the job." This is like saying the length of a centimeter depends on the tape measure you're using. "They're a measure of  relative complexity." Relative to what? Another story you did which didn't have an accurate estimate? "We use shirt sizes/Fibonacci/animals/fruit." Is this supposed to make it easier to understand? Why can't I give a task 4 points? "We have to agree on size befo
Read more