Language

Getting Started with the Azure WebJobs SDK

By Tom Dykstra and Rick Anderson|

The Windows Azure WebJobs SDK is a framework that simplifies the task of adding background processing to Windows Azure Web Sites. This tutorial provides an overview of features in the SDK and walks you through creating and running a simple Hello World background process.

Software versions

Shown in the tutorial Also works with
Visual Studio 2013 Visual Studio 2013 Express for Desktop.

Visual Studio 2012 with the latest update should work, but the tutorial has not been tested with it, and some menu selections and dialog boxes are different.
.NET 4.5  
Windows Azure WebJobs alpha-1  

Questions and Comments

If you have questions that are not directly related to the tutorial, you can post them to the Windows Azure forum, the ASP.NET forum, or StackOverflow.com. For questions and comments regarding the tutorial itself, see the comments section at the bottom of the page.

Contents

Introduction

The WebJobs feature of Windows Azure Web Sites provides an easy way for you to run programs such as services or background tasks in a Web Site. You can upload and run an executable file such as an .exe, .cmd, or .bat file to your web site. However, most useful work that you might want to do requires a lot of complex programming. That's where the Windows Azure WebJobs SDK comes in:  it provides a framework that lets you write a minimum amount of code to get common tasks done.

The WebJobs SDK has a binding and trigger system which works with Windows Azure Storage  Blobs, Queues and Tables. The binding system makes it easy to write code that reads or writes Windows Azure Storage objects. The trigger system calls a function in your code whenever any new data is received in a queue or blob.

 The WebJobs SDK includes the following components:

  • NuGet packages. The NuGet packages enable your code to use the WebJobs SDK binding and trigger system with Windows Azure Storage tables, blobs and queues.
  • Dashboard. Part of the WebJobs SDK is already installed in Windows Azure Web Sites and provides rich monitoring and diagnostics for the programs that you write by using the NuGet packages. You don't have to write code to use these monitoring and diagnostics features.

You don't have to use the WebJobs SDK with the WebJobs feature of Windows Azure Web Sites. As noted earlier, the WebJobs feature enables you to upload and run any executable or script, whether or not it uses the WebJobs SDK framework. But for any work that involves Windows Azure Storage, the WebJobs SDK will make your life easier. You also don’t have to use the WebJobs feature of Windows Azure Web Sites in order to use the Web Jobs SDK NuGet packages. However, the Dashboard and the monitoring and diagnostics information it provides are only available as a site extension when running under a Windows Azure Web Site.

Typical Scenarios

Here are some typical scenarios you can handle more easily with the Windows Azure WebJobs SDK:

  • Image processing or other CPU-intensive work. A common feature of web sites is the ability to upload images or videos. Often you want to manipulate the content after it's uploaded, but you don't want to make the user wait while you do that.
  • Queue processing. A common way for a web frontend to communicate with a backend service is to use queues. When the web site needs to get work done, it pushes a message onto a queue. A backend service pulls messages from the queue and does the work. For example, you could use queues with image processing: after the user uploads a number of files, put the file names in a queue message to be picked up by the backend for processing. Or you could use queues to improve site responsiveness. For example, instead of writing directly to a SQL database, write to a queue, tell the user you're done, and let the backend service handle high-latency relational database work.
  • RSS aggregation. If you have a site that maintains a list of RSS feeds, you could pull in all of the articles from the feeds in a background process.
  • File maintenance, such as aggregating or cleaning up log files.  You might have log files being created by several sites or for separate time spans which you want to combine in order to run analysis jobs on them. Or you might want to schedule a task to run weekly to clean up old log files.
  • Other long-running tasks that you want to run in a background thread, such as sending emails. Until now you couldn't do this in ASP.NET because IIS would recycle your app if your app was idle for some time. Now with AlwaysOn in Windows Azure Web Sites you can keep the web site from being recycled when the app is idle. AlwaysOn ensures that the site does not go to sleep, which means you can run long-running tasks or services using WebJobs and the WebJobs SDK.

Scheduling a program that uses the WebJobs SDK

To create and run a program that uses the WebJobs SDK, all you do is write a Console App that includes the WebJobs NuGet packages, upload a .zip file that contains your .exe, .dll , and .config files to Windows Azure, and tell Windows Azure when to run the process. You have three options:

  • Continuous. For programs that need to be running all the time, such as services that poll queues.
  • Scheduled. For programs that need to be run at particular times, such as nightly file maintenance tasks.
  • On demand. When you want to start a program manually, such as when you want to start an additional run of a file maintenance task outside its normal schedule.

You can also use Git to publish directly from your local computer to a Windows Azure Web Site. With the Local Git approach,  you don’t need to upload a zip at all (and you always get a Continuous job).

Coding a program that uses the WebJobs SDK

The code for handling typical tasks that work with Windows Azure Storage is simple. In a Console Application, you write methods for the background tasks that you want to execute, and you decorate them with attributes from the WebJobs SDK. Your Main method creates a JobHost object that coordinates the calls to methods you write to perform tasks. The WebJobs SDK framework knows when to call your methods based on the WebJobs SDK attributes you use in them. For example:

static void Main()
{
    JobHost host = new JobHost();
    host.RunAndBlock();
}

public static void ProcessQueueMessage([QueueInput("webjobsqueue")]] string inputText, 
                                      [BlobOutput("containername/blobname")]TextWriter writer)
{
    writer.WriteLine(inputText);
}

The JobHost object is a container for a set of background functions. The JobHost object monitors the functions, watches for events that trigger them, and executes them when trigger events occur. You call a JobHost method to indicate whether you want the container process to run on the current thread or a background thread. In the example, the RunAndBlock method runs the process continuously on the current thread.

Because the ProcessQueueMessage method in this example has a QueueInput attribute, the trigger for that function is the creation of a new queue message. The JobHost object watches for new queue messages on the specified queue ("webjobsqueue" in this sample) and when one is found, it calls ProcessQueueMessage. The QueueInput attribute also notifies the framework to bind the inputText parameter to the value of the queue message:

public static void ProcessQueueMessage([QueueInput("webjobsqueue")]] string inputText, 
                                      [BlobOutput("containername/blobname")]TextWriter writer)

The framework also binds a TextWriter object to a blob named "blobname" in a container named "containername":

public static void ProcessQueueMessage([QueueInput("webjobsqueue")]] string inputText, 
                                      [BlobOutput("containername/blobname")]TextWriter writer)

The function then uses these parameters to write the value of the queue message to the blob:

writer.WriteLine(inputText);

As you can see, the trigger and binder features of the WebJobs SDK greatly simplify the code you have to write to work with Windows Azure Storage objects. All of the code required to handle queue and blob processing -- opening the queue, reading queue messages, deleting them when processing for them is completed, creating and writing to containers and blobs, etc., is done for you by the WebJobs SDK framework.

The WebJobs SDK provides many other ways to work with Windows Azure Storage. For example, the parameter you decorate with the QueueInput attribute can be a byte array or a custom type, and it is automatically deserialized from JSON. And you can use a BlobInput attribute to trigger a process whenever a new blob is created in your Windows Azure Storage account. Note that while QueueInput finds new queue messages within a few seconds, BlobInput can take up to 20 minutes to detect a new blob. (BlobInput scans for blobs whenever the JobHost starts and then periodically checks the Windows Azure Storage logs to detect new blobs.)

Monitoring programs that run in WebJobs

The WebJobs SDK provides a dashboard in Windows Azure Web Sites that you can use to monitor the status of the programs you run and examine any exceptions that they throw.

Dashboard

Limitations

As this tutorial is being written, WebJobs for Windows Azure Web Sites is a preview release, and the WebJobs SDK is an alpha release. They are not supported for production use.

In the WebJobs environment, programs run in the context of a Web Site and are not independently scalable. For example, if you have one standard Web Site instance, you can only have 1 instance of your background process running, and it is using some of the server resources that otherwise would be available to serve web content. For high volumes of backend work, consider using a worker role in a Windows Azure Cloud Service.

Prerequisites

Before you start, make sure you meet the following prerequisites:

  • Visual Studio 2013 or 2012, as indicated in Software Versions at the top of the page.
  • A Windows Azure subscription that you can manage. If you don't already have a Windows Azure account, but you do have an MSDN subscription, you can activate your MSDN subscription benefits. Otherwise, you can create a free trial account in just a couple of minutes. For details, see Windows Azure Free Trial.
  • Azure Storage Explorer is required for this tutorial (it's not required for using the WebJobs SDK).

Create a WebJobs project

To get started, you'll create a Console Application project, install the WebJobs SDK NuGet packages, and write the code that will use the WebJobs SDK framework to perform a background task.

  1. Open Visual Studio 2013 or Visual Studio 2013 Express for Desktop.

    (If you use the Express version of Visual Studio, use Express for Desktop because there is no Console Application template in Express for Web.)

  2. In the File menu, click New Project.

  3. In the templates pane, under Installed, expand Visual C#, click Windows, and select the Console Application template.

  4. Name the project WebJob, and then click OK.

    New Project dialog

  5. From the Tools menu click Library Package Manager and then click Package Manager Console.

    In the Package Manager Console window enter the following command:

    Install-Package Microsoft.WindowsAzure.Jobs.Host -pre

    Install the WebJobs package

    This also installs dependent packages, including another WebJobs SDK package, Microsoft.WindowsAzure.Jobs. (You use the other WebJobs SDK package separately only when you create your user functions in a separate DLL; for this tutorial you are writing all of your code in Console Application.)

  6. In Program.cs, replace the Main method with the following code:

    static void Main()
    {
        JobHost host = new JobHost();
        host.RunAndBlock();
    }

    RunAndBlock means the WebJobs SDK framework will run continuously in the current thread. The framework looks for any public static methods that have WebJobs SDK attributes and then watches for the triggers for those methods, such as new queue messages or new blobs. When a triggering event occurs, the framework calls the method.

    You can optionally call the RunOnBackgroundThread method to run the functions on a background thread.  You could call RunOnBackgroundThread more than once to easily create a multi-threaded batch process.

  7. Add a new method:

    public static void ProcessQueueMessage([QueueInput("webjobsqueue")] string inputText, 
                                           [BlobOutput("containername/blobname")]TextWriter writer)
    {
        writer.WriteLine(inputText);
    }

    The QueueInput attribute means that this method will be called when a queue message is received.  There are other trigger attributes, such as BlobInput which means the method will be called when a new blob appears in a specified container.

    The BlobOutput attribute binds a TextWriter object to blob "blobname" in container "containername", and the method body uses that object to write the queue message to the blob.

  8. Add using statements for the references to WebJobs SDK classes and the TextWriter class:

    using Microsoft.WindowsAzure.Jobs;
    using System.IO;
  9. Build the solution to save your work and make sure are no compile errors.

Create a Windows Azure Web Site and Storage Account

You can run your program locally, but the WebJobs SDK framework needs a Windows Azure Storage account for logging done by the framework and for your functions that work with queues, blobs, or tables. The framework and your functions can use the same storage account or separate ones. For this tutorial you'll use one storage account.

You aren't creating any custom web site content for this tutorial, but you'll need a Windows Azure Web Site to monitor your background process when you run it in Visual Studio, so you'll also create a web site in this section.

  1. Go to the Windows Azure Management Portal and sign in to work with the subscription you're going to use.

  2. Click New.

    New button

  3. Click Web Site -- Quick Create

  4. Enter a URL for the web site.

    The URL must be unique within the .azurewebsites.net domain.

  5. Choose the region closest to you, and then click Create Web Site.

    Create site

    Site created

  6. Click New -- Data Services -- Storage -- Quick Create.

  7. Enter a URL for the storage account.

    The URL must be unique within the .core.windows.net domain.

  8. If possible, choose the same region for the storage account that you chose earlier for the web site.

  9. Set Replication to Locally Redundant.

    When geo-replication is enabled for a storage account, the stored content is replicated to a secondary location to enable failover to that location in case of a major disaster in the primary location. Geo-replication can incur additional costs. For test and development accounts, you generally don't need geo-replication. For more information, see How To Manage Storage Accounts.

  10. Click Create Storage Account.

    Create storage account

    After a few seconds the storage account is created.

    Storage account created

Run the project locally

In order to connect to a storage account, the WebJobs SDK framework has to have a connection string with the storage account name and access key. In this section you'll put the connection string in the App.config file and run the WebJobs process locally. You'll test by using Azure Storage Explorer, a tool for viewing and manipulating Windows Azure Storage objects.

  1. In the Windows Azure Management Portal, select your storage account and click Manage Access Keys at the bottom of the page.

    Manage Access Keys button

  2. Copy the Primary Access Key.

    Copy primary access key

  3. In the App.config file, add the following connection strings, replacing <accountname> and <accesskey> with the values for your storage account.  The <accountname> value is the name you entered for the account, not the entire URL that you use to access items in the storage account.

    <configuration>
      <connectionStrings>
        <add name="AzureJobsRuntime" connectionString="DefaultEndpointsProtocol=https;AccountName=[accountname];AccountKey=[accesskey]"/>
        <add name="AzureJobsData" connectionString="DefaultEndpointsProtocol=https;AccountName=[accountname];AccountKey=[accesskey]"/>
      </connectionStrings>
        <startup> 
            <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
        </startup>
    </configuration>
  4. Press CTRL+F5 to run the application.

    The console output starts to slowly display a string of periods to let you know it's waiting for trigger events.

    Periods

The program is now listening for new queue messages on the queue named webjobsqueue in your storage account.  Now you'll use Azure Storage Explorer to create the queue and create a message in the queue.

  1. Open Azure Storage Explorer.

  2. In the Storage Account menu, click Add.

  3. In the Add Storage Account dialog box, enter your storage account name and your primary access key, and then click Add Storage Account.

    These are the same values that you plugged into the connection strings earlier.

    Add storage account

  4. With your storage account selected in the drop-down box, click blobs in the left pane.

    Notice that there is no "containername" blob container in the left pane.

    No blobs

  5. Click queues in the left pane.

  6. In the Queue pane at the bottom of the window, click New.

  7. In the New Queue dialog box, enter webjobsqueue as the Queue name, and then click Create Queue.

    Create queue

  8. In the left pane, click webjobsqueue, and then click New in the Message pane at the bottom of the window.

  9. In the Message content box, enter "Hello World!" and then click Create Message.

    Create queue message

    Soon you'll notice that the console window shows that your ProcessQueueMessage method was executed.

    Execution message

  10. In Azure Storage Explorer, in the left pane, click blobs. and then in the Container pane click Refresh.

    Now you see that there's a container named containername.

  11. Click containername to see the blobs in the container.

    The new container contains a new blob named blobname.

    New container and blob

  12. Double-click blobname.

  13. In the Blob Detail dialog box, click the Text tab to see your "Hello World!" message.

    Text in blob

  14. Close the command window that is running your WebJob project.

Run the project in Windows Azure

To deploy to Windows Azure, you zip and upload the contents of your bin/debug folder (or bin/release if you compile a Release build), and tell Windows Azure when you want the job to run. For this example we'll run the job continuously.

  1. In Solution Explorer, click Show All Files

  2. Expand the bin folder, right-click the Debug folder, and click Open Folder in File Explorer.

    Open debug in File Explorer

  3. Select all of the files in the folder, right-click the selected files, and click Send to -- Compressed Folder.

  4. Name the new .zip file WebJob.zip.

  5. In Windows Azure Management Portal, open your web site and click the Configure tab.

  6. Scroll down to Connection Strings.

  7. Under Connection Strings, enter AzureJobsRuntime in the Name field.

  8. Copy the connection string (without the quotation marks) from the App.config file, and paste it into the Value field.

  9. In the drop-down list click Custom.

  10. Click Save.

    Save connection string

  11. Click the WebJobs tab, and then click Add.

    Web Jobs tab

  12. In the Basic web job settings dialog box, enter HelloWorld as the name for your web job.

  13. Click the folder icon under Content, navigate to your bin\Debug folder, and then click Open.

  14. Leave How to Run set to the default option, Run continuously.

  15. Click the check mark at the lower right hand corner of the dialog box.

    New Web Job dialog

    WebJob running

  16. Open Azure Storage Explorer, and follow the procedures you did earlier to create a queue message, but this time enter Hello World from Windows Azure! as the message.

  17. Click the blobs tab, select containername container, double-click blobname blob, and select the Text tab to see the Hello World from Windows Azure! message in the blob.

    Hello World from Azure

  18. In the management portal, click the link in the Logs column for the HelloWorld web job.

    Before the WebJobs dashboard appears you might get a dialog box asking for credentials. If you do, perform the following steps to get the user name and password that you need to enter.

    • Click the Dashboard tab.
    • Under Quick glance, click Download the publish profile.
    • After you download the .publishsettings file, open it in Notepad or another text editor.
    • Find the first instance of userName= and userPWD=. These are the credentials you need. For example: <publishData><publishProfile profileName="webjobssite - Web Deploy" publishMethod="MSDeploy" . . . msdeploySite="webjobssite" userName="$webjobssite" userPWD="StjwBlJXnrfahHvbWs92MA6HKlvvLEZznr1EFA6kzSGKbY4v" . . .

    When the WebJobs dashboard appears, under Statistics you see a count of how many times your method was executed successfully and how many times it failed.

    Dashboard

  19. Under Invocation Log click Program.ProcessQueueMessage.

    Each time your method is executed, there is a log of its execution.

    Invocation log

  20. Click the containername/blobname link and you see the contents of the blob.

    Contents of blobname

Next Steps

You've now created and run a simple program that processes messages received from a Windows Azure Storage queue. You can do much more with a minimum amount of code and the WebJobs SDK. For example:

  • Bind to custom types serialized as JSON in the queue message.
  • Dynamically change output container and blob names based on properties of incoming queue message JSON objects.
  • Watch for blobs to appear in a container and process them.
  • Write to Windows Azure Storage Tables.

For more information about the WebJobs feature in Windows Azure Web Sites and the Windows Azure WebJobs SDK, see the following resources:

The WebJobs feature of Windows Azure Web Sites and the Windows Azure WebJobs SDK are in preview and not formally supported. Feedback will be considered in changes we make to future versions.

If you have questions that are not directly related to the tutorial, you can post them to the Windows Azure forum, the ASP.NET forum, or StackOverflow.com. Use #AzureWebJobs for Twitter. For questions and comments regarding the tutorial itself, see the comments section at the bottom of the page.

Please leave feedback on how you liked this tutorial and what we could improve. You can also request new topics and code samples at Show Me How With Code.

Author Information

Tom Dykstra

Tom Dykstra – Tom Dykstra is a Senior Programming Writer on Microsoft's Web Platform & Tools Content Team...

Rick Anderson

Rick Anderson – Rick Anderson works as a programmer writer for Microsoft, focusing on ASP.NET MVC, Windows Azure and Entity Framework. You can follow him on twitter via @RickAndMSFT.