Create a Build Task for TFBuild

There are many articles around how to build tasks for TFS 2015’s newer build system (TFBuild). This is yet another one that tries to consolidate the information floating around into a central location and includes information for deploying to an on-premise TFS server.

Setup

Before you can start you need to have TFS 2015. TFS 2013 would also work but some changes may need to be made. This article is going to use an on-premise server. If you are targeting Visual Studio Online then the deployment process will be different. You will also need a solution in source control to test against.

The biggest piece that is probably missing is tfx which is the node.js-based cross-platform tool for working with TFS. You need to have it installed in order to create TFS build tasks. Follow the setup instructions on their site to download and run the program.

If you haven’t already then go ahead and open a command prompt to tfx as you’ll need it. If you followed the steps online (as of Dec 2016) then tfx would have been placed in your AppData\Roaming\npm path. Both this path and npm are needed so if you didn’t add npm to your path you’ll want to create a batch script that includes these paths. Personally I modified the existing Developer Command Prompt’s VSDevCmd.bat file to include both paths.

...
@set VisualStudioVersion=14.0

@rem Adding TFX and NPM
@set PATH=%PATH%;;%AppData%\npm
@goto end

Once that is done, open the command prompt and type tfx to verify it is loaded correctly.

Creating a Simple Build Task

To get started you need to create the task folder structure. For simplicity use tfx to create it.

tfx build tasks create

You will then be prompted for a set of values. Choose these carefully.

Short name: The name of the task. Should be short and not contains spaces. Used to create the folder structure and identify the task. (i.e. ShowVariables)
Friendly Name: The name as shown to the user. Should be easily identify in the available tasks. (i.e. Show Variables Task)
Description: A description of the task. (i.e. Displays the defined variables.)
Author: The author that is shown to the user. (i.e. P3Net)

The task will be created in the current directory you were in using the short name you gave. Open the folder and look at the generated files.

icon.png: This is a standard icon for the task. In general you should create your own icon for your task. There is a default icon that is provided online if you need it.
sample.js: This is a JavaScript sample implementation.
sample.ps1: This is a Powershell same implementation.
task.json: This is the task manifest and will need to be updated to include the information needed to create, publish, install and use the task.

Tasks can be implemented using JavaScript, Powershell or both. For our sample we will only use Powershell so you can delete the JavaScript file. Go ahead and rename the sample.ps1 file to showVariables.ps1 so it matches the task.

Task Manifest

Go ahead and open the task.json file. Most of this information is required but a few items are worthy of special mention. The documentation is available here.

id: The unique ID of the task.
name, friendlyName, description, author: The values provided when creating the task.
category: The category that the task will show up in when defining a build.
visibility: When the task will be visible. For a build task the visibility should include Build and Release.
demands: Any custom demands that this will require. (i.e. test)
version: This is the version number and will need to be updated for each build.
instanceNameFormat: This is the default name the task will get when added to a build. You can change it to be more descriptive.
inputs: This is used to define parameters in the UI and will be discussed later.
execution: This determines what files get run.

The execution section contains the set of JavaScript and Powershell files to run. The names of the files can be anything provided you update the manifest file. Since we aren’t using JavaScript go ahead and remove the entire Node section. Here’s our final version.

{
 
   "id": "ddbf4530-b8b5-11e6-880c-5d93d84789b1",
  
   "name": "ShowVariables",

  "friendlyName": "Show Variables Task",
  
  "description": "Displays the defined variables",
  
   "author": "P3Net",
  
   "helpMarkDown": "Replace with markdown to show in help",
  
   "category": "Utility",
  
   "visibility": [
    
      "Build",
    
      "Release"
 
   ],
 
   "demands": [],
  
   "version": {
   
      "Major": "0",

      "Minor": "1",

      "Patch": "0"

    },
  
    "minimumAgentVersion": "1.95.0",
  
    "instanceNameFormat": "ShowVariables",
  
    "inputs": [],
  
    "execution": {
    
        "PowerShell3": {

           "target": "showVariables.ps1"

         }
  
   }

}

Note the use of PowerShell3 for the execution. Powershell is also a supported value but it tends not to work correctly when it comes to parameters.

Word of warning here: if there are errors in the JSON schema then they won’t get detected until you try to install the task. Therefore be very careful about creating this file.

Defining the Powershell Script

Defining the Powershell script for TFS is no different than any other script you might write. Even better is that you can test your script without even using TFS. Once the script is working the way you want you can then integrate it with TFS. Go ahead and open the example Powershell script that was generated.

The script is already set up to accept parameters, which we’ll add later. It also contains a try-catch block to wrap the invocation of the script. Our simple script will simply display the available environment variables so go ahead and remove everything inside the try block.

[CmdletBinding()]
param()

# For more information on the VSTS Task SDK:
# https://github.com/Microsoft/vsts-task-lib
Trace-VstsEnteringInvocation $MyInvocation
try {    
} finally {
    Trace-VstsLeavingInvocation $MyInvocation
}

We’re using Write-Output here just like we do in a regular script. TFS will capture this and put it in the output log.

VSO Commands

Before going any further it is useful to note that TFS accepts commands using the standard Write commands but with well-defined formats. Specifically if any output is generated using the following syntax then it is interpreted by TFS differently.

##vso[some command property=value]Something else

The ##vso is what lets TFS know there is something to do. The value inside the brackets is the command. Logging occurs this way using the following command.

##vso[task.logissue type=error]Displaying an error

VSTS SDK

TFS has some pre-defined commands available to make it easier to use in Powershell and JavaScript. They are defined in the VSTS Task SDK. Here’s a sample of some of the commands you can use.

Assert-VstsPath
Find-VstsFiles
Get-VstsInput
Get-VstsTaskVariable
Get-VstsTfsService
Trace-VstsEnteringInvocation
Trace-VstsLeavingInvocation
Write-VstsTaskError
Write-VstsTaskWarning

To use these in Powershell you need to download the module and then import it into Powershell. Refer to the documentation on how to do that. The use of the SDK is optional. It does provide some useful functionality but to use it you have to ship the SDK as part of your code. One thing to be aware of if you use the SDK is that you need to change the path it installs to. By default when it downloads it’ll create a VstsTaskSdk\{version} folder. You need to move the VstsTaskSdk folder to the ps_modules subfolder. You also need to move everything in the version-specific folder up to the VstsTaskSdk folder otherwise they will not be found at runtime.

Adding Parameters (Powershell)

If your script needs data to run then you’ll want to define some parameters. TFS will pass parameters to your script based upon the entries in the inputs section of your task.json file. Additionally you can use any of the predefined variables provided by TFS. In most cases they can be used as the default value for parameters for your script. Let’s add some parameters to our task.

currentPath: The current working directory initialized by using the pre-defined SYSTEM_DEFAULTWORKINGDIRECTORY variable.
showEmpty: Determines if we should print empty variables.
logLevel: An integer value where 0 means verbose and 1 means debug.
message: A string indicating the starting message to show, if any.

[CmdletBinding()]
param(
    [string][Parameter(Mandatory=$true)] $logLevel,
    [bool][Parameter(Mandatory=$true)] $showEmpty,
    [string] $message = '',
    [string] $currentPath = $env:SYSTEM_DEFAULTWORKINGDDIRECTORY
)

We don’t have to provide values for all the parameters but any we want to set for a build have to be exposed as part of the task. To do that we need to add the parameters to the inputs section of the task.json file. Within the section we need to provide the following information.

name: The name of the parameter as defined by the script (excluding the $).
label: The display name of the parameter as shown to the user.
type: The type of the parameter (see below).
defaultValue: The optional default value to use.
required: Boolean indicating whether this parameter is required or not.
helpMarkDown: Help markdown describing the parameter.
groupName: The optional group a parameter is contained in (see below).

It is important to ensure that the name property matches the parameter name in the script otherwise it will not be properly passed to the script. The type property can be any supported type. Here is the currently supported types.

boolean: A true or false value.
string: A text value.
multiLine: A multi-line text value.
filePath: A file path.
radio: A set of options defined using the option child object.
pickList: A list of options to choose from using the option child object.

To group inputs together you can use the groups section. This should be done before the inputs section. Groups are useful for grouping related settings (i.e. basic settings that should always be set vs. advanced settings that should rarely be set). Here are the core properties for a group.

name: The name of the group as will be used in the inputs section.
displayName: The text shown in the UI.
isExpanded: Boolean value indicating whether the group should be expanded by default. This is useful for required groups but not optional groups.

Here’s what the sections would look like for the parameters defined earlier.

"groups": [
{
	"name": "basic",
	"displayName": "Required",
	"isExpanded": true
},
{
	"name": "advanced",
	"displayName": "Optional",
	"isExpanded": false
}
],
"inputs": [
{
	"name": "logLevel",
	"type": "pickList",		
	"label": "Log Level",
	"defaultValue": "Debug",
	"required": true,
	"helpMarkDown": "The type of logging to do.",
	"groupName": "basic",
	"options": {
		"Normal": "Normal",
		"Verbose": "Verbose",
		"Debug": "Debug"
	}
},
{
	"name": "showEmpty",
	"type": "boolean",
	"label": "Show Empty Values",
	"defaultValue": "true",
	"required": true,
	"helpMarkDown": "Determines if empty values are shown.",
	"groupName": "basic"
},
{
	"name": "message",
	"type": "string",		
	"label": "Message",
	"defaultValue": "",
	"required": false,
	"helpMarkDown": "The optional message to show.",
	"groupName": "advanced"
}
]

Using the Parameters

Now that we have defined the parameters we can implement the core of the try block.

function Display
{
    param(
        [string][Parameter(Mandatory=$true)] $msg
    )
    
    if ($logLevel -eq "Debug") {
        Write-VstsTaskDebug $msg    
    } elseif ($logLevel -eq "Verbose") {
        Write-VstsTaskVerbose $msg
    } else {
        Write-Output $msg
    }
}

Trace-VstsEnteringInvocation $MyInvocation
try {
    # Display optional message
    if (-not [String]::IsNullOrEmpty($message)) {
        Display $message
    }

    Write-VstsTaskDebug "logLevel = $logLevel"
    Write-VstsTaskDebug "showEmpty = $showEmpty"

    # Display variables
    $count = 0
    $vars = Get-ChildItem env:
    foreach ($var in $vars) {
        if (-not [String]::IsNullOrEmpty($var.Value) -or $showEmpty) {
            Display "$($var.Name) = $($var.Value)"
        }
        $count++
    }    
} finally {
    Trace-VstsLeavingInvocation $MyInvocation
}

Notice that we log the parameters for debugging purposes.

Adding Parameters (VSTS)

I have had quite a bit of problems trying to get mandatory parameters to work properly in Powershell. For one task it just works but for another TFS may refuse to pass the parameters along. This is almost no pattern to when it works. I have gone so far as taking a working task, stripping out the guts and suddenly it fails to get parameters again.

Fortunately there is a more reliable approach provided by VSTS, Get-VstsInput. This command retrieves the input values from the script block. Like a regular parameter you can specify that a parameter is required and convert to boolean or int. To use this approach remove all the parameters from the param block but leave it in. Then call the command for each parameter. Here’s our updated changes.

[CmdletBinding()]
param()

# Get inputs
$logLevel = Get-VstsInput -Name logLevel -Require
$showEmpty = Get-VstsInput -Name showEmpty -AsBool -Require
$message = Get-VstsInput -Name message
$currentPath = $env:SYSTEM_DEFAULTWORKINGDDIRECTORY

This approach seems more reliable and therefore is my recommended approach.

Creating Build Variables

Sometimes it is useful to pass data back to TFS so that it can be used in later build steps. This can be done by setting a new build variable. For Powershell this is done by using the Set-VstsTaskVariable command or by writing ##vso[task.setvariable variable=$name;]$value to the output. As an example we’ll return the total number of environment variables in the p3net_environmentVariableCount variable. Add the following code after the loop.

# Set p3net_environmentVariableCount to the # of variables
Set-VstsTaskVariable -Name "ps3net_environmentVariableCount" -Value $count

Any variable set using the above method will be available to the rest of the build system. This makes it useful for passing data between tasks. Note that, unlike the pre-defined variables, these will be exposed as environment variables and therefore should be valid, unique identifiers. Also note that there are some places where they aren’t currently accessible. I had an issue trying to use one as part of the source label after a successful build.

Packaging

We have a task defined but now we need to wrap it in an extension so that we can upload it to TFS. Using an extension eliminates the need for setting up PATs and other security related features that were previously used.

A single extension can contain any number of tasks so we’ll want each task to be in its own folder. Go to the root folder where your task is defined and create a new folder for the extension (i.e. P3NetBuildExtensions). Then move the task folder into the new folder.

We now need to create the extension manifest file. Create a new file in the root extension folder called vss-extension.json.Then open the file. The documentation for this file is here.

id: This is your extensions unique ID and should probably contain your company name. Note that dots aren’t allowed and it is currently limited to 100 characters. (i.e. p3net-buildextensions)
version: This is the extension version and will need to be updated each time the extension is built.
name: This is the user-friendly name of the extension. (i.e. P3Net Build Extensions)
publisher: This is the publisher and needs to match your publisher information when using Visual Studio Online. For an on-premise TFS it doesn’t matter what you set it to.
targets: This is a list of products that you need. (i.e. Microsoft.VisualStudio.Services)
description: This is an optional description of your extension.
categories: The list of categories (limit 3) this extension falls under. Note that case matters here. (i.e. Build and release)
content: This is the content of the extension and should include at least an overview.md describing the extension.
scopes: This defines the OAuth scopes your extension needs. The available scopes are defined here. (i.e. vso.build, vso.build_execute)
files: The list of files and paths that contain the extension contents. There will be an entry here for each task.
contributions: This is an open element that allows you to define different contribution points. For build tasks we have to specify each build task.

Here’s a sample.

{
    "manifestVersion": 1,
    "id": "p3net-buildextensions",
    "version": "0.1.0",
    "name": "P3Net Build Extensions",
    "publisher": "p3net",
    "targets": [
        {
            "id": "Microsoft.VisualStudio.Services"
        }	
    ],
    "scopes": [
        "vso.build",
        "vso.build_execute"
    ],
    "description": "Build extensions from P3 NET",
    "categories": [
        "Build and Release"
     ],
    "icons": {
         "default": "extension-icon.png"
    },
    "content": {
        "details": {
            "path": "overview.md"
        }
    },
    "files": [
    {
        "path": "ShowVariables"
    }
    ],
    "contributions": [
    {
        "id": "p3net-showvariables-task",
        "type": "ms.vss-distributed-task.task",
        "targets": [
            "ms.vss-distributed-task.tasks"
        ],
        "properties": {
            "name": "ShowVariables"
        }
    }
    ]
}

Adding Build Tasks to an Extension

The formal process is documented here. The files element must specify each file or path that contains content for the extension. Each task will need to have its folder included here.

The contributions element is where we define each task so it can be added to the available build tasks.

id: This is the unique ID of the task. It only has to be unique within this extension and doesn’t need to match the actual task in any way.
type: This must be ms.vss-distributed-task.task.
targets: This identifies the target of the contribution and must be ms.vss-distributed-task.tasks.
properties/name: This is the name of the task and must match the folder name used for the task.

We are now ready to generate the extension. Using tfx execute the following command from the directory where the vss-extension.json file resides.

tfx extension create --manifest-globs vss-extension.json

If everything was successful a .vsix file is generated with your extension name and version number. If there were any errors then they will be shown. You are now ready to deploy it.

A Note on Versioning

It is important that you ensure that each time you prepare to deploy a change to your extension that you increment the version number. For minor changes increment the minor number. For fixes increment the patch. For major changes increment the major number.

Currently TFS will automatically use the latest version of a task. But this can cause compatibility issues so a change has been announced that will allow builds to target a specific task version. Any changes to the minor/patch number will cause any existing builds to use the updated version. But a change to the major number will require each build to explicitly opt into the change.

Another important thing to remember is that there are two versions: extension and task. In general your extension should update its version each time you deploy. A task only needs to update its version if it changed. But it may be a good idea to keep the two values in sync.

Deployment

Now that the extension is built you’ll want to install it into TFS. Unfortunately TFS makes it confusing to find the right page to upload it to. All the documentation mentions using the Manage Extensions page but there are several of these. The one you’re looking for is extensions for the collection. Here’s how to get there from the main TFS home page (with the new UI treatment).

Click the icon to go to the TFS Admin page.
Click the marketplace icon next to your profile and select Browse TFS Extensions.
Scroll to the bottom of the page and find the Manage Extensions button and click it.
For a new extension click the Upload New Extension button to upload the extension.
Find your .vsix file and upload it.

If the extension is already installed then you can select the dropdown menu to the left of the extension and select Update. However, at least with TFS 2013, this does not work correctly. If you are in this situation, remove the extension first, then upload it again.

Once the extension is uploaded it can be installed to the collection(s) you want to use it for by using the Install button.

Using in a Build

Now you need to test your task in a build so edit a build definition. Add a new build task, go to the Utility tasks and select the new Show Variables Task, then click Add. The two groups of variables should be shown and you should be able to set their values. Save the build definition as a draft and then queue a build. When the build runs you should see the task get called and the variables get written to the log.

The easiest way to debug your task is to simply debug it using Powershell ISE as normal. You can specify the parameter values as needed. But if you run into issues when running under TFS itself then you can set the system.debug variable in the build to true. This will generate more debugging information to help diagnose issues.

Next Time

In the next post I’ll show you how to add another task to the extension.

Download the code.