NUnit Deployment Publisher

This publisher builds on the ExecuteDeploymentPublisher to provide additional NUnit processing once the publisher has downloaded the NuGet package.

When a matching trigger HealthCheck result notification is published (see "TargetHealthCheckName" in Configuration section below for more detail) this publisher executes a three step process (The following "Items" in quotes refer to Configuration settings),
  1. The NuGet package containing your test binaries is downloaded from the "Feed" and unpacked into the "DownloadFolder"\"PackageId" folder.
  2. The "Command" is then executed (optionally using identity provided by "Domain", "UserName", "Password") and this is expected to be a command line that will execute an NUnit runner which will produce a test results xml file.
  3. If the command executed successfully it will attempt to load and parse the xml results file specified in "ResultsFile"; a new HealthCheck notification result is published containing the result counts (total, failed, error etc). The Result property of this notification is determined by any failures, errors or invalid tests - if there are none then the Result is "Success" (eg: all tests passed).

Typical Usage

This publisher is intended for use in automated testing, smoke/system test scenarios as part of a Continuous Delivery/Deployment process. To enable this you must first package your tests as a NuGet package - this should be done as part of your CI build process and details on this can be found on the official NuGet website. As part of your deployment process you just simply need to publish this NuGet package to your feed (private/MyGet/official).

A great tip here is to make NUnit.Runner a dependent package of your NuGet package - when this plugin downloads your NuGet package it will also pull down any dependencies so you don't have to pre-deploy the nunit test runner .exe to the target machine or package it inside your package!

Wolfpack provides a HealthCheck plugin that will detect when a new version of a NuGet package has been released - this provides the perfect trigger notification for us to connect this publisher to. More details and instruction for use can be found here. Once set up, test it out; publish a new version of your package and you should receive a Wolfpack notification informing you that "version xyz of package ABC is available"....ok, we have our trigger HealthCheck providing the appropriate notification, next step is to configure this publisher.

Once this is complete publish another new version of your package - this time not only do you receive a notification that a new version is available but this publisher should also execute, downloading the package and executing the tests and hopefully generating another HealthCheck notification containing the test results!

You can have as many of these publishers running as required, just clone the configuration file and adjust the settings to suit. You must make the component "id" attribute and "FriendlyId" element unique though otherwise the configuration will be invalid and remember to set up a new HealthCheck to act as your trigger.

Configuration

Once installed a new configuration file will be available, Config\Publishers\nunit.deployment.contrib.castle.config...
<component id="NUnitDeploymentPublisherConfiguration"
			 lifestyle="singleton"
			 type="Wolfpack.Contrib.Deployment.NUnit.NUnitDeploymentConfig, Wolfpack.Contrib.Deployment">
	<parameters>
		<Enabled>true</Enabled>
		<FriendlyId>NUnitDeploymentPublisher</FriendlyId>
		<PublishOnlyIfFailure>False</PublishOnlyIfFailure>
		<Domain></Domain>
		<UserName></UserName>
		<Password></Password>
		<Command>NUnit.Runners\tools\nunit-console-x86.exe</Command>
		<Args>wolfpack.contrib.demo\lib\net40\wolfpack.contrib.demo.dll /include=DeploymentPassing</Args>
		<ResultsFile>TestResult.xml</ResultsFile>
		<Feed>http://www.myget.org/F/wolfpack/</Feed>
		<PackageId>Wolfpack.Contrib.Demo</PackageId>
		<DownloadFolder>_deploy\demo</DownloadFolder>
		<!-- This is the friendlyId of the NuGet Release Check to monitor -->
		<TargetHealthCheckName>NuGetReleaseHealthCheck</TargetHealthCheckName>
	</parameters>
</component>
  1. To enable the publisher just set the "Enabled" element value to "true" on the component.
  2. If you are only interested in receiving alerts when the unit tests have a failure/error then you can set the "PublishOnlyIfFailure" element to true otherwise you will receive alerts for both success and failure.
  3. "Domain", "Username" & "Password" elements are used to impersonate the identity if the executable needs to be run as someone else.
  4. "Command" element is used to provide the nunit command line to execute to run the tests. This can be absolute or relative - if relative then it is relative to the "DownloadFolder" element value.
  5. "Args" is used to specify the arguments to pass to the nunit runner.
  6. "ResultsFile" is the filename of the nunit xml results file. This can be an absolute filename, if relative then it is so to the "DownloadFolder" element.
  7. "Feed" should be set to the location hosting the NuGet package - leave blank for the official NuGet feed (default).
  8. "PackageId" is the name of the NuGet package to download.
  9. "DownloadFolder" is the folder where the package will be downloaded to. A sub-folder, named with the PackageId will be created and the contents of the NuGet package will extract here. This folder can be absolute or relative, if relative it is to the Wolfpack installation folder.
  10. "TargetHealthCheckName" is the "FriendlyId" of the HealthCheck that this publisher triggers on. Whenever a HealthCheck result notification is published, this publisher will inspect the metadata and if the source HealthCheck matches the "TargetHealthCheckName", the Result is "Success" and there is no Critical Failure it will execute.
  11. Finally, to trigger this deployment publisher we need to enable the "NuGet Release HealthCheck" plugin that was also installed. In the Config\Checks folder should be "triggerdemo.contrib.castle.config". Move this file to the appropriate subfolder, eg: EveryMinute and set the "Enabled" property to true, the remaining properties are already set up to point to the "Demo" package (see below).

Demo

The configuration file contains the settings to download a demo nuget package "Wolfpack.Contrib.Demo" from myget.org. This package contains two dummy nunit tests, one that fails and one that passes. This package also has NUnit.Runner as a dependent package so when Wolfpack downloads it it will also download this package (it also downloads dropkick).

Output

 
nunit-deployment.png
This publisher, once executed will create a new HealthCheck result notification with the following properties set...
  • Tag = name of package updated
  • Duration
  • Result = true if no test failures/errors otherwise false
  • ResultCount = total number of tests executed

Properties...
  • PackageVersion = new version of the package installed
  • ResultCode = result/return code from the nunit console .exe
(The following are test result counts)...
  • Total
  • Errors
  • Failures
  • NotRun
  • Inconclusive
  • Ignored
  • Skipped
  • Invalid

Last edited Aug 22, 2012 at 12:21 PM by jimbobdog, version 20

Comments

No comments yet.