Test Case Management Software Integration

Mixing automated and manual testing (Part 2)

Mixing automated and manual testing (Part 2) : Part 1

This is a two part post on mixing automated and manual testing. In short, I cover solutions to two problems:

  • Converting selenium tests to end-to-end unit tests
  • Reporting end-to-end test via test case management software integration

Test Case Management Software Integration

In my previous post, I gave a few tips on transforming end-to-end selenium test into automated unit tests. This post will be more technical as it covers how to integrate your automated end-to-end selenium-based unit tests into your Test Case Management (TCM) software. TCM software allows manual testers to create test cases, suites, and runs to track the results of manual testing. It’s a great tool when your team needs to do a lot of manual testing. In this example, I’m going to integrate with TestRail.

There are quite a few neat tricks to do this:

  1. Get the names of your unit tests
  2. Add new test run with test cases
  3. Update the run’s test case at the end of each test

Get the names of all test case

Get the test names is a matter of inspecting all methods in the assembly with the TestMethod attribute

public List<string> GetAllTestMethods()
{
	var methods = this.GetType().Assembly.GetTypes()
			  .SelectMany(t => t.GetMethods())
			  .Where(m => m.GetCustomAttributes(typeof(TestMethodAttribute), false).Length > 0)
			  .ToArray();

	var retval = new List<string>();
	foreach (var method in methods)
	{
		retval.Add(method.Name.Replace("Void","").Replace("()",""));	
	}
	return retval;
}

Add new test run with test cases

In this example, I’m using a testRailClient which wraps the REST API methods. Here, I’m doing the following:

  1. Deleting the e2e (end-to-end) test case section
    1. A section is a logical grouping of test cases
  2. Creating a new e2e section
  3. Adding the new cases to the e2e section
  4. Creating a new e2e run with the current timestamp
  5. Adding all the e2e test cases to the run.
  6. Finally, getting a reference to all TestRail run cases.
public void AddMissingCases()
{
	var methodNames = GetAllTestMethods();
	var sections = testRailClient.GetSections(1, 1);
	var section = sections.First(x => x.Name == "e2e");
	testRailClient.DeleteSection(oldSection.ID.Value);
	var section = testRailClient.AddSection(1, 1, "e2e");
	var cases = testRailClient.GetCases(1, 1, section.ID.Value);
	bool changed = false;
	foreach (var methodName in methodNames)
	{
		if (cases.All(x => x.Title != methodName))
		{
			testRailClient.AddCase(section.ID.Value, methodName);
			changed = true;
		}
	}
	if (changed)
		cases = testRailClient.GetCases(1, 1, section.ID.Value);
	var casesToRun = cases.Select(x => x.ID.Value);
	var hs = new HashSet<ulong>(casesToRun);
	var hs = new HashSet<ulong>() {casesToRun.First() };
	var res = testRailClient.AddRun(1, 1, "e2e - " + DateTime.Now, "", 0, null);
	if (!res.WasSuccessful)
	{
		throw new Exception("Unable to create Test Run",res.Exception);
	}
	var update = testRailClient.UpdateRun(res.Value, null, null, null, hs);
	allTests = testRailClient.GetTests(res.Value);

}

Update the run’s test case at the end of each test

MSTest has a great feature where you can inspect the TestContext in the TestCleanup method to get the overall result at the end of each test. It’s here that I update TestRail

private TestContext testContext;

public TestContext TestContext
{
	get { return this.testContext; }
	set { this.testContext = value; }
}

[TestCleanup]
public void CleanUp()
{
	var methodName = TestContext.TestName;
	bool fail = TestContext.CurrentTestOutcome == UnitTestOutcome.Failed;
	AddTestResult(fail,methodName);
	testRailClient = null;
}

protected void AddTestResult(bool fail, string methodName)
{
	var test = allTests.First(x => x.Title == methodName);
	var result = testRailClient.AddResult(test.ID.Value, fail ? ResultStatus.Failed : ResultStatus.Passed);
}

I hope you find this helpful. It’s useful to integrate your end-to-end automated tests into your TCM software.

Wm. Barrett Simms
Follow Me
Latest posts by Wm. Barrett Simms (see all)