Test Automation for Agile - A different strategy

Test Automation has always been a challenge when it comes to running your suites within the time frame especially in an Agile mode with more builds coming the testing way.

So here is a quick strategy that we tried and seems to be working well for now:

What it used to be:
  • Entire test automation runs for 8-10 hours
  • One big test suite or a few smaller test suites
  • Reusable code being used a lot (which was good , but also had a repercussion of redundant flows)
  • Robust suites which broke less but after complete runs the failures where not always application bugs but automation errors.
  • The whole idea was how much test coverage do we have
  • Metrics were based on number of tests run/passed/coverage and so on..

What it is now:
  • Continuous Integration Builds
  • Test suites do not run that long instead there are dedicated 1 hour suites (To keep the suite below the 60 minute time frame and also have the apt coverage , these would be targeted suites for a particular functionality/stories and not cover anything that the previous suite did)
  • The idea is that these one hour suites would be targeted suites and at any cost will not run over an hour on a single automation box
  • The one hour suites will not have any redundant flows nor will there be at any time more than 5 such suites in our latest test set.
  • The one hour suites will have less than 5% automation defects and should be minimized to almost zilch.
  • Metrics are much better now and the tests actually fail only when there is a genuine defect and not an automation failure.

Comments and suggestions are always welcome , until then Happy testing!


Export Test Steps from QC to excel

  1. Go to Dashboard in QC and under Analysis Tab , select the option New Excel Report.
  2. Enter the below query in the Query Area:
          and TEST.TS_TEST_ID in (xxxx,xxxx)

    3.   Click on the Generate button

Note: xxxx,xxxx should be replaced by test case ids. You can know your test case ids from the test plan module.


Skillset - Test Automation Specialist

Note: the Skillset category is just to gauge where you stand in terms of the skills required for the role

Yet another interesting test automation profile for an australian organization:

Key duties of the role include:

•Review requirements, specifications, user documentation, project documentation to assure quality of the products and tests to be developed; i.e., perform manual static testing.

•Select and develop appropriate test automation tools, applying the latest techniques in test automation; e.g., data-driven testing.

•Use risk-driven techniques to develop, maintain, and execute automated test suites for various Software products; i.e., perform automated dynamic testing.

•Work with the Development team to capture and reuse automated Unit Test Cases, Test Stubs and Drivers, and other Development test objects.

•Work with the Release Engineering team to create and maintain an automated nightly build verification (“smoke”) test.

Successful candidates possess extensive experience in UI test automation roles, including proven skills in the following areas:

Programming experience required (Java, VBScript or .Net preferred)

•Experience with designing and scripting functions for business process flow testing

•Experience building, maintaining and expanding upon test automation framework delivery

•Experience and accreditation in HP ALM/QTP, Selenium, IBM RQM/RFT, or MS TFS.

•Experience in Selenium, Cucumber or other proven open source automation tool sets

•Experience in continuous integration build automation with Jenkins (Hudson) or similar tools

•Ability to clearly and succinctly express your opinions and ideas verbally and written

•Ability to communicate and negotiate with stakeholders

•Creation of strategic test plan documents for test automation delivery


The global market for testing services - pegged at $3.4 billion

An article in today's Hindu , a local newspaper in India states the following:

  • The global market for testing services, currently pegged at $3.4 billion, is set to witness an eight-fold increase to $ 24 billion in the next five to six years
  • Shortage of skilled personnel as aspirants continue to prefer software/programme development
  • Testing services is witnessing a compounded annual growth rate (CAGR) of close to 30 per cent compared to development which is around 12 per cent



Configure QTP and Firefox 10.0.3

  1. Un-install any previous instances of Firefox
  2. Install Firefox 10.0.3 by opening the file "WX7-FireFoxESR-10-0-3-R1.EXE" in "Run as Admin" mode
  3. Restart the system
  4. Install the QTP Patches in the following sequence:
    1. QTPWEB_00090.EXE
    2. QTPWEB_00092.EXE
  5. Info about insalled Patches would be available at: C:\Program Files (x86)\HP\QuickTest Professional\HotfixReadmes


HTML mail body content is garbled/distorted with CDO.Message object

This is a common thing that you might see when you are trying to create an automated mail using the CDO.Message object.

Set cdoObj = CreateObject("cdo.message")

Set iConf = CreateObject("CDO.Configuration")
Set iFields = iConf.Fields
iFields.Item("") = 2
iFields.Item("") = "mailhost"
Set cdoObj.Configuration = iConf
cdoObj.Subject =”Test Email”
cdoObj.CreateMHTMLBody "C:\temp\Temp.html"
cdoObj.To = "TO EMAIL ID"
cdoObj.From = "FROM EMAIL ID"
cdoObj.BodyPart.Charset = "utf-8"

Now once you automate and recieve the email , you will see that the HTML source of received email has “!” characters at different location due to which HTML Tags are affected.
Temporaray solution: “vbNewLine” before ever tag

Better solution:  Use .HTMLBodyPart.ContentTransferEncoding = "quoted-printable"
which will remove all references of ! within the text inside the table.


A bug that costed half a million $ and more

Here is a case that shows what a software bug can cost especially when the bug is directly linked to making money.  Imagine being at a casino and finding a bug that can help you get rich in minutes, this is what happened when a Las Vegas local named John Kane discovered a firmware bug present in 10 casino machines at  the Silverton Casino Lodge.The glitch had been hidden for seven years.
The management had been suspicious of Kane for quite some time. On his last day  of  winning, In about one hour he scored five jackpots, and after Kane didn’t collect his last win of $8,200, the Gaming Control Board removed the machine and took it to the lab for testing.

"There they discovered the secret behind Kane’s lucky streak: he was exploiting a previously-unknown firmware bug present in the Game King and nine other IGT machines"

"Now Kane and the bug he exploited are at the center of a high-stakes legal battle before a federal judge in Las Vegas. The question: was it a criminal violation of federal anti-hacking law for Kane and a friend to knowingly take advantage of the glitch to the tune of at least half-a-million dollars? Prosecutors say it was. But in a win for the defense, a federal magistrate found last fall that the Computer Fraud and Abuse Act doesn’t apply, and recommended the hacking charge be dismissed. The issue is now being argued in front of U.S. District Court Judge Miranda Du, who’s likely to rule this month."

Read further :