Category

Per Unit Return

Quantity

Total in

USD

Category

Per Unit Cost

Quantity

Total in

USD

Savings on cost of fixing defect found in production

$10,000 per defect

100 defects

$1,000,000

Resource cost

$34/hour (blended cost)

10 testers for 6 months

$342,000

Cost of fixing defects reported during testing

$1500 per defect

100 defects

$150,000

Defect leakage

$10000 per defect

10 (at 10% leakage)

$100,000

Rejected Defects analysis

$500

per defect

10 (at 10% rejection)

$10,000

Duplicate defects removal

$100 per defect

5 (at 5% duplication rate)

$500

Issue resolution

$100 per issue

250 issues

$25,000

Deliverable & work review

$100 per day

125 working days

$12,500

Communication overhead

$100 per day

125 days of project

$12,500

Training Cost

$1000 per BA day

10 (BA days)

$10,000

Attrition (ramp-up) Cost

$272 (daily blended rate times hours-34*8)

10 days

$2,720

Travel cost

2 onsite resources

$3000 per resource

$6,000

Helpdesk calls

$60 per call

50 (5 calls per resource)

$3,000

Bandwidth

$1000 per month

6 months of IPLC

$6000

Remote desktop /VMWare

$400 per VMWare for 6 months

8 offshore resources

$3200

Governance Cost

4% of resource cost

Resource cost- $342,000

$13,680

Total

$1,000,000

Total

$697,100

Return

$1,000,000.00

Investment

$697,100.00

ROI

0.43

N No incentive for maximizing defect removal

The best testers find most number of defects, find them at earliest stage, uncover the most difficult to find bug and focus on most likely application usage scenario first. The testing contracts may have productivity clause at the most. However tester's productivity per se is of limited use to business. A tester executing 100 test cases per day is of no consequence if no defect is reported. Typically there is no clause related to defect finding capability of team.

Lack of risk sharing in agreements

All organizations assume calculated risk in testing. Testing all possible conditions despite the reduction achieved via techniques such as Orthogonal array testing is always a huge ask. As such, the scope of testing is never 100%. The scope related risk or any other risk such as of defect leakage are entirely borne by customers.

Lack of incentive to retain the team

Productivity gains and cost reductions are achieved over long period of time as the processes are stabilized and the resources acquire application and domain knowledge. However, once the resources reach the peak of knowledge they are lost as their career cannot be furthered inside a project. Promoting a resource engaged in project, reduces the project margin as promotions are mostly unplanned events. As such it is not feasible to retain the resource for long term. This distortion is once again fueled by flat rate structure of contracts.

Lack of promotion of excellence

There is lack of clear articulation in contract on who pays for training. Excellence is not related to individual's performance targets. Excellence is not defined, even when forming a Testing Center of Excellence (TCOE). It is not defined how a TCOE testing project would be superior to a stand-alone testing project.

Inappropriate productivity measurements

Test case based productivity measurements invites exercise to reduce number of steps in a test case and thus boost up the apparent productivity of the team.

Skill & experience based payment

Skill & experience based rates provides best possible talent for the price paid..

Collaborative team pyramid construction

Vendors typically construct team pyramid with most experienced resource at the top. While the decision is primarily driven by margin considerations, it creates an effective team structure. Pyramid construction must be undertaken as a joint exercise so that the vendor margin is not the only driving factor.

Progressive payment increase

Typically vendor resource last in a project at the most for two years. Due to lack of career and remuneration growth it paves the way for their exit. While the customer cannot help with career growth of the resources, hike in payment beyond 2 years can help in retention of some of the resources. While it may appear to increase the cost, it pays for itself through greater defect detection and saving on cost of ramping new resource. Compare this practice to providing salary to in-house resources who are provided raises on annual basis. Not all resources could be retained through higher payments but it will at least eliminate structural bias against the continuation of the resources for a longer term.

Delineate cost items

Flat rate that has in-built costs to cover vendor management effort and other indirect costs invites structural misalignments as the rate structure is based on assumptions made by vendor at the start of the project. For example, for a small team a vendor may decide to allocate a non-billable manager 25% of the time and price the billable engineer rate accordingly to cover manager's cost. However, if the team size grows it may require 100% of test manager's time. As the manager is not billed explicitly and the engineer rates are fixed it leads to mutations such as one of engineer being asked to manage the team over and above his/her job. It is best from client and vendor's perspective to list down and account for all cost components separately. It ensures that customer demands are not pushed back for reasons that vendor is unable to explain and at same time customer get to demand accountability for the price paid. What is not paid for will not be served.

Choose right priorities

Focus on ROI with defect finding capability topping the list. All activities such as process improvements, knowledge management, end to end testing, engaging testing early, engagement model such as managed service or TCOE, specialized testing, domain focused testing e.t.c. need to be ruthlessly tied to single objective and that is maximize defect finding capability of the team and thus it's returns.

Managed testing service, process improvements and all exercises of ilk should never be the end goal. It is suggested that return be goal number one and cost of service the second goal. Customer satisfaction could be the third goal. Return always needs to be the first goal as it typically vastly exceeds cost. To give an example, say an additional high priced white box tester is employed in a team that mostly does black box testing. The cost of this tester for quoted project would be $35K at a rate of $30/hour at offshore for 6 months. If this tester helps uncover even 4 unique defects the cost will be recovered. Typically a single code defect manifests as multiple functional defects and as such ROI would be higher for white box testing. However if the ROI is solely focused on cost aspect then a white box tester may never be employed or a cheaper resource would be employed who spends most of the time learning the skill or reporting trivial defects.

Set correct savings expectations

Offshore testing rarely provides 50% savings unless the indirect costs are added up to wrong buckets in the accounting books. Greater cost savings achieved via cheaper resources is more than compensated by lower productivity, higher rework, more issue resolutions at onsite and defect leakage. On other hand, paying slightly more for more experienced resources at offshore pays for itself through higher productivity and greater defect removal. Even a 30% saving is not a bad deal. Consider other advantages of offshoring as well such as the ability to turn off and turn on the resource tap according to business needs. For a proper evaluation consider the cost of testing through the life of the application.

Reward & risk mechanism

Some part of the payment should be linked to the number of defects reported and the defects leaked to production to provide incentive to vendor to focus on defects

Recognize the career growth aspirations of vendor resources

Customers compete for the best of the resources on vendor bench. Recognition of career growth aspirations of the resources could be the most motivating act that the customers can perform and it doesn't necessarily cost anything.