QE - Offshore Partner Selection Criteria
A couple of big 'infrastructure' issues to take into consideration:
- Test Environment: We, the onshore company, should always host the test environments used for both functional as well as load testing. I have dealt with offshore test environments and you basically end up spending half your time troubleshooting 'perceived' issues in those environments. Issues normally arise because of either the code base or the data is being out of sync between the local environment and the offshore environment. At some point development no longer trusts product issues reported in the offshore environments and you start having to double check everything in the local environment. Luckily with a hosted solution we should not be exposed to these type of issues.
- Licenses for commercial load and performance test tools: An offshore company might have the tools in place that we might be interested in and actively use them for some of their customers, however, in general their license agreements with the tool vendor will not allow them to use the software for our company. Which makes perfect sense from the tool vendor's perspective. We basically have three options. Buy a license to be used by the offshore company and install the software on their servers. The downside of this approach is that if the relationship with the offshore company blows up and you want to move the license to your side, you might have to pay a 'geographical relocation' fee (previous experience with Rational). The second approach is for us to get the license and install it on our local servers. In this case their testers will have to remotely access the software on our servers. Option three is to use open source software which obviously makes this issue completely irrelevant. If option three turns out not to be an option, I strongly favor going with option two - buy a license for our company and install the software locally on our servers.
My quick and dirty list of selection criteria:
Note: At this point I'm excluding any criteria related to Developers
- Company Level Criteria
- Stability - How long have they been in business for
- Is the company based in the US
- Do they have a local representation of the company
- Local account managers in case of escalation / trouble shooting
- Financial stability
- Size of the company
- Number of employees
- Number of offices worldwide
- Number of customers
- Size of customers - Fortune 500 - 1000
- Types and size of outsource projects they have dealt with
- Both from a 'functional' and a 'performance test' perspective
- Do they have people with skillsets that we might be interested in later - doc writers, developers etc.
- What is the general skillset of their employees - advanced degrees
- What is the turn-over rate of their employees
- Quality of their NetOps department
- Network
- Server room(s)
- General technical support for project teams
- Accommodations
- Dedicated space for project teams
- Check customers references, especially SaaS projects
- Process Maturity
- Center of excellence / Best Practices for QE
- Center of excellence / Best Practices for performance testing
- Do they have a clear definition of roles and responsibilities on their side
- Role of the QA Manager and Project Manager
- Clearly defined escalation procedures
- Clearly defined ramp-up plan
- Dedicated support for initial process and technical issues (especially around network and VPN)
- QE Team
- Buy-in into my outsource model
- How much managerial overhead do they want to put on the team. Besides a manager, any other resources that they might eventually want to charge us for?
- Comfort and experience level with agile development
- Can team members travel overseas if there is a need to
- Individuals
- Match of the tester's skillset with the required nGenere product users's profiles
- Experience with SaaS
Click here QE - Generic Profile of Offshore Testers to view the generic profiles of the offshore testers we are looking for:
-
- Availablity of required tools / utilities
- Hardware
- Personal PCs, especially 'special' stuff like Macs etc.
- Software
- Any specific software required for functional testers
- Automated functional test tools
- Load and performance testing test tools
- What tools do they have in place for test case management that we can use
- etc.
- Remote web load testing service
- Costs
- Rates for functional, automation and performance testers
- Is there a cap of how much their rates can increase on an annual baisis
- Rates for overtime
- Cost for software / hardware purchases
- Cost associated with overseas travel - air fare, per diem, lodging, car etc.
- Vacation / Time Off
- What is their policy for overtime. Is it common? How much advanced notice needs to be give?
- What is their policy for compensation for holidays and personal vacations / timeoff?
- Holiday schedule
- Other Operational Aspects:
- How quickly can we ramp up a team - how large in their pool of testers
- How quickly can we ramp down a team - contractual obligations
- Do they work on a project basis i.e. we need 2 performance testers for 4 weeks. Fixed price projects?
- What is the time for us to cancel our contract with the offshore company
|
Valid XHTML v1.0
terry
0 comments
|