Ron Wilson
There has been a long debate around bringing in software testing consultants to perform a software testing assessment. I am seeing a stronger case in my mind, especially as my company is starting to get into the consultant business. I have recently conducted a software testing assessment and I now have both perspectives because I have received many testing assessments and I have now given a software testing assessment. It is a valuable tool for the testing organization because they will be able to provide valuable input and they will also be able gain information from a software testing organization. In general, I am finding that most formal testing organizations that have a software testing manager along with a dedicated software testing team, have a solid knowledge and understanding of the testing process. Either companies have started a software testing organization and have hired testers from the outside, or these individuals within the company have self-taught themselves and have some level of knowledge about software testing practices.
Over a period of time, however, and due to the demands of the business, software testing often gets overlooked, or pressured into reducing their costs. Often, things get so bad, that upper management thinks the software testing process is broken, and they feel a need to bring in an outside organization to perform a software testing assessment to determine what things need to be improved. Within the organization, the software testers immediate reaction is a natural sense of hostility. The group is hurt that upper management does not trust them to get the job done effectively. So the group has a choice, do they put their walls up, or do they allow the organization to come in and help them?
What Happens During The Software Testing Assessment?
In taking a look from the company that does the actual assessment, what I am seeing is very interesting. Once the SOW is signed the following activities occur.
Kickoff Meeting: The first few meetings are awkward, because each are trying to talk the same language and get on the same page. Once the meeting with the test management team is complete, there are subsequent meetings that allow the company who is doing the software testing assessment to interview resources within the company who do the day to day testing activities. What we find is that these people are really happy to sit down with someone who has experience with software testing, and understands the daily frustrations that a software tester experiences. This allow a great level of productivity to be accomplished in a very short period of time. The assessor can often identify the root of the problem very quickly. It is important that the person doing the assessment have many years of software testing experience. This helps, because the assessor truly understands software testing and can quickly determine what is going on, and what recommendations need to be made in order to make the software testing process better.
Interviews: There is usually a series of interviews that will be conducted. This helps to gather information for the software testing assessment. Typically, the majority of the time in interviews is spent talking with people in the software testing group. Project Managers, Business Analysts, Developers, IT Managers, and Agile Scrum Masters are usually other roles that are also interviewed during this process. Once the interviews take place, all the information is collected, and the reporting process begins.
Questionnaires: The organization that is performing the software testing assessment will usually provide a series of questions that that will help in providing some additional information about the software testing process. This will help provide some context and perhaps some comments about how things are going within the organization. It also helps the assessor to see some additional trends that will often be explored in greater detail.
Confidentiality: It is extremely important that the assessor keep all the information as confidential as possible. While some people might be somewhat guarded with their information because they fear the consequences, most people will share information in order to help the organization. It is important for the assessor to emphasize confidentiality with the interviews and questionnaires. I made it a particular point during all my sessions and I believe it helped me to gather more information.
Software Testing Assessment Report: The report typically takes a few weeks to complete, and then the results are published and compiled. Often the assessments from beginning to end are 4-6 weeks depending upon complexity and a proposed roadmap is presented. If assessments are done correctly, the software organization can greatly benefit from the assessment, and get upper management to agree and work towards getting additional things resolved. I also find that the assessments are helpful and I have leveraged these assessments in creating a QA roadmap. So for example, if the recommendation is to add additional resources or have longer testing periods, things will improve within the organization.
So the best recommendation I have, if your company is faced with a software testing assessment, is to welcome that with open arms so in the end you can tell management what you have told them all along “Software testing is important, and it requires the right amount of focus and resources to make it work effectively.” The goal for both the software testing assessment company and the one that is getting the software testing assessment is to improve the software testing process. If this is done properly, then there is a win-win for both organizations. I encourage you to consider having a software testing assessment if you haven’t had one before. I promise you they are worth it!
How Long Does a Software Testing Assessment Take?
The length of the software testing assessment depends upon what is currently in place and the complexity within the organization. I usually see assessments between 3-6 weeks. I wouldn’t recommend anything that is outside that range.
How Much Does a Software Testing Assessment Cost?
This answer varies depending on who is performing the assessment and how long the assessment will take. Most software testing assessments cost between $25,000 and $50,000. While that might see like a lot of money, especially for the smaller testing organizations, they are certainly worth it.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
Ron Wilson