There’s a saying that “you can’t test quality into a product.” That may be true, but it is very difficult to assess the level of quality of a product without testing it. I find that one of the problems with quality is the way that it is talked about. There are all different categories of testing such as regression, black-box, white-box, exploratory, automated, manual, etc. Each of these categories serves a useful purpose, but what is the overall purpose of testing?
Some people argue that testing is how you verify that the product works. Other people argue that the purpose of testing is to uncover what does not work. My view is that the purpose of testing is to show that the software works. By “works” I include that it correctly handles bad input, bad data, and diabolical usage. If you try to use the product in a way that was not intended and it crashes or gives no result at all, then you have found a way in which it does not work.
Quality Assurance and testing are not the same thing. You can implement all of the QA you want: good process, code reviews, coding standards, coding for testability, and whatever else you want to assure quality short of actual testing, but without testing, you have no idea how the product will actually perform. Let’s say you produce a product with absolutely no test plan or you have a great test plan but you don't execute it. I suppose that it is possible that the quality that the customer experiences will be exactly the same as if you had a test plan and executed it. The difference is that your knowledge of the quality of the product will come from your customer after you have already shipped your product. The purpose of testing is to derive knowledge about the quality of the product prior to shipping it so that you can make good decisions about what to do now that you have that knowledge.
So, testing itself does not produce higher quality. It only produces knowledge of current quality. There are only two things that produce higher quality: changes to the process that you use to create the product itself (not including testing), and the responses taken based on the knowledge gained from testing. Testing can influence quality, but it does not directly affect it.
Product quality is an ongoing effort. No matter how much testing you do, users will find ways to use your product that find holes in your test plan. For any change, whether it is a bug fix or a new feature, you don't really know the impact until customers start using it. That means that the maturation of a change doesn't really start until the change ships. If you have a long release cycle, you will have changes that are made early that are basically gathering dust until you ship. You can't really say that they are tested until you test the whole product as it will ship, so again, true maturation doesn't start until you ship.
The moral of this story? Release as often as you reasonably can; which may be once a week, once a month, or once a year, but always strive to release often.
Next: The Role of QA in an Agile Project
10 comments:
I agree with you in some of the points you expose, but I disagree in the overall idea. You make very good points when you say, "Testing itself does not produce quality. There are only two things that produce quality: the process that you use to create the product itself (not including testing), and the responses taken based on the knowledge gained from testing." Therefore, you want to gather information from testing as soon as possible, and the quickest way is to test.
Of course, you cannot possibly think of every scenario your customers are going to run into (or I should say, it is not likely you will be able to define every possible scenario). Under this assumption, releasing often will help you to obtain feedback from your actual customers very soon, which will allow you to deliver fixes to them very quickly. This will increase your software quality, but not from the fact that you release often, but the fact that your customers are testing the scenarios you have not thought about.
You said, "Let’s say you produce a product with absolutely no test plan or you have a great test plan but you don't execute it. There is a chance that the quality that the customer experiences will be exactly the same as if you had a test plan and executed it." Although this is statistically possible, you will have to agree that the actual chances of something like that happening are very slim. If you don't have a test plan, or you have one and don't execute it, it is more likely your final product release will have lower quality than if you executed the plan (no matter how bad the plan was) because you won't have any feedback until your customers actually uses the product. I am sure with your experience you are not trying to convince us that we should release our products without any actual testing. You probably know as well as I do, that it is not a good idea to release untested software, especially if you want to keep your customers around.
The moral of this story and where I disagree with you is that releasing often does not increase quality. Good processes and testing often increase quality. Good processes will allow you to make less mistakes. Testing often will help you to find the little mistakes you make, because we all make mistakes, quicker. With that information, you can act and fix the issues found. Releasing often will help you to gain feedback from your cutomers and act upon what they report, but releasing often will not help you if your often released software is not used; therefore, releasing often does not improve quality; testing often does.
Isaac, thsnks for your thoughtful post. Let's say that you could release every month but instead you release every year. Let's also say that you go above and beyond on your testing efforts. You've got very high test coverage numbers and you are using decision based coverage instead of line coverage. It has been a long time since anybody found a bug, and you've fixed every bug you know about. Lastly, let's say that you developed in 30-day iterations and at the end of every iteration all functionality introduced during that iteration had all of its tests written during that iteration instead of at the end of the one year cycle.
Now at the end of this year you release your product. I guarantee that your customers will find problems. Let's say for the sake of simplicity that they find exactly 12 and each one is linked to functionality introduced in one of the 12 iterations. You waited 11 months to find the issue introduced in the first iteration that your customer would have found right away.
I'm not saying that you should rely on your customers to be part of your testing department. I'm only saying that despite your best efforts, it is inevitable that there will be issues that you only find after you release, so keep on testing, don't stop that. But release as often as you can.
Also in this (contrived) example, your customers were still exposed to the same number of bugs, just not all at the same time.
You are right that I am not suggesting that you forgo testing. My point about possibly having high quality without executing a test plan was that while possible, it definitely is highly unlikely. The point of doing the testing is to gain knowledge about the quality prior to release so that you can then act on that knowledge.
After reading your comment and writing this response I would say that a better moral of the story is "In addition to keeping your testing standards high, it is better to find problems that you are likely to only find by releasing to customers as soon as you possibly can. Therefore, release as often as you can." A bit more wordy, but perhaps a better representation of my point.
Good post, frequent releases are generally good for developers.
But customers may not want it because every download comes at a price and risks.
requent grew for acquiring a strike with grownup males and Onitsuka Tiger Online Store females instantaneously receiving a last final accomplish result of really effective advertising and advertising campaigns undertaken by grownup males and females referred to as Weyden and Kennedy.Hogan Men shoes Of course, this assortment of shoes was named swiftly good ideal after the well-known basketball participant Michael Hogan Men shoes Jordan and was bound to turn in to some strike inside the really bare lowest among the basketball fans, initially. The Jordan sequence of shoes has also been produced for women, which look about in actuality captivating designs. Franklin & Marshall The shoes are obtainable in an exceptionally amount of colors; blue, white, red, black, pink, green, etc. in actuality sporty,Abercrombie Fitch store they get ladies dressed up for that superb game. In add-on they look about in female sizes to fixture the two the tiny footed along utilizing the huge footed.
Nice articTime is an important element which should be made inclusive for the whole activities related with testing toward user story estimates. This can include automated testing and manual probing testing. Inexperienced Scrum teams frequently and habitually over promise or goes overboard with their commitment part in terms of extra work planning compared to what they could feasibly do. le Damon!!For more, please Visit: https://www.scrumstudy.com/blog/all-about-agile-testing/
Grab the thesis statement graders free, which you will definitely like and find useful! This tool will analyze your text in detail, and you will be able to correct your mistakes. I am pleased with the results of his work, and I can safely recommend him to you. The tool is very easy to use, and absolutely free! Good luck friends!
Almost every student wondered how I can grade my essay online free! I too have been looking for the best option to check my essays for a long time and finally found this wonderful tool! Use this Essay Checker if you want to analyze your text and eliminate all types of errors! This is a good and effective method that will definitely help you!
Hello all, do you use the incomplete sentence checker? If not, I recommend this site. You can use it to improve your text in a matter of minutes. It's quick, easy, convenient, and most importantly free. It checks all kinds of sentences and offers suggestions for improvement. Try it out, you'll definitely be surprised by the results!
Catch a cool and convenient online sentence correction app that will help you significantly improve the quality of your text work. This tool uses algorithms to scan text and highlight grammatical structures, punctuation marks, and misspelled words. After it scans your text, you will be able to see the mistakes you made. With this, you can analyze errors and quickly fix them!
Attract Group's quality assurance services stand out due to their meticulous attention to detail and commitment to ensuring top-notch performance and reliability for every project. Their skilled QA team adopts a comprehensive approach to testing, identifying any issues or vulnerabilities and resolving them efficiently. By utilizing the latest tools and methodologies, Attract Group delivers robust solutions that exceed clients' expectations. Trusting them with your quality assurance needs guarantees a seamless and flawless final product.
Post a Comment