Product release readiness: Product testing, product bugs and going live
During product development, companies need to determine their product’s readiness for release. This process takes place after you have spent time learning your market’s problems, building a solution and testing the product, it’s time to determine whether that product is ready for release.
Product release versus product launch: Understanding the difference
Many companies mistakenly equate being ready to release a product with being ready to launch. However, there is a difference between the two:
- Product release: Your product is technically complete and ready for customers to use.
- Product launch: Your company is ready to tell the world about the new release. The launch is not a technical activity. While it does depend on the new product being ready, it also includes other business operations (for example, training, legal considerations, sales activities). Review the article Marketing plan for your product launch for the key aspects of a product launch.
Product testing results: Managing product bugs
During the development process, as your team tests the product, they will find defects (also known as bugs). Use an online tracking tool to track these defects to encourage a streamlined approach to managing and resolving them. Rank each defect according to its severity, ranging from “showstopper” to “low-level/cosmetic.”
The following are sample defect definitions; adjust them as necessary for your organization:
- Showstoppers: These are critical bugs that impede the operation of the product. Example: System does not turn on.
- High-level: These are high-priority bugs that impede the operation of a particular function of the product, and there is no workaround. Example: User cannot log in to the system.
- Medium-level: These are bugs that cause inconvenience to the user, but a workaround exists. Example: User must turn the knob clockwise to engage the latch.
- Low-level/cosmetic: These are minor irritations or cosmetic changes that do not impede the operation of the product. Example: Operation instructions contain several spelling mistakes.
A product release without product bugs?
While it is preferable to release the product with no bugs, this happens rarely. A reasonable guide is to ship with no showstopper or high-level bugs, and minimal medium-level bugs.
During the testing process, negotiate with the engineering team to determine how many of each type of bug you and your customers would tolerate if you were to release.
Your team should be able to outline at regular intervals how many of each type of bug currently exist. While your goal is to ship with no bugs, this could negatively affect your release date and thus your revenue projections.
Product bug triage
As the product manager, review the outstanding bug list on a regular basis with the testing team. Review the status of each open (unresolved) bug, and understand the scenarios through which these bugs were identified.
If you identify High-level bugs that must be fixed, you can choose to delay the release date. In the case of low-level/cosmetic bugs, you might choose to address them during a future release.
Example: You have the following bug list:
- No showstopper bugs
- No high-level bugs
- Two medium-level bugs
- User cannot access the address book without an error message
- When user imports an address, unreadable characters populate the “country” field
- 10 low-level/cosmetic bugs
In this scenario, you might decide that unreadable characters populating the country field is an acceptable bug because the user can easily delete them. However, you accept you do not want to alarm the user with an error message upon accessing the address book.
Similar to when you prioritized your product requirements, place bugs in priority order and make trade-offs where necessary.
Code freeze
After using either the Waterfall or Agile methodology to develop your product requirements, your product development is considered complete.
To that end, you then need to freeze the environment (called “code freeze” in IT). This means that no further development should take place unless it involves fixing a bug that is discovered during testing and deemed necessary to fix.
Your engineers must respect this concept; otherwise, unexpected changes can be introduced into the product, adding unnecessary risk to the project.
Going “live” with your product
Once bug triage is complete and your team has finished fixing the defects, the product is considered “live.”
This means that you have reached the final version of the product, and it is ready for distribution or manufacturing (whichever your product requires). The live product is expected to be very stable, relatively bug-free and ready for customer consumption.
Once you have a live product and you reach the scheduled release date (“live date”), the product launch can begin. For more information, read the following articles:
References
Richardson, J. & Gwaltney, Jr., W. (2005). Ship It! A Practical Guide to Successful Software Projects. Pragmatic Bookshelf.
Software Testing Stuff. (2008). When software is ready to ship or release. Retrieved September 27, 2010, from http://www.softwaretestingstuff.com/2008/12/when-software-is-ready-to-ship-or.html