{"id":3183,"date":"2021-03-02T14:48:32","date_gmt":"2021-03-02T14:48:32","guid":{"rendered":"https:\/\/elementordzyns.com\/subject7\/?p=3183"},"modified":"2024-12-26T14:55:01","modified_gmt":"2024-12-26T14:55:01","slug":"where-do-you-start-building-your-testing-program","status":"publish","type":"post","link":"https:\/\/elementordzyns.com\/subject7\/where-do-you-start-building-your-testing-program\/","title":{"rendered":"Where do you Start Building your Testing Program?"},"content":{"rendered":"

Exactly what to do in your test program is context-dependent. Don\u2019t take this as\u00a0the<\/em><\/strong>\u00a0way to create a test program, but instead as\u00a0one way<\/em><\/strong>, a way I have seen work at several companies, paying less for testing and getting better results than their peers. Some of the readers here won\u2019t really have a test \u201cprogram,\u201d not really anyway, but there are inevitably folks doing various \u201cthings\u201d that will help find bugs before releasing new code. These things may vary per team, per project, per person, or even per day.<\/p>\n

In software engineering terms, I am going to lay out a\u00a0reference implementation<\/em><\/strong>. We\u2019ll start without a test layout, talk a little bit of theory, then pretty quickly move into a real practical design for a middle-small software system. That is, somewhere between three and nine software teams create this \u2014 but I have seen the approach used by as many as fourteen teams on three continents. Beyond that, I find we can break the software into subsystems as big as each of those.<\/p>\n

Getting Started<\/h3>\n

Last time, I mentioned Karen Johnson\u2019s\u00a0RCRCRC heuristic<\/a>\u00a0\u2014 that you can find test ideas by looking at elements of the system that are Recent, Core, Risky, Configuration Sensitive, recently Repaired, and Chronically defective. It is a good source of insights, but not quite a system. Over the years I\u2019ve drawn on version control tools to find Recent changes, logs to find the Core, bug trackers to find the Chronic, Risky and Repaired code. To get started though, I act like an emperor in a science fiction novel: focus on the core. If the core is lost, nothing else matters.<\/p>\n

Within the core, there is the uber-core, the things one simply\u00a0has<\/em><\/strong>\u00a0to do it in order to use the system. In eCommerce, that is the homepage, search, product display, add to cart, and check out. You might add fulfillment (the warehouse gets the order) and making sure the credit card gets charged to that list. If you have logs, you can search through them to see how often operations are performed. We had these logs at Socialtext and they were helpful; there were features we had no idea were so important to the customer that we uncovered that way. It turned out that among our entire user population, \u201cdownload an attached file\u201d was more common than \u201ctag a page,\u201d or even use \u201ctag to search,\u201d for example.<\/p>\n

But let\u2019s say you don\u2019t have those logs. You don\u2019t have any formal documentation on features. Your product managers say they are too busy, no one reads documents, the software changes too often, and they don\u2019t have time for that anyway. Sound familiar?<\/p>\n

Here\u2019s something you can do that will inevitably help.<\/strong>\u00a0Make a list of all core features, along with their critical flows. This is all just one spreadsheet. Tab A contains the core features. Tab B has the flows, and in a second column, lists all the features that flow covers. If the flows are disconnected and independent, you might not have tab B at all, instead, just use a spreadsheet of micro-features that are different enough to have problems that most users will hit 90% of the time. Once the spreadsheet exists, share it with the team and get them to add the micro-features that are missing.<\/p>\n

If you want to get really fancy, and anticipating a post-Covid era, print them on note cards and put them in the biggest conference room you have. Allow the team to walk through and sort the feature in terms of use, from most-used to least. After people have walked around, they walk around again, making corrections and debating. By three or four rounds, people will probably agree on popularity. Then hold up the top three or four note cards; these are your starting point. We can call them the 90+%, basically the most critical features and capabilities in your app. The next group of three or four are your 80-89%, and so on and so forth.<\/p>\n

When you\u2019ve completed that, you have a completed priority list for\u00a0test automation<\/a>. Preparing the spreadsheet might take an hour; the meeting might take an hour. It\u2019s super fast, relatively accurate, and any team can do this.<\/p>\n

Working on the list<\/h3>\n

Now just order and group them in your spreadsheet, so you can pull the test ideas as stories and create them in a tool, one at a time. Assuming the complexity of the stories is roughly the same, you can count the number of stories that are done in two weeks and predict when you get to the 90% cutline, and then the 80%, and so on. If that isn\u2019t fast enough, management can add time, people, or lower their expectations.<\/p>\n

This approach provides a constantly-increasing amount of coverage, starting with the highest \u201cbang for the buck.\u201d As core functionality tends to be relatively stable, the tests you create are less likely to be brittle or \u201cflaky.\u201d Doing the work to create the plan and running it for a week or two also gives management a budget, in terms of time and money, that is backed by real data, instead of wishful thinking. The plan is so inexpensive that it is almost free. I have also seen teams take those lists of features and flows and create main-maps, wikis, and other forms of documentation. As it turns out, the test effort can drive a documentation effort as a side benefit.<\/p>\n

The list itself will change over time. Creating the tests is very different than \u201cimagineering\u201d on a spreadsheet, so test ideas will occur to the testers as they do the work. Expect some small amount of \u201ctest case bloat\u201d and maintenance cost. What this system does provide is that it minimizes this effort and makes those costs explicit.<\/p>\n

As I said before, this isn\u2019t the only way to do it. Next time I\u2019ll provide a different approach based on recent changes to the codebase. It\u2019s a little tricky, and harder to get right \u2013 but for today, I focused on the Core. Try it, and keep us posted on what you find\u2026.<\/p>\n","protected":false},"excerpt":{"rendered":"

Exactly what to do in your test program is context-dependent. Don\u2019t take this as\u00a0the\u00a0way to create a test program, but instead as\u00a0one way, a way I have seen work at several companies, paying less for testing and getting better results than their peers. Some of the readers here won\u2019t really have a test \u201cprogram,\u201d not […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3183","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"acf":[],"_links":{"self":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3183","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/comments?post=3183"}],"version-history":[{"count":3,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3183\/revisions"}],"predecessor-version":[{"id":3192,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3183\/revisions\/3192"}],"wp:attachment":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/media?parent=3183"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/categories?post=3183"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/tags?post=3183"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}