{"id":3164,"date":"2021-04-14T14:46:15","date_gmt":"2021-04-14T14:46:15","guid":{"rendered":"https:\/\/elementordzyns.com\/subject7\/?p=3164"},"modified":"2024-12-26T15:33:44","modified_gmt":"2024-12-26T15:33:44","slug":"how-to-start-performance-testing","status":"publish","type":"post","link":"https:\/\/elementordzyns.com\/subject7\/how-to-start-performance-testing\/","title":{"rendered":"How to Start Performance Testing"},"content":{"rendered":"

Sometimes when people talk about a discipline, they make it harder than it needs to be.<\/p>\n

This does not have to be on purpose. \u00a0When the aspiring performance tester starts out, they have to read a bunch of documentation, thick books with big words, and, frankly, figure out what works by trial and experiment. \u00a0As they grow, the performance tester speaks in that language, uses it at conferences \u2014 they simply have to in order to get hired. \u00a0By the time they have achieved some level of professional success, to admit the process just is not that hard, that the emperor has no clothes, is to devalue their own prior work. \u00a0So the cycle repeats, and, frankly, enables large day rates for project work.<\/p>\n

Don\u2019t get me wrong, there certainly is a fair bit to performance testing. \u00a0There is modeling work, load generation, monitoring, and interpreting results, plus some hard-learned lessons along the way. \u00a0Getting started though? \u00a0Getting started does not need to be hard, but it does need to address some of the counterintuitive realities of simulating large numbers of users.<\/p>\n

Here are my basics of performance and load testing.<\/p>\n

Performance vs. Load Testing<\/h2>\n

First of all, there are no standard definitions for these things. \u00a0I would even go so far as to argue that there are benefits to competing definitions forcing us to fight for shared understanding. \u00a0In this article I will use performance testing as an umbrella term, to see if the system is responsive enough under real-use-like conditions. \u00a0That is different from\u00a0simulating\u00a0<\/em>a large number of users, or \u201cload.\u201d \u00a0With these definitions, load testing is a specific kind of performance testing.<\/p>\n

\"<\/p>\n

To do performance testing with this definition, you can just \u2026 test. \u00a0Explore the software as a single user. \u00a0Notice the problems. \u00a0Re-do things that seem slow. \u00a0Use the browser tools to find a\u00a0waterfall diagram<\/a>\u00a0and figure out what is loading slowly and how quickly.<\/p>\n

Often performance problems appear with just a single user, or are so borderline with a single user that those problems will be the place to start to look with many users. \u00a0Load testing requires simulating multiple users. \u00a0Doing that well requires a bit of modeling.<\/p>\n

Modeling<\/h2>\n

Like a toy plane or car, models are simplified descriptions of what users do in systems. \u00a0If you look at what users actually do on websites, they read them. \u00a0They click around in ways that are difficult to predict. \u00a0Scott Barber, cofounder of the\u00a0Workshop on Performance and Reliability<\/a>,\u00a0invented an entire modeling language called\u00a0User Community Modeling Language<\/a>,\u00a0designed to graph the user experience and to approximate how often users jump between features. \u00a0If the system is in production, there is an easier way. \u00a0Look at log data, collate it by base web page, count the number each appears and calculate the percentage.<\/p>\n

With the wrong model, the load will not realistically match the users. \u00a0The users will go off and do different things, and the load tests will not be as valuable as they could have been. \u00a0Generally modeling is constrained by the tool you choose to generate load.<\/p>\n

Load Generation<\/h2>\n

This is the tool that takes a script of some kind and runs it over and over again. \u00a0Generally, these tools are more interested in the time to get service requests and less interested in the correctness of those results. \u00a0They might report 404 errors and timeouts, but they might not report a bug on-screen. \u00a0In the worst case, login might fail. \u00a0All the results could be \u201cfast\u201d as the software is essentially just re-displaying the failed to login page over and over again.<\/p>\n

Most modern tools accept a script with some variation, including variables, to have different users log in, each on their own \u201cthread.\u201d \u00a0The script may have some randomization in it, either for steps or for what to type. \u00a0The load tools usually \u201cramp up\u201d the number of users slowly, to see where the system \u201cfalls over.\u201d\u00a0 Once you find that spot, try hitting it all at once with slightly less load. \u00a0Another less-common practice is soak testing, where the load running runs for an extended time, say over an entire sprint, to see if there is a memory leak or if a bad file write will overfill a disk.<\/p>\n

Monitoring & Reporting<\/h2>\n

When it comes to data, we care about the time for the entire request on the server (which we can control), the user experience, and the use of the components. \u00a0That\u2019s a fancy way to say we want to know how the CPU, disk, memory, and network are being taxed.\u00a0 Generally, when we hit poor performance at a certain load factor, we look for what component \u201cfell over\u201d due to over-use. \u00a0If the system is cloud-based, then use should hit a certain level, the software should \u201cspin up\u201d a new server, and then use should go back down. \u00a0In that case, it might be helpful to monitor the resources the cluster manager has allocated.<\/p>\n

Front-end performance testing<\/a>\u00a0looks at how long it takes the request to display, usually on a web page. \u00a0In many cases, it is possible to structure the page so that it loads the incremental elements, instead of loading all the files, and then showing the completed page. \u00a0These tools can also identify, for example, when an image file is very large. \u00a0In most cases, a lower resolution image can work just fine.<\/p>\n

You can find time-to-live by injecting time into the server logs, from request-in to request-served. \u00a0The difference between the time on server and the total trip is the\u00a0propagation delay<\/em>, the delay due to the network, or perhaps, limited by bandwidth. \u00a0It can be hard to get utilization, time to live, and the component use into a single test run report. \u00a0Early on, you can just \u201cwatch the dials\u201d as the test runs \u2013 afterward, the load tool should be able to create a test run report that provides at least the front-end experience data over time as the number of users grows.<\/p>\n

Once you have the data of performance under load, you will likely have something like a hockey-stick graph. \u00a0At a certain number of users, the performance falls apart. \u00a0That is likely because some component becomes over-used, the system runs out of disk space, or the network becomes saturated and suddenly everything takes much longer.<\/p>\n

Which brings up the question, once you have the data \u2026 what does it mean?<\/p>\n

Evaluation<\/h2>\n

In an ideal world, you would have a service level agreement (SLA) with the business sponsor on how fast is fast enough, sort of like a specification. \u00a0Sadly, for most clients, that remains as out of reach as a perpetual motion machine.<\/p>\n

My point is that if you are learning about performance testing, you probably don\u2019t have SLAs or any sort of mature process. \u00a0Which, as I alluded earlier, provides an excellent opportunity to experiment and learn. \u00a0It is possible the engineering team can make up SLA\u2019s, collaborating with an empowered product owner. \u00a0More realistically, the actual performance results will be all over the place, and even if you had SLAs, the decision makers might be willing to compromise on this or that feature to get the software out to users. \u00a0In that world, you don\u2019t work to the SLA as much as express a\u00a0service level capability\u00a0<\/em>(SLC) and let the decision makers choose if that is acceptable or not.<\/p>\n

There are a couple different schools of thought on how to present the information. \u00a0You could present the decision makers (read: executives) with the rest results. \u00a0Generally raw data will make no sense to them and need to be interpreted in context. \u00a0The way to do that is to create a\u00a0story<\/em>\u00a0that uses the measurements as supplemental detail. \u00a0This is probably best done in person, perhaps backed up by a one-page document with a few tables and graphs. \u00a0Once the context is understood, to show change over time, a dashboard may suffice. \u00a0In that case, the\u00a0change in performance over time<\/em>\u00a0becomes more important than today\u2019s results out of context.<\/p>\n

One way to get there is to use the software under load yourself and see how it feels. \u00a0That won\u2019t work as part of a continuous integration run, but it can create a story and a visceral experience. \u00a0In the worst of cases, when you present the data, and are told that it\u2019s fine, you could point to a laptop and encourage the decision maker to put fifteen minutes into trying to use the software themselves. \u00a0This creates an opportunity for learning. \u00a0And, should the executive say \u201cnope, it is fine, then when complaints roll in, the problem-resolution discussion is likely to go very differently.<\/p>\n

Right Now<\/h2>\n

Like most skills, performance testing is best learned by doing. \u00a0Today, my goal was to take away the mysticism and magic, to make it approachable. \u00a0Once you understand the concepts, you can dig into a specific application to model, a tool for generating load, an approach to modeling\/reporting. \u00a0By now, you have at least a conversational understanding of performance testing. \u00a0You\u2019ll be able to separate the different pieces of the work, to recognize when a vendor representative is over-simplifying the task, or a technical person might be, perhaps accidentally, over-complicating it.<\/p>\n","protected":false},"excerpt":{"rendered":"

Sometimes when people talk about a discipline, they make it harder than it needs to be. This does not have to be on purpose. \u00a0When the aspiring performance tester starts out, they have to read a bunch of documentation, thick books with big words, and, frankly, figure out what works by trial and experiment. \u00a0As […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3164","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"acf":[],"_links":{"self":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3164","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/comments?post=3164"}],"version-history":[{"count":2,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3164\/revisions"}],"predecessor-version":[{"id":3210,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/posts\/3164\/revisions\/3210"}],"wp:attachment":[{"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/media?parent=3164"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/categories?post=3164"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/elementordzyns.com\/subject7\/wp-json\/wp\/v2\/tags?post=3164"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}