Conference Presentations

STAREAST 2011: Performance Engineering: More Than Just Load Testing

Performance testing that is only done at the last minute–just prior to launch–is the wrong approach for complex systems which have many opportunities for performance bottlenecks. Rex Black discusses a different approach-performance engineering-that is far more comprehensive and valuable than merely performing load testing during the system test. Performance engineering takes a broad look at the environment, platforms, and development processes, and how they affect a system's ability to perform at different load levels on different hardware and networks. Performance engineers use a structured process to conduct a series of performance tests throughout development and after deployment. These tests includes performance modeling, unit performance tests, infrastructure tuning, benchmark testing, code profiling, system validation testing, and production support.

Rex (Red) Black, J9 Technologies
Software Performance Testing: Beyond Record and Playback

Predictable software performance is crucial to the success of almost every enterprise system and, in some cases, to the success of the company itself. Before deploying systems into production, management is demanding comprehensive performance testing and reliable test results. This has created significant pressure on development managers and performance test engineers. Alim Sharif guides you through the basic steps for planning, creating, executing, and reporting performance tests. He explains how to actively involve stakeholders-application developers, database administrators, network engineers, IT infrastructure groups, and senior managers-to identify and resolve performance issues. Alim discusses how to maintain the balance between these stakeholder interests during each step and demonstrates how to effectively lead the performance test effort.

Alim Sharif, Ultimate Software Group
Variations on a Theme: Performance Testing and Functional Unit Testing

The right types of performance tests can reveal functionality problems that would not usually be detected during unit testing. For example, concurrency and thread safety problems can manifest themselves in poor performance or deadlocks, leading to incorrect output. Because unit tests inherently lack concurrent activity, these problems rarely manifest themselves in functional tests. André Bondi describes test structures based on rudimentary models that reveal valuable insights about system scalability, performance, and system function. For example, to ensure that resource utilization increases linearly with the load-a necessary condition for scalability-transactions should be submitted to systems for long periods of time at different rates. Conversely, when the load is constant, performance measures should be constant.

Andre Bondi, Siemens Corporate Research
Focusing Test Efforts with System Usage Patterns

Faced with the reality of tight deadlines and limited resources, many software delivery teams turn to risk-based test planning to ensure that the most critical components of the software are production ready. Although this strategy can prove effective, it is only as good as your underlying risk analysis. Unfortunately, understanding where risk lies within a product is difficult with the analysis often resulting in little more than an “educated guess.” These risk-based testing exercises can lead to uneven test coverage and the uneasy feeling that the team has neglected to test what is really important. Dan Craig describes how to employ system usage patterns and production defect reports to identify the real risks in a system.

Dan Craig, Coveros, Inc.
Performance Testing SQL-based Applications

Often, we discover the "real" software performance issues only after deploying the product in a production environment. Even though performance, scalability, stability, and reliability are standards of today's software development, organizations often wait until the end of the development life cycle to discover these limitations, resulting in late deliveries and even chaos. He embraces agile development's philosophies to explain how performance testers can identify and resolve software performance issues early and continue performance testing throughout the development process. Learn how to optimize the use of performance tuning tools such as SQL profiler and MS PerfMon to identify and fix MS SQL server, application, and Web server performance issues. Institute agile methods in your performance testing efforts to avoid that "Oh, no!" moment when the system goes live.

Alim Sharif, The Ultimate Software Group
Performance Testing Throughout the Life Cycle

Even though it is easy to say that you should continuously test your application for performance during development, how do you really do it? What are the processes for testing performance early and often? What kinds of problems will you find at the different stages? Chris Patterson shares the tools and techniques he recently used during the development of a highly concurrent and highly scalable server that is shipping soon. Chris explores how developers and testers used common tools and frameworks to accelerate the start of performance testing during product development. Explore the challenges they faced while testing a version 1 product, including defining appropriate performance and scale goals, simulating concurrent user access patterns, and generating a real world data set. Learn from his team's mistakes and their successes as Chris shares both the good and the bad of the process and results.

Chris Patterson, Microsoft
Life as a Performance Tester

At the core of most performance testing challenges and failed performance testing projects are serious misunderstandings and miscommunications within the project team. Scott Barber and Dawn Haynes share approaches to overcoming some of the most common frustrations facing performance testers today. Rather than simply telling you how to improve understanding and communicate performance testing concepts, Scott and Dawn demonstrate their approaches through an amusing role play of interactions between a lead performance tester and a non-technical executive.

Scott Barber, PerfTestPlus, Inc.
STARWEST 2008: Performance Engineering: More Than Just Load Testing

Performance testing that is done once or a few times as part of the system test is not the right approach for many systems that must change and grow for years. Rex Black discusses a different approach--performance engineering--that is far more than performing load testing during the system test. Performance engineering takes a broad look at the environment, platforms, and development processes and how they affect a system's ability to perform at different load levels on different hardware and networks. While load testers run a test before product launch to alleviate performance concerns, performance engineers have a plan for conducting a series of performance tests throughout the development lifecycle and after deployment. A comprehensive performance methodology includes performance modeling, unit performance tests, infrastructure tuning, benchmark testing, code profiling, system validation testing, and production support.

Rex Black, QA Software Consultant/Trainer
Avoid Preformance Testing Data Deception

Don't be fooled by your performance test results. Performance testing can easily generate an unwieldy amount of data-some relevant and some not. Testers and their tools often use statistical methods to make sense of the data, but using statistics requires sacrificing accuracy and thoroughness. The good news is that we do not need to understand all the details to make good use of test results. The challenge is to determine what information really matters and how to present it in a useful manner. Join Ben Simo as he addresses common performance test statistical problems including built-in bias, agreeable averages, invisible inadequacies, gargantuan groupings, stingy sets, mountainous molehills, creative charting, alien alliances, and more. Find out how statistical reporting can deceive rather than inform-often unintentionally-and recognize what the numbers do not say.

Ben Simo, Standard & Poor's
Preformance Testing in Enterprise Application Environments

As systems become more complex--serving the enterprise and implemented on the Web and across the Internet-performance testing is becoming more important and more difficult. David Chadwick suggests that the starting point is to design tests that reflect real user activity, including independent arrivals of transactions and varying input data to prevent "cache only" results. David explains how to break down the end-to-end system response time into the distributed components involved in processing the transactions. Learn to use resource-monitoring data to discover bottlenecks on individual systems. By examining the frequency and time spent in various processes, performance testers can determine where resources are being consumed and how to tune a system for better performance.

David Chadwick, IBM

Pages

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.