Performance Engineering: More Art Than Science

If you are responsible for performance engineering, you know there is a true art to understanding how things work.  Performance Engineering requires extensive understanding of the applications that are tested and the performance aspects of how the applications behave.  Early in my career I thought performance was based upon pass and fail criteria only but over the years I have understood that is not always the case.  Here are some performance engineering principles you must consider:

1. There are many components which drive application performance including (but not limited to) CPU, Memory, Cache, Databases, Servers, Network Traffic, and Transactions
2. It is always important to run a baseline test so you have a point of reference in determining what is acceptable
3. I always recommend getting at least 2 acceptable performance runs in in order to eliminate any anomalies
4. The production environment will likely be the only environment that is sized adequately, many organizations will have a dedicated performance testing environment but you will need to extrapolate your information to determine what will be acceptable in production
5. Focus on building performance scripts that cover the top 80-90% of the most used transactions and based your performance tests on those only
6. It would be nice to have a tool such as App Dynamics which will help in troubleshooting performance related issues
7. Run your performance test for at least 3 hours in order to identify any performance bottlenecks
8. Always involve development and infrastructure teams in helping with the performance analysis, they should be major participants in the review and sign off of the performance results
9. Performance Engineers should understand how the applications work and be able to help identify performance issues
10. It is important to view the performance test as a whole versus looking at each transaction.  In general if there are a few high transactions, that will be fine, but the overall test should be satisfactory
11. You need to build a large end to end suite so you can see how the application or applications behave together

The list should help you gain a better understanding of performance engineering.  It is important to constantly work on gaining a greater understanding of performance engineering.  As technology changes and evolves, if will be necessary to keep up with new trends.  In addition, there are some great performance engineering tools that can help complement those tools that you already use.