Nice write up on the topic. I do have two questions/comments on it however. 1. Why disable the use of the vuser cache simulation? If one does that then expect sometimes in excess of 60% more in the hits/sec and other utilization of the system which will not ever happen. 2. Especially when it comes to large tests simulating large user populations a script that simulates your lazy users but just logging in every now and then that may or may not log out or do some other minor movement doesn't have to use multi thousand vusers.. Just one or a handful that tracks and temporarily stores and properly reuses the session (jsessionid/.net keys) etc across several users. The SUT will see several thousand sessions built and memory allocated but you don't need then several thousand vusers to do it. I've seen this a LOT recently. I like the other entries! Makes me want to go back and read the other Best Practices docs.
Thanks for your inputs. regarding the first point... we thought that by disabling the Vuser cache we allow generating more workload against the application under test, not relying on easier client side configuration, as well as reducing some of the in memory cache that happens in case enabling it. I agree with you it is not a mandatory point to be included in the document and it is something the performance engineer should consider when designing the workload in the most realistic way
As for your second comment.... it is a good input as well. Using the new VTS it can be implemented very nicely. We will consider adding it to future versions of this document. once we collect enough feedback we will probably enhance the document.
Glad to help out! Yeah, pushing the session info into VTS would be excellent since it would move the in memory footprint for that storage to VTS instead of the running vuser memory space. NICE!
