Tuesday, 4 February 2014

Distributed Performance Testing With Maven JMeter and Jenkins

In this post, we'll take a look at a practical, simple way to engineer a powerful, distributed performance testing suite, using tools commonly used in the J2EE community

I think that it's commonly accepted at this point that performance testing in software is a useful way of giving us confidence in the robustness and responsiveness of our system under load. The arguments covering these ideas are well covered by better domain experts than myself, so this article will focus on the practicalities of integrating performance testing with your CI environment.

This article assumes familiarity with the following tools

  • JMeter - Widely used and highly adaptable performance testing tool. 
  • Jenkins - Highly popular continuous integration server. 
  • Maven - Build and dependency management tool. 
It's out of scope for this article to introduce, in detail, these tools, but there are a number of online articles that are better suited to providing a better grounding should you be interested. 



Objectives of a well designed performance testing system

 A well designed performance testing system should satisfy some of the following high level properties.


  • Have well designed tests. Often, it's understated how important having well written parameterized tests can be, so i'll dedicate a small section of the article to explaining how you can use some of the often overlooked features in JMeter to improve your own tests. 
  • Support scalable generation of load. Ideally we don't want to rely on one machine to generate all the load for the performance tests. Doing so will push you into a situation where you might find yourself unneccessarily limited by network congestion and physical resource contention. 
  • Provide effective reporting facilities. We need to be able to analyze the results of our tests to determine whether any areas of the system haven't performed the way we expect. 
Luckily, the tools to satisfy these goals already exist, its just a case of putting them together. 



Tips and tricks for your JMeter tests

Ok, lets dive right in. I'm going to try and make this very example centric so you can see whats going on hopefully in the context of your own jmeter tests.

The Thread Group ( Simulating Users )



The thread group is one of the simplest and most powerful ways of simulating a user in your system. Requirements from the business are often expressed in these terms over the classic request per second metric. Thus, it can be a pragmatic approach to model the test in these terms.

Looking at the entries for the Thread Properties gives us a first look at JMeters often overlooked variable dsl, documented here. Lets take the first field as an example :

${__P(users,100)}

So what this statement is trying to do is resolve the value for the users property. If it can't it will use the provided default (100).

Pitfall 1 : Theres some weird truncation logic in play here through the ui ( which i'm assuming 90% of people will be using to edit these fields ), so make sure you check your field names.




Http Request Defaults



So two important factors here about making your test plans generally more maintainable and more robust. 

  1. Configurable domain - We set a default here so we can change the environment which we run our tests against. 
  2. Timeouts - This is a common pitfall. If you don't set your timeout you could potentially be in a situation where your test hangs forever. 
Pitfall 2 : Always set a timeout on http requests. 




Maven Integration (Lazerycode Plugin)

Now we have some tests we need to start thinking about how to run them outside the JMeter GUI. Thankfully, a decent maven plugin exists provided by Lazerycode

So lets assume that you've created a simple maven project in which to put your jmeter test files, properties file and plugin configuration. Lets review what you should probably have in there. 

pom.xml



The JMeter plugin, by default, looks in src/test/jmeter for test plans. If you need to change this you can consult the configuration documentation, but seems like a perfectly sensible place to put test plans.

I've provided some elaboration on the configuration options below:

  • ignoreResultFailures - If a single sampler request fails, you dont want to kill the whole test. We want to collect and display this information in the reporting phase and then make assertions at that point. 
  • remoteConfig -  Adding the remote config section will tell the plugin to send the test plan to your distributed nodes and collect the results. 
  • java.rmi.server.hostname - This will inform the remote nodes where to send results, which can help you avoid a lot of the network configuration problems you might have otherwise when running the test across multiple subnets. 
  • propertiesGlobal - Remember all those properties we were making configurable earlier? Heres where the integration comes in. The env.foo syntax is used because thats how variables will be passed from jenkins to the maven plugin. 
Next, we need to specify a couple of additional properties. Another quick look at the documentation will tell you that the plugin also checks the src/test/jmeter dir for a set of properties files. We want to, at this point, create a jmeter.properties file and populate it with the following values : 

remote_hosts=test-server-1.nerdability.com,test-server-2.nerdability.com,test-server-3.nerdability.com


Obviously using your own remote nodes for in place of the defaults i've specified here.

Thats it. Your plugin is now configured to orchestrate execution of your test plans. Onto reporting.


Reporting with Jenkins Performance Plugin

Now go ahead and install the Jenkins performance plugin, which will be responsible for producing pretty graphs from our test reports. After installing, set up a job which runs mvn verify on your project ( this will run the performance tests ). Next add the post build action Publish Performance test result report. It should look like this : 

Finally, remember all those variables we added in the tests earlier? Now we want to add those as parameters to the job so we can configure them when we run the job. 



These will be provided to the build under the env namespace. 

That's pretty much it, tweak and improve the performance tests and configuration as you see fit. The framework should support whatever you need to do. 

47 comments:

  1. What an Amazing post , really appreciate..

    ReplyDelete
  2. Thanks for sharing such informative blog!!! LoadRunner is very popular tool to validate a software application by generating actual load. It gives precise information about the software performance in real time. Loadrunner training in Chennai

    ReplyDelete
  3. Hi,you sharing to lot of useful information...This information is very popular to all sites....Wonderful..hadoop training chennai

    ReplyDelete
  4. Best provides Salesforce Training in Chennai with years of experienced professionals. Today's most of the IT industry use this software for Saelsforce Training. To know more details about salesforce reach Training in Chennai

    ReplyDelete
  5. Hi, this is Ganesh I am having 3 years of experience as a Dot Net developer and I am certified. I have Knowledge on OOPS Concepts in .NET indepth. After learning Salesforce will be enough to get a good career in IT with good Package? and i crossed Salesforce Training in Chennai website where someone please help me to identify the syllabus covers everything or not??

    Thanks, Ganesh

    ReplyDelete
  6. Hi, this is Ganesh I am having 3 years of experience as a Dot Net developer and I am certified. I have Knowledge on OOPS Concepts in .NET indepth. After learning Salesforce will be enough to get a good career in IT with good Package? and i crossed Salesforce Training in Chennai website where someone please help me to identify the syllabus covers everything or not??

    Thanks, Ganesh

    ReplyDelete
  7. Hello,
    Much obliged to you for the aide.In addition,I can suggest another,how to integrate JMeter tests into building process: install Jenkins/(ex-Hudson);download Jenkins plugin,register to performance testing cloud and upload JMeter script,start build – your test will be begun automatically in the cloud.The primary advantage of utilizing this cloud is that it is conceivable to configure test with up to 80000 clients in 5-10 minutes.
    ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,@
    hidden objects games

    ReplyDelete
  8. The information you posted here is useful to make my career better keep updates..I did Salesforce Training in Chennai at FITA academy. Its really useful for me to make bright future in IT industry.

    ReplyDelete
  9. I have read your blog and i got a very useful and knowledgeable information from your blog.its really a very nice article.You have done a great job . If anyone want to get Salesforce Course in Chennai, Please visit FITA academy located at Chennai Velachery.

    ReplyDelete
  10. The information you posted here is useful to make my career better keep updates...If anyone want to get Cloud Computing Training in Chennai, Please visit FITA academy located at Chennai Velachery which offer best Cloud Computing Course in Chennai.

    ReplyDelete
  11. I have read your blog and i got a very useful and knowledgeable information from your blog.its really a very nice article. I did Loadrunner Course in Chennai. This is really useful for me. Suppose if anyone interested to learn Manual Testing Course in Chennai reach FITA academy located at Chennai Velachery.

    ReplyDelete
  12. Hi, I wish to be a regular contributor of your blog. I have read your blog. Your information is really useful for beginner. I did Selenium Course in Chennai at Fita training and placement academy which offer best Software Testing Course in Chennai with years of experienced professionals. This is really useful for me to make a bright career.

    ReplyDelete
  13. Thanks for sharing this informative blog. FITA provides Salesforce Training in Chennai with years of experienced professionals and fully hands-on classes. Salesforce is a cloud based CRM software. Today's most of the IT industry use this software for customer relationship management. To know more details about salesforce reach FITA Academy. Rated as No.1 Salesforce Training Institutes in Chennai.

    ReplyDelete
  14. Java Training

    Hi I am Johnson lives in Chennai. I am a technology freak. Recently I did Java Course in Chennai at a leading Java Training Institutes in Chennai. This is really helpful for me to make a bright career in IT industry.

    Java Training in Chennai

    ReplyDelete
  15. Excellent post!!! In this competitive market, customer relationship management plays a significant role in determining a business success. That too, cloud based CRM product offer more flexibility to business owners to main strong relationship with the consumers. Salesforce Training Institutes in Chennai | Salesforce Training in Chennai

    ReplyDelete
  16. QTP Training in Chennai

    Hi, I wish to be a regular contributor of your blog. I have read your blog. Your information is really useful for beginner. I did Selenium Training in Chennai at Fita training and placement academy which offer best Software Testing Training in Chennai with years of experienced professionals. This is really useful for me to make a bright career.

    Regards...

    Software Testing Training Institutes in Chennai











    ReplyDelete
  17. Android Training in Chennai

    Your blog is really useful for me. Thanks for sharing this useful blog..Suppose if anyone interested to learn Android Course in Chennai please visit fita academy which offers best Android Training in Chennai at reasonable cost.

    Android Training Institutes in Chennai






    ReplyDelete
  18. Thanks this is the useful blog.I can able to understand about the jmeter basic and why we are studying the jmeter.Software Testing Training in Chennai

    ReplyDelete
  19. The future of software testing is on positive note. It offers huge career prospects for talented professionals to be skilled software testers. Automation training in Chennai | Software training | Software testing training institute Chennai

    ReplyDelete
  20. Unix Training

    Thanks for sharing this informative blog. Suppose if anyone interested to learn Unix Training in Chennai, Please visit Fita Academy located at Chennai, Velachery.

    Regards....

    Unix Training Institutes in Chennai

    ReplyDelete
  21. Expectation on testing is high because every code must be debug using this..

    dot net training in chennai | salesforce training in chennai

    ReplyDelete
  22. Wow that amazing article about software debug ..its very interesting article.. thank you

    dot net training in chennai | salesforce training in chennai

    ReplyDelete
  23. SAP Enterprise Service Oriented Architectonics (or Enterprise SOA) has been authentic by SAP SE as "an accessible architectonics for adaptive business solutions" and "the adapt for an architectonics that enables addition and acclimation in a distinct environment".
    Sap training in chennai | Sucess factor training in chennai | Sap training in chennai

    ReplyDelete

  24. I have read your blog it was nice to follow even I am looking for your future updates. Hadoop is a highly growing & scoopful technology in IT market it’s an open-source software framework for managing big data in a distributed fashion on large commodity computing hardware.
    Hadoop training in chennai | Hadoop courses in Chennai | Hadoop institutes in Chennai

    ReplyDelete
  25. SAP Training

    Thanks for sharing this valuable information.and I gathered some information from this blog. I did SAP Training in Chennai, at FITA Academy which offer best SAP Course Chennai with years of experienced professionals.

    ReplyDelete
  26. Excellent article!!! LoadRunner is popular automation testing tool used for validating a software application/system under load. It delivers most precise information about the performance, functionality and behavior of the software product. Loadrunner training in Chennai | Loadrunner training

    ReplyDelete
  27. Thanks for your wonderful post.It is really very helpful for us and I have gathered some important information from this blog.If anyone wants to get Dot Net Training in Chennai reach FITA, rated as No.1 Dot Net Training Institutes in Chennai.

    ReplyDelete
  28. I am following your blog from the beginning, it was so distinct & I had a chance to collect conglomeration of information that helps me a lot to improvise myself. I hope this will help many readers who are in need of this vital piece of information. Thanks for sharing & keep your blog updated.

    DOT NET Training in Chennai | DOT NET Training Institutes in Chennai

    ReplyDelete
  29. HI Michael,
    Valid information you have shared in this post, really appreciable one...
    Thanks,
    Web design training in Chennai

    ReplyDelete
  30. Hi, I wish to be a regular contributor of your blog. I have read your blog. Your information is really useful for us. I did Software Testing Course in Chennai at Fita training and placement academy which offer best Software Testing Training with years of experienced professionals. This is really useful for me to make a bright career.

    ReplyDelete
  31. Thanks for sharing this informative blog. I have read your blog, your information is really nice and useful for sap beginners. I have completed sap certification course at a leading IT academy. This is really useful for me to make a wonderful career. If anyone wants to get SAP Training Chennai visit FITA Academy.

    ReplyDelete
  32. Amazing post . thanks for your knowledge sharing . keep posting like this kind of activities .
    best data warehousing traning in chennai

    ReplyDelete
  33. Amazing post . thanks for your knowledge sharing .
    java training in chennai

    ReplyDelete
  34. • thank s for sharing the wonderful content with us.it helps us to know more shuttle services for their education services.
    soapui training in chennai

    ReplyDelete
  35. Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

    iOS Training in Chennai

    ReplyDelete