HTTP Simple Table Server

Download

Performance testing with JMeter can be done with several JMeter injectors (on a remote host) and one JMeter controller (with GUI, on your local host). Scripts are sent to JMeter injectors using RMI protocol. Results are brought back periodically to the JMeter controller. Unfortunately the dataset and csv files aren't transferred from the controller to injectors.

The main idea is to use a tiny http server in JMeter Plugins to manage the dataset files with simple commands to get / add rows of data in files.

Configuration

In jmeter.properties file, STS properties are :

jmeterPlugin.sts.port=9191
jmeterPlugin.sts.addTimestamp=true
jmeterPlugin.sts.datasetDirectory=D:/jmeter/dataset
jmeterPlugin.sts.loadAndRunOnStartup=false
jmeterPlugin.sts.daemon=false
jmeterPlugin.sts.charsetEncodingHttpResponse=UTF-8
jmeterPlugin.sts.charsetEncodingReadFile=UTF-8
jmeterPlugin.sts.charsetEncodingWriteFile=UTF-8
jmeterPlugin.sts.initFileAtStartup=article.csv,users.csv
jmeterPlugin.sts.initFileAtStartupRegex=false
jmeterPlugin.sts.initFileAtStartup=file\d+\.csv
jmeterPlugin.sts.initFileAtStartupRegex=true


Do not use '\' in the path directory, it doesn't work well, use '/' or '\' instead. It is also recommended to use UTF-8 as the encoding:

sampleresult.default.encoding=UTF-8

If you want automatically start a Simple Table Server on JMeter STARTUP simply add jsr223.init.file=simple-table-server.groovy in jmeter.properties Be sure that simple-table-server.groovy file is in your JMETER_HOME/bin directory.

The Simple Table Server is a tiny http server which can send http GET/POST requests on port 9191 (by default). You can set a custom port through the graphical user interface or by overriding the jmeterplugin.sts.port property.

Distributed architecture for JMeter

The Simple Table Server runs on the JMeter controller (master) and load generators (slaves) make calls to the STS to get or add some data. At the beginning of the test, the first load generator will load data in memory (initial call) and at the end of the test it asks the STS saving values in a file. All the load generators ask data from the same STS which is started on the JMeter controller.

Getting Started

There are different ways to start the STS:

  • with JMeter GUI:

  • with simple-table-server.cmd (.sh for UNIX) script located in your JMETER_HOME/bin directory. Parameters are read in the jmeter.properties or you could also set parameters like simple-table-server.cmd -DjmeterPlugin.sts.addTimestamp=true -DjmeterPlugin.sts.datasetDirectory=D:/jmeter/dataset, set jmeterPlugin.sts.daemon=true when you want launch the STS with the nohup linux command like process daemon, the STS will not waiting the <ENTER> key to stop.
  • on JMeter CLI startup (Windows : jmeter-n.cmd or Linux jmeter -n) following properties in jmeter.properties file or in user.properties :
jmeterPlugin.sts.loadAndRunOnStartup=true
jmeterPlugin.sts.port=9191
jmeterPlugin.sts.daemon=false
  
jsr223.init.file=simple-table-server.groovy

When the STS is running go to http://<HOST>:<PORT>/sts/ to see all available commands and a short help.

Calls are synchronized, all commands are executed one by one.

Example of a dataset file logins.csv:

login1;password1
login2;password2
login3;password3
login4;password4
login5;password5

INITFILE

Load file in memory. Lines are stored in a linked list, 1 line = 1 element

The filename is limited to 128 characters maxi and must not contains characters \ / : or ..

http://hostname:port/sts/INITFILE?FILENAME=logins.csv

HTML format:

  1. <html><title>OK</title>
  2. <body>5</body><!-- number of lines read -->
  3. </html>

Linked list after this command:

login1;password1
login2;password2
login3;password3
login4;password4
login5;password5

The charset to read the file is jmeterPlugin.sts.charsetEncodingReadFile=<charset>, default value = System property : file.encoding

File could be load when STS startup with

jmeterPlugin.sts.initFileAtStartupRegex=false
jmeterPlugin.sts.initFileAtStartup=article.csv,users.csv
OR
jmeterPlugin.sts.initFileAtStartupRegex=true
jmeterPlugin.sts.initFileAtStartup=file\d+\.csv


jmeterPlugin.sts.initFileAtStartupRegex=false and jmeterPlugin.sts.initFileAtStartup=file1.csv,file2.csv,otherfile.csv read csv files with comma separator (not a regular expression), e.g : read file1.csv and file2.csv and otherfile.csv

OR

jmeterPlugin.sts.initFileAtStartupRegex=true and jmeterPlugin.sts.initFileAtStartup=.+\.csv read all csv files in the dataset directory the jmeterPlugin.sts.initFileAtStartup value is a regular expression

READ

Get one line from list

The charset use in the response is jmeterPlugin.sts.charsetEncodingHttpResponse=<charset>, default value = JMeter property : sampleresult.default.encoding

http://hostname:port/sts/READ?READ_MODE={FIRST, LAST, RANDOM}&KEEP={TRUE, FALSE}&FILENAME=logins.csv

HTML format:

  1. <html><title>OK</title>
  2. <body>login1;password1</body>
  3. </html>

Available options:

  • READ_MODE=FIRST => login1;password1
  • READ_MODE=LAST => login5;password5
  • READ_MODE=RANDOM => login?;password?
  • KEEP=TRUE => the data is kept and put to the end of list
  • KEEP=FALSE => the data is removed

KEEP=TRUE, READ_MODE=FIRST => login1;password1

Linked list after this command:

login2;password2
login3;password3
login4;password4
login5;password5
login1;password1

KEEP=TRUE, READ_MODE=LAST => login5;password5

Linked list after this command:

login1;password1
login2;password2
login3;password3
login4;password4
login5;password5

KEEP=TRUE, READ_MODE=RANDOM => login2;password2

Linked list after this command:

login1;password1
login3;password3
login4;password4
login5;password5
login2;password2

KEEP=FALSE (delete mode), READ_MODE=FIRST => login1;password1

Linked list after this command:

login2;password2
login3;password3
login4;password4
login5;password5

KEEP=FALSE, READ_MODE=LAST => login5;password5

Linked list after this command:

login1;password1
login2;password2
login3;password3
login4;password4

KEEP=FALSE, READ_MODE=RANDOM => login2;password2

Linked list after this command:

login1;password1
login3;password3
login4;password4
login5;password5

ADD

Add a line into a file: (GET OR POST HTTP protocol)

GET Protocol

http://hostname:port/sts/ADD?FILENAME=dossier.csv&LINE=D0001123&ADD_MODE={FIRST, LAST}&UNIQUE={FALSE, TRUE}

GET Parameters : FILENAME=dossier.csv&LINE=D0001123&ADD_MODE={FIRST, LAST}&UNIQUE={FALSE, TRUE}

POST Protocol

http://hostname:port/sts/ADD

POST Parameters : FILENAME=dossier.csv, LINE=D0001123, ADD_MODE={FIRST, LAST}, UNIQUE={FALSE, TRUE}

HTML format:

  1. <html><title>OK</title>
  2. <body></body>
  3. </html>

Available options:

  • ADD_MODE=FIRST => add to the top
  • ADD_MODE=LAST => add to the end
  • FILENAME=dossier.csv => if doesn't already exist it creates a LinkList in memory
  • LINE=1234;98763 => the line to add
  • UNIQUE=TRUE => do not add line if the list already contains such line

POST Protocol with parameters

LENGTH

Return the number of remaining lines of a linked list
http://hostname:port/sts/LENGTH?FILENAME=logins.csv

HTML format:

  1. <html><title>OK</title>
  2. <body>5</body><!-- remaining lines -->
  3. </html>

STATUS

Display the list of loaded files and the number of remaining lines for each linked list
http://hostname:port/sts/STATUS

HTML format:

  1. <html><title>OK</title>
  2. <body>
  3. logins.csv = 5<br />
  4. dossier.csv = 1<br />
  5. </body></html>

SAVE

Save the specified linked list in a file to the specified location

The charset use to write the file is set with meterPlugin.sts.charsetEncodingWriteFile=<charset>, default value = System property : file.encoding

http://hostname:port/sts/SAVE?FILENAME=logins.csv

If jmeterPlugin.sts.addTimestamp is set to true then a timestamp will be add to the filename, the file is stored in the custom directory specified by editing the jmeterPlugin.sts.datasetDirectory property or in JMETER_HOME/bin directory by default:

20140520T16h33m27s.logins.csv

HTML format:

  1. <html><title>OK</title>
  2. <body>5</body><!-- number of lines saved -->
  3. </html>

You can force the addTimestamp value with parameter ADD_TIMESTAMP in the url like :

http://hostname:port/sts/SAVE?FILENAME=logins.csv&ADD_TIMESTAMP=[true,false]

RESET

*Remove all of the elements from the specified list*
http://hostname:port/sts/RESET?FILENAME=logins.csv

HTML format:

  1. <html><title>OK</title>
  2. <body></body>
  3. </html>

Always returns OK even if the file did not exist

STOP

Shutdown the Simple Table Server
http://hostname:port/sts/STOP

When the jmeterPlugin.sts.daemon=true, you need to call http://hostname:port/sts/STOP or kill the process to stop the STS

CONFIG

Display the current configuration for the Simple Table Server
http://hostname:port/sts/CONFIG

See examples below for further information.

Using STS in a Test Plan

Initialize file using a "setUp Thread Group" by calling URL with one or more HTTP Request Sampler.

Reading a row of data is done by calling READ method at each iteration by a HTTP Request Sampler. Then you can use a Regular Expression Extractor to parse the response data.

Reading login:

Reading password:

At the end of your Test Plan you can save remaining/adding data with a HTTP Request Sampler in a "tearDown Thread Group".

If you need to parse more than 2 columns, you could use a Post-Processeur groovy JSR223 like :

  1. // E.g. read line = <body>COL1VALUE;COL2VALUE;COL3VALUE;COL4VALUE;COL5VALUE</body>
  2. // column separator = ';'
  3. String lineData = org.apache.commons.lang3.StringUtils.substringBetween(prev.getResponseDataAsString(),"<body>","</body>");
  4. // lineData = COL1VALUE;COL2VALUE;COL3VALUE;COL4VALUE;COL5VALUE
  5. def tabParam = org.apache.commons.lang3.StringUtils.splitPreserveAllTokens(lineData,';'); // use column separator = ';'
  6.  
  7. if (tabParam.length == 5) {
  8. vars.put("V_COL1PARAM", tabParam[0]); // variable V_COL1PARAM = COL1VALUE (String value)
  9. vars.put("V_COL2PARAM", tabParam[1]); // variable V_COL2PARAM = COL2VALUE (String value)
  10. vars.put("V_COL3PARM", tabParam[2]);
  11. vars.put("V_COL4PARAM", tabParam[3]);
  12. vars.put("V_COL5PARM", tabParam[4]);
  13. }
  14. else {
  15. log.info("ERROR, Need 5 columns");
  16. }

Examples

  • Put the logins.csv file in your JMETER_HOME/bin directory:
Download logins.csv file
  • Run the Simple Table Server manually with the simple-table-server.cmd file or automatically with groovy configuration.
  • Run one of the following scripts:

In a loop, read random values from a file containing a login and a password at each row:

Download Example Test Plan 1 Read random values example

Read value from a file containing a login and a password at each row, each value is unique and cannot be read anymore:

Download Example Test Plan 2 Consume login lines example

Add rows in a new linked list and save it in a file when the test is done:

Download Example Test Plan 3 Save list example

Read in a random mode a dataset located on the controller machine with severals slaves. The first injector loads the dataset in memory while the other injectors are waiting few seconds (you could also use meterPlugin.sts.initFileAtStartup=file1.csv,file2.csv and jmeterPlugin.sts.initFileAtStartupRegex=false to read files when STS startup) The different injectors read randomly the data containing logins and passwords. When the test is done the first injector save the values in a file with a timestamp as prefix:

Download Example Test Plan 4 Multi JMeter injectors and save list examples

You can override STS settings using command-line options:

  • -DjmeterPlugin.sts.port=<port number>
  • -DjmeterPlugin.sts.loadAndRunOnStartup=<true/false>
  • -DjmeterPlugin.sts.datasetDirectory=<path/to/your/directory>
  • -DjmeterPlugin.sts.addTimestamp=<true/false>
  • -DjmeterPlugin.sts.daemon=<true/false>
  • -DjmeterPlugin.sts.charsetEncodingHttpResponse=<charset like UTF-8>
  • -DjmeterPlugin.sts.charsetEncodingReadFile=<charset like UTF-8>
  • -DjmeterPlugin.sts.charsetEncodingWriteFile=<charset like UTF-8>
  • -DjmeterPlugin.sts.initFileAtStartup=<files to read when STS startup, e.g : article.csv,users.csv>
  • -DjmeterPlugin.sts.initFileAtStartupRegex=false=<false : no regular expression, files with comma separator, true : read files matching the regular expression>
jmeter.bat -DjmeterPlugin.sts.loadAndRunOnStartup=true -DjmeterPlugin.sts.port=9191 -DjmeterPlugin.sts.datasetDirectory=d:/data -DjmeterPlugin.sts.charsetEncodingReadFile=UTF-8 -n –t testdemo.jmx

When it's done see results in the Listener Tree View.

JMETER MAVEN PLUGIN

If you want to use the Http Simple Server with the JMeter Maven plugin, you could :

  • put your csv files in <project>/src/test/jmeter directory (e.g : logins.csv)
  • put the simple-table-server.groovy (groovy script) in <project>/src/test/jmeter directory
  • put the your jmeter script in <project>/src/test/jmeter directory (e.g : test_login.jmx)
  • declare in the maven build section, in the configuration > jmeterExtensions > declare the artifact kg.apc:jmeter-plugins-table-server:<version>
  • declare user properties for STS configuration and automatic start

Extract pom.xml dedicated to Http Simple Table Server :

  1. <build>
  2. <plugins>
  3. <plugin>
  4. <groupId>com.lazerycode.jmeter</groupId>
  5. <artifactId>jmeter-maven-plugin</artifactId>
  6. <version>3.5.0</version>
  7. ...
  8. <configuration>
  9. <jmeterExtensions>
  10. <artifact>kg.apc:jmeter-plugins-table-server:3.1</artifact>
  11. </jmeterExtensions>
  12. <propertiesUser>
  13. <!-- properties configuration for Http Simple Table Server with automatic start when JMeter start -->
  14. <jmeterPlugin.sts.port>9191</jmeterPlugin.sts.port>
  15. <jmeterPlugin.sts.addTimestamp>true</jmeterPlugin.sts.addTimestamp>
  16. <jmeterPlugin.sts.datasetDirectory>${project.build.directory}/jmeter/testFiles</jmeterPlugin.sts.datasetDirectory>
  17. <jmeterPlugin.sts.loadAndRunOnStartup>true</jmeterPlugin.sts.loadAndRunOnStartup
  18. <jsr223.init.file>${project.build.directory}/jmeter/testFiles/simple-table-server.groovy</jsr223.init.file>
  19. </propertiesUser>
  20. </configuration>
  21. </plugin>
  22. </plugins>
  23. </build>