[IDCF] A story about tweaking CloudStackAPI when building a jmeter execution environment for load testing

My name is Teraoka and I am an infrastructure engineer.
For a certain project, it was necessary to build a jmeter execution environment on IDCF, and I would
like to introduce some parts that I devised using CloudStack API during the construction stage.

■What did you want to do in the first place?

I wanted to configure it to automatically recognize the IP address of the server that plays the role of the corresponding node when starting jmeter on the server that plays the role of the jmater controller.

Yes, that's what I mean.
Jmeter has a mechanism called
remote testing
by registering the servers (nodes) on which remote jmeter is running in
the configuration file of the controller server, it is possible to generate more load than the number of registered servers. You will be able to do it.

However, the number of registered servers changes depending on the load generated,
and it is troublesome to change the configuration file each time, so

when
starting jmeter on the controller side by determining only the nodes that are running, I made it rewrite automatically.

■What exactly did you do?

When registering a remote server in jmeter,
add settings to the item "remote_hosts" in the file "jmeter.properties".
An example is shown below↓

remote_hosts=10.31.X.XX,10.31.X.XX,10.31.X.XX

As shown above, you will be adding IP addresses separated by ",", so
first you need to obtain the IP addresses of the remote servers all at once.
I used CloudStackAPI to obtain this.

CloudStackAPI is written in python, so it can be installed relatively easily using pip.
The installation method is described on the official IDCF page, so please refer to it.

Getting Started | IDCF Documentation

API references are also summarized on the official page, so please take a look.

API References | IDCF Documentation

Let's get started.
Here is the command to get the IP addresses of remote servers all at once ↓

cloudstack-api listVirtualMachines --name "jmeter-node" --state "Running" | jq -r '.listvirtualmachinesresponse.virtualmachine[].nic[].ipaddress' | tr "\n" "," | sed -e " s/,$//g"

Breaking down what this command does, it looks like this:

① Obtain a list of virtual server information using CloudStackAPI

②Format the json data using the jq command

③Format the formatted json data into a “,” delimited format using the tr command

① Obtain a list of virtual server information using CloudStackAPI

cloudstack-api listVirtualMachines --name "jmeter-node" --state "Running"

This part.

This time, I used "listVirtualMachines" among the many methods of "CloudStackAPI".
As the name suggests, this is a method to obtain a "list of virtual server information."
The execution result of this method is returned in the following json format.

$ cloudstack-api listVirtualMachines { "listvirtualmachinesresponse": { "count": 1, "virtualmachine": [ { "nic": [ { "ipaddress": "XXX.XXX.XXX.XXX" } ] } ] } }

*Many items other than "ipaddress" are returned as a response, but
we have excerpted only the necessary parts in this article.

All you have to do is extract the value of the item “ipaddress” from this json data!
When actually executing the command, use "--name" to narrow down to only the servers that will become nodes
, and from among those, use "--state" to narrow down the virtual servers whose state is Running.
What you mean is "extract information only from nodes that are running, excluding those that are stopped."

②Format the json data using the jq command

jq -r '.listvirtualmachinesresponse.virtualmachine[].nic[].ipaddress'

This part.

Since the CloudStack API response is returned as json data,
you must format the json data so that you can extract only the data you want.
In this case, the item is "ipaddress".
We will tell the jq command the data to be extracted from json in a hierarchical structure.

③Format the formatted json data into a “,” delimited format using the tr command

tr "\n" "," | sed -e "s/,$//g"

This part.

When you format the json data in ②, it will have the following format.

10.31.X.XX 10.31.X.XX 10.31.X.XX

...It's a polite line break, isn't it?
As it is, it cannot be embedded in the jmeter configuration file, so
you need to convert the line feed code to ",".
Also, if you convert, an extra "," will be added at the end, so
use the sed command to remove that part.

Then...

10.31.X.XX,10.31.X.XX,10.31.X.XX

Great job, now you can just embed it in your configuration file.
After that, edit the configuration file before starting jmeter.
Let's create a simple startup script.

$ vi start-controller.sh #!/bin/sh NODE=$(cloudstack-api listVirtualMachines --name "jmeter-node" --state "Running" | jq -r '.listvirtualmachinesresponse.virtualmachine[].nic[] .ipaddress' | tr "\n" "," | sed -e "s/,$//g") sed -i -e "s/^remote_hosts=.*/remote_hosts=${NODE}/" /usr /local/jmeter/bin/jmeter.properties /usr/local/jmeter/bin/jmeter & $ ./start-controller.sh

Yes, now the nodes will be automatically recognized when you start jmeter.
That's all, thank you very much.