Executing Openstack-Rally test cases - rally

Question

I have Openstack installed in my Ubuntu server. I need to run all Rally test cases. I did rally deployment. Now I am able to execute single JSON file and getting the HTML and XML output.
eg:
root#ubuntu:/usr/share/rally/samples/tasks/scenarios/nova# rally task start list-images.json
This way I can execute individual JSON files only.
My requirement:
I have around 250 JSON files to be executed. How to execute them all in one shot?
What tools does the Openstack framework have to execute the entire rally cases(JSON files)?

Answer 1

Actually, you should not want to run 200 separated files. You would like to run one task that contains them. Rally allows you to put as many as you want test cases in single file. For example:
---
NovaServers.boot_and_delete_server:
-
args:
flavor:
name: "m1.tiny"
image:
name: "^cirros.*uec$"
force_delete: false
runner:
type: "constant"
times: 10
concurrency: 2
context:
users:
tenants: 3
users_per_tenant: 2
NovaServers.boot_and_list_server:
-
args:
flavor:
name: "m1.tiny"
image:
name: "^cirros.*uec$"
detailed: True
runner:
type: "constant"
times: 1
concurrency: 1
context:
users:
tenants: 1
users_per_tenant: 1
Take into account that Rally accept jinja2 templates, so you can use all features of jinja2 including "file includ options" Take a look here:
https://rally.readthedocs.io/en/latest/tutorial/step_5_task_templates.html

Answer 2

The best way is to write a script to run all tasks
ex:
#!/bin/bash
cd `dirname $0`
time=`date +%H:%M:%S`
mkdir -p testcase_result/$time
testcase_file=testcase_result/$time/rally_testcase.txt
total_file=testcase_result/$time/rally_total.txt
rally_task_dir=source/samples/tasks/scenarios
#keystone
keystone_case=`find $rally_task_dir/keystone -name "*.yaml"`
keystone_num=`grep -rn '\<Keystone' $keystone_case | wc -l`
echo "Keystone Testcases Number: "$keystone_num > $total_file
echo "Keystone" > $testcase_file
grep -rn '\<Keystone' $keystone_case | awk '{print NR":",$2}' >> $testcase_file
sed -i 's/Keystone.*\.//g' $testcase_file
#glance
glance_case=`find $rally_task_dir/glance -name "*.yaml"`
glance_num=`grep -rn '\<Glance' $glance_case | wc -l`
echo "Glance Testcases Number: "$glance_num >> $total_file
echo "" >> $testcase_file
echo "Glance" >> $testcase_file
grep -rn '\<Glance' $glance_case | awk '{print NR":",$2}' >> $testcase_file
sed -i 's/Glance.*\.//g' $testcase_file
#nova
nova_case=`find $rally_task_dir/nova -name "*.yaml"`
nova_num=`grep -rn '\<Nova' $nova_case | wc -l`
echo "Nova Testcases Number: "$nova_num >> $total_file
echo "" >> $testcase_file
echo "Nova" >> $testcase_file
grep -rn '\<Nova' $nova_case | awk '{print NR":",$2}' >> $testcase_file
sed -i 's/Nova.*\.//g' $testcase_file
#neutron
neutron_case=`find $rally_task_dir/neutron -name "*.yaml"`
neutron_num=`grep -rn '\<Neutron' $neutron_case | wc -l`
echo "Neutron Testcases Number: "$neutron_num >> $total_file
echo "" >> $testcase_file
echo "Neutron" >> $testcase_file
grep -rn '\<Neutron' $neutron_case | awk '{print NR":",$2}' >> $testcase_file
sed -i 's/Neutron.*\.//g' $testcase_file
#cinder
cinder_case=`find $rally_task_dir/cinder -name "*.yaml"`
cinder_num=`grep -rn '\<Cinder' $cinder_case | wc -l`
echo "Cinder Testcases Number: "$cinder_num >> $total_file
echo "" >> $testcase_file
echo "Cinder" >> $testcase_file
grep -rn '\<Cinder' $cinder_case | awk '{print NR":",$2}' >> $testcase_file
sed -i 's/Cinder.*\.//g' $testcase_file
#total
let total=$keystone_num+$glance_num+$nova_num+$neutron_num+$cinder_num
echo "Total Testcases Number: $total" >> $total_file
sed -i 's/:$//' $testcase_file
# Run Scripts tests
cd testcase_result/$time
for i in ../../rally_scripts/*.sh
do
bash $i
done