How to run command on Zabbix agents? - sysadmin

Question

I want to run a command on Zabbix agents:
Some simple unix commands, to obtain our reporting data.
When there is some processing required on the agent side.
There seem to be a variety approaches being talked about. So how to execute such commands on a Zabbix agent?

Answer 1

Run commands from the server directly from a new item.
First, set: EnableRemoteCommands=1 in the agent conf file (for all of your agents). To enable this feature.
Create a new item. A field on the "new item" page says 'key'. Enter:
system.run[command]
As the 'key' string. Where command is the command you want to be downloaded and run on the agent. Here is an example:
system.run[sysctl dev.cpu.0.temperature | cut -d ' ' -f 2 | tr -d C]
Perhaps you need to run something substantially more complex that is too long to fit in there? Then you will need to make a custom script. Put your custom scripts on a local webserver, or somewhere on the web.
Then you might set the item's key to:
system.run[ command -v script && script || wget script_url -O /path/to/script && script]
To fetch and download the missing script to the agent the first time it's executed. However that is a rather crude hack. Not very elegant.
A better way is to go to "Administration" --> "Scripts" in the menu. From there, you can create a new script to use in an item which may be configured to run on any of your agents.
Make a special custom item to re-run your script periodically (like a cron job). The job of the special script item is to update the agent with a collection of your other needed custom scripts.
Of course you could just write all of your custom scripts directly into zabbix's MYSQL database. And it is very tempting to do that. But be aware that then they'd be lost and vulnerable if your zabbix database ever gets fried or corrupted / lost. Zabbix databases always have a habit of growing large, unwieldy and out-of-control. So don't do that. Storing them separately somewhere else and under version control (git or subversion).
Once that's all sorted, we can finally go ahead and create further custom items to run your custom scripts. Again using:
system.run[script]
as the item's key just as before. Where 'script' is the command (plus any arguments), to execute your custom script locally on the agent.

Answer 2

Define the user parameter at the client (where zabbix agent is
located) at /etc/zabbix/zabbix_agentd.conf
The key should be
unique. I am using lsof as an example: UserParameter=open_file,lsof | wc -l
Restart the agent: service zabbix-agent restart
Test if the key is working using zabbix_get utility. To do that from the zabbix server, invoke the following: /usr/local/bin/zabbix_get -s <HOST/IP of the zabbix agent> -k open_file (It should return a number in this case)
Create an item with the key at the zabbix server at the template
level (the return type should be correctly defined, otherwise zabbix
will not accept it):
Type: Zabbix Agent (Active)
Key: open_file
Type of Information: Numeric (unsigned)
Data Type: decimal
You may create a graph using the item to monitor the value at
regular interval.
Here is the official documentation.