Robot Framework: How to avoid a variable value from being displayed in the console during execution - testing

I have the API Key to a server available in clear text in our tests.
To avoid this I came up with a new python script Secure.py (This includes 'encrypt' and 'decrypt' functions) and its working fine.
I have a decrypt keyword in my robot tests.
*** Variables ***
${secret_phrase} abcdefghij
${encrypted_Key} ardfvbjgfrtavhdimdbshajakiugbn
I have used a secret phrase and the API key to encrypt and this is the encrypted key.
*** Test Cases ***
decrypt
${APIKEY} Secure.Decrypt ${secret_phrase} ${encrypted_Key}
Should Not Be Empty ${APIKEY} shell=no
Set Suite Variable ${APIKEY} shell=no
In this keyword I am passing in secret phrase and encrypted key so as a result I get the APIKEY.
#APIKEY result is printed in the Console, I don't want it to be printed in the console is there any way to do so?
Console:
${APIKEY}= asdfghjkl

You can use the Set Log Level to disable all logging in your critical sections of the test case. This keyword will return the previous log level, which you could store in a variable and then you can use it later to restore the original log level.
Sets the log threshold to the specified level and returns the old
level.
Messages below the level will not logged. The default logging level is
INFO, but it can be overridden with the command line option
--loglevel.
The available levels: TRACE, DEBUG, INFO (default), WARN, ERROR and NONE (no logging).
An example:
*** Variables ***
${secret_phrase} abcdefghij
${encrypted_Key} ardfvbjgfrtavhdimdbshajakiugbn
*** Test Case ***
decrypt
${old log level}= Set Log Level NONE # disable logging
${APIKEY} Set Variable ${secret_phrase}_${encrypted_Key}
Should Not Be Empty ${APIKEY}
Set Suite Variable ${APIKEY}
Set Log Level ${old log level} # enable logging
${public}= Set Variable This variable can be visible.
Here you can see the between the two Set Log Level call, nothing is logged, but after the original log level (INFO) has been restored logging is working again.

Related

Robotframework - get failing keyword / stack trace of failure

I have a keyword called "debug teardown" which prints the test status and then runs the debuglibrary Debug keyword, if the test has failed.
I would like to be able to log to console which keyword has caused the failure, so I can more effectively debug my test.
Is it possible to get the stack trace or most recent test keyword, and log it to the console?
Here is my Debug Teardown keyword:
Debug Teardown
Run Keyword If Test Failed Log ${TEST STATUS}: ${TEST MESSAGE} ERROR
Run Keyword If Test Failed Debug
You can get a bit more information if you create a listener and also set the log level to DEBUG. Inside the listener you can save the results of log commands, and then when a keyword fails you can print it out or do whatever else you want.
For example, here's a listener that will print to stdout the last log message when a keyword fails:
from __future__ import print_function
class my_listener():
ROBOT_LISTENER_API_VERSION = 2
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.last_log = None
def _log_message(self, message):
self.last_log = message
def _end_keyword(self, name, attrs):
if attrs['status'] == 'FAIL':
print("\n******\n", self.last_log['message'])
You would use it by importing the listener like a normal library, and also setting the log level to DEBUG (otherwise you'll get the error but no stack trace).
Example:
*** Settings ***
Library my_listener
Suite Setup set log level DEBUG
*** Test cases ***
Example
some keyword
You might be able to utilize set suite variable to update a "global" variable as you go. The last set variable value would be the value that failed.

robot framework: exception handling

Is it possible to handle exceptions from the test case? I have 2 kinds of failure I want to track: a test failed to run, and a test ran but received the wrong output. If I need to raise an exception to fail my test, how can I distinguish between the two failure types? So say I have the following:
*** Test Cases ***
Case 1
Login 1.2.3.4 user pass
Check Log For this log line
If I can't log in, then the Login Keyword would raise an ExecutionError. If the log file doesn't exist, I would also get an ExecutionError. But if the log file does exist and the line isn't in the log, I should get an OutputError.
I may want to immediately fail the test on an ExecutionError, since it means my test did not run and there is some issue that needs to be fixed in the environment or with the test case. But on an OutputError, I may want to continue the test. It may only refer to a single piece of output and the test may be valuable to continue to check the rest of the output.
How can this be done?
Robot has several keywords for dealing with errors, such as Run keyword and ignore error which can be used to run another keyword that might fail. From the documentation:
This keyword returns two values, so that the first is either string
PASS or FAIL, depending on the status of the executed keyword. The
second value is either the return value of the keyword or the received
error message. See Run Keyword And Return Status If you are only
interested in the execution status.
That being said, it might be easier to write a python-based keyword which calls your Login keyword, since it will be easier to deal with multiple exceptions.
You can use something like this
${err_msg}= Run Keyword And Expect Error * <Your keyword>
Should Not Be Empty ${err_msg}
There are couple of different variations you could try like
Run Keyword And Continue On Failure, Run Keyword And Expect Error, Run Keyword And Ignore Error for the first statement above.
Option for the second statement above are Should Be Equal As Strings, Should Contain, Should Match.
You can explore more on Robot keywords

Configuring SLURM so it requires the user to specify --account

I'm trying to figure out how to configure SLURM so that a user is required to specify --account when using the SLURM commands (salloc, sbatch, srun). Effectively I want to disable the default account behavior.
Has anyone out there found a simple way to do this?
I had the same requirement to force users to specify accounts and, after finding several ways to fulfill it with slurm, I decided to revive this post with the shortest/easiest solution.
The slurm lua submit plugin sees the job description before the default account is applied. Hence, you can install the slurm-lua package, add "JobSubmitPlugins=lua" to the slurm.conf, restart the slurmctld, and directly test against whether the account was defined via the job_submit.lua script (create the script wherever you keep your slurm.conf; typically in /etc/slurm/):
-- /etc/slurm/job_submit.lua to reject jobs with no account specified
function slurm_job_submit(job_desc, part_list, submit_uid)
if job_desc.account == nil then
slurm.log_error("User %s did not specify an account.", job_desc.user_id)
slurm.log_user("You must specify an account!")
return slurm.ERROR
end
return slurm.SUCCESS
end
function slurm_job_modify(job_desc, job_rec, part_list, modify_uid)
return slurm.SUCCESS
end
return slurm.SUCCESS
Errors resulting from not specifying an account appear as follows:
# srun --pty bash
srun: error: You must specify an account!
srun: error: Unable to allocate resources: Unspecified error
# sbatch submit.slurm
sbatch: error: You must specify an account!
sbatch: error: Batch job submission failed: Unspecified error
These errors are also printed out to the slurmctld log so that you know what the resource allocation issue was for the particular job:
[2017-09-12T08:32:00.697] error: job_submit.lua: User 0 did not specify an account.
[2017-09-12T08:32:00.697] _slurm_rpc_submit_batch_job: Unspecified error
As an addendum, the Slurm Submit Plugins Guide is only moderately useful and you will probably be much better off simply examining the Lua job_submit plugin implementation for guidance.
One option is to set the AccountingStorageEnforce parameter to associations in slurm.conf.
AccountingStorageEnforce
This controls what level of association-based enforcement to impose on job submissions. Valid options are any combination of
associations, limits, nojobs, nosteps, qos, safe, and wckeys, or all
for all things (expect nojobs and nosteps, they must be requested as
well).
By enforcing Associations no new job is allowed to run unless a corresponding association exists in the system. If limits are enforced
users can be limited by association to whatever job size or run time
limits are defined.
Then, with the sacctmgr command, make sure the default account has no access to the defined partitions. Effectively, the users will be denied submission if they do not specify a valid account.
Another option is to write a custom submission plugin, which you can write in Lua. In that script, you can check whether the --account parameter was set and deny submission with a custom message if it was not.

Redis SET command can't fail, but can?

I'm trying to debug some Redis issues I am experiencing and came by some inconclusive documentation about the SET command.
In my Redis config; I have the following lines (snippet):
# Note: with all the kind of policies, Redis will return an error on write
# operations, when there are not suitable keys for eviction.
#
# At the date of writing this commands are: set setnx setex append
On the documentation page for the SET command I found:
Status code reply: always OK since SET can't fail.
Any insights on the definitive behaviour?
tl;dr: SET will return an error response if the redis instance runs out of memory.
As far as I can tell from the source code in redis.c, esentially when a command is to be processed the flow goes like this (pseudo code):
IF memory is needed
IF we can free keys
Free keys
Process the command
SET -> process and return OK response
ELSE return error response
ELSE
Process command
SET -> process and return OK response
It's not exactly written this way, but the basic idea comes down to that: memory is being checked before the command is processed, so even if the command cannot fail, an error response will be returned if there's no memory regardless the actual response of the command.

CTest build ID not set

I have a CDash configured to accept posts for automatic builds and tests. However, when any system attempts to post results to the CDash, the following error is produced. The result is that each result gets posted four times (presumably the original posting attempt plus the three retries).
Can anyone give me a hint as to what sets this mysterious build ID? I found some code that seems to produce a similar error, but still no lead on what might be happening.
Build::GetNumberOfErrors(): BuildId not set
Build::GetNumberOfWarnings(): BuildId not set
Submit failed, waiting 5 seconds...
Retry submission: Attempt 1 of 3
Server Response:
The buildid for CDash is computed based on the site name, the build name and the build stamp of the submission. You should have a Build.xml file in a Testing/20110311-* directory in your build tree. Open that up and see if any of those fields (near the top) is empty. If so, you need to set BUILDNAME and SITE with -D args when configuring with CMake. Or, set CTEST_BUILD_NAME and CTEST_SITE in your ctest -S script.
If that's not it, then this is a mystery. I've not seen this error occur before...
I'm having the same issue though Site and Buildname are available in test.xml and are visible on cdash (4 times). I can see the jobs increment by refreshing between retries so it seems that the submission succeeds and reports a timeout.
Update: This seems to have started when I added the -j(nprocs) switch to the ctest command. changing CtestSubmitRetryDelay: 20 (was 5) allowed a server response through that indicates the cdash version may not be able to handle the multi-proc option I'll have to look into that for my issue. Perhaps setting CtestSubmitRetryDelay to a larger number will get you back a server response as it did for me. g'luck!
Out of range value for column 'processorclockfrequency'