I am reading a .dat file using the following command
accel_x = np.loadtxt('./GroundMotions/GM/%s_%s_%s_%s_x.dat' % (station, realization, soilpro, profile))
where station, realization, soilpro, and profile are all Strings.
Then python throws me an error:
FileNotFoundError: [Errno 2] No such file or directory: './GroundMotions/GM/Z0XOCS_csz002_C4_WA-DNR-08_14\r_x.dat'
There is an unexpected \r in the error message. How can I avoid this unexpected carriage return?
You could strip it:
accel_x = np.loadtxt('./GroundMotions/GM/%s_%s_%s_%s_x.dat' % (station, realization, soilpro.strip('\r'), profile))
Or to strip all parameters:
'./GroundMotions/GM/%s_%s_%s_%s_x.dat' % tuple(str.strip(s, '\r') for s in (station, realization, soilpro, profile))
Related
Running rsyslogd -N 1, I receive two complains :
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 27: parameter 'RebindInterval' not known -- typo in config file? [v8.27.0 try http://www.rsyslog.com/e/2207 ]
and a complain about last line :
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 43: warnings occured in file '/etc/rsyslog.conf' around line 43 [v8.27.0 try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 43: invalid character '(' - is there an invalid escape sequence somewhere? [v8.27.0 try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 43: warnings occured in file '/etc/rsyslog.conf' around line 43 [v8.27.0 try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 43: invalid character '=' - is there an invalid escape sequence somewhere? [v8.27.0 try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 43: invalid character '"' - is there an invalid escape sequence somewhere? [v8.27.0 try http://www.rsyslog.com/e/2207 ]
It' s strange because rebindInterval should be known : https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html#rebindinterval
Also last line is straight as in the doc : https://www.rsyslog.com/doc/v8-stable/rainerscript/include.html#include-multiple-files
The config is as follows rsyslog 8.28 on Ubuntu 16.04.
# /etc/rsyslog.conf Configuration file for rsyslog
module(load="imuxsock")
module(
load="imklog"
LogPath="/proc/kmsg"
)
module(load="imudp")
input(
type="imudp"
port="514"
)
module(load="imtcp")
input(
type="imtcp"
port="514"
)
module(
load="builtin:omfwd"
# RebindInterval="5000" # ERROR shown by rsyslogd -N 1 : parameter 'RebindInterval' not known.
)
module(
load="builtin:omfile"
template="RSYSLOG_TraditionalFileFormat"
fileOwner="syslog"
fileGroup="syslog"
fileCreateMode="0640"
dirCreateMode="0755"
)
global(
preserveFQDN="on"
umask="0022"
)
include(file="/etc/rsyslog.d/*.conf")
I am running in snakefile on cluster using this default config:
"__default__":
"account" : "myAccount"
"queue" : "myQueue"
"nCPUs" : "16"
"memory" : 20000
"resources" : "\"select[mem>20000] rusage[mem=20000] span[hosts=1]\""
"name" : "JOBNAME.{rule}.{wildcards}"
"output" : "{rule}.{wildcards}.out"
"error" : "{rule}.{wildcards}.err"
"time" : "24:00:00"
It runs ok for some rules, but it raise this error for one of the rules.
Traceback (most recent call last):
File "home/conda3_64/lib/python3.5/site-packages/snakemake/__init__.py", line 537, in snakemake
report=report)
File "home/conda3_64/lib/python3.5/site-packages/snakemake/workflow.py", line 653, in execute
success = scheduler.schedule()
File "home/conda3_64/lib/python3.5/site-packages/snakemake/scheduler.py", line 286, in schedule
self.run(job)
File "home/conda3_64/lib/python3.5/site-packages/snakemake/scheduler.py", line 302, in run
error_callback=self._error)
File "home/conda3_64/lib/python3.5/site-packages/snakemake/executors.py", line 638, in run
jobscript = self.get_jobscript(job)
File "home/conda3_64/lib/python3.5/site-packages/snakemake/executors.py", line 496, in get_jobscript
cluster=self.cluster_wildcards(job))
File "home/conda3_64/lib/python3.5/site-packages/snakemake/executors.py", line 556, in cluster_wildcards
return Wildcards(fromdict=self.cluster_params(job))
File "home/conda3_64/lib/python3.5/site-packages/snakemake/executors.py", line 547, in cluster_params
cluster.update(self.cluster_config.get(job.name, dict()))
ValueError: need more than 1 value to unpack
This is how I run the snakemake;
snakemake -j 20 --cluster-config ./config.yaml --cluster "qsub -A {cluster.account} -l walltime={cluster.time} -q {cluster.queue} -l nodes=1:ppn={cluster.nCPUs},mem={cluster.memory}" -p
If I run it without a cluster snakemake only it will run normally.
similar error is here ValueError: need more than 1 value to unpack python but I could not relate.
I cannot test it right now but based on the error I suspect the issue seems to be with your JOBNAME placeholder not being explicitly specified in call to qsub.
Adding -N some_name to your qsub arguments should resolve it.
I am facing an issue while executing the following script snippet.
$ORACLEHOME/bin/sqlplus -s $DBUSER/$DBPASSWORD <<EOF
set pages 0 feedback off
SELECT * FROM ERR_STG_ROAMING_PARTNER;
EOF > Err_File.txt
The following error message appears.
./Roaming.sh: line 213: warning: here-document at line 206 delimited by end-of-file (wanted `EOF')
./Roaming.sh: line 214: syntax error: unexpected end of file
Any help will be appreciated.
Try this:
$ORACLEHOME/bin/sqlplus -s $DBUSER/$DBPASSWORD > Err_File.txt <<EOF
set pages 0 feedback off
SELECT * FROM ERR_STG_ROAMING_PARTNER;
EOF
That is, specify the output redirection before the input heredoc redirection. The shell expects EOF to be on its own on the line terminating the heredoc.
The error message you are getting is the shell complaining about finding end-of-file (of the script file) before finding EOF. The usage of EOF for the heredoc delimiter might lead to some confusion here!
I ran into an error when importing gzipped tab delimited files into bigquery
The output I got was:
root#a20c6fbdf9b5:/opt/batch/jobs# bq show -j bqjob_r5720e2f2267a5a5b_0000014d09571f27_1
Job infra-bedrock-861:bqjob_r5720e2f2267a5a5b_0000014d09571f27_1
Job Type State Start Time Duration Bytes Processed
---------- --------- ----------------- ---------- -----------------
load FAILURE 30 Apr 08:00:44 0:02:05
Errors encountered during job execution. Bad character (ASCII 0) encountered: field starts with: <H:|\ufc0f\ufffd(>
Failure details:
- File: 1 / Line:1 / Field:1: Bad character (ASCII 0) encountered:
field starts with: <\ufff>
- File: 1 / Line:3 / Field:1: Bad character (ASCII 0) encountered:
field starts with: <\u0475\ufffd=\ufffd\ufffd\u03d6>
- File: 1 / Line:4 / Field:1: Bad character (ASCII 0) encountered:
field starts with: <-\ufffd\ufffdY\u049a\ufffd>
- File: 1 / Line:6 / Field:1: Bad character (ASCII 0) encountered:
field starts with: <\u018e\ufffd\ufffd\ufffd\ufffd>
I tried manually downloading the files, unzipping and then uploading the files again. The uncompressed files could be imported into bigquery without any problems.
This looks like a bug in bigquery with zip files
Inspecting the job configuration, you include a non-gzip file as the first uri, ending in .../20150426/_SUCCESS. BigQuery uses the first file to determine whether compression is enabled.
Assuming this file is empty, you can remove it from your load requests to fix this. If there is data in this file, attach a ".gz" suffix or re-order this file so it is not first in the uri list.
I am having some issues with storing my pig output to a file. This is what I am using to store:
'STORE rel INTO 'simple'; '
If I Dump 'rel' I get:
>(car,0.5,(door,tire,jello,truck,random))
(toy,0.5,(jeep,bunny toy))
(door,0.5,(car,jello,random))
>(jeep,0.5,(toy,bunny toy))
What I get in the file is:
<Yulias-MacBook-Pro:~ yuliatolskaya$ /Users/yuliatolskaya/Documents/misc/pig_clustering/simple/part-r-00000 ; exit;
/Users/yuliatolskaya/Documents/misc/pig_clustering/simple/part-r-00000: line 1: syntax error near unexpected token `('
/Users/yuliatolskaya/Documents/misc/pig_clustering/simple/part-r-00000: line 1: `car 0.5 (door,tire,jello,truck,random)'
logout
[Process completed] >
I am really not sure what the problem is, as there are no errors in the logfiles...Please Help!