I'm new to Redis and would like to play with the RedisTimeSeries module. Following the instructions I load the module using:
redis-server --loadmodule /path/to/module/redistimeseries.so
Obviously adjusting the path to match mine. The server starts and shows the following output:
88537:M 02 Apr 2022 12:19:24.785 # Server initialized
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> RedisTimeSeries version 10609, git_sha=f36e5a703dc9a2487880087a34f6cb0e56d9a459
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> Redis version found by RedisTimeSeries : 6.2.6 - oss
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> loaded default CHUNK_SIZE_BYTES policy: 4096
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> loaded server DUPLICATE_POLICY: block
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> Setting default series ENCODING to: compressed
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> Detected redis oss
88537:M 02 Apr 2022 12:19:24.789 * <timeseries> Enabled diskless replication
88537:M 02 Apr 2022 12:19:24.789 * Module 'timeseries' loaded from lib/redistimeseries.so
88537:M 02 Apr 2022 12:19:24.790 * Loading RDB produced by version 6.2.6
88537:M 02 Apr 2022 12:19:24.790 * RDB age 4 seconds
88537:M 02 Apr 2022 12:19:24.790 * RDB memory usage when created 1.02 Mb
88537:M 02 Apr 2022 12:19:24.790 # Done loading RDB, keys loaded: 0, keys expired: 0.
88537:M 02 Apr 2022 12:19:24.790 * DB loaded from disk: 0.000 seconds
88537:M 02 Apr 2022 12:19:24.790 * Ready to accept connections
When I connect using the redis-cli I run the sample command like so:
TS.CREATE temperature RETENTION 60 LABELS sensor_id 2 area_id 32
However, I receive the following error in the CLI:
(error) ERR unknown command `TS.CREATE`, with args beginning with: `temperature`, `RETENTION`, `60`, `LABELS`, `sensor_id`, `2`, `area_id`, `32`,
No time series commands are available. Can anyone help? I'm stumped.
Related
I'm having a strange Redis issue that causes my redis db0 to get completely wiped periodically for no obvious reason.
I'm using redis with Bull queue for NestJS and inserting volatile keys with 7 days duration.
Everything is hosted on Google Cloud Kubernetes.
As you can see from the logs, queue fills up steadily and all of a sudden its wiped to 0.
1:M 14 Dec 2022 18:02:53.786 - Accepted 10.132.15.228:46851
1:M 14 Dec 2022 18:02:55.818 * DB saved on disk
1:M 14 Dec 2022 18:02:55.836 - DB 0: 6539 keys (1179 volatil
...skipping 1 line
1:M 14 Dec 2022 18:02:55.836 . 25 clients connected (0 repli
cas), 10264320 bytes in use
1:M 14 Dec 2022 18:02:56.338 * DB saved on disk
1:M 14 Dec 2022 18:02:57.975 * DB saved on disk
1:M 14 Dec 2022 18:02:58.444 * DB saved on disk
1:M 14 Dec 2022 18:02:58.687 * DB saved on disk
1:M 14 Dec 2022 18:03:00.863 - DB 0: 10 keys (3 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:00.863 . 25 clients connected (0 repli
cas), 3440672 bytes in use
1:M 14 Dec 2022 18:03:01.056 * DB saved on disk
This part is out of time, like 2 threads are writing to same output:
1:M 14 Dec 2022 18:03:10.900 - DB 0: 12 keys (5 volatile) in ...back 1 page
1:M 14 Dec 2022 18:02:40.773 - DB 0: 6542 keys (1174 volatile) in 8192 slots HT.
Any insights would be greatly appreciated.
Server info:
redis_version:7.0.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:68a9a9d6a665b45c
redis_mode:standalone
os:Linux 5.10.133+ x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:1
process_supervised:no
run_id:ca394137bd91b23622a8facfb97ae86ba2325a01
tcp_port:6379
server_time_usec:1671042411041619
uptime_in_seconds:1752
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:10098026
executable:/data/redis-server
config_file:/redis-config/redis.conf
io_threads_active:0
# Clients
connected_clients:25
cluster_connections:0
Redis config:
save 60 1000
loglevel debug
notify-keyspace-events Ex
dbfilename dump.rdb
dir /data
logfile /data/redis.log
Debug log:
...skipping 1 line
1:M 14 Dec 2022 17:59:55.106 . 24 clients connected (0 repli
cas), 8023104 bytes in use
1:M 14 Dec 2022 18:00:00.125 - DB 0: 5552 keys (1125 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:00.126 . 24 clients connected (0 repli
cas), 7959688 bytes in use
1:M 14 Dec 2022 18:00:05.146 - DB 0: 5552 keys (1128 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:05.146 . 24 clients connected (0 repli
cas), 8038576 bytes in use
1:M 14 Dec 2022 18:00:10.166 - DB 0: 5553 keys (1128 volatil
...skipping 1 line
1:M 14 Dec 2022 18:00:10.166 . 24 clients connected (0 repli
cas), 7977384 bytes in use
1:M 14 Dec 2022 18:00:14.281 * 1000 changes in 60 seconds. S
aving...
1:M 14 Dec 2022 18:00:14.282 * Background saving started by
pid 26
26:C 14 Dec 2022 18:00:14.284 - Fork CoW for RDB: current 0
MB, peak 0 MB, average 0 MB
26:C 14 Dec 2022 18:00:14.378 * DB saved on disk
26:C 14 Dec 2022 18:00:14.378 * Fork CoW for RDB: current 0
MB, peak 0 MB, average 0 MB
...skipping 1 line
1:M 14 Dec 2022 18:00:15.187 - DB 0: 6081 keys (1128 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:15.187 . 24 clients connected (0 repli
cas), 9276960 bytes in use
1:M 14 Dec 2022 18:00:17.755 - Accepted 10.8.1.1:36758
1:M 14 Dec 2022 18:00:20.208 - DB 0: 6570 keys (1131 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:20.208 . 24 clients connected (0 repli
cas), 10252224 bytes in use
1:M 14 Dec 2022 18:00:25.227 - DB 0: 6570 keys (1133 volatil
e) in 8192 slots HT.
...skipping 1 line
1:M 14 Dec 2022 18:00:30.247 - DB 0: 6568 keys (1134 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:30.247 . 24 clients connected (0 repli
cas), 10266176 bytes in use
1:M 14 Dec 2022 18:00:35.267 - DB 0: 6568 keys (1137 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:35.267 . 24 clients connected (0 repli
cas), 10345096 bytes in use
1:M 14 Dec 2022 18:00:40.285 - DB 0: 6569 keys (1137 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:40.285 . 24 clients connected (0 repli
...skipping 1 line
1:M 14 Dec 2022 18:00:45.304 - DB 0: 6563 keys (1140 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:45.304 . 24 clients connected (0 repli
cas), 10315776 bytes in use
1:M 14 Dec 2022 18:00:50.325 - DB 0: 6563 keys (1140 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:50.325 . 24 clients connected (0 repli
cas), 10254504 bytes in use
1:M 14 Dec 2022 18:00:55.342 - DB 0: 6562 keys (1143 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:00:55.342 . 24 clients connected (0 repli
...skipping 1 line
1:M 14 Dec 2022 18:01:00.360 - DB 0: 6562 keys (1143 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:00.360 . 24 clients connected (0 repli
cas), 10209512 bytes in use
1:M 14 Dec 2022 18:01:05.381 - DB 0: 6562 keys (1146 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:05.381 . 24 clients connected (0 repli
cas), 10309016 bytes in use
1:M 14 Dec 2022 18:01:10.399 - DB 0: 6563 keys (1146 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:10.399 . 24 clients connected (0 repli
...skipping 1 line
1:M 14 Dec 2022 18:01:15.019 * 1000 changes in 60 seconds. S
aving...
1:M 14 Dec 2022 18:01:15.020 * Background saving started by
pid 27
27:C 14 Dec 2022 18:01:15.022 - Fork CoW for RDB: current 0
MB, peak 0 MB, average 0 MB
27:C 14 Dec 2022 18:01:15.085 * DB saved on disk
27:C 14 Dec 2022 18:01:15.086 * Fork CoW for RDB: current 0
MB, peak 0 MB, average 0 MB
1:M 14 Dec 2022 18:01:15.120 * Background saving terminated
with success
...skipping 1 line
1:M 14 Dec 2022 18:01:15.421 . 24 clients connected (0 repli
cas), 10212776 bytes in use
1:M 14 Dec 2022 18:01:20.441 - DB 0: 6553 keys (1149 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:20.441 . 24 clients connected (0 repli
cas), 10192304 bytes in use
1:M 14 Dec 2022 18:01:25.460 - DB 0: 6553 keys (1152 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:25.460 . 24 clients connected (0 repli
cas), 10189448 bytes in use
1:M 14 Dec 2022 18:01:30.479 - DB 0: 6553 keys (1152 volatil
...skipping 1 line
1:M 14 Dec 2022 18:01:30.479 . 24 clients connected (0 repli
cas), 10189448 bytes in use
1:M 14 Dec 2022 18:01:35.503 - DB 0: 6553 keys (1155 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:35.503 . 24 clients connected (0 repli
cas), 10186592 bytes in use
1:M 14 Dec 2022 18:01:40.523 - DB 0: 6553 keys (1155 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:40.523 . 24 clients connected (0 repli
cas), 10207064 bytes in use
1:M 14 Dec 2022 18:01:45.543 - DB 0: 6551 keys (1157 volatil
...skipping 1 line
1:M 14 Dec 2022 18:01:45.543 . 24 clients connected (0 repli
cas), 10202824 bytes in use
1:M 14 Dec 2022 18:01:50.561 - DB 0: 6551 keys (1157 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:50.561 . 24 clients connected (0 repli
cas), 10202824 bytes in use
1:M 14 Dec 2022 18:01:55.582 - DB 0: 6546 keys (1160 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:01:55.582 . 24 clients connected (0 repli
cas), 10173504 bytes in use
1:M 14 Dec 2022 18:02:00.603 - DB 0: 6546 keys (1161 volatil
...skipping 1 line
1:M 14 Dec 2022 18:02:00.603 . 24 clients connected (0 repli
cas), 10192880 bytes in use
1:M 14 Dec 2022 18:02:05.622 - DB 0: 6545 keys (1163 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:05.622 . 24 clients connected (0 repli
cas), 10169456 bytes in use
1:M 14 Dec 2022 18:02:10.645 - DB 0: 6546 keys (1167 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:10.645 . 24 clients connected (0 repli
cas), 10290056 bytes in use
1:M 14 Dec 2022 18:02:15.667 - DB 0: 6546 keys (1167 volatil
...skipping 1 line
1:M 14 Dec 2022 18:02:15.667 . 24 clients connected (0 repli
cas), 10208312 bytes in use
1:M 14 Dec 2022 18:02:20.690 - DB 0: 6546 keys (1167 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:20.690 . 24 clients connected (0 repli
cas), 10208312 bytes in use
1:M 14 Dec 2022 18:02:25.713 - DB 0: 6546 keys (1170 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:25.713 . 24 clients connected (0 repli
cas), 10184984 bytes in use
1:M 14 Dec 2022 18:02:30.731 - DB 0: 6545 keys (1171 volatil
...skipping 1 line
1:M 14 Dec 2022 18:02:30.731 . 24 clients connected (0 repli
cas), 10223352 bytes in use
1:M 14 Dec 2022 18:02:35.750 - DB 0: 6541 keys (1173 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:35.750 . 24 clients connected (0 repli
cas), 10154896 bytes in use
1:M 14 Dec 2022 18:02:40.773 - DB 0: 6542 keys (1174 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:40.773 . 24 clients connected (0 repli
cas), 10194824 bytes in use
1:M 14 Dec 2022 18:02:44.425 - Accepted 10.132.15.230:61177
...skipping 1 line
1:M 14 Dec 2022 18:02:45.793 - DB 0: 6541 keys (1176 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:45.793 . 24 clients connected (0 repli
cas), 10172512 bytes in use
1:M 14 Dec 2022 18:02:50.817 - DB 0: 6540 keys (1179 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:50.817 . 24 clients connected (0 repli
cas), 10250272 bytes in use
1:M 14 Dec 2022 18:02:53.786 - Accepted 10.132.15.228:46851
1:M 14 Dec 2022 18:02:55.818 * DB saved on disk
1:M 14 Dec 2022 18:02:55.836 - DB 0: 6539 keys (1179 volatil
...skipping 1 line
1:M 14 Dec 2022 18:02:55.836 . 25 clients connected (0 repli
cas), 10264320 bytes in use
1:M 14 Dec 2022 18:02:56.338 * DB saved on disk
1:M 14 Dec 2022 18:02:57.975 * DB saved on disk
1:M 14 Dec 2022 18:02:58.444 * DB saved on disk
1:M 14 Dec 2022 18:02:58.687 * DB saved on disk
1:M 14 Dec 2022 18:03:00.863 - DB 0: 10 keys (3 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:00.863 . 25 clients connected (0 repli
cas), 3440672 bytes in use
1:M 14 Dec 2022 18:03:01.056 * DB saved on disk
...skipping 1 line
1:M 14 Dec 2022 18:03:01.989 - Client closed connection id=2
9 addr=10.132.15.228:46851 laddr=10.8.1.41:6379 fd=32 name=
age=8 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=
0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=97 obl
=0 oll=0 omem=0 tot-mem=22272 events=r cmd=save user=default
redir=-1 resp=2
1:M 14 Dec 2022 18:03:05.882 - DB 0: 10 keys (3 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:05.882 . 24 clients connected (0 repli
cas), 3418912 bytes in use
1:M 14 Dec 2022 18:03:10.900 - DB 0: 12 keys (5 volatile) in
...back 1 page
1:M 14 Dec 2022 18:02:40.773 - DB 0: 6542 keys (1174 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:40.773 . 24 clients connected (0 repli
cas), 10194824 bytes in use
1:M 14 Dec 2022 18:02:44.425 - Accepted 10.132.15.230:61177
1:M 14 Dec 2022 18:02:44.639 - Reading from client: Connecti
on reset by peer
1:M 14 Dec 2022 18:02:45.793 - DB 0: 6541 keys (1176 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:45.793 . 24 clients connected (0 repli
cas), 10172512 bytes in use
...skipping 1 line
1:M 14 Dec 2022 18:02:50.817 . 24 clients connected (0 repli
cas), 10250272 bytes in use
1:M 14 Dec 2022 18:02:53.786 - Accepted 10.132.15.228:46851
1:M 14 Dec 2022 18:02:55.818 * DB saved on disk
1:M 14 Dec 2022 18:02:55.836 - DB 0: 6539 keys (1179 volatil
e) in 8192 slots HT.
1:M 14 Dec 2022 18:02:55.836 . 25 clients connected (0 repli
cas), 10264320 bytes in use
1:M 14 Dec 2022 18:02:56.338 * DB saved on disk
1:M 14 Dec 2022 18:02:57.975 * DB saved on disk
1:M 14 Dec 2022 18:02:58.444 * DB saved on disk
...skipping 1 line
1:M 14 Dec 2022 18:03:00.863 - DB 0: 10 keys (3 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:00.863 . 25 clients connected (0 repli
cas), 3440672 bytes in use
1:M 14 Dec 2022 18:03:01.056 * DB saved on disk
1:M 14 Dec 2022 18:03:01.756 * DB saved on disk
1:M 14 Dec 2022 18:03:01.989 - Client closed connection id=2
9 addr=10.132.15.228:46851 laddr=10.8.1.41:6379 fd=32 name=
age=8 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=
0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=97 obl
=0 oll=0 omem=0 tot-mem=22272 events=r cmd=save user=default
...skipping 1 line
1:M 14 Dec 2022 18:03:05.882 - DB 0: 10 keys (3 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:05.882 . 24 clients connected (0 repli
cas), 3418912 bytes in use
1:M 14 Dec 2022 18:03:10.900 - DB 0: 12 keys (5 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:10.900 . 24 clients connected (0 repli
cas), 3378320 bytes in use
1:M 14 Dec 2022 18:03:15.917 - DB 0: 12 keys (5 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:15.917 . 24 clients connected (0 repli
cas), 3378336 bytes in use
1:M 14 Dec 2022 18:03:20.936 - DB 0: 12 keys (5 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:20.936 . 24 clients connected (0 repli
cas), 3419296 bytes in use
1:M 14 Dec 2022 18:03:25.954 - DB 0: 12 keys (5 volatile) in
16 slots HT.
1:M 14 Dec 2022 18:03:25.954 . 24 clients connected (0 repli
cas), 3378304 bytes in use
I want the output like this for whole year:
Jan 01 - Jan 14 - 2021
Jan 15 - Jan 28 - 2021
Jan 29 - Feb 11 - 2021
Feb 12 - Feb 25 - 2021
Feb 26 - Mar 11 - 2021
Mar 12 - Mar 25 - 2021
Mar 26 - Apr 08 - 2021
Apr 09 - Apr 22 - 2021
Apr 22 - May 06 - 2021
May 07 - May 20 - 2021
May 21 - Jun 03 - 2021
Jun 04 - Jun 17 - 2021
Jun 18 - Jul 01 - 2021
Jul 02 - Jul 15 - 2021
Jul 16 - Jul 29 - 2021
Jul 30 - Aug 12 - 2021
Aug 13 - Aug 26 - 2021
Aug 27 - Sep 09 - 2021
Sep 10 - Sep 23 - 2021
Sep 24 - Oct 07 - 2021
Oct 08 - Oct 21 - 2021
Oct 22 - Nov 04 - 2021
Nov 05 - Nov 18 - 2021
Dec 19 - Dec 02 - 2021
Dec 03 - Dec 16 - 2021
Dec 17 - Dec 30 - 2021
One option uses generate_series():
select d.dt dt_start, d.dt + '13 days'::interval dt_end
from generate_series(
date_trunc('year', current_date),
date_trunc('year', current_date) + '1 year'::interval - '13 days'::interval,
'14 days'::interval
) d(dt)
order by dt_start
SELECT to_char(d, 'Mon DD - ')
|| to_char(d + INTERVAL '13 days', 'Mon DD - YYYY')
FROM generate_series(
TIMESTAMP '2021-01-01',
TIMESTAMP '2021-12-17',
INTERVAL '14 days'
) AS d;
?column?
------------------------
Jan 01 - Jan 14 - 2021
Jan 15 - Jan 28 - 2021
Jan 29 - Feb 11 - 2021
Feb 12 - Feb 25 - 2021
Feb 26 - Mar 11 - 2021
Mar 12 - Mar 25 - 2021
Mar 26 - Apr 08 - 2021
Apr 09 - Apr 22 - 2021
Apr 23 - May 06 - 2021
May 07 - May 20 - 2021
May 21 - Jun 03 - 2021
Jun 04 - Jun 17 - 2021
Jun 18 - Jul 01 - 2021
Jul 02 - Jul 15 - 2021
Jul 16 - Jul 29 - 2021
Jul 30 - Aug 12 - 2021
Aug 13 - Aug 26 - 2021
Aug 27 - Sep 09 - 2021
Sep 10 - Sep 23 - 2021
Sep 24 - Oct 07 - 2021
Oct 08 - Oct 21 - 2021
Oct 22 - Nov 04 - 2021
Nov 05 - Nov 18 - 2021
Nov 19 - Dec 02 - 2021
Dec 03 - Dec 16 - 2021
Dec 17 - Dec 30 - 2021
(26 rows)
I am really learning a lot here. Thanks to all the support. I have another challenge however.
I want to calculate a weighted Index for a group of rows 'Div' within a particular month as shown below:
Wght tMonth tYear Div Indices
37.5 01 2015 01 157.27
2.7 01 2015 01 127.36
58.7 01 2015 01 142.48
DivIndex 146.11
34.9 01 2015 02 133.33
6.7 01 2015 02 136.49
52.4 01 2015 02 131.34
DivIndex 124.43
43.9 02 2015 01 157.18
44.8 02 2015 01 127.42
DivIndex 126.09
58.7 02 2015 03 145.67
7.5 02 2015 03 134.70
6.7 02 2015 03 137.24
DivIndex 104.72
54.0 03 2015 05 160.61
DivIndex 86.73
48.1 03 2015 04 127.49
58.7 03 2015 04 148.62
DivIndex 148.58
I used Excel to compute the 'DivIndex' and that is what I want to come up with in Sql Server 2008R2.
Thanks in advance for any help.
You can do the calculation as:
select tyear, tmonth,
sum(weight * dev_indices) / sum(weight)
from t
group by tyear, tmonth
order by tyear, tmonth;
I'm new to supporting WAS (Websphere Application Server), currently I'm having issue with my WAS, my WAS was installed under AIX under 2 servers/nodes.
While investigating it, I found in our application log that there are some activity which is "Performing Cache Maintenance":
===================================
2017-01-14 01:31:52,619: [Cache Maintenance] com.ibm.srm.util.db.ServerCache refreshed 2017-01-14 01:31:53,314: [Cache Maintenance] Memory: available=[6884mb] used=[9500mb] %used avail=[58%] max=[16384mb] %used max=[58%] total=[16384mb] free=[6884mb] used by doMaintenance=[-251,201,3 92bytes] Time=[22,818ms] 2017-01-14 01:51:53,325: -------- Performing Cache Maintenance -------- 2017-01-14 01:51:53,325: null : QN=319 Select * from perform.cache_timestamps where row_class_name not like '%Cache' and row_class_name not like '%(SRM 6.0)' 2017-01-14 01:51:53,333: Returning 19 data records, QN=319,2 columns, Time: 8ms conn/query time: 5ms
2017-01-14 01:51:53,333: [Cache Maintenance] Memory: available=[5492mb] used=[10892mb] %used avail=[66%] max=[16384mb] %used max=[66%] total=[16384mb] free=[5492mb] used by doMaintenance=[532kb] Time=[8ms]
===================================
After this activity triggered, I found that mpmstats value for 'bsy' are keep increasing until reach MaxClient maximum value which is '4000':
===================================
[Sat Jan 14 01:38:58 2017] [notice] mpmstats: rdy 166 bsy 234 rd 0 wr 234 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:38:58 2017] [notice] mpmstats: bsy: 234 in mod_was_ap22_http.c [Sat Jan 14 01:48:58 2017] [notice] mpmstats: rdy 195 bsy 505 rd 0 wr 505 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:48:58 2017] [notice] mpmstats: bsy: 505 in mod_was_ap22_http.c [Sat Jan 14 01:58:58 2017] [notice] mpmstats: rdy 180 bsy 720 rd 0 wr 720 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:58:58 2017] [notice] mpmstats: bsy: 720 in mod_was_ap22_http.c [Sat Jan 14 02:08:59 2017] [notice] mpmstats: rdy 105 bsy 895 rd 1 wr 894 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 02:08:59 2017] [notice] mpmstats: bsy: 894 in mod_was_ap22_http.c [Sat Jan 14 02:18:59 2017] [notice] mpmstats: rdy 112 bsy 1088 rd 1 wr 1087 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 02:18:59 2017] [notice] mpmstats: bsy: 1087 in mod_was_ap22_http.c
[Sat Jan 14 02:28:59 2017] [notice] mpmstats: rdy 158 bsy 1242 rd 1 wr 1241 ka 0 log 0 dns 0 cls 0
----
[Sat Jan 14 04:55:34 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 04:55:34 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c [Sat Jan 14 04:57:04 2017] [notice] mpmstats: reached MaxClients (4000/4000) [Sat Jan 14 04:57:04 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 04:57:04 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c [Sat Jan 14 04:58:34 2017] [notice] mpmstats: reached MaxClients (4000/4000) [Sat Jan 14 04:58:34 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0
[Sat Jan 14 04:58:34 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c
===================================
It seem WAS are not processing the client request until it reached the Maximum value.
The questions are:
Is there any log that I can check about why WAS are not processing the Client request until it reached to the max value?
Does the "Cache Maintenance" activity hold WAS from processing the Client request? Because as mentioned from our developer this activity should not lead to this issue.
What is the procedure that I can take to identify/resolve this issue?
Appreciate if can help me for this thing as this issue already occurred for a long time but still not resolve.
OS: SLes 11 sp4
syslog-ng: syslog-ng-2.0.9-27.34.39.2
Hi,
syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config)
I have the following in my syslog-ng configuration file:
source ESRS {
file("/opt/esrsve/gateway/xGate.log");
};
destination esrsfile{ file("/var/log/max.log" );};
log { source(ESRS);
destination(esrsfile);
};
The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines.
ex: Source File :
[ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline
And the destination file looiks like this:
...
...
Aug 29 14:00:02 hostname C
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname r
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname t
Aug 29 14:00:02 hostname
Aug 29 14:00:02 hostname V
Aug 29 14:00:02 hostname a
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname u
Aug 29 14:00:02 hostname e
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname :
Aug 29 14:00:02 hostname O
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname f
Aug 29 14:00:02 hostname l
Aug 29 14:00:02 hostname i
Aug 29 14:00:02 hostname n
Aug 29 14:00:02 hostname e
...
...
What is wrong here please?
When you look closely, the lines are not truncated, but one log entry spreads over multiple lines, character by character. It happens usually, when the log writer flushes after writing every single character.
Make sure the writer buffers lines and sends them as one single message.