I get a syntax error ( #user_script:15: ERR syntax error) when trying to execute a ZADD with optional parameter LT from Lua in Redis 5.0.12:
redis.call('zadd', ASET, 'lt', 100, "A")
but the same exact code works fine on Redis 6.2.1
Is there some alternative way to pass optional parameters in older Redis versions?
It doesn't work in Redis 5.0.12 because you're using a feature (the LT option) that wasn't added until Redis 6.2. From the ZADDdocumentation:
>= 6.2: Added the GT and LT options
Related
Querying DB2 from python using ODBC, I am seeing NULL values converted to 0 (on Linux, seemingly corrupt but close to 0 on Mac M1 -- even more worryingly).
This is using the db2 docker image started like this:
docker run -itd --name db2 --privileged=true -p 50000:50000 -e LICENSE=accept -e DB2INST1_PASSWORD=xxxxx -e DBNAME=testdb -v <db storage dir>:/database ibmcom/db2
Code as follows recreates the issue:
import pyodbc
cs = "Driver={ODBC Driver v11.5.7 for DB2};Database=xxxxx;Hostname=xxxx;Port=50000;Protocol=TCPIP;Uid=xxxx;Pwd=xxxx;"
cnxn = pyodbc.connect(cs)
crsr = cnxn.cursor()
crsr.execute("SELECT CAST(NULL AS INT), CAST(NULL AS REAL) FROM SYSIBM.SYSDUMMY1");
print(crsr.messages)
print(crsr.fetchall())
Outputs:
❯ python float-test.py
[]
[(2, 4.2439915814e-314)]
Is it expected that I can't retrieve NULL values as plain data types? It seems to be allowed in PostgreSQL. I know I can cast around this but would rather not, obviously.
Extra Info
It does seem that the ODBC driver version 11.5.7 from Fix Central suffers this issue whilst the 11.5.6 version from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli does not.
As mentioned in comments, it appears pyodbc is impacted and both plain ibm_db and DBI are not impacted (both return None, None). So at least there is a workaround.
The reason for the behaviour deifference is that pyodbc is using SQLGetData() while the other two use the SQLBindCol() methods of extracting the result set data.
IBM's clidriver on Linux x64, SQLGetData() sets the (SQLLEN *) StrLen_Or_IndPtr parameter to SQL_NULL_DATA when the value of the column in result-set is NULL. But the problem is that IBMs clidrver sets StrLen_or_IndPtr to SQL_NULL_DATA (as int, 4 bytes), when pyodbc code expects it to SQL_NULL_DATA (as SQLLEN, 8bytes on Linux x64) as SQLLEN is the documented datatype for the StrLen_or_IndPtr argument.
Therefore the comparison in pyodbc getdata.cpp GetDataDouble() :
if ( cbFetched == SQL_NULL_DATA )
Py_RETURN_NONE;
will be false, causing the code to return an unitialised variable instead of Py_None.
I do not know if the maintainers of pyodbc run their tests against a Db2-LUW product, but it looks like other parts of the code could suffer the same problem and other issues may lurk. Consider asking on github what is the support policy for Db2-LUW in pyodbc.
If you have a support contract, IBM should also be asked to comment on their reason for not respecting the datatype of StrLen_Or_IndPtr when writing SQL_NULL_DATA to this parameter on Linux x64.
I like to know how to specify unix-timestamp in redis.
for command like SET name xyz EXAT 1617875638 //is wrong format cli says
On my terminal timestamp is something like this 1617875638. this is exception causing. wrong format.
EXAT option is added in Redis 6.2. If your Redis is older than 6.2, it cannot support it. Instead, you can use Lua script to implement it:
eval 'redis.call("set", KEYS[1], ARGV[1]); redis.call("expireat", KEYS[1], ARGV[2])' 1 name xyz 1617875638
I have seen this pass results to another command in redis
and using via command line this command works well :
src/redis-cli keys '*' | xargs src/redis-cli mget
However how can we achieve the same effect via Lettuce (i started trying out 4.0.2.Final)
Also a solution to this is particularly important in the following scenario :
Say we are using geolocation capabilities, and we add a set of locations of "my-location-category"
using GEOADD
GEOADD "category-1" 8.6638775 49.5282537 "location-id:1" 8.3796281 48.9978127 "location-id:2" 8.665351 49.553302 "location-id:3"
Next, say we do a GeoRadius to get locations within 10 km radius of 8.6582361 49.5285495 for "category-1"
Now when we get "location-id:1" & "location-id:3"
Given that I already set values for above keys "location-id:1" & "location-id:3"
I want to pipe commands to do the GEORADIUS as well as do mget on all the matching results.
Does Redis provide feature to do that?
and / or how can we achieve this via the Lettuce client library without first manually iterating through results of GEORADIUS and then do manual mget.
That would be more efficient performance for the program that uses it.
Does anyone know how we can do this ?
Update
This is the piped command for the scenario I discussed above :
src/redis-cli GEORADIUS "category-1" 8.6582361 49.5285495 10 km | xargs src/redis-cli mget
Now we need to know how to do this via Lettuce
IMPORTANT: never use KEYS, always use SCAN instead if you must.
This isn't really a question about Lettuce nor Java so I can actually answer it :)
What you're trying to do is use the results from a read operation (GEORADIUS) as input (key names) for another read operation (MGET). This type of flow can't be pipelined, well, just because of that - pipelining means that you don't need the answers for operations right away but in you case you do.
However.
Since you're reading String keys with MGET, you might as well just denormalize everything (remember, we're NoSQL) and store the contents of these keys in the Sorted Set's members, e.g.:
GEOADD "category-1" 8.6638775 49.5282537 "location-id:1:moredata:evenmoredata:{maybe a JSON document here}:orperhapsmsgpack"
This will allow you to get the locations and their "data" with one GEORADIUS call. Of course, any updates to location:1's data will need to be done across all categories.
A note about Lua scripts: while a Lua script could definitely save on the back and forth in this case, any such script will be against best practices/not cluster safe.
After digging around and studying Lua script, my conclusion is that removing round-trips in such a way can only be done via Lua scripts as suggested by Itamar Haber.
I ended up creating a lua script file (myscript.lua) as below
local locationKeys = redis.call('GEORADIUS', 'category-1', '8.6582361', '49.5285495', '10', 'km' )
if unpack(locationKeys) == nil then
return nil
else
return redis.call('MGET', unpack(locationKeys))
end
** of course we should be sending in parameters to this... this is just a poc :)
now you can execute it via command
src/redis-cli EVAL "$(cat myscript.lua)" 0
Then to reduce the network-overhead of sending across the entire script to Redis for execution, we have the option of registering the script with Redis.
Redis will give us a sha1 digested code for future references for that script, which can be used for next calls to that script.
This can be done as below :
src/redis-cli SCRIPT LOAD "$(cat myscript.lua)"
this should give back a sha1 code something like this : 49730aa2ed3034ee48f818e486tpbdf1b500b19e
next calls can be done using this code
eg
src/redis-cli evalsha 49730aa2ed3034ee48f818e486b2bdf1b500b19e 0
The sad part however here is that the sha1 digest is remembered only so long as the instance of redis is running. If it is restarted, that the sha1 digest is lost. Then you do the SCRIPT LOAD once again. And if nothing changes in the script, then the sha1-digest code will be the same.
Ideally while using through client api, we should first attempt evalsha, if that returns a "No matching script" error, then as a fallback do script load, and procure the sha1 code once again, and create an internal map of that and use that sha1 code for further calls.
This can well be done via Lettuce. I could find the methods for those. Hope this gives a good insight into solution for the problem.
I am having an issue when I try to add "NX" to the zadd command on predis. The redis docs say that ZADD should support NX, but no matter how I set up the predis command, I can not get it working. Does anyone have any experience with this issue?
Here are the commands I have tried:
$redis->zadd($key, "NX", 1, $id);
$redis->executeRaw([ 'ZADD', $key, "NX", 1, $id ]);
Here is the error that keeps getting thrown:
ERROR: exception 'Predis\ServerException' with message 'ERR syntax error'
Looking at the redis-cli monitor, I see the command execute when using the ZADD command, but the executeRaw command does nothing.
Any help would be greatly appreciated!
ZADD's NX switch was added only to the recent version of Redis, see here: https://groups.google.com/forum/#!topic/redis-db/4Y6OqK8gEyk
In all likelihood you aren't running the recent version - use INFO to find out your server's version.
I have a string type key value store in redis having keys like this--
/url-pattern/url-slug-1
/url-pattern/url-slug-2
/url-pattern/url-slug-3
/url-pattern/url-slug-4 ...
I can retrieve all the keys of /url-pattern/ using a wild card query like this --
keys /url-pattern/*
I would like to retrieve the values of all keys corresponding to this wildcard /url-pattern/*
I tried this
mget /url-pattern/*
1) (nil)
but it doesnt returned the array as expected.
How can I retrieve the values of all keys corresponding to /url-pattern/*
I also want to do a sum on the values, but I think there is no such thing called SUM() in redis
MGET accepts multiple arguments where each a key name. It does not do key name patterns.
What you could do is first fetch all the relevant key names (do not use KEYS, use SCAN instead) and then fetch their values with an MGET.
Here is an updated answer for 2015.
If you can upgrade Redis above 2.8, the SCAN command with MATCH will work for this. Before that version, not so much, and do NOT use the KEYS command except in a development environment.
http://redis.io/commands/scan
Example on command line:
$ redis-cli
127.0.0.1:6379> scan match V3.0:*
(error) ERR invalid cursor
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
127.0.0.1:6379> scan 0 match V1.0:*
1) "0"
2) (empty list or set)
127.0.0.1:6379> scan 0 match V3.0:*
1) "0"
2) 1) "V3.0:UNITTEST55660BC7E0C5B"
2) "V3.0:shop.domain.com:route"
3) "V3.0:UNITTEST55660BC4A2548"
Example in PHP:
// Initialize our iterator to NULL
$iterate = null;
// retry when we get no keys back
$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
while ($arr_keys = $redis->scan($iterate, 'match:*')) {
foreach ($arr_keys as $str_key) {
echo "Here is a key: $str_key\n";
}
echo "No more keys to scan!\n";
}
Note, php code is not tested and from the core documentation for example here. Production code would need to be modified depending on the keys needed to look up.
For those on Ubuntu here are the instructions to upgrade php5-redis:
Download the 2.2.7 package here: http://pecl.php.net/package/redis
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.4
Follow instructions in README to phpize, configure, make install
Create a symlink for command line cli package: cd /etc/php5/cli/conf.d && sudo ln -s ../../mods-available/redis.ini 20-redis.ini
$ php -i | grep Redis
Redis Support => enabled
Redis Version => 2.2.7
There is NO command available in REDIS which can return values from wildcard keys.
If you see the documentation for KEYS command: http://redis.io/commands/keys, it says
Consider KEYS as a command that should only be used in production
environments with extreme care. It may ruin performance when it is
executed against large databases. This command is intended for
debugging and special operations. Don't use KEYS in your regular
application code.
I don't know your business use case, but looks like you may have to use different data structure for this requirement. You can use list or set to store similar url patterns.