LUA script for modification object in Redis - redis

I Need a short advise please about LUA script for modification object in Redis.
I Have Redis List with entities like that:
{
"#class": "com.myproject.model.Book",
"bookId": "someId",
"author": "someAuthor"
}
now, I need to change my entity to allow multiple authors for certain book, and create migration script for this:
{
"#class": "com.myproject.model.Book",
"bookId": "someId",
"authors": [
"java.util.ArrayList",
[
"someAuthor"
]
]
}
What I Think I need to do with LUA:
local book
local cacheName --cache name in my case
local authorId;
local size = redis.call('LLEN', cacheName)
if size == 0
then
return -1
end
while size > 0
do
book = redis.call('LPOP', cacheName)
-- modify entity here
affectedEntitiesCount = affectedEntitiesCount + 1
redis.call('RPUSH', cacheName, book)
size = size - 1
end
return affectedEntitiesCount
But I have no idea how to modify book according requirements.
Can someone take a look and suggest?

solved:
local book
local cacheName
local authorPattern= '"author":"[^"]*"'
local authorId
local replacementAuthors = '"authors":["java.util.ArrayList",["%s"]]'
local size = redis.call('LLEN', cacheName)
if size == 0
then
return -1
end
while size > 0
do
book = redis.call('LPOP', cacheName)
authorId = string.match(string.match(book, authorPattern), [["authorId":"([^"]+)]])
replacedAuthors = string.format(replacedAuthors , authorId)
book = string.gsub(book, authorPattern, replacedAuthors)
affectedEntitiesCount = affectedEntitiesCount + 1
redis.call('RPUSH', cacheName, book)
size = size - 1
end
return affectedEntitiesCount

Since it appears that you're storing JSON-encoded values in your list, you can use the cjson library that's embedded in Redis' Lua engine.
For example:
...
book = cjson.decode(redis.call('LPOP', cacheName))
book['bookId'] = book['bookId']..'foo' -- concat 'foo' to bookId
...
redis.call('RPUSH', cacheName, cjson.encode(book))
Note: also make sure that you use the KEYS input array to parameterize the script's input key names (e.g. local cacheName = KEYS[1])

Related

How to call a Redis Function with StackExchange.Redis

I wrote and register this Redis Function:
local function stincr(KEYS, ARGS)
local value = redis.pcall('INCR', KEYS[1])
if value == 1 or value % 100 == 0 then
redis.call('ZADD', KEYS[2],'GT', tostring(value), KEYS[1])
end
return value;
end
redis.register_function('stincr', stincr)
Redis Functions are introduced in Redis 7. How can I call it with StackExchange.Redis?
As of right now StackExchange.Redis doesn't have any higher-level API wrapping up the functions API, however, you can just use the ad-hoc command API pretty easily. I modified your script to add the shebang in the beginning called for by redis and added it to script.lua:
#!lua name=mylib
local function stincr(KEYS, ARGS)
local value = redis.pcall('INCR', KEYS[1])
if value == 1 or value % 100 == 0 then
redis.call('ZADD', KEYS[2],'GT', tostring(value), KEYS[1])
end
return value;
end
redis.register_function('stincr', stincr)
Then loading/calling the function was pretty straight forward:
var script = File.ReadAllText("script.lua");
var muxer = ConnectionMultiplexer.Connect("localhost");
var db = muxer.GetDatabase();
db.Execute("FUNCTION", "LOAD", script);
var res = db.Execute("FCALL", "stincr", 2, "myNum", "myzset");

How to exclude certain images from autosave in Gatan Digital Micrograph (GMS) in DM-script

I am trying to mimic the autosave function in GMS v3 so that I can use in version 1 and 2. I would like to first acknowledge that the main bulk of the script originates from Dr Bernhard Schaffer's "How to script... Digital Micrograph Scripting Handbook". I have modified it a bit, so that any new image recorded by the camera can be autosave into the file. However, I met some problems because if I decide to click on live-view image and move the image around, or using live-fft, the live view image or the FFT image will be saved as well. One of the ideas I have is to use the taggroup information such as the "Acquisition:Parameters:Parameter Set Name" because for live view or live-FFT, this would be either in search or focus mode. Another idea is to use the document ID e.g iDocID = idoc.ImageDocumentGETID() to locate the ID of the live image. However, i am clueless then how to use this information to exclude them from autosaving. Can anyone point to me how i can proceed with this script?
Below is the script
Class PeriodicAutoSave : Object
{
Number output
PeriodicAutoSave(Object self) Result("\n Object ID"+self.ScriptObjectGetID()+" created.")
~PeriodicAutoSave(Object self) Result("\n Object ID"+self.ScriptObjectGetID()+" destroyed")
Void Init2(Object self, Number op)
output=op
Void AutoSave_SaveAll(Object self)
{
String path, name, targettype, targettype1, area, mag, mode, search, result1
ImageDocument idoc
Number nr_idoc, count, index_i, index, iDocID, iDocID_search
path = "c:\\path\\"
name = "test"
targettype=="Gatan Format (*.dm4)"
targettype1 = "dm4"
If (output) Result("\n AutoSave...")
nr_idoc = CountImageDocuments()
For (count = 1; count<nr_idoc; count++)
{
idoc = GetImageDocument(count) //imagedocument
index = 1 // user decide the index to start with
index_i= nr_idoc - index
If (idoc.ImageDocumentIsDirty())
{
idoc = getfrontimagedocument()
iDocID = idoc.ImageDocumentGetID()
TagGroup tg = ImageGetTagGroup(idoc.ImageDocumentGetImage(0)) // cannot declare an 'img' for this line as it will prompt an error?
tg.TagGroupGetTagAsString("Microscope Info:Formatted Indicated Mag", mag)
Try{
{
idoc.ImageDocumentSavetoFile( "Gatan Format", path+index_i+"-"+name+"-"+mag+".dm4")
idoc.ImageDocumentSetName(index_i + "-"+name+"-"+mag+".dm4")
idoc.ImageDocumentClean()
}
If (Output) Result("\n\t saving: "+idoc.ImageDocumentGetCurrentFile())
}
Catch{
Result("\n image cannot be saved at the moment:" + GetExceptionString())
Break
}
Result("\ Continue autosave...")
}
}
}
}
Void LaunchAutoSave()
{
Object obj = Alloc(PeriodicAutoSave)
obj.Init2(2)
Number task_id = obj.AddMainThreadPeriodicTask("AutoSave_SaveALL",6)
//Sleep(10)
while(!shiftdown()) 1==2
RemoveMainThreadTask(task_id)
}
LaunchAutoSave()
thank you very much for your pointers! I have tried and it works very well with my script. as the 'TagGroupDoesTagExist' only refers to the taggroup, I modified further to include the tags I want to filter e.g "Search" or "Focus" and it seems to work well. The script that I modified to your existing ones is as below :
If (idoc.ImageDocumentIsDirty())
{
//now find out what is a filter condition and skip if it is true
skip = 0
TagGroup tg = idoc.ImageDocumentGetImage(0).ImageGetTagGroup()
tg.TagGroupGetTagAsString("Microscope Info:Formatted Indicated Mag", mag)
tg.TagGroupGetTagAsString("Acquisition:Parameters:Parameter Set Name", mode)
skip = tg.TagGroupDoesTagExist("Acquisition:Parameters:Parameter Set Name")
if(skip && (mode == "Search" || mode== "Focus")) continue
Your idea of filtering is a good one, but there is something strange with your for loop.
in
nr_idoc = CountImageDocuments()
For (count = 1; count<nr_idoc; count++)
{
idoc = GetImageDocument(count) //imagedocument
you iterate over all currently open imageDocuments (except the first one!?) and get them one by one, but then in
If (idoc.ImageDocumentIsDirty())
{
idoc = getfrontimagedocument()
you actually get the front-most (selected) document instead each time. Why are you doing this?
Why not go with:
number nr_idoc = CountImageDocuments()
for (number count = 0; count<nr_idoc; count++)
{
imagedocument idoc = GetImageDocument(count)
If (idoc.ImageDocumentIsDirty())
{
// now find out what is a filter condition and skip if it is true
number skip = 0
TagGroup tg = idoc.ImageDocumentGetImage(0).ImageGetTagGroup()
skip = tg.TagGroupDoesTagExist("Microscope Info:Formatted Indicated Mag")
if (skip) continue
// do saving
}
}

Terraform - reference different variable within resource or data source dependent on some logic

With this code, I'm planning on having mulitple variables like "vms". These are key/value maps of VMs and resource groups where I for_each on the data source lookup (backupvm) to get their id. Then I can add these VMs, using id, as a backup item in vault.
data "azurerm_virtual_machine" "backupvm" {
for_each = var.vms
name = each.key
resource_group_name = each.value
}
variable "vms" {
type = map
default = {
# "vm name" = "resource group name"
"vaulttestvm1" = "vaulttestrg"
"vaulttestvm2" = "vaulttestrg"
"vaulttestvm4" = "vaulttestrg"
}
}
resource "azurerm_resource_group" "rg" {
name = var.rg_name
location = var.location
}
resource "azurerm_recovery_services_vault" "vault" {
name = var.vault_name
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
sku = "Standard"
# change later
soft_delete_enabled = false
}
resource "azurerm_backup_protected_vm" "vms" {
for_each = var.vms
recovery_vault_name = azurerm_recovery_services_vault.vault.name
resource_group_name = azurerm_recovery_services_vault.vault.resource_group_name
source_vm_id = data.azurerm_virtual_machine.backupvm[each.key].id
backup_policy_id = azurerm_backup_policy_vm.VMBackupPolicy.id
}
I need a way to reference var.vms, in both the data source & resource, so I could interchange using some logic for another map type variable. So if it was PowerShell, it would be something like:
name = if ($env -eq "prod") { $var.vms } elseif ($env -eq "stage") { $var.vmsstage } elseif ($env -eq "dev") { $var.vmsdev }
I've spent a day trying different things but haven't really got close. I may be somewhat restricted as I need to lookup the vm id using the data source first then loop through my resource (azurerm_backup_protected_vm) dropping that ID in. A solution to this would prevent me from having to use multiple data sources and resources of the same type. Thanks!

Declaring list and map variable values in terraform.tfvars

I recently moved over the values for my variables within my terraform code to terraform.tfvars. I am now getting an error that is due to how I am declaring my list and map variables. The code where I am getting the error is replicated below:
image_id = var.web_amis[var.region]
this is how I have these variables specified in terraform.tfvars:
web_amis = ["ami-0dacb0c129b49f529", "ami-00068cd7555f543d5", ]
this is the error code I am getting:
Error: Invalid index
on autoscaling.tf line 3, in resource "aws_launch_configuration" "web_lc":
3: image_id = var.web_amis[var.region]
|----------------
| var.region is "us-east-2"
| var.web_amis is tuple with 2 elements
The given key does not identify an element in this collection value: a number
is required.
You're trying to access a list element with a non index key instead of by position.
What you probably want instead is to have your web_amis variable be a map that is keyed by the region name:
main.tf
variable "region" {}
variable "web_amis" {}
resource "foo_bar" "baz" {
# ...
image_id = var.web_amis[var.region]
}
terraform.tfvars
web_amis = {
us-east-2 = "ami-0dacb0c129b49f529"
us-west-2 = "ami-00068cd7555f543d5"
}
But, this is a very old school and inelegant way of doing things with Terraform nowadays. Instead you could use the aws_ami data source to look up the AMI for the region based on filters such as tags or the name of the AMI.
A basic example is given in the aws_instance resource documentation:
provider "aws" {
region = "us-west-2"
}
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
instance_type = "t2.micro"
tags = {
Name = "HelloWorld"
}
}

Add help documentation for new command loaded from external redis module within redis-cli

Helps on commands within redis-cli are stored in redis/src/help.h.
I would like to provide my help for commands loaded via redis module (using loadmodule). I could find relevant information from Redis Modules: an introduction to the API
Do you have any suggestion?
I did a check on redis/src/redis-cli.c, the help is created during compile time. Currently there is no possibility to do that.
static void cliInitHelp(void) {
int commandslen = sizeof(commandHelp)/sizeof(struct commandHelp);
int groupslen = sizeof(commandGroups)/sizeof(char*);
int i, len, pos = 0;
helpEntry tmp;
helpEntriesLen = len = commandslen+groupslen;
helpEntries = zmalloc(sizeof(helpEntry)*len);
for (i = 0; i < groupslen; i++) {
tmp.argc = 1;
tmp.argv = zmalloc(sizeof(sds));
tmp.argv[0] = sdscatprintf(sdsempty(),"#%s",commandGroups[i]);
tmp.full = tmp.argv[0];
tmp.type = CLI_HELP_GROUP;
tmp.org = NULL;
helpEntries[pos++] = tmp;
}
for (i = 0; i < commandslen; i++) {
tmp.argv = sdssplitargs(commandHelp[i].name,&tmp.argc);
tmp.full = sdsnew(commandHelp[i].name);
tmp.type = CLI_HELP_COMMAND;
tmp.org = &commandHelp[i];
helpEntries[pos++] = tmp;
}
}
Redis module developers should not write their module command document in redis/src/help/h. I would suggest the following:
Using a new Module API function, module developer register new command documentation (consisting of command syntax, summary, since, group) into a system hash.
redis-cli reads additional command documentation from the system hash, to populate the helpEntries[].