How to use Regex to find a specific string between two special characters? - sql

I`m currently migrating sql data between two forum systems (Woltlab Burning Board 2 -> MyBB 1.8).
I need to change the internal links from http://example.com/thread.php?threadid=XYZ
to http://example.com/showthread.php?tid=ABC. Thread ID's will change between the two systems, so I'm not able to do a simple string replace.
I already catch all posts containing http://example.com/thread.php?threadid=. Now I need to get the unique ID into a variable. As the whole post-string can also contain external links (e.g. http://google.com) I cant just catch Numbers before.
I would like to catch the Thread-ID from this string http://example.com/thread.php?threadid=XYZ, which is part of a bigger string (Forum Post). I guess Regex could be used for this.
Any help would be highly appreciated! Thanks!

In PowerShell, the following will capture the thread ID:
$URI = "http://example.com/thread.php?threadid=2438&Query=GetSomething?2345"
$tid = ($URI | select-string -pattern "threadid=(\d+)").matches.groups.value[1]
$tid
If there is no thread id to be captured, the $tid assignment will throw an error.

Related

How can I put several extracted values from a Json in an array in Kusto?

I'm trying to write a query that returns the vulnerabilities found by "Built-in Qualys vulnerability assessment" in log analytics.
It was all going smoothly I was getting the values from the properties Json and turning then into separated strings but I found out that some of the terms posses more than one value, and I need to get all of them in a single cell.
My query is like this right now
securityresources | where type =~ "microsoft.security/assessments/subassessments"
| extend assessmentKey=extract(#"(?i)providers/Microsoft.Security/assessments/([^/]*)", 1, id), IdAzure=tostring(properties.id)
| extend IdRecurso = tostring(properties.resourceDetails.id)
| extend NomeVulnerabilidade=tostring(properties.displayName),
Correcao=tostring(properties.remediation),
Categoria=tostring(properties.category),
Impacto=tostring(properties.impact),
Ameaca=tostring(properties.additionalData.threat),
severidade=tostring(properties.status.severity),
status=tostring(properties.status.code),
Referencia=tostring(properties.additionalData.vendorReferences[0].link),
CVE=tostring(properties.additionalData.cve[0].link)
| where assessmentKey == "1195afff-c881-495e-9bc5-1486211ae03f"
| where status == "Unhealthy"
| project IdRecurso, IdAzure, NomeVulnerabilidade, severidade, Categoria, CVE, Referencia, status, Impacto, Ameaca, Correcao
Ignore the awkward names of the columns, for they are in Portuguese.
As you can see in the "Referencia" and "CVE" columns, I'm able to extract the values from a specific index of the array, but I want all links of the whole array
Without sample input and expected output it's hard to understand what you need, so trying to guess here...
I think that summarize make_list(...) by ... will help you (see this to learn how to use make_list)
If this is not what you're looking for, please delete the question, and post a new one with minimal sample input (using datatable operator), and expected output, and we'll gladly help.

ImageJ batch processing - opening a series of images containing a specific name and doing stuff on them

I have 25K tif files (please don't ask why) that I want to organize into stacks on image J. Basically for each region of interest (ROI), there are 50 images which breaks down into 25 z-planes for two channels. I want everything in a single stack. And I'd like to batch process the whole folder without opening 50 images 500 times at a time. I've attached a picture of what the file names look like:
Folder organization
r01c01f01p01-ch1.tif - the first 10 characters are unique ID to each ROI, then plane number (p01) then channel - ch1 or ch2, then file extension
Here's what I have so far (which I cobbled together based on other macros so this may not make sense...).This is using the ImageJ macros language.
//Processing loop to process each file in the folder.
for (i=0; i<list.length; i++) {
showProgress(i+1, list.length);
if (endsWith(list[i], ".tif")) { // skip the subfolder (I create a subfolder earlier in the macros)
print("-- Processing file: " + list[i] + " --");
open(dir+list[i]);
imageTitle= getTitle();
newTitle = substring(imageTitle, 0, lengthOf(imageTitle)-10); // r01c01f01p, cutting off plane number and then the rest to just get the ROI ID
//This is where I'm stuck:
// find all files containing newTitle and open them (which would be 50 at a time), then run the following macros on them
run("Images to Stack", "name=Ch1 title=[] use");
run("Duplicate...", "title=Ch2 duplicate");
selectWindow("Ch1");
run("Slice Remover", "first=1 last=50 increment=2");
selectWindow("Ch2");
run("Slice Remover", "first=2 last=50 increment=2");
run("Merge Channels...", "c1=Ch1 c2=Ch2 create");
saveAs("tiff", dirNew + newTitle + "_Stack.tif");
//Close(All)?
}
}
print("-- Done --");
showStatus("Finished.");
setBatchMode(false); // Exit batch mode
run("Collect Garbage");
Thank you!
You could do something like:
for (plane=1; plane<51; plane++) {
open(newTitle+plane+"-ch1.tif");
open(newTitle+place+"-ch2.tif");
}
Which would take care of the opening. I would be inclined to have a loop prior to this which would collate the number of unique "newTitle"'s, as your current setup would end up doing something like opening the first item, assembling the combined TIF, and then repeat the process 25K times if I understand it correctly.
Given that you know the number of unique "r01c01f01p" values, in principle you could do a set of stacked loops akin to:
newTitleArray = newArray();
for (r=1; r<50; r++) {
titleBit = "r0" + toString(r);
for (c=1; c<501; c++) {
titleBit = titleBit + "f0"...
Alternatively, you could set up a loop where you check for unique "r01c01f01p" values and add them to an array. In any case, you'd replace the for "list" loop with the for "newTitleArray" loop, and then continue onto the opener I listed above, instead of your existing one.
If I am understanding correctly, it seems like you might do well to stack by channel first, then merge the two. I am not 100% sure, but I think you could potentially use a macro I have already created to do that. It was originally meant to batch process terabytes of 5D data, so it should be very comfortable handling your volume of images. It is not exactly what you are looking for, but should be super easy to modify (I went a little overboard with the commenting in the code), and I think the only thing it does that you might rather it not is produce max projects from the inputs. I'll throw a link here and look for your reply. If it's of interest, let me know and we can work to make it suit your needs together :-) Otherwise, if you could provide a little more detail about where you're getting stuck and/or where I may have misunderstood, I will do my very best to help!
https://github.com/evanjkiely/FIJIMacros

Sentence segmentation and dependency parser

I’m pretty new to python (using python 3) and spacy (and programming too). Please bear with me.
I have three questions where two are more or less the same I just can’t get it to work.
I took the “syntax specific search with spacy” (example) and tried to make different things work.
My program currently reads txt and the normal extraction
if w.lower_ != 'music':
return False
works.
My first question is: How can I get spacy to extract two words?
For example: “classical music”
With the previous mentioned snippet I can make it extract either classical or music. But if I only search for one of the words I also get results I don’t want like.
Classical – period / era
Or when I look for only music
Music – baroque, modern
The second question is: How can I get the dependencies to work?
The example dependency with:
elif w.dep_ != 'nsubj': # Is it the subject of a verb?
return False
works fine. But everything else I tried does not really work.
For example, I want to extract sentences with the word “birthday” and the dependency ‘DATE’. (so the dependency is an entity)
I got
if d.ent_type_ != ‘DATE’:
return False
To work.
So now it would look like:
def extract_information(w,d):
if w.lower_ != ‘birthday’:
return False
elif d.ent_type_ != ‘DATE’:
return False
else:
return True
Does something like this even work?
If it works the third question would be how I can filter sentences for example with a DATE. So If the sentence contains a certain word and a DATE exclude it.
Last thing maybe, I read somewhere that the dependencies are based on the “Stanford typed dependencies manual”. Is there a list which of those dependencies work with spacy?
Thank you for your patience and help :)
Before I get into offering some simple suggestions to your questions, have you tried using displaCy's visualiser on some of your sentences?
Using an example sentence 'John's birthday was yesterday', you'll find that within the parsed sentence, birthday and yesterday are not necessarily direct dependencies of one another. So searching based on the birthday word having a dependency of a DATE type entity, might not be yield the best of results.
Onto the first question:
A brute force method would be to look for matching subsequent words after you have parsed the sentence.
doc = nlp(u'Mary enjoys classical music.')
for (i,token) in enumerate(doc):
if (token.lower_ == 'classical') and (i != len(doc)-1):
if doc[i+1].lower_ == 'music':
print 'Target Acquired!'
If you're unsure of what enumerate does, look it up. It's the pythonic way of using python.
To questions 2 and 3, one simple (but not elegant) way of solving this is to just identify in a parsed sentence if the word 'birthday' exists and if it contains an entity of type 'DATE'.
doc = nlp(u'John\'s birthday was yesterday.')
for token in doc:
if token.lower_ == 'birthday':
for entities in doc.ents:
if entities.label_ == 'DATE':
print 'Found ya!'
As for the list of dependencies, I presume you're referring to the Part-Of-Speech tags. Check out the documentation on this page.
Good luck! Hope that helped.

Creating a string from a path

I have a chain of nodes, e.g.
{Name: 'X'}->{Name: 'Y'}->{Name: 'Z'}
and I'd like to create a string representing the path in cypher (I know I can do it client-side but I want to do it in a query) that would look like this:
'X->Y->Z'
is that feasible? I've investigated use of collect() and UNWIND, and have googled till I'm blue in the face
thoughts?
* Edit I *
as an addendum (and to make the problem more difficult) my query is going to return a collection of paths (a tree, a DAG), so I'll need to create a string for each of the paths in the tree
REDUCE is your friend here:
WITH reduce(s="",n in nodes(p) | s+n.name+"->") as str
RETURN substring(str,0,length(str)-2)
or if you want to save the extra operation
RETURN reduce(s=head(nodes(p)).name, n in tail(nodes(p)) | s+"->"+n.name)
or with APOC
RETURN apoc.text.join([n in nodes(p) | n.name],"->")
found an old posting (2014) from Michael Hunger with the answer:
http://grokbase.com/t/gg/neo4j/147ch1nj9b/concatenate-nodes-properties

inserting variables into MySQL

I am trying to insert 2 scores into Mysql for two photos for a particular user that already exists in the database. The scores and the photos are both POST variables from a form. I am having great difficulty with the syntax - I am fairly certain the error is related to the position of quotes but despite searching here and finding similar questions I can't seem to get it working. Loathed to bother people with this but need some executive assistance.
$imageT=$_POST[randomimage]."T" ;
$imageH=$_POST[randomimage]."H" ;
$observerid=$_POST[scoreid];
$traction=$_POST[gradeT];
$honeycomb=$_POST[gradeH];
$sql="INSERT INTO scorers ('$imageT', '$imageH')
VALUES ('$imageT', '$imageH') WHERE id=$observerid ";
if (!mysqli_query($con,$sql)) {
die('Error: ' . mysqli_error($con));
} else {
header("Location: testform.php");
} '
$imageT and $imageH are both integers with either T or H appended to them, for example 12T or 14H therefore I assumed they would be treated as strings and I put quotes around them. $traction, $honeycomb and $observerid are all integers. When I echo $imageT, $imageH, $traction, $honeycomb and $observerid the correct values are shown so I am assuming that there is no error in the these, just they way I am placing them within SQL code.
Very much appreciate any help (been learning PHP and My SQL for only 4 weeks so apologies).
At least three main problems at glance
You aren't using prepared statements
You are using WHERE clause in INSERT statement which is useless and erroneous. Either remove WHERE part or change your query to UPDATE.
You didn't post the error with your question. Which you always have to. Error messages is a cornerstone of troubleshooting.