I'm using Grafana plus Prometheus queries to create dashboards in Grafana for Kubernetes. I take the name of the nodes (3 in this case) in a variable and then I pass this values to other query to extract the IPs of the machines. The values extracted are correct. I have the multi-value option enabled.
The problem comes with the query sum(rate(container_cpu_usage_seconds_total{id="/", instance=~"$ip_test:10250"}[1m])) and more than one IP because it only takes one of them. In other query it works but I think it is possible because the other query has not the :10250 after the variable.
My question, do you know any way to concatenate all the ip:port? E.g.: X.X.X.X:pppp|X.X.X.X:pppp
Try it like this:
sum(rate(container_cpu_usage_seconds_total{id="/", instance=~"($ip_test):10250"}[1m]))
From multiple values formating documentation, Prometheus variables are expanded as regex:
InfluxDB and Prometheus uses regex expressions, so the same variable
would be interpolated as (host1|host2|host3). Every value would also
be regex escaped if not, a value with a regex control character would
break the regex expression.
Therefore your variable ip_test = ['127.0.0.1', '127.0.0.2',...] is supposed to be transformed into: (127\.0\.0\.1|127\.0\.0\.2).
This means that your expression =~$ip_test:10250 should be transformed into =~"(127\.0\.0\.1|127\.0\.0\.2):10250" so you don't need the multiple expansion you are asking for.
The reason it is not working is that either the documentation is incorrect or there is a bug in Grafana (tested with version v6.7.2). From my tests, I suspect, the Prometheus expansion doesn't expand with the enclosing () and you end up with the expression =~"127\.0\.0\.1|127\.0\.0\.2:10250" - which is not what you want.
The workaround is to use the regex notation =~"${ip_test:regex}:10250".
Related
I have a big sporadic sql scripts and need to find and replace a few values in it. I am trying to pass my values in REGEX to Notepad++ but I can't seem to make it work. To be more specific, I have around 50 script, each with 5000 lines, and I need to look for a list of values, e.g. "[dbo].[livesales]" "[dbo].[CreditCards]" in all my scripts separately. I undertand that I need either run this separately against each script or merger them all into one file, but I need the proper REGEX command for it. I need to include square bracket and dots as well. I end up building this but it doesn't work for me:
^(?=.*\b[dbo].[LiveSales]\b)(?=.*\b[dbo].[CreditCards]\b).+$
enter image description here
thanks in advance,
I wouldn't bother using word boundaries, as square brackets in SQL Server are pretty ubiquitous for database object names (e.g. database and column names). I suggest the following pattern:
\[dbo\]\.\[(?:LiveSales|CreditCards)\]
Demo
The major changes I have made include not using word boundaries, escaping the [ and ] brackets (since square bracket is a regex metacharacter with a special meaning), and also not try to match the entire input. Presumably you want to find all such occurrences, and so don't bother trying to scope your pattern with ^ and $.
I noticed an inconsistency in how "HANA SQL" escapes single quotes in the context of the PLACEHOLDER clause. For example, consider the following PLACEHOLDER clause snippet:
('PLACEHOLDER' = ('$$CC_PARAM$$','''foo'',''an escaped single quote \'' '''))
The PLACEHOLDER clause above contains multiple values assigned to the CC_PARAM. parameter. We can see that inside of the second argument we have a single quote that's escaped with a backslash. However, we escape the single quotes outside each argument with another single quote (i.e. we do '' instead of \''. It's possible to use the \'' format for the first case, but it's not possible to use the '' format in the second case.
Why is there this discrepancy? It makes escaping quotes in multi-input input parameters tricky. I'm looking to programmatically craft SQL queries for HANA. Am I missing something here? Is it safe to use \'' over '' in all cases? Or do I need logic that can tell where a single quote occurs and escape as appropriate?
The implicit rule here - given by how the software is implemented - is that for parameter values of calculation views, the backslash \ is used to escape the single quotation mark.
For all standard SQL string occurrences, using the single-quotation mark twice '' is the correct way to differentiate between syntax element and string literal.
As for the why:
the PLACEHOLDER syntax is not SQL, but a HANA-specific command extension. So, there is no general standard that the current implementation violates.
that given, this command extension is embedded into, respectively clamped onto the standard SQL syntax and has to be handled by the same parser.
But the parameters are not only parsed once, by the SQL parser but again by the component that instantiates the calculation scenario based on the calculation view. With a bit of squinting it's not hard to see that the parameters interface is a general key-value interface that allows for all sorts of information to be handed over to the calc. engine.
One might argue that the whole approach of providing parameters via key-value pairs is not consistent with the general SQL syntax approach and be correct. On the flip side, this approach allows for general flexibility for adding new command elements to the HANA-specific parts, without structurally changing the syntax (and with it the parser).
The clear downside of this is that both the key names, as well as the values, are string-typed. To avoid losing the required escaping for the "inner string" an escape string different from the main SQL escape string needs to be used.
And here we are with two different ways of handing over a string value to be used as a filter condition.
Funny enough, both approaches may still lead to the same query execution plan.
As a matter of fact, in many scenarios with input parameters, the string value will be internally converted into a SQL conforming form. This is the case when the input parameter is used for filtering or in expressions in the calc. view that can be converted into SQL expressions.
For example
SELECT
"AAA"
FROM "_SYS_BIC"."sp/ESC"
('PLACEHOLDER' = ('$$IP_TEST$$', 'this is a test\''s test'));
shows the following execution plan on my system
OPERATOR_NAME OPERATOR_DETAILS
PROJECT TEST.AAA
COLUMN TABLE FILTER CONDITION: TEST.AAA = 'this is a test's test'
(DETAIL: ([SCAN] TEST.AAA = 'this is a test's test'))
Note how the escape-\' has been removed.
All in all: when using PLACEHOLDER values, the \' escaping needs to be used and in all other cases, the '' escaping.
That should not be terribly difficult to implement for a query builder as you can consider this when dealing with the PLACEHOLDER syntax.
I have one requirement in BODS to execute ABAP dataflows with dynamic global variables. The ABAP code is a custom one with dynamic where clause. I am trying to send the global variable value like Field-name like "%underscroe%". But instead of fetching records with only 'underscore', it is fetching all records from the table.
Do you have any solution for this issue?
Use ESCAPE clause for this. If you wanna search for underscore, set
LIKE '%#_%' ESCAPE '#'.
I am not sure, if it works with BODS though.
From the documentation on properly passing SQL parameters to psycopg2:
"Only variable values should be bound via this method: it shouldn’t be used to set table or field names. For these elements, ordinary string formatting should be used before running execute()."
http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
Why is this? Why would setting a field name not have the same SQL injection problems as setting values?
Because you don't usually receive names of your tables or fields from users, you specify them in your code yourself. If you generate your request based on values got from user input, than yes, you should do some kind of escaping to avoid SQL injections.
I have a command class that abstracts almost all specific database functions (We have the exactly same application running on Mssql 2005 (using ODBC and the native mssql library), MySQL and Oracle. But sometimes we had some problems with our prepare method that, when executed, replaces all placeholders with their respective values. But the problem is that I am using the following:
if(is_array($Parameter['Value']))
{
$Statement = str_ireplace(':'.$Name, implode(', ', $this->Adapter->QuoteValue($Parameter['Value'])), $Statement);
}
else
{
$Statement = str_ireplace(':'.$Name, $this->Adapter->QuoteValue($Parameter['Value']), $Statement);
}
The problem arises when we have two or mer similar parameters names, for example, session_browser and session_browse_version... The first one will partially replace the last one.
Course we learned to go around specifying the parameters within a specific order, but now that I have some "free" time I want to make it better, so I am thinking on switching to preg_replace... and I am not good in regular expression, can anyone give any help with a regex to replace a string like ':parameter_name`?
Best Regards,
Bruno B B Magalhaes
You should use the \b metacharacter to match the word boundary, so you don't accidentally match a short parameter name within a longer parameter name.
Also, you don't need to special-case arrays if you coerce a scalar Value to an array of one entry:
preg_replace("/:$Name\b/",
implode(",", $this->Adapter->QuoteValue( (array) $Parameter['Value'] )),
$Statement);
Note, however, that this can make false positive matches when an identifier or a string literal contains a pattern that looks like a parameter placeholder:
SELECT * FROM ":Name";
SELECT * FROM Table WHERE column = ':Name';
This gets even more complicated when quoted identifiers and string literals can contain escaped quotes.
SELECT * FROM Table WHERE column = 'word\':Name';
You might want to reconsider interpolating variables into SQL strings during prepare, because you're defeating any benefits of prepared statements with respect to security or performance.
I understand why you're doing what you're doing, because not all RDBMS back-end supports named parameters, and also SQL parameters can't be used for lists of values in an IN() predicate. But you're creating an awfully leaky abstraction.