Long story short, I have a sql query pulling data from a database into a spreadsheet. I want to be able to change the Plant filter inside the query via Power Query, I have tried using a custom function and adding that into the query where the variable is set (see below)
Sql.Database("server", "database", [Query="DECLARE #FilterOnPlant AS varchar(3)#(lf)SET #FilterOnPlant = '" + filterOnPlant + "'#(lf)#(lf)SELECT
In doing this I keep getting the following error:
Expression.Error: We cannot apply operator + to types Text and Table.
Details:
Operator=+
Left=DECLARE #FilterOnPlant AS varchar(3)
SET #FilterOnPlant = '
Right=[Table]
I figure it has something to do with the '+' operator used but can't find an alternative.
I have tried using:
'&'
'+'
'and'
but they all yield the same error (with the exception of 'and' that error states that it's not being used in a logical statement which makes sense)
For further info:
The custom function to get the value I want is:
filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]
The intention is to get the Value from a cell Named "Active_Plant"
EDIT:
after further testing I have found the issue lies with variable filterOnPlant, when I exchange that for a string value then the query works as expected. If anyone knows how I can get CELL().Value of a named range in power query would be super helpful
Solved!
In my filterOnPlant function I was missing {0}[Column1]
Original: filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]
New (Working): filterOnPlant = Excel.CurrentWorkbook(){[Name="Active_Plant"]}[Content]{0}[Column1]
Related
I have to remove the accents from the person's name, but I cannot apply the function in Talend while it works in SQL oracle.
this query works in my tDBInput component :
"SELECT '"+((String)globalMap.get("copyOfSORTIE.NOM"))+"' as nom_nom_compl,
'"+((String)globalMap.get("copyOfSORTIE.ENTETE"))+"' entete
FROM DUAL"
However, when I want to add the convert function, it doesn't work
this query does not work :
"SELECT '"+((String)globalMap.get(CONVERT("copyOfSORTIE.NOM",'US7ASCII')))+"' as nom_nom_compl,
'"+((String)globalMap.get("copyOfSORTIE.ENTETE"))+"' entete
FROM DUAL"
In my talend :
I am getting this error
What is the syntax for it to work?
Thank you!
Two things there :
I don't know the CONVERT method, but I can see that you are applying it to the key of your globalMap variable , and not the value (as if you wanted to convert "myKey" and not "myValue" which is attached to the key). Are you sure this is what you want to achieve ? if not, the syntax should be something similar to "SELECT CONVERT('"+((String)globalMap.get("copyOfSORTIE.NOM"))+"','US7ASCII') "
A useful java method implemented in talend is TalendString.removeAccents("") that you can apply directly on your talend variable, thus not using a SQL method.
I am using a SQL notebook in Databricks/Spark community edition
%python
education_DF = sqlContext.sql('select * from global_temp.population_Globaltmp_view where `Education` = "2YD"')
display(education_DF)
The above code cell works fine and retrieves the desired rows from the view.
However the following code cell gives an error.
I am supbstituting the hard-coded value "2YD" in the WHERE clause, with a variable call education_choice that contains the same value.
It seems that I am not using variables correctly in SQL. How would I make this work?
%python
education_DF = sqlContext.sql('select * from global_temp.population_Globaltmp_view where `Education` = education_choice')
display(education_DF)
(This is the error)
org.apache.spark.sql.AnalysisException: cannot resolve '`education_choice`' given input columns: [global_temp.population_globaltmp_view.Salary, global_temp.population_globaltmp_view.Address, global_temp.population_globaltmp_view.Race, global_temp.population_globaltmp_view.MiddleI, global_temp.population_globaltmp_view.Education, global_temp.population_globaltmp_view.HairColor, global_temp.population_globaltmp_view.Age, global_temp.population_globaltmp_view.FullName, global_temp.population_globaltmp_view.City, global_temp.population_globaltmp_view.FirstName, global_temp.population_globaltmp_view.State, global_temp.population_globaltmp_view.LastName, global_temp.population_globaltmp_view.Height, global_temp.population_globaltmp_view.Fertility, global_temp.population_globaltmp_view.Employment, global_temp.population_globaltmp_view.Zip, global_temp.population_globaltmp_view.Weight, global_temp.population_globaltmp_view.Gender]; line 1 pos 72;
I tried the injection method suggested, but got a slightly different error this time. I included a screen shot of the relevant code cells. It seems that the education_choice "2YD" isn't being recognized as a literal but a field or something like that.
Shouldn't 2YD have quotes around it like "2YD"
If I hard code the WHERE clause like this: WHERE Education = "2YD" the queries works fine.
The image bellow is small but if you right click on it and "open in new tap, it is very readable.
*EDIT
You use the variable education_choice as a hardcoded string. Instead, you should inject the value of education_choice inside the SQL expression string like that.
*edit - wrap the variable with quoets.
%python
education_DF = sqlContext.sql('select * from global_temp.population_Globaltmp_view where `Education` = "{}"'.format(education_choice))
display(education_DF)
I am a bit stumped with what happens here. I'm working on a challenge for an online programming course and the solution involves adding a second where statement to the sql query.
when I do this, however, python gives me a typeerror, saying "TypeError: 'NoneType' object is not subscriptable".
I'm pretty sure this is because the fetchone()[1] at the end requires a list or tuple to retrieve the ID. what I don't understand is why adding the second bit of the WHERE clause would change the type of the result.
I have checked the statement itself and it works when testing it on the database, and the first link_id statement in the code below works fine, it returns an int object but the second gives an error. It should return a similar int object.
link_id = self.cursor.execute(self.sql_select + " WHERE " + self.field + "=?", (value,)).fetchone()[1]
print("link_id type:", type(link_id))
link_id = self.cursor.execute(self.sql_select + " WHERE " + self.field + "=? AND albums.artist=?", (value, self.artist_id)).fetchone()[1]
print("link_id type:", type(link_id))
it was because self.artist_id was None. it reset when running this block of code again.
I am using a simple sql block of statements to execute and return a set of results in big query, This is working fine in big query and getting the results,I need to export this data to data studio, so in data studio i use bigquery as connector and select the project and custom query and in that I paste the contents below:
Declare metricType String;
SET metricType="compute.googleapis.com/instance/cpu/utilization";
BEGIN
IF (metricType="compute.googleapis.com/instance/cpu/usage_time")
THEN
SELECT m.value as InstanceName,metric.type as metricType,point.value.double_value as usagetime,point.interval.start_time as StartTime,point.interval.end_time as EndTime,h.value as instance_id FROM `myproject.metric_export.sd_metrics_export_fin`, unnest(resource.labels) as h,unnest(metric.labels) as m where metric.type='compute.googleapis.com/instance/cpu/usage_time' and h.key="project_id";
ELSE IF (metricType="compute.googleapis.com/instance/cpu/utilization")
THEN
SELECT m.value as InstanceName,metric.type as metricType,point.value.double_value as utilizationrate,point.interval.start_time as
StartTime,point.interval.end_time as EndTime,h.value as instance_id FROM `myproject-.metric_export.sd_metrics_export_fin`,unnest(resource.labels) as h,unnest(metric.labels) as m where metric.type='compute.googleapis.com/instance/cpu/utilization' and h.key="project_id";
END IF;
END IF;
END;
but after click "ADD" button i get the below error:
I am not sure what is this error about? I have not used any stored procedure and I am just pasting it as custom query.
Also If I try to save the results of the BigQuery into a view from the Bigquery console results pane, it gives me the error,
Syntax error: Unexpected keyword DECLARE at [1:1]
I am extremely new to datastudio and also to bigquery. Kindly Help thanks
First, I would like to make some considerations about your query. You are using Scripting in order to declare and create a loop within your query. However, since you declare and set the metricsType in the beginning of the query, it will never enter in the first IF. This happens because the value is set and it is not looping through anything.
I would suggest you to use CASE WHEN instead, as below:
SELECT m.value as InstanceName,metric.type as metricType,
CASE WHEN metric.type = #parameter THEN point.value.double_value ELSE 0 END AS usagetime,
CASE WHEN metric.type = #parameter THEN point.value.double_value ELSE 0 END AS utilizationrate,
point.interval.start_time as StartTime,point.interval.end_time as EndTime,h.value as instance_id
FROM `myproject.metric_export.sd_metrics_export_fin`, unnest(resource.labels) as h,unnest(metric.labels) as m
WHERE metric.type=#parameter and h.key="project_id";
Notice that I am using the concept of Parameterized queries. Also, for this reason this query won't work in the console. In addition, pay attention that whem you set the #parameter to "compute.googleapis.com/instance/cpu/utilization", it will have a non-null column with the usagetime and a null column named utilizationrate.
Secondly, in order to add a new data source in DataStudio, you can follow this tutorial from the documentation. After, selecting New Report, click on the BigQuery Connector > Custom Query> Write your Project id, you need to click in ADD PARAMETER (below the query editor). In the above query, I would add:
Name: parameter
Display name: parameter
Data type: text
Default value: leave it in blank
Check the box Allow "parameter" to be modified in reports. This means you will be able to use this parameter as a filter and modify its value within the reports.
Following all the steps above, the data source will be added smoothly.
Lastly, I must point that if your query ran in the Console you are able to save it as a view by clicking on Save view, such as described here.
The following query (qryCurLotNewProducts) produces a data set that I want process further with another query (qryBNP_CFRecordset):
//qryCurLotNewProducts
SELECT tblNewProducts.*
FROM tblNewProducts INNER JOIN tblCurLot ON (tblCurLot.CatalogNum = tblNewProducts.CatalogNum) AND
(tblNewProducts.LotNum = tblCurLot.CurLot);
When I run this second query to list only the "CF" products found in the first query, I get the `Data type mismatch in criteria expression' error.
//qryBNP_CFRecordset
SELECT qryCurLotNewProducts.*, tblABCategory.UNSPSC, tblAmount.ProductSize
FROM tblAmount RIGHT JOIN (tblABCategory RIGHT JOIN qryCurLotNewProducts ON tblABCategory.ABCategory = qryCurLotNewProducts.ABCategory) ON tblAmount.Amount = qryCurLotNewProducts.Amount
WHERE (((qryCurLotNewProducts.CatalogNum) Like "A700-###CF") AND ((qryCurLotNewProducts.DateEntered) Between #1/1/2000# And #3/1/2020#))
ORDER BY qryCurLotNewProducts.CatalogNum, Abs(qryCurLotNewProducts.LotNum);
If I remove the CF from the search string (so "A700-###"), the query correctly outputs a list containing all items that contain that pattern:
If I use strings like "A700-####F" or "A700-###ZZ" or other combinations like that, I don't get an error but rather an empty results set.
Notably, "A700-001CF", "A700-002CF", etc all create the data type error. It seems there is something about the CF key combination that is causing trouble.
Has anybody else ever seen this issue? Do I need to use some kind of delimiter to tell SQL to not view CF as some kind of special switch?
Abs(qryCurLotNewProducts.LotNum) wont work with the values for Products ending in CF. Your LotNum-Column has a text-type.
Edit: Your LotNum-Column has a text-type as you can see in your first screenshot.