How to make Linq to SQL translate to a derived column? - sql

I have a table with a 'Wav' column that is of type 'VARBINARY(max)' (storing a wav file) and would like to be able to check if there is a wav from Linq to SQL.
My first approach was to do the following in Linq:
var result = from row in dc.Table
select new { NoWav = row.Wav != null };
The problem with the code above is it will retreive all the binary content to RAM, and this isn't good (slow and memory hungry).
Any idea how to have Linq query to translate into something like bellow in SQL?
SELECT (CASE WHEN Wav IS NULL THEN 1 ELSE 0 END) As NoWav FROM [Update]

Thanks for all the replies. They all make sense. Indeed, Linq should translate the != null correctly, but it didn't seem to effectively do it: running my code was very slow, so somehow my only explaination is that it got the binary data transfered over to the RAM.... but maybe I'm wrong.
I think I found a work around anyway somewhere else on stackoverflow: Create a computed column on a datetime
I ran the following query against my table:
ALTER TABLE [Table]
ADD WavIsNull AS (CASE WHEN [Wav] IS NULL Then (1) ELSE (0) END)
Now I'll update my DBML to reflect that computed column and see how it goes.

Are you sure that this code will retrieve the data to RAM?
I did some testing using LINQPad and the generated SQL was optimized as you suggest:
from c in Categories
select new
{
Description = c.Description != null
}
SELECT
(CASE
WHEN [t0].[description] IS NOT NULL THEN 1
ELSE 0
END) AS [Description]
FROM [Category] AS [t0]

What about this query:
var result = from row in dc.Table where row.Wav == null
select row.PrimaryKey
for a list of keys where your value is null. For listing of null/not null you could do this:
var result = from row in db.Table
select new
{ Key = row.Key, NoWav = (row.Wav == null ? true : false) };
That will generate SQL code similar to this:
SELECT [t0].[WavID] AS [Key],
(CASE
WHEN [t0].[Wav] IS NULL THEN 1
ELSE 0
END) AS [NoWav]
FROM [tblWave] AS [t0]

I'm not clear here, your SQL code is going to return a list of 1s and 0s from your database. Is that what you are looking for? If you have an ID for your record then you could just retrieve that single record with the a condition on the Wav field, null return would indicate no wav, i.e.
var result = from row in dc.Table
where (row.ID == id) && (row.Wav != null)
select new { row.Wav };

Related

PostgreSQL conditional where clause

In my Ruby on Rails app I'm using blazer(https://github.com/ankane/blazer) and I have the following sql query:
SELECT *
FROM survey_results sr
LEFT JOIN clients c ON c.id = sr.client_id
WHERE sr.client_id = {client_id}
This query works really well. But I need to add conditional logic to check if client_id variable is present. If yes then I filter by this variable, if not then I not launching this where clause. How can I do it in PostgreSQL?
Check if its null OR your condition like this:
WHERE {client_id} IS NULL OR sr.client_id = {client_id}
The WHERE clause evaluate as follow: If the variable is empty, then the WHERE clause evaluate to true, and therefore - no filter. If not, it continue to the next condition of the OR
If anyone faced with the psql operator does not exist: bigint = bytea issue, here is my workaround:
WHERE ({client_id} < 0 AND sr.client_id > {client_id}) OR sr.client_id = {client_id}
Please consider that, client_id generally cannot be negative so you can use that information for eleminating the operation cast issue.
My solution:
I use spring data jpa, native query.
Here is my repository interface signature.
#Query(... where (case when 0 in :itemIds then true else i.id in :itemIds end) ...)
List<Item> getItems(#Param("itemIds) List<Long> itemIds)
Prior calling this method, I check if itemIds is null. If yes, I set value to 0L:
if(itemIds == null) {
itemIds = new ArrayList<Long>();
itemIds.add(0L);
}
itemRepo.getItems(itemIds);
My IDs starts from 1 so there is no case when ID = 0.

how do I join two tables sql

I have an issue that I'm hoping you can help me with. I am trying to create charting data for performance of an application that I am working on. The first step for me to perform two select statements with my feature turned off and on.
SELECT onSet.testName,
avg(onSet.elapsed) as avgOn,
0 as avgOff
FROM Results onSet
WHERE onSet.pll = 'On'
GROUP BY onSet.testName
union
SELECT offSet1.testName,
0 as avgOn,
avg(offSet1.elapsed) as avgOff
FROM Results offSet1
WHERE offSet1.pll = 'Off'
GROUP BY offSet1.testName
This gives me data that looks like this:
Add,0,11.4160277777777778
Add,11.413625,0
Delete,0,4.5245277777777778
Delete,4.0039861111111111,0
Evidently union is not the correct feature. Since the data needs to look like:
Add,11.413625,11.4160277777777778
Delete,4.0039861111111111,4.5245277777777778
I've been trying to get inner joins to work but I can't get the syntax to work.
Removing the union and trying to put this statement after the select statements also doesn't work. I evidently have the wrong syntax.
inner join xxx ON onSet.testName=offset1.testName
After getting the data to be like this I want to apply one last select statement that will subtract one column from another and give me the difference. So for me it's just one step at a time.
Thanks in advance.
-KAP
I think you can use a single query with conditional aggregation:
SELECT
testName,
AVG(CASE WHEN pll = 'On' THEN elapsed ELSE 0 END) AS avgOn,
AVG(CASE WHEN pll = 'Off' THEN elapsed ELSE 0 END) AS avgOff
FROM Results
GROUP BY testName
I just saw the filemaker tag and have no idea if this work there, but on MySQL I would try something along
SELECT testName, sum(if(pll = 'On',elapsed,0)) as sumOn,
sum(if(pll = 'On',1,0)) as numOn,
sum(if(pll ='Off',elapsed,0)) as sumOff,
sum(if(pll ='Off',1,0)) as numOff,
sumOn/numOn as avgOn,
sumOff/numOff as avgOff
FROM Results
WHERE pll = 'On' or pll='Off'
GROUP BY testName ;
If it works for you then this should be rather efficient as you do not need to join. If not, thumbs pressed that this triggers another idea.
The difficulty you have with the join you envisioned is that the filtering in the WHERE clause is performed after the join was completed. So, you would still not know what records to use to compute the averages. If the above is not implementable with FileMaker then check if nested queries work. You would then
SELECT testName, on.avg as avgOn, off.avg as avgOff
FROM ( SELECT ... FROM Results ...) as on, () as off
JOIN on.testName=off.testName
If that is also not possible then I would look for temporary tables.
OK guys... thanks for the help again. Here is the final answer. The statement below is FileMaker custom function that takes 4 arguments (platform, runID, model and user count. You can see the sql statement is specified. FileMaker executeSQL() function does not support nested select statements, does not support IF statements embedded in select statements (calc functions do of course) and finally does not support the SQL keyword VALUES. FileMaker does support the SQL keyword CASE which is a little more powerful but is a bit wordy. The select statement is in a variable named sql and result is placed in a variable named result. The ExecuteSQL() function works like a printf statement for param text so you can see the swaps do occur.
Let(
[
sql =
"SELECT testName, (sum( CASE WHEN PLL='On' THEN elapsed ELSE 0 END)) as sumOn,
sum( CASE WHEN PLL='On' THEN 1 ELSE 0 END) as countOn,
sum( CASE WHEN PLL='Off' THEN elapsed ELSE 0 END) as sumOff,
sum( CASE WHEN PLL='Off' THEN 1 ELSE 0 END) as countOff
FROM Results
WHERE Platform = ?
and RunID = ?
and Model = ?
and UserCnt = ?
GROUP BY testName";
result = ExecuteSQL ( sql ; "" ; ""
; platform
; runID
; model
; userCnt )
];
getAverages ( Result ; "" ; 2 )
)
For those interested the custom function looks like this:
getAverages( result, newList, pos )
Let (
[
curValues = Substitute( GetValue( data; pos ); ","; ¶ );
sumOn = GetValue( curValues; 2 ) ;
countOn = GetValue( curValues; 3 );
sumOff = GetValue( curValues; 4 );
countOff = GetValue( curValues; 5 );
avgOn = sumOn / countOn;
avgOff = sumOff / countOff
newItem = ((avgOff - avgOn) / avgOff ) * 100
];
newList & If ( pos > ValueCount( data); newList;
getAverages( data; If ( not IsEmpty( newList); ¶ ) & newItem; pos + 1 ))
)

How to parse big string U-SQL Regex

I have got a big CSVs that contain big strings. I wanna parse them in U-SQL.
#t1 =
SELECT
Regex.Match("ID=881cf2f5f474579a:T=1489536183:S=ALNI_MZsMMpA4voGE4kQMYxooceW2AOr0Q", "ID=(?<ID>\\w+):T=(?<T>\\w+):S=(?<S>[\\w\\d_]*)") AS p
FROM
(VALUES(1)) AS fe(n);
#t2 =
SELECT
p.Groups["ID"].Value AS gads_id,
p.Groups["T"].Value AS gads_t,
p.Groups["S"].Value AS gads_s
FROM
#t1;
OUTPUT #t
TO "/inhabit/test.csv"
USING Outputters.Csv();
Severity Code Description Project File Line Suppression State
Error E_CSC_USER_INVALIDCOLUMNTYPE:
'System.Text.RegularExpressions.Match' cannot be used as column type.
I know how to do it in a SQL way with EXPLODE/CROSS APPLY/GROUP BY. But may be it is possible to do without these dances?
One more update
#t1 =
SELECT
Regex.Match("ID=881cf2f5f474579a:T=1489536183:S=ALNI_MZsMMpA4voGE4kQMYxooceW2AOr0Q", "ID=(?<ID>\\w+):T=(?<T>\\w+):S=(?<S>[\\w\\d_]*)").Groups["ID"].Value AS id,
Regex.Match("ID=881cf2f5f474579a:T=1489536183:S=ALNI_MZsMMpA4voGE4kQMYxooceW2AOr0Q", "ID=(?<ID>\\w+):T=(?<T>\\w+):S=(?<S>[\\w\\d_]*)").Groups["T"].Value AS t,
Regex.Match("ID=881cf2f5f474579a:T=1489536183:S=ALNI_MZsMMpA4voGE4kQMYxooceW2AOr0Q", "ID=(?<ID>\\w+):T=(?<T>\\w+):S=(?<S>[\\w\\d_]*)").Groups["S"].Value AS s
FROM
(VALUES(1)) AS fe(n);
OUTPUT #t1
TO "/inhabit/test.csv"
USING Outputters.Csv();
This wariant works fine. But there is a question. Will the regex evauated 3 times per row? Does exists any chance to hint U-SQL engine - the function Regex.Match is deterministic.
You should probably be using something more efficient than Regex.Match. But to answer your original question:
System.Text.RegularExpressions.Match is not part of the built-in U-SQL types.
Thus you would need to convert it into a built-in type, such as string or SqlArray<string> or wrap it into a udt that provides an IFormatter to make it a user-defined type.
Looks like it is better to use something like this to parse the simple strings. Regexes are slow for the task and if i will use simple string expressions (instead of CLR calls) they probably will be translated into c++ code at codegen phase... and .net interop will be eliminated (i'm not sure).
#t1 =
SELECT
pv.cust_gads != null ? new SQL.ARRAY<string>(pv.cust_gads.Split(':')) : null AS p
FROM
dwh.raw_page_view_data AS pv
WHERE
pv.year == "2017" AND
pv.month == "04";
#t3 =
SELECT
p != null && p.Count == 3 ? p[0].Split('=')[1] : null AS id,
p != null && p.Count == 3 ? p[1].Split('=')[1] : null AS t,
p != null && p.Count == 3 ? p[2].Split('=')[1] : null AS s
FROM
#t1 AS t1;
OUTPUT #t3
TO "/tmp/test.csv"
USING Outputters.Csv();

Linq Query in VB

Good Day,
I am querying my database using Linq and I have run into a problem, the query searched a column for a search phrase and based on if the column has the phrase, it then returns the results, The query is below,
Dim pdb = New ProductDataContext()
Dim query =
From a In pdb.tblUSSeries
Join b In pdb.tblSizes_ On a.Series Equals b.Series
Where
a.Series.ToString().Equals(searchString) Or
b.Description.Contains(searchString) Or Not b.Description.Contains(Nothing)
Order By b.Series, b.OrderCode Ascending
Select New CustomSearch With
{
.Series = a.Series,
.SeriesDescription= a.Description,
.Coolant = a.Coolant,
.Material = a.Material,
.Standard = a.Standard,
.Surface = a.Surface,
.Type = a.Type,
.PointAngle = a.PointAngle,
.DiaRange = a.DiaRange,
.Shank = b.Shank,
.Flutes = b.Flutes,
.EDPNum = b.EDPNum,
.SizesDescription = b.Description,
.OrderCode = b.OrderCode
}
Return query
I think the problem is that, in the table certain rows are NULL, so when it is checking the column for the phrase and it encounters a row that is null it, breaks and returns this error,
The cast to value type 'System.Int32' failed because the materialized value is null. Either the result type's generic parameter or the query must use a nullable type.
I have ran this query against another column that has all the rows populated with data and it returns the results ok.
So my question is how can I write it in VB to query the db with the supplied searchstring and return the results, when some of the rows in the columns have null values.
Any help would be great.
The exception occurs when you make the projection (i.e. select new CustomSearch)
And yes your trying to assign Null to some int property
(Not sure which one of your properties that is)
one of 2 choices :
1) Use nullalbe types for your properties (or just that one property).
2) project with an inline If ( ?? in C#) , I don't know VB so don't catch me on the syntax.
Taking Series just as an example i don't know if it's an int or if that's the problematic property
Select New CustomSearch With
{
.Series = If(a.Series Is Nothing,0, CInt(a.Series))
}
In C#
Select new CustomSearch
{
Series = a.Series ?? 0;
}

LINQ Incompatibility Issue with SQL Server 2000

I have a Linq to SQL query that was working just fine with SQL Server 2005 but, I have to deploy the web app with a SQL Server 2000 and, when executing that query, I get his error:
"System.Data.SqlClient.SqlException: The column prefix 't0' does not match with a table name or alias name used in the query."
I have more queries but it doesn't seems to have problems with those.
Now, this is the query:
from warningNotices in DBContext_Analyze.FARs
where warningNotices.FAR_Area_ID == filter.WarningAreaID &&
warningNotices.FAR_Seq == filter.WarningSeq &&
warningNotices.FAR_Year == filter.WarningYear
orderby warningNotices.FAR_Seq ascending
select new Search_Result
{
FAR_Area_ID = warningNotices.FAR_Area_ID,
FAR_Seq = warningNotices.FAR_Seq,
FAR_Year = warningNotices.FAR_Year,
DateTime_Entered = (DateTime)warningNotices.DateTime_Entered == null ? DateTime.MaxValue : (DateTime)warningNotices.DateTime_Entered,
Time_Entered = warningNotices.Time_Entered,
OrigDept = warningNotices.OrigDept,
Part_No = warningNotices.Part_No,
DateTime_Analyzed = (DateTime)warningNotices.DateTime_Analyzed == null ? DateTime.MaxValue : (DateTime)warningNotices.DateTime_Analyzed,
Analyzed_By = warningNotices.Analyzed_By,
MDR_Required = (Char)warningNotices.MDR_Required == null ? Char.MinValue : (Char)warningNotices.MDR_Required,
Resp_Dept = (from FARSympt in DBContext_Analyze.FAR_Symptoms
where FARSympt.FAR_Area_ID == warningNotices.FAR_Area_ID &&
FARSympt.FAR_Year == warningNotices.FAR_Year &&
FARSympt.FAR_Seq == warningNotices.FAR_Seq
select new { FARSympt.Resp_Dept}).FirstOrDefault().Resp_Dept,
Sympt_Desc = (from SymptomsCatalog in DBContext_Analyze.Symptoms
where SymptomsCatalog.symptom_ID == filter.Status_ID
select new {
SymptomsCatalog.Sympt_Desc
}).FirstOrDefault().Sympt_Desc,
Status_ID = warningNotices.Status.HasValue ? warningNotices.Status.Value : 0
};
Previously I had a "Distinc" in the subquery for the Resp_Dept field, but I removed it.
Any ideas? Thanks in advance for your comments =)
This is query I get from the SQL Server profiler:
exec sp_executesql N'SELECT [t0].[FAR_Seq], [t0].[FAR_Year],
(CASE
WHEN ([t0].[DateTime_Entered]) IS NULL THEN #p3
ELSE [t0].[DateTime_Entered]
END) AS [DateTime_Entered], [t0].[Time_Entered], [t0].[OrigDept], [t0].[Part_No],
(CASE
WHEN ([t0].[DateTime_Analyzed]) IS NULL THEN #p4
ELSE [t0].[DateTime_Analyzed]
END) AS [DateTime_Analyzed], [t0].[Analyzed_By],
(CASE
WHEN (UNICODE([t0].[MDR_Required])) IS NULL THEN #p5
ELSE CONVERT(NChar(1),[t0].[MDR_Required])
END) AS [MDR_Required], (
SELECT [t2].[Resp_Dept]
FROM (
**SELECT TOP (1)** [t1].[Resp_Dept]
FROM [dbo].[FAR_Symptoms] AS [t1]
WHERE (UNICODE([t1].[FAR_Area_ID]) = UNICODE([t0].[FAR_Area_ID])) AND ([t1].[FAR_Year] = [t0].[FAR_Year]) AND ([t1].[FAR_Seq]
= [t0].[FAR_Seq])
) AS [t2]
) AS [Resp_Dept], (
SELECT [t4].[Sympt_Desc]
FROM (
**SELECT TOP (1)** [t3].[Sympt_Desc]
FROM [dbo].[Symptoms] AS [t3]
WHERE [t3].[symptom_ID] = #p6
) AS [t4]
) AS [Sympt_Desc], [t0].[FAR_Area_ID],
(CASE
WHEN [t0].[Status] IS NOT NULL THEN [t0].[Status]
ELSE #p7
END) AS [Status_ID]
FROM [dbo].[FARs] AS [t0]
WHERE (UNICODE([t0].[FAR_Area_ID]) = #p0) AND ([t0].[FAR_Seq] = #p1) AND ([t0].[FAR_Year] = #p2)
ORDER BY [t0].[FAR_Seq]',N'#p0 int,#p1 int,#p2 varchar(2),#p3 datetime,#p4 datetime,#p5 nchar(1),#p6 int,#p7
int',#p0=76,#p1=7204,#p2='08',#p3=''9999-12-31 23:59:59:997'',#p4=''9999-12-31 23:59:59:997'',#p5=N' ',#p6=0,#p7=0
The only think that I see there that may not in SQL Server 2000 is the '()' in the "Select top..." but I'm not sure if that is what is causing the problem and, also, I don't know how that could be fixed =S
Thanks again =)
My Linq statement worked on SQL2008 but broke with the exact same error message on SQL2000.
Had a very similar Linq query that worked on both, the only real difference was that before calling .ToList() I called the .OrderBy() clause.
Ex:
var query = from t1 in table1 ...
...;
list = query.OrderBy(o => o.Field).ToList()
Tried the same OrderBy clause on the broken Linq query and it worked!
Has to be a bug?
Do you have the latest Service Pack for Visual Studio and the framework?
I just checked some of my Linq generated SQL and it is using "Top 1" correctly against a SQL Server 2000 database.
after several testing and review the DB, I found that the problem was a legacy table I was working on: that table has "text" type fields. Also, I had to remove some "Distinct" instructions in a nested query I had.
I found this and, after review that, I found that I have to change my queries and that the "Distinct" instruction does not work correctly. As a side note, let me say that the nested queries can also generate unexpected behavior.
So, the real lesson here is that if you need to deploy this against a SQL Server 2000, set an instance of the server and test against it!!! XD
Thanks a lot of your help =)