I develop a web app with C# and I want to display these SQL queries in Swagger. So I want to convert them to Linq.
First SQL query:
SELECT TOP 3 HastalikIsmi, COUNT(*)
FROM Hastaliklar
GROUP BY HastalikIsmi
ORDER BY COUNT(*) DESC
Second SQL query:
SELECT TcNo, Isim, Soyisim, Hastaliklar.HastalikIsmi
FROM Calisanlar, Hastaliklar
WHERE Calisanlar.CalisanId = Hastaliklar.CalisanId
AND HastalikIsmi IN (SELECT TOP 3 HastalikIsmi
FROM Hastaliklar
GROUP BY HastalikIsmi
ORDER BY COUNT(*) DESC)
I barely work with raw SQL since I use EF most of the time (so I might have misinterpreted some of the commands) and neither did I have the time to test the second query so I appologize in advance if there are any mistakes.
First SQL query translated:
var result = hastaliklars.GroupBy(hastaliklar => hastaliklar.HastalikIsmi)
.Select(hastaliklarGroup =>
(HastalikIsmi: hastaliklarGroup.Key, Count: hastaliklarGroup.Count()))
.OrderByDescending(hastalikIsmiWithCount => hastalikIsmiWithCount.Count)
.Take(3)
.ToList();
Second SQL query translated:
var result2 = calisanlars.Join(hastaliklars,
calisanlar => calisanlar.CalisanId,
hastaliklar => hastaliklar.CalisanId,
(calisanlar, hastaliklar) =>
(Calisanlar: calisanlar, Hastaliklar: hastaliklar))
.Where(calisanlarAndHastaliklar => hastaliklars
.GroupBy(hastaliklar => hastaliklar.HastalikIsmi)
.OrderByDescending(hastaliklarGroup => hastaliklarGroup.Count())
.Take(3)
.Any(hastaliklarGroup =>
calisanlarAndHastaliklar.Hastaliklar.HastalikIsmi == hastaliklarGroup.Key))
.ToList();
Note that if you're working with an IQueryable<T> instead of an IEnumerable<T> you might get a runtime exception stating that the LINQ expression could not be translated and will be evaluated locally. The reason why this might happen is because an IQueryable<T> is not run locally on C# but rather just contains instructions on what you're trying to do and translates that into an SQL query string. Therefore, specific code can not be translated into SQL which causes this exception.
The easiest way to fix this would be to run the method AsEnumerable() before the method which can not be translated to SQL which forces only all of the previous instructions to be run via SQL and then reads and works with the results locally from there on out. However, it does come at a performance cost, so that it would be slower than running the entire query via SQL. Therefore the optimal solution would be to adjust the code so that all of it could be translated into SQL if possible.
I don't fully remember what code IQueryable<T> supports and what not but as a good start; Here's a link to the microsoft docs that lists all of the LinQ functions it supports: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/supported-and-unsupported-linq-methods-linq-to-entities?redirectedfrom=MSDN
Related
I have to match records in SQL around a zip code with a min/max range
the challenge is that the data qualities are bad, some zipcodes are not numbers only
so I try to match "good zip codes" either by discarding bad ones or even keeping only digits
I dont know how to use Regex.Replace(..., #"[^\d]", "") instead of Regex.Match(..., #"\d") to fit in the query bellow
I get an error with the code bellow at runtime
I tried
Regex.IsMatch
SqlFunctions.IsNumeric
they all cause errors at runtime, here is the code :
var data = context.Leads.AsQueryable();
data = data.Include(p => p.Company).Include(p => p.Contact);
data = data.Where(p => Regex.IsMatch(p.Company.ZipCode, #"\d"));
data = data.Where(p => Convert.ToInt32(p.Company.ZipCode) >= range.Min);
data = data.Where(p => Convert.ToInt32(p.Company.ZipCode) <= range.Max);
here is the error :
System.InvalidOperationException: The LINQ expression 'DbSet<Lead>
.Join(
outer: DbSet<Company>,
inner: l => EF.Property<Nullable<int>>(l, "CompanyId"),
outerKeySelector: c => EF.Property<Nullable<int>>(c, "Id"),
innerKeySelector: (o, i) => new TransparentIdentifier<Lead, Company>(
Outer = o,
Inner = i
))
.Where(l => !(Regex.IsMatch(
input: l.Inner.ZipCode,
pattern: "\d")))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync().
I am not sure how to solve this. I really don't see how AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync() could help here
what do I do wrong ?
thanks for your help
Wnen you use querable list, Ef Core 5 is always trying to translate query to SQl, so you have to use code that SQL server could understand. If you want to use C# function you will have to download data to Server using ToList() or ToArray() at first and after this you can use any c# functions using downloaded data.
You can try something like this:
var data = context.Leads
.Include(p => p.Company)
.Include(p => p.Contact)
.Where(p =>
p.Company.Zipcode.All(char.IsDigit)
&& (Convert.ToInt32(p.Company.ZipCode) >= range.Min) //or >=1
&& ( Convert.ToInt32(p.Company.ZipCode) <= range.Max) ) // or <=99999
.ToArray();
I tried everything imaginable
all sorts of linq/ef trickeries, I even tried to define a DBFunction that was never found
once I had a running stored procedure written dirrectly in SQL, I ended up with a list, not with an IQueryable, so I was back to #1
finaly, I just created a new field in my table :
ZipCodeNum
which holds a filtered , converted version of the zipcode string
In my controller method for the the index view I have the following line.
#students_instance = Student.includes(:memo_tests => {:memo_target => :memo_level})
So for each Student I eager-load all necessary info.
Later on in a .map block, I call the .where() method on one of the relations as shown below.
#all_students = #students_instance.map do |student|
...
last_pass = student.memo_tests.where(:result => true).last.created_at.utc
difference_in_weeks = ((last_pass.to_i - current_date.to_i) / 1.week).round
...
end
This leads to a single SQL query for each student. And since I have over 300+ students, leads to very slow load times and over 300+ SQL queries.
Am I right in thinking that this is caused by the .where() method. I think this because I have checked everything else and these are the two lines that cause all of the queries.
More importantly, is there a better way to do this that reduces these queries to a single query?
The moment you ask where, the statement is translated to a query. Normally, the result should be sql-cached...
Anyway, in order to be sure, you can instead add programming logic to your statement. That way, you are not requesting a NEW sql statement.
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.sort.last
EDIT
I see the OP's question does not require sorting... So, leaving the sorting out:
last_pass = student.memo_tests.map {|m| m.created_at if m.result}.compact.last
compact is required to remove nil results from the array.
I am querying a MS SQL database using Linq and Entity Framework Code First. The requirement is to be able to run a WHERE SomeColumn LIKE '%sometext'clause against the table.
This, on the surface, is a simple requirement that could be accomplished using a simple Linq query like this:
var results = new List<MyTable>();
using(var context = new MyContext())
{
results = context.MyTableQueryable
.Where(x => x.SomeColumn.EndsWith("sometext"))
.ToList();
}
// use results
However, this was not effective in practice. The problem seems to be that the column SomeColumn is not varchar, rather it's a char(31). This means that if a string is saved in the column that is less than 31 characters then there will be spaces added on the end of the string to ensure a length of 31 characters, and that fouls up the .EndsWith() query.
I used SQL Profiler to lookup the exact sql that was generated from the .EndsWith() method. Here is what I found:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE N'%sometext'
So that is interesting. I'm not sure what the N means before '%sometext'. (I'll Google it later.) But I do know that if I take the same query and run it in SSMS without the N like this:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE '%sometext'
Then the query works fine. Is there a way to get Linq and Entity Framework to drop that N from the query?
Please try this...
.Where(x => x.SomeColumn.Trim().EndsWith("sometext"))
Just spoke to my colleague who had a similar issue, see if the following works for you:
[Column(TypeName = "varchar")]
public string SomeColumn
{
get;
set;
}
Apparently setting the type on the column mapping will force the query to recognise it as a VARCHAR, where a string is normally interpreted as an NVARCHAR.
I want to use Second level Cache for my query with eager loading(query is below wrote in 3 different ways, i use query cache). I have standard one to many association. I set entity cache for parent, child, and association between parent and class. And the 2nd level cache doesn't work because i got exceptions.
I wrote my query in 3 different way:
Criteria:
session.CreateCriteria<DictionaryMaster>().SetFetchMode("DictionaryItems", FetchMode.Eager)
.SetResultTransformer(new DistinctRootEntityResultTransformer())
.SetCacheable(true).SetCacheMode(CacheMode.Normal)
.List<DictionaryMaster>().ToList();
When I invoked this query i got exception "Unable to perform find[SQL: SQL not available]"
I think that the problem exists because I use DistinctRootEntityResultTransformer transform. I will start create my custom transform class and I hope that will work.
Query over:
session.QueryOver<DictionaryMaster>().Fetch(x => x.DictionaryItems).Eager
.TransformUsing(new DistinctRootEntityResultTransformer())
.Cacheable().CacheMode(CacheMode.Normal)
.List<DictionaryMaster>().ToList();
Exception is the same as in Criteria.
Linq:
session.Query<DictionaryMaster>().Fetch(x => x.DictionaryItems).Cacheable().CacheMode(CacheMode.Normal).ToList();
Here error depends from the version of nhibernate in 3.1 a got this error https://nhibernate.jira.com/browse/NH-2587?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel
but in 3.2 version i got this: https://nhibernate.jira.com/browse/NH-2856
Thanks in advance
sJHony I have found solution, but for me is more like workaround. Therefore if you know any other solution give me sign.
I utilize QueryOver without any transformation, the fault of this solution is that the query return elements equal amount of childs. Next I retrieve not multiplied list in memory using distinct.
This solution is ok, but when we add one more Fetch for query, then collection in object also be multiplied, that is why i modified collection type from IList(Bag) to ISet(Set).
Code looks like:
var queryCacheResult =
session.QueryOver<DictionaryMaster>()
.Fetch(x => x.DictionaryItems).Eager
.Cacheable().CacheMode(CacheMode.Normal)
.List<DictionaryMaster>().ToList();
return queryCacheResult.Distinct(new KeyEqualityComparer<DictionaryMaster>(x => x.Code)).ToList();
I simply want to include a row number against the returned results of my query.
I found the following post that describes what I am trying to achieve but gives me an exception
http://vaultofthoughts.net/LINQRowNumberColumn.aspx
"An expression tree may not contain an assignment operator"
In MS SQL I would just use the ROWNUMBER() function, I'm simply looking for the equivalent in LINQ.
Use AsEnumerable() to evaluate the final part of your query on the client, and in that final part add a counter column:
int rowNo = 0;
var results = (from data in db.Data
// Add any processing to be performed server side
select data)
.AsEnumerable()
.Select(d => new { Data = d, Count = ++rowNo });
I'm not sure whether LINQ to SQL supports it (but it propably will), but there's an overload to the Queryable.Select method that accepts an lambda with an indexer. You can write your query as follows:
db.Authors.Select((author, index) => new
{
Lp = index, Name = author.Name
});
UPDATE:
I ran a few tests, but unfortunately LINQ to SQL does not support this overload (both 3.5sp1 and 4.0). It throws a NotSupportedException with the message:
Unsupported overload used for query
operator 'Select'.
LINQ to SQL allows you to map a SQL function. While I've not tested this, I think this construct will work:
public partial class YourDataContext : DatContext
{
[Function(Name = "ROWNUMBER")]
public int RowNumber()
{
throw InvalidOperationException("Not called directly.");
}
}
And write a query as follows:
from author in db.Authors
select new { Lp = db.RowNumber(), Name = author.Name };