LINQ & Lambda Expressions equivalent of SQL In - sql

Is there a lambda equivalent of IN? I will like to select all the funds with ids either 4, 5 or 6. One way of writing it is:
List fundHistoricalPrices = lionContext.FundHistoricalPrices.Where(fhp => fhp.Fund.FundId == 5 || fhp.Fund.FundId == 6 || fhp.Fund.FundId == 7).ToList();
However, that quickly becomes unmanageable if I need it to match say 100 different fundIds. Can I do something like:
List
fundHistoricalPrices =
lionContext.FundHistoricalPrices.Where(fhp
=> fhp.Fund.FundId in(5,6,7)).ToList();

It's somewhere along these lines, but I can't quite agree with the approach you have taken. But this will do if you really want to do this:
.Where(fhp => new List<int>{5,6,7}.Contains( fhp.Fund.FundId )).ToList();
You may want to construct the List of ids before your LINQ query...

You can use the Contains() method on a collection to get the equivalent to in.
var fundIds = new [] { 5, 6, 7 };
var fundHistoricalPrices = lionContext.FundHistoricalPrices.Where(fhp => fundIds.Contains(fhp.Fund.FundId)).ToList();

You could write an extension method like this :
public static bool In<T>(this T source, params T[] list)
{
if(null==source) throw new ArgumentNullException("source");
return list.Contains(source);
}
Then :
List fundHistoricalPrices = lionContext.FundHistoricalPrices.Where(fhp => fhp.Fund.FundId.In(5,6,7)).ToList();

No, the only similar operator i'm aware of is the Contains() function.
ANother was is to construct your query dynamically by using the predicate builder out of the LINQkit: http://www.albahari.com/nutshell/predicatebuilder.aspx
Example
int[] fundIds = new int[] { 5,6,7};
var predicate = PredicateBuilder.False<FundHistoricalPrice>();
foreach (int id in fundIds)
{
int tmp = id;
predicate = predicate.Or (fhp => fhp.Fund.FundId == tmp);
}
var query = lionContext.FundHistoricalPrices.Where (predicate);

Related

Get and manipulate data from elasticsearch

I am new to elasticsearch so I will need some help. Unfortunately, I didnt found the answer in other topics here on SO.
I have some .net core application which I inherited and now there is a need to implement some changes.
I already have a method of getting data from elasticsearch, but after getting them, I am not sure how to change it and use it in application.
To be precise, I need to parse first and last name and to remove special characters, specific serbian latin letters like "šđžčć" etc... I already have a method for this parsing written but not sure how to call it...
So, my question is can I and how can I do this?
What I have now is the following:
var result = await _elasticClient.SearchAsync<CachedUserEntity>(
s =>
s.Index(_aliasName)
.Query(q => andQuery));
CachedUserEntity, among others, contains property about FirstName and LastName.
Inside results.Documents, I am getting the data about FirstName and LastName from elasticsearch, but I am not sure how to access it in order to update it via aformentioned NameParser ...
Sorry if the question is too easy, not to say stupid :)
I wont use updateByQuery here, for some reasons. I would scroll on documents (i use matchAll on my exemple, you obviously need to replace it with your query), or, if you dont know how to identify documents to update, only update usefull documents in UpdateManyWithIndex/UpdateManyPartial function.
For performance, we have to update severals documents at once, so we use bulk/updateMany function.
You can use both solution, the classic update, or the second (partial update) with an object containing the targeteds fields.
On server sides, both solutions will have the same cost / performance.
var searchResponse = Client.Search<CachedUserEntity>(s => s
.Query(q => q
MatchAll()
)
.Scroll("10s")
);
while (searchResponse.Documents.Any())
{
List<CachedUserEntity> NewSearchResponse = RemoveChar(searchResponse);
UpdateManyWithIndex<CachedUserEntity>(NewSearchResponse, _aliasName);
searchResponse = Client.Scroll<Project>("2h", searchResponse.ScrollId);
}
public void UpdateManyWithIndex<C>(List<C> obj, string index) where C : class {
var bulkResponse = Client.Bulk(b => b
.Index(index).Refresh(Elasticsearch.Net.Refresh.WaitFor) // explicitly provide index name
.UpdateMany<C>(obj, (bu, d) => bu.Doc(d)));
}
Or, using partial update object
Note: in this case Indix is already set on my client (add .index if needed)
var searchResponse = Client.Search<CachedUserEntity>(s => s
.Query(q => q
MatchAll()
)
.Scroll("2h")
);
while (searchResponse.Documents.Any())
{
List<object> listPocoPartialObj = GetPocoPartialObjList(searchResponse);
UpdateManyPartial(listPocoPartialObj);
searchResponse = Client.Scroll<Project>("2h", searchResponse.ScrollId);
}
private List<object> GetPocoPartialObjList(List<CachedUserEntity> cachedList) {
List<object> listPoco = new List<object>();
//note if you dont have cachedList.Id, take a look at result.source, comments if needed
foreach (var eltCached in cachedList) {
listPoco.Add( new object() { Id = cachedList.Id, FirstName = YOURFIELDWITHOUTSPECIALCHAR, LastName = YOURSECONDFIELDWITHOUTSPECIALCHAR});
}
return listPoco;
}
public bool UpdateManyPartial(List<object> partialObj)
{
var bulkResponse = Client.Bulk(b => b
.Refresh(Elasticsearch.Net.Refresh.WaitFor)
.UpdateMany(partialObj, (bu, d) => bu.Doc(d))
);
if (!bulkResponse.IsValid)
{
GetErrorMsgs(bulkResponse);
}
return (bulkResponse?.IsValid == true);
}

EF could not translate expression to sql

It looks like EF is not able to translate the express in the following code, this is the call
Counter lastCounter = unitOfWork.CounterRepository.FindLast(x => x.Div == counter.Div, x => x.Div);
this is the method
public Counter FindLast(Expression<Func<Counter, bool>> predicate, params Expression<Func<Counter, object>>[] includedProperties)
{
IQueryable<Counter> set = context.Set<Counter>().Where(predicate);
foreach (var includeProperty in includedProperties)
{
set = set.Include(includeProperty);
}
return set.Last();
}
Any idea what could be the problem?
It's quite simple, really: Entity Framework just does not support Last(). The reason for this is that in SQL, you also cannot select the last element (i.e. you have SELECT TOP but don't have SELECT BOTTOM).
See https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/supported-and-unsupported-linq-methods-linq-to-entities

Does Dapper support the like operator?

Using Dapper-dot-net...
The following yields no results in the data object:
var data = conn.Query(#"
select top 25
Term as Label,
Type,
ID
from SearchTerms
WHERE Term like '%#T%'",
new { T = (string)term });
However, when I just use a regular String Format like:
string QueryString = String.Format("select top 25 Term as Label, Type, ID from SearchTerms WHERE Term like '%{0}%'", term);
var data = conn.Query(QueryString);
I get 25 rows back in the collection. Is Dapper not correctly parsing the end of the parameter #T?
Try:
term = "whateverterm";
var encodeForLike = term => term.Replace("[", "[[]").Replace("%", "[%]");
string term = "%" + encodeForLike(term) + "%";
var data = conn.Query(#"
select top 25
Term as Label,
Type,
ID
from SearchTerms
WHERE Term like #term",
new { term });
There is nothing special about like operators, you never want your params inside string literals, they will not work, instead they will be interpreted as a string.
note
The hard-coded example in your second snippet is strongly discouraged, besides being a huge problem with sql injection, it can cause dapper to leak.
caveat
Any like match that is leading with a wildcard is not SARGable, which means it is slow and will require an index scan.
Yes it does. This simple solution has worked for me everytime:
db.Query<Remitente>("SELECT *
FROM Remitentes
WHERE Nombre LIKE #n", new { n = "%" + nombre + "%" })
.ToList();
Best way to use this to add concat function in query as it save in sql injecting as well, but concat function is only support above than sql 2012
string query = "SELECT * from country WHERE Name LIKE CONCAT('%',#name,'%');"
var results = connection.query<country>(query, new {name});
The answer from Sam wasn't working for me so after some testing I came up with using the SQLite CONCAT equivalent which seems to work:
string sql = "SELECT * FROM myTable WHERE Name LIKE '%' || #NAME || '%'";
var data = IEnumerable data = conn.Query(sql, new { NAME = Name });
Just to digress on Sam's answer, here is how I created two helper methods to make searches a bit easier using the LIKE operator.
First, creating a method for generating a parameterized query, this method uses dynamic: , but creating a strongly typed generic method should be more desired in many cases where you want static typing instead of dynamic.
public static dynamic ParameterizedQuery(this IDbConnection connection, string sql, Dictionary<string, object> parametersDictionary)
{
if (string.IsNullOrEmpty(sql))
{
return null;
}
string missingParameters = string.Empty;
foreach (var item in parametersDictionary)
{
if (!sql.Contains(item.Key))
{
missingParameters += $"Missing parameter: {item.Key}";
}
}
if (!string.IsNullOrEmpty(missingParameters))
{
throw new ArgumentException($"Parameterized query failed. {missingParameters}");
}
var parameters = new DynamicParameters(parametersDictionary);
return connection.Query(sql, parameters);
}
Then adding a method to create a Like search term that will work with Dapper.
public static string Like(string searchTerm)
{
if (string.IsNullOrEmpty(searchTerm))
{
return null;
}
Func<string, string> encodeForLike = searchTerm => searchTerm.Replace("[", "[[]").Replace("%", "[%]");
return $"%{encodeForLike(searchTerm)}%";
}
Example usage:
var sql = $"select * from products where ProductName like #ProdName";
var herringsInNorthwindDb = connection.ParameterizedQuery(sql, new Dictionary<string, object> { { "#ProdName", Like("sild") } });
foreach (var herring in herringsInNorthwindDb)
{
Console.WriteLine($"{herring.ProductName}");
}
And we get our sample data from Northwind DB:
I like this approach, since we get helper extension methods to do repetitive work.
My solution simple to this problem :
parameter.Add("#nomeCliente", dfNomeCliPesquisa.Text.ToUpper());
query = "SELECT * FROM cadastrocliente WHERE upper(nome) LIKE " + "'%" + dfNomeCliPesquisa.Text.ToUpper() + "%'";

NHibernate - Linq query using COUNT(DISTINCT)

I'm trying to get a paged query to work properly using LINQ and NHibernate. On a single table, this works perfect, but when more than one table is joined, it's causing me havoc. Here is what I have so far.
public virtual PagedList<Provider> GetPagedProviders(int startIndex, int count, System.Linq.Expressions.Expression<Func<Record, bool>> predicate) {
var firstResult = startIndex == 1 ? 0 : (startIndex - 1) * count;
var query = (from p in session.Query<Record>()
.Where(predicate)
select p.Provider);
var rowCount = query.Select(x => x.Id).Distinct().Count();
var pageOfItems = query.Distinct().Skip(firstResult).Take(count).ToList<Provider>();
return new PagedList<Provider>(pageOfItems, startIndex, count, rowCount);
}
There are a couple issues I'm having with this. First off, the rowCount variable is pulling back a COUNT(*) on a join that has one to many. So my count is way off of what is should be, I'm expecting 129, but getting back over 1K.
The second issue I'm happening is with the results. If I'm on the first page (firstResult = 0), then I'm getting correct results. However, if firstResult is something other than 0, it produces completely different SQL. Below is the SQL generated by both scenarios, I've trimmed out some of the fat to make it a little more readable.
--firstResult = 0
select distinct TOP (30) provider1_.Id as Id12_, provider1_.TaxID as TaxID12_,
provider1_.Facility_Name as Facility5_12_, provider1_.Last_name as Last6_12_,
provider1_.First_name as First7_12_, provider1_.MI as MI12_
from PPORecords record0_ left outer join PPOProviders provider1_ on record0_.Provider_id=provider1_.Id,
PPOProviders provider2_ where record0_.Provider_id=provider2_.Id
and provider2_.TaxID='000000000'
--firstResult = 30
SELECT TOP (30) Id12_, TaxID12_, Facility5_12_, Last6_12_, First7_12_, MI12_,
FROM (select distinct provider1_.Id as Id12_, provider1_.TaxID as TaxID12_,
provider1_.Facility_Name as Facility5_12_,
provider1_.Last_name as Last6_12_,
provider1_.First_name as First7_12_,
provider1_.MI as MI12_,
ROW_NUMBER() OVER(ORDER BY CURRENT_TIMESTAMP) as __hibernate_sort_row
from PPORecords record0_ left outer join PPOProviders provider1_ on record0_.Provider_id=provider1_.Id,
PPOProviders provider2_
where record0_.Provider_id=provider2_.Id and provider2_.TaxID='000000000') as query
WHERE query.__hibernate_sort_row > 30
ORDER BY query.__hibernate_sort_row
The problem with the second query is the "distinct" keyword is not present on the outer query, only the inner query. Any ideas how to correct this?
Thanks for any suggestions!
[UPDATE]
This is the code that is building the the Linq query predicate.
private Expression<Func<Record, bool>> ParseQueryExpression(string Query) {
Expression<Func<Record, bool>> mExpression = x => true;
string[] splitQuery = Query.Split('|');
foreach (string query in splitQuery) {
if (string.IsNullOrEmpty(query))
continue;
int valStartIndex = query.IndexOf('(');
string variable = query.Substring(0, valStartIndex);
string value = query.Substring(valStartIndex + 1, query.IndexOf(')') - valStartIndex - 1);
switch (variable) {
case "tax":
mExpression = x => x.Provider.TaxID == value;
break;
case "net":
mExpression = Combine<Record>(mExpression, x => x.Network.Id == int.Parse(value));
break;
case "con":
mExpression = Combine<Record>(mExpression, x => x.Contract.Id == int.Parse(value));
break;
case "eff":
mExpression = Combine<Record>(mExpression, x => x.Effective_Date >= DateTime.Parse(value));
break;
case "trm":
mExpression = Combine<Record>(mExpression, x => x.Term_Date <= DateTime.Parse(value));
break;
case "pid":
mExpression = Combine<Record>(mExpression, x => x.Provider.Id == long.Parse(value));
break;
case "rid":
mExpression = Combine<Record>(mExpression, x => x.Rate.Id == int.Parse(value));
break;
}
}
return mExpression;
}
It seems as if the Linq provider has some "unexpected" behaviour there. I could reproduce the issue with the rowCount that you are describing and also encountered some issues when trying to fix it with a subquery.
If I understand your intentions correctly you basically want to have a paged list of Providers whose Records match certain criteria.
In that case I would suggest using a subquery. However, I tried implementing the subquery with the Query() method but it did not work. So I tried the QueryOver() method which worked flawlessly. In your case the desired queries would look like this:
Provider pAlias = null;
var query = session.QueryOver<Provider>(() => pAlias).WithSubquery
.WhereExists(QueryOver.Of<Record>()
.Where(predicate)
.And(r => r.Provider.Id == pAlias.Id)
.Select(r => r.Provider));
var rowCount = query.RowCount();
var pageOfItems = query.Skip(firstResult).Take(count).List<Provider>();
That way you do not have to struggle with the Distinct(). And as much as I personally like and use NHibernate.Linq, I suppose, if it does not work in a given situation, one should use something else that works.
Edit: Splitting the query into smaller units.
// the method ParseQueryExpression() does not need to be modified, the Expression should work like that
System.Linq.Expressions.Expression<Func<Record, bool>> predicate = x => x.Provider.TaxID == "000000000";
// Query to evaluate predicate
IQueryable<Record> queryId = session.Query<Record>().Where(predicate).Select(r => r.Provider);
// extracting the IDs to use in a subquery
List<int> idList = queryId.Select(p => p.Id).Distinct().ToList();
// total count of distinct Providers
int rowCount = idList.Count;
// new Query on Provider, using the distinct Ids
var query = session.Query<Provider>().Where(p => idList.Contains(p.Id));
// the List<Provider> to display on the page
var pageOfItems = query.Skip(firstResult).Take(count).ToList();
The problem here is that you have multiple queries and therefore multiple roundtrips to the db. Another problem apperas when you examine the generated sql. The subquery idList.Contains(p.Id) will generate an in-clause with a lot of parameters.
As you can see, these are NHibernate.Linq queries. That is because the new QueryOver feature is not a true Linq provider and does not work well with dynamic Linq expressions.
You could get around that limitation by using Detached QueryOvers. The drawback here is that you would have to modify your ParseQueryExpression() somehow, e.g. like that:
private QueryOver<Record> ParseQueryExpression(string Query)
{
Record rAlias = null;
var detachedQueryOver = QueryOver.Of<Record>(() => rAlias)
.JoinQueryOver(r => r.Provider)
.Where(() => rAlias.Provider.TaxID == "000000000")
.Select(x => x.Id);
// modify your method to match the return value
return detachedQueryOver;
}
// in your main method
var detachedQueryOver = QueryOver.Of<Record>()
.WithSubquery
.WhereProperty(r => r.Id
.In(this.ParseQueryExpression(...));
var queryOverList = session.QueryOver<Provider>()
.WithSubquery
.WhereProperty(x => x.Id)
.In(detachedQueryOver.Select(r => r.Provider.Id));
int rowCount = queryOverList.RowCount();
var pageOfItems = queryOverList.Skip(firstResult).Take(count).List();
Well, I hope you are not too confused by that.

LINQ: Count number of true booleans in multiple columns

I'm using LINQ to SQL to speed up delivery of a project, which it's really helping with. However I'm struggling with a few things I'm used to doing with manual SQL.
I have a LINQ collection containing three columns, each containing a boolean value representing whether an e-mail, mobile or address is availble.
I want to write a LINQ query to give me an count of trues for each column, so how many rows in the e-mail column are set to true (and the same for the other two columns)
If you need a single object containing the results:
var result = new {
HasEmailCount = list.Count(x => x.HasEmail),
HasMobileCount = list.Count(x => x.HasMobile),
HasAddressCount = list.Count(x => x.HasAddress)
};
Or using the aggregate function:
class Result
{
public int HasEmail;
public int HasAddress;
public int HasMobile;
}
var x = data.Aggregate(
new Result(),
(res, next) => {
res.HasEmail += (next.HasEmail ? 0 : 1);
res.HasAddress += (next.HasAddress ? 0 : 1);
res.HasMobile += (next.HasMobile ? 0 : 1);
return res;
}
);
x is of Type Result and contains the aggregated information. This can also be used for more compelx aggregations.
var mobileCount = myTable.Count(user => user.MobileAvailable);
And so on for the other counts.
You can do it like so:
var emailCount = yourDataContext.YourTable.Count(r => r.HasEmail);
etc.