How to change sql generated by linq-to-entities? - sql

I am querying a MS SQL database using Linq and Entity Framework Code First. The requirement is to be able to run a WHERE SomeColumn LIKE '%sometext'clause against the table.
This, on the surface, is a simple requirement that could be accomplished using a simple Linq query like this:
var results = new List<MyTable>();
using(var context = new MyContext())
{
results = context.MyTableQueryable
.Where(x => x.SomeColumn.EndsWith("sometext"))
.ToList();
}
// use results
However, this was not effective in practice. The problem seems to be that the column SomeColumn is not varchar, rather it's a char(31). This means that if a string is saved in the column that is less than 31 characters then there will be spaces added on the end of the string to ensure a length of 31 characters, and that fouls up the .EndsWith() query.
I used SQL Profiler to lookup the exact sql that was generated from the .EndsWith() method. Here is what I found:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE N'%sometext'
So that is interesting. I'm not sure what the N means before '%sometext'. (I'll Google it later.) But I do know that if I take the same query and run it in SSMS without the N like this:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE '%sometext'
Then the query works fine. Is there a way to get Linq and Entity Framework to drop that N from the query?

Please try this...
.Where(x => x.SomeColumn.Trim().EndsWith("sometext"))

Just spoke to my colleague who had a similar issue, see if the following works for you:
[Column(TypeName = "varchar")]
public string SomeColumn
{
get;
set;
}
Apparently setting the type on the column mapping will force the query to recognise it as a VARCHAR, where a string is normally interpreted as an NVARCHAR.

Related

How to convert these SQL queries to linq?

I develop a web app with C# and I want to display these SQL queries in Swagger. So I want to convert them to Linq.
First SQL query:
SELECT TOP 3 HastalikIsmi, COUNT(*)
FROM Hastaliklar
GROUP BY HastalikIsmi
ORDER BY COUNT(*) DESC
Second SQL query:
SELECT TcNo, Isim, Soyisim, Hastaliklar.HastalikIsmi
FROM Calisanlar, Hastaliklar
WHERE Calisanlar.CalisanId = Hastaliklar.CalisanId
AND HastalikIsmi IN (SELECT TOP 3 HastalikIsmi
FROM Hastaliklar
GROUP BY HastalikIsmi
ORDER BY COUNT(*) DESC)
I barely work with raw SQL since I use EF most of the time (so I might have misinterpreted some of the commands) and neither did I have the time to test the second query so I appologize in advance if there are any mistakes.
First SQL query translated:
var result = hastaliklars.GroupBy(hastaliklar => hastaliklar.HastalikIsmi)
.Select(hastaliklarGroup =>
(HastalikIsmi: hastaliklarGroup.Key, Count: hastaliklarGroup.Count()))
.OrderByDescending(hastalikIsmiWithCount => hastalikIsmiWithCount.Count)
.Take(3)
.ToList();
Second SQL query translated:
var result2 = calisanlars.Join(hastaliklars,
calisanlar => calisanlar.CalisanId,
hastaliklar => hastaliklar.CalisanId,
(calisanlar, hastaliklar) =>
(Calisanlar: calisanlar, Hastaliklar: hastaliklar))
.Where(calisanlarAndHastaliklar => hastaliklars
.GroupBy(hastaliklar => hastaliklar.HastalikIsmi)
.OrderByDescending(hastaliklarGroup => hastaliklarGroup.Count())
.Take(3)
.Any(hastaliklarGroup =>
calisanlarAndHastaliklar.Hastaliklar.HastalikIsmi == hastaliklarGroup.Key))
.ToList();
Note that if you're working with an IQueryable<T> instead of an IEnumerable<T> you might get a runtime exception stating that the LINQ expression could not be translated and will be evaluated locally. The reason why this might happen is because an IQueryable<T> is not run locally on C# but rather just contains instructions on what you're trying to do and translates that into an SQL query string. Therefore, specific code can not be translated into SQL which causes this exception.
The easiest way to fix this would be to run the method AsEnumerable() before the method which can not be translated to SQL which forces only all of the previous instructions to be run via SQL and then reads and works with the results locally from there on out. However, it does come at a performance cost, so that it would be slower than running the entire query via SQL. Therefore the optimal solution would be to adjust the code so that all of it could be translated into SQL if possible.
I don't fully remember what code IQueryable<T> supports and what not but as a good start; Here's a link to the microsoft docs that lists all of the LinQ functions it supports: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/supported-and-unsupported-linq-methods-linq-to-entities?redirectedfrom=MSDN

GORM many2many preload error

Currently using GORM to connect to two databases: POSTGRES AND sqlite (using a code switch to choose which one to use). I have a 2 database tables defined in my schema that look like this:
type TableClient struct {
Model
Synchronised bool
FacilityID string `gorm:"primary_key"`
Age int
ClientSexID int
MaritalStatusID int
SpecificNeeds []TableOptionList`gorm:"many2many:options_specific_needs"`
}
type TableOptionList struct {
ID int `gorm:"primary_key"`
Name string
Value string
Text string
SortKey int
}
Previously, I would preload related table with code like this:
var dbClient TableClient
Db.Where("facility_id = ? AND client_id = ? AND id = ?;", URLFacilityID, URLClientID, URLIncidentID).
Preload("ClientSex").
Preload("MaritalStatus").
Preload("CareTakerRelationShip").
Preload("HighestLevelOfEducation").
Preload("Occupation").
Preload("SpecificNeeds").
First(&dbClient)
Now that lookup fails with a syntax error and when I look at the SQL generated, it shows the following SQL is generated:
SELECT * FROM "table_option_lists" INNER JOIN "options_specific_needs" ON "options_specific_needs"."table_option_list_id" = "table_option_lists"."id" WHERE (("options_specific_needs"."table_client_id","options_specific_needs"."table_client_facility_id") IN (('one','LS034')))
Pasting that into the sqlite console also fails with the same error: (near ",": syntax error)
The crux of your issue is in your WHERE clause, you need to use "OR", not ",".
This is likely an issue with GORM and you could open a GitHub issue with them for this. You can get around the issue using db.Raw() from GORM. In general, I prefer to avoid ORMs. Part of the reason is it is often easier to just build an SQL query by hand than try to figure out the strange way to do it in the ORM (for things beyond basic CRUD).

How to use LINQ to Entities to make a left join using a static value

I've got a few tables, Deployment, Deployment_Report and Workflow. In the event that the deployment is being reviewed they join together so you can see all details in the report. If a revision is going out, the new workflow doesn't exist yet new workflow is going into place so I'd like the values to return null as the revision doesn't exist yet.
Complications aside, this is a sample of the SQL that I'd like to have run:
DECLARE #WorkflowID int
SET #WorkflowID = 399 -- Set to -1 if new
SELECT *
FROM Deployment d
LEFT JOIN Deployment_Report r
ON d.FSJ_Deployment_ID = r.FSJ_Deployment_ID
AND r.Workflow_ID = #WorkflowID
WHERE d.FSJ_Deployment_ID = 339
The above in SQL works great and returns the full record if viewing an active workflow, or the left side of the record with empty fields for revision details which haven't been supplied in the event that a new report is being generated.
Using various samples around S.O. I've produced some Entity to SQL based on a few multiple on statements but I feel like I'm missing something fundamental to make this work:
int Workflow_ID = 399 // or -1 if new, just like the above example
from d in context.Deployments
join r in context.Deployment_Reports.DefaultIfEmpty()
on
new { d.Deployment_ID, Workflow_ID }
equals
new { r.Deployment_ID, r.Workflow_ID }
where d.FSJ_Deployment_ID == fsj_deployment_id
select new
{
...
}
Is the SQL query above possible to create using LINQ to Entities without employing Entity SQL? This is the first time I've needed to create such a join since it's very confusing to look at but in the report it's the only way to do it right since it should only return one record at all times.
The workflow ID is a value passed in to the call to retrieve the data source so in the outgoing query it would be considered a static value (for lack of better terminology on my part)
First of all don't kill yourself on learning the intricacies of EF as there are a LOT of things to learn about it. Unfortunately our deadlines don't like the learning curve!
Here's examples to learn over time:
http://msdn.microsoft.com/en-us/library/bb397895.aspx
In the mean time I've found this very nice workaround using EF for this kind of thing:
var query = "SELECT * Deployment d JOIN Deployment_Report r d.FSJ_Deployment_ID = r.Workflow_ID = #WorkflowID d.FSJ_Deployment_ID = 339"
var parm = new SqlParameter(parameterName="WorkFlowID" value = myvalue);
using (var db = new MyEntities()){
db.Database.SqlQuery<MyReturnType>(query, parm.ToArray());
}
All you have to do is create a model for what you want SQL to return and it will fill in all the values you want. The values you are after are all the fields that are returned by the "Select *"...
There's even a really cool way to get EF to help you. First find the table with the most fields, and get EF to generated the model for you. Then you can write another class that inherits from that class adding in the other fields you want. SQL is able to find all fields added regardless of class hierarchy. It makes your job simple.
Warning, make sure your filed names in the class are exactly the same (case sensitive) as those in the database. The goal is to make a super class model that contains all the fields of all the join activity. SQL just knows how to put them into that resultant class giving you strong typing ability and even more important use-ability with LINQ
You can even use dataannotations in the Super Class Model for displaying other names you prefer to the User, this is a super nice way to keep the table field names but show the user something more user friendly.

SQL Injection: is this secure?

I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.

SQL Select Like Keywords in Any Order

I am building a Search function for a shopping cart site, which queries a SQL Server database. When the user enters "Hula Hoops" in the search box, I want results for all records containing both "Hula" and "Hoop", in any order. Furthermore, I need to search multiple columns (i.e. ProductName, Description, ShortName, MaufacturerName, etc.)
All of these product names should be returned, when searching for "Hula hoop":
Hula hoop
Hoop Hula
The Hoopity of xxhula sticks
(Bonus points if these can be ordered by relevance!)
It sounds like you're really looking for full-text search, especially since you want to weight the words.
In order to use LIKE, you'll have to use multiple expressions (one per word, per column), which means dynamic SQL. I don't know which language you're using, so I can't provide an example, but you'll have to produce a statement that's like this:
For "Hula Hoops":
where (ProductName like '%hula%' or ProductName like '%hoops%')
and (Description like '%hula%' or Description like '%hoops%')
and (ShortName like '%hula%' or ShortName like '%hoops%')
etc.
Unfortunately, that's really the only way to do it. Using Full Text Search would allow you to reduce your criteria to one per column, but you'll still have to specify the columns explicitly.
Since you're using SQL Server, I'm going to hazard a guess that this is a C# question. You'd have to do something like this (assuming you're constructing the SqlCommand or DbCommand object yourself; if you're using an ORM, all bets are off and you probably wouldn't be asking this anyway):
SqlCommand command = new SqlCommand();
int paramCount = 0;
string searchTerms = "Hula Hoops";
string commandPrefix = #"select *
from Products";
StringBuilder whereBuilder = new StringBuilder();
foreach(string term in searchTerms.Split(' '))
{
if(whereBuilder.Length == 0)
{
whereBuilder.Append(" where ");
}
else
{
whereBuilder.Append(" and ");
}
paramCount++;
SqlParameter param = new SqlParameter(string.Format("param{0}",paramCount), "%" + term + "%");
command.Parameters.Add(param);
whereBuilder.AppendFormat("(ProductName like #param{0} or Description like #param{0} or ShortName like #param{0})",paramCount);
}
command.CommandText = commandPrefix + whereBuilder.ToString();
SQL Server Full Text Search should help you out. You will basically create indexes on the columns you want to search. in the where clause of your query you will use the CONTAINS operator and pass it your search input.
you can start HERE or HERE to learn more
You might want to check out SOLR too - if you're going to be doing this type of searching. Super cool.
http://lucene.apache.org/solr/