How to intercept and modify SQL query in Linq to SQL - sql

I was wondering if there is any way to intercept and modify the sql generated from linq to Sql before the query is sent off?
Basically, we have a record security layer, that given a query like 'select * from records' it will modify the query to be something like 'select * from records WHERE [somesecurityfilter]'
I am trying to find the best way to intercept and modify the sql before its executed by the linq to sql provider.

Ok, first to directly answer your question (but read on for words of caution ;)), there is a way, albeit a finicky one, to do what you want.
// IQueryable<Customer> L2S query definition, db is DataContext (AdventureWorks)
var cs = from c in db.Customers
select c;
// extract command and append your stuff
DbCommand dbc = db.GetCommand(cs);
dbc.CommandText += " WHERE MiddleName = 'M.'";
// modify command and execute letting data context map it to IEnumerable<T>
var result = db.ExecuteQuery<Customer>(dbc.CommandText, new object[] { });
Now, the caveats.
You have to know which query is generated so you would know how to modify it, this prolongs development.
It falls out of L2S framework and thus creates a possible gaping hole for sustainable development, if anyone modifies a Linq it will hurt.
If your Linq causes parameters (has a where or other extension causing a WHERE section to appear with constants) it complicates things, you'll have to extract and pass those parameters to ExecuteQuery
All in all, possible but very troublesome. That being said you should consider using .Where() extension as Yaakov suggested. If you want to centrally controll security on object level using this approach you can create an extension to handle it for you
static class MySecurityExtensions
{
public static IQueryable<Customer> ApplySecurity(this IQueryable<Customer> source)
{
return source.Where(x => x.MiddleName == "M.");
}
}
//...
// now apply it to any Customer query
var cs = (from c in db.Customers select c).ApplySecurity();
so if you modify ApplySecurity it will automatically be applied to all linq queries on Customer object.

If you want to intercept the SQL generated by L2S and fiddle with that, your best option is to create a wrapper classes for SqlConnection, SqlCommand, DbProviderFactory etc. Give a wrapped instance of SqlConnection to the L2S datacontext constructor overload that takes a db connection. In the wrapped connection you can replace the DbProviderFactory with your own custom DbProviderFactory-derived class that returns wrapped versions of SqlCommand etc.
E.g.:
//sample wrapped SqlConnection:
public class MySqlConnectionWrapper : SqlConnection
{
private SqlConnecction _sqlConn = null;
public MySqlConnectionWrapper(string connectString)
{
_sqlConn = new SqlConnection(connectString);
}
public override void Open()
{
_sqlConn.Open();
}
//TODO: override everything else and pass on to _sqlConn...
protected override DbProviderFactory DbProviderFactory
{
//todo: return wrapped provider factory...
}
}
When using:
using (SomeDataContext dc = new SomeDataContext(new MySqlConnectionWrapper("connect strng"))
{
var q = from x in dc.SomeTable select x;
//...etc...
}
That said, do you really want to go down that road? You'll need to be able to parse the SQL statements and queries generated by L2S in order to modify them properly. If you can instead modify the linq queries to append whatever you want to add to them, that is probably a better alternative.
Remember that Linq queries are composable, so you can add 'extras' in a separate method if you have something that you want to add to many queries.

first thing come to my mind is to modify the query and return the result in Non-LINQ format
//Get linq-query as datatable-schema
public DataTable ToDataTable(System.Data.Linq.DataContext ctx, object query)
{
if (query == null)
{
throw new ArgumentNullException("query");
}
IDbCommand cmd = ctx.GetCommand((IQueryable)query);
System.Data.SqlClient.SqlDataAdapter adapter = new System.Data.SqlClient.SqlDataAdapter();
adapter.SelectCommand = (System.Data.SqlClient.SqlCommand)cmd;
DataTable dt = new DataTable("sd");
try
{
cmd.Connection.Open();
adapter.FillSchema(dt, SchemaType.Source);
adapter.Fill(dt);
}
finally
{
cmd.Connection.Close();
}
return dt;
}
try to add your condition to the selectCommand and see if it helps.

Try setting up a view in the DB that applies the security filter to the records as needed, and then when retrieving records through L2S. This will ensure that the records that you need will not be returned.
Alternatively, add a .Where() to the query before it is submitted that will apply the security filter. This will allow you to apply the filter programmatically (in case it needs to change based on the scenario).

Related

Rewrite Hibernate Criteria with IN clause so can reuse same PreparedStatement with different number of IN clauses

I use the following Hibernate query alot when retrieving multiple records by their primary key
Criteria c = session
.createCriteria(Song.class)
.setLockMode(LockMode.NONE)
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY)
.add(Restrictions.in("recNo", ids));
List<Song> songs = c.list();
The problem is the number of ids can vary from 1 - 50, and every different number of ids requires a different PreparedStatement. That, combined with the fact that any particular prepared statement is tied to a particular database pool connection means that the opportunity to reuse a PreparedStatement is quite low.
Is there way I can rewrite this so that the same statement can be used with different number of in values, I think I read somewhere it could be done by using ANY instead but cannot find the reference.
This is called "in clause parameter padding" and can be activated with a hibernate property:
<property
name="hibernate.query.in_clause_parameter_padding"
value="true"
</property>
Read more about this topic here: https://vladmihalcea.com/improve-statement-caching-efficiency-in-clause-parameter-padding/
With some help I ended up getting a usual SQL connection from Hibernate, and then using standard SQL with ANY instead of IN. As far as I know using ANY means we only need a single prepared statement per connection so is better then using padded IN's. But because just using SQL not much use if you need to modify the data returned
public static List<SongDiff> getReadOnlySongDiffs(List<Integer> ids)
{
Connection connection = null;
try
{
connection = HibernateUtil.getSqlSession();
String SONGDIFFSELECT = "select * from SongDiff where recNo = ANY(?)";
PreparedStatement ps = connection.prepareStatement(SONGDIFFSELECT);
ps.setObject(1, ids.toArray(new Integer[ids.size()]));
ResultSet rs = ps.executeQuery();
List<SongDiff> songDiffs = new ArrayList<>(ids.size());
while(rs.next())
{
SongDiff sd = new SongDiff();
sd.setRecNo(rs.getInt("recNo"));
sd.setDiff(rs.getBytes("diff"));
songDiffs.add(sd);
}
return songDiffs;
}
catch (Exception e)
{
MainWindow.logger.log(Level.SEVERE, "Failed to get SongDiffsFromDb:" + e.getMessage(), e);
throw new RuntimeException(e);
}
finally
{
SessionUtil.close(connection);
}
}
public static Connection getSqlSession() throws SQLException {
if (factory == null || factory.isClosed()) {
createFactory();
}
return ((C3P0ConnectionProvider)factory.getSessionFactoryOptions().getServiceRegistry().getService(C3P0ConnectionProvider.class)).getConnection();
}
If you're still on an old version of Hibernate as suggested in the comments to Simon's answer here, as a workaround, you could use jOOQ's ParsingConnection to transform your SQL by applying the IN list padding feature transparently behind the scenes. You can just wrap your DataSource like this:
// Input DataSource ds1:
DSLContext ctx = DSL.using(ds1, dialect);
ctx.settings().setInListPadding(true);
// Use this DataSource for your code, instead:
DataSource ds2 = ctx.parsingDataSource();
I've written up a blog post to explain this more in detail here.
(Disclaimer: I work for the company behind jOOQ)

Apache Ignite : Ignite Repository query with "IN" clause, returns no records

I am using Apache Ignite as the back-end data store in a SpringBoot Application.
I have a requirement where I need to get all the entities whose name matches one of the names from a set of names.
Hence i am trying to get it implemented using a #Query configuration and a method named findAllByName(Iterable<String> names)as below:
Here on the Query, I am trying to use the 'IN' clause and want to pass an array of names as an input to the 'IN' clause.
#RepositoryConfig(cacheName = "Category")
public interface CategoryRepository extends IgniteRepository<Category, Long>
{
List<Category> findByName(String name);
#Query("SELECT * FROM Category WHERE name IN ( ? )")
Iterable<Category> findAllByName(Iterable<String> names); // this method always returns empty list .
}
In this the method findAllByName always returns empty list, even when ignite has Categories for which the name field matches the data passed in the query.
I am unable to figure out if there is a problem with the Syntax or the query of the method signature or the parameters.
Please try using String[] names instead for supplying parameters.
UPDATE: I have just checked the source, and we don't have tests for such scenario. It means that you're on uncharted territory even if it is somehow possible to get to work.
Otherwise looks unsupported currently.
I know your question is more specific to Spring Data Ignite feature. However, as an alternate, you can achieve it using the SqlQuery abstraction of Ignite.
You will form your query like this. I have pasted the sample below with custom sql function inSet that you will write. Also, the below tells how this is used in your sql.
IgniteCache<String, MyRecord> cache = this.ignite
.cache(this.environment.getProperty(Cache.CACHE_NAME));
String sql = "from “my-ignite-cache”.MyRecord WHERE
MyRecord.city=? AND inSet(?, MyRecord.flight)"
SqlQuery<String, MyRecord> sqlQuery = new SqlQuery<>(MyRecord.class,
sql);
sqlQuery.setArgs(MyCity, [Flight1, Flight2 ] );
QueryCursor<Entry<String, MyRecord>> resultCursor = cache.query(sqlQuery);
You can iterate the result cursor to do something meaningful from the extracted data.
resultCursor.forEach(e -> {
MyRecord record = e.getValue();
// do something with result
});
Below is the Ignite Custom Sql function which is used in the above Query - this will help in replicating the IN clause feature.
#QuerySqlFunction
public static boolean inSet(List<String> filterParamArgIds, String id) {
return filterParamArgIds.contains(id);
}
And finally, as a reference MyRecord referred above can be defined something like this.
public class MyRecord implements Serializable {
#QuerySqlField(name = "city", index = true)
private String city;
#QuerySqlField(name = "flight", index = true)
private String flight;
}

Using ASP.NET Core Web API WITHOUT Entity Framework

I need to build a Web API from ASP.NET Core without Entity Framework. It's an existing database that has some custom stored procedures and we do not want to use EF.
I searched this topic and can't find anything about it, is this even possible?
This is possible.
The first problem you will run into is getting the database connection string. You will want to import the configuration to do so. In a controller, it might look like this:
private readonly IConfiguration _configuration;
public WeatherForecastController(ILogger<WeatherForecastController> logger, IConfiguration configuration)
{
_logger = logger;
_configuration = configuration;
}
Add using System.Data and using System.Data.SqlClient (you'll need NuGet for SqlClient) as well as using Microsoft.Extensions.Configuration. With access to the database, you are writing code "old style", for example:
[HttpGet]
[Route("[controller]/movies")]
public IEnumerable<Movie> GetMovies()
{
List<Movie> movies = new List<Movie>();
string connString = ConfigurationExtensions.GetConnectionString(_configuration, "RazorPagesMovieContext");
SqlConnection conn = new SqlConnection(connString);
conn.Open();
SqlDataAdapter sda = new SqlDataAdapter("SELECT * FROM Movie", conn);
DataSet ds = new DataSet();
sda.Fill(ds);
DataTable dt = ds.Tables[0];
sda.Dispose();
foreach (DataRow dr in dt.Rows)
{
Movie m = new Movie
{
ID = (int)dr["ID"],
Title = dr["Title"].ToString(),
ReleaseDate = (DateTime)dr["ReleaseDate"],
Genre = dr["Genre"].ToString(),
Price = (decimal)dr["Price"],
Rating = dr["Rating"].ToString()
};
movies.Add(m);
}
conn.Close();
return movies.ToArray();
}
The connection string name is in appsettings.json.
"ConnectionStrings": {
"RazorPagesMovieContext": "Server=localhost;Database=Movies;Trusted_Connection=True;MultipleActiveResultSets=true"
}
Yes it is possible. Just implement the API by yourself. Or here is also sample for the identity scaffold, without EF.
https://markjohnson.io/articles/asp-net-core-identity-without-entity-framework/
Just used Dapper as our ORM in a project rather than EF.
https://dapper-tutorial.net/
It is similar to ADO.Net, but it has some additionally features that we leveraged and it was really clean to implement.
I realize this is an old question, but it came up in a search I ran so I figured I'd add to the answers given.
First, if the custom stored procedures are your concern, you can still run them using Entity Framework's .FromSql method (see here for reference: https://www.entityframeworktutorial.net/efcore/working-with-stored-procedure-in-ef-core.aspx)
The relevant info is found at the top of the page:
EF Core provides the following methods to execute a stored procedure:
1. DbSet<TEntity>.FromSql(<sqlcommand>)
2. DbContext.Database.ExecuteSqlCommand(<sqlcommand>)
If you are avoiding Entity Framework for other reasons, it's definitely possible to use any database connection method you want in ASP.NET Core. Just implement your database connection methods using whatever library is relevant to your database and set up your controller to return the data in whatever format you want. Most, if not all, of Microsoft's examples return Entity Framework entities, but you can easily return any data format you want.
As an example, this controller method returns a MemoryStream object after running a query against an MS SQL server (note, in most cases where you want data returned it's my understanding that it should be a "GET" method, not "POST" as is done here, but I needed to send and use information in the HttpPost body)
[HttpPost]
[Route("Query")]
public ActionResult<Stream> Query([FromBody]SqlDto content)
{
return Ok(_msSqlGenericService.Query(content.SqlCommand, content.SqlParameters));
}
Instead of a MemoryStream, you could return a generic DataTable or a List of any custom class you want. Note that you'll also need to determine how you are going to serialize/deserialize your data.

How to work around NHibernate caching?

I'm new to NHibernate and was assigned to a task where I have to change a value of an entity property and then compare if this new value (cached) is different from the actual value stored on the DB. However, every attempt to retrieve this value from the DB resulted in the cached value. As I said, I'm new to NHibernate, maybe this is something easy to do and obviously could be done with plain ADO.NET, but the client demands that we use NHibernate for every access to the DB. In order to make things clearer, those were my "successful" attempts (ie, no errors):
1
DetachedCriteria criteria = DetachedCriteria.For<User>()
.SetProjection(Projections.Distinct(Projections.Property(UserField.JobLoad)))
.Add(Expression.Eq(UserField.Id, userid));
return GetByDetachedCriteria(criteria)[0].Id; //this is the value I want
2
var JobLoadId = DetachedCriteria.For<User>()
.SetProjection(Projections.Distinct(Projections.Property(UserField.JobLoad)))
.Add(Expression.Eq(UserField.Id, userid));
ICriteria criteria = JobLoadId.GetExecutableCriteria(NHibernateSession);
var ids = criteria.List();
return ((JobLoad)ids[0]).Id;
Hope I made myself clear, sometimes is hard to explain a problem when even you don't quite understand the underlying framework.
Edit: Of course, this is a method body.
Edit 2: I found out that it doesn't work properly for the method call is inside a transaction context. If I remove the transaction, it works fine, but I need it to be in this context.
I do that opening a new stateless session for geting the actual object in the database:
User databaseuser;
using (IStatelessSession session = SessionFactory.OpenStatelessSession())
{
databaseuser = db.get<User>("id");
}
//do your checks
Within a session, NHibernate will return the same object from its Level-1 Cache (aka Identity Map). If you need to see the current value in the database, you can open a new session and load the object in that session.
I would do it like this:
public class MyObject : Entity
{
private readonly string myField;
public string MyProperty
{
get { return myField; }
set
{
if (value != myField)
{
myField = value;
DoWhateverYouNeedToDoWhenItIsChanged();
}
}
}
}
googles nhforge
http://nhibernate.info/doc/howto/various/finding-dirty-properties-in-nhibernate.html
This may be able to help you.

Execute a SQL stored procedure before every query generated by EntityFramework

I need to execute a SQL stored procedure every time before I query my ObjectContext. What I want to achieve is setting the CONTEXT_INFO to a value which will be later on used with most of my queries.
Has anyone done that? Is that possible?
[EDIT]
Currently I'm achieving this by opening the connection and executing the stored procedure in my ObjectContext constructor like this:
public partial class MyEntitiesContext
{
public MyEntitiesContext(int contextInfo) : this()
{
if (Connection.State != ConnectionState.Open)
{
Connection.Open(); // open connection if not already open
}
var connection = ((EntityConnection)Connection).StoreConnection;
using (var cmd = connection.CreateCommand())
{
// run stored procedure to set ContextInfo to contextInfo
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "[dbo].[SetContextInfo]";
cmd.Parameters.Add(new SqlParameter("#ci", _contextInfo));
cmd.ExecuteNonQuery();
}
// leave the connection open to reuse later
}
}
Then in my integration test:
[TestMethod]
public void TestMethod1()
{
using (var ctx = new MyEntitiesContext(1))
{
Assert.AreEqual(2, ctx.Roles.ToList().Count);
Assert.AreEqual(2, ctx.Users.ToList().Count);
}
}
But this requires me to leave the connection open - this is error prone since I will always need CONTEXT_INFO, and another developer might easily do:
[TestMethod]
public void TestMethod2()
{
using (var ctx = new MyEntitiesContext(1))
{
// do something here
// ... more here :)
ctx.Connection.Close(); // then out of the blue comes Close();
// do something here
Assert.AreEqual(2, ctx.Roles.ToList().Count);
Assert.AreEqual(2, ctx.Users.ToList().Count); // this fails since the where
// clause will be:
// WHERE ColumnX = CAST(CAST(CONTEXT_INFO() AS BINARY(4)) AS INT)
// and CONTEXT_INFO is empty - there are no users with ColumnX set to 0
// while there are 2 users with it set to 1 so this test should pass
}
}
The above means that I can write the code like in my test and everthing is green (YAY!) but then my colleague uses the code from TestMethod2 somewhere in his business logic and it's all f'd up - and nobody knows where and why since all tests are green :/
[EDIT2]
This blog post certainly does not answer my question but actually solves my problem. Maybe going with NHibernate will be better suited for my purpose :)
We have used this pattern.
But the way we did it was to call the stored procedure as the first opperation inside each db context.
Finally I found the answer. I can wrap the connection using the EFProvider wraper toolkit from EFProviderWrappers.
To do this I mostly have to derive from EFProviderWrapperConnection and override the DbConnection.Open() method. I already tried it with the Tracing provider and it works fine. Once I test it with my solution I will add more information.