I'm trying to sanitize data fields in order to prevent sql injections using ASP.NET and along the way have discovered Uri.EscapeDataString, which I believe is the solution(not sure). Here's my code, this is the execution page and Request[cur] is grabbing from an array "dictArr" which holds the names of my fields on the previous page.
foreach(string cur in dictArr) {
if(Request[cur] != "") {
dictionary[cur] = Request[cur].Trim();
}
index++;
}
To my understanding Uri.EscapeDataString makes the data nothing more than a string and is secure, no need to worry about a database wipe out. My question is, how do I use it? Am I doing it correctly below or is there another step I must take? Thanks
foreach(string cur in dictArr) {
if(Request[cur] != "") {
dictionary[cur] = Uri.EscapeDataString(Request[cur].Trim());
}
index++;
}
That's a bit different, you need to prevent attacks from the server point of view. If I was hacking I don't need to use your web site interface, I can POST whatever I like to your server.
That's a bigger concept but good to get across with the web.
There are a bazillion articles on the web about all that, e.g. http://en.wikipedia.org/wiki/SQL_injection
Start digging :)
Uri.EscapeDataString simply converts strings for safe transmission in a URL. Very much like Server.URLEncode, but with some differences.
I'm trying to sanitize data fields in order to prevent sql injections using ASP.NET and along the way have discovered Uri.EscapeDataString, which I believe is the solution(not sure)
No, this is not the correct way. Although encoding everything with EscapeDataString would encode inline SQL queries in a way that will avoid SQL injection, you will end up with URL encoded data in your database.
e.g. My name is O'Leary would be encoded as My name is O%27Leary
This would mean your application (and every other application now or in the future) will have to decode the data on every read. This is not standard and will add extra complexity to operations in future. For example, reading manipulating and writing data back to the database would have to be decoded, manipulated and then reencoded at the correct points.
The correct way to protect against SQL injection is to use parameterised queries.
String query =
"SELECT account_balance FROM user_data WHERE user_name = ?";
try {
OleDbCommand command = new OleDbCommand(query, connection);
command.Parameters.Add(new OleDbParameter("customerName", CustomerName Name.Text));
OleDbDataReader reader = command.ExecuteReader();
// …
} catch (OleDbException se) {
// error handling
}
This automatically protects against SQL injection as the parameters are treated as strongly typed values rather than part of the query structure.
Related
I used Entity Framework to create a prototype for a project and now that it's working I want to make the program ready for production.
I face many challenges with EF, the biggest one being the concurrency management (it's a financial software).
Given that it seems to have no way to handle pessimistic concurrency with EF, I have to switch to stored procs in SQL.
To be honest I'm a bit afraid of the workload that may represent.
I would like to know if anybody have been in the same situation before and what is the best strategy to convert a .net code using EF to raw SQL.
Edit:
I'm investigating CLR but it's not clear if pessimistic concurency can be manage with it. is it an option more interesting than TSQl in this case ? It would allow me to reuse part of my C# code and structure of function calling another functions, if I understand well.
I was there and the good news is you don't have to give up Entity Framework if you don't want to. The bad news is you have to update the database yourself. Which isn't as hard as it seems. I'm currently using EF 5 but plan to go to EF 6. I don't see why this still wouldn't work for EF 6.
First thing is in the constructor of the DbContext cast it to IObjectContextAdapter and get access to the ObjectContext. I make a property for this
public virtual ObjectContext ObjContext
{
get
{
return ((IObjectContextAdapter)this).ObjectContext;
}
}
Once you have that subscribe to the SavingChanges event - this isn't our exact code some things are copied out of other methods and redone. This just gives you an idea of what you need to do.
ObjContext.SavingChanges += SaveData;
private void SaveData(object sender, EventArgs e)
{
var context = sender as ObjectContext;
if (context != null)
{
context.DetectChanges();
var tsql = new StringBuilder();
var dbParams = new List<KeyValuePair<string, object>>();
var deletedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Deleted);
foreach (var delete in deletedEntites)
{
// Set state to unchanged - so entity framework will ignore
delete.ChangeState(EntityState.Unchanged);
// Method to generate tsql for deleting entities
DeleteData(delete, tsql, dbParams);
}
var addedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added);
foreach (var add in addedEntites)
{
// Set state to unchanged - so entity framework will ignore
add.ChangeState(EntityState.Unchanged);
// Method to generate tsql for added entities
AddData(add, tsql, dbParams);
}
var editedEntites = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified);
foreach (var edit in editedEntites)
{
// Method to generate tsql for updating entities
UpdateEditData(edit, tsql, dbParams);
// Set state to unchanged - so entity framework will ignore
edit.ChangeState(EntityState.Unchanged);
}
if (!tsql.ToString().IsEmpty())
{
var dbcommand = Database.Connection.CreateCommand();
dbcommand.CommandText = tsql.ToString();
foreach (var dbParameter in dbParams)
{
var dbparam = dbcommand.CreateParameter();
dbparam.ParameterName = dbParameter.Key;
dbparam.Value = dbParameter.Value;
dbcommand.Parameters.Add(dbparam);
}
var results = dbcommand.ExecuteNonQuery();
}
}
}
Why we set the entity to unmodified after the update because you can do
var changed properties = edit.GetModifiedProperties();
to get a list of all the changed properties. Since all the entities are now marked as unchanged EF will not send any updates to SQL.
You will also need to mess with the metadata to go from entity to table and property to fields. This isn't that hard to do but messing the metadata does take some time to learn. Something I still struggle with sometimes. I refactored all that out into an IMetaDataHelper interface where I pass it in the entity type and property name to get the table and field back - along with caching the result so I don't have to query metadata all the time.
At the end the tsql is a batch that has all the T-SQL how we want it with the locking hints and containing the transaction level. We also change numeric fields from just being set to nfield = 10 but to be nfield = nfield + 2 in the TSQL if the user updated them by 2 to avoid the concurrency issue as well.
What you wont get to is having SQL locked once someone starts to edit your entity but I don't see how you would get that with stored procedures as well.
All in all it took me about 2 solid days to get this all up and running for us.
I'm working on an web app that collects traffic information for websites that use my service. Think google analytics but far more visual. I'm using SQL Server 2012 for the backbone of my app and am considering using MongoDB as the data gathering analytic side of the site.
If I have 100 users with an average of 20,000 hits a month on their site, that's 2,000,000 records in a single collection that will be getting queried.
Should I use MongoDB to store this information (I'm new to it and new things are intimidating)?
Should I dynamically create new collections/tables for every new user?
Thanks!
With MongoDB the collection (aka sql table) can get quite big without much issue. That is largely what it is designed for. The Mongo is part HuMONGOus (pretty clever eh). This is a great use for mongodb which is great at storing point in time information.
Options :
1. New Collection for each Client
very easy to do I use a GetCollectionSafe Method for this
public class MongoStuff
private static MongoDatabase GetDatabase()
{
var databaseName = "dbName";
var connectionString = "connStr";
var client = new MongoClient(connectionString);
var server = client.GetServer();
return server.GetDatabase(databaseName);
}
public static MongoCollection<T> GetCollection<T>(string collectionName)
{
return GetDatabase().GetCollection<T>(collectionName);
}
public static MongoCollection<T> GetCollectionSafe<T>(string collectionName)
{
//var db = GetDatabase();
var db = GetDatabase();
if (!db.CollectionExists(collectionName)) {
db.CreateCollection(collectionName);
}
return db.GetCollection<T>(collectionName);
}
}
then you can call with :
var collection = MongoStuff.GetCollectionSafe<Record>("ClientName");
Running this script
static void Main(string[] args)
{
var times = new List<long>();
for (int i = 0; i < 1000; i++)
{
Stopwatch watch = new Stopwatch();
watch.Start();
MongoStuff.GetCollectionSafe<Person>(String.Format("Mark{0:000}", i));
watch.Stop();
Console.WriteLine(watch.ElapsedMilliseconds);
times.Add(watch.ElapsedMilliseconds);
}
Console.WriteLine(String.Format("Max : {0} \nMin : {1} \nAvg : {2}", times.Max(f=>f), times.Min(f=> f), times.Average(f=> f)));
Console.ReadKey();
}
Gave me (on my laptop)
Max : 180
Min : 1
Avg : 6.635
Benefits :
Ease of splitting data if one client needs to go on their own
Might match your brain map of the problem
Cons :
Almost impossible to do aggregate data over all collections
Hard to find collections in Management studios (like robomongo)
2. One Large Collection
Use one collection for it all access it this way
var coll = MongoStuff.GetCollection<Record>("Records");
Put an index on the table (the index will make reads orders of magnitude quicker)
coll.EnsureIndex(new IndexKeysBuilder().Ascending("ClientId"));
needs to only be run once (per collection, per index )
Benefits :
One Simple place to find data
Aggregate over all clients possible
More traditional Mongodb setup
Cons :
All Clients Data is intermingled
May not mentally map as well
Just as a reference the mongodb limits for sizes are here :
[http://docs.mongodb.org/manual/reference/limits/][1]
3. Store only aggregated data
If you are never intending to break down to an individual record just save the aggregates themselves.
Page Loads :
# Page Total Time Average Time
15 Default.html 1545 103
I will let someone else tackle the MongoDB side of your question as I don't feel I'm the best person to comment on it, I would point out that MongoDB is a very different animal and you'll lose a lot of the RI you enjoy in SQL.
In terms of SQL design I would not use a different schema for each customer approach. Your database schema and backups could grow uncontrollably, maintaining a dynamically growing schema will be a nightmare.
I would suggest one of two approaches:
Either you can create a new database for each customer:
This is more secure as users cannot access each other's data (just use different credentials) and users are easier to manage/migrate and separate.
However many hosting providers charge per database, it will cost more to run and maintain and should you wish to compare data across users it gets much more challenging.
Your second approach is to simply host all users in a single DB, your tables will grow large (although 2 million rows is not over the top for a well maintained SQL DB). You would simply use a UserID column to discriminate.
The emphasis will be on you to get the performance you need through proper indexing
Users' data will exist in the same system and there's no SQL defense against users accessing each other's data - your code will have to be good!
Afaik Entity Framework 6 doesn't support for batch insert/update/delete.
Is there anyway to make an batch update over an IQueryable object. As an example I have
var query = _db.People.Where(x=>x.Name.Contains(parameter));
an IQueryable (query) object and I want to get the generated sql. Then I hope I can create an update command with this select query like this
Update filteredPerson
Set filteredPerson.Status = 'Updated'
from (here it comes IQueryable Generated SQL :) ) as filteredPerson
over DbContext raw sql execution commands. BTW I don't need EF properties like change tracking and auto detecting. It is just a batch operation.
I know it is pretty risky but I am going to use it for a small piece of code.
Some other logics are appricated. If you know something better, I would like to hear it.
REASON: Why I want to do it this way, because I don't want to spoil the seperation of layers. And there is some validation and filtering comes into the queryable object from other layers. So it is hard to convert it to stored procedure. At the other hand it must be faster than other standard queries.
Again I know there is no support in Entity Framework 6 for batch operations. But other questions are bit outdated. That's another reason why I want to ask this again.
While I was writing the question, I was guessing how I am going to solve it. But I was looking for some more proper way of it. In the end, I know what am I doing and tried to be simple for my colleagues who looking to the same code after me. I know it has some risky usages but I let the exceptions to CLR to handle it. After this excuse :) , I wrote the code like this:
Let's say I have an IQueryable object which is generated with this way:
string parameter = "John";
AdventureWorks2012Entities _db = new AdventureWorks2012Entities();
var query = _db.People.AsQueryable();
//Some parameters added from different layers
query = query.Where(x => x.FirstName.Contains(parameter));
Then I want a batch update over this IQueryable object.
var sqlFrom = query.ToString(); //This is the query which becomes "from table"
var dbq = query.AsDbQuery().GetObjectQuery(); //This does some magic with reflection
var linqParams = dbq.Parameters
.Select(x => new System.Data.SqlClient.SqlParameter(x.Name, x.Value)).ToList();
linqParams.Add(new System.Data.SqlClient.SqlParameter("#ModDate", DateTime.Now));
var sqlBatchUpdate = #"Update filteredPerson Set ModifiedDate = #ModDate From (#FilteredPerson) as filteredPerson"
.Replace("#FilteredPerson", sqlFrom);
var affectedRows = _db.Database.ExecuteSqlCommand(sqlBatchUpdate, linqParams.ToArray());
That's it! Now I don't have to repeat same business logic in stored procedure again. And it is more faster than a foreach and SaveChanges combo.
So I ended up with this for very basic usage. As a fast solution It brings more problems no doubt! But I know I can easily wrap around it for new purposes. So It is up to programmer who wants to use it with more preferences.
Also the code which does the reflection and casting is below and I added a gist for full code:
public static ObjectQuery<T> GetObjectQuery<T>(this DbQuery<T> query)
{
var internalQueryField = query.GetType()
.GetFields(BindingFlags.NonPublic | BindingFlags.Instance)
.Where(f => f.Name.Equals("_internalQuery"))
.FirstOrDefault();
var internalQuery = internalQueryField.GetValue(query);
var objectQueryField = internalQuery.GetType()
.GetFields(BindingFlags.NonPublic | BindingFlags.Instance)
.Where(f => f.Name.Equals("_objectQuery"))
.FirstOrDefault();
var objectQuery = objectQueryField.GetValue(internalQuery) as ObjectQuery<T>;
return objectQuery;
}
Here is the Gist file. Hope It helps somebody out there.
I have saved certain MDX query and I run them using ADOMD.NET. I get CellSet back which I convert into dataset. All this is working fine. Now the DB team has changed the cube structure. They have updated the DimesnionName, Attribute Name etc. Some dimensions got renamed and some got deleted. Becuase of this I am unable to run my saved queries. I want to create a console application, which will take list of keys ([DimensionName].[AttributeName] or [DimensionName].[AttributeName].[MemeberName] format) and it will tell me following keys does not exists.
Please let me know if this is possible programatically. I dont want to check it manually.
Kindly share a link or code which will help me acheive this.
Thank you.
If you're using ADOMD already this should be no problem, just use the metadata queries:
http://msdn.microsoft.com/en-us/library/ms123485.aspx
Alternatively, AMO is nice http://msdn.microsoft.com/en-us/library/microsoft.analysisservices.aspx
I use it in SSIS for processing, you could easily use it in .Net to test existence of elements:
using Microsoft.AnalysisServices;
...
Server server = new Server();
server.Connect(cubeConnectionString);
Database database = server.Databases.FindByName(databaseName);
Cube cube = database.Cubes.FindByName(cubeName);
foreach (MeasureGroup measureGroup in cube.MeasureGroups)
{
foreach (Partition partition in measureGroup.Partitions)
{
...
}
}
foreach (CubeDimension cubeDimension in cube.Dimensions)
{
Dimension dimension = cubeDimension.Dimension;
var dimName = dimension.Name;
...
}
Finding the names in advance for all the elements you need is probably the hard part (And keeping it all up-to-date).
Would it not be easier to fire all the queries at the cube and try to trap the "no such thing" response?
Producing software for customers, mostly using MS SQL but some Oracle, a decision was made to plunge into Nhibernate (and C#).
The task is to delete efficiently e.g. 10 000 rows from 100 000 and still stay sticked to ORM.
I've tried named queries - link already,
IQuery sql = s.GetNamedQuery("native-delete-car").SetString(0, "Kirsten");
sql.ExecuteUpdate();
but the best I have ever found seems to be:
using (ITransaction tx = _session.BeginTransaction())
{
try
{
string cmd = "delete from Customer where Id < GetSomeId()";
var count = _session.CreateSQLQuery(cmd).ExecuteUpdate();
...
Since it may not get into dB to get all complete rows before deleting them.
My questions are:
If there is a better way for this kind of delete.
If there is a possibility to get the Where condition for Delete like this:
Having a select statement (using LinQ to NHibernate) => which will generate appropriate SQL for DB => we get that Where condition and use it for Delete.
If there is a better way for this kind of delete.
Yes, you could use HQL instead of SQL.
If there is a possibility to get the Where condition for Delete [using Expressions]:
No, AFAIK that's not implemented. Since NHibernate is an open source project, I encourage you to find out if anyone has proposed this, and/or discuss it on the mailing list.
Thanks for your quick reply. Now I've probably got the difference.
session.CreateSQLQuery(cmd).ExecuteUpdate();
must have cmd with Delete From DbTable. On the contrary the HQL way
session.CreateQuery(cmd).ExecuteUpdate();
needs cmd with Delete From MappedCollectionOfObjects.
In that case it possibly solves my other question as well.
There now is a better way with NHibernate 5.0:
var biggestId = GetSomeId();
session.Query<Customer>()
.Where(c => c.Id < biggestId)
.Delete();
Documentation:
//
// Summary:
// Delete all entities selected by the specified query. The delete operation is
// performed in the database without reading the entities out of it.
//
// Parameters:
// source:
// The query matching the entities to delete.
//
// Type parameters:
// TSource:
// The type of the elements of source.
//
// Returns:
// The number of deleted entities.
public static int Delete<TSource>(this IQueryable<TSource> source);