MS SQL Query in C# - poor performance - sql

I calculate outstanding customers balance in C# Winforms. The code below works, but it's slow. Is there any way to improve its performance?
public DataTable GetOutStandingCustomers()
{
decimal Tot = 0;
DataTable table = new DataTable();
SqlConnection con = null;
try
{
table.Columns.Add("Code", typeof(Int32));
table.Columns.Add("Name", typeof(string));
table.Columns.Add("City", typeof(string));
table.Columns.Add("Tot", typeof(decimal));
string constr = ConfigHelper.GetConnectionString();
string query = "SELECT Code, Name,City FROM Chart WHERE LEFT(CODE,3)='401' AND Code > 401001 ";
string query0 = " SELECT(SELECT ISNULL( SUM(SalSum.Grand),'0' ) FROM SalSum WHERE SalSum.Code = #Code ) +( SELECT ISNULL(SUM(Journals.Amount),'0' ) FROM Journals WHERE Journals.DrCode = #Code ) -( SELECT ISNULL(SUM(RSalSum.Grand),'0' ) FROM RSalSum WHERE RSalSum.Code = #Code ) -( SELECT ISNULL(SUM(Journals.Amount),'0' ) FROM Journals WHERE Journals.CrCode = #Code )+(SELECT ISNULL(SUM(Chart.Debit),'0' ) FROM Chart WHERE Chart.Code = #Code) - (SELECT ISNULL(SUM(Chart.Credit), '0') FROM Chart WHERE Chart.Code = #Code)";
Person per = new Person();
con = new SqlConnection(constr);
SqlCommand com = new SqlCommand(query, con);
SqlCommand com0 = new SqlCommand(query0, con);
con.Open();
SqlDataReader r = com.ExecuteReader();
if (r.HasRows)
{
while (r.Read())
{
per.Name = Convert.ToString(r["Name"]);
per.City = Convert.ToString(r["City"]);
per.Code = Convert.ToString(r["Code"]);
com0.Parameters.Clear();
com0.Parameters.Add("#Code", SqlDbType.Int).Value = per.Code;
Tot = Convert.ToDecimal(com0.ExecuteScalar());
if (Tot != 0)
{
table.Rows.Add(per.Code, per.Name, per.City, Tot);
}
}
}
r.Close();
con.Close();
return table;
}
catch (Exception)
{
throw new Exception();
}
}

The performance problem is due to you retrieve all data from the server and filter data in the client using the complex computed expression that sum from seven tables:
if (Tot != 0)
{
table.Rows.Add(per.Code, per.Name, per.City, Tot);
}
This represent overhead over network plus you manually add the result to the datatable row by row.
The provided solution do filter in the server based on the computed expression using the CROSS APPLY
and auto create the datatable directly from the DataReader.
The benefit of CROSS APPLY is all columns are feasible to the main sql query, so you can filter on ToT column, filtering is done in the server (not the client).
public void SelctChart()
{
string sql2 = #"
select c.Code, c.Name,c.City ,oo.T
from chart c
cross apply
( select c.code,
(
(select ISNULL( SUM(SalSum.Grand),0 ) FROM SalSum WHERE SalSum.Code = c.code )
+( select ISNULL(SUM(j.Amount),0 ) FROM [dbo].[Jornals] j WHERE j.DrCode = c.code)
-( SELECT ISNULL(SUM(RSalSum.Grand),'0' ) FROM RSalSum WHERE RSalSum.Code = c.Code )
-( SELECT ISNULL(SUM(j.Amount),0 ) FROM [dbo].[Jornals] j WHERE j.CrCode = c.code )
+(SELECT ISNULL(SUM( c0.Debit),0 ) FROM [dbo].Chart c0 WHERE c0.Code = c.code)
- (SELECT ISNULL(SUM(c1.Credit), 0) FROM [dbo].Chart c1 WHERE c1.Code = c.code)
)T
) oo
where
oo.T >0
and LEFT(c.CODE,3)='401' AND c.Code > 401001
";
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(sql2, connection);
//in case you pass #code as a a parameter
//command.Parameters.Add("#code", SqlDbType.Int);
//command.Parameters["#code"].Value = code;
try
{
connection.Open();
var reader = command.ExecuteReader();
while (!reader.IsClosed)
{
DataTable dt = new DataTable();
// Autoload datatable
dt.Load(reader);
Console.WriteLine(dt.Rows.Count);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
You can modify the method and pass code as a parameter

In this case it seems you're looping through and performing multiple queries using multiple Codes, you're also querying Chart twice. In this case you'd want to use a LEFT JOIN from Chart to your other tables.
ON Chart.Code = Salsum.Code
ON Chart.Code = Journal.Code
for example.
You will have to look at GROUP BY as well because you're aggregating some table columns by using SUM.
You may also need to make sure that Code is indexed on the tables you're querying. As long as Code is often queried like this and comparatively rarely Updated or Inserted to, then indexing the Code column on these tables is probably appropriate.
Left Joins : https://technet.microsoft.com/en-us/library/ms187518(v=sql.105).aspx
Indexing: https://technet.microsoft.com/en-us/library/jj835095(v=sql.110).aspx
Sorry I wrote a book on you here, but optimization often leads to a long answer (especially with SQL).
tldr;
Use a LEFT JOIN, grouping by Code
Index the Code columns

Related

SQL Injection on Views

We are using 3-Tier Architecture in ASP.Net.
There are 3 Layers
Presentation
Business
Data Access
The Data Access Layer contains the GetData and ExecuteQuery etc function.
What I want to know is that, that want to call the View directly from the Presentation Layer. Is there any chance of SQL injection in calling a view from front-end without using stored procedure?
Presentation Layer (C#)
protected void btnView_Click(object sender, EventArgs e)
{
DL obj = new DL();
DataTable tb = new DataTable();
string query = "select * from ViewTeacher where FID = " + txtName.Text;
tb = obj.GetData(query);
}
DBAccess
public DataTable GetData(string query)
{
DataTable datatable = new DataTable();
SqlCommand cmd = new SqlCommand();
cmd.Connection = con;
cmd.CommandText = query;
try
{
if (cmd.Connection.State != ConnectionState.Open)
{
cmd.Connection.Open();
}
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(datatable);
}
}
catch (Exception ex)
{
throw new ArgumentException(ex.Message);
}
return datatable;
}
How are you "calling a view"? If you're running an ad-hoc query of:
SELECT <columns> FROM View WHERE ColumnX = 'Y'
and if that query is being constructed using (potentially) hostile input then yes, of course that can be subject to SQL injection - the whole point of injection is that the attacker can change the nature of the query:
SELECT <columns> FROM View WHERE ColumnX = 'Z'
UNION ALL
SELECT name,0,0,0,0 FROM INFORMATION_SCHEMA.TABLES --'
The attacker isn't limited to just the objects that are present in the original query.
The untrustworthy input in the two above queries was:
Y
and
Z'
UNION ALL
SELECT name,0,0,0,0 FROM INFORMATION_SCHEMA.TABLES --
As you are writing the query as follows that takes value from a textbox, 100% there is posibility for sql injection.
string query = "select * from ViewTeacher where FID = " + txtName.Text;
There should be no chance of SQL Injection while calling a view from front end, as views don't take parameters.
reference :
can we pass parameter to a view in sql?

How to Join two tables showing all records Where Table A is Not In Table B

I have a email marketing web application. I want to show which email contacts in (Table B) are not showing up in EmailContacts_Campaign (Table A). In addition, I want to filter table A by the CampaignId field. When I run the below code I get 0 records, yet I know there are a couple of thousand records there. Can anyone tell me where I am messing up?
SELECT * FROM TableA
LEFT JOIN TableB
ON TableA.EmailContactId = TableB.EmailContactId
WHERE TableA.CampaignId = 1
AND TableB.EmailContactId IS NULL
ORDER BY TableB.EmailContactId DESC
I want to show all email contacts in the EmailContact Table that are not showing up in the EmailContactCampaign table. Here is the actual code:
public List<EmailContact> GetNotAssignedContactsForCampaign(int campaignId)
{
string sqlCommand = "SELECT * FROM EmailContactCampaign LEFT JOIN EmailContact";
sqlCommand += " ON EmailContactCampaign.EmailContact_EmailContactId = EmailContact.EmailContactId";
sqlCommand += " WHERE EmailContactCampaign.EmailContact_EmailContactId = " + campaignId.ToString() AND EmailContact.EmailContactId IS NULL ;
sqlCommand += " ORDER BY EmailContact.EmailContactId DESC";
var emailContacts = new List<EmailContact>();
string CS = db.Database.Connection.ConnectionString;
using (SqlConnection con = new SqlConnection(CS))
{
con.Open();
SqlCommand cmd = new SqlCommand(sqlCommand, con);
//Create sql datareader
using (SqlDataReader sqlDataReader = cmd.ExecuteReader())
{
while (sqlDataReader.Read())
{
var emailContact = new EmailContact();
emailContact.Assigned = ((bool)sqlDataReader["Assigned"]);
emailContact.Cell1 = _crypto.DecryptAndSanitize(sqlDataReader["Cell1"] as string);
emailContact.Cell2 = _crypto.DecryptAndSanitize(sqlDataReader["Cell2"] as string);
emailContact.City = _crypto.DecryptAndSanitize(sqlDataReader["City"] as string);
emailContact.Company = _crypto.DecryptAndSanitize(sqlDataReader["Company"] as string);
emailContact.EmailAddress = _crypto.DecryptAndSanitize(sqlDataReader["EmailAddress"] as string);
emailContact.EmailContactId = (int)sqlDataReader["EmailContactId"];
emailContact.FullName = _crypto.DecryptAndSanitize(sqlDataReader["FullName"] as string);
emailContact.Hold = (bool)sqlDataReader["Hold"];
emailContact.Phone1 = _crypto.DecryptAndSanitize(sqlDataReader["Phone1"] as string);
emailContact.Phone2 = _crypto.DecryptAndSanitize(sqlDataReader["Phone2"] as string);
emailContact.State = _crypto.DecryptAndSanitize(sqlDataReader["State"] as string);
emailContact.Status = (Status)sqlDataReader["Status"];
emailContact.Zip = _crypto.DecryptAndSanitize(sqlDataReader["Zip"] as string);
emailContacts.Add(emailContact);
}
}
return (emailContacts);
}
}
Have you tried this?
SELECT * FROM tableB WHERE EmailContactId NOT IN (SELECT EmailContactId FROM tableA)
i think you got 0 probably because of this AND TableB.EmailContactId IS NULL
Please try this one
SELECT * FROM TableA
LEFT JOIN TableB
ON TableA.EmailContactId = TableB.EmailContactId
WHERE TableA.CampaignId = 1
ORDER BY TableB.EmailContactId DESC
I'm sorry my question was not clear enough. Did some digging and found the answer on another post. Sorry but I accidentally closed it and can't find it again. Anyway, here is my implementation of it.
SELECT * FROM EmailContact
WHERE NOT EXISTS
(SELECT * FROM EmailContactCampaign WHERE EmailContactCampaign.EmailContact_EmailContactId = EmailContact.EmailContactId AND EmailContactCampaign.Campaign_CampaignId = 1)
If i understood your question correctly, you are looking for B's that are not in A. But your query will return A's that are not in B. Turn it aroung (tableB left join tableA where a... is NULL)
Your problem was that you had it the wrong way around: your query would return all contacts from EmailContactCampaign that were not in EmailContact.
The correct solution for your problem would look like this:
SELECT * FROM EmailContact
WHERE EmailContactId NOT IN (
SELECT EmailContact_EmailContactId FROM EmailContactCampaign
WHERE Campaign_CampaignId = ?
)
ORDER BY EmailContact.EmailContactId DESC

SQL statement where clause by two dates

I am trying to write a Sql statement which filter the records based on two date variables entered by users. Here is my Sql statement:
public List<DistributionPacking> filterByDeliveryDate()
{
List<DistributionPacking> dateList = new List<DistributionPacking>();
using (var connection = new SqlConnection(FoodBankDB.connectionString)) // get your connection string from the other class here
{
SqlCommand command = new SqlCommand("SELECT d.id, d.packingDate, d.deliveryDate, b.name FROM dbo.Distributions d " +
"INNER JOIN dbo.Beneficiaries b ON d.beneficiary = b.id WHERE d.deliveryDate BETWEEN #fromDate AND #toDate", connection);
command.Parameters.AddWithValue("#fromDate", startDate);
command.Parameters.AddWithValue("#toDate", endDate);
connection.Open();
using (var dr = command.ExecuteReader())
{
while (dr.Read())
{
string id = dr["id"].ToString();
DateTime packingDate = DateTime.Parse(dr["packingDate"].ToString());
DateTime deliveryDate = DateTime.Parse(dr["deliveryDate"].ToString());
string name = dr["name"].ToString();
dateList.Add(new DistributionPacking(id, packingDate, deliveryDate, name));
}
}
}
However, it told me that Conversion failed when converting date and/or time from character string although my data type for packingDate and deliveryDate is DateTime. I wonder why is it so.
Thanks in advance.
Try something like this ...
SELECT d.id, d.packingDate, d.deliveryDate, b.name
FROM dbo.Distributions d INNER JOIN dbo.Beneficiaries b
ON d.beneficiary = b.id
WHERE d.deliveryDate
BETWEEN CAST(#fromDate AS DATETIME) AND CAST(#toDate AS DATETIME)

IDataReader Issue in .Net 1.1 and .Net 4.0 for Sybase DB

I have a sybase DB which fetches results of a query properly as below...
select
S.ipoInternalID,
clientAccount,
clientPrice,
clientAccountType,
interestOnLoan =
CASE WHEN useHIBOR = 1 then
ROUND(financingAmount * (fixedRate + spreadRate) *
I.noOfDaysForInterest/365/100,2)
ELSE
ROUND(financingAmount * (I.fundingRate+ spreadRate) *
I.noOfDaysForInterest/365/100,2) END,
useHIBORSTR =
CASE WHEN useHIBOR = 1 then
"LOCK-IN RATE + SPREAD"
ELSE
"COST OF FUNDING + SPREAD" END,
from subscription S, iPO I, allocation A
where
S.ipoInternalID = #ipoInternalID and
I.ipoInternalID = #ipoInternalID and
A.ipoInternalID = #ipoInternalID and
S.ccassID *= A.ccassID
order by S.ccassID
Notice the way interestOnLoan field is calculated above.
Now when I run this query in SQL Advantage tool, it runs fine and gives me calculated values for interestOnLoan. When I run this query using .Net 1.1 API that loads this query via OleDB it runs fine...
myCommand.CommandText = myQuery;
myAdapter.SelectCommand = myCommand;
int i = myAdapter.Fill(resultSet);
My resultset fills ok.
But when I execute the above code in .net 4.0, the resultset errors out as
"Value was either too large or too small for a Decimal."
The value it has issues with is the interestOnLoan because I also executed the command via IDataReader as below...
using (var dr = myCommand.ExecuteReader())
{
resultSet.Tables.Add(ConvertDataReaderToTableManually(dr));
}
private static DataTable ConvertDataReaderToTableManually(IDataReader dr) {
var dt = new DataTable();
var dtSchema = dr.GetSchemaTable();
var listCols = new List<DataColumn>();
if (dtSchema != null) {
foreach (DataRow drow in dtSchema.Rows) {
var columnName = Convert.ToString(drow["ColumnName"]);
var t = (Type) (drow["DataType"]);
var column = new DataColumn(columnName, t);
column.Unique = (bool) drow["IsUnique"];
column.AllowDBNull = (bool) drow["AllowDBNull"];
column.AutoIncrement = (bool) drow["IsAutoIncrement"];
listCols.Add(column);
dt.Columns.Add(column);
}
}
// Read rows from DataReader and populate the DataTable
int j = 0;
while (dr.Read()) {
j++;
var dataRow = dt.NewRow();
for (int i = 0; i < listCols.Count; i++) {
try {
dataRow[((DataColumn)listCols[i])] = dr[i];
} catch (Exception ex1) { }
}
dt.Rows.Add(dataRow);
}
return dt;
}
Here it errors out at the dataRow[((DataColumn)listCols[i])] = dr[i] where it has issues reading from dr[i];
When observed the ith column is nothing but interestOnLoan.
So somehow .Net 4.0 is not able to read this value. It can read other decimal values correctly such as clientPrice.
Why could this be happening....
Also I wanted to ask is there any way I can load the values in the DataReader as Double (instead of Decimal) by default?
I didnt get the reason why .NET 4.0 had issues ith the above query but when I changed the query as below it worked in both (.Net 1.1 and 4.0)
select
S.ipoInternalID,
clientAccount,
clientPrice,
clientAccountType,
interestOnLoan = ROUND(
(CASE WHEN useHIBOR = 1 THEN
((financingAmount*(fixedRate + spreadRate) * .noOfDaysForInterest)/365.0)
ELSE
((financingAmount*(I.fundingRate+spreadRate)*I.noOfDaysForInterest)/365.0)
END) / 100.0, 2),
useHIBORSTR =
CASE WHEN useHIBOR = 1 then
"LOCK-IN RATE + SPREAD"
ELSE
"COST OF FUNDING + SPREAD" END,
from subscription S, iPO I , allocation A
where
S.ipoInternalID = #ipoInternalID and
I.ipoInternalID = #ipoInternalID and
A.ipoInternalID = #ipoInternalID and
S.ccassID *= A.ccassID
order by S.ccassID

Tree View Query

I created a query to get all addresses(child) with in municipality(parent) but it is coming back empty/ Any help is appreciated.
drop table #tmp1
go
WITH Emp_CTE AS (
SELECT tablesysID, MunicNo, StreetName
FROM table1
UNION ALL
SELECT e.tablesysID,e.MunicNo,e.StreetName
FROM table1 e
INNER JOIN Emp_CTE ecte ON ecte.MunicNo = e.MunicNo
)
SELECT * into #tmp1
FROM Emp_CTE
select * from #tmp1
it will be used in asp.net tree view control.
Thanks Andomar for your answer, which is correct but I just wanted to share how I solved this problem as follows:
1: make the query as stored procedure that returns xml data type as response like:
create PROCEDURE [dbo].[sp_someproc]
#XmlResponse xml output
AS
BEGIN
--
-- Insert statements for procedure here
set #XmlResponse=(SELECT DISTINCT
table1.MunicNo + ' ' + table1.StreetName + ' ' + table1.City AS firstRow, table1.MunicNo, table1.StreetName, table1.City,
table1.XPOS, table1.YPOS, table1.RollID, table2.Asset_ID, table2.Feature_ID, table2.FeatureName + ',' + table2.Feature_ID + ' ' + table2.[DESC] AS secondRow,
table2.FeatureName, table2.xxxID, table2.[DESC],table3.WONOs, table3.WONOs + ', ' + table3.AssetType + ', ' +table3.Feature_ID AS workONumber
FROM table4 INNER JOIN
table3 ON table4.xxxxID = table3.xxxxID INNER JOIN
table2 ON table4.Asset_ID = table2.xxx_ID INNER JOIN
table1 ON table2.StreetName = table1.StreetName AND table3.MunicNo = table1.MunicNo
for xml auto,root('xml'))
END
select #XmlResponse
2: Aspx page code:
<telerik:RadTreeView ID="rtrvxxxxx" runat="server" >
<DataBindings>
<telerik:RadTreeNodeBinding DataMember="table1" TextField="firstRow" ValueField="firstRow" />
<telerik:RadTreeNodeBinding DataMember="table2" TextField="secondRow" />
<telerik:RadTreeNodeBinding DataMember="table3" TextField="workONumber" />
</DataBindings>
</telerik:RadTreeView>
3: C# Code:
private void loadXmlDocument()
{
try
{
#region Load and Bind xml to treeview
XmlDataSource xDS = new XmlDataSource();
XmlDocument xmlDoc = new XmlDocument();
xmlDoc = callingdatalayerclass.list_XML();
xDS.Data = xmlDoc.InnerXml;
xDS.XPath = "/xml/table1";
xDS.EnableCaching = false;
//bind to treeview
rtrvxxxx.DataSource = xDS;
rtrvxxxx.DataBind();
#endregion
}
catch (Exception ex)
{
ex.Message()}
}
}
4: Data access Layer Code:
public static XmlDocument List_XML()
{
XmlDocument xmlDoc = new XmlDocument();
SqlConnection SQLConn = new SqlConnection();
SQLConn.ConnectionString = someclass.someotherclass.GetConnectionString();
try
{
SQLConn.Open();
SqlCommand custCMD = new SqlCommand("sp_someproc", SQLConn);
custCMD.CommandType = CommandType.StoredProcedure;
custCMD.Parameters.Add("#XmlResponse", SqlDbType.Xml).Direction = ParameterDirection.Output;
custCMD.ExecuteNonQuery();
if (custCMD.Parameters["#XmlResponse"].Value != null)
{
string xml = custCMD.Parameters["#XmlResponse"].Value.ToString();
xmlDoc.LoadXml(xml);
}
return xmlDoc;
}
catch (Exception exGEN)
{
throw exGEN;
}
finally
{
SQLConn.Close();
}
}
Notes: if you see query has rows those are first layer of treeview and 2ndrow is inner row of 1st row or can call it a child of parentnode, 3rd row is inner row or child row of 2nd row, so if 1st row has some records it will output that, if 2nd row has some reocrds it will ouput that. if you see the databining section in treeview datamember is table name and
textfiled is firstrow, value field could be some other coulmn. Have not worked with asp.net orginal treeview so don't know how that works this is for telerik treeview control. I also found that the speed is way faster if data is slected as xml. "DS.XPath = "/xml/table1";" <= this code is selecting xml element or root element and then with in that element the first element which by the data is sorted(first row record's table)
Have a look at your recursive condition:
INNER JOIN Emp_CTE ecte ON ecte.MunicNo = e.MunicNo
This recurses into the same municipality. (Except when MinicNo is null.)
You haven't posted enough information to find the proper condition, but it might look like:
INNER JOIN Emp_CTE ecte ON ecte.tablesysID = e.MunicNo