Time Out when Executing a SQL Query. Any Solution? - sql
We use a google script to get data from a SQL database into a spreadsheet. There is a time-driven trigger for the function.
Every time I run the function, there is a time out.
Exception: The query has timed out
When I run the function a second time, it usually works fine. Does anybody has a solution for that?
The address and password of the Database is not shown here.
function onOpen() {
var spreadsheet = SpreadsheetApp.getActive();
var menuItems = [
{name: 'JTL Import', functionName: 'readData'}
];
spreadsheet.addMenu('JTL-Import', menuItems);
}
// Replace the variables in this block with real values.
var address = 'XXXX'; //ex. '10.1.1.1:1433'
var user = 'TAXDOO';
var userPwd = 'XXX';
var db = 'eazybusiness';
var dbUrl = 'jdbc:sqlserver://' + address + ';databaseName=' + db;
function StockUnitsSold30days() {
var conn = Jdbc.getConnection(dbUrl, user, userPwd);
var stmt = conn.createStatement();
var results = stmt.executeQuery("SELECT a.cArtNr SKU, aaI18n.cwertvarchar [Cut_ID], CONVERT(FLOAT, ISNULL(wmsLager.fbestand, 0)) [Stock_WMS], CONVERT(FLOAT, ISNULL(StreckenLager.fbestand, 0)) [Stock_Streckenlager], CONVERT(FLOAT, ISNULL(fba.fanzahlfba, 0)) [Stock_FBA], CONVERT(FLOAT, ISNULL(PendingLager.fbestand, 0)) [Stock_Pending], CONVERT(FLOAT, ISNULL(WmsAusgang.Anzahl, 0)) [Units_WMS_Outgoing], CONVERT(FLOAT, ISNULL(fbaAusgang.SumAnzahl, 0)) [Units_FBA_Outgoing],CONVERT(FLOAT, ISNULL(MengeAufWarenEingang.fAnzahlAktuell,0)) [Stock_Wareneingangsplatz] FROM dbo.tartikel a INNER JOIN dbo.tartikelattribut aa ON aa.kartikel = a.kartikel INNER JOIN dbo.tartikelattributsprache aaI18n ON aaI18n.kartikelattribut = aa.kartikelattribut AND aaI18n.ksprache = 0 INNER JOIN dbo.tattributsprache attributI18n ON attributI18n.kattribut = aa.kattribut AND attributI18n.ksprache = 0 AND attributI18n.cname = 'CutDesign_ID' LEFT OUTER JOIN dbo.vlagerbestandfba fba ON a.kArtikel = fba.kartikel LEFT OUTER JOIN dbo.vlagerbestandprolageralle wmsLager ON wmsLAger.kartikel = a.kArtikel AND wmsLager.kwarenlager = 4 LEFT OUTER JOIN dbo.vlagerbestandprolageralle StreckenLager ON StreckenLager.kartikel = a.kArtikel AND Streckenlager.kwarenlager = 5 LEFT OUTER JOIN dbo.vlagerbestandprolageralle PendingLager ON PendingLager.kartikel = a.kArtikel AND PendingLager.kwarenlager = 7 LEFT OUTER JOIN (SELECT a.kArtikel, SUM(wla.fAnzahl) Anzahl FROM dbo.tBestellung b JOIN dbo.tbestellpos bp ON bp.tBestellung_kBestellung = b.kBestellung JOIN dbo.trechnung r ON r.tBestellung_kBestellung = b.kBestellung JOIN dbo.tLieferschein l ON l.kBestellung = b.kBestellung JOIN dbo.tLieferscheinPos lp ON lp.kBestellPos = bp.kBestellPos JOIN dbo.tWarenLagerAusgang wla ON wla.kLieferscheinPos = lp.kLieferscheinPos JOIN dbo.tArtikel a ON bp.tArtikel_kArtikel = a.kArtikel WHERE DATEDIFF(dd, b.derstellt, GETDATE()) <= 30 GROUP BY a.kArtikel ) AS WmsAusgang ON WmsAusgang.kArtikel = a.kArtikel LEFT OUTER JOIN (SELECT map.kArtikel, SUM(abp.nQuantityPurchased) SumAnzahl FROM dbo.pf_amazon_bestellung ab JOIN dbo.pf_amazon_bestellungpos abp ON abp.kAmazonBestellung = ab.kAmazonBestellung JOIN dbo.tBestellung b ON b.cInetBestellNr = ab.cOrderId JOIN dbo.trechnung r ON r.tBestellung_kBestellung = b.kBestellung JOIN dbo.pf_amazon_angebot_mapping map ON map.cSellerSKU = abp.cArtNr WHERE nFBA = 1 AND cOrderStatus = 'Shipped' AND DATEDIFF(dd, b.derstellt, GETDATE()) <= 30 GROUP BY map.kArtikel) FbaAusgang ON FbaAusgang.kArtikel = a.kArtikel LEFT OUTER JOIN (SELECT DISTINCT ta.kArtikel, twle.fAnzahlAktuell FROM dbo.tArtikel ta JOIN dbo.tWarenLagerEingang twle ON twle.kArtikel = ta.kArtikel JOIN dbo.tWarenlagerPlatz twlp ON twle.kWarenLagerPlatz = twlp.kWarenLagerPlatz WHERE twlp.cName = 'Wareneingangsplatz') MengeAufWarenEingang ON MengeAufWarenEingang.kArtikel = a.kArtikel");
var metaData=results.getMetaData();
var numCols = metaData.getColumnCount();
var spreadsheet = SpreadsheetApp.getActive();
var sheet = spreadsheet.getSheetByName("Test");
//you can use the following line to get the active sheet
//var sheet = SpreadsheetApp.getActiveSheet();
sheet.clearContents();
var arr=[];
for (var col = 0; col < numCols; col++) {
arr.push(metaData.getColumnName(col + 1));
}
sheet.appendRow(arr);
while (results.next()) {
arr=[];
for (var col = 0; col < numCols; col++) {
arr.push(results.getString(col + 1));
}
sheet.appendRow(arr);
}
results.close();
stmt.close();
sheet.autoResizeColumns(1, numCols+1);
}```
Runs slow first time and then fast right after can be a sign of a poorly optimized query.
If there' a table scan, the data might be loaded into cache the first time. It's in cache the second time.
That's a lot of JOINS and a big query. I would recommend a stored procedure if it's an option. The query plan might indicate missing indexes. It should also show the most expensive operations.
If 30 days is is a small fraction of the data and derstellt has an index, then compare the column directly to a date. Wrapping a DATEDIFF around the column prevents effective use of the index on that column.
DECLARE #TodayMinus29DaysMidnight datetime = CAST(GETDATE() - 29 as date)
SELECT...
WHERE b.derstellt < #TodayMinus29DaysMidnight
I would also verify the derived table with the DISINCT. Would a GROUP BY be better?
Related
SQL Server Stored Procedure select
I can't figure out how to set the values if Power, ChangeRate, and Type are null. I still want it to return rest of the values even if those are not in the database. ALTER PROCEDURE dbo.up_getTestData (#Chip int) AS DECLARE #Type char(2), #ChangeRate real, #Power bit SELECT #Type = Type FROM Spectrum WHERE ChipID = #Chip IF(#Type is null) //SET #Type = 'NA' //or Go to the bottom statement ?? SELECT #ChangeRate = ChangeRate FROM TempCycle WHERE ChipID = #Chip IF(#ChangeRate is null) //Go to the bottom statement ?? //SET #ChangeRate = '0.0' SELECT #Power = Power FROM TempCycle WHERE ChipID = #Chip IF(#Power is null) // I want it to still go the bottom statement. I put just return // but that's not right since it wouldn't return anything than?? // or I did SET #Power = 'false' ELSE SELECT Chips.Chip_ID AS ChipID, Chip_Number, Test_Module.ModuleTypeID, Test_Module.PID, Test_Module.Component1, Test_Module.Component2, Test_Module.Parameter1, Test_Module.Parameter2, Test_Module.Parameter3, Test_Module.Parameter4, Test_Module.Parameter5, Test_Module.Parameter6, Spectrum.Type, TempCycle.Power, TempCycle.ChangeRate FROM Chips INNER JOIN Test_Module ON Chips.Chip_Number = Test_Module.Module_Name INNER JOIN Spectrum ON Test_Module.ModuleSpec_TestID = Spectrum.TestID INNER JOIN TempCycle ON Test_Module.ModuleTemp_TestID = TempCycle.TestID WHERE Chip_ID = #Chip RETURN
I think all you need is coalesce (or ISNULL): SELECT Chips.Chip_ID AS ChipID, Chip_Number, Test_Module.ModuleTypeID, Test_Module.PID, Test_Module.Component1, Test_Module.Component2, Test_Module.Parameter1, Test_Module.Parameter2, Test_Module.Parameter3, Test_Module.Parameter4, Test_Module.Parameter5, Test_Module.Parameter6, coalesce(Spectrum.Type, 'NA') as Type, coalesce(TempCycle.Power, 'false') as Power, coalesce(TempCycle.ChangeRate, '0.0') as ChangeRate FROM Chips INNER JOIN Test_Module ON Chips.Chip_Number = Test_Module.Module_Name LEFT OUTER JOIN Spectrum ON Test_Module.ModuleSpec_TestID = Spectrum.TestID LEFT OUTER JOIN TempCycle ON Test_Module.ModuleTemp_TestID = TempCycle.TestID WHERE Chip_ID = #Chip
Track SQL query back to Magento source code
I'm reviewing slow query log and I've found that below query is inefficient. Given the query how can I trace it back to Magento source code (ideally a file)?: SELECT count(DISTINCT e.entity_id) FROM `sales_flat_order_item` AS `order_items` INNER JOIN `sales_flat_order` AS `order` ON `order`.entity_id = order_items.order_id AND `order`.state <> 'canceled' AND (`order`.created_at BETWEEN '2012-08-22 04:17:19' AND '2013-08-22 04:17:19') LEFT JOIN `catalog_product_entity` AS `e` ON (e.type_id NOT IN ('grouped', 'configurable', 'bundle')) AND e.entity_id = order_items.product_id AND e.entity_type_id = 4 INNER JOIN `catalog_product_entity_int` AS `at_visibility_default` ON (`at_visibility_default`.`entity_id` = `e`.`entity_id`) AND (`at_visibility_default`.`attribute_id` = '102') AND `at_visibility_default`.`store_id` = 0 LEFT JOIN `catalog_product_entity_int` AS `at_visibility` ON (`at_visibility`.`entity_id` = `e`.`entity_id`) AND (`at_visibility`.`attribute_id` = '102') AND (`at_visibility`.`store_id` = 1) INNER JOIN `catalog_product_entity_int` AS `at_status_default` ON (`at_status_default`.`entity_id` = `e`.`entity_id`) AND (`at_status_default`.`attribute_id` = '96') AND `at_status_default`.`store_id` = 0 LEFT JOIN `catalog_product_entity_int` AS `at_status` ON (`at_status`.`entity_id` = `e`.`entity_id`) AND (`at_status`.`attribute_id` = '96') AND (`at_status`.`store_id` = 1) WHERE (parent_item_id IS NULL) AND (IF(at_visibility.value_id > 0, at_visibility.value, at_visibility_default.value) IN(2, 4)) AND (IF(at_status.value_id > 0, at_status.value, at_status_default.value) = '1')\G
Open up lib/Varien/Db/Adapter/Pdo/Mysql.php then you will see as follows: /** * Write SQL debug data to file * * #var bool */ protected $_debug = false; /** * Log all queries (ignored minimum query duration time) * * #var bool */ protected $_logAllQueries = false; /** * Add to log call stack data (backtrace) * * #var bool */ protected $_logCallStack = false; So, if you set $_logCallStack to true you will see all you need in the log file.
Problem with Linq query .net MVC
public ActionResult Performances(string id) { var query = from f in _db.Production join g in _db.Run on f.show equals g.Production.show join l in _db.Performance on g.startDate equals l.runStartDate where f.show == id select new ShowPerformance { Venuename = g.venue, Showname = f.show, RunStart = g.startDate, RunEnd = g.endDate, PerformanceDate = l.performanceDate, PerformanceTime = l.performanceTime }; return View(query.ToList()); } The query can not distuingish between a performance in ShowA run1 and Show A run2 it just duplicates all performances ShowA run1 and Show A run2
I think the problem might be how you join Performance to Run/Production var query = from f in _db.Production join g in _db.Run on new {f.show, f.year} equals new {g.show, g.year} join l in _db.Performance on new {g.venue, g.startDate} equals new {l.venue, l.runStartDate} where f.show == id select new ShowPerformance { Venuename = g.venue, Showname = f.show, RunStart = g.startDate, RunEnd = g.endDate, PerformanceDate = l.performanceDate, PerformanceTime = l.performanceTime }; without something like the on g.runId equals l.runId then you will get all the performances for all the productions/runs.
How to improve performance of SQL MERGE statement
I have a job that I am currenlty running to sync data between two databases on different internal servers. The one server is the backend database for the HEAT product from FrontRange. The second is our own reporting database we are using for report writing and other internal uses. Our first approach at the job went something like this: Query all the data from the HEAT database tables we wanted and populate local temp tables. Then copy that data out to the appropriate table. That would work but they use to TRUNCATE the table every time and repopulate without doing anything for indexes or fragmentation. So I thought well this could be a good candidate to use the SQL merge statement. So our second approach used a merge statement for each of the tables. It greatly improved the speed of the process but it seems to be locking the source table up so that users are noticing 15-30 second delays when they try to save information. To make the merge only handle records that have had changes or are new I added the BINARY_CHECKSUM function on the select and store it on my side so that I can avoid updating records that haven't changed. Seems expensive though to call that for every record. This table has about ~300k records. I am wondering is there a better approach to try and synchronize these two tables that I am overlooking. My only constraint is that I really do not want to change anything on the source tables because it is a third party application. Here is my mege statement I am using for the CallLog table: -- Merge CallLog MERGE INTO [CallLog] AS T USING ( SELECT [CallID], [CustID], [CustType], [CallType], [Tracker], [CallStatus], [Priority], [CDuration], [CallCount], [StopWatch], [ClosedBy], [ClosedDate], [ClosedTime], [Cause], [CallDesc], [CloseDesc], [RecvdBy], [RecvdDate], [RecvdTime], [ModBy], [ModDate], [ModTime], [DTLastMod], [CallSource], [PriorityName], [QuickCall], [Category], [TotalAsgnmntTime], [CatHeading], [TotalJournalTime], [TotalTime], [SL_Warn_Goal], [SL_Warn_Date], [SL_Warn_Time], [SL_Complete_Goal], [SL_Complete_Date], [SL_Complete_Time], [SL_Clock_Status], [SL_Button_Status], [FirstResolution], [SL_Complete_Status], [SubCallType], [ImpactValue], [ImpactName], [UrgencyValue], [UrgencyName], [LinkedToProblem], [LinkedToProblemCustID], [LinkedToProblemName], [LinkedToProblemBy], [LinkedToProblemDate], [LinkedToProblemTime], [SLAStatus], [issue_text], [issue_number], [ResCheck], [AsgnAckBy], [AsgnAckDate], [AsgnAckTime], [Resolvedby], [ResolvedDate], [ResolvedTime], [ACheck], [ACKEmail], [LinkedToChange], [LinkedToChangeCustID], [LinkedToChangeName], [LinkedToChangeBy], [LInkedToChangeDate], [LinkedToChangeTime], [IssueTypeProblem], [IssueTypeChange], [RespWarningD], [RespWarningT], [RespMissedD], [RespMissedT], [ResoWarningD], [ResoWarningT], [ResoMissedD], [ResoMissedT], [IssueType], [SubCategory], [Diagnosis], [HSSAlert], [ErrorMessage], [ProblemType], [diagnosising], [KB], [CloseStatus], [SuggestedAssignGrp], [DefaultGrp], [DefaultGrpTF], [OtherAssign], [WorkAround], [ChangeReason], [CloseProblem], [AssgnApp], [AssgnAppRes], [DenyChk], [ImplementationApp], [ImplementationAppRes], [WorkAroundChk], [NoDenyChk], [ImpNoDenyChk], [ImpDenyChk], [ChangeStatus], [ReadyToClose], [ResolveOrReAssign], [TicketLabel], [CatCallType], [IssueType_PK], [Category_PK], [SubCategory_PK], [CallType_PK], [SubCallType_PK], BINARY_CHECKSUM(*) AS [Checksum] FROM [CHLA-HEATDB].SDIT.dbo.calllog ) AS S ON (T.[CallID] = S.[CallID]) WHEN MATCHED AND T.[Checksum] <> S.[Checksum] THEN UPDATE SET T.[CallID] = S.[CallID], T.[CustID] = S.[CustID], T.[CustType] = S.[CustType], T.[CallType] = S.[CallType], T.[Tracker] = S.[Tracker], t.[CallStatus] = S.[CallStatus], T.[Priority] = S.[Priority], T.[CDuration] = S.[CDuration], T.[CallCount] = S.[CallCount], T.[StopWatch] = S.[StopWatch], T.[ClosedBy] = S.[ClosedBy], T.[ClosedDate] = S.[ClosedDate], T.[ClosedTime] = S.[ClosedTime], T.[Cause] = S.[Cause], T.[CallDesc] = S.[CallDesc], T.[CloseDesc] = S.[CloseDesc], T.[RecvdBy] = S.[RecvdBy], T.[RecvdDate] = S.[RecvdDate], T.[RecvdTime] = S.[RecvdTime], T.[ModBy] = S.[ModBy], T.[ModDate] = S.[ModDate], T.[ModTime] = S.[ModTime], T.[DTLastMod] = S.[DTLastMod], T.[CallSource] = S.[CallSource], T.[PriorityName] = S.[PriorityName], T.[QuickCall] = S.[QuickCall], T.[Category] = S.[Category], T.[TotalAsgnmntTime] = S.[TotalAsgnmntTime], T.[CatHeading] = S.[CatHeading], T.[TotalJournalTime] = S.[TotalJournalTime], T.[TotalTime] = S.[TotalTime], T.[SL_Warn_Goal] = S.[SL_Warn_Goal], T.[SL_Warn_Date] = S.[SL_Warn_Date], T.[SL_Warn_Time] = S.[SL_Warn_Time], T.[SL_Complete_Goal] = S.[SL_Complete_Goal], T.[SL_Complete_Date] = S.[SL_Complete_Date], T.[SL_Complete_Time] = S.[SL_Complete_Time], T.[SL_Clock_Status] = S.[SL_Clock_Status], T.[SL_Button_Status] = S.[SL_Button_Status], T.[FirstResolution] = S.[FirstResolution], T.[SL_Complete_Status] = S.[SL_Complete_Status], T.[SubCallType] = S.[SubCallType], T.[ImpactValue] = S.[ImpactValue], T.[ImpactName] = S.[ImpactName], T.[UrgencyValue] = S.[UrgencyValue], T.[UrgencyName] = S.[UrgencyName], T.[LinkedToProblem] = S.[LinkedToProblem], T.[LinkedToProblemCustID] = S.[LinkedToProblemCustID], T.[LinkedToProblemName] = S.[LinkedToProblemName], T.[LinkedToProblemBy] = S.[LinkedToProblemBy], T.[LinkedToProblemDate] = S.[LinkedToProblemDate], T.[LinkedToProblemTime] = S.[LinkedToProblemTime], T.[SLAStatus] = S.[SLAStatus], T.[issue_text] = S.[issue_text], T.[issue_number] = S.[issue_number], T.[ResCheck] = S.[ResCheck], T.[AsgnAckBy] = S.[AsgnAckBy], T.[AsgnAckDate] = S.[AsgnAckDate], T.[AsgnAckTime] = S.[AsgnAckTime], T.[Resolvedby] = S.[Resolvedby], T.[ResolvedDate] = S.[ResolvedDate], T.[ResolvedTime] = S.[ResolvedTime], T.[ACheck] = S.[ACheck], T.[ACKEmail] = S.[ACKEmail], T.[LinkedToChange] = S.[LinkedToChange], T.[LinkedToChangeCustID] = S.[LinkedToChangeCustID], T.[LinkedToChangeName] = S.[LinkedToChangeName], T.[LinkedToChangeBy] = S.[LinkedToChangeBy], T.[LInkedToChangeDate] = S.[LInkedToChangeDate], T.[LinkedToChangeTime] = S.[LinkedToChangeTime], T.[IssueTypeProblem] = S.[IssueTypeProblem], T.[IssueTypeChange] = S.[IssueTypeChange], T.[RespWarningD] = S.[RespWarningD], T.[RespWarningT] = S.[RespWarningT], T.[RespMissedD] = S.[RespMissedD], T.[RespMissedT] = S.[RespMissedT], T.[ResoWarningD] = S.[ResoWarningD], T.[ResoWarningT] = S.[ResoWarningT], T.[ResoMissedD] = S.[ResoMissedD], T.[ResoMissedT] = S.[ResoMissedT], T.[IssueType] = S.[IssueType], T.[SubCategory] = S.[SubCategory], T.[Diagnosis] = S.[Diagnosis], T.[HSSAlert] = S.[HSSAlert], T.[ErrorMessage] = S.[ErrorMessage], T.[ProblemType] = S.[ProblemType], T.[diagnosising] = S.[diagnosising], T.[KB] = S.[KB], T.[CloseStatus] = S.[CloseStatus], T.[SuggestedAssignGrp] = S.[SuggestedAssignGrp], T.[DefaultGrp] = S.[DefaultGrp], T.[DefaultGrpTF] = S.[DefaultGrpTF], T.[OtherAssign] = S.[OtherAssign], T.[WorkAround] = S.[WorkAround], T.[ChangeReason] = S.[ChangeReason], T.[CloseProblem] = S.[CloseProblem], T.[AssgnApp] = S.[AssgnApp], T.[AssgnAppRes] = S.[AssgnAppRes], T.[DenyChk] = S.[DenyChk], T.[ImplementationApp] = S.[ImplementationApp], T.[ImplementationAppRes] = S.[ImplementationAppRes], T.[WorkAroundChk] = S.[WorkAroundChk], T.[NoDenyChk] = S.[NoDenyChk], T.[ImpNoDenyChk] = S.[ImpNoDenyChk], T.[ImpDenyChk] = S.[ImpDenyChk], T.[ChangeStatus] = S.[ChangeStatus], T.[ReadyToClose] = S.[ReadyToClose], T.[ResolveOrReAssign] = S.[ResolveOrReAssign], T.[TicketLabel] = S.[TicketLabel], T.[CatCallType] = S.[CatCallType], T.[IssueType_PK] = S.[IssueType_PK], T.[Category_PK] = S.[Category_PK], T.[SubCategory_PK] = S.[SubCategory_PK], T.[CallType_PK] = S.[CallType_PK], T.[SubCallType_PK] = S.[SubCallType_PK], T.[Checksum] = S.[Checksum] WHEN NOT MATCHED THEN INSERT VALUES ( S.[CallID], S.[CustID], S.[CustType], S.[CallType], S.[Tracker], S.[CallStatus], S.[Priority], S.[CDuration], S.[CallCount], S.[StopWatch], S.[ClosedBy], S.[ClosedDate], S.[ClosedTime], S.[Cause], S.[CallDesc], S.[CloseDesc], S.[RecvdBy], S.[RecvdDate], S.[RecvdTime], S.[ModBy], S.[ModDate], S.[ModTime], S.[DTLastMod], S.[CallSource], S.[PriorityName], S.[QuickCall], S.[Category], S.[TotalAsgnmntTime], S.[CatHeading], S.[TotalJournalTime], S.[TotalTime], S.[SL_Warn_Goal], S.[SL_Warn_Date], S.[SL_Warn_Time], S.[SL_Complete_Goal], S.[SL_Complete_Date], S.[SL_Complete_Time], S.[SL_Clock_Status], S.[SL_Button_Status], S.[FirstResolution], S.[SL_Complete_Status], S.[SubCallType], S.[ImpactValue], S.[ImpactName], S.[UrgencyValue], S.[UrgencyName], S.[LinkedToProblem], S.[LinkedToProblemCustID], S.[LinkedToProblemName], S.[LinkedToProblemBy], S.[LinkedToProblemDate], S.[LinkedToProblemTime], S.[SLAStatus], S.[issue_text], S.[issue_number], S.[ResCheck], S.[AsgnAckBy], S.[AsgnAckDate], S.[AsgnAckTime], S.[Resolvedby], S.[ResolvedDate], S.[ResolvedTime], S.[ACheck], S.[ACKEmail], S.[LinkedToChange], S.[LinkedToChangeCustID], S.[LinkedToChangeName], S.[LinkedToChangeBy], S.[LInkedToChangeDate], S.[LinkedToChangeTime], S.[IssueTypeProblem], S.[IssueTypeChange], S.[RespWarningD], S.[RespWarningT], S.[RespMissedD], S.[RespMissedT], S.[ResoWarningD], S.[ResoWarningT], S.[ResoMissedD], S.[ResoMissedT], S.[IssueType], S.[SubCategory], S.[Diagnosis], S.[HSSAlert], S.[ErrorMessage], S.[ProblemType], S.[diagnosising], S.[KB], S.[CloseStatus], S.[SuggestedAssignGrp], S.[DefaultGrp], S.[DefaultGrpTF], S.[OtherAssign], S.[WorkAround], S.[ChangeReason], S.[CloseProblem], S.[AssgnApp], S.[AssgnAppRes], S.[DenyChk], S.[ImplementationApp], S.[ImplementationAppRes], S.[WorkAroundChk], S.[NoDenyChk], S.[ImpNoDenyChk], S.[ImpDenyChk], S.[ChangeStatus], S.[ReadyToClose], S.[ResolveOrReAssign], S.[TicketLabel], S.[CatCallType], S.[IssueType_PK], S.[Category_PK], S.[SubCategory_PK], S.[CallType_PK], S.[SubCallType_PK], S.[Checksum] ); GO
Since you're on SQL 2008, how about Change Data Capture? You can get the net changes over a period of time and deal only with that (as opposed to replication which pushes every change, even if you update the same data 10 times).
I think you can improve performance by figuring out what rows to modify/insert before the merge statement. You can create a temp table that contains the CallID's that should be affected. Perhaps something like this. ;with cteSource as ( select S.CallID, binary_checksum(*) as [CheckSum] from [CHLA-HEATDB].SDIT.dbo.calllog ) select S.CallID, S.[CheckSum] into #TmpSource from cteSource as S left outer join CallLog as T on S.CallID = T.CallID and S.[CheckSum] = T.[CheckSum] where T.CallID is null Then you can use that table in your using select statement. MERGE INTO [CallLog] AS T USING ( SELECT [CallID], -- A lot of fields T.[CheckSum] AS [Checksum] FROM [CHLA-HEATDB].SDIT.dbo.calllog as C inner join #TmpSource as Tmp on C.CallID = Tmp.CallID ) AS S ON (T.[CallID] = S.[CallID]) WHEN MATCHED THEN -- Probably don't need this AND T.[Checksum] <> S.[Checksum] THEN UPDATE SET T.[CallID] = S.[CallID], -- More fields here WHEN NOT MATCHED THEN INSERT VALUES ( S.[CallID], -- More fields here );
I have some quite simple SQL that I can trying to change to either Linq or LLBLGEN
I want to do something like this... SELECT DISTINCT T1.* FROM T1 INNER JOIN T2 ON T2.ID1 = T1.ID1 INNER JOIN T3 ON T3.ID2 = T2.ID2 --FOLLOWING CAN BE ADDED MULTIPLE TIMES (LOOPS IN C#?) INNER JOIN T2 AS T2A ON T3.ID2 = T2A.ID2 INNER JOIN T1 AS T1A ON T1A.ID1 = T2A.ID1 --END MULTI WHERE T1.ID1 = 1 AND T3.ID3 = 2 AND T3.ID4 = 3 --THE FOLLOWING CONDITIONS WILL ALSO BE FOR EVERY SET OF EXTRA JOINS (LOOPS IN C#?) AND T1A.ID1 = 4 AND T1I.ID5 = 5 --END MULTI ...in either Linq or LLBLGen Code. Any help would be greatly appreciated! Here is the LLBGen I have so far... IPredicateExpression filter = new PredicateExpression(); filter.Add(ProductTypeOptionAttributeFields.OptionId == dl.Key); filter.AddWithAnd(ProductTypeOptionAttributeCombinationFields.ProductTypeId == DataSource.DataItem.ProductTypeId); filter.AddWithAnd(ProductTypeOptionAttributeCombinationFields.ProductId == DataSource.ProductID); bucket.PredicateExpression.Add(filter); bucket.Relations.Add(ProductTypeOptionAttributeEntity.Relations.ProductTypeOptionAttributeCombinationProfileEntityUsingProductTypeOptionAttributeId, JoinHint.Inner); bucket.Relations.Add(ProductTypeOptionAttributeCombinationProfileEntity.Relations.ProductTypeOptionAttributeCombinationEntityUsingProductTypeOptionAttributeCombinationId, JoinHint.Inner); var filtered = _dropdowns.Where(k => ((DropDownList)k.Value[1]).SelectedValue != "-1" && k.Key != dl.Key); foreach (var filteredDdl in filtered) { IPredicateExpression subFilter = new PredicateExpression(); subFilter.AddWithAnd(ProductTypeOptionAttributeFields.AttributeId == int.Parse(((DropDownList)filteredDdl.Value[1]).SelectedValue)); subFilter.AddWithAnd(ProductTypeOptionAttributeFields.OptionId == filteredDdl.Key); bucket.PredicateExpression.AddWithAnd(subFilter); } ProductTypeOptionAttributeCollection attrs = new ProductTypeOptionAttributeCollection(); attrs.GetMulti(bucket.PredicateExpression, -1, null, bucket.Relations); And here is the actual query I want... SELECT DISTINCT PTOA.* FROM ProductTypeOptionAttribute AS PTOA INNER JOIN ProductTypeOPtionAttributeCombinationProfile AS PTOACP ON PTOACP.ProductTypeOPtionAttributeID = PTOA.AttributeID INNER JOIN ProductTypeOPtionAttributeCombination AS PTOAC ON PTOAC.CombinationID = PTOACP.ProductTypeOptionAttributeCombinationID --FOLLOWING CAN BE ADDED MULTIPLE TIMES (LOOPS IN C#?) INNER JOIN ProductTypeOPtionAttributeCombinationProfile AS PTOACP2 ON PTOAC.CombinationID = PTOACP2.ProductTypeOptionAttributeCombinationID INNER JOIN ProductTypeOPtionAttribute AS PTOA2 ON PTOACP2.ProductTypeOPtionAttributeID = PTOA2.AttributeID --END MULTI WHERE PTOA.OptionID = 59 AND PTOAC.ProductTypeID = 11 AND PTOAC.ProductID = 218 --THE FOLLOWING CONDITIONS WILL ALSO BE FOR EVERY SET OF EXTRA JOINS (LOOPS IN C#?) AND PTOA2.AttributeID = 42 AND PTOA2.OptionID = 58 --END MULTI Cheers
LLBLGen Tips: Use SQL Server Profiler to view the SQL emitted (set a breakpoint right after your call to GetMulti, then watch your trace) You have a lot of complicated UI login and cast/converts that could fail -- my personal preference would be to move those to separate code You don't need JoinHint.Inner as that is the default (Personal preference) use a RelationCollection instead of the bucket. I don't entirely understand your situation (especially the multiple joins to the same table?), but this may work. I think you want subFilter.AddWithOr instead of .AddWithAnd. IPredicateExpression filter = new PredicateExpression(); filter.Add(ProductTypeOptionAttributeFields.OptionId == dl.Key); filter.AddWithAnd(ProductTypeOptionAttributeCombinationFields.ProductTypeId == DataSource.DataItem.ProductTypeId); filter.AddWithAnd(ProductTypeOptionAttributeCombinationFields.ProductId == DataSource.ProductID); IRelationCollection relations = new RelationCollection(); relations.Add(ProductTypeOptionAttributeEntity.Relations.ProductTypeOptionAttributeCombinationProfileEntityUsingProductTypeOptionAttributeId); relations.Add(ProductTypeOptionAttributeCombinationProfileEntity.Relations.ProductTypeOptionAttributeCombinationEntityUsingProductTypeOptionAttributeCombinationId); var filtered = _dropdowns.Where(k => ((DropDownList)k.Value[1]).SelectedValue != "-1" && k.Key != dl.Key); foreach (var filteredDdl in filtered) { IPredicateExpression subFilter = new PredicateExpression(); subFilter.AddWithOr(ProductTypeOptionAttributeFields.AttributeId == int.Parse(((DropDownList)filteredDdl.Value[1]).SelectedValue)); subFilter.AddWithOr(ProductTypeOptionAttributeFields.OptionId == filteredDdl.Key); filter.AddWithAnd(subFilter); } ProductTypeOptionAttributeCollection attrs = new ProductTypeOptionAttributeCollection(); attrs.GetMulti(filter, relations)
The basic SQL query you presented should be faily easy to reproduce in LINQ. From t in T1 Where T1.ID == 1 Select; if your not already using it download the free LINQPAD http://www.linqpad.net/ Its got loads of examples to get you up to speed.