I have two tables that are identical to each other on the same database, one is a temporary 'holding' table for uploaded data, the other is the live data table which will store the permanent data.
The reason for this is that the two tables are compared against each other to see changing values from what was there originally and what is newly being updated.
My question is, once the comparison stage has been completed and the user submits the upload, how do I copy over the data from the temporary table to the existing table where it will insert a new record where it does not exist in the existing table and if it does already exist on the permanent table (using a MonthID and yearID), then it should just UPDATE, so that there aren't any duplicates.
So I want it to check if the record exists by seeing if the YearID and MonthID are already there in the other table. Keep in mind that both tables are identical.
Here is some sample data:
Table 1
SpendID Spend MonthNo YearID ChangedBy
47 1444.19 2 4 12
Table 2
SpendID Spend MonthNo YearID ChangedBy
1 1500.54 2 4 12
Sorry, hope this makes sense.
I've tried looking at UPDATE and MERGE, but they seem to work of primary identification keys which I can't use here.
Just need an outside opinion to help with this as I've been struggling.
You can use the following commands to SQLITE: delete-delete, insert, select!!!
You can do this either with MERGE or a INSERT + UPDATE combination. I prefer the latter.
-- Add missing records
INSERT INTO [Table 1] (SpendID, Spend, MonthNo, YearID, ChangedBy)
SELECT
SpendID
, Spend
, MonthNo
, YearID
, ChangedBy
FROM [Table 2] t2
WHERE NOT EXISTS (SELECT 1
FROM [Table 1] t1
WHERE t1.MonthNo = t2.MonthNo
AND t1.YearID = t2.YearID)
-- Update existing records
UPDATE t1
SET t1.Spend = t2.Spend
FROM [Table 1] t1
INNER JOIN [Table 2] t2
ON t1.MonthNo = t2.MonthNo
AND t1.YearID = t2.YearID
The reason I'd prefer this over MERGE is because MERGE can cause performance problems in certain scenarios. There are a lot of articles online, but I'll just post a link to an article that exposes one problem with MERGE and also relates a few more Connect items, at the bottom of the article, that report issues with MERGE.
Related
Long time viewer and my first question. Please be gentle.
I am having issues writing a query that incorporates tables with 1-1 / 1-M relationships.
To keep it simple - I have two tables
Tables
Query - Provide the entire list of cases from Table 1 and add a new column that has a flag (Y/N) if case has a car from table 2 whilst keeping the 1-1 relationship
Outputs
Try using exists logic to check, for each table 1 record, if it has a matching car record in the second table:
SELECT
t1.caseno,
CASE WHEN EXISTS (SELECT 1 FROM Table2 t2
WHERE t1.caseno = t2.caseno AND t2.Product = 'Car')
THEN 'Y' ELSE 'N' END AS car_flag
FROM Table1 t1
ORDER BY
t1.caseno;
Demo
I have 2 tables. Table 1 has data from the bank account. Table 2 aggregates data from multiple other tables; to keep things simple, we will just have 2 tables. I need to append the data from table 1 into table 2.
I have a field in table2, "SrceFk". The concept is that when a record from Table1 appends, it will fill the table2.SrceFk with the table1 primary key and the table name. So record 302 will look like "BANK/302" after it appends. This way, when I run the append query, I can avoid duplicates.
The query is not working. I deleted the record from table2, but when I run the query, it just says "0 records appended". Even though the foreign key is not present.
I am new to SQL, Access, and programming in general. I understand basic concepts. I have googled this issue and looked on stackOverflow, but no luck.
This is my full statement:
INSERT INTO Main ( SrceFK, InvoDate, Descrip, AMT, Ac1, Ac2 )
SELECT Bank.ID &"/"& "BANK", Bank.TransDate, Bank.Descrip, Bank.TtlAmt, Bank.Ac1, Bank.Ac2
FROM Bank
WHERE NOT EXISTS
(
SELECT * FROM Main
WHERE Main.SrceFK = Bank.ID &"/"& "BANK"
);
I expect the query to add records that aren't present in the table, as needed.
When the user clicks a delete button in my UI to delete a product I need to do a fast check to see if it is a foreign key in any of the 4 tables (Table1, Table2, Table3, Table4). If it isn't then I can proceed with the delete. If it is in a single one of them I can't delete it.
Some of these tables have thousands of records and I already learned the hard way that using joins is not the best way because the query takes minutes to complete.
I figured union might be the best way but I am wondering if there is a way I can further enhance it. Or even possibly send back which tables it is involved in so I can give the user a descriptive message on why they can't delete the Product.
Here is what I have so far and it is really fast, but returns thousands of 1's when the product exists all over the place. I suppose I can just do a single or default and if not null then don't let them delete.
select 1
from (
select ProductId from Table1
union all
select ProductId from Table2
union all
select ProductId from Table3
union all
select ProductId from Table4
) tbl
where ProductId = 1000
Here is a method with exists and case:
select (case when exists (select 1 from table1 where productId = 1000) then 1
when exists (select 1 from table2 where productId = 1000) then 1
when exists (select 1 from table3 where productId = 1000) then 1
when exists (select 1 from table4 where productId = 1000) then 1
else 0
end)
One way is to not define any "on delete" clauses when creating the foreign keys. Then just go ahead and try to delete the record. If there is no foreign key in any table referencing that record, the delete will succeed and you go about business as normal. If there is a foreign key in any table referencing that record, the delete will fail. Catch the error and try to give the user a meaningful message: "This customer cannot be deleted as they have at least one order still pending."
This way the system itself will perform the check for you, which will assuredly be more efficient than any check you can perform at the SQL level.
So I have two tables which will store sales figures for products. Table one holds the last 6 weeks sales figures for each product and table 2 shows the last 12 months. I need to find a way to compare these two tables to then produce a 3rd table which will contain the difference between the 2 values as column 2 as well as the products Sage code in column one. What would be the most efficient (in terms of time) way to do this as there will be a fair amount of products to compare and it will only continue to grow? The product Sage code is the key identifier here. The two tables are created as below.
IF OBJECT_ID('tempdb..#Last6WeeksProductSales') IS NOT NULL DROP TABLE #Last6WeeksProductSales;
CREATE TABLE #Last6WeeksProductSales
(
CompoundSageCode varchar(200),
Value decimal(18,2)
)
INSERT INTO #Last6WeeksProductSales
SELECT [SalesOrderLine].[sProductSageCode] AS [CompoundSageCode],
SUM([SalesOrderLine].[fQtyOrdered] * [SalesOrderLine].[fPricePerUnit]) AS [Value]
FROM [SalesOrderLine]
INNER JOIN [SalesOrder] ON (SalesOrder.iSalesOrderID = SalesOrderLine.iSalesOrderID)
WHERE [SalesOrder].[dOrderDateTime] > DateAdd("ww", -6, CURRENT_TIMESTAMP)
GROUP BYsProductSageCode;
SELECT * FROM #Last6WeeksProductSales
ORDER BY CompoundSageCode;
IF OBJECT_ID('tempdb..#Last12MonthsProductSales') IS NOT NULL DROP TABLE #Last12MonthsProductSales;
CREATE TABLE #Last12MonthsProductSales
(
CompoundSageCode varchar(200),
Value decimal(18,2)
)
INSERT INTO #Last12MonthsProductSales SELECT [SalesOrderLine].[sProductSageCode] AS [CompoundSageCode],
SUM([SalesOrderLine].[fQtyOrdered] * [SalesOrderLine].[fPricePerUnit]) AS [Value]
FROM [SalesOrderLine]
INNER JOIN [SalesOrder] ON (SalesOrder.iSalesOrderID = SalesOrderLine.iSalesOrderID)
WHERE [SalesOrder].[dOrderDateTime] > DateAdd(month, -12, CURRENT_TIMESTAMP)
GROUP BY sProductSageCode;
SELECT * FROM#Last12MonthsProductSales
ORDER BY CompoundSageCode;
DROP TABLE #Last6WeeksProductSales;
DROP TABLE #Last12MonthsProductSales;
Use a view. That way you don't have to worry about updating your third table, and it will reflect current information. Base the view on a basic SELECT:
SELECT sixS.CompoundSageCode,
(twelveS.value - sixS.Value ) as diffValue
FROM Last6WeeksProductSales sixS
INNER JOIN Last12MonthsProductSales twelveS ON sixS.CompoundSageCode = twelveS.CompoundSageCode
(I have not tested this code, but it should be a good starting point.)
Computing the difference of two tables is usually done using a FULL OUTER JOIN. SQL Server can implement it using all three of the physical join operators. Apply reasonable indexing and it will run fine.
If you can manage it, create covering indexes on both tables that are sorted by the join key. This will result in a highly efficient merge join plan.
I am sorry if the term m:n is not correct, If you know a better term i will correct. I have the following situation, this is my original data:
gameID
participID
result
the data itself looks like that
1 5 10
1 4 -10
2 5 150
2 2 -100
2 1 -50
when i would extract this table it will easily have some 100mio rows and around 1mio participIDs ore more.
i will need:
show me all results of all games from participant x, where participant y was present
luckily only for a very limited amount of participants, but those are subject to change so I need a complete table and can reduce in a second step.
my idea is the following, it just looks very unoptimized
1) get the list of games where the "point of view participant" is included"
insert into consolidatedtable (gameid, participid, result)
select gameID,participID,sum(result) from mastertable where participID=x and result<>0
2) get all games where other participant is included
insert into consolidatedtable (gameid, participid, result)
where gameID in (select gameID from consolidatedtable)
AND participID=y and result<>0
3) delete all games from consolidate table where count<2
delete from consolidatedDB where gameID in (select gameid from consolidatedtable where count(distinct(participID)<2 group by gameid)
the whole thing looks like a childrens solution to me
I need a consolidated table for each player
I insert way to many games into this table and delete them later on
the whole thing needs to be run participant by participant over
the whole master table, it would not work if i do this for several
participants at the same time
any better ideas, must be, this ones just so bad. the master table will be postgreSQL on the DW server, the consolidated view will be mySQL (but the number crunching will be done in postgreSQL)
my problems
1) how do i build the consolidated table(s - do i need more than one), without having to run a single query for each player over the whole master table (i need to data for players x,y,z and no matter who else is playing) - this is the consolidation task for the DW server, it should create the table for webserver (which is condensed)
2) how can i then extract the at the webserver fast (so the table design of (1) should take this into consideration. we are not talking about a lot of players here i need this info, maybe 100? (so i could then either partition by player ID, or just create single table)
Datawarehouse: postgreSQL 9.2 (48GB, SSD)
Webserver: mySQL 5.5 (4GB Ram, SSD)
master table: gameid BIGINT, participID, Result INT, foreign key on particiP ID (to participants table)
the DW server will hold the master table, the DW server should also prepare the consolidated/extracted Tables (processing power, ssd space is not
an issue)
the webserver should hold the consoldiated tables (only for the 100
players where i need the info) and query this data in a very
efficient manner
so efficient query at webserver >> workload of DW server)
i think this is important, sorry that i didnt include it at the beginning.
the data at the DW server updates daily, but i do not need to query the whole "master table" completely every day. the setup allows me to consolidate only never values. eg: yesterday consolidation was up to ID 500, current ID=550, so today i only consolidate 501-550.
Here is another idea that might work, depending on your database (and my understanding of the question):
SELECT *
FROM table a
WHERE participID = 'x'
AND EXISTS (
SELECT 1 FROM table b
WHERE b.participID = 'y'
AND b.gameID=a.gameID
);
Assuming you have indexes on the two columns (participID and gameID), the performance should be good.
I'd compare it to this and see which runs faster:
SELECT *
FROM table a
JOIN (
SELECT gameID
FROM table
WHERE participID = 'y'
GROUP BY gameID
) b
ON a.gameID=b.gameID
WHERE a.participID = 'x';
Sounds like you just want a self join:
For all participants:
SELECT x.gameID, x.participID, x.results, y.participID, y.results
FROM table as x
JOIN table as y
ON T1.gameID = T2.gameID
WHERE x.participID <> y.participID
The downside of that is you'd get each participant on each side of each game.
For 2 specific particpants:
SELECT x.gameID, x.results, y.results
FROM (SELECT gameID, participID, results
FROM table
WHERE t1.participID = 'x'
and results <> 0)
as x
JOIN (SELECT gameID, participID, results
FROM table
WHERE t1.participID = 'y'
and results <> 0)
as y
ON T1.gameID = T2.gameID
You might not need to select participID in your query, depending on what you're doing with the results.